[{"id":15114,"web_url":"https://patchwork.libcamera.org/comment/15114/","msgid":"","date":"2021-02-12T00:25:00","subject":"Re: [libcamera-devel] [PATCH v7 02/10] utils: ipc: add templates\n\tfor code generation for IPC mechanism","submitter":{"id":2,"url":"https://patchwork.libcamera.org/api/people/2/","name":"Laurent Pinchart","email":"laurent.pinchart@ideasonboard.com"},"content":"Hi Paul,\n\nThank you for the patch.\n\nOn Thu, Feb 11, 2021 at 04:17:57PM +0900, Paul Elder wrote:\n> Add templates to mojo to generate code for the IPC mechanism. These\n> templates generate:\n> - module header\n> - module serializer\n> - IPA proxy cpp, header, and worker\n> \n> Given an input data definition mojom file for a pipeline.\n> \n> Signed-off-by: Paul Elder \n> Acked-by: Jacopo Mondi \n> Acked-by: Kieran Bingham \n> Reviewed-by: Laurent Pinchart \n> \n> ---\n> Changes in v7:\n> - cosmetic changes, add a few todos\n> - use the new sendSync/sendAsync IPCPipe API\n> - save the sequence number in the proxy\n> - construct the header before calling sendSync/sendAsync (from the\n> proxy)\n> - replace genHeader and genSerdes with skipHeader and skipSerdes in\n> core.mojom\n> \n> Changes in v6:\n> - add templates for core_ipa_interface.h and core_ipa_serializer.h\n> - for libcamera types defined in mojom\n> - rename everything to {{module_name}}_ipa_{interface,proxy,proxy_worker}.{c,h}\n> - remove #include - support customizable start()\n> - remove the need for per-pipeline ControlInfoMap\n> - add todo for avoiding intermediate vectors\n> - remove postfix underscore for generated struct fields\n> - support structs that are members of vectors/maps that aren't defined\n> in mojom (in mojom)\n> - fix has_fd detection\n> - namespacing is now required in mojom, in the form of ^ipa\\.[0-9A-Za-z_]+\n> - support consts in mojom\n> - make the pseudo-switch-case in the python generator nicer\n> \n> Changes in v5:\n> - add a usage output to the proxy worker, to document the interface for\n> executing the proxy worker\n> - in the mojom generator python script:\n> - removed unused things (imports, functions, jinja exports)\n> - document GetNameForElement\n> - rename everything cb -> event\n> - refactor Method{Input,Output}HasFd with a helper MethodParamsHaveFd\n> - add Get{Main,Event}Interface to fix the interface_{main,event} jinja\n> exports\n> - add copyright\n> - require that event interfaces have at least one event\n> - expand copyright for templates\n> - use new sendSync/sendAsync API (with IPCMessage)\n> - rename a bunch of things\n> \n> Changes in v4:\n> For the non-template files:\n> - rename IPA{pipeline_name}CallbackInterface to\n> IPA{pipeline_name}EventInterface\n> - to avoid the notion of \"callback\" and emphasize that it's an event\n> - add support for strings in custom structs\n> - add validation, that async methods must not have return values\n> - it throws exception and isn't very clear though...?\n> - rename controls to libcamera::{pipeline_name}::controls (controls is\n> now lowercase)\n> - rename {pipeline_name}_generated.h to {pipeline_name}_ipa_interface.h,\n> and {pipeline_name}_serializer.h to {pipeline_name}_ipa_serializer.h\n> - same for their corresponding template files\n> For the template files:\n> - fix spacing (now it's all {{var}} instead of some {{ var }})\n> - except if it's code, so code is still {{ code }}\n> - move inclusion of corresponding header to first in the inclusion list\n> - fix copy&paste errors\n> - change snake_case to camelCase in the generated code\n> - template code still uses snake_case\n> - change the generated command enums to an enum class, and make it\n> capitalized (instead of allcaps)\n> - add length checks to recvIPC (in proxy)\n> - fix some template spacing\n> - don't use const for PODs in function/signal parameters\n> - add the proper length checks to readPOD/appendPOD\n> - the helper functions for reading and writing PODs to and from\n> serialized data\n> - rename readUInt/appendUInt to readPOD/appendPOD\n> - add support for strings in custom structs\n> \n> Changes in v3:\n> - add support for namespaces\n> - fix enum assignment (used to have +1 for CMD applied to all enums)\n> - use readHeader, writeHeader, and eraseHeader as static class functions\n> of IPAIPCUnixSocket (in the proxy worker)\n> - add requirement that base controls *must* be defined in\n> libcamera::{pipeline_name}::Controls\n> \n> Changes in v2:\n> - mandate the main and callback interfaces, and init(), start(), stop()\n> and their parameters\n> - fix returning single pod value from IPC-called function\n> - add licenses\n> - improve auto-generated message\n> - other fixes related to serdes\n> ---\n> .../core_ipa_interface.h.tmpl | 40 ++\n> .../core_ipa_serializer.h.tmpl | 47 ++\n> .../definition_functions.tmpl | 53 ++\n> .../libcamera_templates/meson.build | 14 +\n> .../module_ipa_interface.h.tmpl | 87 +++\n> .../module_ipa_proxy.cpp.tmpl | 236 ++++++++\n> .../module_ipa_proxy.h.tmpl | 128 +++++\n> .../module_ipa_proxy_worker.cpp.tmpl | 226 ++++++++\n> .../module_ipa_serializer.h.tmpl | 48 ++\n> .../libcamera_templates/proxy_functions.tmpl | 194 +++++++\n> .../libcamera_templates/serializer.tmpl | 313 +++++++++++\n> .../generators/mojom_libcamera_generator.py | 508 ++++++++++++++++++\n> 12 files changed, 1894 insertions(+)\n> create mode 100644 utils/ipc/generators/libcamera_templates/core_ipa_interface.h.tmpl\n> create mode 100644 utils/ipc/generators/libcamera_templates/core_ipa_serializer.h.tmpl\n> create mode 100644 utils/ipc/generators/libcamera_templates/definition_functions.tmpl\n> create mode 100644 utils/ipc/generators/libcamera_templates/meson.build\n> create mode 100644 utils/ipc/generators/libcamera_templates/module_ipa_interface.h.tmpl\n> create mode 100644 utils/ipc/generators/libcamera_templates/module_ipa_proxy.cpp.tmpl\n> create mode 100644 utils/ipc/generators/libcamera_templates/module_ipa_proxy.h.tmpl\n> create mode 100644 utils/ipc/generators/libcamera_templates/module_ipa_proxy_worker.cpp.tmpl\n> create mode 100644 utils/ipc/generators/libcamera_templates/module_ipa_serializer.h.tmpl\n> create mode 100644 utils/ipc/generators/libcamera_templates/proxy_functions.tmpl\n> create mode 100644 utils/ipc/generators/libcamera_templates/serializer.tmpl\n> create mode 100644 utils/ipc/generators/mojom_libcamera_generator.py\n> \n> diff --git a/utils/ipc/generators/libcamera_templates/core_ipa_interface.h.tmpl b/utils/ipc/generators/libcamera_templates/core_ipa_interface.h.tmpl\n> new file mode 100644\n> index 00000000..b253881b\n> --- /dev/null\n> +++ b/utils/ipc/generators/libcamera_templates/core_ipa_interface.h.tmpl\n> @@ -0,0 +1,40 @@\n> +{#-\n> + # SPDX-License-Identifier: LGPL-2.1-or-later\n> + # Copyright (C) 2020, Google Inc.\n> +-#}\n> +{%- import \"definition_functions.tmpl\" as funcs -%}\n> +/* SPDX-License-Identifier: LGPL-2.1-or-later */\n> +/*\n> + * Copyright (C) 2020, Google Inc.\n> + *\n> + * core_ipa_interface.h - libcamera core definitions for Image Processing Algorithms\n> + *\n> + * This file is auto-generated. Do not edit.\n> + */\n> +\n> +#ifndef __LIBCAMERA_IPA_INTERFACE_CORE_GENERATED_H__\n> +#define __LIBCAMERA_IPA_INTERFACE_CORE_GENERATED_H__\n> +\n> +{% if has_map %}#include {% endif %}\n> +{% if has_array %}#include {% endif %}\n> +\n> +#include \n> +\n> +namespace libcamera {\n> +\n> +{# \\todo Use constexpr instead of const after C++20 for std::string #}\n\nI meant \"Use const char * instead of std::string for strings\" :-)\n\nOtherwise the patch looks good to me.\n\n> +{% for const in consts %}\n> +static const {{const.kind|name}} {{const.mojom_name}} = {{const.value}};\n> +{% endfor %}\n> +\n> +{% for enum in enums %}\n> +{{funcs.define_enum(enum)}}\n> +{% endfor %}\n> +\n> +{%- for struct in structs_gen_header %}\n> +{{funcs.define_struct(struct)}}\n> +{% endfor %}\n> +\n> +} /* namespace libcamera */\n> +\n> +#endif /* __LIBCAMERA_IPA_INTERFACE_CORE_GENERATED_H__ */\n> diff --git a/utils/ipc/generators/libcamera_templates/core_ipa_serializer.h.tmpl b/utils/ipc/generators/libcamera_templates/core_ipa_serializer.h.tmpl\n> new file mode 100644\n> index 00000000..37a784f1\n> --- /dev/null\n> +++ b/utils/ipc/generators/libcamera_templates/core_ipa_serializer.h.tmpl\n> @@ -0,0 +1,47 @@\n> +{#-\n> + # SPDX-License-Identifier: LGPL-2.1-or-later\n> + # Copyright (C) 2020, Google Inc.\n> +-#}\n> +{%- import \"serializer.tmpl\" as serializer -%}\n> +\n> +/* SPDX-License-Identifier: LGPL-2.1-or-later */\n> +/*\n> + * Copyright (C) 2020, Google Inc.\n> + *\n> + * core_ipa_serializer.h - Data serializer for core libcamera definitions for IPA\n> + *\n> + * This file is auto-generated. Do not edit.\n> + */\n> +\n> +#ifndef __LIBCAMERA_INTERNAL_IPA_DATA_SERIALIZER_CORE_H__\n> +#define __LIBCAMERA_INTERNAL_IPA_DATA_SERIALIZER_CORE_H__\n> +\n> +#include \n> +#include \n> +\n> +#include \n> +\n> +#include \"libcamera/internal/control_serializer.h\"\n> +#include \"libcamera/internal/ipa_data_serializer.h\"\n> +\n> +namespace libcamera {\n> +\n> +LOG_DECLARE_CATEGORY(IPADataSerializer)\n> +{% for struct in structs_gen_serializer %}\n> +template<>\n> +class IPADataSerializer<{{struct|name}}>\n> +{\n> +public:\n> +{{- serializer.serializer(struct, \"\")}}\n> +{%- if struct|has_fd %}\n> +{{serializer.deserializer_fd(struct, \"\")}}\n> +{%- else %}\n> +{{serializer.deserializer_no_fd(struct, \"\")}}\n> +{{serializer.deserializer_fd_simple(struct, \"\")}}\n> +{%- endif %}\n> +};\n> +{% endfor %}\n> +\n> +} /* namespace libcamera */\n> +\n> +#endif /* __LIBCAMERA_INTERNAL_IPA_DATA_SERIALIZER_CORE_H__ */\n> diff --git a/utils/ipc/generators/libcamera_templates/definition_functions.tmpl b/utils/ipc/generators/libcamera_templates/definition_functions.tmpl\n> new file mode 100644\n> index 00000000..cdd75f89\n> --- /dev/null\n> +++ b/utils/ipc/generators/libcamera_templates/definition_functions.tmpl\n> @@ -0,0 +1,53 @@\n> +{#-\n> + # SPDX-License-Identifier: LGPL-2.1-or-later\n> + # Copyright (C) 2020, Google Inc.\n> +-#}\n> +\n> +{#\n> + # \\brief Generate enum definition\n> + #\n> + # \\param enum Enum object whose definition is to be generated\n> + #}\n> +{%- macro define_enum(enum) -%}\n> +enum {{enum.mojom_name}} {\n> +{%- for field in enum.fields %}\n> +\t{{field.mojom_name}} = {{field.numeric_value}},\n> +{%- endfor %}\n> +};\n> +{%- endmacro -%}\n> +\n> +{#\n> + # \\brief Generate struct definition\n> + #\n> + # \\param struct Struct object whose definition is to be generated\n> + #}\n> +{%- macro define_struct(struct) -%}\n> +struct {{struct.mojom_name}}\n> +{\n> +public:\n> +\t{{struct.mojom_name}}() {%- if struct|has_default_fields %}\n> +\t\t:{% endif %}\n> +{%- for field in struct.fields|with_default_values -%}\n> +{{\" \" if loop.first}}{{field.mojom_name}}({{field|default_value}}){{\", \" if not loop.last}}\n> +{%- endfor %}\n> +\t{\n> +\t}\n> +\n> +\t{{struct.mojom_name}}(\n> +{%- for field in struct.fields -%}\n> +{{\"const \" if not field|is_pod}}{{field|name}} {{\"&\" if not field|is_pod}}_{{field.mojom_name}}{{\", \" if not loop.last}}\n> +{%- endfor -%}\n> +)\n> +\t\t:\n> +{%- for field in struct.fields -%}\n> +{{\" \" if loop.first}}{{field.mojom_name}}(_{{field.mojom_name}}){{\", \" if not loop.last}}\n> +{%- endfor %}\n> +\t{\n> +\t}\n> +{% for field in struct.fields %}\n> +\t{{field|name}} {{field.mojom_name}};\n> +{%- endfor %}\n> +};\n> +{%- endmacro -%}\n> +\n> +\n> diff --git a/utils/ipc/generators/libcamera_templates/meson.build b/utils/ipc/generators/libcamera_templates/meson.build\n> new file mode 100644\n> index 00000000..70664eab\n> --- /dev/null\n> +++ b/utils/ipc/generators/libcamera_templates/meson.build\n> @@ -0,0 +1,14 @@\n> +# SPDX-License-Identifier: CC0-1.0\n> +\n> +mojom_template_files = files([\n> + 'core_ipa_interface.h.tmpl',\n> + 'core_ipa_serializer.h.tmpl',\n> + 'definition_functions.tmpl',\n> + 'module_ipa_interface.h.tmpl',\n> + 'module_ipa_proxy.cpp.tmpl',\n> + 'module_ipa_proxy.h.tmpl',\n> + 'module_ipa_proxy_worker.cpp.tmpl',\n> + 'module_ipa_serializer.h.tmpl',\n> + 'proxy_functions.tmpl',\n> + 'serializer.tmpl',\n> +])\n> diff --git a/utils/ipc/generators/libcamera_templates/module_ipa_interface.h.tmpl b/utils/ipc/generators/libcamera_templates/module_ipa_interface.h.tmpl\n> new file mode 100644\n> index 00000000..ebe811fa\n> --- /dev/null\n> +++ b/utils/ipc/generators/libcamera_templates/module_ipa_interface.h.tmpl\n> @@ -0,0 +1,87 @@\n> +{#-\n> + # SPDX-License-Identifier: LGPL-2.1-or-later\n> + # Copyright (C) 2020, Google Inc.\n> +-#}\n> +{%- import \"definition_functions.tmpl\" as funcs -%}\n> +/* SPDX-License-Identifier: LGPL-2.1-or-later */\n> +/*\n> + * Copyright (C) 2020, Google Inc.\n> + *\n> + * {{module_name}}_ipa_interface.h - Image Processing Algorithm interface for {{module_name}}\n> + *\n> + * This file is auto-generated. Do not edit.\n> + */\n> +\n> +#ifndef __LIBCAMERA_IPA_INTERFACE_{{module_name|upper}}_GENERATED_H__\n> +#define __LIBCAMERA_IPA_INTERFACE_{{module_name|upper}}_GENERATED_H__\n> +\n> +#include \n> +#include \n> +\n> +{% if has_map %}#include {% endif %}\n> +{% if has_array %}#include {% endif %}\n> +\n> +namespace libcamera {\n> +{%- if has_namespace %}\n> +{% for ns in namespace %}\n> +namespace {{ns}} {\n> +{% endfor %}\n> +{%- endif %}\n> +\n> +{% for const in consts %}\n> +const {{const.kind|name}} {{const.mojom_name}} = {{const.value}};\n> +{% endfor %}\n> +\n> +enum class {{cmd_enum_name}} {\n> +\tExit = 0,\n> +{%- for method in interface_main.methods %}\n> +\t{{method.mojom_name|cap}} = {{loop.index}},\n> +{%- endfor %}\n> +};\n> +\n> +enum class {{cmd_event_enum_name}} {\n> +{%- for method in interface_event.methods %}\n> +\t{{method.mojom_name|cap}} = {{loop.index}},\n> +{%- endfor %}\n> +};\n> +\n> +{% for enum in enums %}\n> +{{funcs.define_enum(enum)}}\n> +{% endfor %}\n> +\n> +{%- for struct in structs_nonempty %}\n> +{{funcs.define_struct(struct)}}\n> +{% endfor %}\n> +\n> +{#-\n> +Any consts or #defines should be moved to the mojom file.\n> +#}\n> +class {{interface_name}} : public IPAInterface\n> +{\n> +public:\n> +{% for method in interface_main.methods %}\n> +\tvirtual {{method|method_return_value}} {{method.mojom_name}}(\n> +{%- for param in method|method_parameters %}\n> +\t\t{{param}}{{- \",\" if not loop.last}}\n> +{%- endfor -%}\n> +) = 0;\n> +{% endfor %}\n> +\n> +{%- for method in interface_event.methods %}\n> +\tSignal<\n> +{%- for param in method.parameters -%}\n> +\t\t{{\"const \" if not param|is_pod}}{{param|name}}{{\" &\" if not param|is_pod}}\n> +\t\t{{- \", \" if not loop.last}}\n> +{%- endfor -%}\n> +> {{method.mojom_name}};\n> +{% endfor -%}\n> +};\n> +\n> +{%- if has_namespace %}\n> +{% for ns in namespace|reverse %}\n> +} /* namespace {{ns}} */\n> +{% endfor %}\n> +{%- endif %}\n> +} /* namespace libcamera */\n> +\n> +#endif /* __LIBCAMERA_IPA_INTERFACE_{{module_name|upper}}_GENERATED_H__ */\n> diff --git a/utils/ipc/generators/libcamera_templates/module_ipa_proxy.cpp.tmpl b/utils/ipc/generators/libcamera_templates/module_ipa_proxy.cpp.tmpl\n> new file mode 100644\n> index 00000000..ba34a361\n> --- /dev/null\n> +++ b/utils/ipc/generators/libcamera_templates/module_ipa_proxy.cpp.tmpl\n> @@ -0,0 +1,236 @@\n> +{#-\n> + # SPDX-License-Identifier: LGPL-2.1-or-later\n> + # Copyright (C) 2020, Google Inc.\n> +-#}\n> +{%- import \"proxy_functions.tmpl\" as proxy_funcs -%}\n> +\n> +/* SPDX-License-Identifier: LGPL-2.1-or-later */\n> +/*\n> + * Copyright (C) 2020, Google Inc.\n> + *\n> + * {{module_name}}_ipa_proxy.cpp - Image Processing Algorithm proxy for {{module_name}}\n> + *\n> + * This file is auto-generated. Do not edit.\n> + */\n> +\n> +#include \n> +\n> +#include \n> +#include \n> +#include \n> +\n> +#include \n> +#include \n> +#include \n> +\n> +#include \"libcamera/internal/control_serializer.h\"\n> +#include \"libcamera/internal/ipa_data_serializer.h\"\n> +#include \"libcamera/internal/ipa_module.h\"\n> +#include \"libcamera/internal/ipa_proxy.h\"\n> +#include \"libcamera/internal/ipc_pipe.h\"\n> +#include \"libcamera/internal/ipc_pipe_unixsocket.h\"\n> +#include \"libcamera/internal/ipc_unixsocket.h\"\n> +#include \"libcamera/internal/log.h\"\n> +#include \"libcamera/internal/process.h\"\n> +#include \"libcamera/internal/thread.h\"\n> +\n> +namespace libcamera {\n> +\n> +LOG_DECLARE_CATEGORY(IPAProxy)\n> +\n> +{%- if has_namespace %}\n> +{% for ns in namespace %}\n> +namespace {{ns}} {\n> +{% endfor %}\n> +{%- endif %}\n> +\n> +{{proxy_name}}::{{proxy_name}}(IPAModule *ipam, bool isolate)\n> +\t: IPAProxy(ipam), running_(false),\n> +\t isolate_(isolate), seq_(0)\n> +{\n> +\tLOG(IPAProxy, Debug)\n> +\t\t<< \"initializing {{module_name}} proxy: loading IPA from \"\n> +\t\t<< ipam->path();\n> +\n> +\tif (isolate_) {\n> +\t\tconst std::string proxyWorkerPath = resolvePath(\"{{module_name}}_ipa_proxy\");\n> +\t\tif (proxyWorkerPath.empty()) {\n> +\t\t\tLOG(IPAProxy, Error)\n> +\t\t\t\t<< \"Failed to get proxy worker path\";\n> +\t\t\treturn;\n> +\t\t}\n> +\n> +\t\tipc_ = std::make_unique(ipam->path().c_str(),\n> +\t\t\t\t\t\t\t proxyWorkerPath.c_str());\n> +\t\tif (!ipc_->isConnected()) {\n> +\t\t\tLOG(IPAProxy, Error) << \"Failed to create IPCPipe\";\n> +\t\t\treturn;\n> +\t\t}\n> +\n> +\t\tipc_->recv.connect(this, &{{proxy_name}}::recvMessage);\n> +\n> +\t\tvalid_ = true;\n> +\t\treturn;\n> +\t}\n> +\n> +\tif (!ipam->load())\n> +\t\treturn;\n> +\n> +\tIPAInterface *ipai = ipam->createInterface();\n> +\tif (!ipai) {\n> +\t\tLOG(IPAProxy, Error)\n> +\t\t\t<< \"Failed to create IPA context for \" << ipam->path();\n> +\t\treturn;\n> +\t}\n> +\n> +\tipa_ = std::unique_ptr<{{interface_name}}>(static_cast<{{interface_name}} *>(ipai));\n> +\tproxy_.setIPA(ipa_.get());\n> +\n> +{% for method in interface_event.methods %}\n> +\tipa_->{{method.mojom_name}}.connect(this, &{{proxy_name}}::{{method.mojom_name}}Thread);\n> +{%- endfor %}\n> +\n> +\tvalid_ = true;\n> +}\n> +\n> +{{proxy_name}}::~{{proxy_name}}()\n> +{\n> +\tif (isolate_) {\n> +\t\tIPCMessage::Header header =\n> +\t\t\t{ static_cast({{cmd_enum_name}}::Exit), seq_++ };\n> +\t\tIPCMessage msg(header);\n> +\t\tipc_->sendAsync(msg);\n> +\t}\n> +}\n> +\n> +{% if interface_event.methods|length > 0 %}\n> +void {{proxy_name}}::recvMessage(const IPCMessage &data)\n> +{\n> +\tsize_t dataSize = data.data().size();\n> +\t{{cmd_event_enum_name}} _cmd = static_cast<{{cmd_event_enum_name}}>(data.header().cmd);\n> +\n> +\tswitch (_cmd) {\n> +{%- for method in interface_event.methods %}\n> +\tcase {{cmd_event_enum_name}}::{{method.mojom_name|cap}}: {\n> +\t\t{{method.mojom_name}}IPC(data.data().cbegin(), dataSize, data.fds());\n> +\t\tbreak;\n> +\t}\n> +{%- endfor %}\n> +\tdefault:\n> +\t\tLOG(IPAProxy, Error) << \"Unknown command \" << static_cast(_cmd);\n> +\t}\n> +}\n> +{%- endif %}\n> +\n> +{% for method in interface_main.methods %}\n> +{{proxy_funcs.func_sig(proxy_name, method)}}\n> +{\n> +\tif (isolate_)\n> +\t\t{{\"return \" if method|method_return_value != \"void\"}}{{method.mojom_name}}IPC(\n> +{%- for param in method|method_param_names -%}\n> +\t\t{{param}}{{- \", \" if not loop.last}}\n> +{%- endfor -%}\n> +);\n> +\telse\n> +\t\t{{\"return \" if method|method_return_value != \"void\"}}{{method.mojom_name}}Thread(\n> +{%- for param in method|method_param_names -%}\n> +\t\t{{param}}{{- \", \" if not loop.last}}\n> +{%- endfor -%}\n> +);\n> +}\n> +\n> +{{proxy_funcs.func_sig(proxy_name, method, \"Thread\")}}\n> +{\n> +{%- if method.mojom_name == \"init\" %}\n> +\t{{proxy_funcs.init_thread_body()}}\n> +{%- elif method.mojom_name == \"stop\" %}\n> +\t{{proxy_funcs.stop_thread_body()}}\n> +{%- elif method.mojom_name == \"start\" %}\n> +\trunning_ = true;\n> +\tthread_.start();\n> +\n> +\t{{ \"return \" if method|method_return_value != \"void\" -}}\n> +\tproxy_.invokeMethod(&ThreadProxy::start, ConnectionTypeBlocking\n> +\t{{- \", \" if method|method_param_names}}\n> +\t{%- for param in method|method_param_names -%}\n> +\t\t{{param}}{{- \", \" if not loop.last}}\n> +\t{%- endfor -%}\n> +);\n> +{%- elif not method|is_async %}\n> +\t{{ \"return \" if method|method_return_value != \"void\" -}}\n> +\tipa_->{{method.mojom_name}}(\n> +\t{%- for param in method|method_param_names -%}\n> +\t\t{{param}}{{- \", \" if not loop.last}}\n> +\t{%- endfor -%}\n> +);\n> +{% elif method|is_async %}\n> +\tproxy_.invokeMethod(&ThreadProxy::{{method.mojom_name}}, ConnectionTypeQueued,\n> +\t{%- for param in method|method_param_names -%}\n> +\t\t{{param}}{{- \", \" if not loop.last}}\n> +\t{%- endfor -%}\n> +);\n> +{%- endif %}\n> +}\n> +\n> +{{proxy_funcs.func_sig(proxy_name, method, \"IPC\")}}\n> +{\n> +{%- set has_input = true if method|method_param_inputs|length > 0 %}\n> +{%- set has_output = true if method|method_param_outputs|length > 0 or method|method_return_value != \"void\" %}\n> +{%- set cmd = cmd_enum_name + \"::\" + method.mojom_name|cap %}\n> +\tIPCMessage::Header _header = { static_cast({{cmd}}), seq_++ };\n> +\tIPCMessage _ipcInputBuf(_header);\n> +{%- if has_output %}\n> +\tIPCMessage _ipcOutputBuf;\n> +{%- endif %}\n> +\n> +{{proxy_funcs.serialize_call(method|method_param_inputs, '_ipcInputBuf.data()', '_ipcInputBuf.fds()')}}\n> +\n> +{% if method|is_async %}\n> +\tint _ret = ipc_->sendAsync(_ipcInputBuf);\n> +{%- else %}\n> +\tint _ret = ipc_->sendSync(_ipcInputBuf\n> +{{- \", &_ipcOutputBuf\" if has_output -}}\n> +);\n> +{%- endif %}\n> +\tif (_ret < 0) {\n> +\t\tLOG(IPAProxy, Error) << \"Failed to call {{method.mojom_name}}\";\n> +{%- if method|method_return_value != \"void\" %}\n> +\t\treturn static_cast<{{method|method_return_value}}>(_ret);\n> +{%- else %}\n> +\t\treturn;\n> +{%- endif %}\n> +\t}\n> +{% if method|method_return_value != \"void\" %}\n> +\treturn IPADataSerializer<{{method.response_parameters|first|name}}>::deserialize(_ipcOutputBuf.data(), 0);\n> +{% elif method|method_param_outputs|length > 0 %}\n> +{{proxy_funcs.deserialize_call(method|method_param_outputs, '_ipcOutputBuf.data()', '_ipcOutputBuf.fds()')}}\n> +{% endif -%}\n> +}\n> +\n> +{% endfor %}\n> +\n> +{% for method in interface_event.methods %}\n> +{{proxy_funcs.func_sig(proxy_name, method, \"Thread\")}}\n> +{\n> +\t{{method.mojom_name}}.emit({{method.parameters|params_comma_sep}});\n> +}\n> +\n> +void {{proxy_name}}::{{method.mojom_name}}IPC(\n> +\tstd::vector::const_iterator data,\n> +\tsize_t dataSize,\n> +\t[[maybe_unused]] const std::vector &fds)\n> +{\n> +{%- for param in method.parameters %}\n> +\t{{param|name}} {{param.mojom_name}};\n> +{%- endfor %}\n> +{{proxy_funcs.deserialize_call(method.parameters, 'data', 'fds', false, false, true, 'dataSize')}}\n> +\t{{method.mojom_name}}.emit({{method.parameters|params_comma_sep}});\n> +}\n> +{% endfor %}\n> +\n> +{%- if has_namespace %}\n> +{% for ns in namespace|reverse %}\n> +} /* namespace {{ns}} */\n> +{% endfor %}\n> +{%- endif %}\n> +} /* namespace libcamera */\n> diff --git a/utils/ipc/generators/libcamera_templates/module_ipa_proxy.h.tmpl b/utils/ipc/generators/libcamera_templates/module_ipa_proxy.h.tmpl\n> new file mode 100644\n> index 00000000..2bc187f2\n> --- /dev/null\n> +++ b/utils/ipc/generators/libcamera_templates/module_ipa_proxy.h.tmpl\n> @@ -0,0 +1,128 @@\n> +{#-\n> + # SPDX-License-Identifier: LGPL-2.1-or-later\n> + # Copyright (C) 2020, Google Inc.\n> +-#}\n> +{%- import \"proxy_functions.tmpl\" as proxy_funcs -%}\n> +\n> +/* SPDX-License-Identifier: LGPL-2.1-or-later */\n> +/*\n> + * Copyright (C) 2020, Google Inc.\n> + *\n> + * {{module_name}}_ipa_proxy.h - Image Processing Algorithm proxy for {{module_name}}\n> + *\n> + * This file is auto-generated. Do not edit.\n> + */\n> +\n> +#ifndef __LIBCAMERA_INTERNAL_IPA_PROXY_{{module_name|upper}}_H__\n> +#define __LIBCAMERA_INTERNAL_IPA_PROXY_{{module_name|upper}}_H__\n> +\n> +#include \n> +#include \n> +\n> +#include \"libcamera/internal/control_serializer.h\"\n> +#include \"libcamera/internal/ipa_proxy.h\"\n> +#include \"libcamera/internal/ipc_pipe.h\"\n> +#include \"libcamera/internal/ipc_pipe_unixsocket.h\"\n> +#include \"libcamera/internal/ipc_unixsocket.h\"\n> +#include \"libcamera/internal/thread.h\"\n> +\n> +namespace libcamera {\n> +{%- if has_namespace %}\n> +{% for ns in namespace %}\n> +namespace {{ns}} {\n> +{% endfor %}\n> +{%- endif %}\n> +\n> +class {{proxy_name}} : public IPAProxy, public {{interface_name}}, public Object\n> +{\n> +public:\n> +\t{{proxy_name}}(IPAModule *ipam, bool isolate);\n> +\t~{{proxy_name}}();\n> +\n> +{% for method in interface_main.methods %}\n> +{{proxy_funcs.func_sig(proxy_name, method, \"\", false, true)|indent(8, true)}};\n> +{% endfor %}\n> +\n> +{%- for method in interface_event.methods %}\n> +\tSignal<\n> +{%- for param in method.parameters -%}\n> +\t\t{{\"const \" if not param|is_pod}}{{param|name}}{{\" &\" if not param|is_pod}}\n> +\t\t{{- \", \" if not loop.last}}\n> +{%- endfor -%}\n> +> {{method.mojom_name}};\n> +{% endfor %}\n> +\n> +private:\n> +\tvoid recvMessage(const IPCMessage &data);\n> +\n> +{% for method in interface_main.methods %}\n> +{{proxy_funcs.func_sig(proxy_name, method, \"Thread\", false)|indent(8, true)}};\n> +{{proxy_funcs.func_sig(proxy_name, method, \"IPC\", false)|indent(8, true)}};\n> +{% endfor %}\n> +{% for method in interface_event.methods %}\n> +{{proxy_funcs.func_sig(proxy_name, method, \"Thread\", false)|indent(8, true)}};\n> +\tvoid {{method.mojom_name}}IPC(\n> +\t\tstd::vector::const_iterator data,\n> +\t\tsize_t dataSize,\n> +\t\tconst std::vector &fds);\n> +{% endfor %}\n> +\n> +\t/* Helper class to invoke async functions in another thread. */\n> +\tclass ThreadProxy : public Object\n> +\t{\n> +\tpublic:\n> +\t\tvoid setIPA({{interface_name}} *ipa)\n> +\t\t{\n> +\t\t\tipa_ = ipa;\n> +\t\t}\n> +\n> +\t\tvoid stop()\n> +\t\t{\n> +\t\t\tipa_->stop();\n> +\t\t}\n> +{% for method in interface_main.methods %}\n> +{%- if method|is_async %}\n> +\t\t{{proxy_funcs.func_sig(proxy_name, method, \"\", false)|indent(16)}}\n> +\t\t{\n> +\t\t\tipa_->{{method.mojom_name}}({{method.parameters|params_comma_sep}});\n> +\t\t}\n> +{%- elif method.mojom_name == \"start\" %}\n> +\t\t{{proxy_funcs.func_sig(proxy_name, method, \"\", false)|indent(16)}}\n> +\t\t{\n> +{%- if method|method_return_value != \"void\" %}\n> +\t\t\treturn ipa_->{{method.mojom_name}}({{method.parameters|params_comma_sep}});\n> +{%- else %}\n> +\t\t\tipa_->{{method.mojom_name}}({{method.parameters|params_comma_sep}}\n> +\t{{- \", \" if method|method_param_outputs|params_comma_sep -}}\n> +\t{{- method|method_param_outputs|params_comma_sep}});\n> +{%- endif %}\n> +\t\t}\n> +{%- endif %}\n> +{%- endfor %}\n> +\n> +\tprivate:\n> +\t\t{{interface_name}} *ipa_;\n> +\t};\n> +\n> +\tbool running_;\n> +\tThread thread_;\n> +\tThreadProxy proxy_;\n> +\tstd::unique_ptr<{{interface_name}}> ipa_;\n> +\n> +\tconst bool isolate_;\n> +\n> +\tstd::unique_ptr ipc_;\n> +\n> +\tControlSerializer controlSerializer_;\n> +\n> +\tuint32_t seq_;\n> +};\n> +\n> +{%- if has_namespace %}\n> +{% for ns in namespace|reverse %}\n> +} /* namespace {{ns}} */\n> +{% endfor %}\n> +{%- endif %}\n> +} /* namespace libcamera */\n> +\n> +#endif /* __LIBCAMERA_INTERNAL_IPA_PROXY_{{module_name|upper}}_H__ */\n> diff --git a/utils/ipc/generators/libcamera_templates/module_ipa_proxy_worker.cpp.tmpl b/utils/ipc/generators/libcamera_templates/module_ipa_proxy_worker.cpp.tmpl\n> new file mode 100644\n> index 00000000..ac037fa1\n> --- /dev/null\n> +++ b/utils/ipc/generators/libcamera_templates/module_ipa_proxy_worker.cpp.tmpl\n> @@ -0,0 +1,226 @@\n> +{#-\n> + # SPDX-License-Identifier: LGPL-2.1-or-later\n> + # Copyright (C) 2020, Google Inc.\n> +-#}\n> +{%- import \"proxy_functions.tmpl\" as proxy_funcs -%}\n> +\n> +/* SPDX-License-Identifier: LGPL-2.1-or-later */\n> +/*\n> + * Copyright (C) 2020, Google Inc.\n> + *\n> + * {{module_name}}_ipa_proxy_worker.cpp - Image Processing Algorithm proxy worker for {{module_name}}\n> + *\n> + * This file is auto-generated. Do not edit.\n> + */\n> +\n> +{#- \\todo Split proxy worker into IPC worker and proxy worker. #}\n> +\n> +#include \n> +#include \n> +#include \n> +#include \n> +#include \n> +\n> +#include \n> +#include \n> +#include \n> +#include \n> +\n> +#include \"libcamera/internal/camera_sensor.h\"\n> +#include \"libcamera/internal/control_serializer.h\"\n> +#include \"libcamera/internal/event_dispatcher.h\"\n> +#include \"libcamera/internal/ipa_data_serializer.h\"\n> +#include \"libcamera/internal/ipa_module.h\"\n> +#include \"libcamera/internal/ipa_proxy.h\"\n> +#include \"libcamera/internal/ipc_pipe.h\"\n> +#include \"libcamera/internal/ipc_pipe_unixsocket.h\"\n> +#include \"libcamera/internal/ipc_unixsocket.h\"\n> +#include \"libcamera/internal/log.h\"\n> +#include \"libcamera/internal/thread.h\"\n> +\n> +using namespace libcamera;\n> +\n> +LOG_DEFINE_CATEGORY({{proxy_worker_name}})\n> +\n> +{%- if has_namespace %}\n> +{% for ns in namespace -%}\n> +using namespace {{ns}};\n> +{% endfor %}\n> +{%- endif %}\n> +\n> +class {{proxy_worker_name}}\n> +{\n> +public:\n> +\t{{proxy_worker_name}}()\n> +\t\t: ipa_(nullptr), exit_(false) {}\n> +\n> +\t~{{proxy_worker_name}}() {}\n> +\n> +\tvoid readyRead(IPCUnixSocket *socket)\n> +\t{\n> +\t\tIPCUnixSocket::Payload _message;\n> +\t\tint _retRecv = socket->receive(&_message);\n> +\t\tif (_retRecv) {\n> +\t\t\tLOG({{proxy_worker_name}}, Error)\n> +\t\t\t\t<< \"Receive message failed: \" << _retRecv;\n> +\t\t\treturn;\n> +\t\t}\n> +\n> +\t\tIPCMessage _ipcMessage(_message);\n> +\n> +\t\t{{cmd_enum_name}} _cmd = static_cast<{{cmd_enum_name}}>(_ipcMessage.header().cmd);\n> +\n> +\t\tswitch (_cmd) {\n> +\t\tcase {{cmd_enum_name}}::Exit: {\n> +\t\t\texit_ = true;\n> +\t\t\tbreak;\n> +\t\t}\n> +\n> +{% for method in interface_main.methods %}\n> +\t\tcase {{cmd_enum_name}}::{{method.mojom_name|cap}}: {\n> +\t\t{{proxy_funcs.deserialize_call(method|method_param_inputs, '_ipcMessage.data()', '_ipcMessage.fds()', false, true)|indent(8, true)}}\n> +{% for param in method|method_param_outputs %}\n> +\t\t\t{{param|name}} {{param.mojom_name}};\n> +{% endfor %}\n> +{%- if method|method_return_value != \"void\" %}\n> +\t\t\t{{method|method_return_value}} _callRet = ipa_->{{method.mojom_name}}({{method.parameters|params_comma_sep}});\n> +{%- else %}\n> +\t\t\tipa_->{{method.mojom_name}}({{method.parameters|params_comma_sep}}\n> +{{- \", \" if method|method_param_outputs|params_comma_sep -}}\n> +{%- for param in method|method_param_outputs -%}\n> +&{{param.mojom_name}}{{\", \" if not loop.last}}\n> +{%- endfor -%}\n> +);\n> +{%- endif %}\n> +{% if not method|is_async %}\n> +\t\t\tIPCMessage::Header header = { _ipcMessage.header().cmd, _ipcMessage.header().cookie };\n> +\t\t\tIPCMessage _response(header);\n> +{%- if method|method_return_value != \"void\" %}\n> +\t\t\tstd::vector _callRetBuf;\n> +\t\t\tstd::tie(_callRetBuf, std::ignore) =\n> +\t\t\t\tIPADataSerializer<{{method|method_return_value}}>::serialize(_callRet);\n> +\t\t\t_response.data().insert(_response.data().end(), _callRetBuf.cbegin(), _callRetBuf.cend());\n> +{%- else %}\n> +\t\t{{proxy_funcs.serialize_call(method|method_param_outputs, \"_response.data()\", \"_response.fds()\")|indent(16, true)}}\n> +{%- endif %}\n> +\t\t\tint _ret = socket_.send(_response.payload());\n> +\t\t\tif (_ret < 0) {\n> +\t\t\t\tLOG({{proxy_worker_name}}, Error)\n> +\t\t\t\t\t<< \"Reply to {{method.mojom_name}}() failed: \" << _ret;\n> +\t\t\t}\n> +\t\t\tLOG({{proxy_worker_name}}, Debug) << \"Done replying to {{method.mojom_name}}()\";\n> +{%- endif %}\n> +\t\t\tbreak;\n> +\t\t}\n> +{% endfor %}\n> +\t\tdefault:\n> +\t\t\tLOG({{proxy_worker_name}}, Error) << \"Unknown command \" << _ipcMessage.header().cmd;\n> +\t\t}\n> +\t}\n> +\n> +\tint init(std::unique_ptr &ipam, int socketfd)\n> +\t{\n> +\t\tif (socket_.bind(socketfd) < 0) {\n> +\t\t\tLOG({{proxy_worker_name}}, Error)\n> +\t\t\t\t<< \"IPC socket binding failed\";\n> +\t\t\treturn EXIT_FAILURE;\n> +\t\t}\n> +\t\tsocket_.readyRead.connect(this, &{{proxy_worker_name}}::readyRead);\n> +\n> +\t\tipa_ = dynamic_cast<{{interface_name}} *>(ipam->createInterface());\n> +\t\tif (!ipa_) {\n> +\t\t\tLOG({{proxy_worker_name}}, Error)\n> +\t\t\t\t<< \"Failed to create IPA interface instance\";\n> +\t\t\treturn EXIT_FAILURE;\n> +\t\t}\n> +{% for method in interface_event.methods %}\n> +\t\tipa_->{{method.mojom_name}}.connect(this, &{{proxy_worker_name}}::{{method.mojom_name}});\n> +{%- endfor %}\n> +\t\treturn 0;\n> +\t}\n> +\n> +\tvoid run()\n> +\t{\n> +\t\tEventDispatcher *dispatcher = Thread::current()->eventDispatcher();\n> +\t\twhile (!exit_)\n> +\t\t\tdispatcher->processEvents();\n> +\t}\n> +\n> +\tvoid cleanup()\n> +\t{\n> +\t\tdelete ipa_;\n> +\t\tsocket_.close();\n> +\t}\n> +\n> +private:\n> +\n> +{% for method in interface_event.methods %}\n> +{{proxy_funcs.func_sig(proxy_name, method, \"\", false)|indent(8, true)}}\n> +\t{\n> +\t\tIPCMessage::Header header = {\n> +\t\t\tstatic_cast({{cmd_event_enum_name}}::{{method.mojom_name|cap}}),\n> +\t\t\t0\n> +\t\t};\n> +\t\tIPCMessage _message(header);\n> +\n> +\t\t{{proxy_funcs.serialize_call(method|method_param_inputs, \"_message.data()\", \"_message.fds()\")}}\n> +\n> +\t\tsocket_.send(_message.payload());\n> +\n> +\t\tLOG({{proxy_worker_name}}, Debug) << \"{{method.mojom_name}} done\";\n> +\t}\n> +{% endfor %}\n> +\n> +\t{{interface_name}} *ipa_;\n> +\tIPCUnixSocket socket_;\n> +\n> +\tControlSerializer controlSerializer_;\n> +\n> +\tbool exit_;\n> +};\n> +\n> +int main(int argc, char **argv)\n> +{\n> +{#- \\todo Handle enabling debugging more dynamically. #}\n> +\t/* Uncomment this for debugging. */\n> +#if 0\n> +\tstd::string logPath = \"/tmp/libcamera.worker.\" +\n> +\t\t\t std::to_string(getpid()) + \".log\";\n> +\tlogSetFile(logPath.c_str());\n> +#endif\n> +\n> +\tif (argc < 3) {\n> +\t\tLOG({{proxy_worker_name}}, Error)\n> +\t\t\t<< \"Tried to start worker with no args: \"\n> +\t\t\t<< \"expected \";\n> +\t\treturn EXIT_FAILURE;\n> +\t}\n> +\n> +\tint fd = std::stoi(argv[2]);\n> +\tLOG({{proxy_worker_name}}, Info)\n> +\t\t<< \"Starting worker for IPA module \" << argv[1]\n> +\t\t<< \" with IPC fd = \" << fd;\n> +\n> +\tstd::unique_ptr ipam = std::make_unique(argv[1]);\n> +\tif (!ipam->isValid() || !ipam->load()) {\n> +\t\tLOG({{proxy_worker_name}}, Error)\n> +\t\t\t<< \"IPAModule \" << argv[1] << \" isn't valid\";\n> +\t\treturn EXIT_FAILURE;\n> +\t}\n> +\n> +\t{{proxy_worker_name}} proxyWorker;\n> +\tint ret = proxyWorker.init(ipam, fd);\n> +\tif (ret < 0) {\n> +\t\tLOG({{proxy_worker_name}}, Error)\n> +\t\t\t<< \"Failed to initialize proxy worker\";\n> +\t\treturn ret;\n> +\t}\n> +\n> +\tLOG({{proxy_worker_name}}, Debug) << \"Proxy worker successfully initialized\";\n> +\n> +\tproxyWorker.run();\n> +\n> +\tproxyWorker.cleanup();\n> +\n> +\treturn 0;\n> +}\n> diff --git a/utils/ipc/generators/libcamera_templates/module_ipa_serializer.h.tmpl b/utils/ipc/generators/libcamera_templates/module_ipa_serializer.h.tmpl\n> new file mode 100644\n> index 00000000..64ae99dc\n> --- /dev/null\n> +++ b/utils/ipc/generators/libcamera_templates/module_ipa_serializer.h.tmpl\n> @@ -0,0 +1,48 @@\n> +{#-\n> + # SPDX-License-Identifier: LGPL-2.1-or-later\n> + # Copyright (C) 2020, Google Inc.\n> +-#}\n> +{%- import \"serializer.tmpl\" as serializer -%}\n> +\n> +/* SPDX-License-Identifier: LGPL-2.1-or-later */\n> +/*\n> + * Copyright (C) 2020, Google Inc.\n> + *\n> + * {{module_name}}_ipa_serializer.h - Image Processing Algorithm data serializer for {{module_name}}\n> + *\n> + * This file is auto-generated. Do not edit.\n> + */\n> +\n> +#ifndef __LIBCAMERA_INTERNAL_IPA_DATA_SERIALIZER_{{module_name|upper}}_H__\n> +#define __LIBCAMERA_INTERNAL_IPA_DATA_SERIALIZER_{{module_name|upper}}_H__\n> +\n> +#include \n> +#include \n> +\n> +#include \n> +#include \n> +\n> +#include \"libcamera/internal/control_serializer.h\"\n> +#include \"libcamera/internal/ipa_data_serializer.h\"\n> +\n> +namespace libcamera {\n> +\n> +LOG_DECLARE_CATEGORY(IPADataSerializer)\n> +{% for struct in structs_nonempty %}\n> +template<>\n> +class IPADataSerializer<{{struct|name_full(namespace_str)}}>\n> +{\n> +public:\n> +{{- serializer.serializer(struct, namespace_str)}}\n> +{%- if struct|has_fd %}\n> +{{serializer.deserializer_fd(struct, namespace_str)}}\n> +{%- else %}\n> +{{serializer.deserializer_no_fd(struct, namespace_str)}}\n> +{{serializer.deserializer_fd_simple(struct, namespace_str)}}\n> +{%- endif %}\n> +};\n> +{% endfor %}\n> +\n> +} /* namespace libcamera */\n> +\n> +#endif /* __LIBCAMERA_INTERNAL_IPA_DATA_SERIALIZER_{{module_name|upper}}_H__ */\n> diff --git a/utils/ipc/generators/libcamera_templates/proxy_functions.tmpl b/utils/ipc/generators/libcamera_templates/proxy_functions.tmpl\n> new file mode 100644\n> index 00000000..40611feb\n> --- /dev/null\n> +++ b/utils/ipc/generators/libcamera_templates/proxy_functions.tmpl\n> @@ -0,0 +1,194 @@\n> +{#-\n> + # SPDX-License-Identifier: LGPL-2.1-or-later\n> + # Copyright (C) 2020, Google Inc.\n> +-#}\n> +{#\n> + # \\brief Generate function prototype\n> + #\n> + # \\param class Class name\n> + # \\param method mojom Method object\n> + # \\param suffix Suffix to append to \\a method function name\n> + # \\param need_class_name If true, generate class name with function\n> + # \\param override If true, generate override tag after the function prototype\n> + #}\n> +{%- macro func_sig(class, method, suffix = \"\", need_class_name = true, override = false) -%}\n> +{{method|method_return_value}} {{class + \"::\" if need_class_name}}{{method.mojom_name}}{{suffix}}(\n> +{%- for param in method|method_parameters %}\n> +\t{{param}}{{- \",\" if not loop.last}}\n> +{%- endfor -%}\n> +){{\" override\" if override}}\n> +{%- endmacro -%}\n> +\n> +{#\n> + # \\brief Generate function body for IPA init() function for thread\n> + #}\n> +{%- macro init_thread_body() -%}\n> +\tint ret = ipa_->init(settings);\n> +\tif (ret)\n> +\t\treturn ret;\n> +\n> +\tproxy_.moveToThread(&thread_);\n> +\n> +\treturn 0;\n> +{%- endmacro -%}\n> +\n> +{#\n> + # \\brief Generate function body for IPA stop() function for thread\n> + #}\n> +{%- macro stop_thread_body() -%}\n> +\tif (!running_)\n> +\t\treturn;\n> +\n> +\trunning_ = false;\n> +\n> +\tproxy_.invokeMethod(&ThreadProxy::stop, ConnectionTypeBlocking);\n> +\n> +\tthread_.exit();\n> +\tthread_.wait();\n> +{%- endmacro -%}\n> +\n> +\n> +{#\n> + # \\brief Serialize multiple objects into data buffer and fd vector\n> + #\n> + # Generate code to serialize multiple objects, as specified in \\a params\n> + # (which are the parameters to some function), into \\a buf data buffer and\n> + # \\a fds fd vector.\n> + # This code is meant to be used by the proxy, for serializing prior to IPC calls.\n> + #\n> + # \\todo Avoid intermediate vectors\n> + #}\n> +{%- macro serialize_call(params, buf, fds) %}\n> +{%- for param in params %}\n> +\tstd::vector {{param.mojom_name}}Buf;\n> +{%- if param|has_fd %}\n> +\tstd::vector {{param.mojom_name}}Fds;\n> +\tstd::tie({{param.mojom_name}}Buf, {{param.mojom_name}}Fds) =\n> +{%- else %}\n> +\tstd::tie({{param.mojom_name}}Buf, std::ignore) =\n> +{%- endif %}\n> +\t\tIPADataSerializer<{{param|name}}>::serialize({{param.mojom_name}}\n> +{{- \", &controlSerializer_\" if param|needs_control_serializer -}}\n> +);\n> +{%- endfor %}\n> +\n> +{%- if params|length > 1 %}\n> +{%- for param in params %}\n> +\tappendPOD({{buf}}, {{param.mojom_name}}Buf.size());\n> +{%- if param|has_fd %}\n> +\tappendPOD({{buf}}, {{param.mojom_name}}Fds.size());\n> +{%- endif %}\n> +{%- endfor %}\n> +{%- endif %}\n> +\n> +{%- for param in params %}\n> +\t{{buf}}.insert({{buf}}.end(), {{param.mojom_name}}Buf.begin(), {{param.mojom_name}}Buf.end());\n> +{%- endfor %}\n> +\n> +{%- for param in params %}\n> +{%- if param|has_fd %}\n> +\t{{fds}}.insert({{fds}}.end(), {{param.mojom_name}}Fds.begin(), {{param.mojom_name}}Fds.end());\n> +{%- endif %}\n> +{%- endfor %}\n> +{%- endmacro -%}\n> +\n> +\n> +{#\n> + # \\brief Deserialize a single object from data buffer and fd vector\n> + #\n> + # \\param pointer If true, deserialize the object into a dereferenced pointer\n> + # \\param iter If true, treat \\a buf as an iterator instead of a vector\n> + # \\param data_size Variable that holds the size of the vector referenced by \\a buf\n> + #\n> + # Generate code to deserialize a single object, as specified in \\a param,\n> + # from \\a buf data buffer and \\a fds fd vector.\n> + # This code is meant to be used by macro deserialize_call.\n> + #}\n> +{%- macro deserialize_param(param, pointer, loop, buf, fds, iter, data_size) -%}\n> +{{\"*\" if pointer}}{{param.mojom_name}} = IPADataSerializer<{{param|name}}>::deserialize(\n> +\t{{buf}}{{- \".cbegin()\" if not iter}} + {{param.mojom_name}}Start,\n> +{%- if loop.last and not iter %}\n> +\t{{buf}}.cend()\n> +{%- elif not iter %}\n> +\t{{buf}}.cbegin() + {{param.mojom_name}}Start + {{param.mojom_name}}BufSize\n> +{%- elif iter and loop.length == 1 %}\n> +\t{{buf}} + {{data_size}}\n> +{%- else %}\n> +\t{{buf}} + {{param.mojom_name}}Start + {{param.mojom_name}}BufSize\n> +{%- endif -%}\n> +{{- \",\" if param|has_fd}}\n> +{%- if param|has_fd %}\n> +\t{{fds}}.cbegin() + {{param.mojom_name}}FdStart,\n> +{%- if loop.last %}\n> +\t{{fds}}.cend()\n> +{%- else %}\n> +\t{{fds}}.cbegin() + {{param.mojom_name}}FdStart + {{param.mojom_name}}FdsSize\n> +{%- endif -%}\n> +{%- endif -%}\n> +{{- \",\" if param|needs_control_serializer}}\n> +{%- if param|needs_control_serializer %}\n> +\t&controlSerializer_\n> +{%- endif -%}\n> +);\n> +{%- endmacro -%}\n> +\n> +\n> +{#\n> + # \\brief Deserialize multiple objects from data buffer and fd vector\n> + #\n> + # \\param pointer If true, deserialize objects into pointers, and adds a null check.\n> + # \\param declare If true, declare the objects in addition to deserialization.\n> + # \\param iter if true, treat \\a buf as an iterator instead of a vector\n> + # \\param data_size Variable that holds the size of the vector referenced by \\a buf\n> + #\n> + # Generate code to deserialize multiple objects, as specified in \\a params\n> + # (which are the parameters to some function), from \\a buf data buffer and\n> + # \\a fds fd vector.\n> + # This code is meant to be used by the proxy, for deserializing after IPC calls.\n> + #\n> + # \\todo Avoid intermediate vectors\n> + #}\n> +{%- macro deserialize_call(params, buf, fds, pointer = true, declare = false, iter = false, data_size = '') -%}\n> +{% set ns = namespace(size_offset = 0) %}\n> +{%- if params|length > 1 %}\n> +{%- for param in params %}\n> +\t[[maybe_unused]] const size_t {{param.mojom_name}}BufSize = readPOD({{buf}}, {{ns.size_offset}}\n> +{%- if iter -%}\n> +, {{buf}} + {{data_size}}\n> +{%- endif -%}\n> +);\n> +\t{%- set ns.size_offset = ns.size_offset + 4 %}\n> +{%- if param|has_fd %}\n> +\t[[maybe_unused]] const size_t {{param.mojom_name}}FdsSize = readPOD({{buf}}, {{ns.size_offset}}\n> +{%- if iter -%}\n> +, {{buf}} + {{data_size}}\n> +{%- endif -%}\n> +);\n> +\t{%- set ns.size_offset = ns.size_offset + 4 %}\n> +{%- endif %}\n> +{%- endfor %}\n> +{%- endif %}\n> +{% for param in params %}\n> +{%- if loop.first %}\n> +\tconst size_t {{param.mojom_name}}Start = {{ns.size_offset}};\n> +{%- else %}\n> +\tconst size_t {{param.mojom_name}}Start = {{loop.previtem.mojom_name}}Start + {{loop.previtem.mojom_name}}BufSize;\n> +{%- endif %}\n> +{%- endfor %}\n> +{% for param in params|with_fds %}\n> +{%- if loop.first %}\n> +\tconst size_t {{param.mojom_name}}FdStart = 0;\n> +{%- elif not loop.last %}\n> +\tconst size_t {{param.mojom_name}}FdStart = {{loop.previtem.mojom_name}}FdStart + {{loop.previtem.mojom_name}}FdsSize;\n> +{%- endif %}\n> +{%- endfor %}\n> +{% for param in params %}\n> +\t{%- if pointer %}\n> +\tif ({{param.mojom_name}}) {\n> +{{deserialize_param(param, pointer, loop, buf, fds, iter, data_size)|indent(16, True)}}\n> +\t}\n> +\t{%- else %}\n> +\t{{param|name + \" \" if declare}}{{deserialize_param(param, pointer, loop, buf, fds, iter, data_size)|indent(8)}}\n> +\t{%- endif %}\n> +{% endfor %}\n> +{%- endmacro -%}\n> diff --git a/utils/ipc/generators/libcamera_templates/serializer.tmpl b/utils/ipc/generators/libcamera_templates/serializer.tmpl\n> new file mode 100644\n> index 00000000..af35b9e3\n> --- /dev/null\n> +++ b/utils/ipc/generators/libcamera_templates/serializer.tmpl\n> @@ -0,0 +1,313 @@\n> +{#-\n> + # SPDX-License-Identifier: LGPL-2.1-or-later\n> + # Copyright (C) 2020, Google Inc.\n> +-#}\n> +{#\n> + # \\brief Verify that there is enough bytes to deserialize\n> + #\n> + # Generate code that verifies that \\a size is not greater than \\a dataSize.\n> + # Otherwise log an error with \\a name and \\a typename.\n> + #}\n> +{%- macro check_data_size(size, dataSize, name, typename) %}\n> +\t\tif ({{dataSize}} < {{size}}) {\n> +\t\t\tLOG(IPADataSerializer, Error)\n> +\t\t\t\t<< \"Failed to deserialize \" << \"{{name}}\"\n> +\t\t\t\t<< \": not enough {{typename}}, expected \"\n> +\t\t\t\t<< ({{size}}) << \", got \" << ({{dataSize}});\n> +\t\t\treturn ret;\n> +\t\t}\n> +{%- endmacro %}\n> +\n> +\n> +{#\n> + # \\brief Serialize a field into return vector\n> + #\n> + # Generate code to serialize \\a field into retData, including size of the\n> + # field and fds (where appropriate).\n> + # This code is meant to be used by the IPADataSerializer specialization.\n> + #\n> + # \\todo Avoid intermediate vectors\n> + #}\n> +{%- macro serializer_field(field, namespace, loop) %}\n> +{%- if field|is_pod or field|is_enum %}\n> +\t\tstd::vector {{field.mojom_name}};\n> +\t\tstd::tie({{field.mojom_name}}, std::ignore) =\n> +\t{%- if field|is_pod %}\n> +\t\t\tIPADataSerializer<{{field|name}}>::serialize(data.{{field.mojom_name}});\n> +\t{%- elif field|is_enum %}\n> +\t\t\tIPADataSerializer::serialize(data.{{field.mojom_name}});\n> +\t{%- endif %}\n> +\t\tretData.insert(retData.end(), {{field.mojom_name}}.begin(), {{field.mojom_name}}.end());\n> +{%- elif field|is_fd %}\n> +\t\tstd::vector {{field.mojom_name}};\n> +\t\tstd::vector {{field.mojom_name}}Fds;\n> +\t\tstd::tie({{field.mojom_name}}, {{field.mojom_name}}Fds) =\n> +\t\t\tIPADataSerializer<{{field|name}}>::serialize(data.{{field.mojom_name}});\n> +\t\tretData.insert(retData.end(), {{field.mojom_name}}.begin(), {{field.mojom_name}}.end());\n> +\t\tretFds.insert(retFds.end(), {{field.mojom_name}}Fds.begin(), {{field.mojom_name}}Fds.end());\n> +{%- elif field|is_controls %}\n> +\t\tif (data.{{field.mojom_name}}.size() > 0) {\n> +\t\t\tstd::vector {{field.mojom_name}};\n> +\t\t\tstd::tie({{field.mojom_name}}, std::ignore) =\n> +\t\t\t\tIPADataSerializer<{{field|name}}>::serialize(data.{{field.mojom_name}}, cs);\n> +\t\t\tappendPOD(retData, {{field.mojom_name}}.size());\n> +\t\t\tretData.insert(retData.end(), {{field.mojom_name}}.begin(), {{field.mojom_name}}.end());\n> +\t\t} else {\n> +\t\t\tappendPOD(retData, 0);\n> +\t\t}\n> +{%- elif field|is_plain_struct or field|is_array or field|is_map or field|is_str %}\n> +\t\tstd::vector {{field.mojom_name}};\n> +\t{%- if field|has_fd %}\n> +\t\tstd::vector {{field.mojom_name}}Fds;\n> +\t\tstd::tie({{field.mojom_name}}, {{field.mojom_name}}Fds) =\n> +\t{%- else %}\n> +\t\tstd::tie({{field.mojom_name}}, std::ignore) =\n> +\t{%- endif %}\n> +\t{%- if field|is_array or field|is_map %}\n> +\t\t\tIPADataSerializer<{{field|name}}>::serialize(data.{{field.mojom_name}}, cs);\n> +\t{%- elif field|is_str %}\n> +\t\t\tIPADataSerializer<{{field|name}}>::serialize(data.{{field.mojom_name}});\n> +\t{%- else %}\n> +\t\t\tIPADataSerializer<{{field|name_full(namespace)}}>::serialize(data.{{field.mojom_name}}, cs);\n> +\t{%- endif %}\n> +\t\tappendPOD(retData, {{field.mojom_name}}.size());\n> +\t{%- if field|has_fd %}\n> +\t\tappendPOD(retData, {{field.mojom_name}}Fds.size());\n> +\t{%- endif %}\n> +\t\tretData.insert(retData.end(), {{field.mojom_name}}.begin(), {{field.mojom_name}}.end());\n> +\t{%- if field|has_fd %}\n> +\t\tretFds.insert(retFds.end(), {{field.mojom_name}}Fds.begin(), {{field.mojom_name}}Fds.end());\n> +\t{%- endif %}\n> +{%- else %}\n> +\t\t/* Unknown serialization for {{field.mojom_name}}. */\n> +{%- endif %}\n> +{%- endmacro %}\n> +\n> +\n> +{#\n> + # \\brief Deserialize a field into return struct\n> + #\n> + # Generate code to deserialize \\a field into object ret.\n> + # This code is meant to be used by the IPADataSerializer specialization.\n> + #}\n> +{%- macro deserializer_field(field, namespace, loop) %}\n> +{% if field|is_pod or field|is_enum %}\n> +\t{%- set field_size = (field|bit_width|int / 8)|int %}\n> +\t\t{{- check_data_size(field_size, 'dataSize', field.mojom_name, 'data')}}\n> +\t\t{%- if field|is_pod %}\n> +\t\tret.{{field.mojom_name}} = IPADataSerializer<{{field|name}}>::deserialize(m, m + {{field_size}});\n> +\t\t{%- else %}\n> +\t\tret.{{field.mojom_name}} = static_cast<{{field|name_full(namespace)}}>(IPADataSerializer::deserialize(m, m + {{field_size}}));\n> +\t\t{%- endif %}\n> +\t{%- if not loop.last %}\n> +\t\tm += {{field_size}};\n> +\t\tdataSize -= {{field_size}};\n> +\t{%- endif %}\n> +{% elif field|is_fd %}\n> +\t{%- set field_size = 1 %}\n> +\t\t{{- check_data_size(field_size, 'dataSize', field.mojom_name, 'data')}}\n> +\t\tret.{{field.mojom_name}} = IPADataSerializer<{{field|name}}>::deserialize(m, m + 1, n, n + 1, cs);\n> +\t{%- if not loop.last %}\n> +\t\tm += {{field_size}};\n> +\t\tdataSize -= {{field_size}};\n> +\t\tn += ret.{{field.mojom_name}}.isValid() ? 1 : 0;\n> +\t\tfdsSize -= ret.{{field.mojom_name}}.isValid() ? 1 : 0;\n> +\t{%- endif %}\n> +{% elif field|is_controls %}\n> +\t{%- set field_size = 4 %}\n> +\t\t{{- check_data_size(field_size, 'dataSize', field.mojom_name + 'Size', 'data')}}\n> +\t\tconst size_t {{field.mojom_name}}Size = readPOD(m, 0, dataEnd);\n> +\t\tm += {{field_size}};\n> +\t\tdataSize -= {{field_size}};\n> +\t{%- set field_size = field.mojom_name + 'Size' -%}\n> +\t\t{{- check_data_size(field_size, 'dataSize', field.mojom_name, 'data')}}\n> +\t\tif ({{field.mojom_name}}Size > 0)\n> +\t\t\tret.{{field.mojom_name}} =\n> +\t\t\t\tIPADataSerializer<{{field|name}}>::deserialize(m, m + {{field.mojom_name}}Size, cs);\n> +\t{%- if not loop.last %}\n> +\t\tm += {{field_size}};\n> +\t\tdataSize -= {{field_size}};\n> +\t{%- endif %}\n> +{% elif field|is_plain_struct or field|is_array or field|is_map or field|is_str %}\n> +\t{%- set field_size = 4 %}\n> +\t\t{{- check_data_size(field_size, 'dataSize', field.mojom_name + 'Size', 'data')}}\n> +\t\tconst size_t {{field.mojom_name}}Size = readPOD(m, 0, dataEnd);\n> +\t\tm += {{field_size}};\n> +\t\tdataSize -= {{field_size}};\n> +\t{%- if field|has_fd %}\n> +\t{%- set field_size = 4 %}\n> +\t\t{{- check_data_size(field_size, 'dataSize', field.mojom_name + 'FdsSize', 'data')}}\n> +\t\tconst size_t {{field.mojom_name}}FdsSize = readPOD(m, 0, dataEnd);\n> +\t\tm += {{field_size}};\n> +\t\tdataSize -= {{field_size}};\n> +\t\t{{- check_data_size(field.mojom_name + 'FdsSize', 'fdsSize', field.mojom_name, 'fds')}}\n> +\t{%- endif %}\n> +\t{%- set field_size = field.mojom_name + 'Size' -%}\n> +\t\t{{- check_data_size(field_size, 'dataSize', field.mojom_name, 'data')}}\n> +\t\tret.{{field.mojom_name}} =\n> +\t{%- if field|is_str %}\n> +\t\t\tIPADataSerializer<{{field|name}}>::deserialize(m, m + {{field.mojom_name}}Size);\n> +\t{%- elif field|has_fd and (field|is_array or field|is_map) %}\n> +\t\t\tIPADataSerializer<{{field|name}}>::deserialize(m, m + {{field.mojom_name}}Size, n, n + {{field.mojom_name}}FdsSize, cs);\n> +\t{%- elif field|has_fd and (not (field|is_array or field|is_map)) %}\n> +\t\t\tIPADataSerializer<{{field|name_full(namespace)}}>::deserialize(m, m + {{field.mojom_name}}Size, n, n + {{field.mojom_name}}FdsSize, cs);\n> +\t{%- elif (not field|has_fd) and (field|is_array or field|is_map) %}\n> +\t\t\tIPADataSerializer<{{field|name}}>::deserialize(m, m + {{field.mojom_name}}Size, cs);\n> +\t{%- else %}\n> +\t\t\tIPADataSerializer<{{field|name_full(namespace)}}>::deserialize(m, m + {{field.mojom_name}}Size, cs);\n> +\t{%- endif %}\n> +\t{%- if not loop.last %}\n> +\t\tm += {{field_size}};\n> +\t\tdataSize -= {{field_size}};\n> +\t{%- if field|has_fd %}\n> +\t\tn += {{field.mojom_name}}FdsSize;\n> +\t\tfdsSize -= {{field.mojom_name}}FdsSize;\n> +\t{%- endif %}\n> +\t{%- endif %}\n> +{% else %}\n> +\t\t/* Unknown deserialization for {{field.mojom_name}}. */\n> +{%- endif %}\n> +{%- endmacro %}\n> +\n> +\n> +{#\n> + # \\brief Serialize a struct\n> + #\n> + # Generate code for IPADataSerializer specialization, for serializing\n> + # \\a struct.\n> + #}\n> +{%- macro serializer(struct, namespace) %}\n> +\tstatic std::tuple, std::vector>\n> +\tserialize(const {{struct|name_full(namespace)}} &data,\n> +{%- if struct|needs_control_serializer %}\n> +\t\t ControlSerializer *cs)\n> +{%- else %}\n> +\t\t [[maybe_unused]] ControlSerializer *cs = nullptr)\n> +{%- endif %}\n> +\t{\n> +\t\tstd::vector retData;\n> +{%- if struct|has_fd %}\n> +\t\tstd::vector retFds;\n> +{%- endif %}\n> +{%- for field in struct.fields %}\n> +{{serializer_field(field, namespace, loop)}}\n> +{%- endfor %}\n> +{% if struct|has_fd %}\n> +\t\treturn {retData, retFds};\n> +{%- else %}\n> +\t\treturn {retData, {}};\n> +{%- endif %}\n> +\t}\n> +{%- endmacro %}\n> +\n> +\n> +{#\n> + # \\brief Deserialize a struct that has fds\n> + #\n> + # Generate code for IPADataSerializer specialization, for deserializing\n> + # \\a struct, in the case that \\a struct has file descriptors.\n> + #}\n> +{%- macro deserializer_fd(struct, namespace) %}\n> +\tstatic {{struct|name_full(namespace)}}\n> +\tdeserialize(std::vector &data,\n> +\t\t std::vector &fds,\n> +{%- if struct|needs_control_serializer %}\n> +\t\t ControlSerializer *cs)\n> +{%- else %}\n> +\t\t ControlSerializer *cs = nullptr)\n> +{%- endif %}\n> +\t{\n> +\t\treturn IPADataSerializer<{{struct|name_full(namespace)}}>::deserialize(data.cbegin(), data.cend(), fds.cbegin(), fds.cend(), cs);\n> +\t}\n> +\n> +{# \\todo Don't inline this function #}\n> +\tstatic {{struct|name_full(namespace)}}\n> +\tdeserialize(std::vector::const_iterator dataBegin,\n> +\t\t std::vector::const_iterator dataEnd,\n> +\t\t std::vector::const_iterator fdsBegin,\n> +\t\t std::vector::const_iterator fdsEnd,\n> +{%- if struct|needs_control_serializer %}\n> +\t\t ControlSerializer *cs)\n> +{%- else %}\n> +\t\t [[maybe_unused]] ControlSerializer *cs = nullptr)\n> +{%- endif %}\n> +\t{\n> +\t\t{{struct|name_full(namespace)}} ret;\n> +\t\tstd::vector::const_iterator m = dataBegin;\n> +\t\tstd::vector::const_iterator n = fdsBegin;\n> +\n> +\t\tsize_t dataSize = std::distance(dataBegin, dataEnd);\n> +\t\t[[maybe_unused]] size_t fdsSize = std::distance(fdsBegin, fdsEnd);\n> +{%- for field in struct.fields -%}\n> +{{deserializer_field(field, namespace, loop)}}\n> +{%- endfor %}\n> +\t\treturn ret;\n> +\t}\n> +{%- endmacro %}\n> +\n> +{#\n> + # \\brief Deserialize a struct that has fds, using non-fd\n> + #\n> + # Generate code for IPADataSerializer specialization, for deserializing\n> + # \\a struct, in the case that \\a struct has no file descriptors but requires\n> + # deserializers with file descriptors.\n> + #}\n> +{%- macro deserializer_fd_simple(struct, namespace) %}\n> +\tstatic {{struct|name_full(namespace)}}\n> +\tdeserialize(std::vector &data,\n> +\t\t [[maybe_unused]] std::vector &fds,\n> +\t\t ControlSerializer *cs = nullptr)\n> +\t{\n> +\t\treturn IPADataSerializer<{{struct|name_full(namespace)}}>::deserialize(data.cbegin(), data.cend(), cs);\n> +\t}\n> +\n> +\tstatic {{struct|name_full(namespace)}}\n> +\tdeserialize(std::vector::const_iterator dataBegin,\n> +\t\t std::vector::const_iterator dataEnd,\n> +\t\t [[maybe_unused]] std::vector::const_iterator fdsBegin,\n> +\t\t [[maybe_unused]] std::vector::const_iterator fdsEnd,\n> +\t\t ControlSerializer *cs = nullptr)\n> +\t{\n> +\t\treturn IPADataSerializer<{{struct|name_full(namespace)}}>::deserialize(dataBegin, dataEnd, cs);\n> +\t}\n> +{%- endmacro %}\n> +\n> +\n> +{#\n> + # \\brief Deserialize a struct that has no fds\n> + #\n> + # Generate code for IPADataSerializer specialization, for deserializing\n> + # \\a struct, in the case that \\a struct does not have file descriptors.\n> + #}\n> +{%- macro deserializer_no_fd(struct, namespace) %}\n> +\tstatic {{struct|name_full(namespace)}}\n> +\tdeserialize(std::vector &data,\n> +{%- if struct|needs_control_serializer %}\n> +\t\t ControlSerializer *cs)\n> +{%- else %}\n> +\t\t ControlSerializer *cs = nullptr)\n> +{%- endif %}\n> +\t{\n> +\t\treturn IPADataSerializer<{{struct|name_full(namespace)}}>::deserialize(data.cbegin(), data.cend(), cs);\n> +\t}\n> +\n> +{# \\todo Don't inline this function #}\n> +\tstatic {{struct|name_full(namespace)}}\n> +\tdeserialize(std::vector::const_iterator dataBegin,\n> +\t\t std::vector::const_iterator dataEnd,\n> +{%- if struct|needs_control_serializer %}\n> +\t\t ControlSerializer *cs)\n> +{%- else %}\n> +\t\t [[maybe_unused]] ControlSerializer *cs = nullptr)\n> +{%- endif %}\n> +\t{\n> +\t\t{{struct|name_full(namespace)}} ret;\n> +\t\tstd::vector::const_iterator m = dataBegin;\n> +\n> +\t\tsize_t dataSize = std::distance(dataBegin, dataEnd);\n> +{%- for field in struct.fields -%}\n> +{{deserializer_field(field, namespace, loop)}}\n> +{%- endfor %}\n> +\t\treturn ret;\n> +\t}\n> +{%- endmacro %}\n> diff --git a/utils/ipc/generators/mojom_libcamera_generator.py b/utils/ipc/generators/mojom_libcamera_generator.py\n> new file mode 100644\n> index 00000000..438e41c6\n> --- /dev/null\n> +++ b/utils/ipc/generators/mojom_libcamera_generator.py\n> @@ -0,0 +1,508 @@\n> +#!/usr/bin/env python3\n> +# SPDX-License-Identifier: GPL-2.0-or-later\n> +# Copyright (C) 2020, Google Inc.\n> +#\n> +# Author: Paul Elder \n> +#\n> +# mojom_libcamera_generator.py - Generates libcamera files from a mojom.Module.\n> +\n> +import argparse\n> +import datetime\n> +import os\n> +import re\n> +\n> +import mojom.fileutil as fileutil\n> +import mojom.generate.generator as generator\n> +import mojom.generate.module as mojom\n> +from mojom.generate.template_expander import UseJinja\n> +\n> +\n> +GENERATOR_PREFIX = 'libcamera'\n> +\n> +_kind_to_cpp_type = {\n> + mojom.BOOL: 'bool',\n> + mojom.INT8: 'int8_t',\n> + mojom.UINT8: 'uint8_t',\n> + mojom.INT16: 'int16_t',\n> + mojom.UINT16: 'uint16_t',\n> + mojom.INT32: 'int32_t',\n> + mojom.UINT32: 'uint32_t',\n> + mojom.FLOAT: 'float',\n> + mojom.INT64: 'int64_t',\n> + mojom.UINT64: 'uint64_t',\n> + mojom.DOUBLE: 'double',\n> +}\n> +\n> +_bit_widths = {\n> + mojom.BOOL: '8',\n> + mojom.INT8: '8',\n> + mojom.UINT8: '8',\n> + mojom.INT16: '16',\n> + mojom.UINT16: '16',\n> + mojom.INT32: '32',\n> + mojom.UINT32: '32',\n> + mojom.FLOAT: '32',\n> + mojom.INT64: '64',\n> + mojom.UINT64: '64',\n> + mojom.DOUBLE: '64',\n> +}\n> +\n> +def ModuleName(path):\n> + return path.split('/')[-1].split('.')[0]\n> +\n> +def ModuleClassName(module):\n> + return re.sub(r'^IPA(.*)Interface$', lambda match: match.group(1),\n> + module.interfaces[0].mojom_name)\n> +\n> +def Capitalize(name):\n> + return name[0].upper() + name[1:]\n> +\n> +def ConstantStyle(name):\n> + return generator.ToUpperSnakeCase(name)\n> +\n> +def Choose(cond, t, f):\n> + return t if cond else f\n> +\n> +def CommaSep(l):\n> + return ', '.join([m for m in l])\n> +\n> +def ParamsCommaSep(l):\n> + return ', '.join([m.mojom_name for m in l])\n> +\n> +def GetDefaultValue(element):\n> + if element.default is not None:\n> + return element.default\n> + if type(element.kind) == mojom.Kind:\n> + return '0'\n> + if mojom.IsEnumKind(element.kind):\n> + return f'static_cast<{element.kind.mojom_name}>(0)'\n> + if isinstance(element.kind, mojom.Struct) and \\\n> + element.kind.mojom_name == 'FileDescriptor':\n> + return '-1'\n> + return ''\n> +\n> +def HasDefaultValue(element):\n> + return GetDefaultValue(element) != ''\n> +\n> +def HasDefaultFields(element):\n> + return True in [HasDefaultValue(x) for x in element.fields]\n> +\n> +def GetAllTypes(element):\n> + if mojom.IsArrayKind(element):\n> + return GetAllTypes(element.kind)\n> + if mojom.IsMapKind(element):\n> + return GetAllTypes(element.key_kind) + GetAllTypes(element.value_kind)\n> + if isinstance(element, mojom.Parameter):\n> + return GetAllTypes(element.kind)\n> + if mojom.IsEnumKind(element):\n> + return [element.mojom_name]\n> + if not mojom.IsStructKind(element):\n> + return [element.spec]\n> + if len(element.fields) == 0:\n> + return [element.mojom_name]\n> + ret = [GetAllTypes(x.kind) for x in element.fields]\n> + ret = [x for sublist in ret for x in sublist]\n> + return list(set(ret))\n> +\n> +def GetAllAttrs(element):\n> + if mojom.IsArrayKind(element):\n> + return GetAllAttrs(element.kind)\n> + if mojom.IsMapKind(element):\n> + return {**GetAllAttrs(element.key_kind), **GetAllAttrs(element.value_kind)}\n> + if isinstance(element, mojom.Parameter):\n> + return GetAllAttrs(element.kind)\n> + if mojom.IsEnumKind(element):\n> + return element.attributes if element.attributes is not None else {}\n> + if mojom.IsStructKind(element) and len(element.fields) == 0:\n> + return element.attributes if element.attributes is not None else {}\n> + if not mojom.IsStructKind(element):\n> + if hasattr(element, 'attributes'):\n> + return element.attributes or {}\n> + return {}\n> + attrs = [(x.attributes) for x in element.fields]\n> + ret = {}\n> + for d in attrs:\n> + ret.update(d or {})\n> + if hasattr(element, 'attributes'):\n> + ret.update(element.attributes or {})\n> + return ret\n> +\n> +def NeedsControlSerializer(element):\n> + types = GetAllTypes(element)\n> + return \"ControlList\" in types or \"ControlInfoMap\" in types\n> +\n> +def HasFd(element):\n> + attrs = GetAllAttrs(element)\n> + if isinstance(element, mojom.Kind):\n> + types = GetAllTypes(element)\n> + else:\n> + types = GetAllTypes(element.kind)\n> + return \"FileDescriptor\" in types or (attrs is not None and \"hasFd\" in attrs)\n> +\n> +def WithDefaultValues(element):\n> + return [x for x in element if HasDefaultValue(x)]\n> +\n> +def WithFds(element):\n> + return [x for x in element if HasFd(x)]\n> +\n> +def MethodParamInputs(method):\n> + return method.parameters\n> +\n> +def MethodParamOutputs(method):\n> + if (MethodReturnValue(method) != 'void' or\n> + method.response_parameters is None):\n> + return []\n> + return method.response_parameters\n> +\n> +def MethodParamsHaveFd(parameters):\n> + return len([x for x in parameters if HasFd(x)]) > 0\n> +\n> +def MethodInputHasFd(method):\n> + return MethodParamsHaveFd(method.parameters)\n> +\n> +def MethodOutputHasFd(method):\n> + return MethodParamsHaveFd(MethodParamOutputs(method))\n> +\n> +def MethodParamNames(method):\n> + params = []\n> + for param in method.parameters:\n> + params.append(param.mojom_name)\n> + if MethodReturnValue(method) == 'void':\n> + if method.response_parameters is None:\n> + return params\n> + for param in method.response_parameters:\n> + params.append(param.mojom_name)\n> + return params\n> +\n> +def MethodParameters(method):\n> + params = []\n> + for param in method.parameters:\n> + params.append('const %s %s%s' % (GetNameForElement(param),\n> + '&' if not IsPod(param) else '',\n> + param.mojom_name))\n> + if MethodReturnValue(method) == 'void':\n> + if method.response_parameters is None:\n> + return params\n> + for param in method.response_parameters:\n> + params.append(f'{GetNameForElement(param)} *{param.mojom_name}')\n> + return params\n> +\n> +def MethodReturnValue(method):\n> + if method.response_parameters is None:\n> + return 'void'\n> + if len(method.response_parameters) == 1 and IsPod(method.response_parameters[0]):\n> + return GetNameForElement(method.response_parameters[0])\n> + return 'void'\n> +\n> +def IsAsync(method):\n> + # Events are always async\n> + if re.match(\"^IPA.*EventInterface$\", method.interface.mojom_name):\n> + return True\n> + elif re.match(\"^IPA.*Interface$\", method.interface.mojom_name):\n> + if method.attributes is None:\n> + return False\n> + elif 'async' in method.attributes and method.attributes['async']:\n> + return True\n> + return False\n> +\n> +def IsArray(element):\n> + return mojom.IsArrayKind(element.kind)\n> +\n> +def IsControls(element):\n> + return mojom.IsStructKind(element.kind) and (element.kind.mojom_name == \"ControlList\" or\n> + element.kind.mojom_name == \"ControlInfoMap\")\n> +\n> +def IsEnum(element):\n> + return mojom.IsEnumKind(element.kind)\n> +\n> +def IsFd(element):\n> + return mojom.IsStructKind(element.kind) and element.kind.mojom_name == \"FileDescriptor\"\n> +\n> +def IsMap(element):\n> + return mojom.IsMapKind(element.kind)\n> +\n> +def IsPlainStruct(element):\n> + return mojom.IsStructKind(element.kind) and not IsControls(element) and not IsFd(element)\n> +\n> +def IsPod(element):\n> + return element.kind in _kind_to_cpp_type\n> +\n> +def IsStr(element):\n> + return element.kind.spec == 's'\n> +\n> +def BitWidth(element):\n> + if element.kind in _bit_widths:\n> + return _bit_widths[element.kind]\n> + if mojom.IsEnumKind(element.kind):\n> + return '32'\n> + return ''\n> +\n> +# Get the type name for a given element\n> +def GetNameForElement(element):\n> + # structs\n> + if (mojom.IsEnumKind(element) or\n> + mojom.IsInterfaceKind(element) or\n> + mojom.IsStructKind(element)):\n> + return element.mojom_name\n> + # vectors\n> + if (mojom.IsArrayKind(element)):\n> + elem_name = GetNameForElement(element.kind)\n> + return f'std::vector<{elem_name}>'\n> + # maps\n> + if (mojom.IsMapKind(element)):\n> + key_name = GetNameForElement(element.key_kind)\n> + value_name = GetNameForElement(element.value_kind)\n> + return f'std::map<{key_name}, {value_name}>'\n> + # struct fields and function parameters\n> + if isinstance(element, (mojom.Field, mojom.Method, mojom.Parameter)):\n> + # maps and vectors\n> + if (mojom.IsArrayKind(element.kind) or mojom.IsMapKind(element.kind)):\n> + return GetNameForElement(element.kind)\n> + # strings\n> + if (mojom.IsReferenceKind(element.kind) and element.kind.spec == 's'):\n> + return 'std::string'\n> + # PODs\n> + if element.kind in _kind_to_cpp_type:\n> + return _kind_to_cpp_type[element.kind]\n> + # structs and enums\n> + return element.kind.mojom_name\n> + # PODs that are members of vectors/maps\n> + if (hasattr(element, '__hash__') and element in _kind_to_cpp_type):\n> + return _kind_to_cpp_type[element]\n> + if (hasattr(element, 'spec')):\n> + # strings that are members of vectors/maps\n> + if (element.spec == 's'):\n> + return 'std::string'\n> + # structs that aren't defined in mojom that are members of vectors/maps\n> + if (element.spec[0] == 'x'):\n> + return element.spec.replace('x:', '').replace('.', '::')\n> + if (mojom.IsInterfaceRequestKind(element) or\n> + mojom.IsAssociatedKind(element) or\n> + mojom.IsPendingRemoteKind(element) or\n> + mojom.IsPendingReceiverKind(element) or\n> + mojom.IsUnionKind(element)):\n> + raise Exception('Unsupported element: %s' % element)\n> + raise Exception('Unexpected element: %s' % element)\n> +\n> +def GetFullNameForElement(element, namespace_str):\n> + name = GetNameForElement(element)\n> + if namespace_str == '':\n> + return name\n> + return f'{namespace_str}::{name}'\n> +\n> +def ValidateZeroLength(l, s, cap=True):\n> + if l is None:\n> + return\n> + if len(l) > 0:\n> + raise Exception(f'{s.capitalize() if cap else s} should be empty')\n> +\n> +def ValidateSingleLength(l, s, cap=True):\n> + if len(l) > 1:\n> + raise Exception(f'Only one {s} allowed')\n> + if len(l) < 1:\n> + raise Exception(f'{s.capitalize() if cap else s} is required')\n> +\n> +def GetMainInterface(interfaces):\n> + intf = [x for x in interfaces\n> + if re.match(\"^IPA.*Interface\", x.mojom_name) and\n> + not re.match(\"^IPA.*EventInterface\", x.mojom_name)]\n> + ValidateSingleLength(intf, 'main interface')\n> + return None if len(intf) == 0 else intf[0]\n> +\n> +def GetEventInterface(interfaces):\n> + event = [x for x in interfaces if re.match(\"^IPA.*EventInterface\", x.mojom_name)]\n> + ValidateSingleLength(event, 'event interface')\n> + return None if len(event) == 0 else event[0]\n> +\n> +def ValidateNamespace(namespace):\n> + if namespace == '':\n> + raise Exception('Must have a namespace')\n> +\n> + if not re.match('^ipa\\.[0-9A-Za-z_]+', namespace):\n> + raise Exception('Namespace must be of the form \"ipa.{pipeline_name}\"')\n> +\n> +def ValidateInterfaces(interfaces):\n> + # Validate presence of main interface\n> + intf = GetMainInterface(interfaces)\n> + if intf is None:\n> + raise Exception('Must have main IPA interface')\n> +\n> + # Validate presence of event interface\n> + event = GetEventInterface(interfaces)\n> + if intf is None:\n> + raise Exception('Must have event IPA interface')\n> +\n> + # Validate required main interface functions\n> + f_init = [x for x in intf.methods if x.mojom_name == 'init']\n> + f_start = [x for x in intf.methods if x.mojom_name == 'start']\n> + f_stop = [x for x in intf.methods if x.mojom_name == 'stop']\n> +\n> + ValidateSingleLength(f_init, 'init()', False)\n> + ValidateSingleLength(f_start, 'start()', False)\n> + ValidateSingleLength(f_stop, 'stop()', False)\n> +\n> + f_init = f_init[0]\n> + f_start = f_start[0]\n> + f_stop = f_stop[0]\n> +\n> + # Validate parameters to init()\n> + ValidateSingleLength(f_init.parameters, 'input parameter to init()')\n> + ValidateSingleLength(f_init.response_parameters, 'output parameter from init()')\n> + if f_init.parameters[0].kind.mojom_name != 'IPASettings':\n> + raise Exception('init() must have single IPASettings input parameter')\n> + if f_init.response_parameters[0].kind.spec != 'i32':\n> + raise Exception('init() must have single int32 output parameter')\n> +\n> + # No need to validate start() as it is customizable\n> +\n> + # Validate parameters to stop()\n> + ValidateZeroLength(f_stop.parameters, 'input parameter to stop()')\n> + ValidateZeroLength(f_stop.parameters, 'output parameter from stop()')\n> +\n> + # Validate that event interface has at least one event\n> + if len(event.methods) < 1:\n> + raise Exception('Event interface must have at least one event')\n> +\n> + # Validate that all async methods don't have return values\n> + intf_methods_async = [x for x in intf.methods if IsAsync(x)]\n> + for method in intf_methods_async:\n> + ValidateZeroLength(method.response_parameters,\n> + f'{method.mojom_name} response parameters', False)\n> +\n> + event_methods_async = [x for x in event.methods if IsAsync(x)]\n> + for method in event_methods_async:\n> + ValidateZeroLength(method.response_parameters,\n> + f'{method.mojom_name} response parameters', False)\n> +\n> +class Generator(generator.Generator):\n> + @staticmethod\n> + def GetTemplatePrefix():\n> + return 'libcamera_templates'\n> +\n> + def GetFilters(self):\n> + libcamera_filters = {\n> + 'all_types': GetAllTypes,\n> + 'bit_width': BitWidth,\n> + 'cap': Capitalize,\n> + 'choose': Choose,\n> + 'comma_sep': CommaSep,\n> + 'default_value': GetDefaultValue,\n> + 'has_default_fields': HasDefaultFields,\n> + 'has_fd': HasFd,\n> + 'is_async': IsAsync,\n> + 'is_array': IsArray,\n> + 'is_controls': IsControls,\n> + 'is_enum': IsEnum,\n> + 'is_fd': IsFd,\n> + 'is_map': IsMap,\n> + 'is_plain_struct': IsPlainStruct,\n> + 'is_pod': IsPod,\n> + 'is_str': IsStr,\n> + 'method_input_has_fd': MethodInputHasFd,\n> + 'method_output_has_fd': MethodOutputHasFd,\n> + 'method_param_names': MethodParamNames,\n> + 'method_param_inputs': MethodParamInputs,\n> + 'method_param_outputs': MethodParamOutputs,\n> + 'method_parameters': MethodParameters,\n> + 'method_return_value': MethodReturnValue,\n> + 'name': GetNameForElement,\n> + 'name_full': GetFullNameForElement,\n> + 'needs_control_serializer': NeedsControlSerializer,\n> + 'params_comma_sep': ParamsCommaSep,\n> + 'with_default_values': WithDefaultValues,\n> + 'with_fds': WithFds,\n> + }\n> + return libcamera_filters\n> +\n> + def _GetJinjaExports(self):\n> + return {\n> + 'cmd_enum_name': '_%sCmd' % self.module_name,\n> + 'cmd_event_enum_name': '_%sEventCmd' % self.module_name,\n> + 'consts': self.module.constants,\n> + 'enums': self.module.enums,\n> + 'has_array': len([x for x in self.module.kinds.keys() if x[0] == 'a']) > 0,\n> + 'has_map': len([x for x in self.module.kinds.keys() if x[0] == 'm']) > 0,\n> + 'has_namespace': self.module.mojom_namespace != '',\n> + 'interface_event': GetEventInterface(self.module.interfaces),\n> + 'interface_main': GetMainInterface(self.module.interfaces),\n> + 'interface_name': 'IPA%sInterface' % self.module_name,\n> + 'module_name': ModuleName(self.module.path),\n> + 'namespace': self.module.mojom_namespace.split('.'),\n> + 'namespace_str': self.module.mojom_namespace.replace('.', '::') if\n> + self.module.mojom_namespace is not None else '',\n> + 'proxy_name': 'IPAProxy%s' % self.module_name,\n> + 'proxy_worker_name': 'IPAProxy%sWorker' % self.module_name,\n> + 'structs_nonempty': [x for x in self.module.structs if len(x.fields) > 0],\n> + }\n> +\n> + def _GetJinjaExportsForCore(self):\n> + return {\n> + 'consts': self.module.constants,\n> + 'enums': self.module.enums,\n> + 'has_array': len([x for x in self.module.kinds.keys() if x[0] == 'a']) > 0,\n> + 'has_map': len([x for x in self.module.kinds.keys() if x[0] == 'm']) > 0,\n> + 'structs_gen_header': [x for x in self.module.structs if x.attributes is None or 'skipHeader' not in x.attributes],\n> + 'structs_gen_serializer': [x for x in self.module.structs if x.attributes is None or 'skipSerdes' not in x.attributes],\n> + }\n> +\n> + @UseJinja('core_ipa_interface.h.tmpl')\n> + def _GenerateCoreHeader(self):\n> + return self._GetJinjaExportsForCore()\n> +\n> + @UseJinja('core_ipa_serializer.h.tmpl')\n> + def _GenerateCoreSerializer(self):\n> + return self._GetJinjaExportsForCore()\n> +\n> + @UseJinja('module_ipa_interface.h.tmpl')\n> + def _GenerateDataHeader(self):\n> + return self._GetJinjaExports()\n> +\n> + @UseJinja('module_ipa_serializer.h.tmpl')\n> + def _GenerateSerializer(self):\n> + return self._GetJinjaExports()\n> +\n> + @UseJinja('module_ipa_proxy.cpp.tmpl')\n> + def _GenerateProxyCpp(self):\n> + return self._GetJinjaExports()\n> +\n> + @UseJinja('module_ipa_proxy.h.tmpl')\n> + def _GenerateProxyHeader(self):\n> + return self._GetJinjaExports()\n> +\n> + @UseJinja('module_ipa_proxy_worker.cpp.tmpl')\n> + def _GenerateProxyWorker(self):\n> + return self._GetJinjaExports()\n> +\n> + def GenerateFiles(self, unparsed_args):\n> + parser = argparse.ArgumentParser()\n> + parser.add_argument('--libcamera_generate_core_header', action='store_true')\n> + parser.add_argument('--libcamera_generate_core_serializer', action='store_true')\n> + parser.add_argument('--libcamera_generate_header', action='store_true')\n> + parser.add_argument('--libcamera_generate_serializer', action='store_true')\n> + parser.add_argument('--libcamera_generate_proxy_cpp', action='store_true')\n> + parser.add_argument('--libcamera_generate_proxy_h', action='store_true')\n> + parser.add_argument('--libcamera_generate_proxy_worker', action='store_true')\n> + parser.add_argument('--libcamera_output_path')\n> + args = parser.parse_args(unparsed_args)\n> +\n> + if not args.libcamera_generate_core_header and \\\n> + not args.libcamera_generate_core_serializer:\n> + ValidateNamespace(self.module.mojom_namespace)\n> + ValidateInterfaces(self.module.interfaces)\n> + self.module_name = ModuleClassName(self.module)\n> +\n> + fileutil.EnsureDirectoryExists(os.path.dirname(args.libcamera_output_path))\n> +\n> + gen_funcs = [\n> + [args.libcamera_generate_core_header, self._GenerateCoreHeader],\n> + [args.libcamera_generate_core_serializer, self._GenerateCoreSerializer],\n> + [args.libcamera_generate_header, self._GenerateDataHeader],\n> + [args.libcamera_generate_serializer, self._GenerateSerializer],\n> + [args.libcamera_generate_proxy_cpp, self._GenerateProxyCpp],\n> + [args.libcamera_generate_proxy_h, self._GenerateProxyHeader],\n> + [args.libcamera_generate_proxy_worker, self._GenerateProxyWorker],\n> + ]\n> +\n> + for pair in gen_funcs:\n> + if pair[0]:\n> + self.Write(pair[1](), args.libcamera_output_path)","headers":{"Return-Path":"","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id 8A7A3BD160\n\tfor ;\n\tFri, 12 Feb 2021 00:25:28 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id DC0C463755;\n\tFri, 12 Feb 2021 01:25:27 +0100 (CET)","from perceval.ideasonboard.com (perceval.ideasonboard.com\n\t[213.167.242.64])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id 51EA8602FC\n\tfor ;\n\tFri, 12 Feb 2021 01:25:26 +0100 (CET)","from pendragon.ideasonboard.com (62-78-145-57.bb.dnainternet.fi\n\t[62.78.145.57])\n\tby perceval.ideasonboard.com (Postfix) with ESMTPSA id 667068B5;\n\tFri, 12 Feb 2021 01:25:25 +0100 (CET)"],"Authentication-Results":"lancelot.ideasonboard.com;\n\tdkim=fail reason=\"signature verification failed\" (1024-bit key;\n\tunprotected) header.d=ideasonboard.com header.i=@ideasonboard.com\n\theader.b=\"Vlw83VQH\"; dkim-atps=neutral","DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com;\n\ts=mail; t=1613089525;\n\tbh=vmGOrFjbAH0h7LtmXCaMd5Zukri3znME3sWkpJ6vn+Q=;\n\th=Date:From:To:Cc:Subject:References:In-Reply-To:From;\n\tb=Vlw83VQHv+TPAPEa9i7XfaOzHGcWcURb/8nCcriyAlafjePpvPtCAD5XqR6oOgZMH\n\twG/ULZHYbbyWWdPl9H14tWtAGjCX9yH1Gyj3IpsXBafZo54mDuvu2HNw/mjyH1p1XE\n\tJgPy0+xjjsYuW+sb/2JRDg1Ug0AYjZ14BwUKqCyU=","Date":"Fri, 12 Feb 2021 02:25:00 +0200","From":"Laurent Pinchart ","To":"Paul Elder ","Message-ID":"","References":"<20210211071805.34963-1-paul.elder@ideasonboard.com>\n\t<20210211071805.34963-3-paul.elder@ideasonboard.com>","MIME-Version":"1.0","Content-Disposition":"inline","In-Reply-To":"<20210211071805.34963-3-paul.elder@ideasonboard.com>","Subject":"Re: [libcamera-devel] [PATCH v7 02/10] utils: ipc: add templates\n\tfor code generation for IPC mechanism","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Cc":"libcamera-devel@lists.libcamera.org","Content-Type":"text/plain; charset=\"us-ascii\"","Content-Transfer-Encoding":"7bit","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" "}},{"id":15119,"web_url":"https://patchwork.libcamera.org/comment/15119/","msgid":"","date":"2021-02-12T02:01:27","subject":"Re: [libcamera-devel] [PATCH v7 02/10] utils: ipc: add templates\n\tfor code generation for IPC mechanism","submitter":{"id":2,"url":"https://patchwork.libcamera.org/api/people/2/","name":"Laurent Pinchart","email":"laurent.pinchart@ideasonboard.com"},"content":"Hi Paul,\n\nAnother comment.\n\nOn Fri, Feb 12, 2021 at 02:25:00AM +0200, Laurent Pinchart wrote:\n> On Thu, Feb 11, 2021 at 04:17:57PM +0900, Paul Elder wrote:\n> > Add templates to mojo to generate code for the IPC mechanism. These\n> > templates generate:\n> > - module header\n> > - module serializer\n> > - IPA proxy cpp, header, and worker\n> > \n> > Given an input data definition mojom file for a pipeline.\n> > \n> > Signed-off-by: Paul Elder \n> > Acked-by: Jacopo Mondi \n> > Acked-by: Kieran Bingham \n> > Reviewed-by: Laurent Pinchart \n> > \n> > ---\n> > Changes in v7:\n> > - cosmetic changes, add a few todos\n> > - use the new sendSync/sendAsync IPCPipe API\n> > - save the sequence number in the proxy\n> > - construct the header before calling sendSync/sendAsync (from the\n> > proxy)\n> > - replace genHeader and genSerdes with skipHeader and skipSerdes in\n> > core.mojom\n> > \n> > Changes in v6:\n> > - add templates for core_ipa_interface.h and core_ipa_serializer.h\n> > - for libcamera types defined in mojom\n> > - rename everything to {{module_name}}_ipa_{interface,proxy,proxy_worker}.{c,h}\n> > - remove #include > - support customizable start()\n> > - remove the need for per-pipeline ControlInfoMap\n> > - add todo for avoiding intermediate vectors\n> > - remove postfix underscore for generated struct fields\n> > - support structs that are members of vectors/maps that aren't defined\n> > in mojom (in mojom)\n> > - fix has_fd detection\n> > - namespacing is now required in mojom, in the form of ^ipa\\.[0-9A-Za-z_]+\n> > - support consts in mojom\n> > - make the pseudo-switch-case in the python generator nicer\n> > \n> > Changes in v5:\n> > - add a usage output to the proxy worker, to document the interface for\n> > executing the proxy worker\n> > - in the mojom generator python script:\n> > - removed unused things (imports, functions, jinja exports)\n> > - document GetNameForElement\n> > - rename everything cb -> event\n> > - refactor Method{Input,Output}HasFd with a helper MethodParamsHaveFd\n> > - add Get{Main,Event}Interface to fix the interface_{main,event} jinja\n> > exports\n> > - add copyright\n> > - require that event interfaces have at least one event\n> > - expand copyright for templates\n> > - use new sendSync/sendAsync API (with IPCMessage)\n> > - rename a bunch of things\n> > \n> > Changes in v4:\n> > For the non-template files:\n> > - rename IPA{pipeline_name}CallbackInterface to\n> > IPA{pipeline_name}EventInterface\n> > - to avoid the notion of \"callback\" and emphasize that it's an event\n> > - add support for strings in custom structs\n> > - add validation, that async methods must not have return values\n> > - it throws exception and isn't very clear though...?\n> > - rename controls to libcamera::{pipeline_name}::controls (controls is\n> > now lowercase)\n> > - rename {pipeline_name}_generated.h to {pipeline_name}_ipa_interface.h,\n> > and {pipeline_name}_serializer.h to {pipeline_name}_ipa_serializer.h\n> > - same for their corresponding template files\n> > For the template files:\n> > - fix spacing (now it's all {{var}} instead of some {{ var }})\n> > - except if it's code, so code is still {{ code }}\n> > - move inclusion of corresponding header to first in the inclusion list\n> > - fix copy&paste errors\n> > - change snake_case to camelCase in the generated code\n> > - template code still uses snake_case\n> > - change the generated command enums to an enum class, and make it\n> > capitalized (instead of allcaps)\n> > - add length checks to recvIPC (in proxy)\n> > - fix some template spacing\n> > - don't use const for PODs in function/signal parameters\n> > - add the proper length checks to readPOD/appendPOD\n> > - the helper functions for reading and writing PODs to and from\n> > serialized data\n> > - rename readUInt/appendUInt to readPOD/appendPOD\n> > - add support for strings in custom structs\n> > \n> > Changes in v3:\n> > - add support for namespaces\n> > - fix enum assignment (used to have +1 for CMD applied to all enums)\n> > - use readHeader, writeHeader, and eraseHeader as static class functions\n> > of IPAIPCUnixSocket (in the proxy worker)\n> > - add requirement that base controls *must* be defined in\n> > libcamera::{pipeline_name}::Controls\n> > \n> > Changes in v2:\n> > - mandate the main and callback interfaces, and init(), start(), stop()\n> > and their parameters\n> > - fix returning single pod value from IPC-called function\n> > - add licenses\n> > - improve auto-generated message\n> > - other fixes related to serdes\n> > ---\n> > .../core_ipa_interface.h.tmpl | 40 ++\n> > .../core_ipa_serializer.h.tmpl | 47 ++\n> > .../definition_functions.tmpl | 53 ++\n> > .../libcamera_templates/meson.build | 14 +\n> > .../module_ipa_interface.h.tmpl | 87 +++\n> > .../module_ipa_proxy.cpp.tmpl | 236 ++++++++\n> > .../module_ipa_proxy.h.tmpl | 128 +++++\n> > .../module_ipa_proxy_worker.cpp.tmpl | 226 ++++++++\n> > .../module_ipa_serializer.h.tmpl | 48 ++\n> > .../libcamera_templates/proxy_functions.tmpl | 194 +++++++\n> > .../libcamera_templates/serializer.tmpl | 313 +++++++++++\n> > .../generators/mojom_libcamera_generator.py | 508 ++++++++++++++++++\n> > 12 files changed, 1894 insertions(+)\n> > create mode 100644 utils/ipc/generators/libcamera_templates/core_ipa_interface.h.tmpl\n> > create mode 100644 utils/ipc/generators/libcamera_templates/core_ipa_serializer.h.tmpl\n> > create mode 100644 utils/ipc/generators/libcamera_templates/definition_functions.tmpl\n> > create mode 100644 utils/ipc/generators/libcamera_templates/meson.build\n> > create mode 100644 utils/ipc/generators/libcamera_templates/module_ipa_interface.h.tmpl\n> > create mode 100644 utils/ipc/generators/libcamera_templates/module_ipa_proxy.cpp.tmpl\n> > create mode 100644 utils/ipc/generators/libcamera_templates/module_ipa_proxy.h.tmpl\n> > create mode 100644 utils/ipc/generators/libcamera_templates/module_ipa_proxy_worker.cpp.tmpl\n> > create mode 100644 utils/ipc/generators/libcamera_templates/module_ipa_serializer.h.tmpl\n> > create mode 100644 utils/ipc/generators/libcamera_templates/proxy_functions.tmpl\n> > create mode 100644 utils/ipc/generators/libcamera_templates/serializer.tmpl\n> > create mode 100644 utils/ipc/generators/mojom_libcamera_generator.py\n\n[snip]\n\n> > diff --git a/utils/ipc/generators/libcamera_templates/module_ipa_proxy.h.tmpl b/utils/ipc/generators/libcamera_templates/module_ipa_proxy.h.tmpl\n> > new file mode 100644\n> > index 00000000..2bc187f2\n> > --- /dev/null\n> > +++ b/utils/ipc/generators/libcamera_templates/module_ipa_proxy.h.tmpl\n> > @@ -0,0 +1,128 @@\n> > +{#-\n> > + # SPDX-License-Identifier: LGPL-2.1-or-later\n> > + # Copyright (C) 2020, Google Inc.\n> > +-#}\n> > +{%- import \"proxy_functions.tmpl\" as proxy_funcs -%}\n> > +\n> > +/* SPDX-License-Identifier: LGPL-2.1-or-later */\n> > +/*\n> > + * Copyright (C) 2020, Google Inc.\n> > + *\n> > + * {{module_name}}_ipa_proxy.h - Image Processing Algorithm proxy for {{module_name}}\n> > + *\n> > + * This file is auto-generated. Do not edit.\n> > + */\n> > +\n> > +#ifndef __LIBCAMERA_INTERNAL_IPA_PROXY_{{module_name|upper}}_H__\n> > +#define __LIBCAMERA_INTERNAL_IPA_PROXY_{{module_name|upper}}_H__\n> > +\n> > +#include \n> > +#include \n> > +\n> > +#include \"libcamera/internal/control_serializer.h\"\n> > +#include \"libcamera/internal/ipa_proxy.h\"\n> > +#include \"libcamera/internal/ipc_pipe.h\"\n> > +#include \"libcamera/internal/ipc_pipe_unixsocket.h\"\n> > +#include \"libcamera/internal/ipc_unixsocket.h\"\n> > +#include \"libcamera/internal/thread.h\"\n> > +\n> > +namespace libcamera {\n> > +{%- if has_namespace %}\n> > +{% for ns in namespace %}\n> > +namespace {{ns}} {\n> > +{% endfor %}\n> > +{%- endif %}\n> > +\n> > +class {{proxy_name}} : public IPAProxy, public {{interface_name}}, public Object\n> > +{\n> > +public:\n> > +\t{{proxy_name}}(IPAModule *ipam, bool isolate);\n> > +\t~{{proxy_name}}();\n> > +\n> > +{% for method in interface_main.methods %}\n> > +{{proxy_funcs.func_sig(proxy_name, method, \"\", false, true)|indent(8, true)}};\n> > +{% endfor %}\n> > +\n> > +{%- for method in interface_event.methods %}\n> > +\tSignal<\n> > +{%- for param in method.parameters -%}\n> > +\t\t{{\"const \" if not param|is_pod}}{{param|name}}{{\" &\" if not param|is_pod}}\n> > +\t\t{{- \", \" if not loop.last}}\n> > +{%- endfor -%}\n> > +> {{method.mojom_name}};\n> > +{% endfor %}\n> > +\n> > +private:\n> > +\tvoid recvMessage(const IPCMessage &data);\n> > +\n> > +{% for method in interface_main.methods %}\n> > +{{proxy_funcs.func_sig(proxy_name, method, \"Thread\", false)|indent(8, true)}};\n> > +{{proxy_funcs.func_sig(proxy_name, method, \"IPC\", false)|indent(8, true)}};\n> > +{% endfor %}\n> > +{% for method in interface_event.methods %}\n> > +{{proxy_funcs.func_sig(proxy_name, method, \"Thread\", false)|indent(8, true)}};\n> > +\tvoid {{method.mojom_name}}IPC(\n> > +\t\tstd::vector::const_iterator data,\n> > +\t\tsize_t dataSize,\n> > +\t\tconst std::vector &fds);\n> > +{% endfor %}\n> > +\n> > +\t/* Helper class to invoke async functions in another thread. */\n> > +\tclass ThreadProxy : public Object\n> > +\t{\n> > +\tpublic:\n> > +\t\tvoid setIPA({{interface_name}} *ipa)\n> > +\t\t{\n> > +\t\t\tipa_ = ipa;\n> > +\t\t}\n> > +\n> > +\t\tvoid stop()\n> > +\t\t{\n> > +\t\t\tipa_->stop();\n> > +\t\t}\n> > +{% for method in interface_main.methods %}\n> > +{%- if method|is_async %}\n> > +\t\t{{proxy_funcs.func_sig(proxy_name, method, \"\", false)|indent(16)}}\n> > +\t\t{\n> > +\t\t\tipa_->{{method.mojom_name}}({{method.parameters|params_comma_sep}});\n> > +\t\t}\n> > +{%- elif method.mojom_name == \"start\" %}\n> > +\t\t{{proxy_funcs.func_sig(proxy_name, method, \"\", false)|indent(16)}}\n> > +\t\t{\n> > +{%- if method|method_return_value != \"void\" %}\n> > +\t\t\treturn ipa_->{{method.mojom_name}}({{method.parameters|params_comma_sep}});\n> > +{%- else %}\n> > +\t\t\tipa_->{{method.mojom_name}}({{method.parameters|params_comma_sep}}\n> > +\t{{- \", \" if method|method_param_outputs|params_comma_sep -}}\n> > +\t{{- method|method_param_outputs|params_comma_sep}});\n> > +{%- endif %}\n> > +\t\t}\n> > +{%- endif %}\n> > +{%- endfor %}\n> > +\n> > +\tprivate:\n> > +\t\t{{interface_name}} *ipa_;\n> > +\t};\n> > +\n> > +\tbool running_;\n> > +\tThread thread_;\n> > +\tThreadProxy proxy_;\n> > +\tstd::unique_ptr<{{interface_name}}> ipa_;\n> > +\n> > +\tconst bool isolate_;\n> > +\n> > +\tstd::unique_ptr ipc_;\n> > +\n> > +\tControlSerializer controlSerializer_;\n> > +\n> > +\tuint32_t seq_;\n\nI don't think this belongs here, the sequence number should be an\ninternal property of the IPCPipe implementation. As it won't be\nstraightforward to solve, you can just add a todo comment here.\n\n> > +};\n> > +\n> > +{%- if has_namespace %}\n> > +{% for ns in namespace|reverse %}\n> > +} /* namespace {{ns}} */\n> > +{% endfor %}\n> > +{%- endif %}\n> > +} /* namespace libcamera */\n> > +\n\n[snip]","headers":{"Return-Path":"","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id DE563BD160\n\tfor ;\n\tFri, 12 Feb 2021 02:01:54 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id 5E13663758;\n\tFri, 12 Feb 2021 03:01:54 +0100 (CET)","from perceval.ideasonboard.com (perceval.ideasonboard.com\n\t[IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id 564F7602FC\n\tfor ;\n\tFri, 12 Feb 2021 03:01:53 +0100 (CET)","from pendragon.ideasonboard.com (62-78-145-57.bb.dnainternet.fi\n\t[62.78.145.57])\n\tby perceval.ideasonboard.com (Postfix) with ESMTPSA id B3E4D8B5;\n\tFri, 12 Feb 2021 03:01:52 +0100 (CET)"],"Authentication-Results":"lancelot.ideasonboard.com;\n\tdkim=fail reason=\"signature verification failed\" (1024-bit key;\n\tunprotected) header.d=ideasonboard.com header.i=@ideasonboard.com\n\theader.b=\"BIfiirFo\"; dkim-atps=neutral","DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com;\n\ts=mail; t=1613095312;\n\tbh=n0P7E7mlIY1IjNXU4kr69DDLB8QTSGBGgytfM6yVMq4=;\n\th=Date:From:To:Cc:Subject:References:In-Reply-To:From;\n\tb=BIfiirFoa80VJdbk6+Hs9EMvvBI6tQ85MAu2mSbyU43MOifagQTA1fXAvj8mrPi5s\n\tKvEQbMHadrZ2+2mGfMxa/WgbyJ95qZQsxBwE8OBV7Yuy93rAeiVxxPl+W1cdJmODfC\n\tYPVr9hGxuhP7Glb38fd47+7qkAQ2vckRPQKVBWkM=","Date":"Fri, 12 Feb 2021 04:01:27 +0200","From":"Laurent Pinchart ","To":"Paul Elder ","Message-ID":"","References":"<20210211071805.34963-1-paul.elder@ideasonboard.com>\n\t<20210211071805.34963-3-paul.elder@ideasonboard.com>\n\t","MIME-Version":"1.0","Content-Disposition":"inline","In-Reply-To":"","Subject":"Re: [libcamera-devel] [PATCH v7 02/10] utils: ipc: add templates\n\tfor code generation for IPC mechanism","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Cc":"libcamera-devel@lists.libcamera.org","Content-Type":"text/plain; charset=\"us-ascii\"","Content-Transfer-Encoding":"7bit","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" "}}]