[{"id":14520,"web_url":"https://patchwork.libcamera.org/comment/14520/","msgid":"<3ce53459-3c34-710a-b4e3-01cc4ff6e8f3@ideasonboard.com>","date":"2021-01-11T06:24:48","subject":"Re: [libcamera-devel] [PATCH v6 2/9] utils: ipc: add templates for\n\tcode generation for IPC mechanism","submitter":{"id":4,"url":"https://patchwork.libcamera.org/api/people/4/","name":"Kieran Bingham","email":"kieran.bingham@ideasonboard.com"},"content":"Hi Paul,\n\nOn 24/12/2020 08:15, Paul Elder wrote:\n> Add templates to mojo to generate code for the IPC mechanism. These\n> templates generate:\n> - module header\n> - module serializer\n> - IPA proxy cpp, header, and worker\n> \n> Given an input data definition mojom file for a pipeline.\n> \n> Signed-off-by: Paul Elder \n> Acked-by: Jacopo Mondi \n> \n> ---\n> Changes in v6:\n> - add templates for core_ipa_interface.h and core_ipa_serializer.h\n> - for libcamera types defined in mojom\n> - rename everything to {{module_name}}_ipa_{interface,proxy,proxy_worker}.{c,h}\n> - remove #include - support customizable start()\n> - remove the need for per-pipeline ControlInfoMap\n> - add todo for avoiding intermediate vectors\n> - remove postfix underscore for generated struct fields\n> - support structs that are members of vectors/maps that aren't defined\n> in mojom (in mojom)\n> - fix has_fd detection\n> - namespacing is now required in mojom, in the form of ^ipa\\.[0-9A-Za-z_]+\n> - support consts in mojom\n> - make the pseudo-switch-case in the python generator nicer\n> \n> Changes in v5:\n> - add a usage output to the proxy worker, to document the interface for\n> executing the proxy worker\n> - in the mojom generator python script:\n> - removed unused things (imports, functions, jinja exports)\n> - document GetNameForElement\n> - rename everything cb -> event\n> - refactor Method{Input,Output}HasFd with a helper MethodParamsHaveFd\n> - add Get{Main,Event}Interface to fix the interface_{main,event} jinja\n> exports\n> - add copyright\n> - require that event interfaces have at least one event\n> - expand copyright for templates\n> - use new sendSync/sendAsync API (with IPCMessage)\n> - rename a bunch of things\n> \n> Changes in v4:\n> For the non-template files:\n> - rename IPA{pipeline_name}CallbackInterface to\n> IPA{pipeline_name}EventInterface\n> - to avoid the notion of \"callback\" and emphasize that it's an event\n> - add support for strings in custom structs\n> - add validation, that async methods must not have return values\n> - it throws exception and isn't very clear though...?\n> - rename controls to libcamera::{pipeline_name}::controls (controls is\n> now lowercase)\n> - rename {pipeline_name}_generated.h to {pipeline_name}_ipa_interface.h,\n> and {pipeline_name}_serializer.h to {pipeline_name}_ipa_serializer.h\n> - same for their corresponding template files\n> For the template files:\n> - fix spacing (now it's all {{var}} instead of some {{ var }})\n> - except if it's code, so code is still {{ code }}\n> - move inclusion of corresponding header to first in the inclusion list\n> - fix copy&paste errors\n> - change snake_case to camelCase in the generated code\n> - template code still uses snake_case\n> - change the generated command enums to an enum class, and make it\n> capitalized (instead of allcaps)\n> - add length checks to recvIPC (in proxy)\n> - fix some template spacing\n> - don't use const for PODs in function/signal parameters\n> - add the proper length checks to readPOD/appendPOD\n> - the helper functions for reading and writing PODs to and from\n> serialized data\n> - rename readUInt/appendUInt to readPOD/appendPOD\n> - add support for strings in custom structs\n> \n> Changes in v3:\n> - add support for namespaces\n> - fix enum assignment (used to have +1 for CMD applied to all enums)\n> - use readHeader, writeHeader, and eraseHeader as static class functions\n> of IPAIPCUnixSocket (in the proxy worker)\n> - add requirement that base controls *must* be defined in\n> libcamera::{pipeline_name}::Controls\n> \n> Changes in v2:\n> - mandate the main and callback interfaces, and init(), start(), stop()\n> and their parameters\n> - fix returning single pod value from IPC-called function\n> - add licenses\n> - improve auto-generated message\n> - other fixes related to serdes\n> ---\n> .../core_ipa_interface.h.tmpl | 37 ++\n> .../core_ipa_serializer.h.tmpl | 47 ++\n> .../definition_functions.tmpl | 53 ++\n> .../libcamera_templates/meson.build | 14 +\n> .../module_ipa_interface.h.tmpl | 87 +++\n> .../module_ipa_proxy.cpp.tmpl | 232 ++++++++\n> .../module_ipa_proxy.h.tmpl | 126 +++++\n> .../module_ipa_proxy_worker.cpp.tmpl | 224 ++++++++\n> .../module_ipa_serializer.h.tmpl | 47 ++\n> .../libcamera_templates/proxy_functions.tmpl | 192 +++++++\n> .../libcamera_templates/serializer.tmpl | 313 +++++++++++\n> .../generators/mojom_libcamera_generator.py | 511 ++++++++++++++++++\n> 12 files changed, 1883 insertions(+)\n> create mode 100644 utils/ipc/generators/libcamera_templates/core_ipa_interface.h.tmpl\n> create mode 100644 utils/ipc/generators/libcamera_templates/core_ipa_serializer.h.tmpl\n> create mode 100644 utils/ipc/generators/libcamera_templates/definition_functions.tmpl\n> create mode 100644 utils/ipc/generators/libcamera_templates/meson.build\n> create mode 100644 utils/ipc/generators/libcamera_templates/module_ipa_interface.h.tmpl\n> create mode 100644 utils/ipc/generators/libcamera_templates/module_ipa_proxy.cpp.tmpl\n> create mode 100644 utils/ipc/generators/libcamera_templates/module_ipa_proxy.h.tmpl\n> create mode 100644 utils/ipc/generators/libcamera_templates/module_ipa_proxy_worker.cpp.tmpl\n> create mode 100644 utils/ipc/generators/libcamera_templates/module_ipa_serializer.h.tmpl\n> create mode 100644 utils/ipc/generators/libcamera_templates/proxy_functions.tmpl\n> create mode 100644 utils/ipc/generators/libcamera_templates/serializer.tmpl\n> create mode 100644 utils/ipc/generators/mojom_libcamera_generator.py\n> \n> diff --git a/utils/ipc/generators/libcamera_templates/core_ipa_interface.h.tmpl b/utils/ipc/generators/libcamera_templates/core_ipa_interface.h.tmpl\n> new file mode 100644\n> index 00000000..f11d56fb\n> --- /dev/null\n> +++ b/utils/ipc/generators/libcamera_templates/core_ipa_interface.h.tmpl\n> @@ -0,0 +1,37 @@\n> +{#-\n> + # SPDX-License-Identifier: LGPL-2.1-or-later\n> + # Copyright (C) 2020, Google Inc.\n> +-#}\n> +{%- import \"definition_functions.tmpl\" as funcs -%}\n> +/* SPDX-License-Identifier: LGPL-2.1-or-later */\n> +/*\n> + * Copyright (C) 2020, Google Inc.\n> + *\n> + * core_ipa_interface.h - libcamera core definitions for Image Processing Algorithms\n> + *\n> + * This file is auto-generated. Do not edit.\n> + */\n> +\n> +#ifndef __LIBCAMERA_IPA_INTERFACE_CORE_GENERATED_H__\n> +#define __LIBCAMERA_IPA_INTERFACE_CORE_GENERATED_H__\n> +\n> +{% if has_map %}#include {% endif %}\n> +{% if has_array %}#include {% endif %}\n> +\n> +namespace libcamera {\n> +\n> +{% for const in consts %}\n> +const {{const.kind|name}} {{const.mojom_name}} = {{const.value}};\n> +{% endfor %}\n> +\n> +{% for enum in enums %}\n> +{{funcs.define_enum(enum)}}\n> +{% endfor %}\n> +\n> +{%- for struct in structs_gen_header %}\n> +{{funcs.define_struct(struct)}}\n> +{% endfor %}\n> +\n> +} /* namespace libcamera */\n> +\n> +#endif /* __LIBCAMERA_IPA_INTERFACE_CORE_GENERATED_H__ */\n> diff --git a/utils/ipc/generators/libcamera_templates/core_ipa_serializer.h.tmpl b/utils/ipc/generators/libcamera_templates/core_ipa_serializer.h.tmpl\n> new file mode 100644\n> index 00000000..37a784f1\n> --- /dev/null\n> +++ b/utils/ipc/generators/libcamera_templates/core_ipa_serializer.h.tmpl\n> @@ -0,0 +1,47 @@\n> +{#-\n> + # SPDX-License-Identifier: LGPL-2.1-or-later\n> + # Copyright (C) 2020, Google Inc.\n> +-#}\n> +{%- import \"serializer.tmpl\" as serializer -%}\n> +\n> +/* SPDX-License-Identifier: LGPL-2.1-or-later */\n> +/*\n> + * Copyright (C) 2020, Google Inc.\n> + *\n> + * core_ipa_serializer.h - Data serializer for core libcamera definitions for IPA\n> + *\n> + * This file is auto-generated. Do not edit.\n> + */\n> +\n> +#ifndef __LIBCAMERA_INTERNAL_IPA_DATA_SERIALIZER_CORE_H__\n> +#define __LIBCAMERA_INTERNAL_IPA_DATA_SERIALIZER_CORE_H__\n> +\n> +#include \n> +#include \n> +\n> +#include \n> +\n> +#include \"libcamera/internal/control_serializer.h\"\n> +#include \"libcamera/internal/ipa_data_serializer.h\"\n> +\n> +namespace libcamera {\n> +\n> +LOG_DECLARE_CATEGORY(IPADataSerializer)\n> +{% for struct in structs_gen_serializer %}\n> +template<>\n> +class IPADataSerializer<{{struct|name}}>\n> +{\n> +public:\n> +{{- serializer.serializer(struct, \"\")}}\n> +{%- if struct|has_fd %}\n> +{{serializer.deserializer_fd(struct, \"\")}}\n> +{%- else %}\n> +{{serializer.deserializer_no_fd(struct, \"\")}}\n> +{{serializer.deserializer_fd_simple(struct, \"\")}}\n> +{%- endif %}\n> +};\n> +{% endfor %}\n> +\n> +} /* namespace libcamera */\n> +\n> +#endif /* __LIBCAMERA_INTERNAL_IPA_DATA_SERIALIZER_CORE_H__ */\n> diff --git a/utils/ipc/generators/libcamera_templates/definition_functions.tmpl b/utils/ipc/generators/libcamera_templates/definition_functions.tmpl\n> new file mode 100644\n> index 00000000..cdd75f89\n> --- /dev/null\n> +++ b/utils/ipc/generators/libcamera_templates/definition_functions.tmpl\n> @@ -0,0 +1,53 @@\n> +{#-\n> + # SPDX-License-Identifier: LGPL-2.1-or-later\n> + # Copyright (C) 2020, Google Inc.\n> +-#}\n> +\n> +{#\n> + # \\brief Generate enum definition\n> + #\n> + # \\param enum Enum object whose definition is to be generated\n> + #}\n> +{%- macro define_enum(enum) -%}\n> +enum {{enum.mojom_name}} {\n> +{%- for field in enum.fields %}\n> +\t{{field.mojom_name}} = {{field.numeric_value}},\n> +{%- endfor %}\n> +};\n> +{%- endmacro -%}\n> +\n> +{#\n> + # \\brief Generate struct definition\n> + #\n> + # \\param struct Struct object whose definition is to be generated\n> + #}\n> +{%- macro define_struct(struct) -%}\n> +struct {{struct.mojom_name}}\n> +{\n> +public:\n> +\t{{struct.mojom_name}}() {%- if struct|has_default_fields %}\n> +\t\t:{% endif %}\n> +{%- for field in struct.fields|with_default_values -%}\n> +{{\" \" if loop.first}}{{field.mojom_name}}({{field|default_value}}){{\", \" if not loop.last}}\n> +{%- endfor %}\n> +\t{\n> +\t}\n> +\n> +\t{{struct.mojom_name}}(\n> +{%- for field in struct.fields -%}\n> +{{\"const \" if not field|is_pod}}{{field|name}} {{\"&\" if not field|is_pod}}_{{field.mojom_name}}{{\", \" if not loop.last}}\n> +{%- endfor -%}\n> +)\n> +\t\t:\n> +{%- for field in struct.fields -%}\n> +{{\" \" if loop.first}}{{field.mojom_name}}(_{{field.mojom_name}}){{\", \" if not loop.last}}\n> +{%- endfor %}\n> +\t{\n> +\t}\n> +{% for field in struct.fields %}\n> +\t{{field|name}} {{field.mojom_name}};\n> +{%- endfor %}\n> +};\n> +{%- endmacro -%}\n> +\n> +\n> diff --git a/utils/ipc/generators/libcamera_templates/meson.build b/utils/ipc/generators/libcamera_templates/meson.build\n> new file mode 100644\n> index 00000000..70664eab\n> --- /dev/null\n> +++ b/utils/ipc/generators/libcamera_templates/meson.build\n> @@ -0,0 +1,14 @@\n> +# SPDX-License-Identifier: CC0-1.0\n> +\n> +mojom_template_files = files([\n> + 'core_ipa_interface.h.tmpl',\n> + 'core_ipa_serializer.h.tmpl',\n> + 'definition_functions.tmpl',\n> + 'module_ipa_interface.h.tmpl',\n> + 'module_ipa_proxy.cpp.tmpl',\n> + 'module_ipa_proxy.h.tmpl',\n> + 'module_ipa_proxy_worker.cpp.tmpl',\n> + 'module_ipa_serializer.h.tmpl',\n> + 'proxy_functions.tmpl',\n> + 'serializer.tmpl',\n> +])\n> diff --git a/utils/ipc/generators/libcamera_templates/module_ipa_interface.h.tmpl b/utils/ipc/generators/libcamera_templates/module_ipa_interface.h.tmpl\n> new file mode 100644\n> index 00000000..afbcb1b1\n> --- /dev/null\n> +++ b/utils/ipc/generators/libcamera_templates/module_ipa_interface.h.tmpl\n> @@ -0,0 +1,87 @@\n> +{#-\n> + # SPDX-License-Identifier: LGPL-2.1-or-later\n> + # Copyright (C) 2020, Google Inc.\n> +-#}\n> +{%- import \"definition_functions.tmpl\" as funcs -%}\n> +/* SPDX-License-Identifier: LGPL-2.1-or-later */\n> +/*\n> + * Copyright (C) 2020, Google Inc.\n> + *\n> + * {{module_name}}_ipa_interface.h - Image Processing Algorithm interface for {{module_name}}\n> + *\n> + * This file is auto-generated. Do not edit.\n> + */\n> +\n> +#ifndef __LIBCAMERA_IPA_INTERFACE_{{module_name|upper}}_GENERATED_H__\n> +#define __LIBCAMERA_IPA_INTERFACE_{{module_name|upper}}_GENERATED_H__\n> +\n> +#include \n> +#include \n> +\n> +{% if has_map %}#include {% endif %}\n> +{% if has_array %}#include {% endif %}\n> +\n> +namespace libcamera {\n> +{%- if has_namespace %}\n> +{% for ns in namespace %}\n> +namespace {{ns}} {\n> +{% endfor %}\n> +{%- endif %}\n> +\n> +{% for const in consts %}\n> +const {{const.kind|name}} {{const.mojom_name}} = {{const.value}};\n> +{% endfor %}\n> +\n> +enum class {{cmd_enum_name}} {\n> +\tExit = 0,\n> +{%- for method in interface_main.methods %}\n> +\t{{method.mojom_name|cap}} = {{loop.index}},\n> +{%- endfor %}\n> +};\n> +\n> +enum class {{cmd_event_enum_name}} {\n> +{%- for method in interface_event.methods %}\n> +\t{{method.mojom_name|cap}} = {{loop.index}},\n> +{%- endfor %}\n> +};\n> +\n> +{% for enum in enums %}\n> +{{funcs.define_enum(enum)}}\n> +{% endfor %}\n> +\n> +{%- for struct in structs_nonempty %}\n> +{{funcs.define_struct(struct)}}\n> +{% endfor %}\n> +\n> +{#-\n> +Any consts or #defines should be moved to the mojom file.\n> +#}\n> +class {{interface_name}} : public IPAInterface\n> +{\n> +public:\n> +{% for method in interface_main.methods %}\n> +\tvirtual {{method|method_return_value}} {{method.mojom_name}}(\n> +{%- for param in method|method_parameters %}\n> +\t\t{{param}}{{- \",\" if not loop.last}}\n> +{%- endfor -%}\n> +) = 0;\n> +{% endfor %}\n> +\n> +{%- for method in interface_event.methods %}\n> +\tSignal<\n> +{%- for param in method.parameters -%}\n> +\t\t{{\"const \" if not param|is_pod}}{{param|name}}{{\" &\" if not param|is_pod}}\n> +\t\t{{- \", \" if not loop.last}}\n> +{%- endfor -%}\n> +> {{method.mojom_name}};\n> +{% endfor -%}\n> +};\n> +\n> +{%- if has_namespace %}\n> +{% for ns in namespace|reverse %}\n> +} /* namespace {{ns}} */\n> +{% endfor %}\n> +{%- endif %}\n> +} /* namespace libcamera */\n> +\n> +#endif /* __LIBCAMERA_IPA_INTERFACE_{{module_name|upper}}_GENERATED_H__ */\n> diff --git a/utils/ipc/generators/libcamera_templates/module_ipa_proxy.cpp.tmpl b/utils/ipc/generators/libcamera_templates/module_ipa_proxy.cpp.tmpl\n> new file mode 100644\n> index 00000000..a181cc84\n> --- /dev/null\n> +++ b/utils/ipc/generators/libcamera_templates/module_ipa_proxy.cpp.tmpl\n> @@ -0,0 +1,232 @@\n> +{#-\n> + # SPDX-License-Identifier: LGPL-2.1-or-later\n> + # Copyright (C) 2020, Google Inc.\n> +-#}\n> +{%- import \"proxy_functions.tmpl\" as proxy_funcs -%}\n> +\n> +/* SPDX-License-Identifier: LGPL-2.1-or-later */\n> +/*\n> + * Copyright (C) 2020, Google Inc.\n> + *\n> + * {{module_name}}_ipa_proxy.cpp - Image Processing Algorithm proxy for {{module_name}}\n> + *\n> + * This file is auto-generated. Do not edit.\n> + */\n> +\n> +#include \n> +\n> +#include \n> +\n> +#include \n> +#include \n> +#include \n> +\n> +#include \"libcamera/internal/control_serializer.h\"\n> +#include \"libcamera/internal/ipa_data_serializer.h\"\n> +#include \"libcamera/internal/ipa_module.h\"\n> +#include \"libcamera/internal/ipa_proxy.h\"\n> +#include \"libcamera/internal/ipc_pipe.h\"\n> +#include \"libcamera/internal/ipc_pipe_unixsocket.h\"\n> +#include \"libcamera/internal/ipc_unixsocket.h\"\n> +#include \"libcamera/internal/log.h\"\n> +#include \"libcamera/internal/process.h\"\n> +#include \"libcamera/internal/thread.h\"\n> +\n> +namespace libcamera {\n> +\n> +LOG_DECLARE_CATEGORY(IPAProxy)\n> +\n> +{%- if has_namespace %}\n> +{% for ns in namespace %}\n> +namespace {{ns}} {\n> +{% endfor %}\n> +{%- endif %}\n> +\n> +{{proxy_name}}::{{proxy_name}}(IPAModule *ipam, bool isolate)\n> +\t: IPAProxy(ipam), running_(false),\n> +\t isolate_(isolate)\n> +{\n> +\tLOG(IPAProxy, Debug)\n> +\t\t<< \"initializing {{module_name}} proxy: loading IPA from \"\n> +\t\t<< ipam->path();\n> +\n> +\tif (isolate_) {\n> +\t\tconst std::string proxyWorkerPath = resolvePath(\"{{module_name}}_ipa_proxy\");\n> +\t\tif (proxyWorkerPath.empty()) {\n> +\t\t\tLOG(IPAProxy, Error)\n> +\t\t\t\t<< \"Failed to get proxy worker path\";\n> +\t\t\treturn;\n> +\t\t}\n> +\n> +\t\tipc_ = std::make_unique(ipam->path().c_str(),\n> +\t\t\t\t\t\t\t proxyWorkerPath.c_str());\n> +\t\tif (!ipc_->isConnected()) {\n> +\t\t\tLOG(IPAProxy, Error) << \"Failed to create IPCPipe\";\n> +\t\t\treturn;\n> +\t\t}\n> +\n> +\t\tipc_->recv.connect(this, &{{proxy_name}}::recvMessage);\n> +\n> +\t\tvalid_ = true;\n> +\t\treturn;\n> +\t}\n> +\n> +\tif (!ipam->load())\n> +\t\treturn;\n> +\n> +\tIPAInterface *ipai = ipam->createInterface();\n> +\tif (!ipai) {\n> +\t\tLOG(IPAProxy, Error)\n> +\t\t\t<< \"Failed to create IPA context for \" << ipam->path();\n> +\t\treturn;\n> +\t}\n> +\n> +\tipa_ = std::unique_ptr<{{interface_name}}>(static_cast<{{interface_name}} *>(ipai));\n> +\tproxy_.setIPA(ipa_.get());\n> +\n> +{% for method in interface_event.methods %}\n> +\tipa_->{{method.mojom_name}}.connect(this, &{{proxy_name}}::{{method.mojom_name}}Thread);\n> +{%- endfor %}\n> +\n> +\tvalid_ = true;\n> +}\n> +\n> +{{proxy_name}}::~{{proxy_name}}()\n> +{\n> +\tif (isolate_)\n> +\t\tipc_->sendAsync(static_cast({{cmd_enum_name}}::Exit), {});\n> +}\n> +\n> +{% if interface_event.methods|length > 0 %}\n> +void {{proxy_name}}::recvMessage(uint32_t cmd, const IPCMessage &data)\n> +{\n> +\tsize_t dataSize = data.cdata().size();\n> +\t{{cmd_event_enum_name}} _cmd = static_cast<{{cmd_event_enum_name}}>(cmd);\n> +\n> +\tswitch (_cmd) {\n> +{%- for method in interface_event.methods %}\n> +\tcase {{cmd_event_enum_name}}::{{method.mojom_name|cap}}: {\n> +\t\t{{method.mojom_name}}IPC(data.cdata().cbegin(), dataSize, data.cfds());\n> +\t\tbreak;\n> +\t}\n> +{%- endfor %}\n> +\tdefault:\n> +\t\tLOG(IPAProxy, Error) << \"Unknown command \" << static_cast(_cmd);\n> +\t}\n> +}\n> +{%- endif %}\n> +\n> +{% for method in interface_main.methods %}\n> +{{proxy_funcs.func_sig(proxy_name, method)}}\n> +{\n> +\tif (isolate_)\n> +\t\t{{\"return \" if method|method_return_value != \"void\"}}{{method.mojom_name}}IPC(\n> +{%- for param in method|method_param_names -%}\n> +\t\t{{param}}{{- \", \" if not loop.last}}\n> +{%- endfor -%}\n> +);\n> +\telse\n> +\t\t{{\"return \" if method|method_return_value != \"void\"}}{{method.mojom_name}}Thread(\n> +{%- for param in method|method_param_names -%}\n> +\t\t{{param}}{{- \", \" if not loop.last}}\n> +{%- endfor -%}\n> +);\n> +}\n> +\n> +{{proxy_funcs.func_sig(proxy_name, method, \"Thread\")}}\n> +{\n> +{%- if method.mojom_name == \"init\" %}\n> +\t{{proxy_funcs.init_thread_body()}}\n> +{%- elif method.mojom_name == \"stop\" %}\n> +\t{{proxy_funcs.stop_thread_body()}}\n> +{%- elif method.mojom_name == \"start\" %}\n> +\trunning_ = true;\n> +\tthread_.start();\n> +\n> +\t{{ \"return \" if method|method_return_value != \"void\" -}}\n> +\tproxy_.invokeMethod(&ThreadProxy::start, ConnectionTypeBlocking\n> +\t{{- \", \" if method|method_param_names}}\n> +\t{%- for param in method|method_param_names -%}\n> +\t\t{{param}}{{- \", \" if not loop.last}}\n> +\t{%- endfor -%}\n> +);\n> +{%- elif not method|is_async %}\n> +\t{{ \"return \" if method|method_return_value != \"void\" -}}\n> +\tipa_->{{method.mojom_name}}(\n> +\t{%- for param in method|method_param_names -%}\n> +\t\t{{param}}{{- \", \" if not loop.last}}\n> +\t{%- endfor -%}\n> +);\n> +{% elif method|is_async %}\n> +\tproxy_.invokeMethod(&ThreadProxy::{{method.mojom_name}}, ConnectionTypeQueued,\n> +\t{%- for param in method|method_param_names -%}\n> +\t\t{{param}}{{- \", \" if not loop.last}}\n> +\t{%- endfor -%}\n> +);\n> +{%- endif %}\n> +}\n> +\n> +{{proxy_funcs.func_sig(proxy_name, method, \"IPC\")}}\n> +{\n> +{%- set has_input = true if method|method_param_inputs|length > 0 %}\n> +{%- set has_output = true if method|method_param_outputs|length > 0 or method|method_return_value != \"void\" %}\n> +{%- if has_input %}\n> +\tIPCMessage _ipcInputBuf;\n> +{%- endif %}\n> +{%- if has_output %}\n> +\tIPCMessage _ipcOutputBuf;\n> +{%- endif %}\n> +\n> +{{proxy_funcs.serialize_call(method|method_param_inputs, '_ipcInputBuf.data()', '_ipcInputBuf.fds()')}}\n> +\n> +{%- set input_buf = \"_ipcInputBuf\" if has_input else \"{}\" %}\n> +{%- set cmd = cmd_enum_name + \"::\" + method.mojom_name|cap %}\n> +{% if method|is_async %}\n> +\tint _ret = ipc_->sendAsync(static_cast({{cmd}}), {{input_buf}});\n> +{%- else %}\n> +\tint _ret = ipc_->sendSync(static_cast({{cmd}}), {{input_buf}}\n> +{{- \", &_ipcOutputBuf\" if has_output -}}\n> +);\n> +{%- endif %}\n> +\tif (_ret < 0) {\n> +\t\tLOG(IPAProxy, Error) << \"Failed to call {{method.mojom_name}}\";\n> +{%- if method|method_return_value != \"void\" %}\n> +\t\treturn static_cast<{{method|method_return_value}}>(_ret);\n> +{%- else %}\n> +\t\treturn;\n> +{%- endif %}\n> +\t}\n> +{% if method|method_return_value != \"void\" %}\n> +\treturn IPADataSerializer<{{method.response_parameters|first|name}}>::deserialize(_ipcOutputBuf.data(), 0);\n> +{% elif method|method_param_outputs|length > 0 %}\n> +{{proxy_funcs.deserialize_call(method|method_param_outputs, '_ipcOutputBuf.data()', '_ipcOutputBuf.fds()')}}\n> +{% endif -%}\n> +}\n> +\n> +{% endfor %}\n> +\n> +{% for method in interface_event.methods %}\n> +{{proxy_funcs.func_sig(proxy_name, method, \"Thread\")}}\n> +{\n> +\t{{method.mojom_name}}.emit({{method.parameters|params_comma_sep}});\n> +}\n> +\n> +void {{proxy_name}}::{{method.mojom_name}}IPC(\n> +\tstd::vector::const_iterator data,\n> +\tsize_t dataSize,\n> +\t[[maybe_unused]] const std::vector &fds)\n> +{\n> +{%- for param in method.parameters %}\n> +\t{{param|name}} {{param.mojom_name}};\n> +{%- endfor %}\n> +{{proxy_funcs.deserialize_call(method.parameters, 'data', 'fds', false, false, true, 'dataSize')}}\n> +\t{{method.mojom_name}}.emit({{method.parameters|params_comma_sep}});\n> +}\n> +{% endfor %}\n> +\n> +{%- if has_namespace %}\n> +{% for ns in namespace|reverse %}\n> +} /* namespace {{ns}} */\n> +{% endfor %}\n> +{%- endif %}\n> +} /* namespace libcamera */\n> diff --git a/utils/ipc/generators/libcamera_templates/module_ipa_proxy.h.tmpl b/utils/ipc/generators/libcamera_templates/module_ipa_proxy.h.tmpl\n> new file mode 100644\n> index 00000000..b0f04bf6\n> --- /dev/null\n> +++ b/utils/ipc/generators/libcamera_templates/module_ipa_proxy.h.tmpl\n> @@ -0,0 +1,126 @@\n> +{#-\n> + # SPDX-License-Identifier: LGPL-2.1-or-later\n> + # Copyright (C) 2020, Google Inc.\n> +-#}\n> +{%- import \"proxy_functions.tmpl\" as proxy_funcs -%}\n> +\n> +/* SPDX-License-Identifier: LGPL-2.1-or-later */\n> +/*\n> + * Copyright (C) 2020, Google Inc.\n> + *\n> + * {{module_name}}_ipa_proxy.h - Image Processing Algorithm proxy for {{module_name}}\n> + *\n> + * This file is auto-generated. Do not edit.\n> + */\n> +\n> +#ifndef __LIBCAMERA_INTERNAL_IPA_PROXY_{{module_name|upper}}_H__\n> +#define __LIBCAMERA_INTERNAL_IPA_PROXY_{{module_name|upper}}_H__\n> +\n> +#include \n> +#include \n> +\n> +#include \"libcamera/internal/control_serializer.h\"\n> +#include \"libcamera/internal/ipc_pipe.h\"\n> +#include \"libcamera/internal/ipc_pipe_unixsocket.h\"\n> +#include \"libcamera/internal/ipa_proxy.h\"\n> +#include \"libcamera/internal/ipc_unixsocket.h\"\n> +#include \"libcamera/internal/thread.h\"\n> +\n> +namespace libcamera {\n> +{%- if has_namespace %}\n> +{% for ns in namespace %}\n> +namespace {{ns}} {\n> +{% endfor %}\n> +{%- endif %}\n> +\n> +class {{proxy_name}} : public IPAProxy, public {{interface_name}}, public Object\n> +{\n> +public:\n> +\t{{proxy_name}}(IPAModule *ipam, bool isolate);\n> +\t~{{proxy_name}}();\n> +\n> +{% for method in interface_main.methods %}\n> +{{proxy_funcs.func_sig(proxy_name, method, \"\", false, true)|indent(8, true)}};\n> +{% endfor %}\n> +\n> +{%- for method in interface_event.methods %}\n> +\tSignal<\n> +{%- for param in method.parameters -%}\n> +\t\t{{\"const \" if not param|is_pod}}{{param|name}}{{\" &\" if not param|is_pod}}\n> +\t\t{{- \", \" if not loop.last}}\n> +{%- endfor -%}\n> +> {{method.mojom_name}};\n> +{% endfor %}\n> +\n> +private:\n> +\tvoid recvMessage(uint32_t cmd, const IPCMessage &data);\n> +\n> +{% for method in interface_main.methods %}\n> +{{proxy_funcs.func_sig(proxy_name, method, \"Thread\", false)|indent(8, true)}};\n> +{{proxy_funcs.func_sig(proxy_name, method, \"IPC\", false)|indent(8, true)}};\n> +{% endfor %}\n> +{% for method in interface_event.methods %}\n> +{{proxy_funcs.func_sig(proxy_name, method, \"Thread\", false)|indent(8, true)}};\n> +\tvoid {{method.mojom_name}}IPC(\n> +\t\tstd::vector::const_iterator data,\n> +\t\tsize_t dataSize,\n> +\t\tconst std::vector &fds);\n> +{% endfor %}\n> +\n> +\t/* Helper class to invoke async functions in another thread. */\n> +\tclass ThreadProxy : public Object\n> +\t{\n> +\tpublic:\n> +\t\tvoid setIPA({{interface_name}} *ipa)\n> +\t\t{\n> +\t\t\tipa_ = ipa;\n> +\t\t}\n> +\n> +\t\tvoid stop()\n> +\t\t{\n> +\t\t\tipa_->stop();\n> +\t\t}\n> +{% for method in interface_main.methods %}\n> +{%- if method|is_async %}\n> +\t\t{{proxy_funcs.func_sig(proxy_name, method, \"\", false)|indent(16)}}\n> +\t\t{\n> +\t\t\tipa_->{{method.mojom_name}}({{method.parameters|params_comma_sep}});\n> +\t\t}\n> +{%- elif method.mojom_name == \"start\" %}\n> +\t\t{{proxy_funcs.func_sig(proxy_name, method, \"\", false)|indent(16)}}\n> +\t\t{\n> +{%- if method|method_return_value != \"void\" %}\n> +\t\t\treturn ipa_->{{method.mojom_name}}({{method.parameters|params_comma_sep}});\n> +{%- else %}\n> +\t\t\tipa_->{{method.mojom_name}}({{method.parameters|params_comma_sep}}\n> +\t{{- \", \" if method|method_param_outputs|params_comma_sep -}}\n> +\t{{- method|method_param_outputs|params_comma_sep}});\n> +{%- endif %}\n> +\t\t}\n> +{%- endif %}\n> +{%- endfor %}\n> +\n> +\tprivate:\n> +\t\t{{interface_name}} *ipa_;\n> +\t};\n> +\n> +\tbool running_;\n> +\tThread thread_;\n> +\tThreadProxy proxy_;\n> +\tstd::unique_ptr<{{interface_name}}> ipa_;\n> +\n> +\tconst bool isolate_;\n> +\n> +\tstd::unique_ptr ipc_;\n> +\n> +\tControlSerializer controlSerializer_;\n> +};\n> +\n> +{%- if has_namespace %}\n> +{% for ns in namespace|reverse %}\n> +} /* namespace {{ns}} */\n> +{% endfor %}\n> +{%- endif %}\n> +} /* namespace libcamera */\n> +\n> +#endif /* __LIBCAMERA_INTERNAL_IPA_PROXY_{{module_name|upper}}_H__ */\n> diff --git a/utils/ipc/generators/libcamera_templates/module_ipa_proxy_worker.cpp.tmpl b/utils/ipc/generators/libcamera_templates/module_ipa_proxy_worker.cpp.tmpl\n> new file mode 100644\n> index 00000000..cb3df4eb\n> --- /dev/null\n> +++ b/utils/ipc/generators/libcamera_templates/module_ipa_proxy_worker.cpp.tmpl\n> @@ -0,0 +1,224 @@\n> +{#-\n> + # SPDX-License-Identifier: LGPL-2.1-or-later\n> + # Copyright (C) 2020, Google Inc.\n> +-#}\n> +{%- import \"proxy_functions.tmpl\" as proxy_funcs -%}\n> +\n> +/* SPDX-License-Identifier: LGPL-2.1-or-later */\n> +/*\n> + * Copyright (C) 2020, Google Inc.\n> + *\n> + * {{module_name}}_ipa_proxy_worker.cpp - Image Processing Algorithm proxy worker for {{module_name}}\n> + *\n> + * This file is auto-generated. Do not edit.\n> + */\n> +\n> +{#- \\todo Split proxy worker into IPC worker and proxy worker. #}\n> +\n> +#include \n> +#include \n> +#include \n> +#include \n> +#include \n> +\n> +#include \n> +#include \n> +#include \n> +#include \n> +\n> +#include \"libcamera/internal/camera_sensor.h\"\n> +#include \"libcamera/internal/control_serializer.h\"\n> +#include \"libcamera/internal/event_dispatcher.h\"\n> +#include \"libcamera/internal/ipa_data_serializer.h\"\n> +#include \"libcamera/internal/ipc_pipe.h\"\n> +#include \"libcamera/internal/ipc_pipe_unixsocket.h\"\n> +#include \"libcamera/internal/ipa_module.h\"\n> +#include \"libcamera/internal/ipa_proxy.h\"\n> +#include \"libcamera/internal/ipc_unixsocket.h\"\n> +#include \"libcamera/internal/log.h\"\n> +#include \"libcamera/internal/thread.h\"\n> +\n> +using namespace libcamera;\n> +\n> +LOG_DEFINE_CATEGORY({{proxy_worker_name}})\n> +\n> +{%- if has_namespace %}\n> +{% for ns in namespace -%}\n> +using namespace {{ns}};\n> +{% endfor %}\n> +{%- endif %}\n> +\n> +class {{proxy_worker_name}}\n> +{\n> +public:\n> +\t{{proxy_worker_name}}()\n> +\t\t: ipa_(nullptr), exit_(false) {}\n> +\n> +\t~{{proxy_worker_name}}() {}\n> +\n> +\tvoid readyRead(IPCUnixSocket *socket)\n> +\t{\n> +\t\tIPCUnixSocket::Payload _message;\n> +\t\tint _retRecv = socket->receive(&_message);\n> +\t\tif (_retRecv) {\n> +\t\t\tLOG({{proxy_worker_name}}, Error)\n> +\t\t\t\t<< \"Receive message failed\" << _retRecv;\n> +\t\t\treturn;\n> +\t\t}\n> +\n> +\t\tIPCMessage _ipcMessage(_message);\n> +\n> +\t\t{{cmd_enum_name}} _cmd = static_cast<{{cmd_enum_name}}>(_ipcMessage.header().cmd);\n> +\n> +\t\tswitch (_cmd) {\n> +\t\tcase {{cmd_enum_name}}::Exit: {\n> +\t\t\texit_ = true;\n> +\t\t\tbreak;\n> +\t\t}\n> +\n> +{% for method in interface_main.methods %}\n> +\t\tcase {{cmd_enum_name}}::{{method.mojom_name|cap}}: {\n> +\t\t{{proxy_funcs.deserialize_call(method|method_param_inputs, '_ipcMessage.data()', '_ipcMessage.fds()', false, true)|indent(8, true)}}\n> +{% for param in method|method_param_outputs %}\n> +\t\t\t{{param|name}} {{param.mojom_name}};\n> +{% endfor %}\n> +{%- if method|method_return_value != \"void\" %}\n> +\t\t\t{{method|method_return_value}} _callRet = ipa_->{{method.mojom_name}}({{method.parameters|params_comma_sep}});\n> +{%- else %}\n> +\t\t\tipa_->{{method.mojom_name}}({{method.parameters|params_comma_sep}}\n> +{{- \", \" if method|method_param_outputs|params_comma_sep -}}\n> +{%- for param in method|method_param_outputs -%}\n> +&{{param.mojom_name}}{{\", \" if not loop.last}}\n> +{%- endfor -%}\n> +);\n> +{%- endif %}\n> +{% if not method|is_async %}\n> +\t\t\tIPCMessage::Header header = { _ipcMessage.header().cmd, _ipcMessage.header().cookie };\n> +\t\t\tIPCMessage _response(header);\n> +{%- if method|method_return_value != \"void\" %}\n> +\t\t\tstd::vector _callRetBuf;\n> +\t\t\tstd::tie(_callRetBuf, std::ignore) =\n> +\t\t\t\tIPADataSerializer<{{method|method_return_value}}>::serialize(_callRet);\n> +\t\t\t_response.data().insert(_response.data().end(), _callRetBuf.cbegin(), _callRetBuf.cend());\n> +{%- else %}\n> +\t\t{{proxy_funcs.serialize_call(method|method_param_outputs, \"_response.data()\", \"_response.fds()\")|indent(16, true)}}\n> +{%- endif %}\n> +\t\t\tint _ret = socket_.send(_response.payload());\n> +\t\t\tif (_ret < 0) {\n> +\t\t\t\tLOG({{proxy_worker_name}}, Error)\n> +\t\t\t\t\t<< \"Reply to {{method.mojom_name}}() failed\" << _ret;\n> +\t\t\t}\n> +\t\t\tLOG({{proxy_worker_name}}, Debug) << \"Done replying to {{method.mojom_name}}()\";\n> +{%- endif %}\n> +\t\t\tbreak;\n> +\t\t}\n> +{% endfor %}\n> +\t\tdefault:\n> +\t\t\tLOG({{proxy_worker_name}}, Error) << \"Unknown command \" << _ipcMessage.header().cmd;\n> +\t\t}\n> +\t}\n> +\n> +\tint init(std::unique_ptr &ipam, int socketfd)\n> +\t{\n> +\t\tif (socket_.bind(socketfd) < 0) {\n> +\t\t\tLOG({{proxy_worker_name}}, Error)\n> +\t\t\t\t<< \"IPC socket binding failed\";\n> +\t\t\treturn EXIT_FAILURE;\n> +\t\t}\n> +\t\tsocket_.readyRead.connect(this, &{{proxy_worker_name}}::readyRead);\n> +\n> +\t\tipa_ = dynamic_cast<{{interface_name}} *>(ipam->createInterface());\n> +\t\tif (!ipa_) {\n> +\t\t\tLOG({{proxy_worker_name}}, Error)\n> +\t\t\t\t<< \"Failed to create IPA interface instance\";\n> +\t\t\treturn EXIT_FAILURE;\n> +\t\t}\n> +{% for method in interface_event.methods %}\n> +\t\tipa_->{{method.mojom_name}}.connect(this, &{{proxy_worker_name}}::{{method.mojom_name}});\n> +{%- endfor %}\n> +\t\treturn 0;\n> +\t}\n> +\n> +\tvoid run()\n> +\t{\n> +\t\tEventDispatcher *dispatcher = Thread::current()->eventDispatcher();\n> +\t\twhile (!exit_)\n> +\t\t\tdispatcher->processEvents();\n> +\t}\n> +\n> +\tvoid cleanup()\n> +\t{\n> +\t\tdelete ipa_;\n> +\t\tsocket_.close();\n> +\t}\n> +\n> +private:\n> +\n> +{% for method in interface_event.methods %}\n> +{{proxy_funcs.func_sig(proxy_name, method, \"\", false)|indent(8, true)}}\n> +\t{\n> +\t\tIPCMessage::Header header = {\n> +\t\t\tstatic_cast({{cmd_event_enum_name}}::{{method.mojom_name|cap}}),\n> +\t\t\t0\n> +\t\t};\n> +\t\tIPCMessage _message(header);\n> +\n> +\t\t{{proxy_funcs.serialize_call(method|method_param_inputs, \"_message.data()\", \"_message.fds()\")}}\n> +\n> +\t\tsocket_.send(_message.payload());\n> +\n> +\t\tLOG({{proxy_worker_name}}, Debug) << \"{{method.mojom_name}} done\";\n> +\t}\n> +{% endfor %}\n> +\n> +\t{{interface_name}} *ipa_;\n> +\tIPCUnixSocket socket_;\n> +\n> +\tControlSerializer controlSerializer_;\n> +\n> +\tbool exit_;\n> +};\n> +\n> +int main(int argc, char **argv)\n> +{\n> +{#- \\todo Handle enabling debugging more dynamically. #}\n> +\t/* Uncomment this for debugging. */\n> +\tstd::string logPath = \"/tmp/libcamera.worker.\" +\n> +\t\t\t std::to_string(getpid()) + \".log\";\n> +\tlogSetFile(logPath.c_str());\n> +\n> +\tif (argc < 3) {\n> +\t\tLOG({{proxy_worker_name}}, Error)\n> +\t\t\t<< \"Tried to start worker with no args: \"\n> +\t\t\t<< \"expected \";\n> +\t\treturn EXIT_FAILURE;\n> +\t}\n> +\n> +\tint fd = std::stoi(argv[2]);\n> +\tLOG({{proxy_worker_name}}, Info)\n> +\t\t<< \"Starting worker for IPA module \" << argv[1]\n> +\t\t<< \" with IPC fd = \" << fd;\n> +\n> +\tstd::unique_ptr ipam = std::make_unique(argv[1]);\n> +\tif (!ipam->isValid() || !ipam->load()) {\n> +\t\tLOG({{proxy_worker_name}}, Error)\n> +\t\t\t<< \"IPAModule \" << argv[1] << \" isn't valid\";\n> +\t\treturn EXIT_FAILURE;\n> +\t}\n> +\n> +\t{{proxy_worker_name}} proxyWorker;\n> +\tint ret = proxyWorker.init(ipam, fd);\n> +\tif (ret < 0) {\n> +\t\tLOG({{proxy_worker_name}}, Error)\n> +\t\t\t<< \"Failed to initialize proxy worker\";\n> +\t\treturn ret;\n> +\t}\n> +\n> +\tLOG({{proxy_worker_name}}, Debug) << \"Proxy worker successfully started\";\n> +\n> +\tproxyWorker.run();\n> +\n> +\tproxyWorker.cleanup();\n> +\n> +\treturn 0;\n> +}\n> diff --git a/utils/ipc/generators/libcamera_templates/module_ipa_serializer.h.tmpl b/utils/ipc/generators/libcamera_templates/module_ipa_serializer.h.tmpl\n> new file mode 100644\n> index 00000000..ad522964\n> --- /dev/null\n> +++ b/utils/ipc/generators/libcamera_templates/module_ipa_serializer.h.tmpl\n> @@ -0,0 +1,47 @@\n> +{#-\n> + # SPDX-License-Identifier: LGPL-2.1-or-later\n> + # Copyright (C) 2020, Google Inc.\n> +-#}\n> +{%- import \"serializer.tmpl\" as serializer -%}\n> +\n> +/* SPDX-License-Identifier: LGPL-2.1-or-later */\n> +/*\n> + * Copyright (C) 2020, Google Inc.\n> + *\n> + * {{module_name}}_ipa_serializer.h - Image Processing Algorithm data serializer for {{module_name}}\n> + *\n> + * This file is auto-generated. Do not edit.\n> + */\n> +\n> +#ifndef __LIBCAMERA_INTERNAL_IPA_DATA_SERIALIZER_{{module_name|upper}}_H__\n> +#define __LIBCAMERA_INTERNAL_IPA_DATA_SERIALIZER_{{module_name|upper}}_H__\n> +\n> +#include \n> +#include \n> +\n> +#include \n> +#include \n> +\n> +#include \"libcamera/internal/control_serializer.h\"\n> +#include \"libcamera/internal/ipa_data_serializer.h\"\n> +\n> +namespace libcamera {\n> +\n> +LOG_DECLARE_CATEGORY(IPADataSerializer)\n> +{% for struct in structs_nonempty %}\n> +template<>\n> +class IPADataSerializer<{{struct|name_full(namespace_str)}}>\n> +{\n> +public:\n> +{{- serializer.serializer(struct, namespace_str)}}\n> +{%- if struct|has_fd %}\n> +{{serializer.deserializer_fd(struct, namespace_str)}}\n> +{%- else %}\n> +{{serializer.deserializer_no_fd(struct, namespace_str)}}\n> +{%- endif %}\n> +};\n> +{% endfor %}\n> +\n> +} /* namespace libcamera */\n> +\n> +#endif /* __LIBCAMERA_INTERNAL_IPA_DATA_SERIALIZER_{{module_name|upper}}_H__ */\n> diff --git a/utils/ipc/generators/libcamera_templates/proxy_functions.tmpl b/utils/ipc/generators/libcamera_templates/proxy_functions.tmpl\n> new file mode 100644\n> index 00000000..3f2143b1\n> --- /dev/null\n> +++ b/utils/ipc/generators/libcamera_templates/proxy_functions.tmpl\n> @@ -0,0 +1,192 @@\n> +{#-\n> + # SPDX-License-Identifier: LGPL-2.1-or-later\n> + # Copyright (C) 2020, Google Inc.\n> +-#}\n> +{#\n> + # \\brief Generate fuction prototype\n> + #\n> + # \\param class Class name\n> + # \\param method mojom Method object\n> + # \\param suffix Suffix to append to \\a method function name\n> + # \\param need_class_name If true, generate class name with function\n> + # \\param override If true, generate override tag after the function prototype\n> + #}\n> +{%- macro func_sig(class, method, suffix = \"\", need_class_name = true, override = false) -%}\n> +{{method|method_return_value}} {{class + \"::\" if need_class_name}}{{method.mojom_name}}{{suffix}}(\n> +{%- for param in method|method_parameters %}\n> +\t{{param}}{{- \",\" if not loop.last}}\n> +{%- endfor -%}\n> +){{\" override\" if override}}\n> +{%- endmacro -%}\n> +\n> +{#\n> + # \\brief Generate function body for IPA init() function for thread\n> + #}\n> +{%- macro init_thread_body() -%}\n> +\tint ret = ipa_->init(settings);\n> +\tif (ret)\n> +\t\treturn ret;\n> +\n> +\tproxy_.moveToThread(&thread_);\n> +\n> +\treturn 0;\n> +{%- endmacro -%}\n> +\n> +{#\n> + # \\brief Generate function body for IPA stop() function for thread\n> + #}\n> +{%- macro stop_thread_body() -%}\n> +\tif (!running_)\n> +\t\treturn;\n> +\n> +\trunning_ = false;\n> +\n> +\tproxy_.invokeMethod(&ThreadProxy::stop, ConnectionTypeBlocking);\n> +\n> +\tthread_.exit();\n> +\tthread_.wait();\n> +{%- endmacro -%}\n> +\n> +\n> +{#\n> + # \\brief Serialize multiple objects into data buffer and fd vector\n> + #\n> + # Generate code to serialize multiple objects, as specified in \\a params\n> + # (which are the parameters to some function), into \\a buf data buffer and\n> + # \\a fds fd vector.\n> + # This code is meant to be used by the proxy, for serializing prior to IPC calls.\n> + #}\n> +{%- macro serialize_call(params, buf, fds) %}\n> +{%- for param in params %}\n> +\tstd::vector {{param.mojom_name}}Buf;\n> +{%- if param|has_fd %}\n> +\tstd::vector {{param.mojom_name}}Fds;\n> +\tstd::tie({{param.mojom_name}}Buf, {{param.mojom_name}}Fds) =\n> +{%- else %}\n> +\tstd::tie({{param.mojom_name}}Buf, std::ignore) =\n> +{%- endif %}\n> +\t\tIPADataSerializer<{{param|name}}>::serialize({{param.mojom_name}}\n> +{{- \", &controlSerializer_\" if param|needs_control_serializer -}}\n> +);\n> +{%- endfor %}\n> +\n> +{%- if params|length > 1 %}\n> +{%- for param in params %}\n> +\tappendPOD({{buf}}, {{param.mojom_name}}Buf.size());\n> +{%- if param|has_fd %}\n> +\tappendPOD({{buf}}, {{param.mojom_name}}Fds.size());\n> +{%- endif %}\n> +{%- endfor %}\n> +{%- endif %}\n> +\n> +{%- for param in params %}\n> +\t{{buf}}.insert({{buf}}.end(), {{param.mojom_name}}Buf.begin(), {{param.mojom_name}}Buf.end());\n> +{%- endfor %}\n> +\n> +{%- for param in params %}\n> +{%- if param|has_fd %}\n> +\t{{fds}}.insert({{fds}}.end(), {{param.mojom_name}}Fds.begin(), {{param.mojom_name}}Fds.end());\n> +{%- endif %}\n> +{%- endfor %}\n> +{%- endmacro -%}\n> +\n> +\n> +{#\n> + # \\brief Deserialize a single object from data buffer and fd vector\n> + #\n> + # \\param pointer If true, deserialize the object into a dereferenced pointer\n> + # \\param iter If true, treat \\a buf as an iterator instead of a vector\n> + # \\param data_size Variable that holds the size of the vector referenced by \\a buf\n> + #\n> + # Generate code to deserialize a single object, as specified in \\a param,\n> + # from \\a buf data buffer and \\a fds fd vector.\n> + # This code is meant to be used by macro deserialize_call.\n> + #}\n> +{%- macro deserialize_param(param, pointer, loop, buf, fds, iter, data_size) -%}\n> +{{\"*\" if pointer}}{{param.mojom_name}} = IPADataSerializer<{{param|name}}>::deserialize(\n> +\t{{buf}}{{- \".cbegin()\" if not iter}} + {{param.mojom_name}}Start,\n> +{%- if loop.last and not iter %}\n> +\t{{buf}}.cend()\n> +{%- elif not iter %}\n> +\t{{buf}}.cbegin() + {{param.mojom_name}}Start + {{param.mojom_name}}BufSize\n> +{%- elif iter and loop.length == 1 %}\n> +\t{{buf}} + {{data_size}}\n> +{%- else %}\n> +\t{{buf}} + {{param.mojom_name}}Start + {{param.mojom_name}}BufSize\n> +{%- endif -%}\n> +{{- \",\" if param|has_fd}}\n> +{%- if param|has_fd %}\n> +\t{{fds}}.cbegin() + {{param.mojom_name}}FdStart,\n> +{%- if loop.last %}\n> +\t{{fds}}.cend()\n> +{%- else %}\n> +\t{{fds}}.cbegin() + {{param.mojom_name}}FdStart + {{param.mojom_name}}FdsSize\n> +{%- endif -%}\n> +{%- endif -%}\n> +{{- \",\" if param|needs_control_serializer}}\n> +{%- if param|needs_control_serializer %}\n> +\t&controlSerializer_\n> +{%- endif -%}\n> +);\n> +{%- endmacro -%}\n> +\n> +\n> +{#\n> + # \\brief Deserialize multiple objects from data buffer and fd vector\n> + #\n> + # \\param pointer If true, deserialize objects into pointers, and adds a null check.\n> + # \\param declare If true, declare the objects in addition to deserialization.\n> + # \\param iter if true, treat \\a buf as an iterator instead of a vector\n> + # \\param data_size Variable that holds the size of the vector referenced by \\a buf\n> + #\n> + # Generate code to deserialize multiple objects, as specified in \\a params\n> + # (which are the parameters to some function), from \\a buf data buffer and\n> + # \\a fds fd vector.\n> + # This code is meant to be used by the proxy, for deserializing after IPC calls.\n> + #\n> + # \\todo Avoid intermediate vectors\n> + #}\n> +{%- macro deserialize_call(params, buf, fds, pointer = true, declare = false, iter = false, data_size = '') -%}\n> +{% set ns = namespace(size_offset = 0) %}\n> +{%- if params|length > 1 %}\n> +{%- for param in params %}\n> +\t[[maybe_unused]] const size_t {{param.mojom_name}}BufSize = readPOD({{buf}}, {{ns.size_offset}}\n> +{%- if iter -%}\n> +, {{buf}} + {{data_size}}\n> +{%- endif -%}\n> +);\n> +\t{%- set ns.size_offset = ns.size_offset + 4 %}\n> +{%- if param|has_fd %}\n> +\t[[maybe_unused]] const size_t {{param.mojom_name}}FdsSize = readPOD({{buf}}, {{ns.size_offset}}\n> +{%- if iter -%}\n> +, {{buf}} + {{data_size}}\n> +{%- endif -%}\n> +);\n> +\t{%- set ns.size_offset = ns.size_offset + 4 %}\n> +{%- endif %}\n> +{%- endfor %}\n> +{%- endif %}\n> +{% for param in params %}\n> +{%- if loop.first %}\n> +\tconst size_t {{param.mojom_name}}Start = {{ns.size_offset}};\n> +{%- else %}\n> +\tconst size_t {{param.mojom_name}}Start = {{loop.previtem.mojom_name}}Start + {{loop.previtem.mojom_name}}BufSize;\n> +{%- endif %}\n> +{%- endfor %}\n> +{% for param in params|with_fds %}\n> +{%- if loop.first %}\n> +\tconst size_t {{param.mojom_name}}FdStart = 0;\n> +{%- elif not loop.last %}\n> +\tconst size_t {{param.mojom_name}}FdStart = {{loop.previtem.mojom_name}}FdStart + {{loop.previtem.mojom_name}}FdsSize;\n> +{%- endif %}\n> +{%- endfor %}\n> +{% for param in params %}\n> +\t{%- if pointer %}\n> +\tif ({{param.mojom_name}}) {\n> +{{deserialize_param(param, pointer, loop, buf, fds, iter, data_size)|indent(16, True)}}\n> +\t}\n> +\t{%- else %}\n> +\t{{param|name + \" \" if declare}}{{deserialize_param(param, pointer, loop, buf, fds, iter, data_size)|indent(8)}}\n> +\t{%- endif %}\n> +{% endfor %}\n> +{%- endmacro -%}\n> diff --git a/utils/ipc/generators/libcamera_templates/serializer.tmpl b/utils/ipc/generators/libcamera_templates/serializer.tmpl\n> new file mode 100644\n> index 00000000..af4800bf\n> --- /dev/null\n> +++ b/utils/ipc/generators/libcamera_templates/serializer.tmpl\n> @@ -0,0 +1,313 @@\n> +{#-\n> + # SPDX-License-Identifier: LGPL-2.1-or-later\n> + # Copyright (C) 2020, Google Inc.\n> +-#}\n> +{# Turn this into a C macro? #}\n\nIs this a todo?\n\n\n\n> +{#\n> + # \\brief Verify that there is enough bytes to deserialize\n> + #\n> + # Generate code that verifies that \\a size is not greater than \\a dataSize.\n> + # Otherwise log an error with \\a name and \\a typename.\n> + #}\n> +{%- macro check_data_size(size, dataSize, name, typename) %}\n> +\t\tif ({{dataSize}} < {{size}}) {\n> +\t\t\tLOG(IPADataSerializer, Error)\n> +\t\t\t\t<< \"Failed to deserialize \" << \"{{name}}\"\n> +\t\t\t\t<< \": not enough {{typename}}, expected \"\n> +\t\t\t\t<< ({{size}}) << \", got \" << ({{dataSize}});\n> +\t\t\treturn ret;\n> +\t\t}\n> +{%- endmacro %}\n> +\n> +\n> +{#\n> + # \\brief Serialize some field into return vector\n\n'some' field?\na field?\n\n> + #\n> + # Generate code to serialize \\a field into retData, including size of the\n> + # field and fds (where appropriate).\n> + # This code is meant to be used by the IPADataSerializer specialization.\n> + #\n> + # \\todo Avoid intermediate vectors\n> + #}\n> +{%- macro serializer_field(field, namespace, loop) %}\n> +{%- if field|is_pod or field|is_enum %}\n> +\t\tstd::vector {{field.mojom_name}};\n> +\t\tstd::tie({{field.mojom_name}}, std::ignore) =\n> +\t{%- if field|is_pod %}\n> +\t\t\tIPADataSerializer<{{field|name}}>::serialize(data.{{field.mojom_name}});\n> +\t{%- elif field|is_enum %}\n> +\t\t\tIPADataSerializer::serialize(data.{{field.mojom_name}});\n> +\t{%- endif %}\n> +\t\tretData.insert(retData.end(), {{field.mojom_name}}.begin(), {{field.mojom_name}}.end());\n> +{%- elif field|is_fd %}\n> +\t\tstd::vector {{field.mojom_name}};\n> +\t\tstd::vector {{field.mojom_name}}Fds;\n> +\t\tstd::tie({{field.mojom_name}}, {{field.mojom_name}}Fds) =\n> +\t\t\tIPADataSerializer<{{field|name}}>::serialize(data.{{field.mojom_name}});\n> +\t\tretData.insert(retData.end(), {{field.mojom_name}}.begin(), {{field.mojom_name}}.end());\n> +\t\tretFds.insert(retFds.end(), {{field.mojom_name}}Fds.begin(), {{field.mojom_name}}Fds.end());\n> +{%- elif field|is_controls %}\n> +\t\tif (data.{{field.mojom_name}}.size() > 0) {\n> +\t\t\tstd::vector {{field.mojom_name}};\n> +\t\t\tstd::tie({{field.mojom_name}}, std::ignore) =\n> +\t\t\t\tIPADataSerializer<{{field|name}}>::serialize(data.{{field.mojom_name}}, cs);\n> +\t\t\tappendPOD(retData, {{field.mojom_name}}.size());\n> +\t\t\tretData.insert(retData.end(), {{field.mojom_name}}.begin(), {{field.mojom_name}}.end());\n> +\t\t} else {\n> +\t\t\tappendPOD(retData, 0);\n> +\t\t}\n> +{%- elif field|is_plain_struct or field|is_array or field|is_map or field|is_str %}\n> +\t\tstd::vector {{field.mojom_name}};\n> +\t{%- if field|has_fd %}\n> +\t\tstd::vector {{field.mojom_name}}Fds;\n> +\t\tstd::tie({{field.mojom_name}}, {{field.mojom_name}}Fds) =\n> +\t{%- else %}\n> +\t\tstd::tie({{field.mojom_name}}, std::ignore) =\n> +\t{%- endif %}\n> +\t{%- if field|is_array or field|is_map %}\n> +\t\t\tIPADataSerializer<{{field|name}}>::serialize(data.{{field.mojom_name}}, cs);\n> +\t{%- elif field|is_str %}\n> +\t\t\tIPADataSerializer<{{field|name}}>::serialize(data.{{field.mojom_name}});\n> +\t{%- else %}\n> +\t\t\tIPADataSerializer<{{field|name_full(namespace)}}>::serialize(data.{{field.mojom_name}}, cs);\n> +\t{%- endif %}\n> +\t\tappendPOD(retData, {{field.mojom_name}}.size());\n> +\t{%- if field|has_fd %}\n> +\t\tappendPOD(retData, {{field.mojom_name}}Fds.size());\n> +\t{%- endif %}\n> +\t\tretData.insert(retData.end(), {{field.mojom_name}}.begin(), {{field.mojom_name}}.end());\n> +\t{%- if field|has_fd %}\n> +\t\tretFds.insert(retFds.end(), {{field.mojom_name}}Fds.begin(), {{field.mojom_name}}Fds.end());\n> +\t{%- endif %}\n> +{%- else %}\n> +\t\t/* Unknown serialization for {{field.mojom_name}}. */\n> +{%- endif %}\n> +{%- endmacro %}\n> +\n> +\n> +{#\n> + # \\brief Deserialize some field into return struct\n\nAgain, 'some field' sounds like a random field.\nPresumably this is going to handle a quite specific one each time...\n\n\n> + #\n> + # Generate code to deserialize \\a field into object ret.\n> + # This code is meant to be used by the IPADataSerializer specialization.\n> + #}\n> +{%- macro deserializer_field(field, namespace, loop) %}\n> +{% if field|is_pod or field|is_enum %}\n> +\t{%- set field_size = (field|bit_width|int / 8)|int %}\n> +\t\t{{- check_data_size(field_size, 'dataSize', field.mojom_name, 'data')}}\n> +\t\t{%- if field|is_pod %}\n> +\t\tret.{{field.mojom_name}} = IPADataSerializer<{{field|name}}>::deserialize(m, m + {{field_size}});\n> +\t\t{%- else %}\n> +\t\tret.{{field.mojom_name}} = static_cast<{{field|name_full(namespace)}}>(IPADataSerializer::deserialize(m, m + {{field_size}}));\n> +\t\t{%- endif %}\n> +\t{%- if not loop.last %}\n> +\t\tm += {{field_size}};\n> +\t\tdataSize -= {{field_size}};\n> +\t{%- endif %}\n> +{% elif field|is_fd %}\n> +\t{%- set field_size = 1 %}\n> +\t\t{{- check_data_size(field_size, 'dataSize', field.mojom_name, 'data')}}\n> +\t\tret.{{field.mojom_name}} = IPADataSerializer<{{field|name}}>::deserialize(m, m + 1, n, n + 1, cs);\n> +\t{%- if not loop.last %}\n> +\t\tm += {{field_size}};\n> +\t\tdataSize -= {{field_size}};\n> +\t\tn += ret.{{field.mojom_name}}.isValid() ? 1 : 0;\n> +\t\tfdsSize -= ret.{{field.mojom_name}}.isValid() ? 1 : 0;\n> +\t{%- endif %}\n> +{% elif field|is_controls %}\n> +\t{%- set field_size = 4 %}\n> +\t\t{{- check_data_size(field_size, 'dataSize', field.mojom_name + 'Size', 'data')}}\n> +\t\tconst size_t {{field.mojom_name}}Size = readPOD(m, 0, dataEnd);\n> +\t\tm += {{field_size}};\n> +\t\tdataSize -= {{field_size}};\n> +\t{%- set field_size = field.mojom_name + 'Size' -%}\n> +\t\t{{- check_data_size(field_size, 'dataSize', field.mojom_name, 'data')}}\n> +\t\tif ({{field.mojom_name}}Size > 0)\n> +\t\t\tret.{{field.mojom_name}} =\n> +\t\t\t\tIPADataSerializer<{{field|name}}>::deserialize(m, m + {{field.mojom_name}}Size, cs);\n> +\t{%- if not loop.last %}\n> +\t\tm += {{field_size}};\n> +\t\tdataSize -= {{field_size}};\n> +\t{%- endif %}\n> +{% elif field|is_plain_struct or field|is_array or field|is_map or field|is_str %}\n> +\t{%- set field_size = 4 %}\n> +\t\t{{- check_data_size(field_size, 'dataSize', field.mojom_name + 'Size', 'data')}}\n> +\t\tconst size_t {{field.mojom_name}}Size = readPOD(m, 0, dataEnd);\n> +\t\tm += {{field_size}};\n> +\t\tdataSize -= {{field_size}};\n> +\t{%- if field|has_fd %}\n> +\t{%- set field_size = 4 %}\n> +\t\t{{- check_data_size(field_size, 'dataSize', field.mojom_name + 'FdsSize', 'data')}}\n> +\t\tconst size_t {{field.mojom_name}}FdsSize = readPOD(m, 0, dataEnd);\n> +\t\tm += {{field_size}};\n> +\t\tdataSize -= {{field_size}};\n> +\t\t{{- check_data_size(field.mojom_name + 'FdsSize', 'fdsSize', field.mojom_name, 'fds')}}\n> +\t{%- endif %}\n> +\t{%- set field_size = field.mojom_name + 'Size' -%}\n> +\t\t{{- check_data_size(field_size, 'dataSize', field.mojom_name, 'data')}}\n> +\t\tret.{{field.mojom_name}} =\n> +\t{%- if field|is_str %}\n> +\t\t\tIPADataSerializer<{{field|name}}>::deserialize(m, m + {{field.mojom_name}}Size);\n> +\t{%- elif field|has_fd and (field|is_array or field|is_map) %}\n> +\t\t\tIPADataSerializer<{{field|name}}>::deserialize(m, m + {{field.mojom_name}}Size, n, n + {{field.mojom_name}}FdsSize, cs);\n> +\t{%- elif field|has_fd and (not (field|is_array or field|is_map)) %}\n> +\t\t\tIPADataSerializer<{{field|name_full(namespace)}}>::deserialize(m, m + {{field.mojom_name}}Size, n, n + {{field.mojom_name}}FdsSize, cs);\n> +\t{%- elif (not field|has_fd) and (field|is_array or field|is_map) %}\n> +\t\t\tIPADataSerializer<{{field|name}}>::deserialize(m, m + {{field.mojom_name}}Size, cs);\n> +\t{%- else %}\n> +\t\t\tIPADataSerializer<{{field|name_full(namespace)}}>::deserialize(m, m + {{field.mojom_name}}Size, cs);\n> +\t{%- endif %}\n> +\t{%- if not loop.last %}\n> +\t\tm += {{field_size}};\n> +\t\tdataSize -= {{field_size}};\n> +\t{%- if field|has_fd %}\n> +\t\tn += {{field.mojom_name}}FdsSize;\n> +\t\tfdsSize -= {{field.mojom_name}}FdsSize;\n> +\t{%- endif %}\n> +\t{%- endif %}\n> +{% else %}\n> +\t\t/* Unknown deserialization for {{field.mojom_name}}. */\n> +{%- endif %}\n> +{%- endmacro %}\n> +\n> +\n> +{#\n> + # \\brief Serialize a struct\n> + #\n> + # Generate code for IPADataSerializer specialization, for serializing\n> + # \\a struct.\n> + #}\n> +{%- macro serializer(struct, namespace) %}\n> +\tstatic std::tuple, std::vector>\n> +\tserialize(const {{struct|name_full(namespace)}} &data,\n> +{%- if struct|needs_control_serializer %}\n> +\t\t ControlSerializer *cs)\n> +{%- else %}\n> +\t\t [[maybe_unused]] ControlSerializer *cs = nullptr)\n> +{%- endif %}\n> +\t{\n> +\t\tstd::vector retData;\n> +{%- if struct|has_fd %}\n> +\t\tstd::vector retFds;\n> +{%- endif %}\n> +{%- for field in struct.fields %}\n> +{{serializer_field(field, namespace, loop)}}\n> +{%- endfor %}\n> +{% if struct|has_fd %}\n> +\t\treturn {retData, retFds};\n> +{%- else %}\n> +\t\treturn {retData, {}};\n> +{%- endif %}\n> +\t}\n> +{%- endmacro %}\n> +\n> +\n> +{#\n> + # \\brief Deserialize a struct that has fds\n> + #\n> + # Generate code for IPADataSerializer specialization, for deserializing\n> + # \\a struct, in the case that \\a struct has file descriptors.\n> + # fd parameters\n> + #}\n> +{%- macro deserializer_fd(struct, namespace) %}\n> +\tstatic {{struct|name_full(namespace)}}\n> +\tdeserialize(std::vector &data,\n> +\t\t std::vector &fds,\n> +{%- if struct|needs_control_serializer %}\n> +\t\t ControlSerializer *cs)\n> +{%- else %}\n> +\t\t [[maybe_unused]] ControlSerializer *cs = nullptr)\n> +{%- endif %}\n> +\t{\n> +\t\treturn IPADataSerializer<{{struct|name_full(namespace)}}>::deserialize(data.cbegin(), data.cend(), fds.cbegin(), fds.cend(), cs);\n> +\t}\n> +\n> +\tstatic {{struct|name_full(namespace)}}\n> +\tdeserialize(std::vector::const_iterator dataBegin,\n> +\t\t std::vector::const_iterator dataEnd,\n> +\t\t std::vector::const_iterator fdsBegin,\n> +\t\t std::vector::const_iterator fdsEnd,\n> +{%- if struct|needs_control_serializer %}\n> +\t\t ControlSerializer *cs)\n> +{%- else %}\n> +\t\t [[maybe_unused]] ControlSerializer *cs = nullptr)\n> +{%- endif %}\n> +\t{\n> +\t\t{{struct|name_full(namespace)}} ret;\n> +\t\tstd::vector::const_iterator m = dataBegin;\n> +\t\tstd::vector::const_iterator n = fdsBegin;\n> +\n> +\t\tsize_t dataSize = std::distance(dataBegin, dataEnd);\n> +\t\t[[maybe_unused]] size_t fdsSize = std::distance(fdsBegin, fdsEnd);\n> +{%- for field in struct.fields -%}\n> +{{deserializer_field(field, namespace, loop)}}\n> +{%- endfor %}\n> +\t\treturn ret;\n> +\t}\n> +{%- endmacro %}\n> +\n> +{#\n> + # \\brief Deserialize a struct that has fds, using non-fd\n> + #\n> + # Generate code for IPADataSerializer specialization, for deserializing\n> + # \\a struct, in the case that \\a struct has no file descriptors but requires\n> + # deserializers with file descriptors.\n\nIf the struct has no file descriptors, why does it need a deserialiszer\nwith file descriptors?\n\n\n\nOther than the documentation comments, I don't think I can hope to\nprovide much more parsing of all this.\n\nAcked-by: Kieran Bingham \n\n\n\n\n> + #}\n> +{%- macro deserializer_fd_simple(struct, namespace) %}\n> +\tstatic {{struct|name_full(namespace)}}\n> +\tdeserialize(std::vector &data,\n> +\t\t [[maybe_unused]] std::vector &fds,\n> +\t\t [[maybe_unused]] ControlSerializer *cs = nullptr)\n> +\t{\n> +\t\treturn IPADataSerializer<{{struct|name_full(namespace)}}>::deserialize(data.cbegin(), data.cend(), cs);\n> +\t}\n> +\n> +\tstatic {{struct|name_full(namespace)}}\n> +\tdeserialize(std::vector::const_iterator dataBegin,\n> +\t\t std::vector::const_iterator dataEnd,\n> +\t\t [[maybe_unused]] std::vector::const_iterator fdsBegin,\n> +\t\t [[maybe_unused]] std::vector::const_iterator fdsEnd,\n> +\t\t [[maybe_unused]] ControlSerializer *cs = nullptr)\n> +\t{\n> +\t\treturn IPADataSerializer<{{struct|name_full(namespace)}}>::deserialize(dataBegin, dataEnd, cs);\n> +\t}\n> +{%- endmacro %}\n> +\n> +\n> +{#\n> + # \\brief Deserialize a struct that has no fds\n> + #\n> + # Generate code for IPADataSerializer specialization, for deserializing\n> + # \\a struct, in the case that \\a struct does not have file descriptors.\n> + #}\n> +{%- macro deserializer_no_fd(struct, namespace) %}\n> +\tstatic {{struct|name_full(namespace)}}\n> +\tdeserialize(std::vector &data,\n> +{%- if struct|needs_control_serializer %}\n> +\t\t ControlSerializer *cs)\n> +{%- else %}\n> +\t\t [[maybe_unused]] ControlSerializer *cs = nullptr)\n> +{%- endif %}\n> +\t{\n> +\t\treturn IPADataSerializer<{{struct|name_full(namespace)}}>::deserialize(data.cbegin(), data.cend(), cs);\n> +\t}\n> +\n> +\tstatic {{struct|name_full(namespace)}}\n> +\tdeserialize(std::vector::const_iterator dataBegin,\n> +\t\t std::vector::const_iterator dataEnd,\n> +{%- if struct|needs_control_serializer %}\n> +\t\t ControlSerializer *cs)\n> +{%- else %}\n> +\t\t [[maybe_unused]] ControlSerializer *cs = nullptr)\n> +{%- endif %}\n> +\t{\n> +\t\t{{struct|name_full(namespace)}} ret;\n> +\t\tstd::vector::const_iterator m = dataBegin;\n> +\n> +\t\tsize_t dataSize = std::distance(dataBegin, dataEnd);\n> +{%- for field in struct.fields -%}\n> +{{deserializer_field(field, namespace, loop)}}\n> +{%- endfor %}\n> +\t\treturn ret;\n> +\t}\n> +{%- endmacro %}\n> diff --git a/utils/ipc/generators/mojom_libcamera_generator.py b/utils/ipc/generators/mojom_libcamera_generator.py\n> new file mode 100644\n> index 00000000..5c4ad4fe\n> --- /dev/null\n> +++ b/utils/ipc/generators/mojom_libcamera_generator.py\n> @@ -0,0 +1,511 @@\n> +#!/usr/bin/env python3\n> +# SPDX-License-Identifier: GPL-2.0-or-later\n> +# Copyright (C) 2020, Google Inc.\n> +#\n> +# Author: Paul Elder \n> +#\n> +# mojom_libcamera_generator.py - Generates libcamera files from a mojom.Module.\n> +\n> +import argparse\n> +import datetime\n> +import os\n> +import re\n> +\n> +import mojom.fileutil as fileutil\n> +import mojom.generate.generator as generator\n> +import mojom.generate.module as mojom\n> +from mojom.generate.template_expander import UseJinja\n> +\n> +\n> +GENERATOR_PREFIX = 'libcamera'\n> +\n> +_kind_to_cpp_type = {\n> + mojom.BOOL: 'bool',\n> + mojom.INT8: 'int8_t',\n> + mojom.UINT8: 'uint8_t',\n> + mojom.INT16: 'int16_t',\n> + mojom.UINT16: 'uint16_t',\n> + mojom.INT32: 'int32_t',\n> + mojom.UINT32: 'uint32_t',\n> + mojom.FLOAT: 'float',\n> + mojom.INT64: 'int64_t',\n> + mojom.UINT64: 'uint64_t',\n> + mojom.DOUBLE: 'double',\n> +}\n> +\n> +_bit_widths = {\n> + mojom.BOOL: '8',\n> + mojom.INT8: '8',\n> + mojom.UINT8: '8',\n> + mojom.INT16: '16',\n> + mojom.UINT16: '16',\n> + mojom.INT32: '32',\n> + mojom.UINT32: '32',\n> + mojom.FLOAT: '32',\n> + mojom.INT64: '64',\n> + mojom.UINT64: '64',\n> + mojom.DOUBLE: '64',\n> +}\n> +\n> +def ModuleName(path):\n> + return path.split('/')[-1].split('.')[0]\n> +\n> +def ModuleClassName(module):\n> + return re.sub(r'^IPA(.*)Interface$', lambda match: match.group(1),\n> + module.interfaces[0].mojom_name)\n> +\n> +def Capitalize(name):\n> + return name[0].upper() + name[1:]\n> +\n> +def ConstantStyle(name):\n> + return generator.ToUpperSnakeCase(name)\n> +\n> +def Choose(cond, t, f):\n> + return t if cond else f\n> +\n> +def CommaSep(l):\n> + return ', '.join([m for m in l])\n> +\n> +def ParamsCommaSep(l):\n> + return ', '.join([m.mojom_name for m in l])\n> +\n> +def GetDefaultValue(element):\n> + if element.default is not None:\n> + return element.default\n> + if type(element.kind) == mojom.Kind:\n> + return '0'\n> + if mojom.IsEnumKind(element.kind):\n> + return f'static_cast<{element.kind.mojom_name}>(0)'\n> + if isinstance(element.kind, mojom.Struct) and \\\n> + element.kind.mojom_name == 'FileDescriptor':\n> + return '-1'\n> + return ''\n> +\n> +def HasDefaultValue(element):\n> + return GetDefaultValue(element) != ''\n> +\n> +def HasDefaultFields(element):\n> + return True in [HasDefaultValue(x) for x in element.fields]\n> +\n> +def GetAllTypes(element):\n> + if mojom.IsArrayKind(element):\n> + return GetAllTypes(element.kind)\n> + if mojom.IsMapKind(element):\n> + return GetAllTypes(element.key_kind) + GetAllTypes(element.value_kind)\n> + if isinstance(element, mojom.Parameter):\n> + return GetAllTypes(element.kind)\n> + if mojom.IsEnumKind(element):\n> + return [element.mojom_name]\n> + if not mojom.IsStructKind(element):\n> + return [element.spec]\n> + if len(element.fields) == 0:\n> + return [element.mojom_name]\n> + ret = [GetAllTypes(x.kind) for x in element.fields]\n> + ret = [x for sublist in ret for x in sublist]\n> + return list(set(ret))\n> +\n> +def GetAllAttrs(element):\n> + if mojom.IsArrayKind(element):\n> + return GetAllAttrs(element.kind)\n> + if mojom.IsMapKind(element):\n> + return {**GetAllAttrs(element.key_kind), **GetAllAttrs(element.value_kind)}\n> + if isinstance(element, mojom.Parameter):\n> + return GetAllAttrs(element.kind)\n> + if mojom.IsEnumKind(element):\n> + return element.attributes if element.attributes is not None else {}\n> + if mojom.IsStructKind(element) and len(element.fields) == 0:\n> + return element.attributes if element.attributes is not None else {}\n> + if not mojom.IsStructKind(element):\n> + if hasattr(element, 'attributes'):\n> + return element.attributes or {}\n> + return {}\n> + attrs = [(x.attributes) for x in element.fields]\n> + ret = {}\n> + for d in attrs:\n> + ret.update(d or {})\n> + if hasattr(element, 'attributes'):\n> + ret.update(element.attributes or {})\n> + return ret\n> +\n> +def NeedsControlSerializer(element):\n> + types = GetAllTypes(element)\n> + return \"ControlList\" in types or \"ControlInfoMap\" in types\n> +\n> +def HasFd(element):\n> + attrs = GetAllAttrs(element)\n> + if isinstance(element, mojom.Kind):\n> + types = GetAllTypes(element)\n> + else:\n> + types = GetAllTypes(element.kind)\n> + return \"FileDescriptor\" in types or (attrs is not None and \"hasFd\" in attrs)\n> +\n> +def WithDefaultValues(element):\n> + return [x for x in element if HasDefaultValue(x)]\n> +\n> +def WithFds(element):\n> + return [x for x in element if HasFd(x)]\n> +\n> +def MethodParamInputs(method):\n> + return method.parameters\n> +\n> +def MethodParamOutputs(method):\n> + if (MethodReturnValue(method) != 'void' or\n> + method.response_parameters is None):\n> + return []\n> + return method.response_parameters\n> +\n> +def MethodParamsHaveFd(parameters):\n> + return len([x for x in parameters if HasFd(x)]) > 0\n> +\n> +def MethodInputHasFd(method):\n> + return MethodParamsHaveFd(method.parameters)\n> +\n> +def MethodOutputHasFd(method):\n> + if (MethodReturnValue(method) != 'void' or\n> + method.response_parameters is None):\n> + return False\n> + return MethodParamsHaveFd(method.response_parameters)\n> +\n> +def MethodParamNames(method):\n> + params = []\n> + for param in method.parameters:\n> + params.append(param.mojom_name)\n> + if MethodReturnValue(method) == 'void':\n> + if method.response_parameters is None:\n> + return params\n> + for param in method.response_parameters:\n> + params.append(param.mojom_name)\n> + return params\n> +\n> +def MethodParameters(method):\n> + params = []\n> + for param in method.parameters:\n> + params.append('const %s %s%s' % (GetNameForElement(param),\n> + '&' if not IsPod(param) else '',\n> + param.mojom_name))\n> + if MethodReturnValue(method) == 'void':\n> + if method.response_parameters is None:\n> + return params\n> + for param in method.response_parameters:\n> + params.append(f'{GetNameForElement(param)} *{param.mojom_name}')\n> + return params\n> +\n> +def MethodReturnValue(method):\n> + if method.response_parameters is None:\n> + return 'void'\n> + if len(method.response_parameters) == 1 and IsPod(method.response_parameters[0]):\n> + return GetNameForElement(method.response_parameters[0])\n> + return 'void'\n> +\n> +def IsAsync(method):\n> + # Events are always async\n> + if re.match(\"^IPA.*EventInterface$\", method.interface.mojom_name):\n> + return True\n> + elif re.match(\"^IPA.*Interface$\", method.interface.mojom_name):\n> + if method.attributes is None:\n> + return False\n> + elif 'async' in method.attributes and method.attributes['async']:\n> + return True\n> + return False\n> +\n> +def IsArray(element):\n> + return mojom.IsArrayKind(element.kind)\n> +\n> +def IsControls(element):\n> + return mojom.IsStructKind(element.kind) and (element.kind.mojom_name == \"ControlList\" or\n> + element.kind.mojom_name == \"ControlInfoMap\")\n> +\n> +def IsEnum(element):\n> + return mojom.IsEnumKind(element.kind)\n> +\n> +def IsFd(element):\n> + return mojom.IsStructKind(element.kind) and element.kind.mojom_name == \"FileDescriptor\"\n> +\n> +def IsMap(element):\n> + return mojom.IsMapKind(element.kind)\n> +\n> +def IsPlainStruct(element):\n> + return mojom.IsStructKind(element.kind) and not IsControls(element) and not IsFd(element)\n> +\n> +def IsPod(element):\n> + return element.kind in _kind_to_cpp_type\n> +\n> +def IsStr(element):\n> + return element.kind.spec == 's'\n> +\n> +def BitWidth(element):\n> + if element.kind in _bit_widths:\n> + return _bit_widths[element.kind]\n> + if mojom.IsEnumKind(element.kind):\n> + return '32'\n> + return ''\n> +\n> +# Get the type name for a given element\n> +def GetNameForElement(element):\n> + # structs\n> + if (mojom.IsEnumKind(element) or\n> + mojom.IsInterfaceKind(element) or\n> + mojom.IsStructKind(element)):\n> + return element.mojom_name\n> + # vectors\n> + if (mojom.IsArrayKind(element)):\n> + elem_name = GetNameForElement(element.kind)\n> + return f'std::vector<{elem_name}>'\n> + # maps\n> + if (mojom.IsMapKind(element)):\n> + key_name = GetNameForElement(element.key_kind)\n> + value_name = GetNameForElement(element.value_kind)\n> + return f'std::map<{key_name}, {value_name}>'\n> + # struct fields and function parameters\n> + if isinstance(element, (mojom.Field, mojom.Method, mojom.Parameter)):\n> + # maps and vectors\n> + if (mojom.IsArrayKind(element.kind) or mojom.IsMapKind(element.kind)):\n> + return GetNameForElement(element.kind)\n> + # strings\n> + if (mojom.IsReferenceKind(element.kind) and element.kind.spec == 's'):\n> + return 'std::string'\n> + # PODs\n> + if element.kind in _kind_to_cpp_type:\n> + return _kind_to_cpp_type[element.kind]\n> + # structs and enums\n> + return element.kind.mojom_name\n> + # PODs that are members of vectors/maps\n> + if (hasattr(element, '__hash__') and element in _kind_to_cpp_type):\n> + return _kind_to_cpp_type[element]\n> + if (hasattr(element, 'spec')):\n> + # strings that are members of vectors/maps\n> + if (element.spec == 's'):\n> + return 'std::string'\n> + # structs that aren't defined in mojom that are members of vectors/maps\n> + if (element.spec[0] == 'x'):\n> + return element.spec.replace('x:', '').replace('.', '::')\n> + if (mojom.IsInterfaceRequestKind(element) or\n> + mojom.IsAssociatedKind(element) or\n> + mojom.IsPendingRemoteKind(element) or\n> + mojom.IsPendingReceiverKind(element) or\n> + mojom.IsUnionKind(element)):\n> + raise Exception('Unsupported element: %s' % element)\n> + raise Exception('Unexpected element: %s' % element)\n> +\n> +def GetFullNameForElement(element, namespace_str):\n> + name = GetNameForElement(element)\n> + if namespace_str == '':\n> + return name\n> + return f'{namespace_str}::{name}'\n> +\n> +def ValidateZeroLength(l, s, cap=True):\n> + if l is None:\n> + return\n> + if len(l) > 0:\n> + raise Exception(f'{s.capitalize() if cap else s} should be empty')\n> +\n> +def ValidateSingleLength(l, s, cap=True):\n> + if len(l) > 1:\n> + raise Exception(f'Only one {s} allowed')\n> + if len(l) < 1:\n> + raise Exception(f'{s.capitalize() if cap else s} is required')\n> +\n> +def GetMainInterface(interfaces):\n> + intf = [x for x in interfaces\n> + if re.match(\"^IPA.*Interface\", x.mojom_name) and\n> + not re.match(\"^IPA.*EventInterface\", x.mojom_name)]\n> + ValidateSingleLength(intf, 'main interface')\n> + return None if len(intf) == 0 else intf[0]\n> +\n> +def GetEventInterface(interfaces):\n> + event = [x for x in interfaces if re.match(\"^IPA.*EventInterface\", x.mojom_name)]\n> + ValidateSingleLength(event, 'event interface')\n> + return None if len(event) == 0 else event[0]\n> +\n> +def ValidateNamespace(namespace):\n> + if namespace == '':\n> + raise Exception('Must have a namespace')\n> +\n> + if not re.match('^ipa\\.[0-9A-Za-z_]+', namespace):\n> + raise Exception('Namespace must be of the form \"ipa.{pipeline_name}\"')\n> +\n> +def ValidateInterfaces(interfaces):\n> + # Validate presence of main interface\n> + intf = GetMainInterface(interfaces)\n> + if intf is None:\n> + raise Exception('Must have main IPA interface')\n> +\n> + # Validate presence of event interface\n> + event = GetEventInterface(interfaces)\n> + if intf is None:\n> + raise Exception('Must have event IPA interface')\n> +\n> + # Validate required main interface functions\n> + f_init = [x for x in intf.methods if x.mojom_name == 'init']\n> + f_start = [x for x in intf.methods if x.mojom_name == 'start']\n> + f_stop = [x for x in intf.methods if x.mojom_name == 'stop']\n> +\n> + ValidateSingleLength(f_init, 'init()', False)\n> + ValidateSingleLength(f_start, 'start()', False)\n> + ValidateSingleLength(f_stop, 'stop()', False)\n> +\n> + f_init = f_init[0]\n> + f_start = f_start[0]\n> + f_stop = f_stop[0]\n> +\n> + # Validate parameters to init()\n> + ValidateSingleLength(f_init.parameters, 'input parameter to init()')\n> + ValidateSingleLength(f_init.response_parameters, 'output parameter from init()')\n> + if f_init.parameters[0].kind.mojom_name != 'IPASettings':\n> + raise Exception('init() must have single IPASettings input parameter')\n> + if f_init.response_parameters[0].kind.spec != 'i32':\n> + raise Exception('init() must have single int32 output parameter')\n> +\n> + # No need to validate start() as it is customizable\n> +\n> + # Validate parameters to stop()\n> + ValidateZeroLength(f_stop.parameters, 'input parameter to stop()')\n> + ValidateZeroLength(f_stop.parameters, 'output parameter from stop()')\n> +\n> + # Validate that event interface has at least one event\n> + if len(event.methods) < 1:\n> + raise Exception('Event interface must have at least one event')\n> +\n> + # Validate that all async methods don't have return values\n> + intf_methods_async = [x for x in intf.methods if IsAsync(x)]\n> + for method in intf_methods_async:\n> + ValidateZeroLength(method.response_parameters,\n> + f'{method.mojom_name} response parameters', False)\n> +\n> + event_methods_async = [x for x in event.methods if IsAsync(x)]\n> + for method in event_methods_async:\n> + ValidateZeroLength(method.response_parameters,\n> + f'{method.mojom_name} response parameters', False)\n> +\n> +class Generator(generator.Generator):\n> + @staticmethod\n> + def GetTemplatePrefix():\n> + return 'libcamera_templates'\n> +\n> + def GetFilters(self):\n> + libcamera_filters = {\n> + 'all_types': GetAllTypes,\n> + 'bit_width': BitWidth,\n> + 'cap': Capitalize,\n> + 'choose': Choose,\n> + 'comma_sep': CommaSep,\n> + 'default_value': GetDefaultValue,\n> + 'has_default_fields': HasDefaultFields,\n> + 'has_fd': HasFd,\n> + 'is_async': IsAsync,\n> + 'is_array': IsArray,\n> + 'is_controls': IsControls,\n> + 'is_enum': IsEnum,\n> + 'is_fd': IsFd,\n> + 'is_map': IsMap,\n> + 'is_plain_struct': IsPlainStruct,\n> + 'is_pod': IsPod,\n> + 'is_str': IsStr,\n> + 'method_input_has_fd': MethodInputHasFd,\n> + 'method_output_has_fd': MethodOutputHasFd,\n> + 'method_param_names': MethodParamNames,\n> + 'method_param_inputs': MethodParamInputs,\n> + 'method_param_outputs': MethodParamOutputs,\n> + 'method_parameters': MethodParameters,\n> + 'method_return_value': MethodReturnValue,\n> + 'name': GetNameForElement,\n> + 'name_full': GetFullNameForElement,\n> + 'needs_control_serializer': NeedsControlSerializer,\n> + 'params_comma_sep': ParamsCommaSep,\n> + 'with_default_values': WithDefaultValues,\n> + 'with_fds': WithFds,\n> + }\n> + return libcamera_filters\n> +\n> + def _GetJinjaExports(self):\n> + return {\n> + 'cmd_enum_name': '_%sCmd' % self.module_name,\n> + 'cmd_event_enum_name': '_%sEventCmd' % self.module_name,\n> + 'consts': self.module.constants,\n> + 'enums': self.module.enums,\n> + 'has_array': len([x for x in self.module.kinds.keys() if x[0] == 'a']) > 0,\n> + 'has_map': len([x for x in self.module.kinds.keys() if x[0] == 'm']) > 0,\n> + 'has_namespace': self.module.mojom_namespace != '',\n> + 'interface_event': GetEventInterface(self.module.interfaces),\n> + 'interface_main': GetMainInterface(self.module.interfaces),\n> + 'interface_name': 'IPA%sInterface' % self.module_name,\n> + 'module_name': ModuleName(self.module.path),\n> + 'namespace': self.module.mojom_namespace.split('.'),\n> + 'namespace_str': self.module.mojom_namespace.replace('.', '::') if\n> + self.module.mojom_namespace is not None else '',\n> + 'proxy_name': 'IPAProxy%s' % self.module_name,\n> + 'proxy_worker_name': 'IPAProxy%sWorker' % self.module_name,\n> + 'structs_nonempty': [x for x in self.module.structs if len(x.fields) > 0],\n> + }\n> +\n> + def _GetJinjaExportsForCore(self):\n> + return {\n> + 'consts': self.module.constants,\n> + 'enums': self.module.enums,\n> + 'has_array': len([x for x in self.module.kinds.keys() if x[0] == 'a']) > 0,\n> + 'has_map': len([x for x in self.module.kinds.keys() if x[0] == 'm']) > 0,\n> + 'structs_gen_header': [x for x in self.module.structs if x.attributes is not None and 'genHeader' in x.attributes],\n> + 'structs_gen_serializer': [x for x in self.module.structs if x.attributes is not None and 'genSerdes' in x.attributes],\n> + }\n> +\n> + @UseJinja('core_ipa_interface.h.tmpl')\n> + def _GenerateCoreHeader(self):\n> + return self._GetJinjaExportsForCore()\n> +\n> + @UseJinja('core_ipa_serializer.h.tmpl')\n> + def _GenerateCoreSerializer(self):\n> + return self._GetJinjaExportsForCore()\n> +\n> + @UseJinja('module_ipa_interface.h.tmpl')\n> + def _GenerateDataHeader(self):\n> + return self._GetJinjaExports()\n> +\n> + @UseJinja('module_ipa_serializer.h.tmpl')\n> + def _GenerateSerializer(self):\n> + return self._GetJinjaExports()\n> +\n> + @UseJinja('module_ipa_proxy.cpp.tmpl')\n> + def _GenerateProxyCpp(self):\n> + return self._GetJinjaExports()\n> +\n> + @UseJinja('module_ipa_proxy.h.tmpl')\n> + def _GenerateProxyHeader(self):\n> + return self._GetJinjaExports()\n> +\n> + @UseJinja('module_ipa_proxy_worker.cpp.tmpl')\n> + def _GenerateProxyWorker(self):\n> + return self._GetJinjaExports()\n> +\n> + def GenerateFiles(self, unparsed_args):\n> + parser = argparse.ArgumentParser()\n> + parser.add_argument('--libcamera_generate_core_header', action='store_true')\n> + parser.add_argument('--libcamera_generate_core_serializer', action='store_true')\n> + parser.add_argument('--libcamera_generate_header', action='store_true')\n> + parser.add_argument('--libcamera_generate_serializer', action='store_true')\n> + parser.add_argument('--libcamera_generate_proxy_cpp', action='store_true')\n> + parser.add_argument('--libcamera_generate_proxy_h', action='store_true')\n> + parser.add_argument('--libcamera_generate_proxy_worker', action='store_true')\n> + parser.add_argument('--libcamera_output_path')\n> + args = parser.parse_args(unparsed_args)\n> +\n> + if not args.libcamera_generate_core_header and \\\n> + not args.libcamera_generate_core_serializer:\n> + ValidateNamespace(self.module.mojom_namespace)\n> + ValidateInterfaces(self.module.interfaces)\n> + self.module_name = ModuleClassName(self.module)\n> +\n> + fileutil.EnsureDirectoryExists(os.path.dirname(args.libcamera_output_path))\n> +\n> + gen_funcs = [\n> + [args.libcamera_generate_core_header, self._GenerateCoreHeader],\n> + [args.libcamera_generate_core_serializer, self._GenerateCoreSerializer],\n> + [args.libcamera_generate_header, self._GenerateDataHeader],\n> + [args.libcamera_generate_serializer, self._GenerateSerializer],\n> + [args.libcamera_generate_proxy_cpp, self._GenerateProxyCpp],\n> + [args.libcamera_generate_proxy_h, self._GenerateProxyHeader],\n> + [args.libcamera_generate_proxy_worker, self._GenerateProxyWorker],\n> + ]\n> +\n> + for pair in gen_funcs:\n> + if pair[0]:\n> + self.Write(pair[1](), args.libcamera_output_path)\n>","headers":{"Return-Path":"","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id 53DBFBD80C\n\tfor ;\n\tMon, 11 Jan 2021 06:24:53 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id B93F8680A5;\n\tMon, 11 Jan 2021 07:24:52 +0100 (CET)","from perceval.ideasonboard.com (perceval.ideasonboard.com\n\t[213.167.242.64])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id 8D97960317\n\tfor ;\n\tMon, 11 Jan 2021 07:24:51 +0100 (CET)","from [192.168.0.20]\n\t(cpc89244-aztw30-2-0-cust3082.18-1.cable.virginm.net [86.31.172.11])\n\tby perceval.ideasonboard.com (Postfix) with ESMTPSA id 19DD9EC;\n\tMon, 11 Jan 2021 07:24:51 +0100 (CET)"],"Authentication-Results":"lancelot.ideasonboard.com;\n\tdkim=fail reason=\"signature verification failed\" (1024-bit key;\n\tunprotected) header.d=ideasonboard.com header.i=@ideasonboard.com\n\theader.b=\"bafHffBh\"; dkim-atps=neutral","DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com;\n\ts=mail; t=1610346291;\n\tbh=yf3ekDTdiJR6r/UdIOvylL7GINk4cMQD3F0dngqj3jE=;\n\th=Reply-To:Subject:To:References:From:Date:In-Reply-To:From;\n\tb=bafHffBhHIgpffy+nwHcKNzGv8eNiEBmZAcAwzCh4k3jKWDB5Ypsj7CXPNngUsnyN\n\twGHHv41gXQyY1Vio8eX6mCqvdFPOlrjeHWNWQKsDHRglHkkT6r9cP7oGZOOnZ+u2Ux\n\tCqN+iuSox4UvBAcCL6lkzJFH6YZdGFU8z3Yx9Jmk=","To":"Paul Elder ,\n\tlibcamera-devel@lists.libcamera.org","References":"<20201224081534.41601-1-paul.elder@ideasonboard.com>\n\t<20201224081534.41601-3-paul.elder@ideasonboard.com>","From":"Kieran Bingham ","Autocrypt":"addr=kieran.bingham@ideasonboard.com; keydata=\n\tmQINBFYE/WYBEACs1PwjMD9rgCu1hlIiUA1AXR4rv2v+BCLUq//vrX5S5bjzxKAryRf0uHat\n\tV/zwz6hiDrZuHUACDB7X8OaQcwhLaVlq6byfoBr25+hbZG7G3+5EUl9cQ7dQEdvNj6V6y/SC\n\trRanWfelwQThCHckbobWiQJfK9n7rYNcPMq9B8e9F020LFH7Kj6YmO95ewJGgLm+idg1Kb3C\n\tpotzWkXc1xmPzcQ1fvQMOfMwdS+4SNw4rY9f07Xb2K99rjMwZVDgESKIzhsDB5GY465sCsiQ\n\tcSAZRxqE49RTBq2+EQsbrQpIc8XiffAB8qexh5/QPzCmR4kJgCGeHIXBtgRj+nIkCJPZvZtf\n\tKr2EAbc6tgg6DkAEHJb+1okosV09+0+TXywYvtEop/WUOWQ+zo+Y/OBd+8Ptgt1pDRyOBzL8\n\tRXa8ZqRf0Mwg75D+dKntZeJHzPRJyrlfQokngAAs4PaFt6UfS+ypMAF37T6CeDArQC41V3ko\n\tlPn1yMsVD0p+6i3DPvA/GPIksDC4owjnzVX9kM8Zc5Cx+XoAN0w5Eqo4t6qEVbuettxx55gq\n\t8K8FieAjgjMSxngo/HST8TpFeqI5nVeq0/lqtBRQKumuIqDg+Bkr4L1V/PSB6XgQcOdhtd36\n\tOe9X9dXB8YSNt7VjOcO7BTmFn/Z8r92mSAfHXpb07YJWJosQOQARAQABtDBLaWVyYW4gQmlu\n\tZ2hhbSA8a2llcmFuLmJpbmdoYW1AaWRlYXNvbmJvYXJkLmNvbT6JAlcEEwEKAEECGwMFCwkI\n\tBwIGFQgJCgsCBBYCAwECHgECF4ACGQEWIQSQLdeYP70o/eNy1HqhHkZyEKRh/QUCXWTtygUJ\n\tCyJXZAAKCRChHkZyEKRh/f8dEACTDsbLN2nioNZMwyLuQRUAFcXNolDX48xcUXsWS2QjxaPm\n\tVsJx8Uy8aYkS85mdPBh0C83OovQR/OVbr8AxhGvYqBs3nQvbWuTl/+4od7DfK2VZOoKBAu5S\n\tQK2FYuUcikDqYcFWJ8DQnubxfE8dvzojHEkXw0sA4igINHDDFX3HJGZtLio+WpEFQtCbfTAG\n\tYZslasz1YZRbwEdSsmO3/kqy5eMnczlm8a21A3fKUo3g8oAZEFM+f4DUNzqIltg31OAB/kZS\n\tenKZQ/SWC8PmLg/ZXBrReYakxXtkP6w3FwMlzOlhGxqhIRNiAJfXJBaRhuUWzPOpEDE9q5YJ\n\tBmqQL2WJm1VSNNVxbXJHpaWMH1sA2R00vmvRrPXGwyIO0IPYeUYQa3gsy6k+En/aMQJd27dp\n\taScf9am9PFICPY5T4ppneeJLif2lyLojo0mcHOV+uyrds9XkLpp14GfTkeKPdPMrLLTsHRfH\n\tfA4I4OBpRrEPiGIZB/0im98MkGY/Mu6qxeZmYLCcgD6qz4idOvfgVOrNh+aA8HzIVR+RMW8H\n\tQGBN9f0E3kfwxuhl3omo6V7lDw8XOdmuWZNC9zPq1UfryVHANYbLGz9KJ4Aw6M+OgBC2JpkD\n\thXMdHUkC+d20dwXrwHTlrJi1YNp6rBc+xald3wsUPOZ5z8moTHUX/uPA/qhGsbkCDQRWBP1m\n\tARAAzijkb+Sau4hAncr1JjOY+KyFEdUNxRy+hqTJdJfaYihxyaj0Ee0P0zEi35CbE6lgU0Uz\n\ttih9fiUbSV3wfsWqg1Ut3/5rTKu7kLFp15kF7eqvV4uezXRD3Qu4yjv/rMmEJbbD4cTvGCYI\n\td6MDC417f7vK3hCbCVIZSp3GXxyC1LU+UQr3fFcOyCwmP9vDUR9JV0BSqHHxRDdpUXE26Dk6\n\tmhf0V1YkspE5St814ETXpEus2urZE5yJIUROlWPIL+hm3NEWfAP06vsQUyLvr/GtbOT79vXl\n\tEn1aulcYyu20dRRxhkQ6iILaURcxIAVJJKPi8dsoMnS8pB0QW12AHWuirPF0g6DiuUfPmrA5\n\tPKe56IGlpkjc8cO51lIxHkWTpCMWigRdPDexKX+Sb+W9QWK/0JjIc4t3KBaiG8O4yRX8ml2R\n\t+rxfAVKM6V769P/hWoRGdgUMgYHFpHGSgEt80OKK5HeUPy2cngDUXzwrqiM5Sz6Od0qw5pCk\n\tNlXqI0W/who0iSVM+8+RmyY0OEkxEcci7rRLsGnM15B5PjLJjh1f2ULYkv8s4SnDwMZ/kE04\n\t/UqCMK/KnX8pwXEMCjz0h6qWNpGwJ0/tYIgQJZh6bqkvBrDogAvuhf60Sogw+mH8b+PBlx1L\n\toeTK396wc+4c3BfiC6pNtUS5GpsPMMjYMk7kVvEAEQEAAYkCPAQYAQoAJgIbDBYhBJAt15g/\n\tvSj943LUeqEeRnIQpGH9BQJdizzIBQkLSKZiAAoJEKEeRnIQpGH9eYgQAJpjaWNgqNOnMTmD\n\tMJggbwjIotypzIXfhHNCeTkG7+qCDlSaBPclcPGYrTwCt0YWPU2TgGgJrVhYT20ierN8LUvj\n\t6qOPTd+Uk7NFzL65qkh80ZKNBFddx1AabQpSVQKbdcLb8OFs85kuSvFdgqZwgxA1vl4TFhNz\n\tPZ79NAmXLackAx3sOVFhk4WQaKRshCB7cSl+RIng5S/ThOBlwNlcKG7j7W2MC06BlTbdEkUp\n\tECzuuRBv8wX4OQl+hbWbB/VKIx5HKlLu1eypen/5lNVzSqMMIYkkZcjV2SWQyUGxSwq0O/sx\n\tS0A8/atCHUXOboUsn54qdxrVDaK+6jIAuo8JiRWctP16KjzUM7MO0/+4zllM8EY57rXrj48j\n\tsbEYX0YQnzaj+jO6kJtoZsIaYR7rMMq9aUAjyiaEZpmP1qF/2sYenDx0Fg2BSlLvLvXM0vU8\n\tpQk3kgDu7kb/7PRYrZvBsr21EIQoIjXbZxDz/o7z95frkP71EaICttZ6k9q5oxxA5WC6sTXc\n\tMW8zs8avFNuA9VpXt0YupJd2ijtZy2mpZNG02fFVXhIn4G807G7+9mhuC4XG5rKlBBUXTvPU\n\tAfYnB4JBDLmLzBFavQfvonSfbitgXwCG3vS+9HEwAjU30Bar1PEOmIbiAoMzuKeRm2LVpmq4\n\tWZw01QYHU/GUV/zHJSFk","Organization":"Ideas on Board","Message-ID":"<3ce53459-3c34-710a-b4e3-01cc4ff6e8f3@ideasonboard.com>","Date":"Mon, 11 Jan 2021 06:24:48 +0000","User-Agent":"Mozilla/5.0 (X11; Linux x86_64; rv:68.0) Gecko/20100101\n\tThunderbird/68.10.0","MIME-Version":"1.0","In-Reply-To":"<20201224081534.41601-3-paul.elder@ideasonboard.com>","Content-Language":"en-GB","Subject":"Re: [libcamera-devel] [PATCH v6 2/9] utils: ipc: add templates for\n\tcode generation for IPC mechanism","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Reply-To":"kieran.bingham@ideasonboard.com","Content-Type":"text/plain; charset=\"us-ascii\"","Content-Transfer-Encoding":"7bit","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" "}},{"id":14546,"web_url":"https://patchwork.libcamera.org/comment/14546/","msgid":"<20210115051141.GA1885@pyrite.rasen.tech>","date":"2021-01-15T05:11:41","subject":"Re: [libcamera-devel] [PATCH v6 2/9] utils: ipc: add templates for\n\tcode generation for IPC mechanism","submitter":{"id":17,"url":"https://patchwork.libcamera.org/api/people/17/","name":"Paul Elder","email":"paul.elder@ideasonboard.com"},"content":"Hi Kieran,\n\n\n\n> > diff --git a/utils/ipc/generators/libcamera_templates/serializer.tmpl b/utils/ipc/generators/libcamera_templates/serializer.tmpl\n> > new file mode 100644\n> > index 00000000..af4800bf\n> > --- /dev/null\n> > +++ b/utils/ipc/generators/libcamera_templates/serializer.tmpl\n> > @@ -0,0 +1,313 @@\n> > +{#-\n> > + # SPDX-License-Identifier: LGPL-2.1-or-later\n> > + # Copyright (C) 2020, Google Inc.\n> > +-#}\n> > +{# Turn this into a C macro? #}\n> \n> Is this a todo?\n\nIt was more of a question,\n\n> \n> \n> > +{#\n> > + # \\brief Verify that there is enough bytes to deserialize\n> > + #\n> > + # Generate code that verifies that \\a size is not greater than \\a dataSize.\n> > + # Otherwise log an error with \\a name and \\a typename.\n> > + #}\n> > +{%- macro check_data_size(size, dataSize, name, typename) %}\n> > +\t\tif ({{dataSize}} < {{size}}) {\n> > +\t\t\tLOG(IPADataSerializer, Error)\n> > +\t\t\t\t<< \"Failed to deserialize \" << \"{{name}}\"\n> > +\t\t\t\t<< \": not enough {{typename}}, expected \"\n> > +\t\t\t\t<< ({{size}}) << \", got \" << ({{dataSize}});\n> > +\t\t\treturn ret;\n> > +\t\t}\n> > +{%- endmacro %}\n\nbecause this block is kind of big, so I wasn't sure if this shold be\ngenerated at every check, or if this should be a macro and put the macro\nat every generated check. In either case it's generated.\n\n> > +\n> > +\n> > +{#\n> > + # \\brief Serialize some field into return vector\n> \n> 'some' field?\n> a field?\n\nMathematics vocabulary :p\n\nYeah, I'll change it.\n\n> > + #\n> > + # Generate code to serialize \\a field into retData, including size of the\n> > + # field and fds (where appropriate).\n> > + # This code is meant to be used by the IPADataSerializer specialization.\n> > + #\n> > + # \\todo Avoid intermediate vectors\n> > + #}\n> > +{%- macro serializer_field(field, namespace, loop) %}\n> > +{%- if field|is_pod or field|is_enum %}\n> > +\t\tstd::vector {{field.mojom_name}};\n> > +\t\tstd::tie({{field.mojom_name}}, std::ignore) =\n> > +\t{%- if field|is_pod %}\n> > +\t\t\tIPADataSerializer<{{field|name}}>::serialize(data.{{field.mojom_name}});\n> > +\t{%- elif field|is_enum %}\n> > +\t\t\tIPADataSerializer::serialize(data.{{field.mojom_name}});\n> > +\t{%- endif %}\n> > +\t\tretData.insert(retData.end(), {{field.mojom_name}}.begin(), {{field.mojom_name}}.end());\n> > +{%- elif field|is_fd %}\n> > +\t\tstd::vector {{field.mojom_name}};\n> > +\t\tstd::vector {{field.mojom_name}}Fds;\n> > +\t\tstd::tie({{field.mojom_name}}, {{field.mojom_name}}Fds) =\n> > +\t\t\tIPADataSerializer<{{field|name}}>::serialize(data.{{field.mojom_name}});\n> > +\t\tretData.insert(retData.end(), {{field.mojom_name}}.begin(), {{field.mojom_name}}.end());\n> > +\t\tretFds.insert(retFds.end(), {{field.mojom_name}}Fds.begin(), {{field.mojom_name}}Fds.end());\n> > +{%- elif field|is_controls %}\n> > +\t\tif (data.{{field.mojom_name}}.size() > 0) {\n> > +\t\t\tstd::vector {{field.mojom_name}};\n> > +\t\t\tstd::tie({{field.mojom_name}}, std::ignore) =\n> > +\t\t\t\tIPADataSerializer<{{field|name}}>::serialize(data.{{field.mojom_name}}, cs);\n> > +\t\t\tappendPOD(retData, {{field.mojom_name}}.size());\n> > +\t\t\tretData.insert(retData.end(), {{field.mojom_name}}.begin(), {{field.mojom_name}}.end());\n> > +\t\t} else {\n> > +\t\t\tappendPOD(retData, 0);\n> > +\t\t}\n> > +{%- elif field|is_plain_struct or field|is_array or field|is_map or field|is_str %}\n> > +\t\tstd::vector {{field.mojom_name}};\n> > +\t{%- if field|has_fd %}\n> > +\t\tstd::vector {{field.mojom_name}}Fds;\n> > +\t\tstd::tie({{field.mojom_name}}, {{field.mojom_name}}Fds) =\n> > +\t{%- else %}\n> > +\t\tstd::tie({{field.mojom_name}}, std::ignore) =\n> > +\t{%- endif %}\n> > +\t{%- if field|is_array or field|is_map %}\n> > +\t\t\tIPADataSerializer<{{field|name}}>::serialize(data.{{field.mojom_name}}, cs);\n> > +\t{%- elif field|is_str %}\n> > +\t\t\tIPADataSerializer<{{field|name}}>::serialize(data.{{field.mojom_name}});\n> > +\t{%- else %}\n> > +\t\t\tIPADataSerializer<{{field|name_full(namespace)}}>::serialize(data.{{field.mojom_name}}, cs);\n> > +\t{%- endif %}\n> > +\t\tappendPOD(retData, {{field.mojom_name}}.size());\n> > +\t{%- if field|has_fd %}\n> > +\t\tappendPOD(retData, {{field.mojom_name}}Fds.size());\n> > +\t{%- endif %}\n> > +\t\tretData.insert(retData.end(), {{field.mojom_name}}.begin(), {{field.mojom_name}}.end());\n> > +\t{%- if field|has_fd %}\n> > +\t\tretFds.insert(retFds.end(), {{field.mojom_name}}Fds.begin(), {{field.mojom_name}}Fds.end());\n> > +\t{%- endif %}\n> > +{%- else %}\n> > +\t\t/* Unknown serialization for {{field.mojom_name}}. */\n> > +{%- endif %}\n> > +{%- endmacro %}\n> > +\n> > +\n> > +{#\n> > + # \\brief Deserialize some field into return struct\n> \n> Again, 'some field' sounds like a random field.\n> Presumably this is going to handle a quite specific one each time...\n> \n> \n> > + #\n> > + # Generate code to deserialize \\a field into object ret.\n> > + # This code is meant to be used by the IPADataSerializer specialization.\n> > + #}\n> > +{%- macro deserializer_field(field, namespace, loop) %}\n> > +{% if field|is_pod or field|is_enum %}\n> > +\t{%- set field_size = (field|bit_width|int / 8)|int %}\n> > +\t\t{{- check_data_size(field_size, 'dataSize', field.mojom_name, 'data')}}\n> > +\t\t{%- if field|is_pod %}\n> > +\t\tret.{{field.mojom_name}} = IPADataSerializer<{{field|name}}>::deserialize(m, m + {{field_size}});\n> > +\t\t{%- else %}\n> > +\t\tret.{{field.mojom_name}} = static_cast<{{field|name_full(namespace)}}>(IPADataSerializer::deserialize(m, m + {{field_size}}));\n> > +\t\t{%- endif %}\n> > +\t{%- if not loop.last %}\n> > +\t\tm += {{field_size}};\n> > +\t\tdataSize -= {{field_size}};\n> > +\t{%- endif %}\n> > +{% elif field|is_fd %}\n> > +\t{%- set field_size = 1 %}\n> > +\t\t{{- check_data_size(field_size, 'dataSize', field.mojom_name, 'data')}}\n> > +\t\tret.{{field.mojom_name}} = IPADataSerializer<{{field|name}}>::deserialize(m, m + 1, n, n + 1, cs);\n> > +\t{%- if not loop.last %}\n> > +\t\tm += {{field_size}};\n> > +\t\tdataSize -= {{field_size}};\n> > +\t\tn += ret.{{field.mojom_name}}.isValid() ? 1 : 0;\n> > +\t\tfdsSize -= ret.{{field.mojom_name}}.isValid() ? 1 : 0;\n> > +\t{%- endif %}\n> > +{% elif field|is_controls %}\n> > +\t{%- set field_size = 4 %}\n> > +\t\t{{- check_data_size(field_size, 'dataSize', field.mojom_name + 'Size', 'data')}}\n> > +\t\tconst size_t {{field.mojom_name}}Size = readPOD(m, 0, dataEnd);\n> > +\t\tm += {{field_size}};\n> > +\t\tdataSize -= {{field_size}};\n> > +\t{%- set field_size = field.mojom_name + 'Size' -%}\n> > +\t\t{{- check_data_size(field_size, 'dataSize', field.mojom_name, 'data')}}\n> > +\t\tif ({{field.mojom_name}}Size > 0)\n> > +\t\t\tret.{{field.mojom_name}} =\n> > +\t\t\t\tIPADataSerializer<{{field|name}}>::deserialize(m, m + {{field.mojom_name}}Size, cs);\n> > +\t{%- if not loop.last %}\n> > +\t\tm += {{field_size}};\n> > +\t\tdataSize -= {{field_size}};\n> > +\t{%- endif %}\n> > +{% elif field|is_plain_struct or field|is_array or field|is_map or field|is_str %}\n> > +\t{%- set field_size = 4 %}\n> > +\t\t{{- check_data_size(field_size, 'dataSize', field.mojom_name + 'Size', 'data')}}\n> > +\t\tconst size_t {{field.mojom_name}}Size = readPOD(m, 0, dataEnd);\n> > +\t\tm += {{field_size}};\n> > +\t\tdataSize -= {{field_size}};\n> > +\t{%- if field|has_fd %}\n> > +\t{%- set field_size = 4 %}\n> > +\t\t{{- check_data_size(field_size, 'dataSize', field.mojom_name + 'FdsSize', 'data')}}\n> > +\t\tconst size_t {{field.mojom_name}}FdsSize = readPOD(m, 0, dataEnd);\n> > +\t\tm += {{field_size}};\n> > +\t\tdataSize -= {{field_size}};\n> > +\t\t{{- check_data_size(field.mojom_name + 'FdsSize', 'fdsSize', field.mojom_name, 'fds')}}\n> > +\t{%- endif %}\n> > +\t{%- set field_size = field.mojom_name + 'Size' -%}\n> > +\t\t{{- check_data_size(field_size, 'dataSize', field.mojom_name, 'data')}}\n> > +\t\tret.{{field.mojom_name}} =\n> > +\t{%- if field|is_str %}\n> > +\t\t\tIPADataSerializer<{{field|name}}>::deserialize(m, m + {{field.mojom_name}}Size);\n> > +\t{%- elif field|has_fd and (field|is_array or field|is_map) %}\n> > +\t\t\tIPADataSerializer<{{field|name}}>::deserialize(m, m + {{field.mojom_name}}Size, n, n + {{field.mojom_name}}FdsSize, cs);\n> > +\t{%- elif field|has_fd and (not (field|is_array or field|is_map)) %}\n> > +\t\t\tIPADataSerializer<{{field|name_full(namespace)}}>::deserialize(m, m + {{field.mojom_name}}Size, n, n + {{field.mojom_name}}FdsSize, cs);\n> > +\t{%- elif (not field|has_fd) and (field|is_array or field|is_map) %}\n> > +\t\t\tIPADataSerializer<{{field|name}}>::deserialize(m, m + {{field.mojom_name}}Size, cs);\n> > +\t{%- else %}\n> > +\t\t\tIPADataSerializer<{{field|name_full(namespace)}}>::deserialize(m, m + {{field.mojom_name}}Size, cs);\n> > +\t{%- endif %}\n> > +\t{%- if not loop.last %}\n> > +\t\tm += {{field_size}};\n> > +\t\tdataSize -= {{field_size}};\n> > +\t{%- if field|has_fd %}\n> > +\t\tn += {{field.mojom_name}}FdsSize;\n> > +\t\tfdsSize -= {{field.mojom_name}}FdsSize;\n> > +\t{%- endif %}\n> > +\t{%- endif %}\n> > +{% else %}\n> > +\t\t/* Unknown deserialization for {{field.mojom_name}}. */\n> > +{%- endif %}\n> > +{%- endmacro %}\n> > +\n> > +\n> > +{#\n> > + # \\brief Serialize a struct\n> > + #\n> > + # Generate code for IPADataSerializer specialization, for serializing\n> > + # \\a struct.\n> > + #}\n> > +{%- macro serializer(struct, namespace) %}\n> > +\tstatic std::tuple, std::vector>\n> > +\tserialize(const {{struct|name_full(namespace)}} &data,\n> > +{%- if struct|needs_control_serializer %}\n> > +\t\t ControlSerializer *cs)\n> > +{%- else %}\n> > +\t\t [[maybe_unused]] ControlSerializer *cs = nullptr)\n> > +{%- endif %}\n> > +\t{\n> > +\t\tstd::vector retData;\n> > +{%- if struct|has_fd %}\n> > +\t\tstd::vector retFds;\n> > +{%- endif %}\n> > +{%- for field in struct.fields %}\n> > +{{serializer_field(field, namespace, loop)}}\n> > +{%- endfor %}\n> > +{% if struct|has_fd %}\n> > +\t\treturn {retData, retFds};\n> > +{%- else %}\n> > +\t\treturn {retData, {}};\n> > +{%- endif %}\n> > +\t}\n> > +{%- endmacro %}\n> > +\n> > +\n> > +{#\n> > + # \\brief Deserialize a struct that has fds\n> > + #\n> > + # Generate code for IPADataSerializer specialization, for deserializing\n> > + # \\a struct, in the case that \\a struct has file descriptors.\n> > + # fd parameters\n> > + #}\n> > +{%- macro deserializer_fd(struct, namespace) %}\n> > +\tstatic {{struct|name_full(namespace)}}\n> > +\tdeserialize(std::vector &data,\n> > +\t\t std::vector &fds,\n> > +{%- if struct|needs_control_serializer %}\n> > +\t\t ControlSerializer *cs)\n> > +{%- else %}\n> > +\t\t [[maybe_unused]] ControlSerializer *cs = nullptr)\n> > +{%- endif %}\n> > +\t{\n> > +\t\treturn IPADataSerializer<{{struct|name_full(namespace)}}>::deserialize(data.cbegin(), data.cend(), fds.cbegin(), fds.cend(), cs);\n> > +\t}\n> > +\n> > +\tstatic {{struct|name_full(namespace)}}\n> > +\tdeserialize(std::vector::const_iterator dataBegin,\n> > +\t\t std::vector::const_iterator dataEnd,\n> > +\t\t std::vector::const_iterator fdsBegin,\n> > +\t\t std::vector::const_iterator fdsEnd,\n> > +{%- if struct|needs_control_serializer %}\n> > +\t\t ControlSerializer *cs)\n> > +{%- else %}\n> > +\t\t [[maybe_unused]] ControlSerializer *cs = nullptr)\n> > +{%- endif %}\n> > +\t{\n> > +\t\t{{struct|name_full(namespace)}} ret;\n> > +\t\tstd::vector::const_iterator m = dataBegin;\n> > +\t\tstd::vector::const_iterator n = fdsBegin;\n> > +\n> > +\t\tsize_t dataSize = std::distance(dataBegin, dataEnd);\n> > +\t\t[[maybe_unused]] size_t fdsSize = std::distance(fdsBegin, fdsEnd);\n> > +{%- for field in struct.fields -%}\n> > +{{deserializer_field(field, namespace, loop)}}\n> > +{%- endfor %}\n> > +\t\treturn ret;\n> > +\t}\n> > +{%- endmacro %}\n> > +\n> > +{#\n> > + # \\brief Deserialize a struct that has fds, using non-fd\n> > + #\n> > + # Generate code for IPADataSerializer specialization, for deserializing\n> > + # \\a struct, in the case that \\a struct has no file descriptors but requires\n> > + # deserializers with file descriptors.\n> \n> If the struct has no file descriptors, why does it need a deserialiszer\n> with file descriptors?\n\nIt's for structs that are generated from core.mojom, to allow them to be\nembedded in arrays/maps. The array/map (de)serializer has no way of\nknowing if the struct it's (de)serializing has fds or not, so any struct\nthat will be contained in an array/map must have (de)serializers defined\nfor fds.\n\nThat's why the hand-written (de)serializer implementations for\narithmetic types and ControlList and ControlInfoMap also have them.\n\nIt's actually also required for structs that are generated in the\nper-pipeline mojom files.\n\n> Other than the documentation comments, I don't think I can hope to\n> provide much more parsing of all this.\n> \n> Acked-by: Kieran Bingham \n\n\nThanks,\n\nPaul\n\n\n","headers":{"Return-Path":"","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id 575C5BD80C\n\tfor ;\n\tFri, 15 Jan 2021 05:11:52 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id BCB21680E6;\n\tFri, 15 Jan 2021 06:11:51 +0100 (CET)","from perceval.ideasonboard.com (perceval.ideasonboard.com\n\t[IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id BDB8360314\n\tfor ;\n\tFri, 15 Jan 2021 06:11:49 +0100 (CET)","from pyrite.rasen.tech (unknown\n\t[IPv6:2400:4051:61:600:2c71:1b79:d06d:5032])\n\tby perceval.ideasonboard.com (Postfix) with ESMTPSA id 36FC2527;\n\tFri, 15 Jan 2021 06:11:47 +0100 (CET)"],"Authentication-Results":"lancelot.ideasonboard.com;\n\tdkim=fail reason=\"signature verification failed\" (1024-bit key;\n\tunprotected) header.d=ideasonboard.com header.i=@ideasonboard.com\n\theader.b=\"TC37XeQe\"; dkim-atps=neutral","DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com;\n\ts=mail; t=1610687509;\n\tbh=2j+Gsz4qth6fRl6of0QWboHAD3/nnKluxPIkq3yds4s=;\n\th=Date:From:To:Cc:Subject:References:In-Reply-To:From;\n\tb=TC37XeQegCUE8TDPx0gd7K9h7tfJJ+RFFIeV9NKXfgK2X1cR0N2mYODzxF9j+n04g\n\tZ4IbFArkb6gqP21SzE8LgmYIOCweWgy1a2fBSe+F0jAWLTkli8Y+9IA/EHAE4054jH\n\tWH/bZMDvBy3uo7AShrtKbUw+RBirywWfHlh/uUsM=","Date":"Fri, 15 Jan 2021 14:11:41 +0900","From":"paul.elder@ideasonboard.com","To":"Kieran Bingham ","Message-ID":"<20210115051141.GA1885@pyrite.rasen.tech>","References":"<20201224081534.41601-1-paul.elder@ideasonboard.com>\n\t<20201224081534.41601-3-paul.elder@ideasonboard.com>\n\t<3ce53459-3c34-710a-b4e3-01cc4ff6e8f3@ideasonboard.com>","MIME-Version":"1.0","Content-Disposition":"inline","In-Reply-To":"<3ce53459-3c34-710a-b4e3-01cc4ff6e8f3@ideasonboard.com>","Subject":"Re: [libcamera-devel] [PATCH v6 2/9] utils: ipc: add templates for\n\tcode generation for IPC mechanism","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Cc":"libcamera-devel@lists.libcamera.org","Content-Type":"text/plain; charset=\"us-ascii\"","Content-Transfer-Encoding":"7bit","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" "}},{"id":14547,"web_url":"https://patchwork.libcamera.org/comment/14547/","msgid":"<20210115052447.GB1885@pyrite.rasen.tech>","date":"2021-01-15T05:24:47","subject":"Re: [libcamera-devel] [PATCH v6 2/9] utils: ipc: add templates for\n\tcode generation for IPC mechanism","submitter":{"id":17,"url":"https://patchwork.libcamera.org/api/people/17/","name":"Paul Elder","email":"paul.elder@ideasonboard.com"},"content":"On Fri, Jan 15, 2021 at 02:11:41PM +0900, paul.elder@ideasonboard.com wrote:\n> Hi Kieran,\n> \n> \n> \n> > > diff --git a/utils/ipc/generators/libcamera_templates/serializer.tmpl b/utils/ipc/generators/libcamera_templates/serializer.tmpl\n> > > new file mode 100644\n> > > index 00000000..af4800bf\n> > > --- /dev/null\n> > > +++ b/utils/ipc/generators/libcamera_templates/serializer.tmpl\n> > > @@ -0,0 +1,313 @@\n> > > +{#-\n> > > + # SPDX-License-Identifier: LGPL-2.1-or-later\n> > > + # Copyright (C) 2020, Google Inc.\n> > > +-#}\n> > > +{# Turn this into a C macro? #}\n> > \n> > Is this a todo?\n> \n> It was more of a question,\n> \n> > \n> > \n> > > +{#\n> > > + # \\brief Verify that there is enough bytes to deserialize\n> > > + #\n> > > + # Generate code that verifies that \\a size is not greater than \\a dataSize.\n> > > + # Otherwise log an error with \\a name and \\a typename.\n> > > + #}\n> > > +{%- macro check_data_size(size, dataSize, name, typename) %}\n> > > +\t\tif ({{dataSize}} < {{size}}) {\n> > > +\t\t\tLOG(IPADataSerializer, Error)\n> > > +\t\t\t\t<< \"Failed to deserialize \" << \"{{name}}\"\n> > > +\t\t\t\t<< \": not enough {{typename}}, expected \"\n> > > +\t\t\t\t<< ({{size}}) << \", got \" << ({{dataSize}});\n> > > +\t\t\treturn ret;\n> > > +\t\t}\n> > > +{%- endmacro %}\n> \n> because this block is kind of big, so I wasn't sure if this shold be\n> generated at every check, or if this should be a macro and put the macro\n> at every generated check. In either case it's generated.\n> \n> > > +\n> > > +\n> > > +{#\n> > > + # \\brief Serialize some field into return vector\n> > \n> > 'some' field?\n> > a field?\n> \n> Mathematics vocabulary :p\n> \n> Yeah, I'll change it.\n> \n> > > + #\n> > > + # Generate code to serialize \\a field into retData, including size of the\n> > > + # field and fds (where appropriate).\n> > > + # This code is meant to be used by the IPADataSerializer specialization.\n> > > + #\n> > > + # \\todo Avoid intermediate vectors\n> > > + #}\n> > > +{%- macro serializer_field(field, namespace, loop) %}\n> > > +{%- if field|is_pod or field|is_enum %}\n> > > +\t\tstd::vector {{field.mojom_name}};\n> > > +\t\tstd::tie({{field.mojom_name}}, std::ignore) =\n> > > +\t{%- if field|is_pod %}\n> > > +\t\t\tIPADataSerializer<{{field|name}}>::serialize(data.{{field.mojom_name}});\n> > > +\t{%- elif field|is_enum %}\n> > > +\t\t\tIPADataSerializer::serialize(data.{{field.mojom_name}});\n> > > +\t{%- endif %}\n> > > +\t\tretData.insert(retData.end(), {{field.mojom_name}}.begin(), {{field.mojom_name}}.end());\n> > > +{%- elif field|is_fd %}\n> > > +\t\tstd::vector {{field.mojom_name}};\n> > > +\t\tstd::vector {{field.mojom_name}}Fds;\n> > > +\t\tstd::tie({{field.mojom_name}}, {{field.mojom_name}}Fds) =\n> > > +\t\t\tIPADataSerializer<{{field|name}}>::serialize(data.{{field.mojom_name}});\n> > > +\t\tretData.insert(retData.end(), {{field.mojom_name}}.begin(), {{field.mojom_name}}.end());\n> > > +\t\tretFds.insert(retFds.end(), {{field.mojom_name}}Fds.begin(), {{field.mojom_name}}Fds.end());\n> > > +{%- elif field|is_controls %}\n> > > +\t\tif (data.{{field.mojom_name}}.size() > 0) {\n> > > +\t\t\tstd::vector {{field.mojom_name}};\n> > > +\t\t\tstd::tie({{field.mojom_name}}, std::ignore) =\n> > > +\t\t\t\tIPADataSerializer<{{field|name}}>::serialize(data.{{field.mojom_name}}, cs);\n> > > +\t\t\tappendPOD(retData, {{field.mojom_name}}.size());\n> > > +\t\t\tretData.insert(retData.end(), {{field.mojom_name}}.begin(), {{field.mojom_name}}.end());\n> > > +\t\t} else {\n> > > +\t\t\tappendPOD(retData, 0);\n> > > +\t\t}\n> > > +{%- elif field|is_plain_struct or field|is_array or field|is_map or field|is_str %}\n> > > +\t\tstd::vector {{field.mojom_name}};\n> > > +\t{%- if field|has_fd %}\n> > > +\t\tstd::vector {{field.mojom_name}}Fds;\n> > > +\t\tstd::tie({{field.mojom_name}}, {{field.mojom_name}}Fds) =\n> > > +\t{%- else %}\n> > > +\t\tstd::tie({{field.mojom_name}}, std::ignore) =\n> > > +\t{%- endif %}\n> > > +\t{%- if field|is_array or field|is_map %}\n> > > +\t\t\tIPADataSerializer<{{field|name}}>::serialize(data.{{field.mojom_name}}, cs);\n> > > +\t{%- elif field|is_str %}\n> > > +\t\t\tIPADataSerializer<{{field|name}}>::serialize(data.{{field.mojom_name}});\n> > > +\t{%- else %}\n> > > +\t\t\tIPADataSerializer<{{field|name_full(namespace)}}>::serialize(data.{{field.mojom_name}}, cs);\n> > > +\t{%- endif %}\n> > > +\t\tappendPOD(retData, {{field.mojom_name}}.size());\n> > > +\t{%- if field|has_fd %}\n> > > +\t\tappendPOD(retData, {{field.mojom_name}}Fds.size());\n> > > +\t{%- endif %}\n> > > +\t\tretData.insert(retData.end(), {{field.mojom_name}}.begin(), {{field.mojom_name}}.end());\n> > > +\t{%- if field|has_fd %}\n> > > +\t\tretFds.insert(retFds.end(), {{field.mojom_name}}Fds.begin(), {{field.mojom_name}}Fds.end());\n> > > +\t{%- endif %}\n> > > +{%- else %}\n> > > +\t\t/* Unknown serialization for {{field.mojom_name}}. */\n> > > +{%- endif %}\n> > > +{%- endmacro %}\n> > > +\n> > > +\n> > > +{#\n> > > + # \\brief Deserialize some field into return struct\n> > \n> > Again, 'some field' sounds like a random field.\n> > Presumably this is going to handle a quite specific one each time...\n> > \n> > \n> > > + #\n> > > + # Generate code to deserialize \\a field into object ret.\n> > > + # This code is meant to be used by the IPADataSerializer specialization.\n> > > + #}\n> > > +{%- macro deserializer_field(field, namespace, loop) %}\n> > > +{% if field|is_pod or field|is_enum %}\n> > > +\t{%- set field_size = (field|bit_width|int / 8)|int %}\n> > > +\t\t{{- check_data_size(field_size, 'dataSize', field.mojom_name, 'data')}}\n> > > +\t\t{%- if field|is_pod %}\n> > > +\t\tret.{{field.mojom_name}} = IPADataSerializer<{{field|name}}>::deserialize(m, m + {{field_size}});\n> > > +\t\t{%- else %}\n> > > +\t\tret.{{field.mojom_name}} = static_cast<{{field|name_full(namespace)}}>(IPADataSerializer::deserialize(m, m + {{field_size}}));\n> > > +\t\t{%- endif %}\n> > > +\t{%- if not loop.last %}\n> > > +\t\tm += {{field_size}};\n> > > +\t\tdataSize -= {{field_size}};\n> > > +\t{%- endif %}\n> > > +{% elif field|is_fd %}\n> > > +\t{%- set field_size = 1 %}\n> > > +\t\t{{- check_data_size(field_size, 'dataSize', field.mojom_name, 'data')}}\n> > > +\t\tret.{{field.mojom_name}} = IPADataSerializer<{{field|name}}>::deserialize(m, m + 1, n, n + 1, cs);\n> > > +\t{%- if not loop.last %}\n> > > +\t\tm += {{field_size}};\n> > > +\t\tdataSize -= {{field_size}};\n> > > +\t\tn += ret.{{field.mojom_name}}.isValid() ? 1 : 0;\n> > > +\t\tfdsSize -= ret.{{field.mojom_name}}.isValid() ? 1 : 0;\n> > > +\t{%- endif %}\n> > > +{% elif field|is_controls %}\n> > > +\t{%- set field_size = 4 %}\n> > > +\t\t{{- check_data_size(field_size, 'dataSize', field.mojom_name + 'Size', 'data')}}\n> > > +\t\tconst size_t {{field.mojom_name}}Size = readPOD(m, 0, dataEnd);\n> > > +\t\tm += {{field_size}};\n> > > +\t\tdataSize -= {{field_size}};\n> > > +\t{%- set field_size = field.mojom_name + 'Size' -%}\n> > > +\t\t{{- check_data_size(field_size, 'dataSize', field.mojom_name, 'data')}}\n> > > +\t\tif ({{field.mojom_name}}Size > 0)\n> > > +\t\t\tret.{{field.mojom_name}} =\n> > > +\t\t\t\tIPADataSerializer<{{field|name}}>::deserialize(m, m + {{field.mojom_name}}Size, cs);\n> > > +\t{%- if not loop.last %}\n> > > +\t\tm += {{field_size}};\n> > > +\t\tdataSize -= {{field_size}};\n> > > +\t{%- endif %}\n> > > +{% elif field|is_plain_struct or field|is_array or field|is_map or field|is_str %}\n> > > +\t{%- set field_size = 4 %}\n> > > +\t\t{{- check_data_size(field_size, 'dataSize', field.mojom_name + 'Size', 'data')}}\n> > > +\t\tconst size_t {{field.mojom_name}}Size = readPOD(m, 0, dataEnd);\n> > > +\t\tm += {{field_size}};\n> > > +\t\tdataSize -= {{field_size}};\n> > > +\t{%- if field|has_fd %}\n> > > +\t{%- set field_size = 4 %}\n> > > +\t\t{{- check_data_size(field_size, 'dataSize', field.mojom_name + 'FdsSize', 'data')}}\n> > > +\t\tconst size_t {{field.mojom_name}}FdsSize = readPOD(m, 0, dataEnd);\n> > > +\t\tm += {{field_size}};\n> > > +\t\tdataSize -= {{field_size}};\n> > > +\t\t{{- check_data_size(field.mojom_name + 'FdsSize', 'fdsSize', field.mojom_name, 'fds')}}\n> > > +\t{%- endif %}\n> > > +\t{%- set field_size = field.mojom_name + 'Size' -%}\n> > > +\t\t{{- check_data_size(field_size, 'dataSize', field.mojom_name, 'data')}}\n> > > +\t\tret.{{field.mojom_name}} =\n> > > +\t{%- if field|is_str %}\n> > > +\t\t\tIPADataSerializer<{{field|name}}>::deserialize(m, m + {{field.mojom_name}}Size);\n> > > +\t{%- elif field|has_fd and (field|is_array or field|is_map) %}\n> > > +\t\t\tIPADataSerializer<{{field|name}}>::deserialize(m, m + {{field.mojom_name}}Size, n, n + {{field.mojom_name}}FdsSize, cs);\n> > > +\t{%- elif field|has_fd and (not (field|is_array or field|is_map)) %}\n> > > +\t\t\tIPADataSerializer<{{field|name_full(namespace)}}>::deserialize(m, m + {{field.mojom_name}}Size, n, n + {{field.mojom_name}}FdsSize, cs);\n> > > +\t{%- elif (not field|has_fd) and (field|is_array or field|is_map) %}\n> > > +\t\t\tIPADataSerializer<{{field|name}}>::deserialize(m, m + {{field.mojom_name}}Size, cs);\n> > > +\t{%- else %}\n> > > +\t\t\tIPADataSerializer<{{field|name_full(namespace)}}>::deserialize(m, m + {{field.mojom_name}}Size, cs);\n> > > +\t{%- endif %}\n> > > +\t{%- if not loop.last %}\n> > > +\t\tm += {{field_size}};\n> > > +\t\tdataSize -= {{field_size}};\n> > > +\t{%- if field|has_fd %}\n> > > +\t\tn += {{field.mojom_name}}FdsSize;\n> > > +\t\tfdsSize -= {{field.mojom_name}}FdsSize;\n> > > +\t{%- endif %}\n> > > +\t{%- endif %}\n> > > +{% else %}\n> > > +\t\t/* Unknown deserialization for {{field.mojom_name}}. */\n> > > +{%- endif %}\n> > > +{%- endmacro %}\n> > > +\n> > > +\n> > > +{#\n> > > + # \\brief Serialize a struct\n> > > + #\n> > > + # Generate code for IPADataSerializer specialization, for serializing\n> > > + # \\a struct.\n> > > + #}\n> > > +{%- macro serializer(struct, namespace) %}\n> > > +\tstatic std::tuple, std::vector>\n> > > +\tserialize(const {{struct|name_full(namespace)}} &data,\n> > > +{%- if struct|needs_control_serializer %}\n> > > +\t\t ControlSerializer *cs)\n> > > +{%- else %}\n> > > +\t\t [[maybe_unused]] ControlSerializer *cs = nullptr)\n> > > +{%- endif %}\n> > > +\t{\n> > > +\t\tstd::vector retData;\n> > > +{%- if struct|has_fd %}\n> > > +\t\tstd::vector retFds;\n> > > +{%- endif %}\n> > > +{%- for field in struct.fields %}\n> > > +{{serializer_field(field, namespace, loop)}}\n> > > +{%- endfor %}\n> > > +{% if struct|has_fd %}\n> > > +\t\treturn {retData, retFds};\n> > > +{%- else %}\n> > > +\t\treturn {retData, {}};\n> > > +{%- endif %}\n> > > +\t}\n> > > +{%- endmacro %}\n> > > +\n> > > +\n> > > +{#\n> > > + # \\brief Deserialize a struct that has fds\n> > > + #\n> > > + # Generate code for IPADataSerializer specialization, for deserializing\n> > > + # \\a struct, in the case that \\a struct has file descriptors.\n> > > + # fd parameters\n> > > + #}\n> > > +{%- macro deserializer_fd(struct, namespace) %}\n> > > +\tstatic {{struct|name_full(namespace)}}\n> > > +\tdeserialize(std::vector &data,\n> > > +\t\t std::vector &fds,\n> > > +{%- if struct|needs_control_serializer %}\n> > > +\t\t ControlSerializer *cs)\n> > > +{%- else %}\n> > > +\t\t [[maybe_unused]] ControlSerializer *cs = nullptr)\n> > > +{%- endif %}\n> > > +\t{\n> > > +\t\treturn IPADataSerializer<{{struct|name_full(namespace)}}>::deserialize(data.cbegin(), data.cend(), fds.cbegin(), fds.cend(), cs);\n> > > +\t}\n> > > +\n> > > +\tstatic {{struct|name_full(namespace)}}\n> > > +\tdeserialize(std::vector::const_iterator dataBegin,\n> > > +\t\t std::vector::const_iterator dataEnd,\n> > > +\t\t std::vector::const_iterator fdsBegin,\n> > > +\t\t std::vector::const_iterator fdsEnd,\n> > > +{%- if struct|needs_control_serializer %}\n> > > +\t\t ControlSerializer *cs)\n> > > +{%- else %}\n> > > +\t\t [[maybe_unused]] ControlSerializer *cs = nullptr)\n> > > +{%- endif %}\n> > > +\t{\n> > > +\t\t{{struct|name_full(namespace)}} ret;\n> > > +\t\tstd::vector::const_iterator m = dataBegin;\n> > > +\t\tstd::vector::const_iterator n = fdsBegin;\n> > > +\n> > > +\t\tsize_t dataSize = std::distance(dataBegin, dataEnd);\n> > > +\t\t[[maybe_unused]] size_t fdsSize = std::distance(fdsBegin, fdsEnd);\n> > > +{%- for field in struct.fields -%}\n> > > +{{deserializer_field(field, namespace, loop)}}\n> > > +{%- endfor %}\n> > > +\t\treturn ret;\n> > > +\t}\n> > > +{%- endmacro %}\n> > > +\n> > > +{#\n> > > + # \\brief Deserialize a struct that has fds, using non-fd\n> > > + #\n> > > + # Generate code for IPADataSerializer specialization, for deserializing\n> > > + # \\a struct, in the case that \\a struct has no file descriptors but requires\n> > > + # deserializers with file descriptors.\n> > \n> > If the struct has no file descriptors, why does it need a deserialiszer\n> > with file descriptors?\n> \n> It's for structs that are generated from core.mojom, to allow them to be\n> embedded in arrays/maps. The array/map (de)serializer has no way of\n> knowing if the struct it's (de)serializing has fds or not, so any struct\n> that will be contained in an array/map must have (de)serializers defined\n> for fds.\n> \n> That's why the hand-written (de)serializer implementations for\n> arithmetic types and ControlList and ControlInfoMap also have them.\n> \n> It's actually also required for structs that are generated in the\n> per-pipeline mojom files.\n\n...which apparently wasn't implemented. Gotta go add this. And tests.\n\n\nPaul\n\n> > Other than the documentation comments, I don't think I can hope to\n> > provide much more parsing of all this.\n> > \n> > Acked-by: Kieran Bingham \n> \n> \n> Thanks,\n> \n> Paul\n> \n> \n> \n> _______________________________________________\n> libcamera-devel mailing list\n> libcamera-devel@lists.libcamera.org\n> https://lists.libcamera.org/listinfo/libcamera-devel","headers":{"Return-Path":"","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id 8406EBD808\n\tfor ;\n\tFri, 15 Jan 2021 05:24:57 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id 19E47680EA;\n\tFri, 15 Jan 2021 06:24:57 +0100 (CET)","from perceval.ideasonboard.com (perceval.ideasonboard.com\n\t[IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id DF28860314\n\tfor ;\n\tFri, 15 Jan 2021 06:24:55 +0100 (CET)","from pyrite.rasen.tech (unknown\n\t[IPv6:2400:4051:61:600:2c71:1b79:d06d:5032])\n\tby perceval.ideasonboard.com (Postfix) with ESMTPSA id 3E3C9527;\n\tFri, 15 Jan 2021 06:24:53 +0100 (CET)"],"Authentication-Results":"lancelot.ideasonboard.com;\n\tdkim=fail reason=\"signature verification failed\" (1024-bit key;\n\tunprotected) header.d=ideasonboard.com header.i=@ideasonboard.com\n\theader.b=\"nAbH57zb\"; dkim-atps=neutral","DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com;\n\ts=mail; t=1610688295;\n\tbh=EY95W0CtjYoeTF2HsD4dEX8Fg6t7m3lTOJnE9apL8V8=;\n\th=Date:From:To:Cc:Subject:References:In-Reply-To:From;\n\tb=nAbH57zbdZqQEHr7xIug4QXlIh/DAwVjtJd7hkxOMfkhvX50j0FmzkjRAgfkkNufk\n\tnuBXyw3FA9Q5nD5oFuhdRtPw+1Yazhk35icpe0i1DVLN3ssyHfn0i2uMiU5o8IFu74\n\tOgONsBg+YYzGC2kteekRNCNBl2n9OBJuUb56OZds=","Date":"Fri, 15 Jan 2021 14:24:47 +0900","From":"paul.elder@ideasonboard.com","To":"Kieran Bingham ","Message-ID":"<20210115052447.GB1885@pyrite.rasen.tech>","References":"<20201224081534.41601-1-paul.elder@ideasonboard.com>\n\t<20201224081534.41601-3-paul.elder@ideasonboard.com>\n\t<3ce53459-3c34-710a-b4e3-01cc4ff6e8f3@ideasonboard.com>\n\t<20210115051141.GA1885@pyrite.rasen.tech>","MIME-Version":"1.0","Content-Disposition":"inline","In-Reply-To":"<20210115051141.GA1885@pyrite.rasen.tech>","Subject":"Re: [libcamera-devel] [PATCH v6 2/9] utils: ipc: add templates for\n\tcode generation for IPC mechanism","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Cc":"libcamera-devel@lists.libcamera.org","Content-Type":"text/plain; charset=\"us-ascii\"","Content-Transfer-Encoding":"7bit","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" "}},{"id":14893,"web_url":"https://patchwork.libcamera.org/comment/14893/","msgid":"","date":"2021-02-02T00:33:43","subject":"Re: [libcamera-devel] [PATCH v6 2/9] utils: ipc: add templates for\n\tcode generation for IPC mechanism","submitter":{"id":2,"url":"https://patchwork.libcamera.org/api/people/2/","name":"Laurent Pinchart","email":"laurent.pinchart@ideasonboard.com"},"content":"Hi Paul,\n\nThank you for the patch.\n\nOn Thu, Dec 24, 2020 at 05:15:27PM +0900, Paul Elder wrote:\n> Add templates to mojo to generate code for the IPC mechanism. These\n> templates generate:\n> - module header\n> - module serializer\n> - IPA proxy cpp, header, and worker\n> \n> Given an input data definition mojom file for a pipeline.\n> \n> Signed-off-by: Paul Elder \n> Acked-by: Jacopo Mondi \n> \n> ---\n> Changes in v6:\n> - add templates for core_ipa_interface.h and core_ipa_serializer.h\n> - for libcamera types defined in mojom\n> - rename everything to {{module_name}}_ipa_{interface,proxy,proxy_worker}.{c,h}\n> - remove #include - support customizable start()\n> - remove the need for per-pipeline ControlInfoMap\n> - add todo for avoiding intermediate vectors\n> - remove postfix underscore for generated struct fields\n> - support structs that are members of vectors/maps that aren't defined\n> in mojom (in mojom)\n> - fix has_fd detection\n> - namespacing is now required in mojom, in the form of ^ipa\\.[0-9A-Za-z_]+\n> - support consts in mojom\n> - make the pseudo-switch-case in the python generator nicer\n> \n> Changes in v5:\n> - add a usage output to the proxy worker, to document the interface for\n> executing the proxy worker\n> - in the mojom generator python script:\n> - removed unused things (imports, functions, jinja exports)\n> - document GetNameForElement\n> - rename everything cb -> event\n> - refactor Method{Input,Output}HasFd with a helper MethodParamsHaveFd\n> - add Get{Main,Event}Interface to fix the interface_{main,event} jinja\n> exports\n> - add copyright\n> - require that event interfaces have at least one event\n> - expand copyright for templates\n> - use new sendSync/sendAsync API (with IPCMessage)\n> - rename a bunch of things\n> \n> Changes in v4:\n> For the non-template files:\n> - rename IPA{pipeline_name}CallbackInterface to\n> IPA{pipeline_name}EventInterface\n> - to avoid the notion of \"callback\" and emphasize that it's an event\n> - add support for strings in custom structs\n> - add validation, that async methods must not have return values\n> - it throws exception and isn't very clear though...?\n> - rename controls to libcamera::{pipeline_name}::controls (controls is\n> now lowercase)\n> - rename {pipeline_name}_generated.h to {pipeline_name}_ipa_interface.h,\n> and {pipeline_name}_serializer.h to {pipeline_name}_ipa_serializer.h\n> - same for their corresponding template files\n> For the template files:\n> - fix spacing (now it's all {{var}} instead of some {{ var }})\n> - except if it's code, so code is still {{ code }}\n> - move inclusion of corresponding header to first in the inclusion list\n> - fix copy&paste errors\n> - change snake_case to camelCase in the generated code\n> - template code still uses snake_case\n> - change the generated command enums to an enum class, and make it\n> capitalized (instead of allcaps)\n> - add length checks to recvIPC (in proxy)\n> - fix some template spacing\n> - don't use const for PODs in function/signal parameters\n> - add the proper length checks to readPOD/appendPOD\n> - the helper functions for reading and writing PODs to and from\n> serialized data\n> - rename readUInt/appendUInt to readPOD/appendPOD\n> - add support for strings in custom structs\n> \n> Changes in v3:\n> - add support for namespaces\n> - fix enum assignment (used to have +1 for CMD applied to all enums)\n> - use readHeader, writeHeader, and eraseHeader as static class functions\n> of IPAIPCUnixSocket (in the proxy worker)\n> - add requirement that base controls *must* be defined in\n> libcamera::{pipeline_name}::Controls\n> \n> Changes in v2:\n> - mandate the main and callback interfaces, and init(), start(), stop()\n> and their parameters\n> - fix returning single pod value from IPC-called function\n> - add licenses\n> - improve auto-generated message\n> - other fixes related to serdes\n> ---\n> .../core_ipa_interface.h.tmpl | 37 ++\n> .../core_ipa_serializer.h.tmpl | 47 ++\n> .../definition_functions.tmpl | 53 ++\n> .../libcamera_templates/meson.build | 14 +\n> .../module_ipa_interface.h.tmpl | 87 +++\n> .../module_ipa_proxy.cpp.tmpl | 232 ++++++++\n> .../module_ipa_proxy.h.tmpl | 126 +++++\n> .../module_ipa_proxy_worker.cpp.tmpl | 224 ++++++++\n> .../module_ipa_serializer.h.tmpl | 47 ++\n> .../libcamera_templates/proxy_functions.tmpl | 192 +++++++\n> .../libcamera_templates/serializer.tmpl | 313 +++++++++++\n> .../generators/mojom_libcamera_generator.py | 511 ++++++++++++++++++\n> 12 files changed, 1883 insertions(+)\n> create mode 100644 utils/ipc/generators/libcamera_templates/core_ipa_interface.h.tmpl\n> create mode 100644 utils/ipc/generators/libcamera_templates/core_ipa_serializer.h.tmpl\n> create mode 100644 utils/ipc/generators/libcamera_templates/definition_functions.tmpl\n> create mode 100644 utils/ipc/generators/libcamera_templates/meson.build\n> create mode 100644 utils/ipc/generators/libcamera_templates/module_ipa_interface.h.tmpl\n> create mode 100644 utils/ipc/generators/libcamera_templates/module_ipa_proxy.cpp.tmpl\n> create mode 100644 utils/ipc/generators/libcamera_templates/module_ipa_proxy.h.tmpl\n> create mode 100644 utils/ipc/generators/libcamera_templates/module_ipa_proxy_worker.cpp.tmpl\n> create mode 100644 utils/ipc/generators/libcamera_templates/module_ipa_serializer.h.tmpl\n> create mode 100644 utils/ipc/generators/libcamera_templates/proxy_functions.tmpl\n> create mode 100644 utils/ipc/generators/libcamera_templates/serializer.tmpl\n> create mode 100644 utils/ipc/generators/mojom_libcamera_generator.py\n> \n> diff --git a/utils/ipc/generators/libcamera_templates/core_ipa_interface.h.tmpl b/utils/ipc/generators/libcamera_templates/core_ipa_interface.h.tmpl\n> new file mode 100644\n> index 00000000..f11d56fb\n> --- /dev/null\n> +++ b/utils/ipc/generators/libcamera_templates/core_ipa_interface.h.tmpl\n> @@ -0,0 +1,37 @@\n> +{#-\n> + # SPDX-License-Identifier: LGPL-2.1-or-later\n> + # Copyright (C) 2020, Google Inc.\n> +-#}\n> +{%- import \"definition_functions.tmpl\" as funcs -%}\n> +/* SPDX-License-Identifier: LGPL-2.1-or-later */\n> +/*\n> + * Copyright (C) 2020, Google Inc.\n> + *\n> + * core_ipa_interface.h - libcamera core definitions for Image Processing Algorithms\n> + *\n> + * This file is auto-generated. Do not edit.\n> + */\n> +\n> +#ifndef __LIBCAMERA_IPA_INTERFACE_CORE_GENERATED_H__\n> +#define __LIBCAMERA_IPA_INTERFACE_CORE_GENERATED_H__\n> +\n> +{% if has_map %}#include {% endif %}\n> +{% if has_array %}#include {% endif %}\n> +\n> +namespace libcamera {\n> +\n> +{% for const in consts %}\n> +const {{const.kind|name}} {{const.mojom_name}} = {{const.value}};\n\ns/const/static const/ ?\n\nSame for module_ipa_interface.h.tmpl.\n\nIt would be best to use constexpr instead of const, but that won't work\nwith strings as the std::string constructor only becomes constexpr in\nC++20 :-S Switching strings to char * won't be easy. Could you add a\n\\todo comment about this ?\n\n> +{% endfor %}\n> +\n> +{% for enum in enums %}\n> +{{funcs.define_enum(enum)}}\n> +{% endfor %}\n> +\n> +{%- for struct in structs_gen_header %}\n> +{{funcs.define_struct(struct)}}\n> +{% endfor %}\n> +\n> +} /* namespace libcamera */\n> +\n> +#endif /* __LIBCAMERA_IPA_INTERFACE_CORE_GENERATED_H__ */\n> diff --git a/utils/ipc/generators/libcamera_templates/core_ipa_serializer.h.tmpl b/utils/ipc/generators/libcamera_templates/core_ipa_serializer.h.tmpl\n> new file mode 100644\n> index 00000000..37a784f1\n> --- /dev/null\n> +++ b/utils/ipc/generators/libcamera_templates/core_ipa_serializer.h.tmpl\n> @@ -0,0 +1,47 @@\n> +{#-\n> + # SPDX-License-Identifier: LGPL-2.1-or-later\n> + # Copyright (C) 2020, Google Inc.\n> +-#}\n> +{%- import \"serializer.tmpl\" as serializer -%}\n> +\n> +/* SPDX-License-Identifier: LGPL-2.1-or-later */\n> +/*\n> + * Copyright (C) 2020, Google Inc.\n> + *\n> + * core_ipa_serializer.h - Data serializer for core libcamera definitions for IPA\n> + *\n> + * This file is auto-generated. Do not edit.\n> + */\n> +\n> +#ifndef __LIBCAMERA_INTERNAL_IPA_DATA_SERIALIZER_CORE_H__\n> +#define __LIBCAMERA_INTERNAL_IPA_DATA_SERIALIZER_CORE_H__\n> +\n> +#include \n> +#include \n> +\n> +#include \n> +\n> +#include \"libcamera/internal/control_serializer.h\"\n> +#include \"libcamera/internal/ipa_data_serializer.h\"\n> +\n> +namespace libcamera {\n> +\n> +LOG_DECLARE_CATEGORY(IPADataSerializer)\n> +{% for struct in structs_gen_serializer %}\n> +template<>\n> +class IPADataSerializer<{{struct|name}}>\n> +{\n> +public:\n> +{{- serializer.serializer(struct, \"\")}}\n> +{%- if struct|has_fd %}\n> +{{serializer.deserializer_fd(struct, \"\")}}\n> +{%- else %}\n> +{{serializer.deserializer_no_fd(struct, \"\")}}\n> +{{serializer.deserializer_fd_simple(struct, \"\")}}\n> +{%- endif %}\n> +};\n> +{% endfor %}\n> +\n> +} /* namespace libcamera */\n> +\n> +#endif /* __LIBCAMERA_INTERNAL_IPA_DATA_SERIALIZER_CORE_H__ */\n> diff --git a/utils/ipc/generators/libcamera_templates/definition_functions.tmpl b/utils/ipc/generators/libcamera_templates/definition_functions.tmpl\n> new file mode 100644\n> index 00000000..cdd75f89\n> --- /dev/null\n> +++ b/utils/ipc/generators/libcamera_templates/definition_functions.tmpl\n> @@ -0,0 +1,53 @@\n> +{#-\n> + # SPDX-License-Identifier: LGPL-2.1-or-later\n> + # Copyright (C) 2020, Google Inc.\n> +-#}\n> +\n> +{#\n> + # \\brief Generate enum definition\n> + #\n> + # \\param enum Enum object whose definition is to be generated\n> + #}\n> +{%- macro define_enum(enum) -%}\n> +enum {{enum.mojom_name}} {\n> +{%- for field in enum.fields %}\n> +\t{{field.mojom_name}} = {{field.numeric_value}},\n> +{%- endfor %}\n> +};\n> +{%- endmacro -%}\n> +\n> +{#\n> + # \\brief Generate struct definition\n> + #\n> + # \\param struct Struct object whose definition is to be generated\n> + #}\n> +{%- macro define_struct(struct) -%}\n> +struct {{struct.mojom_name}}\n> +{\n> +public:\n> +\t{{struct.mojom_name}}() {%- if struct|has_default_fields %}\n> +\t\t:{% endif %}\n> +{%- for field in struct.fields|with_default_values -%}\n> +{{\" \" if loop.first}}{{field.mojom_name}}({{field|default_value}}){{\", \" if not loop.last}}\n> +{%- endfor %}\n> +\t{\n> +\t}\n> +\n> +\t{{struct.mojom_name}}(\n> +{%- for field in struct.fields -%}\n> +{{\"const \" if not field|is_pod}}{{field|name}} {{\"&\" if not field|is_pod}}_{{field.mojom_name}}{{\", \" if not loop.last}}\n> +{%- endfor -%}\n> +)\n> +\t\t:\n> +{%- for field in struct.fields -%}\n> +{{\" \" if loop.first}}{{field.mojom_name}}(_{{field.mojom_name}}){{\", \" if not loop.last}}\n> +{%- endfor %}\n> +\t{\n> +\t}\n> +{% for field in struct.fields %}\n> +\t{{field|name}} {{field.mojom_name}};\n> +{%- endfor %}\n> +};\n> +{%- endmacro -%}\n> +\n> +\n> diff --git a/utils/ipc/generators/libcamera_templates/meson.build b/utils/ipc/generators/libcamera_templates/meson.build\n> new file mode 100644\n> index 00000000..70664eab\n> --- /dev/null\n> +++ b/utils/ipc/generators/libcamera_templates/meson.build\n> @@ -0,0 +1,14 @@\n> +# SPDX-License-Identifier: CC0-1.0\n> +\n> +mojom_template_files = files([\n> + 'core_ipa_interface.h.tmpl',\n> + 'core_ipa_serializer.h.tmpl',\n> + 'definition_functions.tmpl',\n> + 'module_ipa_interface.h.tmpl',\n> + 'module_ipa_proxy.cpp.tmpl',\n> + 'module_ipa_proxy.h.tmpl',\n> + 'module_ipa_proxy_worker.cpp.tmpl',\n> + 'module_ipa_serializer.h.tmpl',\n> + 'proxy_functions.tmpl',\n> + 'serializer.tmpl',\n> +])\n> diff --git a/utils/ipc/generators/libcamera_templates/module_ipa_interface.h.tmpl b/utils/ipc/generators/libcamera_templates/module_ipa_interface.h.tmpl\n> new file mode 100644\n> index 00000000..afbcb1b1\n> --- /dev/null\n> +++ b/utils/ipc/generators/libcamera_templates/module_ipa_interface.h.tmpl\n> @@ -0,0 +1,87 @@\n> +{#-\n> + # SPDX-License-Identifier: LGPL-2.1-or-later\n> + # Copyright (C) 2020, Google Inc.\n> +-#}\n> +{%- import \"definition_functions.tmpl\" as funcs -%}\n> +/* SPDX-License-Identifier: LGPL-2.1-or-later */\n> +/*\n> + * Copyright (C) 2020, Google Inc.\n> + *\n> + * {{module_name}}_ipa_interface.h - Image Processing Algorithm interface for {{module_name}}\n> + *\n> + * This file is auto-generated. Do not edit.\n> + */\n> +\n> +#ifndef __LIBCAMERA_IPA_INTERFACE_{{module_name|upper}}_GENERATED_H__\n> +#define __LIBCAMERA_IPA_INTERFACE_{{module_name|upper}}_GENERATED_H__\n> +\n> +#include \n> +#include \n\nWrong alphabetical order.\n\n> +\n> +{% if has_map %}#include {% endif %}\n> +{% if has_array %}#include {% endif %}\n> +\n> +namespace libcamera {\n> +{%- if has_namespace %}\n> +{% for ns in namespace %}\n> +namespace {{ns}} {\n> +{% endfor %}\n> +{%- endif %}\n> +\n> +{% for const in consts %}\n> +const {{const.kind|name}} {{const.mojom_name}} = {{const.value}};\n> +{% endfor %}\n> +\n> +enum class {{cmd_enum_name}} {\n> +\tExit = 0,\n> +{%- for method in interface_main.methods %}\n> +\t{{method.mojom_name|cap}} = {{loop.index}},\n> +{%- endfor %}\n> +};\n> +\n> +enum class {{cmd_event_enum_name}} {\n> +{%- for method in interface_event.methods %}\n> +\t{{method.mojom_name|cap}} = {{loop.index}},\n> +{%- endfor %}\n> +};\n> +\n> +{% for enum in enums %}\n> +{{funcs.define_enum(enum)}}\n> +{% endfor %}\n> +\n> +{%- for struct in structs_nonempty %}\n> +{{funcs.define_struct(struct)}}\n> +{% endfor %}\n> +\n> +{#-\n> +Any consts or #defines should be moved to the mojom file.\n> +#}\n> +class {{interface_name}} : public IPAInterface\n> +{\n> +public:\n> +{% for method in interface_main.methods %}\n> +\tvirtual {{method|method_return_value}} {{method.mojom_name}}(\n> +{%- for param in method|method_parameters %}\n> +\t\t{{param}}{{- \",\" if not loop.last}}\n> +{%- endfor -%}\n> +) = 0;\n> +{% endfor %}\n> +\n> +{%- for method in interface_event.methods %}\n> +\tSignal<\n> +{%- for param in method.parameters -%}\n> +\t\t{{\"const \" if not param|is_pod}}{{param|name}}{{\" &\" if not param|is_pod}}\n> +\t\t{{- \", \" if not loop.last}}\n> +{%- endfor -%}\n> +> {{method.mojom_name}};\n> +{% endfor -%}\n> +};\n> +\n> +{%- if has_namespace %}\n> +{% for ns in namespace|reverse %}\n> +} /* namespace {{ns}} */\n> +{% endfor %}\n> +{%- endif %}\n> +} /* namespace libcamera */\n> +\n> +#endif /* __LIBCAMERA_IPA_INTERFACE_{{module_name|upper}}_GENERATED_H__ */\n> diff --git a/utils/ipc/generators/libcamera_templates/module_ipa_proxy.cpp.tmpl b/utils/ipc/generators/libcamera_templates/module_ipa_proxy.cpp.tmpl\n> new file mode 100644\n> index 00000000..a181cc84\n> --- /dev/null\n> +++ b/utils/ipc/generators/libcamera_templates/module_ipa_proxy.cpp.tmpl\n> @@ -0,0 +1,232 @@\n> +{#-\n> + # SPDX-License-Identifier: LGPL-2.1-or-later\n> + # Copyright (C) 2020, Google Inc.\n> +-#}\n> +{%- import \"proxy_functions.tmpl\" as proxy_funcs -%}\n> +\n> +/* SPDX-License-Identifier: LGPL-2.1-or-later */\n> +/*\n> + * Copyright (C) 2020, Google Inc.\n> + *\n> + * {{module_name}}_ipa_proxy.cpp - Image Processing Algorithm proxy for {{module_name}}\n> + *\n> + * This file is auto-generated. Do not edit.\n> + */\n> +\n> +#include \n> +\n\n#include \n#include \n\n(for std::unique_ptr and std::string)\n\n> +#include \n> +\n> +#include \n> +#include \n> +#include \n> +\n> +#include \"libcamera/internal/control_serializer.h\"\n> +#include \"libcamera/internal/ipa_data_serializer.h\"\n> +#include \"libcamera/internal/ipa_module.h\"\n> +#include \"libcamera/internal/ipa_proxy.h\"\n> +#include \"libcamera/internal/ipc_pipe.h\"\n> +#include \"libcamera/internal/ipc_pipe_unixsocket.h\"\n> +#include \"libcamera/internal/ipc_unixsocket.h\"\n> +#include \"libcamera/internal/log.h\"\n> +#include \"libcamera/internal/process.h\"\n> +#include \"libcamera/internal/thread.h\"\n> +\n> +namespace libcamera {\n> +\n> +LOG_DECLARE_CATEGORY(IPAProxy)\n> +\n> +{%- if has_namespace %}\n> +{% for ns in namespace %}\n> +namespace {{ns}} {\n> +{% endfor %}\n> +{%- endif %}\n> +\n> +{{proxy_name}}::{{proxy_name}}(IPAModule *ipam, bool isolate)\n> +\t: IPAProxy(ipam), running_(false),\n> +\t isolate_(isolate)\n> +{\n> +\tLOG(IPAProxy, Debug)\n> +\t\t<< \"initializing {{module_name}} proxy: loading IPA from \"\n> +\t\t<< ipam->path();\n> +\n> +\tif (isolate_) {\n> +\t\tconst std::string proxyWorkerPath = resolvePath(\"{{module_name}}_ipa_proxy\");\n> +\t\tif (proxyWorkerPath.empty()) {\n> +\t\t\tLOG(IPAProxy, Error)\n> +\t\t\t\t<< \"Failed to get proxy worker path\";\n> +\t\t\treturn;\n> +\t\t}\n> +\n> +\t\tipc_ = std::make_unique(ipam->path().c_str(),\n> +\t\t\t\t\t\t\t proxyWorkerPath.c_str());\n> +\t\tif (!ipc_->isConnected()) {\n> +\t\t\tLOG(IPAProxy, Error) << \"Failed to create IPCPipe\";\n> +\t\t\treturn;\n> +\t\t}\n> +\n> +\t\tipc_->recv.connect(this, &{{proxy_name}}::recvMessage);\n> +\n> +\t\tvalid_ = true;\n> +\t\treturn;\n> +\t}\n> +\n> +\tif (!ipam->load())\n> +\t\treturn;\n> +\n> +\tIPAInterface *ipai = ipam->createInterface();\n> +\tif (!ipai) {\n> +\t\tLOG(IPAProxy, Error)\n> +\t\t\t<< \"Failed to create IPA context for \" << ipam->path();\n> +\t\treturn;\n> +\t}\n> +\n> +\tipa_ = std::unique_ptr<{{interface_name}}>(static_cast<{{interface_name}} *>(ipai));\n> +\tproxy_.setIPA(ipa_.get());\n> +\n> +{% for method in interface_event.methods %}\n> +\tipa_->{{method.mojom_name}}.connect(this, &{{proxy_name}}::{{method.mojom_name}}Thread);\n> +{%- endfor %}\n> +\n> +\tvalid_ = true;\n> +}\n> +\n> +{{proxy_name}}::~{{proxy_name}}()\n> +{\n> +\tif (isolate_)\n> +\t\tipc_->sendAsync(static_cast({{cmd_enum_name}}::Exit), {});\n> +}\n> +\n> +{% if interface_event.methods|length > 0 %}\n> +void {{proxy_name}}::recvMessage(uint32_t cmd, const IPCMessage &data)\n> +{\n> +\tsize_t dataSize = data.cdata().size();\n> +\t{{cmd_event_enum_name}} _cmd = static_cast<{{cmd_event_enum_name}}>(cmd);\n> +\n> +\tswitch (_cmd) {\n> +{%- for method in interface_event.methods %}\n> +\tcase {{cmd_event_enum_name}}::{{method.mojom_name|cap}}: {\n> +\t\t{{method.mojom_name}}IPC(data.cdata().cbegin(), dataSize, data.cfds());\n> +\t\tbreak;\n> +\t}\n> +{%- endfor %}\n> +\tdefault:\n> +\t\tLOG(IPAProxy, Error) << \"Unknown command \" << static_cast(_cmd);\n> +\t}\n> +}\n> +{%- endif %}\n> +\n> +{% for method in interface_main.methods %}\n> +{{proxy_funcs.func_sig(proxy_name, method)}}\n> +{\n> +\tif (isolate_)\n> +\t\t{{\"return \" if method|method_return_value != \"void\"}}{{method.mojom_name}}IPC(\n> +{%- for param in method|method_param_names -%}\n> +\t\t{{param}}{{- \", \" if not loop.last}}\n> +{%- endfor -%}\n> +);\n> +\telse\n> +\t\t{{\"return \" if method|method_return_value != \"void\"}}{{method.mojom_name}}Thread(\n> +{%- for param in method|method_param_names -%}\n> +\t\t{{param}}{{- \", \" if not loop.last}}\n> +{%- endfor -%}\n> +);\n> +}\n> +\n> +{{proxy_funcs.func_sig(proxy_name, method, \"Thread\")}}\n> +{\n> +{%- if method.mojom_name == \"init\" %}\n> +\t{{proxy_funcs.init_thread_body()}}\n> +{%- elif method.mojom_name == \"stop\" %}\n> +\t{{proxy_funcs.stop_thread_body()}}\n> +{%- elif method.mojom_name == \"start\" %}\n> +\trunning_ = true;\n> +\tthread_.start();\n> +\n> +\t{{ \"return \" if method|method_return_value != \"void\" -}}\n> +\tproxy_.invokeMethod(&ThreadProxy::start, ConnectionTypeBlocking\n> +\t{{- \", \" if method|method_param_names}}\n> +\t{%- for param in method|method_param_names -%}\n> +\t\t{{param}}{{- \", \" if not loop.last}}\n> +\t{%- endfor -%}\n> +);\n> +{%- elif not method|is_async %}\n> +\t{{ \"return \" if method|method_return_value != \"void\" -}}\n> +\tipa_->{{method.mojom_name}}(\n> +\t{%- for param in method|method_param_names -%}\n> +\t\t{{param}}{{- \", \" if not loop.last}}\n> +\t{%- endfor -%}\n> +);\n> +{% elif method|is_async %}\n> +\tproxy_.invokeMethod(&ThreadProxy::{{method.mojom_name}}, ConnectionTypeQueued,\n> +\t{%- for param in method|method_param_names -%}\n> +\t\t{{param}}{{- \", \" if not loop.last}}\n> +\t{%- endfor -%}\n> +);\n> +{%- endif %}\n> +}\n> +\n> +{{proxy_funcs.func_sig(proxy_name, method, \"IPC\")}}\n> +{\n> +{%- set has_input = true if method|method_param_inputs|length > 0 %}\n> +{%- set has_output = true if method|method_param_outputs|length > 0 or method|method_return_value != \"void\" %}\n> +{%- if has_input %}\n> +\tIPCMessage _ipcInputBuf;\n> +{%- endif %}\n> +{%- if has_output %}\n> +\tIPCMessage _ipcOutputBuf;\n> +{%- endif %}\n> +\n> +{{proxy_funcs.serialize_call(method|method_param_inputs, '_ipcInputBuf.data()', '_ipcInputBuf.fds()')}}\n> +\n> +{%- set input_buf = \"_ipcInputBuf\" if has_input else \"{}\" %}\n> +{%- set cmd = cmd_enum_name + \"::\" + method.mojom_name|cap %}\n> +{% if method|is_async %}\n> +\tint _ret = ipc_->sendAsync(static_cast({{cmd}}), {{input_buf}});\n> +{%- else %}\n> +\tint _ret = ipc_->sendSync(static_cast({{cmd}}), {{input_buf}}\n> +{{- \", &_ipcOutputBuf\" if has_output -}}\n> +);\n> +{%- endif %}\n> +\tif (_ret < 0) {\n> +\t\tLOG(IPAProxy, Error) << \"Failed to call {{method.mojom_name}}\";\n> +{%- if method|method_return_value != \"void\" %}\n> +\t\treturn static_cast<{{method|method_return_value}}>(_ret);\n> +{%- else %}\n> +\t\treturn;\n> +{%- endif %}\n> +\t}\n> +{% if method|method_return_value != \"void\" %}\n> +\treturn IPADataSerializer<{{method.response_parameters|first|name}}>::deserialize(_ipcOutputBuf.data(), 0);\n> +{% elif method|method_param_outputs|length > 0 %}\n> +{{proxy_funcs.deserialize_call(method|method_param_outputs, '_ipcOutputBuf.data()', '_ipcOutputBuf.fds()')}}\n> +{% endif -%}\n> +}\n> +\n> +{% endfor %}\n> +\n> +{% for method in interface_event.methods %}\n> +{{proxy_funcs.func_sig(proxy_name, method, \"Thread\")}}\n> +{\n> +\t{{method.mojom_name}}.emit({{method.parameters|params_comma_sep}});\n> +}\n> +\n> +void {{proxy_name}}::{{method.mojom_name}}IPC(\n> +\tstd::vector::const_iterator data,\n> +\tsize_t dataSize,\n> +\t[[maybe_unused]] const std::vector &fds)\n> +{\n> +{%- for param in method.parameters %}\n> +\t{{param|name}} {{param.mojom_name}};\n> +{%- endfor %}\n> +{{proxy_funcs.deserialize_call(method.parameters, 'data', 'fds', false, false, true, 'dataSize')}}\n> +\t{{method.mojom_name}}.emit({{method.parameters|params_comma_sep}});\n> +}\n> +{% endfor %}\n> +\n> +{%- if has_namespace %}\n> +{% for ns in namespace|reverse %}\n> +} /* namespace {{ns}} */\n> +{% endfor %}\n> +{%- endif %}\n> +} /* namespace libcamera */\n> diff --git a/utils/ipc/generators/libcamera_templates/module_ipa_proxy.h.tmpl b/utils/ipc/generators/libcamera_templates/module_ipa_proxy.h.tmpl\n> new file mode 100644\n> index 00000000..b0f04bf6\n> --- /dev/null\n> +++ b/utils/ipc/generators/libcamera_templates/module_ipa_proxy.h.tmpl\n> @@ -0,0 +1,126 @@\n> +{#-\n> + # SPDX-License-Identifier: LGPL-2.1-or-later\n> + # Copyright (C) 2020, Google Inc.\n> +-#}\n> +{%- import \"proxy_functions.tmpl\" as proxy_funcs -%}\n> +\n> +/* SPDX-License-Identifier: LGPL-2.1-or-later */\n> +/*\n> + * Copyright (C) 2020, Google Inc.\n> + *\n> + * {{module_name}}_ipa_proxy.h - Image Processing Algorithm proxy for {{module_name}}\n> + *\n> + * This file is auto-generated. Do not edit.\n> + */\n> +\n> +#ifndef __LIBCAMERA_INTERNAL_IPA_PROXY_{{module_name|upper}}_H__\n> +#define __LIBCAMERA_INTERNAL_IPA_PROXY_{{module_name|upper}}_H__\n> +\n> +#include \n> +#include \n> +\n> +#include \"libcamera/internal/control_serializer.h\"\n> +#include \"libcamera/internal/ipc_pipe.h\"\n> +#include \"libcamera/internal/ipc_pipe_unixsocket.h\"\n> +#include \"libcamera/internal/ipa_proxy.h\"\n\nThis one goes before ipc_pipe.h.\n\n> +#include \"libcamera/internal/ipc_unixsocket.h\"\n> +#include \"libcamera/internal/thread.h\"\n> +\n> +namespace libcamera {\n> +{%- if has_namespace %}\n> +{% for ns in namespace %}\n> +namespace {{ns}} {\n> +{% endfor %}\n> +{%- endif %}\n> +\n> +class {{proxy_name}} : public IPAProxy, public {{interface_name}}, public Object\n> +{\n> +public:\n> +\t{{proxy_name}}(IPAModule *ipam, bool isolate);\n> +\t~{{proxy_name}}();\n> +\n> +{% for method in interface_main.methods %}\n> +{{proxy_funcs.func_sig(proxy_name, method, \"\", false, true)|indent(8, true)}};\n> +{% endfor %}\n> +\n> +{%- for method in interface_event.methods %}\n> +\tSignal<\n> +{%- for param in method.parameters -%}\n> +\t\t{{\"const \" if not param|is_pod}}{{param|name}}{{\" &\" if not param|is_pod}}\n> +\t\t{{- \", \" if not loop.last}}\n> +{%- endfor -%}\n> +> {{method.mojom_name}};\n> +{% endfor %}\n> +\n> +private:\n> +\tvoid recvMessage(uint32_t cmd, const IPCMessage &data);\n> +\n> +{% for method in interface_main.methods %}\n> +{{proxy_funcs.func_sig(proxy_name, method, \"Thread\", false)|indent(8, true)}};\n> +{{proxy_funcs.func_sig(proxy_name, method, \"IPC\", false)|indent(8, true)}};\n> +{% endfor %}\n> +{% for method in interface_event.methods %}\n> +{{proxy_funcs.func_sig(proxy_name, method, \"Thread\", false)|indent(8, true)}};\n> +\tvoid {{method.mojom_name}}IPC(\n> +\t\tstd::vector::const_iterator data,\n> +\t\tsize_t dataSize,\n> +\t\tconst std::vector &fds);\n\nReally nice work on the constification of the API, and the usage of\nIPCMessage !\n\n> +{% endfor %}\n> +\n> +\t/* Helper class to invoke async functions in another thread. */\n> +\tclass ThreadProxy : public Object\n> +\t{\n> +\tpublic:\n> +\t\tvoid setIPA({{interface_name}} *ipa)\n> +\t\t{\n> +\t\t\tipa_ = ipa;\n> +\t\t}\n> +\n> +\t\tvoid stop()\n> +\t\t{\n> +\t\t\tipa_->stop();\n> +\t\t}\n> +{% for method in interface_main.methods %}\n> +{%- if method|is_async %}\n> +\t\t{{proxy_funcs.func_sig(proxy_name, method, \"\", false)|indent(16)}}\n> +\t\t{\n> +\t\t\tipa_->{{method.mojom_name}}({{method.parameters|params_comma_sep}});\n> +\t\t}\n> +{%- elif method.mojom_name == \"start\" %}\n> +\t\t{{proxy_funcs.func_sig(proxy_name, method, \"\", false)|indent(16)}}\n> +\t\t{\n> +{%- if method|method_return_value != \"void\" %}\n> +\t\t\treturn ipa_->{{method.mojom_name}}({{method.parameters|params_comma_sep}});\n> +{%- else %}\n> +\t\t\tipa_->{{method.mojom_name}}({{method.parameters|params_comma_sep}}\n> +\t{{- \", \" if method|method_param_outputs|params_comma_sep -}}\n> +\t{{- method|method_param_outputs|params_comma_sep}});\n> +{%- endif %}\n> +\t\t}\n> +{%- endif %}\n> +{%- endfor %}\n> +\n> +\tprivate:\n> +\t\t{{interface_name}} *ipa_;\n> +\t};\n> +\n> +\tbool running_;\n> +\tThread thread_;\n> +\tThreadProxy proxy_;\n> +\tstd::unique_ptr<{{interface_name}}> ipa_;\n> +\n> +\tconst bool isolate_;\n> +\n> +\tstd::unique_ptr ipc_;\n> +\n> +\tControlSerializer controlSerializer_;\n> +};\n> +\n> +{%- if has_namespace %}\n> +{% for ns in namespace|reverse %}\n> +} /* namespace {{ns}} */\n> +{% endfor %}\n> +{%- endif %}\n> +} /* namespace libcamera */\n> +\n> +#endif /* __LIBCAMERA_INTERNAL_IPA_PROXY_{{module_name|upper}}_H__ */\n> diff --git a/utils/ipc/generators/libcamera_templates/module_ipa_proxy_worker.cpp.tmpl b/utils/ipc/generators/libcamera_templates/module_ipa_proxy_worker.cpp.tmpl\n> new file mode 100644\n> index 00000000..cb3df4eb\n> --- /dev/null\n> +++ b/utils/ipc/generators/libcamera_templates/module_ipa_proxy_worker.cpp.tmpl\n> @@ -0,0 +1,224 @@\n> +{#-\n> + # SPDX-License-Identifier: LGPL-2.1-or-later\n> + # Copyright (C) 2020, Google Inc.\n> +-#}\n> +{%- import \"proxy_functions.tmpl\" as proxy_funcs -%}\n> +\n> +/* SPDX-License-Identifier: LGPL-2.1-or-later */\n> +/*\n> + * Copyright (C) 2020, Google Inc.\n> + *\n> + * {{module_name}}_ipa_proxy_worker.cpp - Image Processing Algorithm proxy worker for {{module_name}}\n> + *\n> + * This file is auto-generated. Do not edit.\n> + */\n> +\n> +{#- \\todo Split proxy worker into IPC worker and proxy worker. #}\n> +\n> +#include \n> +#include \n> +#include \n> +#include \n> +#include \n> +\n> +#include \n> +#include \n> +#include \n> +#include \n> +\n> +#include \"libcamera/internal/camera_sensor.h\"\n> +#include \"libcamera/internal/control_serializer.h\"\n> +#include \"libcamera/internal/event_dispatcher.h\"\n> +#include \"libcamera/internal/ipa_data_serializer.h\"\n> +#include \"libcamera/internal/ipc_pipe.h\"\n> +#include \"libcamera/internal/ipc_pipe_unixsocket.h\"\n> +#include \"libcamera/internal/ipa_module.h\"\n> +#include \"libcamera/internal/ipa_proxy.h\"\n\nThese two go before ipc_pipe.h.\n\n> +#include \"libcamera/internal/ipc_unixsocket.h\"\n> +#include \"libcamera/internal/log.h\"\n> +#include \"libcamera/internal/thread.h\"\n> +\n> +using namespace libcamera;\n> +\n> +LOG_DEFINE_CATEGORY({{proxy_worker_name}})\n> +\n> +{%- if has_namespace %}\n> +{% for ns in namespace -%}\n> +using namespace {{ns}};\n> +{% endfor %}\n> +{%- endif %}\n> +\n> +class {{proxy_worker_name}}\n> +{\n> +public:\n> +\t{{proxy_worker_name}}()\n> +\t\t: ipa_(nullptr), exit_(false) {}\n> +\n> +\t~{{proxy_worker_name}}() {}\n> +\n> +\tvoid readyRead(IPCUnixSocket *socket)\n> +\t{\n> +\t\tIPCUnixSocket::Payload _message;\n> +\t\tint _retRecv = socket->receive(&_message);\n> +\t\tif (_retRecv) {\n> +\t\t\tLOG({{proxy_worker_name}}, Error)\n> +\t\t\t\t<< \"Receive message failed\" << _retRecv;\n\ns/failed/failed: /\n\n> +\t\t\treturn;\n> +\t\t}\n> +\n> +\t\tIPCMessage _ipcMessage(_message);\n> +\n> +\t\t{{cmd_enum_name}} _cmd = static_cast<{{cmd_enum_name}}>(_ipcMessage.header().cmd);\n> +\n> +\t\tswitch (_cmd) {\n> +\t\tcase {{cmd_enum_name}}::Exit: {\n> +\t\t\texit_ = true;\n> +\t\t\tbreak;\n> +\t\t}\n> +\n> +{% for method in interface_main.methods %}\n> +\t\tcase {{cmd_enum_name}}::{{method.mojom_name|cap}}: {\n> +\t\t{{proxy_funcs.deserialize_call(method|method_param_inputs, '_ipcMessage.data()', '_ipcMessage.fds()', false, true)|indent(8, true)}}\n> +{% for param in method|method_param_outputs %}\n> +\t\t\t{{param|name}} {{param.mojom_name}};\n> +{% endfor %}\n> +{%- if method|method_return_value != \"void\" %}\n> +\t\t\t{{method|method_return_value}} _callRet = ipa_->{{method.mojom_name}}({{method.parameters|params_comma_sep}});\n> +{%- else %}\n> +\t\t\tipa_->{{method.mojom_name}}({{method.parameters|params_comma_sep}}\n> +{{- \", \" if method|method_param_outputs|params_comma_sep -}}\n> +{%- for param in method|method_param_outputs -%}\n> +&{{param.mojom_name}}{{\", \" if not loop.last}}\n> +{%- endfor -%}\n> +);\n> +{%- endif %}\n> +{% if not method|is_async %}\n> +\t\t\tIPCMessage::Header header = { _ipcMessage.header().cmd, _ipcMessage.header().cookie };\n> +\t\t\tIPCMessage _response(header);\n> +{%- if method|method_return_value != \"void\" %}\n> +\t\t\tstd::vector _callRetBuf;\n> +\t\t\tstd::tie(_callRetBuf, std::ignore) =\n> +\t\t\t\tIPADataSerializer<{{method|method_return_value}}>::serialize(_callRet);\n> +\t\t\t_response.data().insert(_response.data().end(), _callRetBuf.cbegin(), _callRetBuf.cend());\n> +{%- else %}\n> +\t\t{{proxy_funcs.serialize_call(method|method_param_outputs, \"_response.data()\", \"_response.fds()\")|indent(16, true)}}\n> +{%- endif %}\n> +\t\t\tint _ret = socket_.send(_response.payload());\n> +\t\t\tif (_ret < 0) {\n> +\t\t\t\tLOG({{proxy_worker_name}}, Error)\n> +\t\t\t\t\t<< \"Reply to {{method.mojom_name}}() failed\" << _ret;\n\ns/failed/failed: /\n\n> +\t\t\t}\n> +\t\t\tLOG({{proxy_worker_name}}, Debug) << \"Done replying to {{method.mojom_name}}()\";\n> +{%- endif %}\n> +\t\t\tbreak;\n> +\t\t}\n> +{% endfor %}\n> +\t\tdefault:\n> +\t\t\tLOG({{proxy_worker_name}}, Error) << \"Unknown command \" << _ipcMessage.header().cmd;\n> +\t\t}\n> +\t}\n> +\n> +\tint init(std::unique_ptr &ipam, int socketfd)\n> +\t{\n> +\t\tif (socket_.bind(socketfd) < 0) {\n> +\t\t\tLOG({{proxy_worker_name}}, Error)\n> +\t\t\t\t<< \"IPC socket binding failed\";\n> +\t\t\treturn EXIT_FAILURE;\n> +\t\t}\n> +\t\tsocket_.readyRead.connect(this, &{{proxy_worker_name}}::readyRead);\n> +\n> +\t\tipa_ = dynamic_cast<{{interface_name}} *>(ipam->createInterface());\n> +\t\tif (!ipa_) {\n> +\t\t\tLOG({{proxy_worker_name}}, Error)\n> +\t\t\t\t<< \"Failed to create IPA interface instance\";\n> +\t\t\treturn EXIT_FAILURE;\n> +\t\t}\n> +{% for method in interface_event.methods %}\n> +\t\tipa_->{{method.mojom_name}}.connect(this, &{{proxy_worker_name}}::{{method.mojom_name}});\n> +{%- endfor %}\n> +\t\treturn 0;\n> +\t}\n> +\n> +\tvoid run()\n> +\t{\n> +\t\tEventDispatcher *dispatcher = Thread::current()->eventDispatcher();\n> +\t\twhile (!exit_)\n> +\t\t\tdispatcher->processEvents();\n> +\t}\n> +\n> +\tvoid cleanup()\n> +\t{\n> +\t\tdelete ipa_;\n> +\t\tsocket_.close();\n> +\t}\n> +\n> +private:\n> +\n> +{% for method in interface_event.methods %}\n> +{{proxy_funcs.func_sig(proxy_name, method, \"\", false)|indent(8, true)}}\n> +\t{\n> +\t\tIPCMessage::Header header = {\n> +\t\t\tstatic_cast({{cmd_event_enum_name}}::{{method.mojom_name|cap}}),\n> +\t\t\t0\n> +\t\t};\n> +\t\tIPCMessage _message(header);\n> +\n> +\t\t{{proxy_funcs.serialize_call(method|method_param_inputs, \"_message.data()\", \"_message.fds()\")}}\n> +\n> +\t\tsocket_.send(_message.payload());\n> +\n> +\t\tLOG({{proxy_worker_name}}, Debug) << \"{{method.mojom_name}} done\";\n> +\t}\n> +{% endfor %}\n> +\n> +\t{{interface_name}} *ipa_;\n> +\tIPCUnixSocket socket_;\n> +\n> +\tControlSerializer controlSerializer_;\n> +\n> +\tbool exit_;\n> +};\n> +\n> +int main(int argc, char **argv)\n> +{\n> +{#- \\todo Handle enabling debugging more dynamically. #}\n> +\t/* Uncomment this for debugging. */\n\nIt's not commented :-)\n\n> +\tstd::string logPath = \"/tmp/libcamera.worker.\" +\n> +\t\t\t std::to_string(getpid()) + \".log\";\n> +\tlogSetFile(logPath.c_str());\n> +\n> +\tif (argc < 3) {\n> +\t\tLOG({{proxy_worker_name}}, Error)\n> +\t\t\t<< \"Tried to start worker with no args: \"\n> +\t\t\t<< \"expected \";\n> +\t\treturn EXIT_FAILURE;\n> +\t}\n> +\n> +\tint fd = std::stoi(argv[2]);\n> +\tLOG({{proxy_worker_name}}, Info)\n> +\t\t<< \"Starting worker for IPA module \" << argv[1]\n> +\t\t<< \" with IPC fd = \" << fd;\n> +\n> +\tstd::unique_ptr ipam = std::make_unique(argv[1]);\n> +\tif (!ipam->isValid() || !ipam->load()) {\n> +\t\tLOG({{proxy_worker_name}}, Error)\n> +\t\t\t<< \"IPAModule \" << argv[1] << \" isn't valid\";\n> +\t\treturn EXIT_FAILURE;\n> +\t}\n> +\n> +\t{{proxy_worker_name}} proxyWorker;\n> +\tint ret = proxyWorker.init(ipam, fd);\n> +\tif (ret < 0) {\n> +\t\tLOG({{proxy_worker_name}}, Error)\n> +\t\t\t<< \"Failed to initialize proxy worker\";\n> +\t\treturn ret;\n> +\t}\n> +\n> +\tLOG({{proxy_worker_name}}, Debug) << \"Proxy worker successfully started\";\n\ns/started/initialized/ ?\n\n> +\n> +\tproxyWorker.run();\n> +\n> +\tproxyWorker.cleanup();\n> +\n> +\treturn 0;\n> +}\n> diff --git a/utils/ipc/generators/libcamera_templates/module_ipa_serializer.h.tmpl b/utils/ipc/generators/libcamera_templates/module_ipa_serializer.h.tmpl\n> new file mode 100644\n> index 00000000..ad522964\n> --- /dev/null\n> +++ b/utils/ipc/generators/libcamera_templates/module_ipa_serializer.h.tmpl\n> @@ -0,0 +1,47 @@\n> +{#-\n> + # SPDX-License-Identifier: LGPL-2.1-or-later\n> + # Copyright (C) 2020, Google Inc.\n> +-#}\n> +{%- import \"serializer.tmpl\" as serializer -%}\n> +\n> +/* SPDX-License-Identifier: LGPL-2.1-or-later */\n> +/*\n> + * Copyright (C) 2020, Google Inc.\n> + *\n> + * {{module_name}}_ipa_serializer.h - Image Processing Algorithm data serializer for {{module_name}}\n> + *\n> + * This file is auto-generated. Do not edit.\n> + */\n> +\n> +#ifndef __LIBCAMERA_INTERNAL_IPA_DATA_SERIALIZER_{{module_name|upper}}_H__\n> +#define __LIBCAMERA_INTERNAL_IPA_DATA_SERIALIZER_{{module_name|upper}}_H__\n> +\n> +#include \n> +#include \n> +\n> +#include \n> +#include \n> +\n> +#include \"libcamera/internal/control_serializer.h\"\n> +#include \"libcamera/internal/ipa_data_serializer.h\"\n> +\n> +namespace libcamera {\n> +\n> +LOG_DECLARE_CATEGORY(IPADataSerializer)\n> +{% for struct in structs_nonempty %}\n> +template<>\n> +class IPADataSerializer<{{struct|name_full(namespace_str)}}>\n> +{\n> +public:\n> +{{- serializer.serializer(struct, namespace_str)}}\n> +{%- if struct|has_fd %}\n> +{{serializer.deserializer_fd(struct, namespace_str)}}\n> +{%- else %}\n> +{{serializer.deserializer_no_fd(struct, namespace_str)}}\n> +{%- endif %}\n> +};\n> +{% endfor %}\n> +\n> +} /* namespace libcamera */\n> +\n> +#endif /* __LIBCAMERA_INTERNAL_IPA_DATA_SERIALIZER_{{module_name|upper}}_H__ */\n> diff --git a/utils/ipc/generators/libcamera_templates/proxy_functions.tmpl b/utils/ipc/generators/libcamera_templates/proxy_functions.tmpl\n> new file mode 100644\n> index 00000000..3f2143b1\n> --- /dev/null\n> +++ b/utils/ipc/generators/libcamera_templates/proxy_functions.tmpl\n> @@ -0,0 +1,192 @@\n> +{#-\n> + # SPDX-License-Identifier: LGPL-2.1-or-later\n> + # Copyright (C) 2020, Google Inc.\n> +-#}\n> +{#\n> + # \\brief Generate fuction prototype\n\ns/fuction/function/\n\n> + #\n> + # \\param class Class name\n> + # \\param method mojom Method object\n> + # \\param suffix Suffix to append to \\a method function name\n> + # \\param need_class_name If true, generate class name with function\n> + # \\param override If true, generate override tag after the function prototype\n> + #}\n> +{%- macro func_sig(class, method, suffix = \"\", need_class_name = true, override = false) -%}\n> +{{method|method_return_value}} {{class + \"::\" if need_class_name}}{{method.mojom_name}}{{suffix}}(\n> +{%- for param in method|method_parameters %}\n> +\t{{param}}{{- \",\" if not loop.last}}\n> +{%- endfor -%}\n> +){{\" override\" if override}}\n> +{%- endmacro -%}\n> +\n> +{#\n> + # \\brief Generate function body for IPA init() function for thread\n> + #}\n> +{%- macro init_thread_body() -%}\n> +\tint ret = ipa_->init(settings);\n> +\tif (ret)\n> +\t\treturn ret;\n> +\n> +\tproxy_.moveToThread(&thread_);\n> +\n> +\treturn 0;\n> +{%- endmacro -%}\n> +\n> +{#\n> + # \\brief Generate function body for IPA stop() function for thread\n> + #}\n> +{%- macro stop_thread_body() -%}\n> +\tif (!running_)\n> +\t\treturn;\n> +\n> +\trunning_ = false;\n> +\n> +\tproxy_.invokeMethod(&ThreadProxy::stop, ConnectionTypeBlocking);\n> +\n> +\tthread_.exit();\n> +\tthread_.wait();\n> +{%- endmacro -%}\n> +\n> +\n> +{#\n> + # \\brief Serialize multiple objects into data buffer and fd vector\n> + #\n> + # Generate code to serialize multiple objects, as specified in \\a params\n> + # (which are the parameters to some function), into \\a buf data buffer and\n> + # \\a fds fd vector.\n> + # This code is meant to be used by the proxy, for serializing prior to IPC calls.\n> + #}\n> +{%- macro serialize_call(params, buf, fds) %}\n> +{%- for param in params %}\n> +\tstd::vector {{param.mojom_name}}Buf;\n\nThere's room for optimization here too,\n\n # \\todo Avoid intermediate vectors\n\nI'm not sure how such optimization would work, but given that we know\nwhat we're serializing, I'm confident we could do something. One useful\ntask would be to see if using the same binary format as mojo could help\n(I assume it has been designed with performance in mind).\n\n> +{%- if param|has_fd %}\n> +\tstd::vector {{param.mojom_name}}Fds;\n> +\tstd::tie({{param.mojom_name}}Buf, {{param.mojom_name}}Fds) =\n> +{%- else %}\n> +\tstd::tie({{param.mojom_name}}Buf, std::ignore) =\n> +{%- endif %}\n> +\t\tIPADataSerializer<{{param|name}}>::serialize({{param.mojom_name}}\n> +{{- \", &controlSerializer_\" if param|needs_control_serializer -}}\n> +);\n> +{%- endfor %}\n> +\n> +{%- if params|length > 1 %}\n> +{%- for param in params %}\n> +\tappendPOD({{buf}}, {{param.mojom_name}}Buf.size());\n> +{%- if param|has_fd %}\n> +\tappendPOD({{buf}}, {{param.mojom_name}}Fds.size());\n> +{%- endif %}\n> +{%- endfor %}\n> +{%- endif %}\n> +\n> +{%- for param in params %}\n> +\t{{buf}}.insert({{buf}}.end(), {{param.mojom_name}}Buf.begin(), {{param.mojom_name}}Buf.end());\n> +{%- endfor %}\n> +\n> +{%- for param in params %}\n> +{%- if param|has_fd %}\n> +\t{{fds}}.insert({{fds}}.end(), {{param.mojom_name}}Fds.begin(), {{param.mojom_name}}Fds.end());\n> +{%- endif %}\n> +{%- endfor %}\n> +{%- endmacro -%}\n> +\n> +\n> +{#\n> + # \\brief Deserialize a single object from data buffer and fd vector\n> + #\n> + # \\param pointer If true, deserialize the object into a dereferenced pointer\n> + # \\param iter If true, treat \\a buf as an iterator instead of a vector\n> + # \\param data_size Variable that holds the size of the vector referenced by \\a buf\n> + #\n> + # Generate code to deserialize a single object, as specified in \\a param,\n> + # from \\a buf data buffer and \\a fds fd vector.\n> + # This code is meant to be used by macro deserialize_call.\n> + #}\n> +{%- macro deserialize_param(param, pointer, loop, buf, fds, iter, data_size) -%}\n> +{{\"*\" if pointer}}{{param.mojom_name}} = IPADataSerializer<{{param|name}}>::deserialize(\n> +\t{{buf}}{{- \".cbegin()\" if not iter}} + {{param.mojom_name}}Start,\n> +{%- if loop.last and not iter %}\n> +\t{{buf}}.cend()\n> +{%- elif not iter %}\n> +\t{{buf}}.cbegin() + {{param.mojom_name}}Start + {{param.mojom_name}}BufSize\n> +{%- elif iter and loop.length == 1 %}\n> +\t{{buf}} + {{data_size}}\n> +{%- else %}\n> +\t{{buf}} + {{param.mojom_name}}Start + {{param.mojom_name}}BufSize\n> +{%- endif -%}\n> +{{- \",\" if param|has_fd}}\n> +{%- if param|has_fd %}\n> +\t{{fds}}.cbegin() + {{param.mojom_name}}FdStart,\n> +{%- if loop.last %}\n> +\t{{fds}}.cend()\n> +{%- else %}\n> +\t{{fds}}.cbegin() + {{param.mojom_name}}FdStart + {{param.mojom_name}}FdsSize\n> +{%- endif -%}\n> +{%- endif -%}\n> +{{- \",\" if param|needs_control_serializer}}\n> +{%- if param|needs_control_serializer %}\n> +\t&controlSerializer_\n> +{%- endif -%}\n> +);\n> +{%- endmacro -%}\n> +\n> +\n> +{#\n> + # \\brief Deserialize multiple objects from data buffer and fd vector\n> + #\n> + # \\param pointer If true, deserialize objects into pointers, and adds a null check.\n> + # \\param declare If true, declare the objects in addition to deserialization.\n> + # \\param iter if true, treat \\a buf as an iterator instead of a vector\n> + # \\param data_size Variable that holds the size of the vector referenced by \\a buf\n> + #\n> + # Generate code to deserialize multiple objects, as specified in \\a params\n> + # (which are the parameters to some function), from \\a buf data buffer and\n> + # \\a fds fd vector.\n> + # This code is meant to be used by the proxy, for deserializing after IPC calls.\n> + #\n> + # \\todo Avoid intermediate vectors\n> + #}\n> +{%- macro deserialize_call(params, buf, fds, pointer = true, declare = false, iter = false, data_size = '') -%}\n> +{% set ns = namespace(size_offset = 0) %}\n> +{%- if params|length > 1 %}\n> +{%- for param in params %}\n> +\t[[maybe_unused]] const size_t {{param.mojom_name}}BufSize = readPOD({{buf}}, {{ns.size_offset}}\n\nExtra space before const.\n\n> +{%- if iter -%}\n> +, {{buf}} + {{data_size}}\n> +{%- endif -%}\n> +);\n> +\t{%- set ns.size_offset = ns.size_offset + 4 %}\n> +{%- if param|has_fd %}\n> +\t[[maybe_unused]] const size_t {{param.mojom_name}}FdsSize = readPOD({{buf}}, {{ns.size_offset}}\n> +{%- if iter -%}\n> +, {{buf}} + {{data_size}}\n> +{%- endif -%}\n> +);\n> +\t{%- set ns.size_offset = ns.size_offset + 4 %}\n> +{%- endif %}\n> +{%- endfor %}\n> +{%- endif %}\n> +{% for param in params %}\n> +{%- if loop.first %}\n> +\tconst size_t {{param.mojom_name}}Start = {{ns.size_offset}};\n> +{%- else %}\n> +\tconst size_t {{param.mojom_name}}Start = {{loop.previtem.mojom_name}}Start + {{loop.previtem.mojom_name}}BufSize;\n> +{%- endif %}\n> +{%- endfor %}\n> +{% for param in params|with_fds %}\n> +{%- if loop.first %}\n> +\tconst size_t {{param.mojom_name}}FdStart = 0;\n> +{%- elif not loop.last %}\n> +\tconst size_t {{param.mojom_name}}FdStart = {{loop.previtem.mojom_name}}FdStart + {{loop.previtem.mojom_name}}FdsSize;\n> +{%- endif %}\n> +{%- endfor %}\n> +{% for param in params %}\n> +\t{%- if pointer %}\n> +\tif ({{param.mojom_name}}) {\n> +{{deserialize_param(param, pointer, loop, buf, fds, iter, data_size)|indent(16, True)}}\n> +\t}\n> +\t{%- else %}\n> +\t{{param|name + \" \" if declare}}{{deserialize_param(param, pointer, loop, buf, fds, iter, data_size)|indent(8)}}\n> +\t{%- endif %}\n> +{% endfor %}\n> +{%- endmacro -%}\n> diff --git a/utils/ipc/generators/libcamera_templates/serializer.tmpl b/utils/ipc/generators/libcamera_templates/serializer.tmpl\n> new file mode 100644\n> index 00000000..af4800bf\n> --- /dev/null\n> +++ b/utils/ipc/generators/libcamera_templates/serializer.tmpl\n> @@ -0,0 +1,313 @@\n> +{#-\n> + # SPDX-License-Identifier: LGPL-2.1-or-later\n> + # Copyright (C) 2020, Google Inc.\n> +-#}\n> +{# Turn this into a C macro? #}\n> +{#\n> + # \\brief Verify that there is enough bytes to deserialize\n> + #\n> + # Generate code that verifies that \\a size is not greater than \\a dataSize.\n> + # Otherwise log an error with \\a name and \\a typename.\n> + #}\n> +{%- macro check_data_size(size, dataSize, name, typename) %}\n> +\t\tif ({{dataSize}} < {{size}}) {\n> +\t\t\tLOG(IPADataSerializer, Error)\n> +\t\t\t\t<< \"Failed to deserialize \" << \"{{name}}\"\n> +\t\t\t\t<< \": not enough {{typename}}, expected \"\n> +\t\t\t\t<< ({{size}}) << \", got \" << ({{dataSize}});\n> +\t\t\treturn ret;\n> +\t\t}\n> +{%- endmacro %}\n> +\n> +\n> +{#\n> + # \\brief Serialize some field into return vector\n> + #\n> + # Generate code to serialize \\a field into retData, including size of the\n> + # field and fds (where appropriate).\n> + # This code is meant to be used by the IPADataSerializer specialization.\n> + #\n> + # \\todo Avoid intermediate vectors\n> + #}\n> +{%- macro serializer_field(field, namespace, loop) %}\n> +{%- if field|is_pod or field|is_enum %}\n> +\t\tstd::vector {{field.mojom_name}};\n> +\t\tstd::tie({{field.mojom_name}}, std::ignore) =\n> +\t{%- if field|is_pod %}\n> +\t\t\tIPADataSerializer<{{field|name}}>::serialize(data.{{field.mojom_name}});\n> +\t{%- elif field|is_enum %}\n> +\t\t\tIPADataSerializer::serialize(data.{{field.mojom_name}});\n> +\t{%- endif %}\n> +\t\tretData.insert(retData.end(), {{field.mojom_name}}.begin(), {{field.mojom_name}}.end());\n> +{%- elif field|is_fd %}\n> +\t\tstd::vector {{field.mojom_name}};\n> +\t\tstd::vector {{field.mojom_name}}Fds;\n> +\t\tstd::tie({{field.mojom_name}}, {{field.mojom_name}}Fds) =\n> +\t\t\tIPADataSerializer<{{field|name}}>::serialize(data.{{field.mojom_name}});\n> +\t\tretData.insert(retData.end(), {{field.mojom_name}}.begin(), {{field.mojom_name}}.end());\n> +\t\tretFds.insert(retFds.end(), {{field.mojom_name}}Fds.begin(), {{field.mojom_name}}Fds.end());\n> +{%- elif field|is_controls %}\n> +\t\tif (data.{{field.mojom_name}}.size() > 0) {\n> +\t\t\tstd::vector {{field.mojom_name}};\n> +\t\t\tstd::tie({{field.mojom_name}}, std::ignore) =\n> +\t\t\t\tIPADataSerializer<{{field|name}}>::serialize(data.{{field.mojom_name}}, cs);\n> +\t\t\tappendPOD(retData, {{field.mojom_name}}.size());\n> +\t\t\tretData.insert(retData.end(), {{field.mojom_name}}.begin(), {{field.mojom_name}}.end());\n> +\t\t} else {\n> +\t\t\tappendPOD(retData, 0);\n> +\t\t}\n> +{%- elif field|is_plain_struct or field|is_array or field|is_map or field|is_str %}\n> +\t\tstd::vector {{field.mojom_name}};\n> +\t{%- if field|has_fd %}\n> +\t\tstd::vector {{field.mojom_name}}Fds;\n> +\t\tstd::tie({{field.mojom_name}}, {{field.mojom_name}}Fds) =\n> +\t{%- else %}\n> +\t\tstd::tie({{field.mojom_name}}, std::ignore) =\n> +\t{%- endif %}\n> +\t{%- if field|is_array or field|is_map %}\n> +\t\t\tIPADataSerializer<{{field|name}}>::serialize(data.{{field.mojom_name}}, cs);\n> +\t{%- elif field|is_str %}\n> +\t\t\tIPADataSerializer<{{field|name}}>::serialize(data.{{field.mojom_name}});\n> +\t{%- else %}\n> +\t\t\tIPADataSerializer<{{field|name_full(namespace)}}>::serialize(data.{{field.mojom_name}}, cs);\n> +\t{%- endif %}\n> +\t\tappendPOD(retData, {{field.mojom_name}}.size());\n> +\t{%- if field|has_fd %}\n> +\t\tappendPOD(retData, {{field.mojom_name}}Fds.size());\n> +\t{%- endif %}\n> +\t\tretData.insert(retData.end(), {{field.mojom_name}}.begin(), {{field.mojom_name}}.end());\n> +\t{%- if field|has_fd %}\n> +\t\tretFds.insert(retFds.end(), {{field.mojom_name}}Fds.begin(), {{field.mojom_name}}Fds.end());\n> +\t{%- endif %}\n> +{%- else %}\n> +\t\t/* Unknown serialization for {{field.mojom_name}}. */\n> +{%- endif %}\n> +{%- endmacro %}\n> +\n> +\n> +{#\n> + # \\brief Deserialize some field into return struct\n> + #\n> + # Generate code to deserialize \\a field into object ret.\n> + # This code is meant to be used by the IPADataSerializer specialization.\n> + #}\n> +{%- macro deserializer_field(field, namespace, loop) %}\n> +{% if field|is_pod or field|is_enum %}\n> +\t{%- set field_size = (field|bit_width|int / 8)|int %}\n> +\t\t{{- check_data_size(field_size, 'dataSize', field.mojom_name, 'data')}}\n> +\t\t{%- if field|is_pod %}\n> +\t\tret.{{field.mojom_name}} = IPADataSerializer<{{field|name}}>::deserialize(m, m + {{field_size}});\n> +\t\t{%- else %}\n> +\t\tret.{{field.mojom_name}} = static_cast<{{field|name_full(namespace)}}>(IPADataSerializer::deserialize(m, m + {{field_size}}));\n> +\t\t{%- endif %}\n> +\t{%- if not loop.last %}\n> +\t\tm += {{field_size}};\n> +\t\tdataSize -= {{field_size}};\n> +\t{%- endif %}\n> +{% elif field|is_fd %}\n> +\t{%- set field_size = 1 %}\n> +\t\t{{- check_data_size(field_size, 'dataSize', field.mojom_name, 'data')}}\n> +\t\tret.{{field.mojom_name}} = IPADataSerializer<{{field|name}}>::deserialize(m, m + 1, n, n + 1, cs);\n> +\t{%- if not loop.last %}\n> +\t\tm += {{field_size}};\n> +\t\tdataSize -= {{field_size}};\n> +\t\tn += ret.{{field.mojom_name}}.isValid() ? 1 : 0;\n> +\t\tfdsSize -= ret.{{field.mojom_name}}.isValid() ? 1 : 0;\n> +\t{%- endif %}\n> +{% elif field|is_controls %}\n> +\t{%- set field_size = 4 %}\n> +\t\t{{- check_data_size(field_size, 'dataSize', field.mojom_name + 'Size', 'data')}}\n> +\t\tconst size_t {{field.mojom_name}}Size = readPOD(m, 0, dataEnd);\n> +\t\tm += {{field_size}};\n> +\t\tdataSize -= {{field_size}};\n> +\t{%- set field_size = field.mojom_name + 'Size' -%}\n> +\t\t{{- check_data_size(field_size, 'dataSize', field.mojom_name, 'data')}}\n> +\t\tif ({{field.mojom_name}}Size > 0)\n> +\t\t\tret.{{field.mojom_name}} =\n> +\t\t\t\tIPADataSerializer<{{field|name}}>::deserialize(m, m + {{field.mojom_name}}Size, cs);\n> +\t{%- if not loop.last %}\n> +\t\tm += {{field_size}};\n> +\t\tdataSize -= {{field_size}};\n> +\t{%- endif %}\n> +{% elif field|is_plain_struct or field|is_array or field|is_map or field|is_str %}\n> +\t{%- set field_size = 4 %}\n> +\t\t{{- check_data_size(field_size, 'dataSize', field.mojom_name + 'Size', 'data')}}\n> +\t\tconst size_t {{field.mojom_name}}Size = readPOD(m, 0, dataEnd);\n> +\t\tm += {{field_size}};\n> +\t\tdataSize -= {{field_size}};\n> +\t{%- if field|has_fd %}\n> +\t{%- set field_size = 4 %}\n> +\t\t{{- check_data_size(field_size, 'dataSize', field.mojom_name + 'FdsSize', 'data')}}\n> +\t\tconst size_t {{field.mojom_name}}FdsSize = readPOD(m, 0, dataEnd);\n> +\t\tm += {{field_size}};\n> +\t\tdataSize -= {{field_size}};\n> +\t\t{{- check_data_size(field.mojom_name + 'FdsSize', 'fdsSize', field.mojom_name, 'fds')}}\n> +\t{%- endif %}\n> +\t{%- set field_size = field.mojom_name + 'Size' -%}\n> +\t\t{{- check_data_size(field_size, 'dataSize', field.mojom_name, 'data')}}\n> +\t\tret.{{field.mojom_name}} =\n> +\t{%- if field|is_str %}\n> +\t\t\tIPADataSerializer<{{field|name}}>::deserialize(m, m + {{field.mojom_name}}Size);\n> +\t{%- elif field|has_fd and (field|is_array or field|is_map) %}\n> +\t\t\tIPADataSerializer<{{field|name}}>::deserialize(m, m + {{field.mojom_name}}Size, n, n + {{field.mojom_name}}FdsSize, cs);\n> +\t{%- elif field|has_fd and (not (field|is_array or field|is_map)) %}\n> +\t\t\tIPADataSerializer<{{field|name_full(namespace)}}>::deserialize(m, m + {{field.mojom_name}}Size, n, n + {{field.mojom_name}}FdsSize, cs);\n> +\t{%- elif (not field|has_fd) and (field|is_array or field|is_map) %}\n> +\t\t\tIPADataSerializer<{{field|name}}>::deserialize(m, m + {{field.mojom_name}}Size, cs);\n> +\t{%- else %}\n> +\t\t\tIPADataSerializer<{{field|name_full(namespace)}}>::deserialize(m, m + {{field.mojom_name}}Size, cs);\n> +\t{%- endif %}\n> +\t{%- if not loop.last %}\n> +\t\tm += {{field_size}};\n> +\t\tdataSize -= {{field_size}};\n> +\t{%- if field|has_fd %}\n> +\t\tn += {{field.mojom_name}}FdsSize;\n> +\t\tfdsSize -= {{field.mojom_name}}FdsSize;\n> +\t{%- endif %}\n> +\t{%- endif %}\n> +{% else %}\n> +\t\t/* Unknown deserialization for {{field.mojom_name}}. */\n> +{%- endif %}\n> +{%- endmacro %}\n> +\n> +\n> +{#\n> + # \\brief Serialize a struct\n> + #\n> + # Generate code for IPADataSerializer specialization, for serializing\n> + # \\a struct.\n> + #}\n> +{%- macro serializer(struct, namespace) %}\n> +\tstatic std::tuple, std::vector>\n> +\tserialize(const {{struct|name_full(namespace)}} &data,\n> +{%- if struct|needs_control_serializer %}\n> +\t\t ControlSerializer *cs)\n> +{%- else %}\n> +\t\t [[maybe_unused]] ControlSerializer *cs = nullptr)\n> +{%- endif %}\n> +\t{\n> +\t\tstd::vector retData;\n> +{%- if struct|has_fd %}\n> +\t\tstd::vector retFds;\n> +{%- endif %}\n> +{%- for field in struct.fields %}\n> +{{serializer_field(field, namespace, loop)}}\n> +{%- endfor %}\n> +{% if struct|has_fd %}\n> +\t\treturn {retData, retFds};\n> +{%- else %}\n> +\t\treturn {retData, {}};\n> +{%- endif %}\n> +\t}\n> +{%- endmacro %}\n> +\n> +\n> +{#\n> + # \\brief Deserialize a struct that has fds\n> + #\n> + # Generate code for IPADataSerializer specialization, for deserializing\n> + # \\a struct, in the case that \\a struct has file descriptors.\n> + # fd parameters\n\nLeftover ?\n\n> + #}\n> +{%- macro deserializer_fd(struct, namespace) %}\n> +\tstatic {{struct|name_full(namespace)}}\n> +\tdeserialize(std::vector &data,\n> +\t\t std::vector &fds,\n> +{%- if struct|needs_control_serializer %}\n> +\t\t ControlSerializer *cs)\n> +{%- else %}\n> +\t\t [[maybe_unused]] ControlSerializer *cs = nullptr)\n> +{%- endif %}\n> +\t{\n> +\t\treturn IPADataSerializer<{{struct|name_full(namespace)}}>::deserialize(data.cbegin(), data.cend(), fds.cbegin(), fds.cend(), cs);\n> +\t}\n> +\n\nThe next function can be quite large, inlining it isn't great. You don't\nhave to fix this now, but a \"\\todo Don't inline this function\" would be\nAgood. Same for the \"no fds\" version.\n\n> +\tstatic {{struct|name_full(namespace)}}\n> +\tdeserialize(std::vector::const_iterator dataBegin,\n> +\t\t std::vector::const_iterator dataEnd,\n> +\t\t std::vector::const_iterator fdsBegin,\n> +\t\t std::vector::const_iterator fdsEnd,\n> +{%- if struct|needs_control_serializer %}\n> +\t\t ControlSerializer *cs)\n> +{%- else %}\n> +\t\t [[maybe_unused]] ControlSerializer *cs = nullptr)\n> +{%- endif %}\n> +\t{\n> +\t\t{{struct|name_full(namespace)}} ret;\n> +\t\tstd::vector::const_iterator m = dataBegin;\n> +\t\tstd::vector::const_iterator n = fdsBegin;\n> +\n> +\t\tsize_t dataSize = std::distance(dataBegin, dataEnd);\n> +\t\t[[maybe_unused]] size_t fdsSize = std::distance(fdsBegin, fdsEnd);\n> +{%- for field in struct.fields -%}\n> +{{deserializer_field(field, namespace, loop)}}\n> +{%- endfor %}\n> +\t\treturn ret;\n> +\t}\n> +{%- endmacro %}\n> +\n> +{#\n> + # \\brief Deserialize a struct that has fds, using non-fd\n> + #\n> + # Generate code for IPADataSerializer specialization, for deserializing\n> + # \\a struct, in the case that \\a struct has no file descriptors but requires\n> + # deserializers with file descriptors.\n> + #}\n> +{%- macro deserializer_fd_simple(struct, namespace) %}\n> +\tstatic {{struct|name_full(namespace)}}\n> +\tdeserialize(std::vector &data,\n> +\t\t [[maybe_unused]] std::vector &fds,\n> +\t\t [[maybe_unused]] ControlSerializer *cs = nullptr)\n\nGiven that you're passing cs to the deserialize() call on the next line,\ndo you need [[maybe_unused]] here ? Same below, and in a few other\nlocations.\n\n> +\t{\n> +\t\treturn IPADataSerializer<{{struct|name_full(namespace)}}>::deserialize(data.cbegin(), data.cend(), cs);\n> +\t}\n> +\n> +\tstatic {{struct|name_full(namespace)}}\n> +\tdeserialize(std::vector::const_iterator dataBegin,\n> +\t\t std::vector::const_iterator dataEnd,\n> +\t\t [[maybe_unused]] std::vector::const_iterator fdsBegin,\n> +\t\t [[maybe_unused]] std::vector::const_iterator fdsEnd,\n> +\t\t [[maybe_unused]] ControlSerializer *cs = nullptr)\n> +\t{\n> +\t\treturn IPADataSerializer<{{struct|name_full(namespace)}}>::deserialize(dataBegin, dataEnd, cs);\n> +\t}\n> +{%- endmacro %}\n> +\n> +\n> +{#\n> + # \\brief Deserialize a struct that has no fds\n> + #\n> + # Generate code for IPADataSerializer specialization, for deserializing\n> + # \\a struct, in the case that \\a struct does not have file descriptors.\n> + #}\n> +{%- macro deserializer_no_fd(struct, namespace) %}\n> +\tstatic {{struct|name_full(namespace)}}\n> +\tdeserialize(std::vector &data,\n> +{%- if struct|needs_control_serializer %}\n> +\t\t ControlSerializer *cs)\n> +{%- else %}\n> +\t\t [[maybe_unused]] ControlSerializer *cs = nullptr)\n> +{%- endif %}\n> +\t{\n> +\t\treturn IPADataSerializer<{{struct|name_full(namespace)}}>::deserialize(data.cbegin(), data.cend(), cs);\n> +\t}\n> +\n> +\tstatic {{struct|name_full(namespace)}}\n> +\tdeserialize(std::vector::const_iterator dataBegin,\n> +\t\t std::vector::const_iterator dataEnd,\n> +{%- if struct|needs_control_serializer %}\n> +\t\t ControlSerializer *cs)\n> +{%- else %}\n> +\t\t [[maybe_unused]] ControlSerializer *cs = nullptr)\n> +{%- endif %}\n> +\t{\n> +\t\t{{struct|name_full(namespace)}} ret;\n> +\t\tstd::vector::const_iterator m = dataBegin;\n> +\n> +\t\tsize_t dataSize = std::distance(dataBegin, dataEnd);\n> +{%- for field in struct.fields -%}\n> +{{deserializer_field(field, namespace, loop)}}\n> +{%- endfor %}\n> +\t\treturn ret;\n> +\t}\n> +{%- endmacro %}\n> diff --git a/utils/ipc/generators/mojom_libcamera_generator.py b/utils/ipc/generators/mojom_libcamera_generator.py\n> new file mode 100644\n> index 00000000..5c4ad4fe\n> --- /dev/null\n> +++ b/utils/ipc/generators/mojom_libcamera_generator.py\n> @@ -0,0 +1,511 @@\n> +#!/usr/bin/env python3\n> +# SPDX-License-Identifier: GPL-2.0-or-later\n> +# Copyright (C) 2020, Google Inc.\n> +#\n> +# Author: Paul Elder \n> +#\n> +# mojom_libcamera_generator.py - Generates libcamera files from a mojom.Module.\n> +\n> +import argparse\n> +import datetime\n> +import os\n> +import re\n> +\n> +import mojom.fileutil as fileutil\n> +import mojom.generate.generator as generator\n> +import mojom.generate.module as mojom\n> +from mojom.generate.template_expander import UseJinja\n> +\n> +\n> +GENERATOR_PREFIX = 'libcamera'\n> +\n> +_kind_to_cpp_type = {\n> + mojom.BOOL: 'bool',\n> + mojom.INT8: 'int8_t',\n> + mojom.UINT8: 'uint8_t',\n> + mojom.INT16: 'int16_t',\n> + mojom.UINT16: 'uint16_t',\n> + mojom.INT32: 'int32_t',\n> + mojom.UINT32: 'uint32_t',\n> + mojom.FLOAT: 'float',\n> + mojom.INT64: 'int64_t',\n> + mojom.UINT64: 'uint64_t',\n> + mojom.DOUBLE: 'double',\n> +}\n> +\n> +_bit_widths = {\n> + mojom.BOOL: '8',\n> + mojom.INT8: '8',\n> + mojom.UINT8: '8',\n> + mojom.INT16: '16',\n> + mojom.UINT16: '16',\n> + mojom.INT32: '32',\n> + mojom.UINT32: '32',\n> + mojom.FLOAT: '32',\n> + mojom.INT64: '64',\n> + mojom.UINT64: '64',\n> + mojom.DOUBLE: '64',\n> +}\n> +\n> +def ModuleName(path):\n> + return path.split('/')[-1].split('.')[0]\n> +\n> +def ModuleClassName(module):\n> + return re.sub(r'^IPA(.*)Interface$', lambda match: match.group(1),\n> + module.interfaces[0].mojom_name)\n> +\n> +def Capitalize(name):\n> + return name[0].upper() + name[1:]\n> +\n> +def ConstantStyle(name):\n> + return generator.ToUpperSnakeCase(name)\n> +\n> +def Choose(cond, t, f):\n> + return t if cond else f\n> +\n> +def CommaSep(l):\n> + return ', '.join([m for m in l])\n> +\n> +def ParamsCommaSep(l):\n> + return ', '.join([m.mojom_name for m in l])\n> +\n> +def GetDefaultValue(element):\n> + if element.default is not None:\n> + return element.default\n> + if type(element.kind) == mojom.Kind:\n> + return '0'\n> + if mojom.IsEnumKind(element.kind):\n> + return f'static_cast<{element.kind.mojom_name}>(0)'\n> + if isinstance(element.kind, mojom.Struct) and \\\n> + element.kind.mojom_name == 'FileDescriptor':\n> + return '-1'\n> + return ''\n> +\n> +def HasDefaultValue(element):\n> + return GetDefaultValue(element) != ''\n> +\n> +def HasDefaultFields(element):\n> + return True in [HasDefaultValue(x) for x in element.fields]\n> +\n> +def GetAllTypes(element):\n> + if mojom.IsArrayKind(element):\n> + return GetAllTypes(element.kind)\n> + if mojom.IsMapKind(element):\n> + return GetAllTypes(element.key_kind) + GetAllTypes(element.value_kind)\n> + if isinstance(element, mojom.Parameter):\n> + return GetAllTypes(element.kind)\n> + if mojom.IsEnumKind(element):\n> + return [element.mojom_name]\n> + if not mojom.IsStructKind(element):\n> + return [element.spec]\n> + if len(element.fields) == 0:\n> + return [element.mojom_name]\n> + ret = [GetAllTypes(x.kind) for x in element.fields]\n> + ret = [x for sublist in ret for x in sublist]\n> + return list(set(ret))\n> +\n> +def GetAllAttrs(element):\n> + if mojom.IsArrayKind(element):\n> + return GetAllAttrs(element.kind)\n> + if mojom.IsMapKind(element):\n> + return {**GetAllAttrs(element.key_kind), **GetAllAttrs(element.value_kind)}\n> + if isinstance(element, mojom.Parameter):\n> + return GetAllAttrs(element.kind)\n> + if mojom.IsEnumKind(element):\n> + return element.attributes if element.attributes is not None else {}\n> + if mojom.IsStructKind(element) and len(element.fields) == 0:\n> + return element.attributes if element.attributes is not None else {}\n> + if not mojom.IsStructKind(element):\n> + if hasattr(element, 'attributes'):\n> + return element.attributes or {}\n> + return {}\n> + attrs = [(x.attributes) for x in element.fields]\n> + ret = {}\n> + for d in attrs:\n> + ret.update(d or {})\n> + if hasattr(element, 'attributes'):\n> + ret.update(element.attributes or {})\n> + return ret\n> +\n> +def NeedsControlSerializer(element):\n> + types = GetAllTypes(element)\n> + return \"ControlList\" in types or \"ControlInfoMap\" in types\n> +\n> +def HasFd(element):\n> + attrs = GetAllAttrs(element)\n> + if isinstance(element, mojom.Kind):\n> + types = GetAllTypes(element)\n> + else:\n> + types = GetAllTypes(element.kind)\n> + return \"FileDescriptor\" in types or (attrs is not None and \"hasFd\" in attrs)\n> +\n> +def WithDefaultValues(element):\n> + return [x for x in element if HasDefaultValue(x)]\n> +\n> +def WithFds(element):\n> + return [x for x in element if HasFd(x)]\n> +\n> +def MethodParamInputs(method):\n> + return method.parameters\n> +\n> +def MethodParamOutputs(method):\n> + if (MethodReturnValue(method) != 'void' or\n> + method.response_parameters is None):\n\nNo need for outer parentheses.\n\n> + return []\n> + return method.response_parameters\n> +\n> +def MethodParamsHaveFd(parameters):\n> + return len([x for x in parameters if HasFd(x)]) > 0\n> +\n> +def MethodInputHasFd(method):\n> + return MethodParamsHaveFd(method.parameters)\n> +\n> +def MethodOutputHasFd(method):\n> + if (MethodReturnValue(method) != 'void' or\n> + method.response_parameters is None):\n\nSame here.\n\n> + return False\n> + return MethodParamsHaveFd(method.response_parameters)\n\nCould this be written\n\n return MethodParamHasFd(MethodParamOutputs(method))\n\n?\n\n> +\n> +def MethodParamNames(method):\n> + params = []\n> + for param in method.parameters:\n> + params.append(param.mojom_name)\n> + if MethodReturnValue(method) == 'void':\n> + if method.response_parameters is None:\n> + return params\n> + for param in method.response_parameters:\n> + params.append(param.mojom_name)\n> + return params\n> +\n> +def MethodParameters(method):\n> + params = []\n> + for param in method.parameters:\n> + params.append('const %s %s%s' % (GetNameForElement(param),\n> + '&' if not IsPod(param) else '',\n> + param.mojom_name))\n> + if MethodReturnValue(method) == 'void':\n> + if method.response_parameters is None:\n> + return params\n> + for param in method.response_parameters:\n> + params.append(f'{GetNameForElement(param)} *{param.mojom_name}')\n> + return params\n> +\n> +def MethodReturnValue(method):\n> + if method.response_parameters is None:\n> + return 'void'\n> + if len(method.response_parameters) == 1 and IsPod(method.response_parameters[0]):\n> + return GetNameForElement(method.response_parameters[0])\n> + return 'void'\n> +\n> +def IsAsync(method):\n> + # Events are always async\n> + if re.match(\"^IPA.*EventInterface$\", method.interface.mojom_name):\n> + return True\n> + elif re.match(\"^IPA.*Interface$\", method.interface.mojom_name):\n> + if method.attributes is None:\n> + return False\n> + elif 'async' in method.attributes and method.attributes['async']:\n> + return True\n> + return False\n> +\n> +def IsArray(element):\n> + return mojom.IsArrayKind(element.kind)\n> +\n> +def IsControls(element):\n> + return mojom.IsStructKind(element.kind) and (element.kind.mojom_name == \"ControlList\" or\n> + element.kind.mojom_name == \"ControlInfoMap\")\n> +\n> +def IsEnum(element):\n> + return mojom.IsEnumKind(element.kind)\n> +\n> +def IsFd(element):\n> + return mojom.IsStructKind(element.kind) and element.kind.mojom_name == \"FileDescriptor\"\n> +\n> +def IsMap(element):\n> + return mojom.IsMapKind(element.kind)\n> +\n> +def IsPlainStruct(element):\n> + return mojom.IsStructKind(element.kind) and not IsControls(element) and not IsFd(element)\n> +\n> +def IsPod(element):\n> + return element.kind in _kind_to_cpp_type\n> +\n> +def IsStr(element):\n> + return element.kind.spec == 's'\n> +\n> +def BitWidth(element):\n> + if element.kind in _bit_widths:\n> + return _bit_widths[element.kind]\n> + if mojom.IsEnumKind(element.kind):\n> + return '32'\n> + return ''\n> +\n> +# Get the type name for a given element\n> +def GetNameForElement(element):\n> + # structs\n> + if (mojom.IsEnumKind(element) or\n> + mojom.IsInterfaceKind(element) or\n> + mojom.IsStructKind(element)):\n\nYou really like your outer parentheses :-) Same below.\n\nReviewed-by: Laurent Pinchart \n\n> + return element.mojom_name\n> + # vectors\n> + if (mojom.IsArrayKind(element)):\n> + elem_name = GetNameForElement(element.kind)\n> + return f'std::vector<{elem_name}>'\n> + # maps\n> + if (mojom.IsMapKind(element)):\n> + key_name = GetNameForElement(element.key_kind)\n> + value_name = GetNameForElement(element.value_kind)\n> + return f'std::map<{key_name}, {value_name}>'\n> + # struct fields and function parameters\n> + if isinstance(element, (mojom.Field, mojom.Method, mojom.Parameter)):\n> + # maps and vectors\n> + if (mojom.IsArrayKind(element.kind) or mojom.IsMapKind(element.kind)):\n> + return GetNameForElement(element.kind)\n> + # strings\n> + if (mojom.IsReferenceKind(element.kind) and element.kind.spec == 's'):\n> + return 'std::string'\n> + # PODs\n> + if element.kind in _kind_to_cpp_type:\n> + return _kind_to_cpp_type[element.kind]\n> + # structs and enums\n> + return element.kind.mojom_name\n> + # PODs that are members of vectors/maps\n> + if (hasattr(element, '__hash__') and element in _kind_to_cpp_type):\n> + return _kind_to_cpp_type[element]\n> + if (hasattr(element, 'spec')):\n> + # strings that are members of vectors/maps\n> + if (element.spec == 's'):\n> + return 'std::string'\n> + # structs that aren't defined in mojom that are members of vectors/maps\n> + if (element.spec[0] == 'x'):\n> + return element.spec.replace('x:', '').replace('.', '::')\n> + if (mojom.IsInterfaceRequestKind(element) or\n> + mojom.IsAssociatedKind(element) or\n> + mojom.IsPendingRemoteKind(element) or\n> + mojom.IsPendingReceiverKind(element) or\n> + mojom.IsUnionKind(element)):\n> + raise Exception('Unsupported element: %s' % element)\n> + raise Exception('Unexpected element: %s' % element)\n> +\n> +def GetFullNameForElement(element, namespace_str):\n> + name = GetNameForElement(element)\n> + if namespace_str == '':\n> + return name\n> + return f'{namespace_str}::{name}'\n> +\n> +def ValidateZeroLength(l, s, cap=True):\n> + if l is None:\n> + return\n> + if len(l) > 0:\n> + raise Exception(f'{s.capitalize() if cap else s} should be empty')\n> +\n> +def ValidateSingleLength(l, s, cap=True):\n> + if len(l) > 1:\n> + raise Exception(f'Only one {s} allowed')\n> + if len(l) < 1:\n> + raise Exception(f'{s.capitalize() if cap else s} is required')\n> +\n> +def GetMainInterface(interfaces):\n> + intf = [x for x in interfaces\n> + if re.match(\"^IPA.*Interface\", x.mojom_name) and\n> + not re.match(\"^IPA.*EventInterface\", x.mojom_name)]\n> + ValidateSingleLength(intf, 'main interface')\n> + return None if len(intf) == 0 else intf[0]\n> +\n> +def GetEventInterface(interfaces):\n> + event = [x for x in interfaces if re.match(\"^IPA.*EventInterface\", x.mojom_name)]\n> + ValidateSingleLength(event, 'event interface')\n> + return None if len(event) == 0 else event[0]\n> +\n> +def ValidateNamespace(namespace):\n> + if namespace == '':\n> + raise Exception('Must have a namespace')\n> +\n> + if not re.match('^ipa\\.[0-9A-Za-z_]+', namespace):\n> + raise Exception('Namespace must be of the form \"ipa.{pipeline_name}\"')\n> +\n> +def ValidateInterfaces(interfaces):\n> + # Validate presence of main interface\n> + intf = GetMainInterface(interfaces)\n> + if intf is None:\n> + raise Exception('Must have main IPA interface')\n> +\n> + # Validate presence of event interface\n> + event = GetEventInterface(interfaces)\n> + if intf is None:\n> + raise Exception('Must have event IPA interface')\n> +\n> + # Validate required main interface functions\n> + f_init = [x for x in intf.methods if x.mojom_name == 'init']\n> + f_start = [x for x in intf.methods if x.mojom_name == 'start']\n> + f_stop = [x for x in intf.methods if x.mojom_name == 'stop']\n> +\n> + ValidateSingleLength(f_init, 'init()', False)\n> + ValidateSingleLength(f_start, 'start()', False)\n> + ValidateSingleLength(f_stop, 'stop()', False)\n> +\n> + f_init = f_init[0]\n> + f_start = f_start[0]\n> + f_stop = f_stop[0]\n> +\n> + # Validate parameters to init()\n> + ValidateSingleLength(f_init.parameters, 'input parameter to init()')\n> + ValidateSingleLength(f_init.response_parameters, 'output parameter from init()')\n> + if f_init.parameters[0].kind.mojom_name != 'IPASettings':\n> + raise Exception('init() must have single IPASettings input parameter')\n> + if f_init.response_parameters[0].kind.spec != 'i32':\n> + raise Exception('init() must have single int32 output parameter')\n> +\n> + # No need to validate start() as it is customizable\n> +\n> + # Validate parameters to stop()\n> + ValidateZeroLength(f_stop.parameters, 'input parameter to stop()')\n> + ValidateZeroLength(f_stop.parameters, 'output parameter from stop()')\n> +\n> + # Validate that event interface has at least one event\n> + if len(event.methods) < 1:\n> + raise Exception('Event interface must have at least one event')\n> +\n> + # Validate that all async methods don't have return values\n> + intf_methods_async = [x for x in intf.methods if IsAsync(x)]\n> + for method in intf_methods_async:\n> + ValidateZeroLength(method.response_parameters,\n> + f'{method.mojom_name} response parameters', False)\n> +\n> + event_methods_async = [x for x in event.methods if IsAsync(x)]\n> + for method in event_methods_async:\n> + ValidateZeroLength(method.response_parameters,\n> + f'{method.mojom_name} response parameters', False)\n> +\n> +class Generator(generator.Generator):\n> + @staticmethod\n> + def GetTemplatePrefix():\n> + return 'libcamera_templates'\n> +\n> + def GetFilters(self):\n> + libcamera_filters = {\n> + 'all_types': GetAllTypes,\n> + 'bit_width': BitWidth,\n> + 'cap': Capitalize,\n> + 'choose': Choose,\n> + 'comma_sep': CommaSep,\n> + 'default_value': GetDefaultValue,\n> + 'has_default_fields': HasDefaultFields,\n> + 'has_fd': HasFd,\n> + 'is_async': IsAsync,\n> + 'is_array': IsArray,\n> + 'is_controls': IsControls,\n> + 'is_enum': IsEnum,\n> + 'is_fd': IsFd,\n> + 'is_map': IsMap,\n> + 'is_plain_struct': IsPlainStruct,\n> + 'is_pod': IsPod,\n> + 'is_str': IsStr,\n> + 'method_input_has_fd': MethodInputHasFd,\n> + 'method_output_has_fd': MethodOutputHasFd,\n> + 'method_param_names': MethodParamNames,\n> + 'method_param_inputs': MethodParamInputs,\n> + 'method_param_outputs': MethodParamOutputs,\n> + 'method_parameters': MethodParameters,\n> + 'method_return_value': MethodReturnValue,\n> + 'name': GetNameForElement,\n> + 'name_full': GetFullNameForElement,\n> + 'needs_control_serializer': NeedsControlSerializer,\n> + 'params_comma_sep': ParamsCommaSep,\n> + 'with_default_values': WithDefaultValues,\n> + 'with_fds': WithFds,\n> + }\n> + return libcamera_filters\n> +\n> + def _GetJinjaExports(self):\n> + return {\n> + 'cmd_enum_name': '_%sCmd' % self.module_name,\n> + 'cmd_event_enum_name': '_%sEventCmd' % self.module_name,\n> + 'consts': self.module.constants,\n> + 'enums': self.module.enums,\n> + 'has_array': len([x for x in self.module.kinds.keys() if x[0] == 'a']) > 0,\n> + 'has_map': len([x for x in self.module.kinds.keys() if x[0] == 'm']) > 0,\n> + 'has_namespace': self.module.mojom_namespace != '',\n> + 'interface_event': GetEventInterface(self.module.interfaces),\n> + 'interface_main': GetMainInterface(self.module.interfaces),\n> + 'interface_name': 'IPA%sInterface' % self.module_name,\n> + 'module_name': ModuleName(self.module.path),\n> + 'namespace': self.module.mojom_namespace.split('.'),\n> + 'namespace_str': self.module.mojom_namespace.replace('.', '::') if\n> + self.module.mojom_namespace is not None else '',\n> + 'proxy_name': 'IPAProxy%s' % self.module_name,\n> + 'proxy_worker_name': 'IPAProxy%sWorker' % self.module_name,\n> + 'structs_nonempty': [x for x in self.module.structs if len(x.fields) > 0],\n> + }\n> +\n> + def _GetJinjaExportsForCore(self):\n> + return {\n> + 'consts': self.module.constants,\n> + 'enums': self.module.enums,\n> + 'has_array': len([x for x in self.module.kinds.keys() if x[0] == 'a']) > 0,\n> + 'has_map': len([x for x in self.module.kinds.keys() if x[0] == 'm']) > 0,\n> + 'structs_gen_header': [x for x in self.module.structs if x.attributes is not None and 'genHeader' in x.attributes],\n> + 'structs_gen_serializer': [x for x in self.module.structs if x.attributes is not None and 'genSerdes' in x.attributes],\n> + }\n> +\n> + @UseJinja('core_ipa_interface.h.tmpl')\n> + def _GenerateCoreHeader(self):\n> + return self._GetJinjaExportsForCore()\n> +\n> + @UseJinja('core_ipa_serializer.h.tmpl')\n> + def _GenerateCoreSerializer(self):\n> + return self._GetJinjaExportsForCore()\n> +\n> + @UseJinja('module_ipa_interface.h.tmpl')\n> + def _GenerateDataHeader(self):\n> + return self._GetJinjaExports()\n> +\n> + @UseJinja('module_ipa_serializer.h.tmpl')\n> + def _GenerateSerializer(self):\n> + return self._GetJinjaExports()\n> +\n> + @UseJinja('module_ipa_proxy.cpp.tmpl')\n> + def _GenerateProxyCpp(self):\n> + return self._GetJinjaExports()\n> +\n> + @UseJinja('module_ipa_proxy.h.tmpl')\n> + def _GenerateProxyHeader(self):\n> + return self._GetJinjaExports()\n> +\n> + @UseJinja('module_ipa_proxy_worker.cpp.tmpl')\n> + def _GenerateProxyWorker(self):\n> + return self._GetJinjaExports()\n> +\n> + def GenerateFiles(self, unparsed_args):\n> + parser = argparse.ArgumentParser()\n> + parser.add_argument('--libcamera_generate_core_header', action='store_true')\n> + parser.add_argument('--libcamera_generate_core_serializer', action='store_true')\n> + parser.add_argument('--libcamera_generate_header', action='store_true')\n> + parser.add_argument('--libcamera_generate_serializer', action='store_true')\n> + parser.add_argument('--libcamera_generate_proxy_cpp', action='store_true')\n> + parser.add_argument('--libcamera_generate_proxy_h', action='store_true')\n> + parser.add_argument('--libcamera_generate_proxy_worker', action='store_true')\n> + parser.add_argument('--libcamera_output_path')\n> + args = parser.parse_args(unparsed_args)\n> +\n> + if not args.libcamera_generate_core_header and \\\n> + not args.libcamera_generate_core_serializer:\n> + ValidateNamespace(self.module.mojom_namespace)\n> + ValidateInterfaces(self.module.interfaces)\n> + self.module_name = ModuleClassName(self.module)\n> +\n> + fileutil.EnsureDirectoryExists(os.path.dirname(args.libcamera_output_path))\n> +\n> + gen_funcs = [\n> + [args.libcamera_generate_core_header, self._GenerateCoreHeader],\n> + [args.libcamera_generate_core_serializer, self._GenerateCoreSerializer],\n> + [args.libcamera_generate_header, self._GenerateDataHeader],\n> + [args.libcamera_generate_serializer, self._GenerateSerializer],\n> + [args.libcamera_generate_proxy_cpp, self._GenerateProxyCpp],\n> + [args.libcamera_generate_proxy_h, self._GenerateProxyHeader],\n> + [args.libcamera_generate_proxy_worker, self._GenerateProxyWorker],\n> + ]\n> +\n> + for pair in gen_funcs:\n> + if pair[0]:\n> + self.Write(pair[1](), args.libcamera_output_path)","headers":{"Return-Path":"","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id F1095BD808\n\tfor ;\n\tTue, 2 Feb 2021 00:34:07 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id 68D38683D0;\n\tTue, 2 Feb 2021 01:34:07 +0100 (CET)","from perceval.ideasonboard.com (perceval.ideasonboard.com\n\t[IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id 2379C683D0\n\tfor ;\n\tTue, 2 Feb 2021 01:34:05 +0100 (CET)","from pendragon.ideasonboard.com (62-78-145-57.bb.dnainternet.fi\n\t[62.78.145.57])\n\tby perceval.ideasonboard.com (Postfix) with ESMTPSA id 539A8556;\n\tTue, 2 Feb 2021 01:34:04 +0100 (CET)"],"Authentication-Results":"lancelot.ideasonboard.com;\n\tdkim=fail reason=\"signature verification failed\" (1024-bit key;\n\tunprotected) header.d=ideasonboard.com header.i=@ideasonboard.com\n\theader.b=\"trmkA4JS\"; dkim-atps=neutral","DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com;\n\ts=mail; t=1612226044;\n\tbh=+jWNFfaba1LcqcxKhd5VqQqZFpQS5bes4Ttu+sh5jvM=;\n\th=Date:From:To:Cc:Subject:References:In-Reply-To:From;\n\tb=trmkA4JSGIu4IhR+jN4fv4Dlz+UJ/sfpB8iWUjXnSjGYiwmqLseaMjnkIy7Ppxgbk\n\tixnN3zsI7SeRtB7HMNNXAyNw3ToRSAR0O1JNWh8HlLfJXl5+dyAkPTcfxFCpy5qp7Q\n\t5+HIUEDG6UXMoxjQR9UGYYEmQXdXLuITBBP1FjjM=","Date":"Tue, 2 Feb 2021 02:33:43 +0200","From":"Laurent Pinchart ","To":"Paul Elder ","Message-ID":"","References":"<20201224081534.41601-1-paul.elder@ideasonboard.com>\n\t<20201224081534.41601-3-paul.elder@ideasonboard.com>","MIME-Version":"1.0","Content-Disposition":"inline","In-Reply-To":"<20201224081534.41601-3-paul.elder@ideasonboard.com>","Subject":"Re: [libcamera-devel] [PATCH v6 2/9] utils: ipc: add templates for\n\tcode generation for IPC mechanism","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Cc":"libcamera-devel@lists.libcamera.org","Content-Type":"text/plain; charset=\"us-ascii\"","Content-Transfer-Encoding":"7bit","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" "}},{"id":14895,"web_url":"https://patchwork.libcamera.org/comment/14895/","msgid":"","date":"2021-02-02T01:25:29","subject":"Re: [libcamera-devel] [PATCH v6 2/9] utils: ipc: add templates for\n\tcode generation for IPC mechanism","submitter":{"id":2,"url":"https://patchwork.libcamera.org/api/people/2/","name":"Laurent Pinchart","email":"laurent.pinchart@ideasonboard.com"},"content":"Hi Paul,\n\nOn Fri, Jan 15, 2021 at 02:11:41PM +0900, paul.elder@ideasonboard.com wrote:\n> Hi Kieran,\n> \n> \n> \n> > > diff --git a/utils/ipc/generators/libcamera_templates/serializer.tmpl b/utils/ipc/generators/libcamera_templates/serializer.tmpl\n> > > new file mode 100644\n> > > index 00000000..af4800bf\n> > > --- /dev/null\n> > > +++ b/utils/ipc/generators/libcamera_templates/serializer.tmpl\n> > > @@ -0,0 +1,313 @@\n> > > +{#-\n> > > + # SPDX-License-Identifier: LGPL-2.1-or-later\n> > > + # Copyright (C) 2020, Google Inc.\n> > > +-#}\n> > > +{# Turn this into a C macro? #}\n> > \n> > Is this a todo?\n> \n> It was more of a question,\n> \n> > > +{#\n> > > + # \\brief Verify that there is enough bytes to deserialize\n> > > + #\n> > > + # Generate code that verifies that \\a size is not greater than \\a dataSize.\n> > > + # Otherwise log an error with \\a name and \\a typename.\n> > > + #}\n> > > +{%- macro check_data_size(size, dataSize, name, typename) %}\n> > > +\t\tif ({{dataSize}} < {{size}}) {\n> > > +\t\t\tLOG(IPADataSerializer, Error)\n> > > +\t\t\t\t<< \"Failed to deserialize \" << \"{{name}}\"\n> > > +\t\t\t\t<< \": not enough {{typename}}, expected \"\n> > > +\t\t\t\t<< ({{size}}) << \", got \" << ({{dataSize}});\n> > > +\t\t\treturn ret;\n> > > +\t\t}\n> > > +{%- endmacro %}\n> \n> because this block is kind of big, so I wasn't sure if this shold be\n> generated at every check, or if this should be a macro and put the macro\n> at every generated check. In either case it's generated.\n> \n> > > +\n> > > +\n> > > +{#\n> > > + # \\brief Serialize some field into return vector\n> > \n> > 'some' field?\n> > a field?\n> \n> Mathematics vocabulary :p\n> \n> Yeah, I'll change it.\n> \n> > > + #\n> > > + # Generate code to serialize \\a field into retData, including size of the\n> > > + # field and fds (where appropriate).\n> > > + # This code is meant to be used by the IPADataSerializer specialization.\n> > > + #\n> > > + # \\todo Avoid intermediate vectors\n> > > + #}\n> > > +{%- macro serializer_field(field, namespace, loop) %}\n> > > +{%- if field|is_pod or field|is_enum %}\n> > > +\t\tstd::vector {{field.mojom_name}};\n> > > +\t\tstd::tie({{field.mojom_name}}, std::ignore) =\n> > > +\t{%- if field|is_pod %}\n> > > +\t\t\tIPADataSerializer<{{field|name}}>::serialize(data.{{field.mojom_name}});\n> > > +\t{%- elif field|is_enum %}\n> > > +\t\t\tIPADataSerializer::serialize(data.{{field.mojom_name}});\n> > > +\t{%- endif %}\n> > > +\t\tretData.insert(retData.end(), {{field.mojom_name}}.begin(), {{field.mojom_name}}.end());\n> > > +{%- elif field|is_fd %}\n> > > +\t\tstd::vector {{field.mojom_name}};\n> > > +\t\tstd::vector {{field.mojom_name}}Fds;\n> > > +\t\tstd::tie({{field.mojom_name}}, {{field.mojom_name}}Fds) =\n> > > +\t\t\tIPADataSerializer<{{field|name}}>::serialize(data.{{field.mojom_name}});\n> > > +\t\tretData.insert(retData.end(), {{field.mojom_name}}.begin(), {{field.mojom_name}}.end());\n> > > +\t\tretFds.insert(retFds.end(), {{field.mojom_name}}Fds.begin(), {{field.mojom_name}}Fds.end());\n> > > +{%- elif field|is_controls %}\n> > > +\t\tif (data.{{field.mojom_name}}.size() > 0) {\n> > > +\t\t\tstd::vector {{field.mojom_name}};\n> > > +\t\t\tstd::tie({{field.mojom_name}}, std::ignore) =\n> > > +\t\t\t\tIPADataSerializer<{{field|name}}>::serialize(data.{{field.mojom_name}}, cs);\n> > > +\t\t\tappendPOD(retData, {{field.mojom_name}}.size());\n> > > +\t\t\tretData.insert(retData.end(), {{field.mojom_name}}.begin(), {{field.mojom_name}}.end());\n> > > +\t\t} else {\n> > > +\t\t\tappendPOD(retData, 0);\n> > > +\t\t}\n> > > +{%- elif field|is_plain_struct or field|is_array or field|is_map or field|is_str %}\n> > > +\t\tstd::vector {{field.mojom_name}};\n> > > +\t{%- if field|has_fd %}\n> > > +\t\tstd::vector {{field.mojom_name}}Fds;\n> > > +\t\tstd::tie({{field.mojom_name}}, {{field.mojom_name}}Fds) =\n> > > +\t{%- else %}\n> > > +\t\tstd::tie({{field.mojom_name}}, std::ignore) =\n> > > +\t{%- endif %}\n> > > +\t{%- if field|is_array or field|is_map %}\n> > > +\t\t\tIPADataSerializer<{{field|name}}>::serialize(data.{{field.mojom_name}}, cs);\n> > > +\t{%- elif field|is_str %}\n> > > +\t\t\tIPADataSerializer<{{field|name}}>::serialize(data.{{field.mojom_name}});\n> > > +\t{%- else %}\n> > > +\t\t\tIPADataSerializer<{{field|name_full(namespace)}}>::serialize(data.{{field.mojom_name}}, cs);\n> > > +\t{%- endif %}\n> > > +\t\tappendPOD(retData, {{field.mojom_name}}.size());\n> > > +\t{%- if field|has_fd %}\n> > > +\t\tappendPOD(retData, {{field.mojom_name}}Fds.size());\n> > > +\t{%- endif %}\n> > > +\t\tretData.insert(retData.end(), {{field.mojom_name}}.begin(), {{field.mojom_name}}.end());\n> > > +\t{%- if field|has_fd %}\n> > > +\t\tretFds.insert(retFds.end(), {{field.mojom_name}}Fds.begin(), {{field.mojom_name}}Fds.end());\n> > > +\t{%- endif %}\n> > > +{%- else %}\n> > > +\t\t/* Unknown serialization for {{field.mojom_name}}. */\n> > > +{%- endif %}\n> > > +{%- endmacro %}\n> > > +\n> > > +\n> > > +{#\n> > > + # \\brief Deserialize some field into return struct\n> > \n> > Again, 'some field' sounds like a random field.\n> > Presumably this is going to handle a quite specific one each time...\n> > \n> > \n> > > + #\n> > > + # Generate code to deserialize \\a field into object ret.\n> > > + # This code is meant to be used by the IPADataSerializer specialization.\n> > > + #}\n> > > +{%- macro deserializer_field(field, namespace, loop) %}\n> > > +{% if field|is_pod or field|is_enum %}\n> > > +\t{%- set field_size = (field|bit_width|int / 8)|int %}\n> > > +\t\t{{- check_data_size(field_size, 'dataSize', field.mojom_name, 'data')}}\n> > > +\t\t{%- if field|is_pod %}\n> > > +\t\tret.{{field.mojom_name}} = IPADataSerializer<{{field|name}}>::deserialize(m, m + {{field_size}});\n> > > +\t\t{%- else %}\n> > > +\t\tret.{{field.mojom_name}} = static_cast<{{field|name_full(namespace)}}>(IPADataSerializer::deserialize(m, m + {{field_size}}));\n> > > +\t\t{%- endif %}\n> > > +\t{%- if not loop.last %}\n> > > +\t\tm += {{field_size}};\n> > > +\t\tdataSize -= {{field_size}};\n> > > +\t{%- endif %}\n> > > +{% elif field|is_fd %}\n> > > +\t{%- set field_size = 1 %}\n> > > +\t\t{{- check_data_size(field_size, 'dataSize', field.mojom_name, 'data')}}\n> > > +\t\tret.{{field.mojom_name}} = IPADataSerializer<{{field|name}}>::deserialize(m, m + 1, n, n + 1, cs);\n> > > +\t{%- if not loop.last %}\n> > > +\t\tm += {{field_size}};\n> > > +\t\tdataSize -= {{field_size}};\n> > > +\t\tn += ret.{{field.mojom_name}}.isValid() ? 1 : 0;\n> > > +\t\tfdsSize -= ret.{{field.mojom_name}}.isValid() ? 1 : 0;\n> > > +\t{%- endif %}\n> > > +{% elif field|is_controls %}\n> > > +\t{%- set field_size = 4 %}\n> > > +\t\t{{- check_data_size(field_size, 'dataSize', field.mojom_name + 'Size', 'data')}}\n> > > +\t\tconst size_t {{field.mojom_name}}Size = readPOD(m, 0, dataEnd);\n> > > +\t\tm += {{field_size}};\n> > > +\t\tdataSize -= {{field_size}};\n> > > +\t{%- set field_size = field.mojom_name + 'Size' -%}\n> > > +\t\t{{- check_data_size(field_size, 'dataSize', field.mojom_name, 'data')}}\n> > > +\t\tif ({{field.mojom_name}}Size > 0)\n> > > +\t\t\tret.{{field.mojom_name}} =\n> > > +\t\t\t\tIPADataSerializer<{{field|name}}>::deserialize(m, m + {{field.mojom_name}}Size, cs);\n> > > +\t{%- if not loop.last %}\n> > > +\t\tm += {{field_size}};\n> > > +\t\tdataSize -= {{field_size}};\n> > > +\t{%- endif %}\n> > > +{% elif field|is_plain_struct or field|is_array or field|is_map or field|is_str %}\n> > > +\t{%- set field_size = 4 %}\n> > > +\t\t{{- check_data_size(field_size, 'dataSize', field.mojom_name + 'Size', 'data')}}\n> > > +\t\tconst size_t {{field.mojom_name}}Size = readPOD(m, 0, dataEnd);\n> > > +\t\tm += {{field_size}};\n> > > +\t\tdataSize -= {{field_size}};\n> > > +\t{%- if field|has_fd %}\n> > > +\t{%- set field_size = 4 %}\n> > > +\t\t{{- check_data_size(field_size, 'dataSize', field.mojom_name + 'FdsSize', 'data')}}\n> > > +\t\tconst size_t {{field.mojom_name}}FdsSize = readPOD(m, 0, dataEnd);\n> > > +\t\tm += {{field_size}};\n> > > +\t\tdataSize -= {{field_size}};\n> > > +\t\t{{- check_data_size(field.mojom_name + 'FdsSize', 'fdsSize', field.mojom_name, 'fds')}}\n> > > +\t{%- endif %}\n> > > +\t{%- set field_size = field.mojom_name + 'Size' -%}\n> > > +\t\t{{- check_data_size(field_size, 'dataSize', field.mojom_name, 'data')}}\n> > > +\t\tret.{{field.mojom_name}} =\n> > > +\t{%- if field|is_str %}\n> > > +\t\t\tIPADataSerializer<{{field|name}}>::deserialize(m, m + {{field.mojom_name}}Size);\n> > > +\t{%- elif field|has_fd and (field|is_array or field|is_map) %}\n> > > +\t\t\tIPADataSerializer<{{field|name}}>::deserialize(m, m + {{field.mojom_name}}Size, n, n + {{field.mojom_name}}FdsSize, cs);\n> > > +\t{%- elif field|has_fd and (not (field|is_array or field|is_map)) %}\n> > > +\t\t\tIPADataSerializer<{{field|name_full(namespace)}}>::deserialize(m, m + {{field.mojom_name}}Size, n, n + {{field.mojom_name}}FdsSize, cs);\n> > > +\t{%- elif (not field|has_fd) and (field|is_array or field|is_map) %}\n> > > +\t\t\tIPADataSerializer<{{field|name}}>::deserialize(m, m + {{field.mojom_name}}Size, cs);\n> > > +\t{%- else %}\n> > > +\t\t\tIPADataSerializer<{{field|name_full(namespace)}}>::deserialize(m, m + {{field.mojom_name}}Size, cs);\n> > > +\t{%- endif %}\n> > > +\t{%- if not loop.last %}\n> > > +\t\tm += {{field_size}};\n> > > +\t\tdataSize -= {{field_size}};\n> > > +\t{%- if field|has_fd %}\n> > > +\t\tn += {{field.mojom_name}}FdsSize;\n> > > +\t\tfdsSize -= {{field.mojom_name}}FdsSize;\n> > > +\t{%- endif %}\n> > > +\t{%- endif %}\n> > > +{% else %}\n> > > +\t\t/* Unknown deserialization for {{field.mojom_name}}. */\n> > > +{%- endif %}\n> > > +{%- endmacro %}\n> > > +\n> > > +\n> > > +{#\n> > > + # \\brief Serialize a struct\n> > > + #\n> > > + # Generate code for IPADataSerializer specialization, for serializing\n> > > + # \\a struct.\n> > > + #}\n> > > +{%- macro serializer(struct, namespace) %}\n> > > +\tstatic std::tuple, std::vector>\n> > > +\tserialize(const {{struct|name_full(namespace)}} &data,\n> > > +{%- if struct|needs_control_serializer %}\n> > > +\t\t ControlSerializer *cs)\n> > > +{%- else %}\n> > > +\t\t [[maybe_unused]] ControlSerializer *cs = nullptr)\n> > > +{%- endif %}\n> > > +\t{\n> > > +\t\tstd::vector retData;\n> > > +{%- if struct|has_fd %}\n> > > +\t\tstd::vector retFds;\n> > > +{%- endif %}\n> > > +{%- for field in struct.fields %}\n> > > +{{serializer_field(field, namespace, loop)}}\n> > > +{%- endfor %}\n> > > +{% if struct|has_fd %}\n> > > +\t\treturn {retData, retFds};\n> > > +{%- else %}\n> > > +\t\treturn {retData, {}};\n> > > +{%- endif %}\n> > > +\t}\n> > > +{%- endmacro %}\n> > > +\n> > > +\n> > > +{#\n> > > + # \\brief Deserialize a struct that has fds\n> > > + #\n> > > + # Generate code for IPADataSerializer specialization, for deserializing\n> > > + # \\a struct, in the case that \\a struct has file descriptors.\n> > > + # fd parameters\n> > > + #}\n> > > +{%- macro deserializer_fd(struct, namespace) %}\n> > > +\tstatic {{struct|name_full(namespace)}}\n> > > +\tdeserialize(std::vector &data,\n> > > +\t\t std::vector &fds,\n> > > +{%- if struct|needs_control_serializer %}\n> > > +\t\t ControlSerializer *cs)\n> > > +{%- else %}\n> > > +\t\t [[maybe_unused]] ControlSerializer *cs = nullptr)\n> > > +{%- endif %}\n> > > +\t{\n> > > +\t\treturn IPADataSerializer<{{struct|name_full(namespace)}}>::deserialize(data.cbegin(), data.cend(), fds.cbegin(), fds.cend(), cs);\n> > > +\t}\n> > > +\n> > > +\tstatic {{struct|name_full(namespace)}}\n> > > +\tdeserialize(std::vector::const_iterator dataBegin,\n> > > +\t\t std::vector::const_iterator dataEnd,\n> > > +\t\t std::vector::const_iterator fdsBegin,\n> > > +\t\t std::vector::const_iterator fdsEnd,\n> > > +{%- if struct|needs_control_serializer %}\n> > > +\t\t ControlSerializer *cs)\n> > > +{%- else %}\n> > > +\t\t [[maybe_unused]] ControlSerializer *cs = nullptr)\n> > > +{%- endif %}\n> > > +\t{\n> > > +\t\t{{struct|name_full(namespace)}} ret;\n> > > +\t\tstd::vector::const_iterator m = dataBegin;\n> > > +\t\tstd::vector::const_iterator n = fdsBegin;\n> > > +\n> > > +\t\tsize_t dataSize = std::distance(dataBegin, dataEnd);\n> > > +\t\t[[maybe_unused]] size_t fdsSize = std::distance(fdsBegin, fdsEnd);\n> > > +{%- for field in struct.fields -%}\n> > > +{{deserializer_field(field, namespace, loop)}}\n> > > +{%- endfor %}\n> > > +\t\treturn ret;\n> > > +\t}\n> > > +{%- endmacro %}\n> > > +\n> > > +{#\n> > > + # \\brief Deserialize a struct that has fds, using non-fd\n> > > + #\n> > > + # Generate code for IPADataSerializer specialization, for deserializing\n> > > + # \\a struct, in the case that \\a struct has no file descriptors but requires\n> > > + # deserializers with file descriptors.\n> > \n> > If the struct has no file descriptors, why does it need a deserialiszer\n> > with file descriptors?\n> \n> It's for structs that are generated from core.mojom, to allow them to be\n> embedded in arrays/maps. The array/map (de)serializer has no way of\n> knowing if the struct it's (de)serializing has fds or not, so any struct\n> that will be contained in an array/map must have (de)serializers defined\n> for fds.\n> \n> That's why the hand-written (de)serializer implementations for\n> arithmetic types and ControlList and ControlInfoMap also have them.\n> \n> It's actually also required for structs that are generated in the\n> per-pipeline mojom files.\n\nWould it make sense to pass the fds to all (de)serialization functions,\nunconditionally, to lower the complexity ? Up to you.\n\n> > Other than the documentation comments, I don't think I can hope to\n> > provide much more parsing of all this.\n> > \n> > Acked-by: Kieran Bingham ","headers":{"Return-Path":"","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id 72076BD808\n\tfor ;\n\tTue, 2 Feb 2021 01:25:53 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id 0722F6841B;\n\tTue, 2 Feb 2021 02:25:53 +0100 (CET)","from perceval.ideasonboard.com (perceval.ideasonboard.com\n\t[213.167.242.64])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id 8EE9E60106\n\tfor ;\n\tTue, 2 Feb 2021 02:25:51 +0100 (CET)","from pendragon.ideasonboard.com (62-78-145-57.bb.dnainternet.fi\n\t[62.78.145.57])\n\tby perceval.ideasonboard.com (Postfix) with ESMTPSA id CE8AE556;\n\tTue, 2 Feb 2021 02:25:50 +0100 (CET)"],"Authentication-Results":"lancelot.ideasonboard.com;\n\tdkim=fail reason=\"signature verification failed\" (1024-bit key;\n\tunprotected) header.d=ideasonboard.com header.i=@ideasonboard.com\n\theader.b=\"AIlrDA7W\"; dkim-atps=neutral","DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com;\n\ts=mail; t=1612229151;\n\tbh=n1zyMAfNVhS5asEuYgYd10cCmLHFWUXQs7LQOnehIEQ=;\n\th=Date:From:To:Cc:Subject:References:In-Reply-To:From;\n\tb=AIlrDA7WGp9kbk59icq1FfurTAQxwSm/hCrvIzh0f5vo5KP5SpYo+JRIybPdsXh/p\n\tpwvnCSB9/nRKWZQlWcI8xU0FvykYNadM/lNNe3huI/e3n49jvm6KU6uOcNBEWQuySV\n\tQUqBli/Z6XDxl0rh+oXvoTNBKc0Ss4zlURkhdpwY=","Date":"Tue, 2 Feb 2021 03:25:29 +0200","From":"Laurent Pinchart ","To":"paul.elder@ideasonboard.com","Message-ID":"","References":"<20201224081534.41601-1-paul.elder@ideasonboard.com>\n\t<20201224081534.41601-3-paul.elder@ideasonboard.com>\n\t<3ce53459-3c34-710a-b4e3-01cc4ff6e8f3@ideasonboard.com>\n\t<20210115051141.GA1885@pyrite.rasen.tech>","MIME-Version":"1.0","Content-Disposition":"inline","In-Reply-To":"<20210115051141.GA1885@pyrite.rasen.tech>","Subject":"Re: [libcamera-devel] [PATCH v6 2/9] utils: ipc: add templates for\n\tcode generation for IPC mechanism","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Cc":"libcamera-devel@lists.libcamera.org","Content-Type":"text/plain; charset=\"us-ascii\"","Content-Transfer-Encoding":"7bit","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" "}},{"id":15021,"web_url":"https://patchwork.libcamera.org/comment/15021/","msgid":"<20210205060156.GA57940@pyrite.rasen.tech>","date":"2021-02-05T06:01:56","subject":"Re: [libcamera-devel] [PATCH v6 2/9] utils: ipc: add templates for\n\tcode generation for IPC mechanism","submitter":{"id":17,"url":"https://patchwork.libcamera.org/api/people/17/","name":"Paul Elder","email":"paul.elder@ideasonboard.com"},"content":"Hi Laurent,\n\nOn Tue, Feb 02, 2021 at 03:25:29AM +0200, Laurent Pinchart wrote:\n> Hi Paul,\n> \n> On Fri, Jan 15, 2021 at 02:11:41PM +0900, paul.elder@ideasonboard.com wrote:\n> > Hi Kieran,\n> > \n> > \n> > \n> > > > diff --git a/utils/ipc/generators/libcamera_templates/serializer.tmpl b/utils/ipc/generators/libcamera_templates/serializer.tmpl\n> > > > new file mode 100644\n> > > > index 00000000..af4800bf\n> > > > --- /dev/null\n> > > > +++ b/utils/ipc/generators/libcamera_templates/serializer.tmpl\n> > > > @@ -0,0 +1,313 @@\n> > > > +{#-\n> > > > + # SPDX-License-Identifier: LGPL-2.1-or-later\n> > > > + # Copyright (C) 2020, Google Inc.\n> > > > +-#}\n> > > > +{# Turn this into a C macro? #}\n> > > \n> > > Is this a todo?\n> > \n> > It was more of a question,\n> > \n> > > > +{#\n> > > > + # \\brief Verify that there is enough bytes to deserialize\n> > > > + #\n> > > > + # Generate code that verifies that \\a size is not greater than \\a dataSize.\n> > > > + # Otherwise log an error with \\a name and \\a typename.\n> > > > + #}\n> > > > +{%- macro check_data_size(size, dataSize, name, typename) %}\n> > > > +\t\tif ({{dataSize}} < {{size}}) {\n> > > > +\t\t\tLOG(IPADataSerializer, Error)\n> > > > +\t\t\t\t<< \"Failed to deserialize \" << \"{{name}}\"\n> > > > +\t\t\t\t<< \": not enough {{typename}}, expected \"\n> > > > +\t\t\t\t<< ({{size}}) << \", got \" << ({{dataSize}});\n> > > > +\t\t\treturn ret;\n> > > > +\t\t}\n> > > > +{%- endmacro %}\n> > \n> > because this block is kind of big, so I wasn't sure if this shold be\n> > generated at every check, or if this should be a macro and put the macro\n> > at every generated check. In either case it's generated.\n> > \n> > > > +\n> > > > +\n> > > > +{#\n> > > > + # \\brief Serialize some field into return vector\n> > > \n> > > 'some' field?\n> > > a field?\n> > \n> > Mathematics vocabulary :p\n> > \n> > Yeah, I'll change it.\n> > \n> > > > + #\n> > > > + # Generate code to serialize \\a field into retData, including size of the\n> > > > + # field and fds (where appropriate).\n> > > > + # This code is meant to be used by the IPADataSerializer specialization.\n> > > > + #\n> > > > + # \\todo Avoid intermediate vectors\n> > > > + #}\n> > > > +{%- macro serializer_field(field, namespace, loop) %}\n> > > > +{%- if field|is_pod or field|is_enum %}\n> > > > +\t\tstd::vector {{field.mojom_name}};\n> > > > +\t\tstd::tie({{field.mojom_name}}, std::ignore) =\n> > > > +\t{%- if field|is_pod %}\n> > > > +\t\t\tIPADataSerializer<{{field|name}}>::serialize(data.{{field.mojom_name}});\n> > > > +\t{%- elif field|is_enum %}\n> > > > +\t\t\tIPADataSerializer::serialize(data.{{field.mojom_name}});\n> > > > +\t{%- endif %}\n> > > > +\t\tretData.insert(retData.end(), {{field.mojom_name}}.begin(), {{field.mojom_name}}.end());\n> > > > +{%- elif field|is_fd %}\n> > > > +\t\tstd::vector {{field.mojom_name}};\n> > > > +\t\tstd::vector {{field.mojom_name}}Fds;\n> > > > +\t\tstd::tie({{field.mojom_name}}, {{field.mojom_name}}Fds) =\n> > > > +\t\t\tIPADataSerializer<{{field|name}}>::serialize(data.{{field.mojom_name}});\n> > > > +\t\tretData.insert(retData.end(), {{field.mojom_name}}.begin(), {{field.mojom_name}}.end());\n> > > > +\t\tretFds.insert(retFds.end(), {{field.mojom_name}}Fds.begin(), {{field.mojom_name}}Fds.end());\n> > > > +{%- elif field|is_controls %}\n> > > > +\t\tif (data.{{field.mojom_name}}.size() > 0) {\n> > > > +\t\t\tstd::vector {{field.mojom_name}};\n> > > > +\t\t\tstd::tie({{field.mojom_name}}, std::ignore) =\n> > > > +\t\t\t\tIPADataSerializer<{{field|name}}>::serialize(data.{{field.mojom_name}}, cs);\n> > > > +\t\t\tappendPOD(retData, {{field.mojom_name}}.size());\n> > > > +\t\t\tretData.insert(retData.end(), {{field.mojom_name}}.begin(), {{field.mojom_name}}.end());\n> > > > +\t\t} else {\n> > > > +\t\t\tappendPOD(retData, 0);\n> > > > +\t\t}\n> > > > +{%- elif field|is_plain_struct or field|is_array or field|is_map or field|is_str %}\n> > > > +\t\tstd::vector {{field.mojom_name}};\n> > > > +\t{%- if field|has_fd %}\n> > > > +\t\tstd::vector {{field.mojom_name}}Fds;\n> > > > +\t\tstd::tie({{field.mojom_name}}, {{field.mojom_name}}Fds) =\n> > > > +\t{%- else %}\n> > > > +\t\tstd::tie({{field.mojom_name}}, std::ignore) =\n> > > > +\t{%- endif %}\n> > > > +\t{%- if field|is_array or field|is_map %}\n> > > > +\t\t\tIPADataSerializer<{{field|name}}>::serialize(data.{{field.mojom_name}}, cs);\n> > > > +\t{%- elif field|is_str %}\n> > > > +\t\t\tIPADataSerializer<{{field|name}}>::serialize(data.{{field.mojom_name}});\n> > > > +\t{%- else %}\n> > > > +\t\t\tIPADataSerializer<{{field|name_full(namespace)}}>::serialize(data.{{field.mojom_name}}, cs);\n> > > > +\t{%- endif %}\n> > > > +\t\tappendPOD(retData, {{field.mojom_name}}.size());\n> > > > +\t{%- if field|has_fd %}\n> > > > +\t\tappendPOD(retData, {{field.mojom_name}}Fds.size());\n> > > > +\t{%- endif %}\n> > > > +\t\tretData.insert(retData.end(), {{field.mojom_name}}.begin(), {{field.mojom_name}}.end());\n> > > > +\t{%- if field|has_fd %}\n> > > > +\t\tretFds.insert(retFds.end(), {{field.mojom_name}}Fds.begin(), {{field.mojom_name}}Fds.end());\n> > > > +\t{%- endif %}\n> > > > +{%- else %}\n> > > > +\t\t/* Unknown serialization for {{field.mojom_name}}. */\n> > > > +{%- endif %}\n> > > > +{%- endmacro %}\n> > > > +\n> > > > +\n> > > > +{#\n> > > > + # \\brief Deserialize some field into return struct\n> > > \n> > > Again, 'some field' sounds like a random field.\n> > > Presumably this is going to handle a quite specific one each time...\n> > > \n> > > \n> > > > + #\n> > > > + # Generate code to deserialize \\a field into object ret.\n> > > > + # This code is meant to be used by the IPADataSerializer specialization.\n> > > > + #}\n> > > > +{%- macro deserializer_field(field, namespace, loop) %}\n> > > > +{% if field|is_pod or field|is_enum %}\n> > > > +\t{%- set field_size = (field|bit_width|int / 8)|int %}\n> > > > +\t\t{{- check_data_size(field_size, 'dataSize', field.mojom_name, 'data')}}\n> > > > +\t\t{%- if field|is_pod %}\n> > > > +\t\tret.{{field.mojom_name}} = IPADataSerializer<{{field|name}}>::deserialize(m, m + {{field_size}});\n> > > > +\t\t{%- else %}\n> > > > +\t\tret.{{field.mojom_name}} = static_cast<{{field|name_full(namespace)}}>(IPADataSerializer::deserialize(m, m + {{field_size}}));\n> > > > +\t\t{%- endif %}\n> > > > +\t{%- if not loop.last %}\n> > > > +\t\tm += {{field_size}};\n> > > > +\t\tdataSize -= {{field_size}};\n> > > > +\t{%- endif %}\n> > > > +{% elif field|is_fd %}\n> > > > +\t{%- set field_size = 1 %}\n> > > > +\t\t{{- check_data_size(field_size, 'dataSize', field.mojom_name, 'data')}}\n> > > > +\t\tret.{{field.mojom_name}} = IPADataSerializer<{{field|name}}>::deserialize(m, m + 1, n, n + 1, cs);\n> > > > +\t{%- if not loop.last %}\n> > > > +\t\tm += {{field_size}};\n> > > > +\t\tdataSize -= {{field_size}};\n> > > > +\t\tn += ret.{{field.mojom_name}}.isValid() ? 1 : 0;\n> > > > +\t\tfdsSize -= ret.{{field.mojom_name}}.isValid() ? 1 : 0;\n> > > > +\t{%- endif %}\n> > > > +{% elif field|is_controls %}\n> > > > +\t{%- set field_size = 4 %}\n> > > > +\t\t{{- check_data_size(field_size, 'dataSize', field.mojom_name + 'Size', 'data')}}\n> > > > +\t\tconst size_t {{field.mojom_name}}Size = readPOD(m, 0, dataEnd);\n> > > > +\t\tm += {{field_size}};\n> > > > +\t\tdataSize -= {{field_size}};\n> > > > +\t{%- set field_size = field.mojom_name + 'Size' -%}\n> > > > +\t\t{{- check_data_size(field_size, 'dataSize', field.mojom_name, 'data')}}\n> > > > +\t\tif ({{field.mojom_name}}Size > 0)\n> > > > +\t\t\tret.{{field.mojom_name}} =\n> > > > +\t\t\t\tIPADataSerializer<{{field|name}}>::deserialize(m, m + {{field.mojom_name}}Size, cs);\n> > > > +\t{%- if not loop.last %}\n> > > > +\t\tm += {{field_size}};\n> > > > +\t\tdataSize -= {{field_size}};\n> > > > +\t{%- endif %}\n> > > > +{% elif field|is_plain_struct or field|is_array or field|is_map or field|is_str %}\n> > > > +\t{%- set field_size = 4 %}\n> > > > +\t\t{{- check_data_size(field_size, 'dataSize', field.mojom_name + 'Size', 'data')}}\n> > > > +\t\tconst size_t {{field.mojom_name}}Size = readPOD(m, 0, dataEnd);\n> > > > +\t\tm += {{field_size}};\n> > > > +\t\tdataSize -= {{field_size}};\n> > > > +\t{%- if field|has_fd %}\n> > > > +\t{%- set field_size = 4 %}\n> > > > +\t\t{{- check_data_size(field_size, 'dataSize', field.mojom_name + 'FdsSize', 'data')}}\n> > > > +\t\tconst size_t {{field.mojom_name}}FdsSize = readPOD(m, 0, dataEnd);\n> > > > +\t\tm += {{field_size}};\n> > > > +\t\tdataSize -= {{field_size}};\n> > > > +\t\t{{- check_data_size(field.mojom_name + 'FdsSize', 'fdsSize', field.mojom_name, 'fds')}}\n> > > > +\t{%- endif %}\n> > > > +\t{%- set field_size = field.mojom_name + 'Size' -%}\n> > > > +\t\t{{- check_data_size(field_size, 'dataSize', field.mojom_name, 'data')}}\n> > > > +\t\tret.{{field.mojom_name}} =\n> > > > +\t{%- if field|is_str %}\n> > > > +\t\t\tIPADataSerializer<{{field|name}}>::deserialize(m, m + {{field.mojom_name}}Size);\n> > > > +\t{%- elif field|has_fd and (field|is_array or field|is_map) %}\n> > > > +\t\t\tIPADataSerializer<{{field|name}}>::deserialize(m, m + {{field.mojom_name}}Size, n, n + {{field.mojom_name}}FdsSize, cs);\n> > > > +\t{%- elif field|has_fd and (not (field|is_array or field|is_map)) %}\n> > > > +\t\t\tIPADataSerializer<{{field|name_full(namespace)}}>::deserialize(m, m + {{field.mojom_name}}Size, n, n + {{field.mojom_name}}FdsSize, cs);\n> > > > +\t{%- elif (not field|has_fd) and (field|is_array or field|is_map) %}\n> > > > +\t\t\tIPADataSerializer<{{field|name}}>::deserialize(m, m + {{field.mojom_name}}Size, cs);\n> > > > +\t{%- else %}\n> > > > +\t\t\tIPADataSerializer<{{field|name_full(namespace)}}>::deserialize(m, m + {{field.mojom_name}}Size, cs);\n> > > > +\t{%- endif %}\n> > > > +\t{%- if not loop.last %}\n> > > > +\t\tm += {{field_size}};\n> > > > +\t\tdataSize -= {{field_size}};\n> > > > +\t{%- if field|has_fd %}\n> > > > +\t\tn += {{field.mojom_name}}FdsSize;\n> > > > +\t\tfdsSize -= {{field.mojom_name}}FdsSize;\n> > > > +\t{%- endif %}\n> > > > +\t{%- endif %}\n> > > > +{% else %}\n> > > > +\t\t/* Unknown deserialization for {{field.mojom_name}}. */\n> > > > +{%- endif %}\n> > > > +{%- endmacro %}\n> > > > +\n> > > > +\n> > > > +{#\n> > > > + # \\brief Serialize a struct\n> > > > + #\n> > > > + # Generate code for IPADataSerializer specialization, for serializing\n> > > > + # \\a struct.\n> > > > + #}\n> > > > +{%- macro serializer(struct, namespace) %}\n> > > > +\tstatic std::tuple, std::vector>\n> > > > +\tserialize(const {{struct|name_full(namespace)}} &data,\n> > > > +{%- if struct|needs_control_serializer %}\n> > > > +\t\t ControlSerializer *cs)\n> > > > +{%- else %}\n> > > > +\t\t [[maybe_unused]] ControlSerializer *cs = nullptr)\n> > > > +{%- endif %}\n> > > > +\t{\n> > > > +\t\tstd::vector retData;\n> > > > +{%- if struct|has_fd %}\n> > > > +\t\tstd::vector retFds;\n> > > > +{%- endif %}\n> > > > +{%- for field in struct.fields %}\n> > > > +{{serializer_field(field, namespace, loop)}}\n> > > > +{%- endfor %}\n> > > > +{% if struct|has_fd %}\n> > > > +\t\treturn {retData, retFds};\n> > > > +{%- else %}\n> > > > +\t\treturn {retData, {}};\n> > > > +{%- endif %}\n> > > > +\t}\n> > > > +{%- endmacro %}\n> > > > +\n> > > > +\n> > > > +{#\n> > > > + # \\brief Deserialize a struct that has fds\n> > > > + #\n> > > > + # Generate code for IPADataSerializer specialization, for deserializing\n> > > > + # \\a struct, in the case that \\a struct has file descriptors.\n> > > > + # fd parameters\n> > > > + #}\n> > > > +{%- macro deserializer_fd(struct, namespace) %}\n> > > > +\tstatic {{struct|name_full(namespace)}}\n> > > > +\tdeserialize(std::vector &data,\n> > > > +\t\t std::vector &fds,\n> > > > +{%- if struct|needs_control_serializer %}\n> > > > +\t\t ControlSerializer *cs)\n> > > > +{%- else %}\n> > > > +\t\t [[maybe_unused]] ControlSerializer *cs = nullptr)\n> > > > +{%- endif %}\n> > > > +\t{\n> > > > +\t\treturn IPADataSerializer<{{struct|name_full(namespace)}}>::deserialize(data.cbegin(), data.cend(), fds.cbegin(), fds.cend(), cs);\n> > > > +\t}\n> > > > +\n> > > > +\tstatic {{struct|name_full(namespace)}}\n> > > > +\tdeserialize(std::vector::const_iterator dataBegin,\n> > > > +\t\t std::vector::const_iterator dataEnd,\n> > > > +\t\t std::vector::const_iterator fdsBegin,\n> > > > +\t\t std::vector::const_iterator fdsEnd,\n> > > > +{%- if struct|needs_control_serializer %}\n> > > > +\t\t ControlSerializer *cs)\n> > > > +{%- else %}\n> > > > +\t\t [[maybe_unused]] ControlSerializer *cs = nullptr)\n> > > > +{%- endif %}\n> > > > +\t{\n> > > > +\t\t{{struct|name_full(namespace)}} ret;\n> > > > +\t\tstd::vector::const_iterator m = dataBegin;\n> > > > +\t\tstd::vector::const_iterator n = fdsBegin;\n> > > > +\n> > > > +\t\tsize_t dataSize = std::distance(dataBegin, dataEnd);\n> > > > +\t\t[[maybe_unused]] size_t fdsSize = std::distance(fdsBegin, fdsEnd);\n> > > > +{%- for field in struct.fields -%}\n> > > > +{{deserializer_field(field, namespace, loop)}}\n> > > > +{%- endfor %}\n> > > > +\t\treturn ret;\n> > > > +\t}\n> > > > +{%- endmacro %}\n> > > > +\n> > > > +{#\n> > > > + # \\brief Deserialize a struct that has fds, using non-fd\n> > > > + #\n> > > > + # Generate code for IPADataSerializer specialization, for deserializing\n> > > > + # \\a struct, in the case that \\a struct has no file descriptors but requires\n> > > > + # deserializers with file descriptors.\n> > > \n> > > If the struct has no file descriptors, why does it need a deserialiszer\n> > > with file descriptors?\n> > \n> > It's for structs that are generated from core.mojom, to allow them to be\n> > embedded in arrays/maps. The array/map (de)serializer has no way of\n> > knowing if the struct it's (de)serializing has fds or not, so any struct\n> > that will be contained in an array/map must have (de)serializers defined\n> > for fds.\n> > \n> > That's why the hand-written (de)serializer implementations for\n> > arithmetic types and ControlList and ControlInfoMap also have them.\n> > \n> > It's actually also required for structs that are generated in the\n> > per-pipeline mojom files.\n> \n> Would it make sense to pass the fds to all (de)serialization functions,\n> unconditionally, to lower the complexity ? Up to you.\n\nI don't think so. It ends up needing to create dummy fds\nvectors/iterators to pass into the ones that don't need it, and if we\ndefault to empty ones then it's dangerous for those that do need it.\n\n\nPaul\n\n> > > Other than the documentation comments, I don't think I can hope to\n> > > provide much more parsing of all this.\n> > > \n> > > Acked-by: Kieran Bingham ","headers":{"Return-Path":"","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id DDB56BD162\n\tfor ;\n\tFri, 5 Feb 2021 06:02:06 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id 61940614A6;\n\tFri, 5 Feb 2021 07:02:06 +0100 (CET)","from perceval.ideasonboard.com (perceval.ideasonboard.com\n\t[IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id CB93B6149C\n\tfor ;\n\tFri, 5 Feb 2021 07:02:04 +0100 (CET)","from pyrite.rasen.tech (unknown\n\t[IPv6:2400:4051:61:600:2c71:1b79:d06d:5032])\n\tby perceval.ideasonboard.com (Postfix) with ESMTPSA id 12934EE;\n\tFri, 5 Feb 2021 07:02:02 +0100 (CET)"],"Authentication-Results":"lancelot.ideasonboard.com;\n\tdkim=fail reason=\"signature verification failed\" (1024-bit key;\n\tunprotected) header.d=ideasonboard.com header.i=@ideasonboard.com\n\theader.b=\"c9jAoffi\"; dkim-atps=neutral","DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com;\n\ts=mail; t=1612504924;\n\tbh=EL7NriFGtJXFLlvyoYF8OLtE3MgSSNWzM225p6Jdvhk=;\n\th=Date:From:To:Cc:Subject:References:In-Reply-To:From;\n\tb=c9jAoffiHZc4J9ZzYzjC5TWdHfxIsCWOzqGnVPZHqYeQXr/Krw0ceabLgW1zG5DMm\n\tVo7jHTo+tm6EdKLgUzG6ROFe2/O2+0tUzyqQKmEviSK/y1eLl6T7xh2UTaQc6ZkVvb\n\t1FSmMVvANsw5a2IsxXrvoOutlapTgycjo0+L8qHU=","Date":"Fri, 5 Feb 2021 15:01:56 +0900","From":"paul.elder@ideasonboard.com","To":"Laurent Pinchart ","Message-ID":"<20210205060156.GA57940@pyrite.rasen.tech>","References":"<20201224081534.41601-1-paul.elder@ideasonboard.com>\n\t<20201224081534.41601-3-paul.elder@ideasonboard.com>\n\t<3ce53459-3c34-710a-b4e3-01cc4ff6e8f3@ideasonboard.com>\n\t<20210115051141.GA1885@pyrite.rasen.tech>\n\t","MIME-Version":"1.0","Content-Disposition":"inline","In-Reply-To":"","Subject":"Re: [libcamera-devel] [PATCH v6 2/9] utils: ipc: add templates for\n\tcode generation for IPC mechanism","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Cc":"libcamera-devel@lists.libcamera.org","Content-Type":"text/plain; charset=\"us-ascii\"","Content-Transfer-Encoding":"7bit","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" "}}]