[00/10] libcamera: Improve code generation for controls

20850 mbox series

Hello,

This patch series started small by scratching an itch and wanting to
have the ability to share some of the code generation code for controls
with the upcoming controls support in libcamerasrc. As is too often the
case, it then grew to 10 patches long, with a conversion of the
templates to jinja2.

I don't have much else to add here, the diffstat will hopefully speak by
itself.

Laurent Pinchart (10):
  meson: Store controls and properties YAML files in variables
  meson: Fix mismatch in controls and properties generated file names
  libcamera: Drop remaining file names from header comment blocks
  meson: Move all code generation scripts to utils/codegen/
  meson: utils: Provide environment for Python scripts
  utils: codegen: gen-header.sh: Generate libcamera.h based on
    meson.build
  utils: codegen: gen-controls.py: Convert to jinja2 templates
  utils: codegen: gen-controls.py: Move helper classes to separate file
  py: gen-py-controls: Use Control class
  py: gen-py-controls: Convert to jinja2 templates

 include/libcamera/control_ids.h.in            |  40 +-
 include/libcamera/internal/meson.build        |   2 +-
 include/libcamera/ipa/meson.build             |  21 +-
 include/libcamera/meson.build                 |  31 +-
 include/libcamera/property_ids.h.in           |  34 --
 meson.build                                   |   4 +-
 src/libcamera/control_ids.cpp.in              | 101 ++++-
 src/libcamera/meson.build                     |  30 +-
 src/libcamera/property_ids.cpp.in             |  48 ---
 src/libcamera/proxy/meson.build               |   3 +-
 src/libcamera/proxy/worker/meson.build        |   3 +-
 src/py/libcamera/gen-py-controls.py           | 122 +++---
 src/py/libcamera/meson.build                  |  27 +-
 src/py/libcamera/py_controls_generated.cpp.in |  35 +-
 .../libcamera/py_properties_generated.cpp.in  |  30 --
 .../include/libcamera/ipa/meson.build         |   9 +-
 utils/codegen/controls.py                     | 112 +++++
 utils/codegen/gen-controls.py                 | 111 +++++
 utils/{ => codegen}/gen-formats.py            |   0
 utils/{ => codegen}/gen-header.sh             |   7 +-
 utils/{ => codegen}/gen-ipa-pub-key.py        |   0
 .../{tracepoints => codegen}/gen-tp-header.py |   0
 utils/{ => codegen}/ipc/extract-docs.py       |   0
 utils/{ => codegen}/ipc/generate.py           |   3 -
 .../{ => codegen}/ipc/generators/__init__.py  |   0
 .../core_ipa_interface.h.tmpl                 |   0
 .../core_ipa_serializer.h.tmpl                |   0
 .../definition_functions.tmpl                 |   0
 .../libcamera_templates/meson.build           |   0
 .../module_ipa_interface.h.tmpl               |   0
 .../module_ipa_proxy.cpp.tmpl                 |   0
 .../module_ipa_proxy.h.tmpl                   |   0
 .../module_ipa_proxy_worker.cpp.tmpl          |   0
 .../module_ipa_serializer.h.tmpl              |   0
 .../libcamera_templates/proxy_functions.tmpl  |   0
 .../libcamera_templates/serializer.tmpl       |   0
 .../{ => codegen}/ipc/generators/meson.build  |   0
 .../generators/mojom_libcamera_generator.py   |   0
 utils/{ => codegen}/ipc/meson.build           |   3 +-
 utils/{ => codegen}/ipc/mojo/README           |   0
 utils/{ => codegen}/ipc/mojo/public/LICENSE   |   0
 .../ipc/mojo/public/tools/.style.yapf         |   0
 .../ipc/mojo/public/tools/BUILD.gn            |   0
 .../ipc/mojo/public/tools/bindings/BUILD.gn   |   0
 .../ipc/mojo/public/tools/bindings/README.md  |   0
 .../public/tools/bindings/checks/__init__.py  |   0
 .../bindings/checks/mojom_attributes_check.py |   0
 .../checks/mojom_attributes_check_unittest.py |   0
 .../checks/mojom_definitions_check.py         |   0
 .../checks/mojom_interface_feature_check.py   |   0
 .../mojom_interface_feature_check_unittest.py |   0
 .../checks/mojom_restrictions_check.py        |   0
 .../mojom_restrictions_checks_unittest.py     |   0
 .../tools/bindings/concatenate-files.py       |   0
 ...concatenate_and_replace_closure_exports.py |   0
 .../tools/bindings/gen_data_files_list.py     |   0
 .../tools/bindings/generate_type_mappings.py  |   0
 .../tools/bindings/minify_with_terser.py      |   0
 .../ipc/mojo/public/tools/bindings/mojom.gni  |   0
 .../bindings/mojom_bindings_generator.py      |   0
 .../mojom_bindings_generator_unittest.py      |   0
 .../tools/bindings/validate_typemap_config.py |   0
 .../ipc/mojo/public/tools/mojom/BUILD.gn      |   0
 .../ipc/mojo/public/tools/mojom/README.md     |   0
 .../mojom/check_stable_mojom_compatibility.py |   0
 ...eck_stable_mojom_compatibility_unittest.py |   0
 .../mojo/public/tools/mojom/const_unittest.py |   0
 .../mojo/public/tools/mojom/enum_unittest.py  |   0
 .../public/tools/mojom/feature_unittest.py    |   0
 .../mojo/public/tools/mojom/mojom/BUILD.gn    |   0
 .../mojo/public/tools/mojom/mojom/__init__.py |   0
 .../mojo/public/tools/mojom/mojom/error.py    |   0
 .../mojo/public/tools/mojom/mojom/fileutil.py |   0
 .../tools/mojom/mojom/fileutil_unittest.py    |   0
 .../tools/mojom/mojom/generate/__init__.py    |   0
 .../tools/mojom/mojom/generate/check.py       |   0
 .../tools/mojom/mojom/generate/generator.py   |   0
 .../mojom/generate/generator_unittest.py      |   0
 .../tools/mojom/mojom/generate/module.py      |   0
 .../mojom/mojom/generate/module_unittest.py   |   0
 .../public/tools/mojom/mojom/generate/pack.py |   0
 .../mojom/mojom/generate/pack_unittest.py     |   0
 .../mojom/mojom/generate/template_expander.py |   0
 .../tools/mojom/mojom/generate/translate.py   |   0
 .../mojom/generate/translate_unittest.py      |   0
 .../tools/mojom/mojom/parse/__init__.py       |   0
 .../public/tools/mojom/mojom/parse/ast.py     |   0
 .../tools/mojom/mojom/parse/ast_unittest.py   |   0
 .../mojom/mojom/parse/conditional_features.py |   0
 .../parse/conditional_features_unittest.py    |   0
 .../public/tools/mojom/mojom/parse/lexer.py   |   0
 .../tools/mojom/mojom/parse/lexer_unittest.py |   0
 .../public/tools/mojom/mojom/parse/parser.py  |   0
 .../mojom/mojom/parse/parser_unittest.py      |   0
 .../mojo/public/tools/mojom/mojom_parser.py   |   0
 .../tools/mojom/mojom_parser_test_case.py     |   0
 .../tools/mojom/mojom_parser_unittest.py      |   0
 .../tools/mojom/stable_attribute_unittest.py  |   0
 .../mojo/public/tools/mojom/union_unittest.py |   0
 .../mojom/version_compatibility_unittest.py   |   0
 .../public/tools/run_all_python_unittests.py  |   0
 utils/{ => codegen}/ipc/parser.py             |   3 -
 utils/{ => codegen}/ipc/tools/README          |   0
 .../ipc/tools/diagnosis/crbug_1001171.py      |   0
 utils/codegen/meson.build                     |  18 +
 utils/gen-controls.py                         | 389 ------------------
 utils/meson.build                             |  10 +-
 utils/tracepoints/meson.build                 |   5 -
 108 files changed, 513 insertions(+), 688 deletions(-)
 delete mode 100644 include/libcamera/property_ids.h.in
 delete mode 100644 src/libcamera/property_ids.cpp.in
 delete mode 100644 src/py/libcamera/py_properties_generated.cpp.in
 create mode 100644 utils/codegen/controls.py
 create mode 100755 utils/codegen/gen-controls.py
 rename utils/{ => codegen}/gen-formats.py (100%)
 rename utils/{ => codegen}/gen-header.sh (76%)
 rename utils/{ => codegen}/gen-ipa-pub-key.py (100%)
 rename utils/{tracepoints => codegen}/gen-tp-header.py (100%)
 rename utils/{ => codegen}/ipc/extract-docs.py (100%)
 rename utils/{ => codegen}/ipc/generate.py (94%)
 rename utils/{ => codegen}/ipc/generators/__init__.py (100%)
 rename utils/{ => codegen}/ipc/generators/libcamera_templates/core_ipa_interface.h.tmpl (100%)
 rename utils/{ => codegen}/ipc/generators/libcamera_templates/core_ipa_serializer.h.tmpl (100%)
 rename utils/{ => codegen}/ipc/generators/libcamera_templates/definition_functions.tmpl (100%)
 rename utils/{ => codegen}/ipc/generators/libcamera_templates/meson.build (100%)
 rename utils/{ => codegen}/ipc/generators/libcamera_templates/module_ipa_interface.h.tmpl (100%)
 rename utils/{ => codegen}/ipc/generators/libcamera_templates/module_ipa_proxy.cpp.tmpl (100%)
 rename utils/{ => codegen}/ipc/generators/libcamera_templates/module_ipa_proxy.h.tmpl (100%)
 rename utils/{ => codegen}/ipc/generators/libcamera_templates/module_ipa_proxy_worker.cpp.tmpl (100%)
 rename utils/{ => codegen}/ipc/generators/libcamera_templates/module_ipa_serializer.h.tmpl (100%)
 rename utils/{ => codegen}/ipc/generators/libcamera_templates/proxy_functions.tmpl (100%)
 rename utils/{ => codegen}/ipc/generators/libcamera_templates/serializer.tmpl (100%)
 rename utils/{ => codegen}/ipc/generators/meson.build (100%)
 rename utils/{ => codegen}/ipc/generators/mojom_libcamera_generator.py (100%)
 rename utils/{ => codegen}/ipc/meson.build (86%)
 rename utils/{ => codegen}/ipc/mojo/README (100%)
 rename utils/{ => codegen}/ipc/mojo/public/LICENSE (100%)
 rename utils/{ => codegen}/ipc/mojo/public/tools/.style.yapf (100%)
 rename utils/{ => codegen}/ipc/mojo/public/tools/BUILD.gn (100%)
 rename utils/{ => codegen}/ipc/mojo/public/tools/bindings/BUILD.gn (100%)
 rename utils/{ => codegen}/ipc/mojo/public/tools/bindings/README.md (100%)
 rename utils/{ => codegen}/ipc/mojo/public/tools/bindings/checks/__init__.py (100%)
 rename utils/{ => codegen}/ipc/mojo/public/tools/bindings/checks/mojom_attributes_check.py (100%)
 rename utils/{ => codegen}/ipc/mojo/public/tools/bindings/checks/mojom_attributes_check_unittest.py (100%)
 rename utils/{ => codegen}/ipc/mojo/public/tools/bindings/checks/mojom_definitions_check.py (100%)
 rename utils/{ => codegen}/ipc/mojo/public/tools/bindings/checks/mojom_interface_feature_check.py (100%)
 rename utils/{ => codegen}/ipc/mojo/public/tools/bindings/checks/mojom_interface_feature_check_unittest.py (100%)
 rename utils/{ => codegen}/ipc/mojo/public/tools/bindings/checks/mojom_restrictions_check.py (100%)
 rename utils/{ => codegen}/ipc/mojo/public/tools/bindings/checks/mojom_restrictions_checks_unittest.py (100%)
 rename utils/{ => codegen}/ipc/mojo/public/tools/bindings/concatenate-files.py (100%)
 rename utils/{ => codegen}/ipc/mojo/public/tools/bindings/concatenate_and_replace_closure_exports.py (100%)
 rename utils/{ => codegen}/ipc/mojo/public/tools/bindings/gen_data_files_list.py (100%)
 rename utils/{ => codegen}/ipc/mojo/public/tools/bindings/generate_type_mappings.py (100%)
 rename utils/{ => codegen}/ipc/mojo/public/tools/bindings/minify_with_terser.py (100%)
 rename utils/{ => codegen}/ipc/mojo/public/tools/bindings/mojom.gni (100%)
 rename utils/{ => codegen}/ipc/mojo/public/tools/bindings/mojom_bindings_generator.py (100%)
 rename utils/{ => codegen}/ipc/mojo/public/tools/bindings/mojom_bindings_generator_unittest.py (100%)
 rename utils/{ => codegen}/ipc/mojo/public/tools/bindings/validate_typemap_config.py (100%)
 rename utils/{ => codegen}/ipc/mojo/public/tools/mojom/BUILD.gn (100%)
 rename utils/{ => codegen}/ipc/mojo/public/tools/mojom/README.md (100%)
 rename utils/{ => codegen}/ipc/mojo/public/tools/mojom/check_stable_mojom_compatibility.py (100%)
 rename utils/{ => codegen}/ipc/mojo/public/tools/mojom/check_stable_mojom_compatibility_unittest.py (100%)
 rename utils/{ => codegen}/ipc/mojo/public/tools/mojom/const_unittest.py (100%)
 rename utils/{ => codegen}/ipc/mojo/public/tools/mojom/enum_unittest.py (100%)
 rename utils/{ => codegen}/ipc/mojo/public/tools/mojom/feature_unittest.py (100%)
 rename utils/{ => codegen}/ipc/mojo/public/tools/mojom/mojom/BUILD.gn (100%)
 rename utils/{ => codegen}/ipc/mojo/public/tools/mojom/mojom/__init__.py (100%)
 rename utils/{ => codegen}/ipc/mojo/public/tools/mojom/mojom/error.py (100%)
 rename utils/{ => codegen}/ipc/mojo/public/tools/mojom/mojom/fileutil.py (100%)
 rename utils/{ => codegen}/ipc/mojo/public/tools/mojom/mojom/fileutil_unittest.py (100%)
 rename utils/{ => codegen}/ipc/mojo/public/tools/mojom/mojom/generate/__init__.py (100%)
 rename utils/{ => codegen}/ipc/mojo/public/tools/mojom/mojom/generate/check.py (100%)
 rename utils/{ => codegen}/ipc/mojo/public/tools/mojom/mojom/generate/generator.py (100%)
 rename utils/{ => codegen}/ipc/mojo/public/tools/mojom/mojom/generate/generator_unittest.py (100%)
 rename utils/{ => codegen}/ipc/mojo/public/tools/mojom/mojom/generate/module.py (100%)
 rename utils/{ => codegen}/ipc/mojo/public/tools/mojom/mojom/generate/module_unittest.py (100%)
 rename utils/{ => codegen}/ipc/mojo/public/tools/mojom/mojom/generate/pack.py (100%)
 rename utils/{ => codegen}/ipc/mojo/public/tools/mojom/mojom/generate/pack_unittest.py (100%)
 rename utils/{ => codegen}/ipc/mojo/public/tools/mojom/mojom/generate/template_expander.py (100%)
 rename utils/{ => codegen}/ipc/mojo/public/tools/mojom/mojom/generate/translate.py (100%)
 rename utils/{ => codegen}/ipc/mojo/public/tools/mojom/mojom/generate/translate_unittest.py (100%)
 rename utils/{ => codegen}/ipc/mojo/public/tools/mojom/mojom/parse/__init__.py (100%)
 rename utils/{ => codegen}/ipc/mojo/public/tools/mojom/mojom/parse/ast.py (100%)
 rename utils/{ => codegen}/ipc/mojo/public/tools/mojom/mojom/parse/ast_unittest.py (100%)
 rename utils/{ => codegen}/ipc/mojo/public/tools/mojom/mojom/parse/conditional_features.py (100%)
 rename utils/{ => codegen}/ipc/mojo/public/tools/mojom/mojom/parse/conditional_features_unittest.py (100%)
 rename utils/{ => codegen}/ipc/mojo/public/tools/mojom/mojom/parse/lexer.py (100%)
 rename utils/{ => codegen}/ipc/mojo/public/tools/mojom/mojom/parse/lexer_unittest.py (100%)
 rename utils/{ => codegen}/ipc/mojo/public/tools/mojom/mojom/parse/parser.py (100%)
 rename utils/{ => codegen}/ipc/mojo/public/tools/mojom/mojom/parse/parser_unittest.py (100%)
 rename utils/{ => codegen}/ipc/mojo/public/tools/mojom/mojom_parser.py (100%)
 rename utils/{ => codegen}/ipc/mojo/public/tools/mojom/mojom_parser_test_case.py (100%)
 rename utils/{ => codegen}/ipc/mojo/public/tools/mojom/mojom_parser_unittest.py (100%)
 rename utils/{ => codegen}/ipc/mojo/public/tools/mojom/stable_attribute_unittest.py (100%)
 rename utils/{ => codegen}/ipc/mojo/public/tools/mojom/union_unittest.py (100%)
 rename utils/{ => codegen}/ipc/mojo/public/tools/mojom/version_compatibility_unittest.py (100%)
 rename utils/{ => codegen}/ipc/mojo/public/tools/run_all_python_unittests.py (100%)
 rename utils/{ => codegen}/ipc/parser.py (83%)
 rename utils/{ => codegen}/ipc/tools/README (100%)
 rename utils/{ => codegen}/ipc/tools/diagnosis/crbug_1001171.py (100%)
 create mode 100644 utils/codegen/meson.build
 delete mode 100755 utils/gen-controls.py
 delete mode 100644 utils/tracepoints/meson.build


base-commit: 1045522af99c9f26e0f583aa2310174f7ff42178