From patchwork Fri Aug 9 00:59:09 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Laurent Pinchart X-Patchwork-Id: 20855 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 43815C324E for ; Fri, 9 Aug 2024 00:59:52 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id BF47E633CC; Fri, 9 Aug 2024 02:59:51 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="lmBxY0tJ"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 6129B633B5 for ; Fri, 9 Aug 2024 02:59:46 +0200 (CEST) Received: from pendragon.ideasonboard.com (81-175-209-231.bb.dnainternet.fi [81.175.209.231]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 6F67A83F for ; Fri, 9 Aug 2024 02:58:52 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1723165132; bh=94VQjMKqgNrCpBOkdblxoiuR+MwLFIQit34kpjxveUg=; h=From:To:Subject:Date:In-Reply-To:References:From; b=lmBxY0tJlImRmWdJKyUnyv4oEtwfwgImXlveumCU6210QmJ/dI+Ipb5OWd2jgAe0T 2Qy8zOoIRqeW45hxcPGzxEKd7gbPyGieFMZZ+lJrY+62VW9rUwzgKjRd2qlX1Cx43x flS4VVKfHoyID4rUKyFRX/QTb5usdf+EhNNFKPPo= From: Laurent Pinchart To: libcamera-devel@lists.libcamera.org Subject: [PATCH 05/10] meson: utils: Provide environment for Python scripts Date: Fri, 9 Aug 2024 03:59:09 +0300 Message-ID: <20240809005914.20662-6-laurent.pinchart@ideasonboard.com> X-Mailer: git-send-email 2.44.2 In-Reply-To: <20240809005914.20662-1-laurent.pinchart@ideasonboard.com> References: <20240809005914.20662-1-laurent.pinchart@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Python scripts run as part of the build process need to take a few actions specific to the environment in which they operate. One of those is disabling the Python bytecode cache, to avoid writing file to the source tree. This is done manually in the IPC generate.py and parser.py scripts. The current implementation is not ideal because it hardcodes in the scripts information related to the environment in which they operate. As those scripts are part of libcamera this is more of a theoretical issue than a practical one. A second issue is that future Python scripts will need to duplicate similar mechanisms, resulting in a higher maintenance burden. Address the issue with a different approach, by creating a meson environment for the Python scripts, and passing it to the custom_target() functions. The environment only disables the bytecode cache for now. The diffstat shows an increase in code size. This is expected to be offset by usage of the environment for more Python scripts, as well as support of more variables in the environment. Signed-off-by: Laurent Pinchart Reviewed-by: Tomi Valkeinen Reviewed-by: Daniel Scally Reviewed-by: Paul Elder --- include/libcamera/ipa/meson.build | 21 ++++++++++++------- src/libcamera/proxy/meson.build | 3 ++- src/libcamera/proxy/worker/meson.build | 3 ++- .../include/libcamera/ipa/meson.build | 9 +++++--- utils/codegen/ipc/generate.py | 3 --- utils/codegen/ipc/meson.build | 3 ++- utils/codegen/ipc/parser.py | 3 --- utils/codegen/meson.build | 4 ++++ 8 files changed, 30 insertions(+), 19 deletions(-) diff --git a/include/libcamera/ipa/meson.build b/include/libcamera/ipa/meson.build index 96fca42cc0b8..bf55e124e97e 100644 --- a/include/libcamera/ipa/meson.build +++ b/include/libcamera/ipa/meson.build @@ -26,7 +26,8 @@ ipa_mojom_core = custom_target(core_mojom_file.split('.')[0] + '_mojom_module', '--output-root', meson.project_build_root(), '--input-root', meson.project_source_root(), '--mojoms', '@INPUT@' - ]) + ], + env : py_build_env) # core_ipa_interface.h libcamera_ipa_headers += custom_target('core_ipa_interface_h', @@ -42,7 +43,8 @@ libcamera_ipa_headers += custom_target('core_ipa_interface_h', '--libcamera_generate_core_header', '--libcamera_output_path=@OUTPUT@', './' +'@INPUT@' - ]) + ], + env : py_build_env) # core_ipa_serializer.h libcamera_ipa_headers += custom_target('core_ipa_serializer_h', @@ -56,7 +58,8 @@ libcamera_ipa_headers += custom_target('core_ipa_serializer_h', '--libcamera_generate_core_serializer', '--libcamera_output_path=@OUTPUT@', './' +'@INPUT@' - ]) + ], + env : py_build_env) # Mapping from pipeline handler name to mojom file pipeline_ipa_mojom_mapping = { @@ -99,7 +102,8 @@ foreach pipeline, file : pipeline_ipa_mojom_mapping '--output-root', meson.project_build_root(), '--input-root', meson.project_source_root(), '--mojoms', '@INPUT@' - ]) + ], + env : py_build_env) # {interface}_ipa_interface.h header = custom_target(name + '_ipa_interface_h', @@ -115,7 +119,8 @@ foreach pipeline, file : pipeline_ipa_mojom_mapping '--libcamera_generate_header', '--libcamera_output_path=@OUTPUT@', './' +'@INPUT@' - ]) + ], + env : py_build_env) # {interface}_ipa_serializer.h serializer = custom_target(name + '_ipa_serializer_h', @@ -129,7 +134,8 @@ foreach pipeline, file : pipeline_ipa_mojom_mapping '--libcamera_generate_serializer', '--libcamera_output_path=@OUTPUT@', './' +'@INPUT@' - ]) + ], + env : py_build_env) # {interface}_ipa_proxy.h proxy_header = custom_target(name + '_proxy_h', @@ -143,7 +149,8 @@ foreach pipeline, file : pipeline_ipa_mojom_mapping '--libcamera_generate_proxy_h', '--libcamera_output_path=@OUTPUT@', './' +'@INPUT@' - ]) + ], + env : py_build_env) ipa_mojoms += { 'name': name, diff --git a/src/libcamera/proxy/meson.build b/src/libcamera/proxy/meson.build index d7de518a0549..8bd1b135d0f5 100644 --- a/src/libcamera/proxy/meson.build +++ b/src/libcamera/proxy/meson.build @@ -13,7 +13,8 @@ foreach mojom : ipa_mojoms '--libcamera_generate_proxy_cpp', '--libcamera_output_path=@OUTPUT@', './' + '@INPUT@' - ]) + ], + env : py_build_env) libcamera_internal_sources += proxy endforeach diff --git a/src/libcamera/proxy/worker/meson.build b/src/libcamera/proxy/worker/meson.build index b5ab9794c622..8c54a2e206a5 100644 --- a/src/libcamera/proxy/worker/meson.build +++ b/src/libcamera/proxy/worker/meson.build @@ -15,7 +15,8 @@ foreach mojom : ipa_mojoms '--libcamera_generate_proxy_worker', '--libcamera_output_path=@OUTPUT@', './' + '@INPUT@' - ]) + ], + env : py_build_env) proxy = executable(mojom['name'] + '_ipa_proxy', worker, install : true, diff --git a/test/serialization/generated_serializer/include/libcamera/ipa/meson.build b/test/serialization/generated_serializer/include/libcamera/ipa/meson.build index 6f8794c188a3..ae08e9bee6b9 100644 --- a/test/serialization/generated_serializer/include/libcamera/ipa/meson.build +++ b/test/serialization/generated_serializer/include/libcamera/ipa/meson.build @@ -9,7 +9,8 @@ mojom = custom_target('test_mojom_module', '--output-root', meson.project_build_root(), '--input-root', meson.project_source_root(), '--mojoms', '@INPUT@' - ]) + ], + env : py_build_env) # test_ipa_interface.h generated_test_header = custom_target('test_ipa_interface_h', @@ -23,7 +24,8 @@ generated_test_header = custom_target('test_ipa_interface_h', '--libcamera_generate_header', '--libcamera_output_path=@OUTPUT@', './' +'@INPUT@' - ]) + ], + env : py_build_env) # test_ipa_serializer.h generated_test_serializer = custom_target('test_ipa_serializer_h', @@ -37,4 +39,5 @@ generated_test_serializer = custom_target('test_ipa_serializer_h', '--libcamera_generate_serializer', '--libcamera_output_path=@OUTPUT@', './' +'@INPUT@' - ]) + ], + env : py_build_env) diff --git a/utils/codegen/ipc/generate.py b/utils/codegen/ipc/generate.py index c2b3fcb72e1f..dfbe659bc0ca 100755 --- a/utils/codegen/ipc/generate.py +++ b/utils/codegen/ipc/generate.py @@ -9,9 +9,6 @@ import os import sys -# TODO set sys.pycache_prefix for >= python3.8 -sys.dont_write_bytecode = True - sys.path.insert(0, f'{os.path.dirname(__file__)}/mojo/public/tools/bindings') import mojo.public.tools.bindings.mojom_bindings_generator as generator diff --git a/utils/codegen/ipc/meson.build b/utils/codegen/ipc/meson.build index 973a5417dc99..f77bf32497ba 100644 --- a/utils/codegen/ipc/meson.build +++ b/utils/codegen/ipc/meson.build @@ -13,6 +13,7 @@ mojom_docs_extractor = find_program('./extract-docs.py') mojom_templates = custom_target('mojom_templates', input : mojom_template_files, output : 'libcamera_templates.zip', - command : [mojom_generator, '-o', '@OUTDIR@', 'precompile']) + command : [mojom_generator, '-o', '@OUTDIR@', 'precompile'], + env : py_build_env) mojom_templates_dir = meson.current_build_dir() diff --git a/utils/codegen/ipc/parser.py b/utils/codegen/ipc/parser.py index cb5608b7c165..8e70322d1bdb 100755 --- a/utils/codegen/ipc/parser.py +++ b/utils/codegen/ipc/parser.py @@ -9,9 +9,6 @@ import os import sys -# TODO set sys.pycache_prefix for >= python3.8 -sys.dont_write_bytecode = True - # Make sure that mojom_parser.py can import mojom sys.path.insert(0, f'{os.path.dirname(__file__)}/mojo/public/tools/mojom') diff --git a/utils/codegen/meson.build b/utils/codegen/meson.build index 7dd312e16559..fb2196ee0d20 100644 --- a/utils/codegen/meson.build +++ b/utils/codegen/meson.build @@ -2,6 +2,10 @@ ## Code generation +py_build_env = environment() +# \todo Investigate usage of PYTHONPYCACHEPREFIX for Python >= 3.8 +py_build_env.set('PYTHONDONTWRITEBYTECODE', '1') + py_modules += ['jinja2', 'yaml'] gen_controls = files('gen-controls.py')