Patch Detail
Show a patch.
GET /api/1.1/patches/20855/?format=api
{ "id": 20855, "url": "https://patchwork.libcamera.org/api/1.1/patches/20855/?format=api", "web_url": "https://patchwork.libcamera.org/patch/20855/", "project": { "id": 1, "url": "https://patchwork.libcamera.org/api/1.1/projects/1/?format=api", "name": "libcamera", "link_name": "libcamera", "list_id": "libcamera_core", "list_email": "libcamera-devel@lists.libcamera.org", "web_url": "", "scm_url": "", "webscm_url": "" }, "msgid": "<20240809005914.20662-6-laurent.pinchart@ideasonboard.com>", "date": "2024-08-09T00:59:09", "name": "[05/10] meson: utils: Provide environment for Python scripts", "commit_ref": null, "pull_url": null, "state": "accepted", "archived": false, "hash": "cb8fccf8fb57d60cccf82636cb015a35aa2244f3", "submitter": { "id": 2, "url": "https://patchwork.libcamera.org/api/1.1/people/2/?format=api", "name": "Laurent Pinchart", "email": "laurent.pinchart@ideasonboard.com" }, "delegate": null, "mbox": "https://patchwork.libcamera.org/patch/20855/mbox/", "series": [ { "id": 4506, "url": "https://patchwork.libcamera.org/api/1.1/series/4506/?format=api", "web_url": "https://patchwork.libcamera.org/project/libcamera/list/?series=4506", "date": "2024-08-09T00:59:04", "name": "libcamera: Improve code generation for controls", "version": 1, "mbox": "https://patchwork.libcamera.org/series/4506/mbox/" } ], "comments": "https://patchwork.libcamera.org/api/patches/20855/comments/", "check": "pending", "checks": "https://patchwork.libcamera.org/api/patches/20855/checks/", "tags": {}, "headers": { "Return-Path": "<libcamera-devel-bounces@lists.libcamera.org>", "X-Original-To": "parsemail@patchwork.libcamera.org", "Delivered-To": "parsemail@patchwork.libcamera.org", "Received": [ "from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id 43815C324E\n\tfor <parsemail@patchwork.libcamera.org>;\n\tFri, 9 Aug 2024 00:59:52 +0000 (UTC)", "from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id BF47E633CC;\n\tFri, 9 Aug 2024 02:59:51 +0200 (CEST)", "from perceval.ideasonboard.com (perceval.ideasonboard.com\n\t[IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id 6129B633B5\n\tfor <libcamera-devel@lists.libcamera.org>;\n\tFri, 9 Aug 2024 02:59:46 +0200 (CEST)", "from pendragon.ideasonboard.com (81-175-209-231.bb.dnainternet.fi\n\t[81.175.209.231])\n\tby perceval.ideasonboard.com (Postfix) with ESMTPSA id 6F67A83F\n\tfor <libcamera-devel@lists.libcamera.org>;\n\tFri, 9 Aug 2024 02:58:52 +0200 (CEST)" ], "Authentication-Results": "lancelot.ideasonboard.com; dkim=pass (1024-bit key;\n\tunprotected) header.d=ideasonboard.com header.i=@ideasonboard.com\n\theader.b=\"lmBxY0tJ\"; dkim-atps=neutral", "DKIM-Signature": "v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com;\n\ts=mail; t=1723165132;\n\tbh=94VQjMKqgNrCpBOkdblxoiuR+MwLFIQit34kpjxveUg=;\n\th=From:To:Subject:Date:In-Reply-To:References:From;\n\tb=lmBxY0tJlImRmWdJKyUnyv4oEtwfwgImXlveumCU6210QmJ/dI+Ipb5OWd2jgAe0T\n\t2Qy8zOoIRqeW45hxcPGzxEKd7gbPyGieFMZZ+lJrY+62VW9rUwzgKjRd2qlX1Cx43x\n\tflS4VVKfHoyID4rUKyFRX/QTb5usdf+EhNNFKPPo=", "From": "Laurent Pinchart <laurent.pinchart@ideasonboard.com>", "To": "libcamera-devel@lists.libcamera.org", "Subject": "[PATCH 05/10] meson: utils: Provide environment for Python scripts", "Date": "Fri, 9 Aug 2024 03:59:09 +0300", "Message-ID": "<20240809005914.20662-6-laurent.pinchart@ideasonboard.com>", "X-Mailer": "git-send-email 2.44.2", "In-Reply-To": "<20240809005914.20662-1-laurent.pinchart@ideasonboard.com>", "References": "<20240809005914.20662-1-laurent.pinchart@ideasonboard.com>", "MIME-Version": "1.0", "Content-Transfer-Encoding": "8bit", "X-BeenThere": "libcamera-devel@lists.libcamera.org", "X-Mailman-Version": "2.1.29", "Precedence": "list", "List-Id": "<libcamera-devel.lists.libcamera.org>", "List-Unsubscribe": "<https://lists.libcamera.org/options/libcamera-devel>,\n\t<mailto:libcamera-devel-request@lists.libcamera.org?subject=unsubscribe>", "List-Archive": "<https://lists.libcamera.org/pipermail/libcamera-devel/>", "List-Post": "<mailto:libcamera-devel@lists.libcamera.org>", "List-Help": "<mailto:libcamera-devel-request@lists.libcamera.org?subject=help>", "List-Subscribe": "<https://lists.libcamera.org/listinfo/libcamera-devel>,\n\t<mailto:libcamera-devel-request@lists.libcamera.org?subject=subscribe>", "Errors-To": "libcamera-devel-bounces@lists.libcamera.org", "Sender": "\"libcamera-devel\" <libcamera-devel-bounces@lists.libcamera.org>" }, "content": "Python scripts run as part of the build process need to take a few\nactions specific to the environment in which they operate. One of those\nis disabling the Python bytecode cache, to avoid writing file to the\nsource tree. This is done manually in the IPC generate.py and parser.py\nscripts.\n\nThe current implementation is not ideal because it hardcodes in the\nscripts information related to the environment in which they operate. As\nthose scripts are part of libcamera this is more of a theoretical issue\nthan a practical one. A second issue is that future Python scripts will\nneed to duplicate similar mechanisms, resulting in a higher maintenance\nburden.\n\nAddress the issue with a different approach, by creating a meson\nenvironment for the Python scripts, and passing it to the\ncustom_target() functions. The environment only disables the bytecode\ncache for now.\n\nThe diffstat shows an increase in code size. This is expected to be\noffset by usage of the environment for more Python scripts, as well as\nsupport of more variables in the environment.\n\nSigned-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>\n---\n include/libcamera/ipa/meson.build | 21 ++++++++++++-------\n src/libcamera/proxy/meson.build | 3 ++-\n src/libcamera/proxy/worker/meson.build | 3 ++-\n .../include/libcamera/ipa/meson.build | 9 +++++---\n utils/codegen/ipc/generate.py | 3 ---\n utils/codegen/ipc/meson.build | 3 ++-\n utils/codegen/ipc/parser.py | 3 ---\n utils/codegen/meson.build | 4 ++++\n 8 files changed, 30 insertions(+), 19 deletions(-)", "diff": "diff --git a/include/libcamera/ipa/meson.build b/include/libcamera/ipa/meson.build\nindex 96fca42cc0b8..bf55e124e97e 100644\n--- a/include/libcamera/ipa/meson.build\n+++ b/include/libcamera/ipa/meson.build\n@@ -26,7 +26,8 @@ ipa_mojom_core = custom_target(core_mojom_file.split('.')[0] + '_mojom_module',\n '--output-root', meson.project_build_root(),\n '--input-root', meson.project_source_root(),\n '--mojoms', '@INPUT@'\n- ])\n+ ],\n+ env : py_build_env)\n \n # core_ipa_interface.h\n libcamera_ipa_headers += custom_target('core_ipa_interface_h',\n@@ -42,7 +43,8 @@ libcamera_ipa_headers += custom_target('core_ipa_interface_h',\n '--libcamera_generate_core_header',\n '--libcamera_output_path=@OUTPUT@',\n './' +'@INPUT@'\n- ])\n+ ],\n+ env : py_build_env)\n \n # core_ipa_serializer.h\n libcamera_ipa_headers += custom_target('core_ipa_serializer_h',\n@@ -56,7 +58,8 @@ libcamera_ipa_headers += custom_target('core_ipa_serializer_h',\n '--libcamera_generate_core_serializer',\n '--libcamera_output_path=@OUTPUT@',\n './' +'@INPUT@'\n- ])\n+ ],\n+ env : py_build_env)\n \n # Mapping from pipeline handler name to mojom file\n pipeline_ipa_mojom_mapping = {\n@@ -99,7 +102,8 @@ foreach pipeline, file : pipeline_ipa_mojom_mapping\n '--output-root', meson.project_build_root(),\n '--input-root', meson.project_source_root(),\n '--mojoms', '@INPUT@'\n- ])\n+ ],\n+ env : py_build_env)\n \n # {interface}_ipa_interface.h\n header = custom_target(name + '_ipa_interface_h',\n@@ -115,7 +119,8 @@ foreach pipeline, file : pipeline_ipa_mojom_mapping\n '--libcamera_generate_header',\n '--libcamera_output_path=@OUTPUT@',\n './' +'@INPUT@'\n- ])\n+ ],\n+ env : py_build_env)\n \n # {interface}_ipa_serializer.h\n serializer = custom_target(name + '_ipa_serializer_h',\n@@ -129,7 +134,8 @@ foreach pipeline, file : pipeline_ipa_mojom_mapping\n '--libcamera_generate_serializer',\n '--libcamera_output_path=@OUTPUT@',\n './' +'@INPUT@'\n- ])\n+ ],\n+ env : py_build_env)\n \n # {interface}_ipa_proxy.h\n proxy_header = custom_target(name + '_proxy_h',\n@@ -143,7 +149,8 @@ foreach pipeline, file : pipeline_ipa_mojom_mapping\n '--libcamera_generate_proxy_h',\n '--libcamera_output_path=@OUTPUT@',\n './' +'@INPUT@'\n- ])\n+ ],\n+ env : py_build_env)\n \n ipa_mojoms += {\n 'name': name,\ndiff --git a/src/libcamera/proxy/meson.build b/src/libcamera/proxy/meson.build\nindex d7de518a0549..8bd1b135d0f5 100644\n--- a/src/libcamera/proxy/meson.build\n+++ b/src/libcamera/proxy/meson.build\n@@ -13,7 +13,8 @@ foreach mojom : ipa_mojoms\n '--libcamera_generate_proxy_cpp',\n '--libcamera_output_path=@OUTPUT@',\n './' + '@INPUT@'\n- ])\n+ ],\n+ env : py_build_env)\n \n libcamera_internal_sources += proxy\n endforeach\ndiff --git a/src/libcamera/proxy/worker/meson.build b/src/libcamera/proxy/worker/meson.build\nindex b5ab9794c622..8c54a2e206a5 100644\n--- a/src/libcamera/proxy/worker/meson.build\n+++ b/src/libcamera/proxy/worker/meson.build\n@@ -15,7 +15,8 @@ foreach mojom : ipa_mojoms\n '--libcamera_generate_proxy_worker',\n '--libcamera_output_path=@OUTPUT@',\n './' + '@INPUT@'\n- ])\n+ ],\n+ env : py_build_env)\n \n proxy = executable(mojom['name'] + '_ipa_proxy', worker,\n install : true,\ndiff --git a/test/serialization/generated_serializer/include/libcamera/ipa/meson.build b/test/serialization/generated_serializer/include/libcamera/ipa/meson.build\nindex 6f8794c188a3..ae08e9bee6b9 100644\n--- a/test/serialization/generated_serializer/include/libcamera/ipa/meson.build\n+++ b/test/serialization/generated_serializer/include/libcamera/ipa/meson.build\n@@ -9,7 +9,8 @@ mojom = custom_target('test_mojom_module',\n '--output-root', meson.project_build_root(),\n '--input-root', meson.project_source_root(),\n '--mojoms', '@INPUT@'\n- ])\n+ ],\n+ env : py_build_env)\n \n # test_ipa_interface.h\n generated_test_header = custom_target('test_ipa_interface_h',\n@@ -23,7 +24,8 @@ generated_test_header = custom_target('test_ipa_interface_h',\n '--libcamera_generate_header',\n '--libcamera_output_path=@OUTPUT@',\n './' +'@INPUT@'\n- ])\n+ ],\n+ env : py_build_env)\n \n # test_ipa_serializer.h\n generated_test_serializer = custom_target('test_ipa_serializer_h',\n@@ -37,4 +39,5 @@ generated_test_serializer = custom_target('test_ipa_serializer_h',\n '--libcamera_generate_serializer',\n '--libcamera_output_path=@OUTPUT@',\n './' +'@INPUT@'\n- ])\n+ ],\n+ env : py_build_env)\ndiff --git a/utils/codegen/ipc/generate.py b/utils/codegen/ipc/generate.py\nindex c2b3fcb72e1f..dfbe659bc0ca 100755\n--- a/utils/codegen/ipc/generate.py\n+++ b/utils/codegen/ipc/generate.py\n@@ -9,9 +9,6 @@\n import os\n import sys\n \n-# TODO set sys.pycache_prefix for >= python3.8\n-sys.dont_write_bytecode = True\n-\n sys.path.insert(0, f'{os.path.dirname(__file__)}/mojo/public/tools/bindings')\n \n import mojo.public.tools.bindings.mojom_bindings_generator as generator\ndiff --git a/utils/codegen/ipc/meson.build b/utils/codegen/ipc/meson.build\nindex 973a5417dc99..f77bf32497ba 100644\n--- a/utils/codegen/ipc/meson.build\n+++ b/utils/codegen/ipc/meson.build\n@@ -13,6 +13,7 @@ mojom_docs_extractor = find_program('./extract-docs.py')\n mojom_templates = custom_target('mojom_templates',\n input : mojom_template_files,\n output : 'libcamera_templates.zip',\n- command : [mojom_generator, '-o', '@OUTDIR@', 'precompile'])\n+ command : [mojom_generator, '-o', '@OUTDIR@', 'precompile'],\n+ env : py_build_env)\n \n mojom_templates_dir = meson.current_build_dir()\ndiff --git a/utils/codegen/ipc/parser.py b/utils/codegen/ipc/parser.py\nindex cb5608b7c165..8e70322d1bdb 100755\n--- a/utils/codegen/ipc/parser.py\n+++ b/utils/codegen/ipc/parser.py\n@@ -9,9 +9,6 @@\n import os\n import sys\n \n-# TODO set sys.pycache_prefix for >= python3.8\n-sys.dont_write_bytecode = True\n-\n # Make sure that mojom_parser.py can import mojom\n sys.path.insert(0, f'{os.path.dirname(__file__)}/mojo/public/tools/mojom')\n \ndiff --git a/utils/codegen/meson.build b/utils/codegen/meson.build\nindex 7dd312e16559..fb2196ee0d20 100644\n--- a/utils/codegen/meson.build\n+++ b/utils/codegen/meson.build\n@@ -2,6 +2,10 @@\n \n ## Code generation\n \n+py_build_env = environment()\n+# \\todo Investigate usage of PYTHONPYCACHEPREFIX for Python >= 3.8\n+py_build_env.set('PYTHONDONTWRITEBYTECODE', '1')\n+\n py_modules += ['jinja2', 'yaml']\n \n gen_controls = files('gen-controls.py')\n", "prefixes": [ "05/10" ] }