From patchwork Tue Aug 19 01:23:59 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Laurent Pinchart X-Patchwork-Id: 24160 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 3142BBDCC1 for ; Tue, 19 Aug 2025 01:24:30 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 7AA516926D; Tue, 19 Aug 2025 03:24:29 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="Ps00IPT3"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id F1C53613C5 for ; Tue, 19 Aug 2025 03:24:25 +0200 (CEST) Received: from pendragon.ideasonboard.com (81-175-209-231.bb.dnainternet.fi [81.175.209.231]) by perceval.ideasonboard.com (Postfix) with UTF8SMTPSA id 5D2FBC6D; Tue, 19 Aug 2025 03:23:28 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1755566608; bh=HFVJ7KHekzH6I2S+XJMkq2TsCTyNeUVcVZws6ggkezg=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=Ps00IPT3c372dgCpHP3yk9lTEVkJeF0Q6EsREMaa4UHHQOg6g3Ab3k9Ig0hzCANd9 3BircFuXuE86PoerqotBOgwTUVqFU2b8zStC+UqnsK/JMqI8ftvYiN8CAKI8GOkH5q Aa4WQrNBI1R4xZJQX7qmLKc5H+bPO51Hj6Wrau/g= From: Laurent Pinchart To: libcamera-devel@lists.libcamera.org Cc: William Vinnicombe , Tomi Valkeinen Subject: [PATCH v3 1/3] py: libcamera: Improve python binding installation Date: Tue, 19 Aug 2025 04:23:59 +0300 Message-ID: <20250819012402.8395-2-laurent.pinchart@ideasonboard.com> X-Mailer: git-send-email 2.49.1 In-Reply-To: <20250819012402.8395-1-laurent.pinchart@ideasonboard.com> References: <20250819012402.8395-1-laurent.pinchart@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" From: William Vinnicombe The existing meson.build file installs the bindings to a manually constructed directory that is not included in the Python path in most distributions. For instance, on a Debian 12 system, the modules is intalled in /usr/lib/x86_64-linux-gnu/python3.11/site-packages/, while the Python interpreter looks for site packages in /usr/lib/python3/dist-packages/. It also always builds the bindings using the system Python, as it searches for the Python library using the standard dependency() function. This prevents build the Python bindings for a different interpreter version without changing the system default interpreter. Modify the build process to use the meson python module to build the Python bindings targets, so it installs them to the correct directories for Python. This also allows specifying a different target Python interpreter through the '[binaries]' section of a meson native file. The behaviour is not changed for cross-compilation, as the meson python module has known issues in that case. Signed-off-by: William Vinnicombe Co-developed-by: Laurent Pinchart Signed-off-by: Laurent Pinchart --- src/py/libcamera/meson.build | 41 ++++++++++++++++++++++++------------ src/py/meson.build | 13 +++++++++++- test/py/meson.build | 3 --- 3 files changed, 40 insertions(+), 17 deletions(-) diff --git a/src/py/libcamera/meson.build b/src/py/libcamera/meson.build index 33ab65798559..fe22ffd2f70b 100644 --- a/src/py/libcamera/meson.build +++ b/src/py/libcamera/meson.build @@ -57,16 +57,35 @@ pycamera_args = [ '-DPYBIND11_USE_SMART_HOLDER_AS_DEFAULT', ] -destdir = get_option('libdir') / ('python' + py3_dep.version()) / 'site-packages' / 'libcamera' +if meson.is_cross_build() + # Meson doesn't correctly support cross-compilation through the python + # module. There's work in progress to fix this, based on PEP 739 + # (https://github.com/mesonbuild/meson/pull/14657). While waiting for this + # to become available, work around the issue by using shared_module(). + destdir = get_option('libdir') / ('python' + py3_dep.version()) / 'site-packages' / 'libcamera' -pycamera = shared_module('_libcamera', - pycamera_sources, - install : true, - install_dir : destdir, - install_tag : 'python-runtime', - name_prefix : '', - dependencies : pycamera_deps, - cpp_args : pycamera_args) + pycamera = shared_module('_libcamera', + pycamera_sources, + install : true, + install_dir : destdir, + install_tag : 'python-runtime', + name_prefix : '', + dependencies : pycamera_deps, + cpp_args : pycamera_args) + install_data(['__init__.py'], + install_dir : destdir, + install_tag : 'python-runtime') +else + pycamera = py3.extension_module('_libcamera', + pycamera_sources, + install : true, + subdir : 'libcamera', + dependencies : pycamera_deps, + cpp_args : pycamera_args) + py3.install_sources(['__init__.py'], + subdir : 'libcamera', + pure : false) +endif # Create symlinks from the build dir to the source dir so that we can use the # Python module directly from the build dir. @@ -79,10 +98,6 @@ run_command('ln', '-fsrT', meson.current_source_dir() / 'utils', meson.current_build_dir() / 'utils', check : true) -install_data(['__init__.py'], - install_dir : destdir, - install_tag : 'python-runtime') - # \todo Generate stubs when building. See https://peps.python.org/pep-0484/#stub-files # Note: Depends on pybind11-stubgen. To generate pylibcamera stubs: # $ PYTHONPATH=build/src/py pybind11-stubgen --no-setup-py -o build/src/py libcamera diff --git a/src/py/meson.build b/src/py/meson.build index 9228069711be..24102f83d2e1 100644 --- a/src/py/meson.build +++ b/src/py/meson.build @@ -1,6 +1,17 @@ # SPDX-License-Identifier: CC0-1.0 -py3_dep = dependency('python3', required : get_option('pycamera')) +if meson.is_cross_build() + py3_dep = dependency('python3', required : get_option('pycamera')) +else + py3 = import('python').find_installation('python3', required : get_option('pycamera')) + if not py3.found() + pycamera_enabled = false + subdir_done() + endif + + py3_dep = py3.dependency(required : get_option('pycamera')) +endif + pybind11_dep = dependency('pybind11', required : get_option('pycamera')) pycamera_enabled = py3_dep.found() and pybind11_dep.found() diff --git a/test/py/meson.build b/test/py/meson.build index b922e8578c29..d014aac4573e 100644 --- a/test/py/meson.build +++ b/test/py/meson.build @@ -15,9 +15,6 @@ endif py_env = environment() -pymod = import('python') -py3 = pymod.find_installation('python3') - pypathdir = meson.project_build_root() / 'src' / 'py' py_env.append('PYTHONPATH', pypathdir) From patchwork Tue Aug 19 01:24:00 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Laurent Pinchart X-Patchwork-Id: 24161 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 8F6F9BDCC1 for ; Tue, 19 Aug 2025 01:24:32 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id D7EFC6926B; Tue, 19 Aug 2025 03:24:31 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="mC/Lan3w"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 7E2CE69265 for ; Tue, 19 Aug 2025 03:24:27 +0200 (CEST) Received: from pendragon.ideasonboard.com (81-175-209-231.bb.dnainternet.fi [81.175.209.231]) by perceval.ideasonboard.com (Postfix) with UTF8SMTPSA id C3C39446; Tue, 19 Aug 2025 03:23:29 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1755566610; bh=FoV1eZQLOo1+3azC6yGFmYeK5O88hxR+Qgc+i0I/ySw=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=mC/Lan3wCbjNTedogO8wkvAwuO9C0oUxewGF6cdbWf6OdMROtlvoyf4WNC8eOxLaz zXFN8LvIPQw7JKS4XfXYcH8gbPxCX+JgQ04hxdiXx05cfmUxr7MDtthx3608HVUmZt PG3vFFtMHkzENsB/zfn2sgzgy3ZExMbuWeC6VK2Q= From: Laurent Pinchart To: libcamera-devel@lists.libcamera.org Cc: William Vinnicombe , Tomi Valkeinen Subject: [PATCH v3 2/3] py: libcamera: Get dependency from meson python module unconditionally Date: Tue, 19 Aug 2025 04:24:00 +0300 Message-ID: <20250819012402.8395-3-laurent.pinchart@ideasonboard.com> X-Mailer: git-send-email 2.49.1 In-Reply-To: <20250819012402.8395-1-laurent.pinchart@ideasonboard.com> References: <20250819012402.8395-1-laurent.pinchart@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" libcamera uses the meson python module to handle native compilation of Python extension modules. When cross-compiling, the module uses the build machine suffix instead of the host machine suffix in some enviroments (for instance naming the shared object file _libcamera.cpython-313-x86_64-linux-gnu.so instead of _libcamera.cpython-313-aarch64-linux-gnu.so when cross-compiling from x86_64 to aarch64). This prevents using the python module in that case, and libcamera uses the normal dependency() function to locate the Python libraries, and the shared_module() function to build the module. Not using the meson python module to get the Python dependency prevents selecting a specific Python interpreter, the same way as it does for native builds. While having multiple Python interpreter versions in a cross-build environment is likely less common, different behaviours and features between native and cross-compilation are still not optimal. Improve this situation by getting the dependency from the python module for cross-compilation as well. This also prepares for usage of py.extension_module() once the file suffix issue will be fixed in meson. The user will need to ensure that the Python interpreter for the build machine matches the version of the interpreter in the cross-compilation environment for the host machine. Otherwise, meson will fail to find the Python dependency. Cross-compilation environment provided by Linux distributions (such as Debian multi-arch support) should work out of the box, but compiling libcamera manually against a cross-compilation environment provided by Buildroot or Yocto may require manual configuration. When the interpreters versions do not match, meson needs to be pointed to the build ùachine interpreter from the cross-compilation environment using the cross file. For instance, assuming a 'br_host_dir' variable pointing to the host directory from Buildroot, the cross file should contain [binaries] python = br_host_dir / 'bin/python3' Signed-off-by: Laurent Pinchart --- src/py/libcamera/meson.build | 4 ++-- src/py/meson.build | 16 +++------------- 2 files changed, 5 insertions(+), 15 deletions(-) diff --git a/src/py/libcamera/meson.build b/src/py/libcamera/meson.build index fe22ffd2f70b..d5433fbc1e3a 100644 --- a/src/py/libcamera/meson.build +++ b/src/py/libcamera/meson.build @@ -47,7 +47,7 @@ pycamera_sources += custom_target('py_gen_formats', pycamera_deps = [ libcamera_private, - py3_dep, + py3.dependency(), pybind11_dep, ] @@ -62,7 +62,7 @@ if meson.is_cross_build() # module. There's work in progress to fix this, based on PEP 739 # (https://github.com/mesonbuild/meson/pull/14657). While waiting for this # to become available, work around the issue by using shared_module(). - destdir = get_option('libdir') / ('python' + py3_dep.version()) / 'site-packages' / 'libcamera' + destdir = get_option('libdir') / ('python' + py3.language_version()) / 'site-packages' / 'libcamera' pycamera = shared_module('_libcamera', pycamera_sources, diff --git a/src/py/meson.build b/src/py/meson.build index 24102f83d2e1..88ad8b7ec26e 100644 --- a/src/py/meson.build +++ b/src/py/meson.build @@ -1,20 +1,10 @@ # SPDX-License-Identifier: CC0-1.0 -if meson.is_cross_build() - py3_dep = dependency('python3', required : get_option('pycamera')) -else - py3 = import('python').find_installation('python3', required : get_option('pycamera')) - if not py3.found() - pycamera_enabled = false - subdir_done() - endif - - py3_dep = py3.dependency(required : get_option('pycamera')) -endif - +py_mod = import('python') +py3 = py_mod.find_installation('python3', required : get_option('pycamera')) pybind11_dep = dependency('pybind11', required : get_option('pycamera')) -pycamera_enabled = py3_dep.found() and pybind11_dep.found() +pycamera_enabled = py3.found() and pybind11_dep.found() if not pycamera_enabled subdir_done() endif From patchwork Tue Aug 19 01:24:01 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Laurent Pinchart X-Patchwork-Id: 24162 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 0022DBDCC1 for ; Tue, 19 Aug 2025 01:24:35 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 8DBDE6926B; Tue, 19 Aug 2025 03:24:35 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="Spy0n/ut"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id CE38D69259 for ; Tue, 19 Aug 2025 03:24:28 +0200 (CEST) Received: from pendragon.ideasonboard.com (81-175-209-231.bb.dnainternet.fi [81.175.209.231]) by perceval.ideasonboard.com (Postfix) with UTF8SMTPSA id 364D9446; Tue, 19 Aug 2025 03:23:31 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1755566611; bh=mM1MlgwoX48G/vAdbqdW17U+FLxSBc8pb2/LFb09Ua0=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=Spy0n/utVQrlG5kN8MsWfTs33RCbdgw38hlYEGTZI1d5r7kdh/r450CMYb04PrnVG F+ePec2649iu4OsjqSNy9TLZBwDBKyQOZmabLBkQVwb7Pe9wk9mp5Tg5+IQVSauq1r 8YZNsE9uI7XmcymKqNM2bwgHJ3d/rUZKwPM0JvB0= From: Laurent Pinchart To: libcamera-devel@lists.libcamera.org Cc: William Vinnicombe , Tomi Valkeinen Subject: [PATCH v3 3/3] py: libcamera: Always use install path from meson python module Date: Tue, 19 Aug 2025 04:24:01 +0300 Message-ID: <20250819012402.8395-4-laurent.pinchart@ideasonboard.com> X-Mailer: git-send-email 2.49.1 In-Reply-To: <20250819012402.8395-1-laurent.pinchart@ideasonboard.com> References: <20250819012402.8395-1-laurent.pinchart@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" libcamera uses the meson python module to handle native compilation of Python extension modules, and uses the shared_module() function when cross-compiling due to an issue in the python module. The difference between native and cross compilation also extends to the installation path: native compilation lets the python module handle the paths, while cross compilation constructs a path manually using a heuristic based on the Python version and hardcoded components. This manually-constructed installation path is problematic for cross compilation for the same reason it caused issue when used for native compilation: it is not guaranteed to be right, and it can't be overridden by users. Switch to obtaining the installation path from the meson python module for cross-compilation as well. This also prepares for usage of py.extension_module() once the file suffix issue will be fixed in meson. On Debian 13, this change replaces the incorrect path /usr/local/lib/python3.12/site-packages/libcamera with the still (but differently) incorrect /usr/local/lib/python3/dist-packages/libcamera. Future fixes in meson to address this issue will make the path correct by default. When the path calculated by the python module is not correct, it can now be overridden by the user through the meson python.platlibdir configuration variable. Signed-off-by: Laurent Pinchart --- src/py/libcamera/meson.build | 12 +++++------- 1 file changed, 5 insertions(+), 7 deletions(-) diff --git a/src/py/libcamera/meson.build b/src/py/libcamera/meson.build index d5433fbc1e3a..b0bd291fa9d6 100644 --- a/src/py/libcamera/meson.build +++ b/src/py/libcamera/meson.build @@ -62,7 +62,7 @@ if meson.is_cross_build() # module. There's work in progress to fix this, based on PEP 739 # (https://github.com/mesonbuild/meson/pull/14657). While waiting for this # to become available, work around the issue by using shared_module(). - destdir = get_option('libdir') / ('python' + py3.language_version()) / 'site-packages' / 'libcamera' + destdir = py3.get_install_dir(subdir : 'libcamera', pure : false) pycamera = shared_module('_libcamera', pycamera_sources, @@ -72,9 +72,6 @@ if meson.is_cross_build() name_prefix : '', dependencies : pycamera_deps, cpp_args : pycamera_args) - install_data(['__init__.py'], - install_dir : destdir, - install_tag : 'python-runtime') else pycamera = py3.extension_module('_libcamera', pycamera_sources, @@ -82,11 +79,12 @@ else subdir : 'libcamera', dependencies : pycamera_deps, cpp_args : pycamera_args) - py3.install_sources(['__init__.py'], - subdir : 'libcamera', - pure : false) endif +py3.install_sources(['__init__.py'], + subdir : 'libcamera', + pure : false) + # Create symlinks from the build dir to the source dir so that we can use the # Python module directly from the build dir.