[{"id":35203,"web_url":"https://patchwork.libcamera.org/comment/35203/","msgid":"<129a070e-850c-4303-8dda-bd138f841934@ideasonboard.com>","date":"2025-07-28T14:18:02","subject":"Re: [libcamera-ci] [PATCH 2/3] setup-container: Generalize package\n\tinstallation through pip","submitter":{"id":216,"url":"https://patchwork.libcamera.org/api/people/216/","name":"Barnabás Pőcze","email":"barnabas.pocze@ideasonboard.com"},"content":"Hi\n\n2025. 07. 27. 3:11 keltezéssel, Laurent Pinchart írta:\n> When building containers, we favour packages from the Debian\n> repositories, and fall back to pip when minimum version requirements are\n> not met by Debian. This is open-coded for two packages (meson and\n> python3-pygments). Generalize the mechanism in preparation for adding\n> more packages.\n> \n> Signed-off-by: Laurent Pinchart \n> ---\n> .gitlab-ci/setup-container.sh | 43 +++++++++++++++++++----------------\n> 1 file changed, 24 insertions(+), 19 deletions(-)\n> \n> diff --git a/.gitlab-ci/setup-container.sh b/.gitlab-ci/setup-container.sh\n> index 5e9b89da8431..c8b631c387b5 100755\n> --- a/.gitlab-ci/setup-container.sh\n> +++ b/.gitlab-ci/setup-container.sh\n> @@ -93,6 +93,11 @@ PKGS_VIRTME_RUNTIME=(\n> \tudev\n> )\n> \n> +PKGS_PIP_MIN_VERSIONS=(\n> +\t'meson meson 1.2.0 remove'\n> +\t'python3-pygments pygments 2.10.0 keep'\n> +)\n> +\n> archs=( amd64 )\n> \n> declare -A components\n> @@ -192,34 +197,34 @@ dpkg_check_version() {\n> cbuild_fixups() {\n> \techo \"Applying miscellaneous fixups\"\n> \n> -\tlocal min_version\n> \tlocal pip3_options\n> -\tlocal version\n> +\tlocal pkg\n> \n> \tif [[ $FDO_DISTRIBUTION_VERSION != 'bullseye' ]] ; then\n> \t\tpip3_options=--break-system-packages\n> \tfi\n> \n> -\t# Install meson from pip.\n> -\tmin_version=1.2.0\n> -\tversion=$(dpkg_version meson)\n> -\tif dpkg_check_version $version $min_version ; then\n> -\t\techo \"meson $version too old, installing $min_version from pip\"\n> +\t# Install packages from pip to ensure minimum versions.\n> +\tfor pkg in \"${PKGS_PIP_MIN_VERSIONS[@]}\" ; do\n> +\t\tpkg=($pkg)\n\nAnd people say C++ is unreadable...\n\n\n> +\t\tlocal pkg_name=${pkg[0]}\n> +\t\tlocal pip_name=${pkg[1]}\n> +\t\tlocal min_version=${pkg[2]}\n> +\t\tlocal action=${pkg[3]}\n> \n> -\t\tapt remove -y meson\n> -\t\tapt install -y python3-pip\n> -\t\tpip3 install ${pip3_options} meson==${min_version}\n> -\tfi\n> +\t\tlocal version=$(dpkg_version $pkg_name)\n> \n> -\t# Install pygments from pip.\n> -\tmin_version=2.10.0\n> -\tversion=$(dpkg_version python3-pygments)\n> -\tif dpkg_check_version $version $min_version ; then\n> -\t\techo \"pygments $version too old, installing $min_version from pip\"\n> +\t\tif dpkg_check_version $version $min_version ; then\n> +\t\t\techo \"$pkg_name $version too old, installing $min_version from pip\"\n> \n> -\t\tapt install -y python3-pip\n> -\t\tpip3 install ${pip3_options} pygments==${min_version}\n> -\tfi\n> +\t\t\tif [[ $action == 'remove' ]] ; then\n\nI haven't found out why some packages need to be kept/removed, and it's not\nclear to me at all. But it seems to do what I expect the code to do:\nhttps://gitlab.freedesktop.org/pobrn/libcamera/-/jobs/81380023\n\nReviewed-by: Barnabás Pőcze \n\n> +\t\t\t\tapt remove -y ${pkg_name}\n> +\t\t\tfi\n> +\n> +\t\t\tapt install -y python3-pip\n> +\t\t\tpip3 install ${pip3_options} ${pip_name}==${min_version}\n> +\t\tfi\n> +\tdone\n> \n> \t# Create pybind11.pc manually if not provided by the distribution\n> \t# package.","headers":{"Return-Path":"","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id 5F91DBDCC1\n\tfor ;\n\tMon, 28 Jul 2025 14:18:08 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id E8CA569189;\n\tMon, 28 Jul 2025 16:18:07 +0200 (CEST)","from perceval.ideasonboard.com (perceval.ideasonboard.com\n\t[IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id C5D3A69158\n\tfor ;\n\tMon, 28 Jul 2025 16:18:04 +0200 (CEST)","from [192.168.33.18] (185.221.140.39.nat.pool.zt.hu\n\t[185.221.140.39])\n\tby perceval.ideasonboard.com (Postfix) with ESMTPSA id D19E0446;\n\tMon, 28 Jul 2025 16:17:22 +0200 (CEST)"],"Authentication-Results":"lancelot.ideasonboard.com; dkim=pass (1024-bit key;\n\tunprotected) header.d=ideasonboard.com header.i=@ideasonboard.com\n\theader.b=\"Na5V/4hm\"; dkim-atps=neutral","DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com;\n\ts=mail; t=1753712242;\n\tbh=0JWEwlBBXiZpwjXYxvgpFKuk7Ti1ec9PMhteWQa3z/w=;\n\th=Date:Subject:To:References:From:In-Reply-To:From;\n\tb=Na5V/4hmsf2gLKwDrTWwD+G53R4nt5ZFZAJtPBwysIIbwxTsKjpl9uY5fH3bTaGxH\n\tirz89OXs4gS8ZvGDNS/QqgvB6q4p535WZBiZ3pdRPdEXMi+z3xW7L8XD0uI1HsAKsk\n\tSu4ipdbD0UYQ2KDa0jz6pQuMw6cwbAl8QE0p5BWk=","Message-ID":"<129a070e-850c-4303-8dda-bd138f841934@ideasonboard.com>","Date":"Mon, 28 Jul 2025 16:18:02 +0200","MIME-Version":"1.0","User-Agent":"Mozilla Thunderbird","Subject":"Re: [libcamera-ci] [PATCH 2/3] setup-container: Generalize package\n\tinstallation through pip","To":"Laurent Pinchart ,\n\tlibcamera-devel@lists.libcamera.org","References":"<20250727011123.31634-1-laurent.pinchart@ideasonboard.com>\n\t<20250727011123.31634-3-laurent.pinchart@ideasonboard.com>","From":"=?utf-8?q?Barnab=C3=A1s_P=C5=91cze?= ","Content-Language":"en-US, hu-HU","In-Reply-To":"<20250727011123.31634-3-laurent.pinchart@ideasonboard.com>","Content-Type":"text/plain; charset=UTF-8; format=flowed","Content-Transfer-Encoding":"8bit","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" "}},{"id":35210,"web_url":"https://patchwork.libcamera.org/comment/35210/","msgid":"<20250728151459.GM787@pendragon.ideasonboard.com>","date":"2025-07-28T15:14:59","subject":"Re: [libcamera-ci] [PATCH 2/3] setup-container: Generalize package\n\tinstallation through pip","submitter":{"id":2,"url":"https://patchwork.libcamera.org/api/people/2/","name":"Laurent Pinchart","email":"laurent.pinchart@ideasonboard.com"},"content":"On Mon, Jul 28, 2025 at 04:18:02PM +0200, Barnabás Pőcze wrote:\n> 2025. 07. 27. 3:11 keltezéssel, Laurent Pinchart írta:\n> > When building containers, we favour packages from the Debian\n> > repositories, and fall back to pip when minimum version requirements are\n> > not met by Debian. This is open-coded for two packages (meson and\n> > python3-pygments). Generalize the mechanism in preparation for adding\n> > more packages.\n> > \n> > Signed-off-by: Laurent Pinchart \n> > ---\n> > .gitlab-ci/setup-container.sh | 43 +++++++++++++++++++----------------\n> > 1 file changed, 24 insertions(+), 19 deletions(-)\n> > \n> > diff --git a/.gitlab-ci/setup-container.sh b/.gitlab-ci/setup-container.sh\n> > index 5e9b89da8431..c8b631c387b5 100755\n> > --- a/.gitlab-ci/setup-container.sh\n> > +++ b/.gitlab-ci/setup-container.sh\n> > @@ -93,6 +93,11 @@ PKGS_VIRTME_RUNTIME=(\n> > \tudev\n> > )\n> > \n> > +PKGS_PIP_MIN_VERSIONS=(\n> > +\t'meson meson 1.2.0 remove'\n> > +\t'python3-pygments pygments 2.10.0 keep'\n> > +)\n> > +\n> > archs=( amd64 )\n> > \n> > declare -A components\n> > @@ -192,34 +197,34 @@ dpkg_check_version() {\n> > cbuild_fixups() {\n> > \techo \"Applying miscellaneous fixups\"\n> > \n> > -\tlocal min_version\n> > \tlocal pip3_options\n> > -\tlocal version\n> > +\tlocal pkg\n> > \n> > \tif [[ $FDO_DISTRIBUTION_VERSION != 'bullseye' ]] ; then\n> > \t\tpip3_options=--break-system-packages\n> > \tfi\n> > \n> > -\t# Install meson from pip.\n> > -\tmin_version=1.2.0\n> > -\tversion=$(dpkg_version meson)\n> > -\tif dpkg_check_version $version $min_version ; then\n> > -\t\techo \"meson $version too old, installing $min_version from pip\"\n> > +\t# Install packages from pip to ensure minimum versions.\n> > +\tfor pkg in \"${PKGS_PIP_MIN_VERSIONS[@]}\" ; do\n> > +\t\tpkg=($pkg)\n> \n> And people say C++ is unreadable...\n\n:-D\n\n> > +\t\tlocal pkg_name=${pkg[0]}\n> > +\t\tlocal pip_name=${pkg[1]}\n> > +\t\tlocal min_version=${pkg[2]}\n> > +\t\tlocal action=${pkg[3]}\n> > \n> > -\t\tapt remove -y meson\n> > -\t\tapt install -y python3-pip\n> > -\t\tpip3 install ${pip3_options} meson==${min_version}\n> > -\tfi\n> > +\t\tlocal version=$(dpkg_version $pkg_name)\n> > \n> > -\t# Install pygments from pip.\n> > -\tmin_version=2.10.0\n> > -\tversion=$(dpkg_version python3-pygments)\n> > -\tif dpkg_check_version $version $min_version ; then\n> > -\t\techo \"pygments $version too old, installing $min_version from pip\"\n> > +\t\tif dpkg_check_version $version $min_version ; then\n> > +\t\t\techo \"$pkg_name $version too old, installing $min_version from pip\"\n> > \n> > -\t\tapt install -y python3-pip\n> > -\t\tpip3 install ${pip3_options} pygments==${min_version}\n> > -\tfi\n> > +\t\t\tif [[ $action == 'remove' ]] ; then\n> \n> I haven't found out why some packages need to be kept/removed, and it's not\n> clear to me at all. But it seems to do what I expect the code to do:\n> https://gitlab.freedesktop.org/pobrn/libcamera/-/jobs/81380023\n\nIf python3-pygments is uninstalled, python3-sphinx will be uninstalled\nas well as it depends on it. I'll add a comment to the code and send a\nv2.\n\n> Reviewed-by: Barnabás Pőcze \n> \n> > +\t\t\t\tapt remove -y ${pkg_name}\n> > +\t\t\tfi\n> > +\n> > +\t\t\tapt install -y python3-pip\n> > +\t\t\tpip3 install ${pip3_options} ${pip_name}==${min_version}\n> > +\t\tfi\n> > +\tdone\n> > \n> > \t# Create pybind11.pc manually if not provided by the distribution\n> > \t# package.","headers":{"Return-Path":"","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id 94386BDCC1\n\tfor ;\n\tMon, 28 Jul 2025 15:15:09 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id 6EC0A6918F;\n\tMon, 28 Jul 2025 17:15:08 +0200 (CEST)","from perceval.ideasonboard.com (perceval.ideasonboard.com\n\t[213.167.242.64])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id 025E869158\n\tfor ;\n\tMon, 28 Jul 2025 17:15:06 +0200 (CEST)","from pendragon.ideasonboard.com (81-175-209-231.bb.dnainternet.fi\n\t[81.175.209.231])\n\tby perceval.ideasonboard.com (Postfix) with UTF8SMTPSA id ADF363D5;\n\tMon, 28 Jul 2025 17:14:24 +0200 (CEST)"],"Authentication-Results":"lancelot.ideasonboard.com; dkim=pass (1024-bit key;\n\tunprotected) header.d=ideasonboard.com header.i=@ideasonboard.com\n\theader.b=\"nGwcSrtd\"; dkim-atps=neutral","DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com;\n\ts=mail; t=1753715664;\n\tbh=CVhqjY/Be0hC8phWnhMco5auUfmLfFxBITvDttgExvg=;\n\th=Date:From:To:Cc:Subject:References:In-Reply-To:From;\n\tb=nGwcSrtdHDi6dCyenaWcdTx+UxpachJJaS6l+hd0cKw2rWHFOGbzrqyHXHaiBLfqi\n\tm2QqE343t8FSQlKQ7FQPhfxzP0Z+XqdSDQDAxD14f8bfAC/xRFhhIXb33tvIsIiM0Q\n\tosW+mXEbqOIlVy/4QqwIWKLCvmGVRPd8A0O4z2OE=","Date":"Mon, 28 Jul 2025 18:14:59 +0300","From":"Laurent Pinchart ","To":"=?utf-8?q?Barnab=C3=A1s_P=C5=91cze?= ","Cc":"libcamera-devel@lists.libcamera.org","Subject":"Re: [libcamera-ci] [PATCH 2/3] setup-container: Generalize package\n\tinstallation through pip","Message-ID":"<20250728151459.GM787@pendragon.ideasonboard.com>","References":"<20250727011123.31634-1-laurent.pinchart@ideasonboard.com>\n\t<20250727011123.31634-3-laurent.pinchart@ideasonboard.com>\n\t<129a070e-850c-4303-8dda-bd138f841934@ideasonboard.com>","MIME-Version":"1.0","Content-Type":"text/plain; charset=utf-8","Content-Disposition":"inline","Content-Transfer-Encoding":"8bit","In-Reply-To":"<129a070e-850c-4303-8dda-bd138f841934@ideasonboard.com>","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" "}}]