[{"id":32733,"web_url":"https://patchwork.libcamera.org/comment/32733/","msgid":"<20241215190408.GL9975@pendragon.ideasonboard.com>","date":"2024-12-15T19:04:08","subject":"Re: [libcamera-ci] [RFC PATCH v1 2/3] Separate the building and\n\trunning of unit tests","submitter":{"id":2,"url":"https://patchwork.libcamera.org/api/people/2/","name":"Laurent Pinchart","email":"laurent.pinchart@ideasonboard.com"},"content":"Hi Barnabás,\n\nThank you for the patch.\n\nOn Thu, Dec 12, 2024 at 07:16:54PM +0100, Barnabás Pőcze wrote:\n> The built artifacts will be reused in a later job, so split\n> the \"test-unit\" into the \"build-test\" and \"test-unit\" jobs.\n> \n> The `libevent` development package cannot be installed in the container\n\nI've write `libevent-dev` here to avoid ambiguities.\n\n> directly because it is not multiarch compatible. It is, however, installed\n> in the architecture specific build jobs, right before building. To ensure\n> that the it is available for already built executables in different jobs,\n\n\"that the it is\" ?\n\n> install just the libraries in the container.\n\nAnd name here `libevent`.\n\n> \n> Signed-off-by: Barnabás Pőcze \n> ---\n> .gitlab-ci/setup-container.sh | 3 +++\n> gitlab-ci.yml | 42 +++++++++++++++++++++++------------\n> 2 files changed, 31 insertions(+), 14 deletions(-)\n> \n> diff --git a/.gitlab-ci/setup-container.sh b/.gitlab-ci/setup-container.sh\n> index d2909c7..0658368 100755\n> --- a/.gitlab-ci/setup-container.sh\n> +++ b/.gitlab-ci/setup-container.sh\n> @@ -103,6 +103,9 @@ case $FDO_DISTRIBUTION_VERSION in\n> 'bookworm')\n> \t# libclang-rt-dev for the clang ASan runtime.\n> \tPKGS_LIBCAMERA_RUNTIME_MULTIARCH+=( libclang-rt-dev )\n> +\t# For cam and lc-compliance\n> +\t# libevent-dev cannot be used here, see build-libcamera-common.sh\n> +\tPKGS_LIBCAMERA_RUNTIME_MULTIARCH+=( libevent-2.1-7 libevent-pthreads-2.1-7 )\n> \t;;\n> 'trixie')\n> \t# gcc 13 to expand compilation testing coverage.\n> diff --git a/gitlab-ci.yml b/gitlab-ci.yml\n> index 8bc8bc2..c7448b8 100644\n> --- a/gitlab-ci.yml\n> +++ b/gitlab-ci.yml\n> @@ -64,7 +64,7 @@ include:\n> .libcamera-ci.debian:12:\n> variables:\n> FDO_DISTRIBUTION_VERSION: 'bookworm'\n> - FDO_DISTRIBUTION_TAG: '2024-12-12.1'\n> + FDO_DISTRIBUTION_TAG: '2024-12-12.2'\n> \n> .libcamera-ci.debian:13:\n> variables:\n> @@ -363,28 +363,18 @@ test-soraka:\n> script:\n> - submit .gitlab-ci/lava/soraka-camera-test.yml\n> \n> -# Run the unit tests in a virtual machine. Enable only the options exercised by\n> -# the unit tests.\n> -test-unit:\n> +# Enable only the options exercised by the unit tests.\n> +build-test:debug:\n\nI'd call this build-package:amd64, as we have build-package:arm64 and\nbuild-package:cros. I think it would also make sense to use the same\nbuild options for the amd64 and arm64 packages (beside possibly the\nselected pipeline handlers, although the 'auto' option may work for\nboth).\n\n> extends:\n> - .fdo.distribution-image@debian\n> - .libcamera-ci.debian:12\n> - .libcamera-ci.scripts\n> - stage: test\n> + stage: build\n> needs:\n> - job: container-debian:12\n> artifacts: false\n> - tags:\n> - - kvm\n> script:\n> - $CI_PROJECT_DIR/.gitlab-ci/build-libcamera.sh\n> - - $CI_PROJECT_DIR/.gitlab-ci/test-libcamera-qemu.sh\n> - artifacts:\n> - name: libcamera-unit-tests-${CI_COMMIT_SHA}\n> - when: always\n> - expire_in: 1 week\n> - paths:\n> - - build/meson-logs/\n> variables:\n> BUILD_TYPE: debug\n> MESON_OPTIONS: >-\n> @@ -399,6 +389,30 @@ test-unit:\n> -D qcam=disabled\n> -D test=true\n> -D v4l2=true\n> + artifacts:\n> + paths:\n> + - build/\n\nThe whole build directory can be very large. Can't we do the same as\nbuild-package:arm64 and run package-libcamera.sh to only package what we\nneed ? We'll need probably need an unpackage script for the test-unit\njob.\n\n> + expire_in: 1 day\n> +\n> +# Run the unit tests in a virtual machine.\n> +test-unit:\n> + extends:\n> + - .fdo.distribution-image@debian\n> + - .libcamera-ci.debian:12\n> + - .libcamera-ci.scripts\n> + stage: test\n> + needs:\n> + - job: build-test:debug\n> + tags:\n> + - kvm\n> + script:\n> + - $CI_PROJECT_DIR/.gitlab-ci/test-libcamera-qemu.sh\n> + artifacts:\n> + name: libcamera-unit-tests-${CI_COMMIT_SHA}\n> + when: always\n> + expire_in: 1 week\n> + paths:\n> + - build/meson-logs/\n> \n> # meson prior to 1.2.0 doesn't correctly escape non-printable characters\n> # when generating the testlog XML. This results in an unparseable file.","headers":{"Return-Path":"","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id 3A3FEC32F5\n\tfor ;\n\tSun, 15 Dec 2024 19:04:28 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id 6283967F0D;\n\tSun, 15 Dec 2024 20:04:27 +0100 (CET)","from perceval.ideasonboard.com (perceval.ideasonboard.com\n\t[213.167.242.64])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id E712267EEE\n\tfor ;\n\tSun, 15 Dec 2024 20:04:25 +0100 (CET)","from pendragon.ideasonboard.com (81-175-209-231.bb.dnainternet.fi\n\t[81.175.209.231])\n\tby perceval.ideasonboard.com (Postfix) with ESMTPSA id A841399F;\n\tSun, 15 Dec 2024 20:03:49 +0100 (CET)"],"Authentication-Results":"lancelot.ideasonboard.com; dkim=pass (1024-bit key;\n\tunprotected) header.d=ideasonboard.com header.i=@ideasonboard.com\n\theader.b=\"cMaQ4K+E\"; dkim-atps=neutral","DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com;\n\ts=mail; t=1734289429;\n\tbh=eZujpTgV9Ivwe0lR2JOqcTM8XdXvBZ7Y/FcaUKHhabs=;\n\th=Date:From:To:Cc:Subject:References:In-Reply-To:From;\n\tb=cMaQ4K+Eq9dIHydNq3OY0t7lQXldGVQ/20KIWWm1DPUtZ+qLBbwtT6ewIUQTb4SpQ\n\tHRxcYTesjzmmau+YKno8s+1g2iq6ggtz/l+ZtRF/egk51Ra4AUcQLNMKNTg97/5OuK\n\tDeU43Gf1hCEtK/dCqW+UK6UEsfa8v0PWyq2qN9sA=","Date":"Sun, 15 Dec 2024 21:04:08 +0200","From":"Laurent Pinchart ","To":"=?utf-8?q?Barnab=C3=A1s_P=C5=91cze?= ","Cc":"libcamera-devel@lists.libcamera.org","Subject":"Re: [libcamera-ci] [RFC PATCH v1 2/3] Separate the building and\n\trunning of unit tests","Message-ID":"<20241215190408.GL9975@pendragon.ideasonboard.com>","References":"<20241212181655.112958-1-barnabas.pocze@ideasonboard.com>\n\t<20241212181655.112958-2-barnabas.pocze@ideasonboard.com>","MIME-Version":"1.0","Content-Type":"text/plain; charset=utf-8","Content-Disposition":"inline","Content-Transfer-Encoding":"8bit","In-Reply-To":"<20241212181655.112958-2-barnabas.pocze@ideasonboard.com>","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" "}},{"id":32735,"web_url":"https://patchwork.libcamera.org/comment/32735/","msgid":"<20241215194320.GN9975@pendragon.ideasonboard.com>","date":"2024-12-15T19:43:20","subject":"Re: [libcamera-ci] [RFC PATCH v1 2/3] Separate the building and\n\trunning of unit tests","submitter":{"id":2,"url":"https://patchwork.libcamera.org/api/people/2/","name":"Laurent Pinchart","email":"laurent.pinchart@ideasonboard.com"},"content":"On Sun, Dec 15, 2024 at 09:04:08PM +0200, Laurent Pinchart wrote:\n> Hi Barnabás,\n> \n> Thank you for the patch.\n> \n> On Thu, Dec 12, 2024 at 07:16:54PM +0100, Barnabás Pőcze wrote:\n> > The built artifacts will be reused in a later job, so split\n> > the \"test-unit\" into the \"build-test\" and \"test-unit\" jobs.\n> > \n> > The `libevent` development package cannot be installed in the container\n> \n> I've write `libevent-dev` here to avoid ambiguities.\n> \n> > directly because it is not multiarch compatible. It is, however, installed\n> > in the architecture specific build jobs, right before building. To ensure\n> > that the it is available for already built executables in different jobs,\n> \n> \"that the it is\" ?\n> \n> > install just the libraries in the container.\n> \n> And name here `libevent`.\n> \n> > \n> > Signed-off-by: Barnabás Pőcze \n> > ---\n> > .gitlab-ci/setup-container.sh | 3 +++\n> > gitlab-ci.yml | 42 +++++++++++++++++++++++------------\n> > 2 files changed, 31 insertions(+), 14 deletions(-)\n> > \n> > diff --git a/.gitlab-ci/setup-container.sh b/.gitlab-ci/setup-container.sh\n> > index d2909c7..0658368 100755\n> > --- a/.gitlab-ci/setup-container.sh\n> > +++ b/.gitlab-ci/setup-container.sh\n> > @@ -103,6 +103,9 @@ case $FDO_DISTRIBUTION_VERSION in\n> > 'bookworm')\n> > \t# libclang-rt-dev for the clang ASan runtime.\n> > \tPKGS_LIBCAMERA_RUNTIME_MULTIARCH+=( libclang-rt-dev )\n> > +\t# For cam and lc-compliance\n> > +\t# libevent-dev cannot be used here, see build-libcamera-common.sh\n> > +\tPKGS_LIBCAMERA_RUNTIME_MULTIARCH+=( libevent-2.1-7 libevent-pthreads-2.1-7 )\n> > \t;;\n> > 'trixie')\n> > \t# gcc 13 to expand compilation testing coverage.\n> > diff --git a/gitlab-ci.yml b/gitlab-ci.yml\n> > index 8bc8bc2..c7448b8 100644\n> > --- a/gitlab-ci.yml\n> > +++ b/gitlab-ci.yml\n> > @@ -64,7 +64,7 @@ include:\n> > .libcamera-ci.debian:12:\n> > variables:\n> > FDO_DISTRIBUTION_VERSION: 'bookworm'\n> > - FDO_DISTRIBUTION_TAG: '2024-12-12.1'\n> > + FDO_DISTRIBUTION_TAG: '2024-12-12.2'\n> > \n> > .libcamera-ci.debian:13:\n> > variables:\n> > @@ -363,28 +363,18 @@ test-soraka:\n> > script:\n> > - submit .gitlab-ci/lava/soraka-camera-test.yml\n> > \n> > -# Run the unit tests in a virtual machine. Enable only the options exercised by\n> > -# the unit tests.\n> > -test-unit:\n> > +# Enable only the options exercised by the unit tests.\n> > +build-test:debug:\n> \n> I'd call this build-package:amd64, as we have build-package:arm64 and\n> build-package:cros. I think it would also make sense to use the same\n> build options for the amd64 and arm64 packages (beside possibly the\n> selected pipeline handlers, although the 'auto' option may work for\n> both).\n> \n> > extends:\n> > - .fdo.distribution-image@debian\n> > - .libcamera-ci.debian:12\n> > - .libcamera-ci.scripts\n> > - stage: test\n> > + stage: build\n> > needs:\n> > - job: container-debian:12\n> > artifacts: false\n> > - tags:\n> > - - kvm\n> > script:\n> > - $CI_PROJECT_DIR/.gitlab-ci/build-libcamera.sh\n> > - - $CI_PROJECT_DIR/.gitlab-ci/test-libcamera-qemu.sh\n> > - artifacts:\n> > - name: libcamera-unit-tests-${CI_COMMIT_SHA}\n> > - when: always\n> > - expire_in: 1 week\n> > - paths:\n> > - - build/meson-logs/\n> > variables:\n> > BUILD_TYPE: debug\n> > MESON_OPTIONS: >-\n> > @@ -399,6 +389,30 @@ test-unit:\n> > -D qcam=disabled\n> > -D test=true\n> > -D v4l2=true\n> > + artifacts:\n> > + paths:\n> > + - build/\n> \n> The whole build directory can be very large. Can't we do the same as\n> build-package:arm64 and run package-libcamera.sh to only package what we\n> need ? We'll need probably need an unpackage script for the test-unit\n> job.\n\nBut of course the unit test binaries don't get installed... Can we fix\nthat and install them ? You can specify \"install_tag : 'tests'\" in\nmeson.build so they won't be installed by default (an appropriate\ninstall_dir is also needed). This in turn requires bumping the minimum\nmeson version from 0.63.0 to 0.64.0, which shouldn't be an issue.\n\nAnd now that I've said this, I realize we wouldn't be able to run \"meson\ntest\" to run the tests :-/ I'm not sure there's an appropriate solution\nfor this. If not, given the size of the build directory, and to avoid\ntransferring a large amount of data between runners, we may need to keep\nbuilding libcamera within the test-unit job :-(\n\nA separate build-package target for lc-compliance would still make\nsense.\n\n> > + expire_in: 1 day\n> > +\n> > +# Run the unit tests in a virtual machine.\n> > +test-unit:\n> > + extends:\n> > + - .fdo.distribution-image@debian\n> > + - .libcamera-ci.debian:12\n> > + - .libcamera-ci.scripts\n> > + stage: test\n> > + needs:\n> > + - job: build-test:debug\n> > + tags:\n> > + - kvm\n> > + script:\n> > + - $CI_PROJECT_DIR/.gitlab-ci/test-libcamera-qemu.sh\n> > + artifacts:\n> > + name: libcamera-unit-tests-${CI_COMMIT_SHA}\n> > + when: always\n> > + expire_in: 1 week\n> > + paths:\n> > + - build/meson-logs/\n> > \n> > # meson prior to 1.2.0 doesn't correctly escape non-printable characters\n> > # when generating the testlog XML. This results in an unparseable file.","headers":{"Return-Path":"","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id 2C8D2C32F6\n\tfor ;\n\tSun, 15 Dec 2024 19:43:39 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id 84F9A67F10;\n\tSun, 15 Dec 2024 20:43:38 +0100 (CET)","from perceval.ideasonboard.com (perceval.ideasonboard.com\n\t[IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id 40C7667EEE\n\tfor ;\n\tSun, 15 Dec 2024 20:43:37 +0100 (CET)","from pendragon.ideasonboard.com (81-175-209-231.bb.dnainternet.fi\n\t[81.175.209.231])\n\tby perceval.ideasonboard.com (Postfix) with ESMTPSA id 0469C99F;\n\tSun, 15 Dec 2024 20:43:00 +0100 (CET)"],"Authentication-Results":"lancelot.ideasonboard.com; dkim=pass (1024-bit key;\n\tunprotected) header.d=ideasonboard.com header.i=@ideasonboard.com\n\theader.b=\"XcAec1on\"; dkim-atps=neutral","DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com;\n\ts=mail; t=1734291781;\n\tbh=Z1gRchtKR154Zc/n1CcngMMivyXiNKCGLsVR+981PV4=;\n\th=Date:From:To:Cc:Subject:References:In-Reply-To:From;\n\tb=XcAec1onsTs/agm3VTyKqMDuLxkjJIrRxKKZjKbHovXhJVI5tl+W+x9ZZMpuvvCii\n\tKtrikbezG5K1mM+ktJDEAJTeMgTmDVbnEv0w3k4sZb4NGUH7p6tx/0sCr2VX5nNGgE\n\tnZdEbBjmnA35JIBFWNPiS3T8h48HBvIRj4Apza/c=","Date":"Sun, 15 Dec 2024 21:43:20 +0200","From":"Laurent Pinchart ","To":"=?utf-8?q?Barnab=C3=A1s_P=C5=91cze?= ","Cc":"libcamera-devel@lists.libcamera.org","Subject":"Re: [libcamera-ci] [RFC PATCH v1 2/3] Separate the building and\n\trunning of unit tests","Message-ID":"<20241215194320.GN9975@pendragon.ideasonboard.com>","References":"<20241212181655.112958-1-barnabas.pocze@ideasonboard.com>\n\t<20241212181655.112958-2-barnabas.pocze@ideasonboard.com>\n\t<20241215190408.GL9975@pendragon.ideasonboard.com>","MIME-Version":"1.0","Content-Type":"text/plain; charset=utf-8","Content-Disposition":"inline","Content-Transfer-Encoding":"8bit","In-Reply-To":"<20241215190408.GL9975@pendragon.ideasonboard.com>","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" "}},{"id":32737,"web_url":"https://patchwork.libcamera.org/comment/32737/","msgid":"<20241215204350.GP9975@pendragon.ideasonboard.com>","date":"2024-12-15T20:43:50","subject":"Re: [libcamera-ci] [RFC PATCH v1 2/3] Separate the building and\n\trunning of unit tests","submitter":{"id":2,"url":"https://patchwork.libcamera.org/api/people/2/","name":"Laurent Pinchart","email":"laurent.pinchart@ideasonboard.com"},"content":"On Sun, Dec 15, 2024 at 09:43:20PM +0200, Laurent Pinchart wrote:\n> On Sun, Dec 15, 2024 at 09:04:08PM +0200, Laurent Pinchart wrote:\n> > Hi Barnabás,\n> > \n> > Thank you for the patch.\n> > \n> > On Thu, Dec 12, 2024 at 07:16:54PM +0100, Barnabás Pőcze wrote:\n> > > The built artifacts will be reused in a later job, so split\n> > > the \"test-unit\" into the \"build-test\" and \"test-unit\" jobs.\n> > > \n> > > The `libevent` development package cannot be installed in the container\n> > \n> > I've write `libevent-dev` here to avoid ambiguities.\n> > \n> > > directly because it is not multiarch compatible. It is, however, installed\n> > > in the architecture specific build jobs, right before building. To ensure\n> > > that the it is available for already built executables in different jobs,\n> > \n> > \"that the it is\" ?\n> > \n> > > install just the libraries in the container.\n> > \n> > And name here `libevent`.\n> > \n> > > \n> > > Signed-off-by: Barnabás Pőcze \n> > > ---\n> > > .gitlab-ci/setup-container.sh | 3 +++\n> > > gitlab-ci.yml | 42 +++++++++++++++++++++++------------\n> > > 2 files changed, 31 insertions(+), 14 deletions(-)\n> > > \n> > > diff --git a/.gitlab-ci/setup-container.sh b/.gitlab-ci/setup-container.sh\n> > > index d2909c7..0658368 100755\n> > > --- a/.gitlab-ci/setup-container.sh\n> > > +++ b/.gitlab-ci/setup-container.sh\n> > > @@ -103,6 +103,9 @@ case $FDO_DISTRIBUTION_VERSION in\n> > > 'bookworm')\n> > > \t# libclang-rt-dev for the clang ASan runtime.\n> > > \tPKGS_LIBCAMERA_RUNTIME_MULTIARCH+=( libclang-rt-dev )\n> > > +\t# For cam and lc-compliance\n> > > +\t# libevent-dev cannot be used here, see build-libcamera-common.sh\n> > > +\tPKGS_LIBCAMERA_RUNTIME_MULTIARCH+=( libevent-2.1-7 libevent-pthreads-2.1-7 )\n> > > \t;;\n> > > 'trixie')\n> > > \t# gcc 13 to expand compilation testing coverage.\n> > > diff --git a/gitlab-ci.yml b/gitlab-ci.yml\n> > > index 8bc8bc2..c7448b8 100644\n> > > --- a/gitlab-ci.yml\n> > > +++ b/gitlab-ci.yml\n> > > @@ -64,7 +64,7 @@ include:\n> > > .libcamera-ci.debian:12:\n> > > variables:\n> > > FDO_DISTRIBUTION_VERSION: 'bookworm'\n> > > - FDO_DISTRIBUTION_TAG: '2024-12-12.1'\n> > > + FDO_DISTRIBUTION_TAG: '2024-12-12.2'\n> > > \n> > > .libcamera-ci.debian:13:\n> > > variables:\n> > > @@ -363,28 +363,18 @@ test-soraka:\n> > > script:\n> > > - submit .gitlab-ci/lava/soraka-camera-test.yml\n> > > \n> > > -# Run the unit tests in a virtual machine. Enable only the options exercised by\n> > > -# the unit tests.\n> > > -test-unit:\n> > > +# Enable only the options exercised by the unit tests.\n> > > +build-test:debug:\n> > \n> > I'd call this build-package:amd64, as we have build-package:arm64 and\n> > build-package:cros. I think it would also make sense to use the same\n> > build options for the amd64 and arm64 packages (beside possibly the\n> > selected pipeline handlers, although the 'auto' option may work for\n> > both).\n> > \n> > > extends:\n> > > - .fdo.distribution-image@debian\n> > > - .libcamera-ci.debian:12\n> > > - .libcamera-ci.scripts\n> > > - stage: test\n> > > + stage: build\n> > > needs:\n> > > - job: container-debian:12\n> > > artifacts: false\n> > > - tags:\n> > > - - kvm\n> > > script:\n> > > - $CI_PROJECT_DIR/.gitlab-ci/build-libcamera.sh\n> > > - - $CI_PROJECT_DIR/.gitlab-ci/test-libcamera-qemu.sh\n> > > - artifacts:\n> > > - name: libcamera-unit-tests-${CI_COMMIT_SHA}\n> > > - when: always\n> > > - expire_in: 1 week\n> > > - paths:\n> > > - - build/meson-logs/\n> > > variables:\n> > > BUILD_TYPE: debug\n> > > MESON_OPTIONS: >-\n> > > @@ -399,6 +389,30 @@ test-unit:\n> > > -D qcam=disabled\n> > > -D test=true\n> > > -D v4l2=true\n> > > + artifacts:\n> > > + paths:\n> > > + - build/\n> > \n> > The whole build directory can be very large. Can't we do the same as\n> > build-package:arm64 and run package-libcamera.sh to only package what we\n> > need ? We'll need probably need an unpackage script for the test-unit\n> > job.\n> \n> But of course the unit test binaries don't get installed... Can we fix\n> that and install them ? You can specify \"install_tag : 'tests'\" in\n> meson.build so they won't be installed by default (an appropriate\n> install_dir is also needed). This in turn requires bumping the minimum\n> meson version from 0.63.0 to 0.64.0, which shouldn't be an issue.\n\nI've been told on IRC that the motivation for the \"tests\" install tag in\nmeson is https://gitlab.gnome.org/GNOME/gnome-desktop-testing. I don't\nthink we should switch to a separate runner for unit tests (the pain is\nnot worth the gain at this point in my opinion), but it could be useful\nto tag lc-compliance with install_tag = 'tests'.\n\n> And now that I've said this, I realize we wouldn't be able to run \"meson\n> test\" to run the tests :-/ I'm not sure there's an appropriate solution\n> for this. If not, given the size of the build directory, and to avoid\n> transferring a large amount of data between runners, we may need to keep\n> building libcamera within the test-unit job :-(\n> \n> A separate build-package target for lc-compliance would still make\n> sense.\n> \n> > > + expire_in: 1 day\n> > > +\n> > > +# Run the unit tests in a virtual machine.\n> > > +test-unit:\n> > > + extends:\n> > > + - .fdo.distribution-image@debian\n> > > + - .libcamera-ci.debian:12\n> > > + - .libcamera-ci.scripts\n> > > + stage: test\n> > > + needs:\n> > > + - job: build-test:debug\n> > > + tags:\n> > > + - kvm\n> > > + script:\n> > > + - $CI_PROJECT_DIR/.gitlab-ci/test-libcamera-qemu.sh\n> > > + artifacts:\n> > > + name: libcamera-unit-tests-${CI_COMMIT_SHA}\n> > > + when: always\n> > > + expire_in: 1 week\n> > > + paths:\n> > > + - build/meson-logs/\n> > > \n> > > # meson prior to 1.2.0 doesn't correctly escape non-printable characters\n> > > # when generating the testlog XML. This results in an unparseable file.","headers":{"Return-Path":"","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id 67D4FC32F6\n\tfor ;\n\tSun, 15 Dec 2024 20:44:09 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id 7B29267EEE;\n\tSun, 15 Dec 2024 21:44:08 +0100 (CET)","from perceval.ideasonboard.com (perceval.ideasonboard.com\n\t[IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id 1830A67EEE\n\tfor ;\n\tSun, 15 Dec 2024 21:44:07 +0100 (CET)","from pendragon.ideasonboard.com (81-175-209-231.bb.dnainternet.fi\n\t[81.175.209.231])\n\tby perceval.ideasonboard.com (Postfix) with ESMTPSA id C675F7E9;\n\tSun, 15 Dec 2024 21:43:30 +0100 (CET)"],"Authentication-Results":"lancelot.ideasonboard.com; dkim=pass (1024-bit key;\n\tunprotected) header.d=ideasonboard.com header.i=@ideasonboard.com\n\theader.b=\"UorPg7ya\"; dkim-atps=neutral","DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com;\n\ts=mail; t=1734295411;\n\tbh=mIr61Fitazl0ZpJBsN3PPSSfNrr5qS1i8XlSMd4SV4s=;\n\th=Date:From:To:Cc:Subject:References:In-Reply-To:From;\n\tb=UorPg7yabYXYHWK6sXhwz2kKc37VO51/mH51EqGHea1Z2vglw3Ymr7+PBD34Ii7hY\n\tsWYA3D+P6/VTd3b0h9Fge5aCw/hXB5dubIf+4/0aorQqQuZZQH6iyPweGCe//3dPEK\n\t0SnZnGeP0+vSiHLgZCkXbaM8FPVv+BWJDP4O7/gs=","Date":"Sun, 15 Dec 2024 22:43:50 +0200","From":"Laurent Pinchart ","To":"=?utf-8?q?Barnab=C3=A1s_P=C5=91cze?= ","Cc":"libcamera-devel@lists.libcamera.org","Subject":"Re: [libcamera-ci] [RFC PATCH v1 2/3] Separate the building and\n\trunning of unit tests","Message-ID":"<20241215204350.GP9975@pendragon.ideasonboard.com>","References":"<20241212181655.112958-1-barnabas.pocze@ideasonboard.com>\n\t<20241212181655.112958-2-barnabas.pocze@ideasonboard.com>\n\t<20241215190408.GL9975@pendragon.ideasonboard.com>\n\t<20241215194320.GN9975@pendragon.ideasonboard.com>","MIME-Version":"1.0","Content-Type":"text/plain; charset=utf-8","Content-Disposition":"inline","Content-Transfer-Encoding":"8bit","In-Reply-To":"<20241215194320.GN9975@pendragon.ideasonboard.com>","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" "}},{"id":32751,"web_url":"https://patchwork.libcamera.org/comment/32751/","msgid":"<7948749f-b11a-4397-8332-29c681f188d0@ideasonboard.com>","date":"2024-12-16T09:13:54","subject":"Re: [libcamera-ci] [RFC PATCH v1 2/3] Separate the building and\n\trunning of unit tests","submitter":{"id":216,"url":"https://patchwork.libcamera.org/api/people/216/","name":"Barnabás Pőcze","email":"barnabas.pocze@ideasonboard.com"},"content":"Hi\n\n\n2024. 12. 15. 21:43 keltezéssel, Laurent Pinchart írta:\n> On Sun, Dec 15, 2024 at 09:43:20PM +0200, Laurent Pinchart wrote:\n>> On Sun, Dec 15, 2024 at 09:04:08PM +0200, Laurent Pinchart wrote:\n>>> Hi Barnabás,\n>>>\n>>> Thank you for the patch.\n>>>\n>>> On Thu, Dec 12, 2024 at 07:16:54PM +0100, Barnabás Pőcze wrote:\n>>>> The built artifacts will be reused in a later job, so split\n>>>> the \"test-unit\" into the \"build-test\" and \"test-unit\" jobs.\n>>>>\n>>>> The `libevent` development package cannot be installed in the container\n>>>\n>>> I've write `libevent-dev` here to avoid ambiguities.\n>>>\n>>>> directly because it is not multiarch compatible. It is, however, installed\n>>>> in the architecture specific build jobs, right before building. To ensure\n>>>> that the it is available for already built executables in different jobs,\n>>>\n>>> \"that the it is\" ?\n>>>\n>>>> install just the libraries in the container.\n>>>\n>>> And name here `libevent`.\n>>>\n>>>>\n>>>> Signed-off-by: Barnabás Pőcze \n>>>> ---\n>>>> .gitlab-ci/setup-container.sh | 3 +++\n>>>> gitlab-ci.yml | 42 +++++++++++++++++++++++------------\n>>>> 2 files changed, 31 insertions(+), 14 deletions(-)\n>>>>\n>>>> diff --git a/.gitlab-ci/setup-container.sh b/.gitlab-ci/setup-container.sh\n>>>> index d2909c7..0658368 100755\n>>>> --- a/.gitlab-ci/setup-container.sh\n>>>> +++ b/.gitlab-ci/setup-container.sh\n>>>> @@ -103,6 +103,9 @@ case $FDO_DISTRIBUTION_VERSION in\n>>>> 'bookworm')\n>>>> \t# libclang-rt-dev for the clang ASan runtime.\n>>>> \tPKGS_LIBCAMERA_RUNTIME_MULTIARCH+=( libclang-rt-dev )\n>>>> +\t# For cam and lc-compliance\n>>>> +\t# libevent-dev cannot be used here, see build-libcamera-common.sh\n>>>> +\tPKGS_LIBCAMERA_RUNTIME_MULTIARCH+=( libevent-2.1-7 libevent-pthreads-2.1-7 )\n>>>> \t;;\n>>>> 'trixie')\n>>>> \t# gcc 13 to expand compilation testing coverage.\n>>>> diff --git a/gitlab-ci.yml b/gitlab-ci.yml\n>>>> index 8bc8bc2..c7448b8 100644\n>>>> --- a/gitlab-ci.yml\n>>>> +++ b/gitlab-ci.yml\n>>>> @@ -64,7 +64,7 @@ include:\n>>>> .libcamera-ci.debian:12:\n>>>> variables:\n>>>> FDO_DISTRIBUTION_VERSION: 'bookworm'\n>>>> - FDO_DISTRIBUTION_TAG: '2024-12-12.1'\n>>>> + FDO_DISTRIBUTION_TAG: '2024-12-12.2'\n>>>>\n>>>> .libcamera-ci.debian:13:\n>>>> variables:\n>>>> @@ -363,28 +363,18 @@ test-soraka:\n>>>> script:\n>>>> - submit .gitlab-ci/lava/soraka-camera-test.yml\n>>>>\n>>>> -# Run the unit tests in a virtual machine. Enable only the options exercised by\n>>>> -# the unit tests.\n>>>> -test-unit:\n>>>> +# Enable only the options exercised by the unit tests.\n>>>> +build-test:debug:\n>>>\n>>> I'd call this build-package:amd64, as we have build-package:arm64 and\n>>> build-package:cros. I think it would also make sense to use the same\n>>> build options for the amd64 and arm64 packages (beside possibly the\n>>> selected pipeline handlers, although the 'auto' option may work for\n>>> both).\n>>>\n>>>> extends:\n>>>> - .fdo.distribution-image@debian\n>>>> - .libcamera-ci.debian:12\n>>>> - .libcamera-ci.scripts\n>>>> - stage: test\n>>>> + stage: build\n>>>> needs:\n>>>> - job: container-debian:12\n>>>> artifacts: false\n>>>> - tags:\n>>>> - - kvm\n>>>> script:\n>>>> - $CI_PROJECT_DIR/.gitlab-ci/build-libcamera.sh\n>>>> - - $CI_PROJECT_DIR/.gitlab-ci/test-libcamera-qemu.sh\n>>>> - artifacts:\n>>>> - name: libcamera-unit-tests-${CI_COMMIT_SHA}\n>>>> - when: always\n>>>> - expire_in: 1 week\n>>>> - paths:\n>>>> - - build/meson-logs/\n>>>> variables:\n>>>> BUILD_TYPE: debug\n>>>> MESON_OPTIONS: >-\n>>>> @@ -399,6 +389,30 @@ test-unit:\n>>>> -D qcam=disabled\n>>>> -D test=true\n>>>> -D v4l2=true\n>>>> + artifacts:\n>>>> + paths:\n>>>> + - build/\n>>>\n>>> The whole build directory can be very large. Can't we do the same as\n>>> build-package:arm64 and run package-libcamera.sh to only package what we\n>>> need ? We'll need probably need an unpackage script for the test-unit\n>>> job.\n>>\n>> But of course the unit test binaries don't get installed... Can we fix\n>> that and install them ? You can specify \"install_tag : 'tests'\" in\n>> meson.build so they won't be installed by default (an appropriate\n>> install_dir is also needed). This in turn requires bumping the minimum\n>> meson version from 0.63.0 to 0.64.0, which shouldn't be an issue.\n> \n> I've been told on IRC that the motivation for the \"tests\" install tag in\n> meson is https://gitlab.gnome.org/GNOME/gnome-desktop-testing. I don't\n> think we should switch to a separate runner for unit tests (the pain is\n> not worth the gain at this point in my opinion), but it could be useful\n> to tag lc-compliance with install_tag = 'tests'.\n> \n>> And now that I've said this, I realize we wouldn't be able to run \"meson\n>> test\" to run the tests :-/ I'm not sure there's an appropriate solution\n>> for this. If not, given the size of the build directory, and to avoid\n>> transferring a large amount of data between runners, we may need to keep\n>> building libcamera within the test-unit job :-(\n>>\n>> A separate build-package target for lc-compliance would still make\n>> sense.\n\nI think it would be unfortunate to give up the usage `meson test` as you\nmentioned. I have not noticed that these build artifacts would put any\nappreciable strain on the infrastructure. The compressed build directory\ncomes out to around 167 MiB; I am not sure if I would consider that a\nlarge amount of data. It is definitely cheaper, in terms of time, than\nbuilding libcamera twice. Clearing the object files could be another\noption. With `artifacts:exclude: build/**/*.o` we can seemingly\nremove more than half of the uncompressed size, and about 1/4 of\nthe compressed size. Does this look acceptable?\n\n\nRegards,\nBarnabás Pőcze\n\n\n>>\n>>>> + expire_in: 1 day\n>>>> +\n>>>> +# Run the unit tests in a virtual machine.\n>>>> +test-unit:\n>>>> + extends:\n>>>> + - .fdo.distribution-image@debian\n>>>> + - .libcamera-ci.debian:12\n>>>> + - .libcamera-ci.scripts\n>>>> + stage: test\n>>>> + needs:\n>>>> + - job: build-test:debug\n>>>> + tags:\n>>>> + - kvm\n>>>> + script:\n>>>> + - $CI_PROJECT_DIR/.gitlab-ci/test-libcamera-qemu.sh\n>>>> + artifacts:\n>>>> + name: libcamera-unit-tests-${CI_COMMIT_SHA}\n>>>> + when: always\n>>>> + expire_in: 1 week\n>>>> + paths:\n>>>> + - build/meson-logs/\n>>>>\n>>>> # meson prior to 1.2.0 doesn't correctly escape non-printable characters\n>>>> # when generating the testlog XML. This results in an unparseable file.\n>","headers":{"Return-Path":"","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id 33ED0C32F9\n\tfor ;\n\tMon, 16 Dec 2024 09:14:01 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id 4FC4467ECA;\n\tMon, 16 Dec 2024 10:14:00 +0100 (CET)","from perceval.ideasonboard.com (perceval.ideasonboard.com\n\t[IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id 0164867ECA\n\tfor ;\n\tMon, 16 Dec 2024 10:13:57 +0100 (CET)","from [192.168.33.16] (185.221.140.157.nat.pool.zt.hu\n\t[185.221.140.157])\n\tby perceval.ideasonboard.com (Postfix) with ESMTPSA id 6148513C;\n\tMon, 16 Dec 2024 10:13:21 +0100 (CET)"],"Authentication-Results":"lancelot.ideasonboard.com; dkim=pass (1024-bit key;\n\tunprotected) header.d=ideasonboard.com header.i=@ideasonboard.com\n\theader.b=\"MMIon+3U\"; dkim-atps=neutral","DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com;\n\ts=mail; t=1734340401;\n\tbh=YpLjpikNjRT5ks7Xqm2SZ1+ooc5YGfb3ibZW16GZLKo=;\n\th=Date:Subject:To:Cc:References:From:In-Reply-To:From;\n\tb=MMIon+3UOtlRwL6nyok3ABhlf3W39SjPeYJzKcn66F4BRAE+Ki4aUMLKqf7PMvgBP\n\t+OPrQ4SOLv0T6UuFCol/J6fbDJ4XLdmY2a5XqO9+bgpDC3E0OR77QYk/rUE0rldvfS\n\t5qKpPCz8aEvop60O6iRncx4xYH5YqxlO697wo9dc=","Message-ID":"<7948749f-b11a-4397-8332-29c681f188d0@ideasonboard.com>","Date":"Mon, 16 Dec 2024 10:13:54 +0100","MIME-Version":"1.0","User-Agent":"Mozilla Thunderbird","Subject":"Re: [libcamera-ci] [RFC PATCH v1 2/3] Separate the building and\n\trunning of unit tests","To":"Laurent Pinchart ","Cc":"libcamera-devel@lists.libcamera.org","References":"<20241212181655.112958-1-barnabas.pocze@ideasonboard.com>\n\t<20241212181655.112958-2-barnabas.pocze@ideasonboard.com>\n\t<20241215190408.GL9975@pendragon.ideasonboard.com>\n\t<20241215194320.GN9975@pendragon.ideasonboard.com>\n\t<20241215204350.GP9975@pendragon.ideasonboard.com>","Content-Language":"en-US, hu-HU","From":"=?utf-8?q?Barnab=C3=A1s_P=C5=91cze?= ","In-Reply-To":"<20241215204350.GP9975@pendragon.ideasonboard.com>","Content-Type":"text/plain; charset=UTF-8; format=flowed","Content-Transfer-Encoding":"8bit","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" "}},{"id":32759,"web_url":"https://patchwork.libcamera.org/comment/32759/","msgid":"<20241216110442.GH32204@pendragon.ideasonboard.com>","date":"2024-12-16T11:04:42","subject":"Re: [libcamera-ci] [RFC PATCH v1 2/3] Separate the building and\n\trunning of unit tests","submitter":{"id":2,"url":"https://patchwork.libcamera.org/api/people/2/","name":"Laurent Pinchart","email":"laurent.pinchart@ideasonboard.com"},"content":"On Mon, Dec 16, 2024 at 10:13:54AM +0100, Barnabás Pőcze wrote:\n> 2024. 12. 15. 21:43 keltezéssel, Laurent Pinchart írta:\n> > On Sun, Dec 15, 2024 at 09:43:20PM +0200, Laurent Pinchart wrote:\n> >> On Sun, Dec 15, 2024 at 09:04:08PM +0200, Laurent Pinchart wrote:\n> >>> Hi Barnabás,\n> >>>\n> >>> Thank you for the patch.\n> >>>\n> >>> On Thu, Dec 12, 2024 at 07:16:54PM +0100, Barnabás Pőcze wrote:\n> >>>> The built artifacts will be reused in a later job, so split\n> >>>> the \"test-unit\" into the \"build-test\" and \"test-unit\" jobs.\n> >>>>\n> >>>> The `libevent` development package cannot be installed in the container\n> >>>\n> >>> I've write `libevent-dev` here to avoid ambiguities.\n> >>>\n> >>>> directly because it is not multiarch compatible. It is, however, installed\n> >>>> in the architecture specific build jobs, right before building. To ensure\n> >>>> that the it is available for already built executables in different jobs,\n> >>>\n> >>> \"that the it is\" ?\n> >>>\n> >>>> install just the libraries in the container.\n> >>>\n> >>> And name here `libevent`.\n> >>>\n> >>>>\n> >>>> Signed-off-by: Barnabás Pőcze \n> >>>> ---\n> >>>> .gitlab-ci/setup-container.sh | 3 +++\n> >>>> gitlab-ci.yml | 42 +++++++++++++++++++++++------------\n> >>>> 2 files changed, 31 insertions(+), 14 deletions(-)\n> >>>>\n> >>>> diff --git a/.gitlab-ci/setup-container.sh b/.gitlab-ci/setup-container.sh\n> >>>> index d2909c7..0658368 100755\n> >>>> --- a/.gitlab-ci/setup-container.sh\n> >>>> +++ b/.gitlab-ci/setup-container.sh\n> >>>> @@ -103,6 +103,9 @@ case $FDO_DISTRIBUTION_VERSION in\n> >>>> 'bookworm')\n> >>>> \t# libclang-rt-dev for the clang ASan runtime.\n> >>>> \tPKGS_LIBCAMERA_RUNTIME_MULTIARCH+=( libclang-rt-dev )\n> >>>> +\t# For cam and lc-compliance\n> >>>> +\t# libevent-dev cannot be used here, see build-libcamera-common.sh\n> >>>> +\tPKGS_LIBCAMERA_RUNTIME_MULTIARCH+=( libevent-2.1-7 libevent-pthreads-2.1-7 )\n> >>>> \t;;\n> >>>> 'trixie')\n> >>>> \t# gcc 13 to expand compilation testing coverage.\n> >>>> diff --git a/gitlab-ci.yml b/gitlab-ci.yml\n> >>>> index 8bc8bc2..c7448b8 100644\n> >>>> --- a/gitlab-ci.yml\n> >>>> +++ b/gitlab-ci.yml\n> >>>> @@ -64,7 +64,7 @@ include:\n> >>>> .libcamera-ci.debian:12:\n> >>>> variables:\n> >>>> FDO_DISTRIBUTION_VERSION: 'bookworm'\n> >>>> - FDO_DISTRIBUTION_TAG: '2024-12-12.1'\n> >>>> + FDO_DISTRIBUTION_TAG: '2024-12-12.2'\n> >>>>\n> >>>> .libcamera-ci.debian:13:\n> >>>> variables:\n> >>>> @@ -363,28 +363,18 @@ test-soraka:\n> >>>> script:\n> >>>> - submit .gitlab-ci/lava/soraka-camera-test.yml\n> >>>>\n> >>>> -# Run the unit tests in a virtual machine. Enable only the options exercised by\n> >>>> -# the unit tests.\n> >>>> -test-unit:\n> >>>> +# Enable only the options exercised by the unit tests.\n> >>>> +build-test:debug:\n> >>>\n> >>> I'd call this build-package:amd64, as we have build-package:arm64 and\n> >>> build-package:cros. I think it would also make sense to use the same\n> >>> build options for the amd64 and arm64 packages (beside possibly the\n> >>> selected pipeline handlers, although the 'auto' option may work for\n> >>> both).\n> >>>\n> >>>> extends:\n> >>>> - .fdo.distribution-image@debian\n> >>>> - .libcamera-ci.debian:12\n> >>>> - .libcamera-ci.scripts\n> >>>> - stage: test\n> >>>> + stage: build\n> >>>> needs:\n> >>>> - job: container-debian:12\n> >>>> artifacts: false\n> >>>> - tags:\n> >>>> - - kvm\n> >>>> script:\n> >>>> - $CI_PROJECT_DIR/.gitlab-ci/build-libcamera.sh\n> >>>> - - $CI_PROJECT_DIR/.gitlab-ci/test-libcamera-qemu.sh\n> >>>> - artifacts:\n> >>>> - name: libcamera-unit-tests-${CI_COMMIT_SHA}\n> >>>> - when: always\n> >>>> - expire_in: 1 week\n> >>>> - paths:\n> >>>> - - build/meson-logs/\n> >>>> variables:\n> >>>> BUILD_TYPE: debug\n> >>>> MESON_OPTIONS: >-\n> >>>> @@ -399,6 +389,30 @@ test-unit:\n> >>>> -D qcam=disabled\n> >>>> -D test=true\n> >>>> -D v4l2=true\n> >>>> + artifacts:\n> >>>> + paths:\n> >>>> + - build/\n> >>>\n> >>> The whole build directory can be very large. Can't we do the same as\n> >>> build-package:arm64 and run package-libcamera.sh to only package what we\n> >>> need ? We'll need probably need an unpackage script for the test-unit\n> >>> job.\n> >>\n> >> But of course the unit test binaries don't get installed... Can we fix\n> >> that and install them ? You can specify \"install_tag : 'tests'\" in\n> >> meson.build so they won't be installed by default (an appropriate\n> >> install_dir is also needed). This in turn requires bumping the minimum\n> >> meson version from 0.63.0 to 0.64.0, which shouldn't be an issue.\n> > \n> > I've been told on IRC that the motivation for the \"tests\" install tag in\n> > meson is https://gitlab.gnome.org/GNOME/gnome-desktop-testing. I don't\n> > think we should switch to a separate runner for unit tests (the pain is\n> > not worth the gain at this point in my opinion), but it could be useful\n> > to tag lc-compliance with install_tag = 'tests'.\n> > \n> >> And now that I've said this, I realize we wouldn't be able to run \"meson\n> >> test\" to run the tests :-/ I'm not sure there's an appropriate solution\n> >> for this. If not, given the size of the build directory, and to avoid\n> >> transferring a large amount of data between runners, we may need to keep\n> >> building libcamera within the test-unit job :-(\n> >>\n> >> A separate build-package target for lc-compliance would still make\n> >> sense.\n> \n> I think it would be unfortunate to give up the usage `meson test` as you\n> mentioned.\n\nWe could work on a replacement, but it would require a significant\namount of work and I think there are better things to do.\n\n> I have not noticed that these build artifacts would put any\n> appreciable strain on the infrastructure. The compressed build directory\n> comes out to around 167 MiB; I am not sure if I would consider that a\n> large amount of data. It is definitely cheaper, in terms of time, than\n> building libcamera twice. Clearing the object files could be another\n> option. With `artifacts:exclude: build/**/*.o` we can seemingly\n> remove more than half of the uncompressed size, and about 1/4 of\n> the compressed size. Does this look acceptable?\n\nPossibly. We should probably ask the fdo sysadmins about what is\nacceptable.\n\nI gave it a try locally though, and deleting all *.o files in the build\ndirectory results in \"meson test\" rebuilding everything.\n\nFor other uses of the artifacts (in particular deployment on real\ndevices), I would still prefer minimizing the bandwidth, creating a\npackage similarly to what build-package:arm64 does. How about keeping\ntest-unit as-is, at the cost of a recompilation, and creating a\nbuild-package:amd64 that will be used by the lc-compliance test job ? We\ncan try to improve on top when/if needed.\n\n> >>>> + expire_in: 1 day\n> >>>> +\n> >>>> +# Run the unit tests in a virtual machine.\n> >>>> +test-unit:\n> >>>> + extends:\n> >>>> + - .fdo.distribution-image@debian\n> >>>> + - .libcamera-ci.debian:12\n> >>>> + - .libcamera-ci.scripts\n> >>>> + stage: test\n> >>>> + needs:\n> >>>> + - job: build-test:debug\n> >>>> + tags:\n> >>>> + - kvm\n> >>>> + script:\n> >>>> + - $CI_PROJECT_DIR/.gitlab-ci/test-libcamera-qemu.sh\n> >>>> + artifacts:\n> >>>> + name: libcamera-unit-tests-${CI_COMMIT_SHA}\n> >>>> + when: always\n> >>>> + expire_in: 1 week\n> >>>> + paths:\n> >>>> + - build/meson-logs/\n> >>>>\n> >>>> # meson prior to 1.2.0 doesn't correctly escape non-printable characters\n> >>>> # when generating the testlog XML. This results in an unparseable file.","headers":{"Return-Path":"","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id BA75CC32F6\n\tfor ;\n\tMon, 16 Dec 2024 11:05:00 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id 6FA3B67F56;\n\tMon, 16 Dec 2024 12:05:00 +0100 (CET)","from perceval.ideasonboard.com (perceval.ideasonboard.com\n\t[213.167.242.64])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id 4C9C167EF1\n\tfor ;\n\tMon, 16 Dec 2024 12:04:58 +0100 (CET)","from pendragon.ideasonboard.com (81-175-209-231.bb.dnainternet.fi\n\t[81.175.209.231])\n\tby perceval.ideasonboard.com (Postfix) with ESMTPSA id B7EF913C;\n\tMon, 16 Dec 2024 12:04:21 +0100 (CET)"],"Authentication-Results":"lancelot.ideasonboard.com; dkim=pass (1024-bit key;\n\tunprotected) header.d=ideasonboard.com header.i=@ideasonboard.com\n\theader.b=\"e5iyyurW\"; dkim-atps=neutral","DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com;\n\ts=mail; t=1734347062;\n\tbh=0SSP1XHjBucb1bHPgVBO2YdPG3N3qHMc8eIStY4CMwg=;\n\th=Date:From:To:Cc:Subject:References:In-Reply-To:From;\n\tb=e5iyyurWT4gaGiYATXPxYUpDKO/c2h+0Pk/DBgy2zsLwyV91E/cPmmv7/IDEYQ1Bq\n\tkP7D3ChPqP+YTQAbUrId7jVb3YWe7wGRdRsCTQVOs6iQduAwstN4HcurXgCD6yMKYP\n\tBMdJ1Gx7916wMuXhsmvUDTK34FlK9Jyj1C2QusvU=","Date":"Mon, 16 Dec 2024 13:04:42 +0200","From":"Laurent Pinchart ","To":"=?utf-8?q?Barnab=C3=A1s_P=C5=91cze?= ","Cc":"libcamera-devel@lists.libcamera.org","Subject":"Re: [libcamera-ci] [RFC PATCH v1 2/3] Separate the building and\n\trunning of unit tests","Message-ID":"<20241216110442.GH32204@pendragon.ideasonboard.com>","References":"<20241212181655.112958-1-barnabas.pocze@ideasonboard.com>\n\t<20241212181655.112958-2-barnabas.pocze@ideasonboard.com>\n\t<20241215190408.GL9975@pendragon.ideasonboard.com>\n\t<20241215194320.GN9975@pendragon.ideasonboard.com>\n\t<20241215204350.GP9975@pendragon.ideasonboard.com>\n\t<7948749f-b11a-4397-8332-29c681f188d0@ideasonboard.com>","MIME-Version":"1.0","Content-Type":"text/plain; charset=utf-8","Content-Disposition":"inline","Content-Transfer-Encoding":"8bit","In-Reply-To":"<7948749f-b11a-4397-8332-29c681f188d0@ideasonboard.com>","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" "}},{"id":32774,"web_url":"https://patchwork.libcamera.org/comment/32774/","msgid":"<5c441936-60e0-40e5-b542-a02ccf0739ca@ideasonboard.com>","date":"2024-12-16T17:28:46","subject":"Re: [libcamera-ci] [RFC PATCH v1 2/3] Separate the building and\n\trunning of unit tests","submitter":{"id":216,"url":"https://patchwork.libcamera.org/api/people/216/","name":"Barnabás Pőcze","email":"barnabas.pocze@ideasonboard.com"},"content":"2024. 12. 16. 12:04 keltezéssel, Laurent Pinchart írta:\n> On Mon, Dec 16, 2024 at 10:13:54AM +0100, Barnabás Pőcze wrote:\n>> 2024. 12. 15. 21:43 keltezéssel, Laurent Pinchart írta:\n>>> On Sun, Dec 15, 2024 at 09:43:20PM +0200, Laurent Pinchart wrote:\n>>>> On Sun, Dec 15, 2024 at 09:04:08PM +0200, Laurent Pinchart wrote:\n>>>>> Hi Barnabás,\n>>>>>\n>>>>> Thank you for the patch.\n>>>>>\n>>>>> On Thu, Dec 12, 2024 at 07:16:54PM +0100, Barnabás Pőcze wrote:\n>>>>>> The built artifacts will be reused in a later job, so split\n>>>>>> the \"test-unit\" into the \"build-test\" and \"test-unit\" jobs.\n>>>>>>\n>>>>>> The `libevent` development package cannot be installed in the container\n>>>>>\n>>>>> I've write `libevent-dev` here to avoid ambiguities.\n>>>>>\n>>>>>> directly because it is not multiarch compatible. It is, however, installed\n>>>>>> in the architecture specific build jobs, right before building. To ensure\n>>>>>> that the it is available for already built executables in different jobs,\n>>>>>\n>>>>> \"that the it is\" ?\n>>>>>\n>>>>>> install just the libraries in the container.\n>>>>>\n>>>>> And name here `libevent`.\n>>>>>\n>>>>>>\n>>>>>> Signed-off-by: Barnabás Pőcze \n>>>>>> ---\n>>>>>> .gitlab-ci/setup-container.sh | 3 +++\n>>>>>> gitlab-ci.yml | 42 +++++++++++++++++++++++------------\n>>>>>> 2 files changed, 31 insertions(+), 14 deletions(-)\n>>>>>>\n>>>>>> diff --git a/.gitlab-ci/setup-container.sh b/.gitlab-ci/setup-container.sh\n>>>>>> index d2909c7..0658368 100755\n>>>>>> --- a/.gitlab-ci/setup-container.sh\n>>>>>> +++ b/.gitlab-ci/setup-container.sh\n>>>>>> @@ -103,6 +103,9 @@ case $FDO_DISTRIBUTION_VERSION in\n>>>>>> 'bookworm')\n>>>>>> \t# libclang-rt-dev for the clang ASan runtime.\n>>>>>> \tPKGS_LIBCAMERA_RUNTIME_MULTIARCH+=( libclang-rt-dev )\n>>>>>> +\t# For cam and lc-compliance\n>>>>>> +\t# libevent-dev cannot be used here, see build-libcamera-common.sh\n>>>>>> +\tPKGS_LIBCAMERA_RUNTIME_MULTIARCH+=( libevent-2.1-7 libevent-pthreads-2.1-7 )\n>>>>>> \t;;\n>>>>>> 'trixie')\n>>>>>> \t# gcc 13 to expand compilation testing coverage.\n>>>>>> diff --git a/gitlab-ci.yml b/gitlab-ci.yml\n>>>>>> index 8bc8bc2..c7448b8 100644\n>>>>>> --- a/gitlab-ci.yml\n>>>>>> +++ b/gitlab-ci.yml\n>>>>>> @@ -64,7 +64,7 @@ include:\n>>>>>> .libcamera-ci.debian:12:\n>>>>>> variables:\n>>>>>> FDO_DISTRIBUTION_VERSION: 'bookworm'\n>>>>>> - FDO_DISTRIBUTION_TAG: '2024-12-12.1'\n>>>>>> + FDO_DISTRIBUTION_TAG: '2024-12-12.2'\n>>>>>>\n>>>>>> .libcamera-ci.debian:13:\n>>>>>> variables:\n>>>>>> @@ -363,28 +363,18 @@ test-soraka:\n>>>>>> script:\n>>>>>> - submit .gitlab-ci/lava/soraka-camera-test.yml\n>>>>>>\n>>>>>> -# Run the unit tests in a virtual machine. Enable only the options exercised by\n>>>>>> -# the unit tests.\n>>>>>> -test-unit:\n>>>>>> +# Enable only the options exercised by the unit tests.\n>>>>>> +build-test:debug:\n>>>>>\n>>>>> I'd call this build-package:amd64, as we have build-package:arm64 and\n>>>>> build-package:cros. I think it would also make sense to use the same\n>>>>> build options for the amd64 and arm64 packages (beside possibly the\n>>>>> selected pipeline handlers, although the 'auto' option may work for\n>>>>> both).\n>>>>>\n>>>>>> extends:\n>>>>>> - .fdo.distribution-image@debian\n>>>>>> - .libcamera-ci.debian:12\n>>>>>> - .libcamera-ci.scripts\n>>>>>> - stage: test\n>>>>>> + stage: build\n>>>>>> needs:\n>>>>>> - job: container-debian:12\n>>>>>> artifacts: false\n>>>>>> - tags:\n>>>>>> - - kvm\n>>>>>> script:\n>>>>>> - $CI_PROJECT_DIR/.gitlab-ci/build-libcamera.sh\n>>>>>> - - $CI_PROJECT_DIR/.gitlab-ci/test-libcamera-qemu.sh\n>>>>>> - artifacts:\n>>>>>> - name: libcamera-unit-tests-${CI_COMMIT_SHA}\n>>>>>> - when: always\n>>>>>> - expire_in: 1 week\n>>>>>> - paths:\n>>>>>> - - build/meson-logs/\n>>>>>> variables:\n>>>>>> BUILD_TYPE: debug\n>>>>>> MESON_OPTIONS: >-\n>>>>>> @@ -399,6 +389,30 @@ test-unit:\n>>>>>> -D qcam=disabled\n>>>>>> -D test=true\n>>>>>> -D v4l2=true\n>>>>>> + artifacts:\n>>>>>> + paths:\n>>>>>> + - build/\n>>>>>\n>>>>> The whole build directory can be very large. Can't we do the same as\n>>>>> build-package:arm64 and run package-libcamera.sh to only package what we\n>>>>> need ? We'll need probably need an unpackage script for the test-unit\n>>>>> job.\n>>>>\n>>>> But of course the unit test binaries don't get installed... Can we fix\n>>>> that and install them ? You can specify \"install_tag : 'tests'\" in\n>>>> meson.build so they won't be installed by default (an appropriate\n>>>> install_dir is also needed). This in turn requires bumping the minimum\n>>>> meson version from 0.63.0 to 0.64.0, which shouldn't be an issue.\n>>>\n>>> I've been told on IRC that the motivation for the \"tests\" install tag in\n>>> meson is https://gitlab.gnome.org/GNOME/gnome-desktop-testing. I don't\n>>> think we should switch to a separate runner for unit tests (the pain is\n>>> not worth the gain at this point in my opinion), but it could be useful\n>>> to tag lc-compliance with install_tag = 'tests'.\n>>>\n>>>> And now that I've said this, I realize we wouldn't be able to run \"meson\n>>>> test\" to run the tests :-/ I'm not sure there's an appropriate solution\n>>>> for this. If not, given the size of the build directory, and to avoid\n>>>> transferring a large amount of data between runners, we may need to keep\n>>>> building libcamera within the test-unit job :-(\n>>>>\n>>>> A separate build-package target for lc-compliance would still make\n>>>> sense.\n>>\n>> I think it would be unfortunate to give up the usage `meson test` as you\n>> mentioned.\n> \n> We could work on a replacement, but it would require a significant\n> amount of work and I think there are better things to do.\n> \n>> I have not noticed that these build artifacts would put any\n>> appreciable strain on the infrastructure. The compressed build directory\n>> comes out to around 167 MiB; I am not sure if I would consider that a\n>> large amount of data. It is definitely cheaper, in terms of time, than\n>> building libcamera twice. Clearing the object files could be another\n>> option. With `artifacts:exclude: build/**/*.o` we can seemingly\n>> remove more than half of the uncompressed size, and about 1/4 of\n>> the compressed size. Does this look acceptable?\n> \n> Possibly. We should probably ask the fdo sysadmins about what is\n> acceptable.\n> \n> I gave it a try locally though, and deleting all *.o files in the build\n> directory results in \"meson test\" rebuilding everything.\n\nAs far as I can see that does not happen with the `--no-rebuild` option,\nwhich is already used in `test-libcamera-qemu.sh`.\n\n> \n> For other uses of the artifacts (in particular deployment on real\n> devices), I would still prefer minimizing the bandwidth, creating a\n> package similarly to what build-package:arm64 does. How about keeping\n> test-unit as-is, at the cost of a recompilation, and creating a\n> build-package:amd64 that will be used by the lc-compliance test job ? We\n> can try to improve on top when/if needed.\n\nCouple observations:\n\n1. The virtual pipeline handler configuration is not installed, so\n that needs to be addressed. (Was this omitted intentionally?)\n\n2. I am not a fan of the extra `tar` and `ldconfig` calls that\n need to be sprinkled in. I think this would be much better\n if the package was not a mere tar archive but a proper deb/etc.\n package. I imagine that is a prerequisite of deploying on real\n hardware in any case, correct?\n\n3. Ignoring `*.o` makes the sizes a lot closer in my opinion (for one\n particular build: uncompressed 163M vs. 143M, compressed 28M vs. 15M\n [probably largely due to the different compression schemes]).\n\n\nIn any case, please see the new version of these changes.\n\n\nRegards,\nBarnabás Pőcze\n\n\n> \n>>>>>> + expire_in: 1 day\n>>>>>> +\n>>>>>> +# Run the unit tests in a virtual machine.\n>>>>>> +test-unit:\n>>>>>> + extends:\n>>>>>> + - .fdo.distribution-image@debian\n>>>>>> + - .libcamera-ci.debian:12\n>>>>>> + - .libcamera-ci.scripts\n>>>>>> + stage: test\n>>>>>> + needs:\n>>>>>> + - job: build-test:debug\n>>>>>> + tags:\n>>>>>> + - kvm\n>>>>>> + script:\n>>>>>> + - $CI_PROJECT_DIR/.gitlab-ci/test-libcamera-qemu.sh\n>>>>>> + artifacts:\n>>>>>> + name: libcamera-unit-tests-${CI_COMMIT_SHA}\n>>>>>> + when: always\n>>>>>> + expire_in: 1 week\n>>>>>> + paths:\n>>>>>> + - build/meson-logs/\n>>>>>>\n>>>>>> # meson prior to 1.2.0 doesn't correctly escape non-printable characters\n>>>>>> # when generating the testlog XML. This results in an unparseable file.\n>","headers":{"Return-Path":"","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id BA805C32F6\n\tfor ;\n\tMon, 16 Dec 2024 17:28:51 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id 6FE9967F97;\n\tMon, 16 Dec 2024 18:28:51 +0100 (CET)","from perceval.ideasonboard.com (perceval.ideasonboard.com\n\t[IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id 9122F67F8E\n\tfor ;\n\tMon, 16 Dec 2024 18:28:49 +0100 (CET)","from [192.168.33.16] (185.221.140.157.nat.pool.zt.hu\n\t[185.221.140.157])\n\tby perceval.ideasonboard.com (Postfix) with ESMTPSA id E4A99675;\n\tMon, 16 Dec 2024 18:28:12 +0100 (CET)"],"Authentication-Results":"lancelot.ideasonboard.com; dkim=pass (1024-bit key;\n\tunprotected) header.d=ideasonboard.com header.i=@ideasonboard.com\n\theader.b=\"VgnqyXDc\"; dkim-atps=neutral","DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com;\n\ts=mail; t=1734370093;\n\tbh=EpykPfpuhhiWHSSXyHstk1L/XFTBUYXqhyXM5CfFGQU=;\n\th=Date:Subject:To:Cc:References:From:In-Reply-To:From;\n\tb=VgnqyXDcBK9TvdwuqexdefOSD/Dw0CAH3PGQFildE0Gt506s06kJQRa08mRlK/LnM\n\t+APiSVQwB+Suw8941TtVn4SXl777O0pelsgPRTXPgrdUkVr+lIzOReiORZzQ6PMRtC\n\t2Tfb9HAmmf3jDNgSEqDzAHRam2lUSevsFSS9G3z8=","Message-ID":"<5c441936-60e0-40e5-b542-a02ccf0739ca@ideasonboard.com>","Date":"Mon, 16 Dec 2024 18:28:46 +0100","MIME-Version":"1.0","User-Agent":"Mozilla Thunderbird","Subject":"Re: [libcamera-ci] [RFC PATCH v1 2/3] Separate the building and\n\trunning of unit tests","To":"Laurent Pinchart ","Cc":"libcamera-devel@lists.libcamera.org","References":"<20241212181655.112958-1-barnabas.pocze@ideasonboard.com>\n\t<20241212181655.112958-2-barnabas.pocze@ideasonboard.com>\n\t<20241215190408.GL9975@pendragon.ideasonboard.com>\n\t<20241215194320.GN9975@pendragon.ideasonboard.com>\n\t<20241215204350.GP9975@pendragon.ideasonboard.com>\n\t<7948749f-b11a-4397-8332-29c681f188d0@ideasonboard.com>\n\t<20241216110442.GH32204@pendragon.ideasonboard.com>","Content-Language":"en-US, hu-HU","From":"=?utf-8?q?Barnab=C3=A1s_P=C5=91cze?= ","In-Reply-To":"<20241216110442.GH32204@pendragon.ideasonboard.com>","Content-Type":"text/plain; charset=UTF-8; format=flowed","Content-Transfer-Encoding":"8bit","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" "}},{"id":32780,"web_url":"https://patchwork.libcamera.org/comment/32780/","msgid":"<173437357450.543771.15768411058221148888@ping.linuxembedded.co.uk>","date":"2024-12-16T18:26:14","subject":"Re: [libcamera-ci] [RFC PATCH v1 2/3] Separate the building and\n\trunning of unit tests","submitter":{"id":4,"url":"https://patchwork.libcamera.org/api/people/4/","name":"Kieran Bingham","email":"kieran.bingham@ideasonboard.com"},"content":"Quoting Barnabás Pőcze (2024-12-16 17:28:46)\n> 2024. 12. 16. 12:04 keltezéssel, Laurent Pinchart írta:\n> > On Mon, Dec 16, 2024 at 10:13:54AM +0100, Barnabás Pőcze wrote:\n> >> 2024. 12. 15. 21:43 keltezéssel, Laurent Pinchart írta:\n> >>> On Sun, Dec 15, 2024 at 09:43:20PM +0200, Laurent Pinchart wrote:\n> >>>> On Sun, Dec 15, 2024 at 09:04:08PM +0200, Laurent Pinchart wrote:\n> >>>>> Hi Barnabás,\n> >>>>>\n> >>>>> Thank you for the patch.\n> >>>>>\n> >>>>> On Thu, Dec 12, 2024 at 07:16:54PM +0100, Barnabás Pőcze wrote:\n> >>>>>> The built artifacts will be reused in a later job, so split\n> >>>>>> the \"test-unit\" into the \"build-test\" and \"test-unit\" jobs.\n> >>>>>>\n> >>>>>> The `libevent` development package cannot be installed in the container\n> >>>>>\n> >>>>> I've write `libevent-dev` here to avoid ambiguities.\n> >>>>>\n> >>>>>> directly because it is not multiarch compatible. It is, however, installed\n> >>>>>> in the architecture specific build jobs, right before building. To ensure\n> >>>>>> that the it is available for already built executables in different jobs,\n> >>>>>\n> >>>>> \"that the it is\" ?\n> >>>>>\n> >>>>>> install just the libraries in the container.\n> >>>>>\n> >>>>> And name here `libevent`.\n> >>>>>\n> >>>>>>\n> >>>>>> Signed-off-by: Barnabás Pőcze \n> >>>>>> ---\n> >>>>>> .gitlab-ci/setup-container.sh | 3 +++\n> >>>>>> gitlab-ci.yml | 42 +++++++++++++++++++++++------------\n> >>>>>> 2 files changed, 31 insertions(+), 14 deletions(-)\n> >>>>>>\n> >>>>>> diff --git a/.gitlab-ci/setup-container.sh b/.gitlab-ci/setup-container.sh\n> >>>>>> index d2909c7..0658368 100755\n> >>>>>> --- a/.gitlab-ci/setup-container.sh\n> >>>>>> +++ b/.gitlab-ci/setup-container.sh\n> >>>>>> @@ -103,6 +103,9 @@ case $FDO_DISTRIBUTION_VERSION in\n> >>>>>> 'bookworm')\n> >>>>>> # libclang-rt-dev for the clang ASan runtime.\n> >>>>>> PKGS_LIBCAMERA_RUNTIME_MULTIARCH+=( libclang-rt-dev )\n> >>>>>> + # For cam and lc-compliance\n> >>>>>> + # libevent-dev cannot be used here, see build-libcamera-common.sh\n> >>>>>> + PKGS_LIBCAMERA_RUNTIME_MULTIARCH+=( libevent-2.1-7 libevent-pthreads-2.1-7 )\n> >>>>>> ;;\n> >>>>>> 'trixie')\n> >>>>>> # gcc 13 to expand compilation testing coverage.\n> >>>>>> diff --git a/gitlab-ci.yml b/gitlab-ci.yml\n> >>>>>> index 8bc8bc2..c7448b8 100644\n> >>>>>> --- a/gitlab-ci.yml\n> >>>>>> +++ b/gitlab-ci.yml\n> >>>>>> @@ -64,7 +64,7 @@ include:\n> >>>>>> .libcamera-ci.debian:12:\n> >>>>>> variables:\n> >>>>>> FDO_DISTRIBUTION_VERSION: 'bookworm'\n> >>>>>> - FDO_DISTRIBUTION_TAG: '2024-12-12.1'\n> >>>>>> + FDO_DISTRIBUTION_TAG: '2024-12-12.2'\n> >>>>>>\n> >>>>>> .libcamera-ci.debian:13:\n> >>>>>> variables:\n> >>>>>> @@ -363,28 +363,18 @@ test-soraka:\n> >>>>>> script:\n> >>>>>> - submit .gitlab-ci/lava/soraka-camera-test.yml\n> >>>>>>\n> >>>>>> -# Run the unit tests in a virtual machine. Enable only the options exercised by\n> >>>>>> -# the unit tests.\n> >>>>>> -test-unit:\n> >>>>>> +# Enable only the options exercised by the unit tests.\n> >>>>>> +build-test:debug:\n> >>>>>\n> >>>>> I'd call this build-package:amd64, as we have build-package:arm64 and\n> >>>>> build-package:cros. I think it would also make sense to use the same\n> >>>>> build options for the amd64 and arm64 packages (beside possibly the\n> >>>>> selected pipeline handlers, although the 'auto' option may work for\n> >>>>> both).\n> >>>>>\n> >>>>>> extends:\n> >>>>>> - .fdo.distribution-image@debian\n> >>>>>> - .libcamera-ci.debian:12\n> >>>>>> - .libcamera-ci.scripts\n> >>>>>> - stage: test\n> >>>>>> + stage: build\n> >>>>>> needs:\n> >>>>>> - job: container-debian:12\n> >>>>>> artifacts: false\n> >>>>>> - tags:\n> >>>>>> - - kvm\n> >>>>>> script:\n> >>>>>> - $CI_PROJECT_DIR/.gitlab-ci/build-libcamera.sh\n> >>>>>> - - $CI_PROJECT_DIR/.gitlab-ci/test-libcamera-qemu.sh\n> >>>>>> - artifacts:\n> >>>>>> - name: libcamera-unit-tests-${CI_COMMIT_SHA}\n> >>>>>> - when: always\n> >>>>>> - expire_in: 1 week\n> >>>>>> - paths:\n> >>>>>> - - build/meson-logs/\n> >>>>>> variables:\n> >>>>>> BUILD_TYPE: debug\n> >>>>>> MESON_OPTIONS: >-\n> >>>>>> @@ -399,6 +389,30 @@ test-unit:\n> >>>>>> -D qcam=disabled\n> >>>>>> -D test=true\n> >>>>>> -D v4l2=true\n> >>>>>> + artifacts:\n> >>>>>> + paths:\n> >>>>>> + - build/\n> >>>>>\n> >>>>> The whole build directory can be very large. Can't we do the same as\n> >>>>> build-package:arm64 and run package-libcamera.sh to only package what we\n> >>>>> need ? We'll need probably need an unpackage script for the test-unit\n> >>>>> job.\n> >>>>\n> >>>> But of course the unit test binaries don't get installed... Can we fix\n> >>>> that and install them ? You can specify \"install_tag : 'tests'\" in\n> >>>> meson.build so they won't be installed by default (an appropriate\n> >>>> install_dir is also needed). This in turn requires bumping the minimum\n> >>>> meson version from 0.63.0 to 0.64.0, which shouldn't be an issue.\n> >>>\n> >>> I've been told on IRC that the motivation for the \"tests\" install tag in\n> >>> meson is https://gitlab.gnome.org/GNOME/gnome-desktop-testing. I don't\n> >>> think we should switch to a separate runner for unit tests (the pain is\n> >>> not worth the gain at this point in my opinion), but it could be useful\n> >>> to tag lc-compliance with install_tag = 'tests'.\n> >>>\n> >>>> And now that I've said this, I realize we wouldn't be able to run \"meson\n> >>>> test\" to run the tests :-/ I'm not sure there's an appropriate solution\n> >>>> for this. If not, given the size of the build directory, and to avoid\n> >>>> transferring a large amount of data between runners, we may need to keep\n> >>>> building libcamera within the test-unit job :-(\n> >>>>\n> >>>> A separate build-package target for lc-compliance would still make\n> >>>> sense.\n> >>\n> >> I think it would be unfortunate to give up the usage `meson test` as you\n> >> mentioned.\n> > \n> > We could work on a replacement, but it would require a significant\n> > amount of work and I think there are better things to do.\n> > \n> >> I have not noticed that these build artifacts would put any\n> >> appreciable strain on the infrastructure. The compressed build directory\n> >> comes out to around 167 MiB; I am not sure if I would consider that a\n> >> large amount of data. It is definitely cheaper, in terms of time, than\n> >> building libcamera twice. Clearing the object files could be another\n> >> option. With `artifacts:exclude: build/**/*.o` we can seemingly\n> >> remove more than half of the uncompressed size, and about 1/4 of\n> >> the compressed size. Does this look acceptable?\n> > \n> > Possibly. We should probably ask the fdo sysadmins about what is\n> > acceptable.\n> > \n> > I gave it a try locally though, and deleting all *.o files in the build\n> > directory results in \"meson test\" rebuilding everything.\n> \n> As far as I can see that does not happen with the `--no-rebuild` option,\n> which is already used in `test-libcamera-qemu.sh`.\n> \n> > \n> > For other uses of the artifacts (in particular deployment on real\n> > devices), I would still prefer minimizing the bandwidth, creating a\n> > package similarly to what build-package:arm64 does. How about keeping\n> > test-unit as-is, at the cost of a recompilation, and creating a\n> > build-package:amd64 that will be used by the lc-compliance test job ? We\n> > can try to improve on top when/if needed.\n> \n> Couple observations:\n> \n> 1. The virtual pipeline handler configuration is not installed, so\n> that needs to be addressed. (Was this omitted intentionally?)\n> \n> 2. I am not a fan of the extra `tar` and `ldconfig` calls that\n> need to be sprinkled in. I think this would be much better\n> if the package was not a mere tar archive but a proper deb/etc.\n> package. I imagine that is a prerequisite of deploying on real\n> hardware in any case, correct?\n\nHaving the server build a 'real' deb would be a real benefit IMO, and\nindeed could help with installation/set up on real targets for testing\nin defined environments.\n\nI've wanted to tackle that for a while but never got time.\n\n--\nKieran","headers":{"Return-Path":"","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id 50F46C326C\n\tfor ;\n\tMon, 16 Dec 2024 18:26:20 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id 72EE767F98;\n\tMon, 16 Dec 2024 19:26:19 +0100 (CET)","from perceval.ideasonboard.com (perceval.ideasonboard.com\n\t[IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id E9BBD67F93\n\tfor ;\n\tMon, 16 Dec 2024 19:26:17 +0100 (CET)","from pendragon.ideasonboard.com\n\t(cpc89244-aztw30-2-0-cust6594.18-1.cable.virginm.net [86.31.185.195])\n\tby perceval.ideasonboard.com (Postfix) with ESMTPSA id 0F1A3160;\n\tMon, 16 Dec 2024 19:25:41 +0100 (CET)"],"Authentication-Results":"lancelot.ideasonboard.com; dkim=pass (1024-bit key;\n\tunprotected) header.d=ideasonboard.com header.i=@ideasonboard.com\n\theader.b=\"F3Y7vdUO\"; dkim-atps=neutral","DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com;\n\ts=mail; t=1734373541;\n\tbh=gQb2az+PmqoqzZNvWGTy8WMw04pLeEoAmCeaKZLkpvs=;\n\th=In-Reply-To:References:Subject:From:Cc:To:Date:From;\n\tb=F3Y7vdUOSyUXIBFqdROjPZIqLmtkOtLVEKwh5KQx1GiKBkJMDn6xhpRM2HRvm69fm\n\tG+c5G3W9LNLCoOLcouvGC38ppJ6bk0j0YjHiZwL821JpUf+mXW0qfbvK4aVvEz78V7\n\twH6wBxwDs7/Xvdr5YSDnXDxSYbrIEfO5ZHUhUKRU=","Content-Type":"text/plain; charset=\"utf-8\"","MIME-Version":"1.0","Content-Transfer-Encoding":"quoted-printable","In-Reply-To":"<5c441936-60e0-40e5-b542-a02ccf0739ca@ideasonboard.com>","References":"<20241212181655.112958-1-barnabas.pocze@ideasonboard.com>\n\t<20241212181655.112958-2-barnabas.pocze@ideasonboard.com>\n\t<20241215190408.GL9975@pendragon.ideasonboard.com>\n\t<20241215194320.GN9975@pendragon.ideasonboard.com>\n\t<20241215204350.GP9975@pendragon.ideasonboard.com>\n\t<7948749f-b11a-4397-8332-29c681f188d0@ideasonboard.com>\n\t<20241216110442.GH32204@pendragon.ideasonboard.com>\n\t<5c441936-60e0-40e5-b542-a02ccf0739ca@ideasonboard.com>","Subject":"Re: [libcamera-ci] [RFC PATCH v1 2/3] Separate the building and\n\trunning of unit tests","From":"Kieran Bingham ","Cc":"libcamera-devel@lists.libcamera.org","To":"=?utf-8?q?Barnab=C3=A1s_P=C5=91cze?= ,\n\tLaurent Pinchart ","Date":"Mon, 16 Dec 2024 18:26:14 +0000","Message-ID":"<173437357450.543771.15768411058221148888@ping.linuxembedded.co.uk>","User-Agent":"alot/0.10","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" "}},{"id":32786,"web_url":"https://patchwork.libcamera.org/comment/32786/","msgid":"<20241216200141.GC1304@pendragon.ideasonboard.com>","date":"2024-12-16T20:01:41","subject":"Re: [libcamera-ci] [RFC PATCH v1 2/3] Separate the building and\n\trunning of unit tests","submitter":{"id":2,"url":"https://patchwork.libcamera.org/api/people/2/","name":"Laurent Pinchart","email":"laurent.pinchart@ideasonboard.com"},"content":"On Mon, Dec 16, 2024 at 06:26:14PM +0000, Kieran Bingham wrote:\n> Quoting Barnabás Pőcze (2024-12-16 17:28:46)\n> > 2024. 12. 16. 12:04 keltezéssel, Laurent Pinchart írta:\n> > > On Mon, Dec 16, 2024 at 10:13:54AM +0100, Barnabás Pőcze wrote:\n> > >> 2024. 12. 15. 21:43 keltezéssel, Laurent Pinchart írta:\n> > >>> On Sun, Dec 15, 2024 at 09:43:20PM +0200, Laurent Pinchart wrote:\n> > >>>> On Sun, Dec 15, 2024 at 09:04:08PM +0200, Laurent Pinchart wrote:\n> > >>>>> Hi Barnabás,\n> > >>>>>\n> > >>>>> Thank you for the patch.\n> > >>>>>\n> > >>>>> On Thu, Dec 12, 2024 at 07:16:54PM +0100, Barnabás Pőcze wrote:\n> > >>>>>> The built artifacts will be reused in a later job, so split\n> > >>>>>> the \"test-unit\" into the \"build-test\" and \"test-unit\" jobs.\n> > >>>>>>\n> > >>>>>> The `libevent` development package cannot be installed in the container\n> > >>>>>\n> > >>>>> I've write `libevent-dev` here to avoid ambiguities.\n> > >>>>>\n> > >>>>>> directly because it is not multiarch compatible. It is, however, installed\n> > >>>>>> in the architecture specific build jobs, right before building. To ensure\n> > >>>>>> that the it is available for already built executables in different jobs,\n> > >>>>>\n> > >>>>> \"that the it is\" ?\n> > >>>>>\n> > >>>>>> install just the libraries in the container.\n> > >>>>>\n> > >>>>> And name here `libevent`.\n> > >>>>>\n> > >>>>>>\n> > >>>>>> Signed-off-by: Barnabás Pőcze \n> > >>>>>> ---\n> > >>>>>> .gitlab-ci/setup-container.sh | 3 +++\n> > >>>>>> gitlab-ci.yml | 42 +++++++++++++++++++++++------------\n> > >>>>>> 2 files changed, 31 insertions(+), 14 deletions(-)\n> > >>>>>>\n> > >>>>>> diff --git a/.gitlab-ci/setup-container.sh b/.gitlab-ci/setup-container.sh\n> > >>>>>> index d2909c7..0658368 100755\n> > >>>>>> --- a/.gitlab-ci/setup-container.sh\n> > >>>>>> +++ b/.gitlab-ci/setup-container.sh\n> > >>>>>> @@ -103,6 +103,9 @@ case $FDO_DISTRIBUTION_VERSION in\n> > >>>>>> 'bookworm')\n> > >>>>>> # libclang-rt-dev for the clang ASan runtime.\n> > >>>>>> PKGS_LIBCAMERA_RUNTIME_MULTIARCH+=( libclang-rt-dev )\n> > >>>>>> + # For cam and lc-compliance\n> > >>>>>> + # libevent-dev cannot be used here, see build-libcamera-common.sh\n> > >>>>>> + PKGS_LIBCAMERA_RUNTIME_MULTIARCH+=( libevent-2.1-7 libevent-pthreads-2.1-7 )\n> > >>>>>> ;;\n> > >>>>>> 'trixie')\n> > >>>>>> # gcc 13 to expand compilation testing coverage.\n> > >>>>>> diff --git a/gitlab-ci.yml b/gitlab-ci.yml\n> > >>>>>> index 8bc8bc2..c7448b8 100644\n> > >>>>>> --- a/gitlab-ci.yml\n> > >>>>>> +++ b/gitlab-ci.yml\n> > >>>>>> @@ -64,7 +64,7 @@ include:\n> > >>>>>> .libcamera-ci.debian:12:\n> > >>>>>> variables:\n> > >>>>>> FDO_DISTRIBUTION_VERSION: 'bookworm'\n> > >>>>>> - FDO_DISTRIBUTION_TAG: '2024-12-12.1'\n> > >>>>>> + FDO_DISTRIBUTION_TAG: '2024-12-12.2'\n> > >>>>>>\n> > >>>>>> .libcamera-ci.debian:13:\n> > >>>>>> variables:\n> > >>>>>> @@ -363,28 +363,18 @@ test-soraka:\n> > >>>>>> script:\n> > >>>>>> - submit .gitlab-ci/lava/soraka-camera-test.yml\n> > >>>>>>\n> > >>>>>> -# Run the unit tests in a virtual machine. Enable only the options exercised by\n> > >>>>>> -# the unit tests.\n> > >>>>>> -test-unit:\n> > >>>>>> +# Enable only the options exercised by the unit tests.\n> > >>>>>> +build-test:debug:\n> > >>>>>\n> > >>>>> I'd call this build-package:amd64, as we have build-package:arm64 and\n> > >>>>> build-package:cros. I think it would also make sense to use the same\n> > >>>>> build options for the amd64 and arm64 packages (beside possibly the\n> > >>>>> selected pipeline handlers, although the 'auto' option may work for\n> > >>>>> both).\n> > >>>>>\n> > >>>>>> extends:\n> > >>>>>> - .fdo.distribution-image@debian\n> > >>>>>> - .libcamera-ci.debian:12\n> > >>>>>> - .libcamera-ci.scripts\n> > >>>>>> - stage: test\n> > >>>>>> + stage: build\n> > >>>>>> needs:\n> > >>>>>> - job: container-debian:12\n> > >>>>>> artifacts: false\n> > >>>>>> - tags:\n> > >>>>>> - - kvm\n> > >>>>>> script:\n> > >>>>>> - $CI_PROJECT_DIR/.gitlab-ci/build-libcamera.sh\n> > >>>>>> - - $CI_PROJECT_DIR/.gitlab-ci/test-libcamera-qemu.sh\n> > >>>>>> - artifacts:\n> > >>>>>> - name: libcamera-unit-tests-${CI_COMMIT_SHA}\n> > >>>>>> - when: always\n> > >>>>>> - expire_in: 1 week\n> > >>>>>> - paths:\n> > >>>>>> - - build/meson-logs/\n> > >>>>>> variables:\n> > >>>>>> BUILD_TYPE: debug\n> > >>>>>> MESON_OPTIONS: >-\n> > >>>>>> @@ -399,6 +389,30 @@ test-unit:\n> > >>>>>> -D qcam=disabled\n> > >>>>>> -D test=true\n> > >>>>>> -D v4l2=true\n> > >>>>>> + artifacts:\n> > >>>>>> + paths:\n> > >>>>>> + - build/\n> > >>>>>\n> > >>>>> The whole build directory can be very large. Can't we do the same as\n> > >>>>> build-package:arm64 and run package-libcamera.sh to only package what we\n> > >>>>> need ? We'll need probably need an unpackage script for the test-unit\n> > >>>>> job.\n> > >>>>\n> > >>>> But of course the unit test binaries don't get installed... Can we fix\n> > >>>> that and install them ? You can specify \"install_tag : 'tests'\" in\n> > >>>> meson.build so they won't be installed by default (an appropriate\n> > >>>> install_dir is also needed). This in turn requires bumping the minimum\n> > >>>> meson version from 0.63.0 to 0.64.0, which shouldn't be an issue.\n> > >>>\n> > >>> I've been told on IRC that the motivation for the \"tests\" install tag in\n> > >>> meson is https://gitlab.gnome.org/GNOME/gnome-desktop-testing. I don't\n> > >>> think we should switch to a separate runner for unit tests (the pain is\n> > >>> not worth the gain at this point in my opinion), but it could be useful\n> > >>> to tag lc-compliance with install_tag = 'tests'.\n> > >>>\n> > >>>> And now that I've said this, I realize we wouldn't be able to run \"meson\n> > >>>> test\" to run the tests :-/ I'm not sure there's an appropriate solution\n> > >>>> for this. If not, given the size of the build directory, and to avoid\n> > >>>> transferring a large amount of data between runners, we may need to keep\n> > >>>> building libcamera within the test-unit job :-(\n> > >>>>\n> > >>>> A separate build-package target for lc-compliance would still make\n> > >>>> sense.\n> > >>\n> > >> I think it would be unfortunate to give up the usage `meson test` as you\n> > >> mentioned.\n> > > \n> > > We could work on a replacement, but it would require a significant\n> > > amount of work and I think there are better things to do.\n> > > \n> > >> I have not noticed that these build artifacts would put any\n> > >> appreciable strain on the infrastructure. The compressed build directory\n> > >> comes out to around 167 MiB; I am not sure if I would consider that a\n> > >> large amount of data. It is definitely cheaper, in terms of time, than\n> > >> building libcamera twice. Clearing the object files could be another\n> > >> option. With `artifacts:exclude: build/**/*.o` we can seemingly\n> > >> remove more than half of the uncompressed size, and about 1/4 of\n> > >> the compressed size. Does this look acceptable?\n> > > \n> > > Possibly. We should probably ask the fdo sysadmins about what is\n> > > acceptable.\n> > > \n> > > I gave it a try locally though, and deleting all *.o files in the build\n> > > directory results in \"meson test\" rebuilding everything.\n> > \n> > As far as I can see that does not happen with the `--no-rebuild` option,\n> > which is already used in `test-libcamera-qemu.sh`.\n\nAh, good point, I missed that.\n\n> > > For other uses of the artifacts (in particular deployment on real\n> > > devices), I would still prefer minimizing the bandwidth, creating a\n> > > package similarly to what build-package:arm64 does. How about keeping\n> > > test-unit as-is, at the cost of a recompilation, and creating a\n> > > build-package:amd64 that will be used by the lc-compliance test job ? We\n> > > can try to improve on top when/if needed.\n> > \n> > Couple observations:\n> > \n> > 1. The virtual pipeline handler configuration is not installed, so\n> > that needs to be addressed. (Was this omitted intentionally?)\n\nI don't know. We have to be careful here, we don't want to virtual\npipeline handler to end up being enabled with a default valid\nconfiguration in distributions, otherwise people will all of a sudden\nsee virtual cameras poluting their devices list.\n\n> > 2. I am not a fan of the extra `tar` and `ldconfig` calls that\n> > need to be sprinkled in. I think this would be much better\n> > if the package was not a mere tar archive but a proper deb/etc.\n> > package. I imagine that is a prerequisite of deploying on real\n> > hardware in any case, correct?\n> \n> Having the server build a 'real' deb would be a real benefit IMO, and\n> indeed could help with installation/set up on real targets for testing\n> in defined environments.\n> \n> I've wanted to tackle that for a while but never got time.\n\nSame, I think it's probably worth it, but it will require some effort\nand I didn't have enough time when I implemented build-package:arm64.","headers":{"Return-Path":"","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id 49FEEC32F6\n\tfor ;\n\tMon, 16 Dec 2024 20:02:00 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id 66D2967FAC;\n\tMon, 16 Dec 2024 21:01:59 +0100 (CET)","from perceval.ideasonboard.com (perceval.ideasonboard.com\n\t[213.167.242.64])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id 08E0B62C8B\n\tfor ;\n\tMon, 16 Dec 2024 21:01:58 +0100 (CET)","from pendragon.ideasonboard.com (81-175-209-231.bb.dnainternet.fi\n\t[81.175.209.231])\n\tby perceval.ideasonboard.com (Postfix) with ESMTPSA id F3CE7675;\n\tMon, 16 Dec 2024 21:01:20 +0100 (CET)"],"Authentication-Results":"lancelot.ideasonboard.com; dkim=pass (1024-bit key;\n\tunprotected) header.d=ideasonboard.com header.i=@ideasonboard.com\n\theader.b=\"D0YFpmlr\"; dkim-atps=neutral","DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com;\n\ts=mail; t=1734379281;\n\tbh=DKqufLG5Wv6ge9yawhQcKhJFXPYcnpRACMozEYF/fY0=;\n\th=Date:From:To:Cc:Subject:References:In-Reply-To:From;\n\tb=D0YFpmlrCQgKDr5zq68G/SGjlUgwhsoQXsTxcMMNtVB41yFiXsQeVa0uGiUpxI7MV\n\tBah5R6P4vQwkW/Scm7Bdnnl196w+0yoCh4pYzY1Fz0uIbdzbDmfxUGwerl4g+jw+WW\n\tPoACUk/YhqGAq598cGFHckHvKQPaE4bbcSVLkNn0=","Date":"Mon, 16 Dec 2024 22:01:41 +0200","From":"Laurent Pinchart ","To":"Kieran Bingham ","Cc":"=?utf-8?q?Barnab=C3=A1s_P=C5=91cze?= ,\n\tlibcamera-devel@lists.libcamera.org","Subject":"Re: [libcamera-ci] [RFC PATCH v1 2/3] Separate the building and\n\trunning of unit tests","Message-ID":"<20241216200141.GC1304@pendragon.ideasonboard.com>","References":"<20241212181655.112958-1-barnabas.pocze@ideasonboard.com>\n\t<20241212181655.112958-2-barnabas.pocze@ideasonboard.com>\n\t<20241215190408.GL9975@pendragon.ideasonboard.com>\n\t<20241215194320.GN9975@pendragon.ideasonboard.com>\n\t<20241215204350.GP9975@pendragon.ideasonboard.com>\n\t<7948749f-b11a-4397-8332-29c681f188d0@ideasonboard.com>\n\t<20241216110442.GH32204@pendragon.ideasonboard.com>\n\t<5c441936-60e0-40e5-b542-a02ccf0739ca@ideasonboard.com>\n\t<173437357450.543771.15768411058221148888@ping.linuxembedded.co.uk>","MIME-Version":"1.0","Content-Type":"text/plain; charset=utf-8","Content-Disposition":"inline","Content-Transfer-Encoding":"8bit","In-Reply-To":"<173437357450.543771.15768411058221148888@ping.linuxembedded.co.uk>","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" "}},{"id":32824,"web_url":"https://patchwork.libcamera.org/comment/32824/","msgid":"<2d23688e-7770-4a44-9495-ec31363fb769@ideasonboard.com>","date":"2024-12-17T16:09:45","subject":"Re: [libcamera-ci] [RFC PATCH v1 2/3] Separate the building and\n\trunning of unit tests","submitter":{"id":216,"url":"https://patchwork.libcamera.org/api/people/216/","name":"Barnabás Pőcze","email":"barnabas.pocze@ideasonboard.com"},"content":"2024. 12. 16. 21:01 keltezéssel, Laurent Pinchart írta:\n> On Mon, Dec 16, 2024 at 06:26:14PM +0000, Kieran Bingham wrote:\n>> Quoting Barnabás Pőcze (2024-12-16 17:28:46)\n>>> 2024. 12. 16. 12:04 keltezéssel, Laurent Pinchart írta:\n>>>> On Mon, Dec 16, 2024 at 10:13:54AM +0100, Barnabás Pőcze wrote:\n>>>>> 2024. 12. 15. 21:43 keltezéssel, Laurent Pinchart írta:\n>>>>>> On Sun, Dec 15, 2024 at 09:43:20PM +0200, Laurent Pinchart wrote:\n>>>>>>> On Sun, Dec 15, 2024 at 09:04:08PM +0200, Laurent Pinchart wrote:\n>>>>>>>> Hi Barnabás,\n>>>>>>>>\n>>>>>>>> Thank you for the patch.\n>>>>>>>>\n>>>>>>>> On Thu, Dec 12, 2024 at 07:16:54PM +0100, Barnabás Pőcze wrote:\n>>>>>>>>> The built artifacts will be reused in a later job, so split\n>>>>>>>>> the \"test-unit\" into the \"build-test\" and \"test-unit\" jobs.\n>>>>>>>>>\n>>>>>>>>> The `libevent` development package cannot be installed in the container\n>>>>>>>>\n>>>>>>>> I've write `libevent-dev` here to avoid ambiguities.\n>>>>>>>>\n>>>>>>>>> directly because it is not multiarch compatible. It is, however, installed\n>>>>>>>>> in the architecture specific build jobs, right before building. To ensure\n>>>>>>>>> that the it is available for already built executables in different jobs,\n>>>>>>>>\n>>>>>>>> \"that the it is\" ?\n>>>>>>>>\n>>>>>>>>> install just the libraries in the container.\n>>>>>>>>\n>>>>>>>> And name here `libevent`.\n>>>>>>>>\n>>>>>>>>>\n>>>>>>>>> Signed-off-by: Barnabás Pőcze \n>>>>>>>>> ---\n>>>>>>>>> .gitlab-ci/setup-container.sh | 3 +++\n>>>>>>>>> gitlab-ci.yml | 42 +++++++++++++++++++++++------------\n>>>>>>>>> 2 files changed, 31 insertions(+), 14 deletions(-)\n>>>>>>>>>\n>>>>>>>>> diff --git a/.gitlab-ci/setup-container.sh b/.gitlab-ci/setup-container.sh\n>>>>>>>>> index d2909c7..0658368 100755\n>>>>>>>>> --- a/.gitlab-ci/setup-container.sh\n>>>>>>>>> +++ b/.gitlab-ci/setup-container.sh\n>>>>>>>>> @@ -103,6 +103,9 @@ case $FDO_DISTRIBUTION_VERSION in\n>>>>>>>>> 'bookworm')\n>>>>>>>>> # libclang-rt-dev for the clang ASan runtime.\n>>>>>>>>> PKGS_LIBCAMERA_RUNTIME_MULTIARCH+=( libclang-rt-dev )\n>>>>>>>>> + # For cam and lc-compliance\n>>>>>>>>> + # libevent-dev cannot be used here, see build-libcamera-common.sh\n>>>>>>>>> + PKGS_LIBCAMERA_RUNTIME_MULTIARCH+=( libevent-2.1-7 libevent-pthreads-2.1-7 )\n>>>>>>>>> ;;\n>>>>>>>>> 'trixie')\n>>>>>>>>> # gcc 13 to expand compilation testing coverage.\n>>>>>>>>> diff --git a/gitlab-ci.yml b/gitlab-ci.yml\n>>>>>>>>> index 8bc8bc2..c7448b8 100644\n>>>>>>>>> --- a/gitlab-ci.yml\n>>>>>>>>> +++ b/gitlab-ci.yml\n>>>>>>>>> @@ -64,7 +64,7 @@ include:\n>>>>>>>>> .libcamera-ci.debian:12:\n>>>>>>>>> variables:\n>>>>>>>>> FDO_DISTRIBUTION_VERSION: 'bookworm'\n>>>>>>>>> - FDO_DISTRIBUTION_TAG: '2024-12-12.1'\n>>>>>>>>> + FDO_DISTRIBUTION_TAG: '2024-12-12.2'\n>>>>>>>>>\n>>>>>>>>> .libcamera-ci.debian:13:\n>>>>>>>>> variables:\n>>>>>>>>> @@ -363,28 +363,18 @@ test-soraka:\n>>>>>>>>> script:\n>>>>>>>>> - submit .gitlab-ci/lava/soraka-camera-test.yml\n>>>>>>>>>\n>>>>>>>>> -# Run the unit tests in a virtual machine. Enable only the options exercised by\n>>>>>>>>> -# the unit tests.\n>>>>>>>>> -test-unit:\n>>>>>>>>> +# Enable only the options exercised by the unit tests.\n>>>>>>>>> +build-test:debug:\n>>>>>>>>\n>>>>>>>> I'd call this build-package:amd64, as we have build-package:arm64 and\n>>>>>>>> build-package:cros. I think it would also make sense to use the same\n>>>>>>>> build options for the amd64 and arm64 packages (beside possibly the\n>>>>>>>> selected pipeline handlers, although the 'auto' option may work for\n>>>>>>>> both).\n>>>>>>>>\n>>>>>>>>> extends:\n>>>>>>>>> - .fdo.distribution-image@debian\n>>>>>>>>> - .libcamera-ci.debian:12\n>>>>>>>>> - .libcamera-ci.scripts\n>>>>>>>>> - stage: test\n>>>>>>>>> + stage: build\n>>>>>>>>> needs:\n>>>>>>>>> - job: container-debian:12\n>>>>>>>>> artifacts: false\n>>>>>>>>> - tags:\n>>>>>>>>> - - kvm\n>>>>>>>>> script:\n>>>>>>>>> - $CI_PROJECT_DIR/.gitlab-ci/build-libcamera.sh\n>>>>>>>>> - - $CI_PROJECT_DIR/.gitlab-ci/test-libcamera-qemu.sh\n>>>>>>>>> - artifacts:\n>>>>>>>>> - name: libcamera-unit-tests-${CI_COMMIT_SHA}\n>>>>>>>>> - when: always\n>>>>>>>>> - expire_in: 1 week\n>>>>>>>>> - paths:\n>>>>>>>>> - - build/meson-logs/\n>>>>>>>>> variables:\n>>>>>>>>> BUILD_TYPE: debug\n>>>>>>>>> MESON_OPTIONS: >-\n>>>>>>>>> @@ -399,6 +389,30 @@ test-unit:\n>>>>>>>>> -D qcam=disabled\n>>>>>>>>> -D test=true\n>>>>>>>>> -D v4l2=true\n>>>>>>>>> + artifacts:\n>>>>>>>>> + paths:\n>>>>>>>>> + - build/\n>>>>>>>>\n>>>>>>>> The whole build directory can be very large. Can't we do the same as\n>>>>>>>> build-package:arm64 and run package-libcamera.sh to only package what we\n>>>>>>>> need ? We'll need probably need an unpackage script for the test-unit\n>>>>>>>> job.\n>>>>>>>\n>>>>>>> But of course the unit test binaries don't get installed... Can we fix\n>>>>>>> that and install them ? You can specify \"install_tag : 'tests'\" in\n>>>>>>> meson.build so they won't be installed by default (an appropriate\n>>>>>>> install_dir is also needed). This in turn requires bumping the minimum\n>>>>>>> meson version from 0.63.0 to 0.64.0, which shouldn't be an issue.\n>>>>>>\n>>>>>> I've been told on IRC that the motivation for the \"tests\" install tag in\n>>>>>> meson is https://gitlab.gnome.org/GNOME/gnome-desktop-testing. I don't\n>>>>>> think we should switch to a separate runner for unit tests (the pain is\n>>>>>> not worth the gain at this point in my opinion), but it could be useful\n>>>>>> to tag lc-compliance with install_tag = 'tests'.\n>>>>>>\n>>>>>>> And now that I've said this, I realize we wouldn't be able to run \"meson\n>>>>>>> test\" to run the tests :-/ I'm not sure there's an appropriate solution\n>>>>>>> for this. If not, given the size of the build directory, and to avoid\n>>>>>>> transferring a large amount of data between runners, we may need to keep\n>>>>>>> building libcamera within the test-unit job :-(\n>>>>>>>\n>>>>>>> A separate build-package target for lc-compliance would still make\n>>>>>>> sense.\n>>>>>\n>>>>> I think it would be unfortunate to give up the usage `meson test` as you\n>>>>> mentioned.\n>>>>\n>>>> We could work on a replacement, but it would require a significant\n>>>> amount of work and I think there are better things to do.\n>>>>\n>>>>> I have not noticed that these build artifacts would put any\n>>>>> appreciable strain on the infrastructure. The compressed build directory\n>>>>> comes out to around 167 MiB; I am not sure if I would consider that a\n>>>>> large amount of data. It is definitely cheaper, in terms of time, than\n>>>>> building libcamera twice. Clearing the object files could be another\n>>>>> option. With `artifacts:exclude: build/**/*.o` we can seemingly\n>>>>> remove more than half of the uncompressed size, and about 1/4 of\n>>>>> the compressed size. Does this look acceptable?\n>>>>\n>>>> Possibly. We should probably ask the fdo sysadmins about what is\n>>>> acceptable.\n>>>>\n>>>> I gave it a try locally though, and deleting all *.o files in the build\n>>>> directory results in \"meson test\" rebuilding everything.\n>>>\n>>> As far as I can see that does not happen with the `--no-rebuild` option,\n>>> which is already used in `test-libcamera-qemu.sh`.\n> \n> Ah, good point, I missed that.\n> \n>>>> For other uses of the artifacts (in particular deployment on real\n>>>> devices), I would still prefer minimizing the bandwidth, creating a\n>>>> package similarly to what build-package:arm64 does. How about keeping\n>>>> test-unit as-is, at the cost of a recompilation, and creating a\n>>>> build-package:amd64 that will be used by the lc-compliance test job ? We\n>>>> can try to improve on top when/if needed.\n>>>\n>>> Couple observations:\n>>>\n>>> 1. The virtual pipeline handler configuration is not installed, so\n>>> that needs to be addressed. (Was this omitted intentionally?)\n> \n> I don't know. We have to be careful here, we don't want to virtual\n> pipeline handler to end up being enabled with a default valid\n> configuration in distributions, otherwise people will all of a sudden\n> see virtual cameras poluting their devices list.\n\nI am not sure what should be done. Maybe a separate meson `install_tag`.\nOr I suppose the configuration file could be created right before\nrunning it?\n\n\n> \n>>> 2. I am not a fan of the extra `tar` and `ldconfig` calls that\n>>> need to be sprinkled in. I think this would be much better\n>>> if the package was not a mere tar archive but a proper deb/etc.\n>>> package. I imagine that is a prerequisite of deploying on real\n>>> hardware in any case, correct?\n>>\n>> Having the server build a 'real' deb would be a real benefit IMO, and\n>> indeed could help with installation/set up on real targets for testing\n>> in defined environments.\n>>\n>> I've wanted to tackle that for a while but never got time.\n> \n> Same, I think it's probably worth it, but it will require some effort\n> and I didn't have enough time when I implemented build-package:arm64.\n>","headers":{"Return-Path":"","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id 8B3A7C32F6\n\tfor ;\n\tTue, 17 Dec 2024 16:09:51 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id 4528667FE5;\n\tTue, 17 Dec 2024 17:09:51 +0100 (CET)","from perceval.ideasonboard.com (perceval.ideasonboard.com\n\t[213.167.242.64])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id BA0C467FDD\n\tfor ;\n\tTue, 17 Dec 2024 17:09:49 +0100 (CET)","from [192.168.33.28] (185.221.140.157.nat.pool.zt.hu\n\t[185.221.140.157])\n\tby perceval.ideasonboard.com (Postfix) with ESMTPSA id 625614C7;\n\tTue, 17 Dec 2024 17:09:12 +0100 (CET)"],"Authentication-Results":"lancelot.ideasonboard.com; dkim=pass (1024-bit key;\n\tunprotected) header.d=ideasonboard.com header.i=@ideasonboard.com\n\theader.b=\"f78R2v8U\"; dkim-atps=neutral","DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com;\n\ts=mail; t=1734451752;\n\tbh=/AYOff/0cr7n4I663UW/P6rECBqWQLyUUOLaPAlXqIY=;\n\th=Date:Subject:To:Cc:References:From:In-Reply-To:From;\n\tb=f78R2v8Uk/JHfioolYpB4XsAR1Tyh+hDTMp/r9EeVSE156GLfYPYZpc/GAteDDsBE\n\tYxS70JBNxGkNWGzaM8f6tVALA3ghYYeE/ZLm0srzPdzXZUV2O7Ysc/ZT3NCpt0Vmhi\n\tbBlPyCMxQCfvNMzUNgvmxI89JV06AKR1uQ/AznIs=","Message-ID":"<2d23688e-7770-4a44-9495-ec31363fb769@ideasonboard.com>","Date":"Tue, 17 Dec 2024 17:09:45 +0100","MIME-Version":"1.0","User-Agent":"Mozilla Thunderbird","Subject":"Re: [libcamera-ci] [RFC PATCH v1 2/3] Separate the building and\n\trunning of unit tests","To":"Laurent Pinchart ,\n\tKieran Bingham ","Cc":"libcamera-devel@lists.libcamera.org","References":"<20241212181655.112958-1-barnabas.pocze@ideasonboard.com>\n\t<20241212181655.112958-2-barnabas.pocze@ideasonboard.com>\n\t<20241215190408.GL9975@pendragon.ideasonboard.com>\n\t<20241215194320.GN9975@pendragon.ideasonboard.com>\n\t<20241215204350.GP9975@pendragon.ideasonboard.com>\n\t<7948749f-b11a-4397-8332-29c681f188d0@ideasonboard.com>\n\t<20241216110442.GH32204@pendragon.ideasonboard.com>\n\t<5c441936-60e0-40e5-b542-a02ccf0739ca@ideasonboard.com>\n\t<173437357450.543771.15768411058221148888@ping.linuxembedded.co.uk>\n\t<20241216200141.GC1304@pendragon.ideasonboard.com>","Content-Language":"en-US, hu-HU","From":"=?utf-8?q?Barnab=C3=A1s_P=C5=91cze?= ","In-Reply-To":"<20241216200141.GC1304@pendragon.ideasonboard.com>","Content-Type":"text/plain; charset=UTF-8; format=flowed","Content-Transfer-Encoding":"8bit","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" "}},{"id":32843,"web_url":"https://patchwork.libcamera.org/comment/32843/","msgid":"<20241217172459.GM20432@pendragon.ideasonboard.com>","date":"2024-12-17T17:24:59","subject":"Re: [libcamera-ci] [RFC PATCH v1 2/3] Separate the building and\n\trunning of unit tests","submitter":{"id":2,"url":"https://patchwork.libcamera.org/api/people/2/","name":"Laurent Pinchart","email":"laurent.pinchart@ideasonboard.com"},"content":"Hi Barnabás,\n\nOn Tue, Dec 17, 2024 at 05:09:45PM +0100, Barnabás Pőcze wrote:\n> 2024. 12. 16. 21:01 keltezéssel, Laurent Pinchart írta:\n> > On Mon, Dec 16, 2024 at 06:26:14PM +0000, Kieran Bingham wrote:\n> >> Quoting Barnabás Pőcze (2024-12-16 17:28:46)\n> >>> 2024. 12. 16. 12:04 keltezéssel, Laurent Pinchart írta:\n> >>>> On Mon, Dec 16, 2024 at 10:13:54AM +0100, Barnabás Pőcze wrote:\n> >>>>> 2024. 12. 15. 21:43 keltezéssel, Laurent Pinchart írta:\n> >>>>>> On Sun, Dec 15, 2024 at 09:43:20PM +0200, Laurent Pinchart wrote:\n> >>>>>>> On Sun, Dec 15, 2024 at 09:04:08PM +0200, Laurent Pinchart wrote:\n> >>>>>>>> On Thu, Dec 12, 2024 at 07:16:54PM +0100, Barnabás Pőcze wrote:\n> >>>>>>>>> The built artifacts will be reused in a later job, so split\n> >>>>>>>>> the \"test-unit\" into the \"build-test\" and \"test-unit\" jobs.\n> >>>>>>>>>\n> >>>>>>>>> The `libevent` development package cannot be installed in the container\n> >>>>>>>>\n> >>>>>>>> I've write `libevent-dev` here to avoid ambiguities.\n> >>>>>>>>\n> >>>>>>>>> directly because it is not multiarch compatible. It is, however, installed\n> >>>>>>>>> in the architecture specific build jobs, right before building. To ensure\n> >>>>>>>>> that the it is available for already built executables in different jobs,\n> >>>>>>>>\n> >>>>>>>> \"that the it is\" ?\n> >>>>>>>>\n> >>>>>>>>> install just the libraries in the container.\n> >>>>>>>>\n> >>>>>>>> And name here `libevent`.\n> >>>>>>>>\n> >>>>>>>>>\n> >>>>>>>>> Signed-off-by: Barnabás Pőcze \n> >>>>>>>>> ---\n> >>>>>>>>> .gitlab-ci/setup-container.sh | 3 +++\n> >>>>>>>>> gitlab-ci.yml | 42 +++++++++++++++++++++++------------\n> >>>>>>>>> 2 files changed, 31 insertions(+), 14 deletions(-)\n> >>>>>>>>>\n> >>>>>>>>> diff --git a/.gitlab-ci/setup-container.sh b/.gitlab-ci/setup-container.sh\n> >>>>>>>>> index d2909c7..0658368 100755\n> >>>>>>>>> --- a/.gitlab-ci/setup-container.sh\n> >>>>>>>>> +++ b/.gitlab-ci/setup-container.sh\n> >>>>>>>>> @@ -103,6 +103,9 @@ case $FDO_DISTRIBUTION_VERSION in\n> >>>>>>>>> 'bookworm')\n> >>>>>>>>> # libclang-rt-dev for the clang ASan runtime.\n> >>>>>>>>> PKGS_LIBCAMERA_RUNTIME_MULTIARCH+=( libclang-rt-dev )\n> >>>>>>>>> + # For cam and lc-compliance\n> >>>>>>>>> + # libevent-dev cannot be used here, see build-libcamera-common.sh\n> >>>>>>>>> + PKGS_LIBCAMERA_RUNTIME_MULTIARCH+=( libevent-2.1-7 libevent-pthreads-2.1-7 )\n> >>>>>>>>> ;;\n> >>>>>>>>> 'trixie')\n> >>>>>>>>> # gcc 13 to expand compilation testing coverage.\n> >>>>>>>>> diff --git a/gitlab-ci.yml b/gitlab-ci.yml\n> >>>>>>>>> index 8bc8bc2..c7448b8 100644\n> >>>>>>>>> --- a/gitlab-ci.yml\n> >>>>>>>>> +++ b/gitlab-ci.yml\n> >>>>>>>>> @@ -64,7 +64,7 @@ include:\n> >>>>>>>>> .libcamera-ci.debian:12:\n> >>>>>>>>> variables:\n> >>>>>>>>> FDO_DISTRIBUTION_VERSION: 'bookworm'\n> >>>>>>>>> - FDO_DISTRIBUTION_TAG: '2024-12-12.1'\n> >>>>>>>>> + FDO_DISTRIBUTION_TAG: '2024-12-12.2'\n> >>>>>>>>>\n> >>>>>>>>> .libcamera-ci.debian:13:\n> >>>>>>>>> variables:\n> >>>>>>>>> @@ -363,28 +363,18 @@ test-soraka:\n> >>>>>>>>> script:\n> >>>>>>>>> - submit .gitlab-ci/lava/soraka-camera-test.yml\n> >>>>>>>>>\n> >>>>>>>>> -# Run the unit tests in a virtual machine. Enable only the options exercised by\n> >>>>>>>>> -# the unit tests.\n> >>>>>>>>> -test-unit:\n> >>>>>>>>> +# Enable only the options exercised by the unit tests.\n> >>>>>>>>> +build-test:debug:\n> >>>>>>>>\n> >>>>>>>> I'd call this build-package:amd64, as we have build-package:arm64 and\n> >>>>>>>> build-package:cros. I think it would also make sense to use the same\n> >>>>>>>> build options for the amd64 and arm64 packages (beside possibly the\n> >>>>>>>> selected pipeline handlers, although the 'auto' option may work for\n> >>>>>>>> both).\n> >>>>>>>>\n> >>>>>>>>> extends:\n> >>>>>>>>> - .fdo.distribution-image@debian\n> >>>>>>>>> - .libcamera-ci.debian:12\n> >>>>>>>>> - .libcamera-ci.scripts\n> >>>>>>>>> - stage: test\n> >>>>>>>>> + stage: build\n> >>>>>>>>> needs:\n> >>>>>>>>> - job: container-debian:12\n> >>>>>>>>> artifacts: false\n> >>>>>>>>> - tags:\n> >>>>>>>>> - - kvm\n> >>>>>>>>> script:\n> >>>>>>>>> - $CI_PROJECT_DIR/.gitlab-ci/build-libcamera.sh\n> >>>>>>>>> - - $CI_PROJECT_DIR/.gitlab-ci/test-libcamera-qemu.sh\n> >>>>>>>>> - artifacts:\n> >>>>>>>>> - name: libcamera-unit-tests-${CI_COMMIT_SHA}\n> >>>>>>>>> - when: always\n> >>>>>>>>> - expire_in: 1 week\n> >>>>>>>>> - paths:\n> >>>>>>>>> - - build/meson-logs/\n> >>>>>>>>> variables:\n> >>>>>>>>> BUILD_TYPE: debug\n> >>>>>>>>> MESON_OPTIONS: >-\n> >>>>>>>>> @@ -399,6 +389,30 @@ test-unit:\n> >>>>>>>>> -D qcam=disabled\n> >>>>>>>>> -D test=true\n> >>>>>>>>> -D v4l2=true\n> >>>>>>>>> + artifacts:\n> >>>>>>>>> + paths:\n> >>>>>>>>> + - build/\n> >>>>>>>>\n> >>>>>>>> The whole build directory can be very large. Can't we do the same as\n> >>>>>>>> build-package:arm64 and run package-libcamera.sh to only package what we\n> >>>>>>>> need ? We'll need probably need an unpackage script for the test-unit\n> >>>>>>>> job.\n> >>>>>>>\n> >>>>>>> But of course the unit test binaries don't get installed... Can we fix\n> >>>>>>> that and install them ? You can specify \"install_tag : 'tests'\" in\n> >>>>>>> meson.build so they won't be installed by default (an appropriate\n> >>>>>>> install_dir is also needed). This in turn requires bumping the minimum\n> >>>>>>> meson version from 0.63.0 to 0.64.0, which shouldn't be an issue.\n> >>>>>>\n> >>>>>> I've been told on IRC that the motivation for the \"tests\" install tag in\n> >>>>>> meson is https://gitlab.gnome.org/GNOME/gnome-desktop-testing. I don't\n> >>>>>> think we should switch to a separate runner for unit tests (the pain is\n> >>>>>> not worth the gain at this point in my opinion), but it could be useful\n> >>>>>> to tag lc-compliance with install_tag = 'tests'.\n> >>>>>>\n> >>>>>>> And now that I've said this, I realize we wouldn't be able to run \"meson\n> >>>>>>> test\" to run the tests :-/ I'm not sure there's an appropriate solution\n> >>>>>>> for this. If not, given the size of the build directory, and to avoid\n> >>>>>>> transferring a large amount of data between runners, we may need to keep\n> >>>>>>> building libcamera within the test-unit job :-(\n> >>>>>>>\n> >>>>>>> A separate build-package target for lc-compliance would still make\n> >>>>>>> sense.\n> >>>>>\n> >>>>> I think it would be unfortunate to give up the usage `meson test` as you\n> >>>>> mentioned.\n> >>>>\n> >>>> We could work on a replacement, but it would require a significant\n> >>>> amount of work and I think there are better things to do.\n> >>>>\n> >>>>> I have not noticed that these build artifacts would put any\n> >>>>> appreciable strain on the infrastructure. The compressed build directory\n> >>>>> comes out to around 167 MiB; I am not sure if I would consider that a\n> >>>>> large amount of data. It is definitely cheaper, in terms of time, than\n> >>>>> building libcamera twice. Clearing the object files could be another\n> >>>>> option. With `artifacts:exclude: build/**/*.o` we can seemingly\n> >>>>> remove more than half of the uncompressed size, and about 1/4 of\n> >>>>> the compressed size. Does this look acceptable?\n> >>>>\n> >>>> Possibly. We should probably ask the fdo sysadmins about what is\n> >>>> acceptable.\n> >>>>\n> >>>> I gave it a try locally though, and deleting all *.o files in the build\n> >>>> directory results in \"meson test\" rebuilding everything.\n> >>>\n> >>> As far as I can see that does not happen with the `--no-rebuild` option,\n> >>> which is already used in `test-libcamera-qemu.sh`.\n> > \n> > Ah, good point, I missed that.\n> > \n> >>>> For other uses of the artifacts (in particular deployment on real\n> >>>> devices), I would still prefer minimizing the bandwidth, creating a\n> >>>> package similarly to what build-package:arm64 does. How about keeping\n> >>>> test-unit as-is, at the cost of a recompilation, and creating a\n> >>>> build-package:amd64 that will be used by the lc-compliance test job ? We\n> >>>> can try to improve on top when/if needed.\n> >>>\n> >>> Couple observations:\n> >>>\n> >>> 1. The virtual pipeline handler configuration is not installed, so\n> >>> that needs to be addressed. (Was this omitted intentionally?)\n> > \n> > I don't know. We have to be careful here, we don't want to virtual\n> > pipeline handler to end up being enabled with a default valid\n> > configuration in distributions, otherwise people will all of a sudden\n> > see virtual cameras poluting their devices list.\n> \n> I am not sure what should be done. Maybe a separate meson `install_tag`.\n> Or I suppose the configuration file could be created right before\n> running it?\n\nHmmmm... If this is a sample configuration file that we don't want to\ninstall by default in the location where it will be lookup up, how about\ninstalling it as an example in doc_install_dir (e.g.\n/usr/share/doc/libcamera/) and name it virtual.yaml.example ? That seems\nto be a fairly common pattern.\n\nYou can move the doc_install_dir definition from\nDocumentation/meson.build to src/meson.build and rename it to\nlibcamera_docdir.\n\n> >>> 2. I am not a fan of the extra `tar` and `ldconfig` calls that\n> >>> need to be sprinkled in. I think this would be much better\n> >>> if the package was not a mere tar archive but a proper deb/etc.\n> >>> package. I imagine that is a prerequisite of deploying on real\n> >>> hardware in any case, correct?\n> >>\n> >> Having the server build a 'real' deb would be a real benefit IMO, and\n> >> indeed could help with installation/set up on real targets for testing\n> >> in defined environments.\n> >>\n> >> I've wanted to tackle that for a while but never got time.\n> > \n> > Same, I think it's probably worth it, but it will require some effort\n> > and I didn't have enough time when I implemented build-package:arm64.","headers":{"Return-Path":"","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id 4D2C4BD1F1\n\tfor ;\n\tTue, 17 Dec 2024 17:25:05 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id 95DDC67FF3;\n\tTue, 17 Dec 2024 18:25:03 +0100 (CET)","from perceval.ideasonboard.com (perceval.ideasonboard.com\n\t[213.167.242.64])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id 5643867FD3\n\tfor ;\n\tTue, 17 Dec 2024 18:25:01 +0100 (CET)","from pendragon.ideasonboard.com (81-175-209-231.bb.dnainternet.fi\n\t[81.175.209.231])\n\tby perceval.ideasonboard.com (Postfix) with ESMTPSA id 9C85D4C7;\n\tTue, 17 Dec 2024 18:24:23 +0100 (CET)"],"Authentication-Results":"lancelot.ideasonboard.com; dkim=pass (1024-bit key;\n\tunprotected) header.d=ideasonboard.com header.i=@ideasonboard.com\n\theader.b=\"lUR2WXs1\"; dkim-atps=neutral","DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com;\n\ts=mail; t=1734456263;\n\tbh=PZqwLGS0F94eUuCV3oGHEJ8IhMVn6rAN8NstTj4Q4sE=;\n\th=Date:From:To:Cc:Subject:References:In-Reply-To:From;\n\tb=lUR2WXs1NaA96cRlIQUyEhx1HbOvcj5fyBN++ZCQXlZ390zf7atIMxa0qqFN5NdZ9\n\tCb3DPeE7jdkcPwhvJrhPZfTFeLB9ybma6wltoJ+KcBw2iK8Zxgrou93zO/AlhAvGOH\n\tmaAiHc8oLQ6T8zYBk4Qr6S61r5IS8JId5mxOG5Nc=","Date":"Tue, 17 Dec 2024 19:24:59 +0200","From":"Laurent Pinchart ","To":"=?utf-8?q?Barnab=C3=A1s_P=C5=91cze?= ","Cc":"Kieran Bingham ,\n\tlibcamera-devel@lists.libcamera.org","Subject":"Re: [libcamera-ci] [RFC PATCH v1 2/3] Separate the building and\n\trunning of unit tests","Message-ID":"<20241217172459.GM20432@pendragon.ideasonboard.com>","References":"<20241212181655.112958-2-barnabas.pocze@ideasonboard.com>\n\t<20241215190408.GL9975@pendragon.ideasonboard.com>\n\t<20241215194320.GN9975@pendragon.ideasonboard.com>\n\t<20241215204350.GP9975@pendragon.ideasonboard.com>\n\t<7948749f-b11a-4397-8332-29c681f188d0@ideasonboard.com>\n\t<20241216110442.GH32204@pendragon.ideasonboard.com>\n\t<5c441936-60e0-40e5-b542-a02ccf0739ca@ideasonboard.com>\n\t<173437357450.543771.15768411058221148888@ping.linuxembedded.co.uk>\n\t<20241216200141.GC1304@pendragon.ideasonboard.com>\n\t<2d23688e-7770-4a44-9495-ec31363fb769@ideasonboard.com>","MIME-Version":"1.0","Content-Type":"text/plain; charset=utf-8","Content-Disposition":"inline","Content-Transfer-Encoding":"8bit","In-Reply-To":"<2d23688e-7770-4a44-9495-ec31363fb769@ideasonboard.com>","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" "}},{"id":32844,"web_url":"https://patchwork.libcamera.org/comment/32844/","msgid":"<20241217173225.GN20432@pendragon.ideasonboard.com>","date":"2024-12-17T17:32:25","subject":"Re: [libcamera-ci] [RFC PATCH v1 2/3] Separate the building and\n\trunning of unit tests","submitter":{"id":2,"url":"https://patchwork.libcamera.org/api/people/2/","name":"Laurent Pinchart","email":"laurent.pinchart@ideasonboard.com"},"content":"On Tue, Dec 17, 2024 at 07:25:00PM +0200, Laurent Pinchart wrote:\n> On Tue, Dec 17, 2024 at 05:09:45PM +0100, Barnabás Pőcze wrote:\n> > 2024. 12. 16. 21:01 keltezéssel, Laurent Pinchart írta:\n> > > On Mon, Dec 16, 2024 at 06:26:14PM +0000, Kieran Bingham wrote:\n> > >> Quoting Barnabás Pőcze (2024-12-16 17:28:46)\n> > >>> 2024. 12. 16. 12:04 keltezéssel, Laurent Pinchart írta:\n> > >>>> On Mon, Dec 16, 2024 at 10:13:54AM +0100, Barnabás Pőcze wrote:\n> > >>>>> 2024. 12. 15. 21:43 keltezéssel, Laurent Pinchart írta:\n> > >>>>>> On Sun, Dec 15, 2024 at 09:43:20PM +0200, Laurent Pinchart wrote:\n> > >>>>>>> On Sun, Dec 15, 2024 at 09:04:08PM +0200, Laurent Pinchart wrote:\n> > >>>>>>>> On Thu, Dec 12, 2024 at 07:16:54PM +0100, Barnabás Pőcze wrote:\n> > >>>>>>>>> The built artifacts will be reused in a later job, so split\n> > >>>>>>>>> the \"test-unit\" into the \"build-test\" and \"test-unit\" jobs.\n> > >>>>>>>>>\n> > >>>>>>>>> The `libevent` development package cannot be installed in the container\n> > >>>>>>>>\n> > >>>>>>>> I've write `libevent-dev` here to avoid ambiguities.\n> > >>>>>>>>\n> > >>>>>>>>> directly because it is not multiarch compatible. It is, however, installed\n> > >>>>>>>>> in the architecture specific build jobs, right before building. To ensure\n> > >>>>>>>>> that the it is available for already built executables in different jobs,\n> > >>>>>>>>\n> > >>>>>>>> \"that the it is\" ?\n> > >>>>>>>>\n> > >>>>>>>>> install just the libraries in the container.\n> > >>>>>>>>\n> > >>>>>>>> And name here `libevent`.\n> > >>>>>>>>\n> > >>>>>>>>>\n> > >>>>>>>>> Signed-off-by: Barnabás Pőcze \n> > >>>>>>>>> ---\n> > >>>>>>>>> .gitlab-ci/setup-container.sh | 3 +++\n> > >>>>>>>>> gitlab-ci.yml | 42 +++++++++++++++++++++++------------\n> > >>>>>>>>> 2 files changed, 31 insertions(+), 14 deletions(-)\n> > >>>>>>>>>\n> > >>>>>>>>> diff --git a/.gitlab-ci/setup-container.sh b/.gitlab-ci/setup-container.sh\n> > >>>>>>>>> index d2909c7..0658368 100755\n> > >>>>>>>>> --- a/.gitlab-ci/setup-container.sh\n> > >>>>>>>>> +++ b/.gitlab-ci/setup-container.sh\n> > >>>>>>>>> @@ -103,6 +103,9 @@ case $FDO_DISTRIBUTION_VERSION in\n> > >>>>>>>>> 'bookworm')\n> > >>>>>>>>> # libclang-rt-dev for the clang ASan runtime.\n> > >>>>>>>>> PKGS_LIBCAMERA_RUNTIME_MULTIARCH+=( libclang-rt-dev )\n> > >>>>>>>>> + # For cam and lc-compliance\n> > >>>>>>>>> + # libevent-dev cannot be used here, see build-libcamera-common.sh\n> > >>>>>>>>> + PKGS_LIBCAMERA_RUNTIME_MULTIARCH+=( libevent-2.1-7 libevent-pthreads-2.1-7 )\n> > >>>>>>>>> ;;\n> > >>>>>>>>> 'trixie')\n> > >>>>>>>>> # gcc 13 to expand compilation testing coverage.\n> > >>>>>>>>> diff --git a/gitlab-ci.yml b/gitlab-ci.yml\n> > >>>>>>>>> index 8bc8bc2..c7448b8 100644\n> > >>>>>>>>> --- a/gitlab-ci.yml\n> > >>>>>>>>> +++ b/gitlab-ci.yml\n> > >>>>>>>>> @@ -64,7 +64,7 @@ include:\n> > >>>>>>>>> .libcamera-ci.debian:12:\n> > >>>>>>>>> variables:\n> > >>>>>>>>> FDO_DISTRIBUTION_VERSION: 'bookworm'\n> > >>>>>>>>> - FDO_DISTRIBUTION_TAG: '2024-12-12.1'\n> > >>>>>>>>> + FDO_DISTRIBUTION_TAG: '2024-12-12.2'\n> > >>>>>>>>>\n> > >>>>>>>>> .libcamera-ci.debian:13:\n> > >>>>>>>>> variables:\n> > >>>>>>>>> @@ -363,28 +363,18 @@ test-soraka:\n> > >>>>>>>>> script:\n> > >>>>>>>>> - submit .gitlab-ci/lava/soraka-camera-test.yml\n> > >>>>>>>>>\n> > >>>>>>>>> -# Run the unit tests in a virtual machine. Enable only the options exercised by\n> > >>>>>>>>> -# the unit tests.\n> > >>>>>>>>> -test-unit:\n> > >>>>>>>>> +# Enable only the options exercised by the unit tests.\n> > >>>>>>>>> +build-test:debug:\n> > >>>>>>>>\n> > >>>>>>>> I'd call this build-package:amd64, as we have build-package:arm64 and\n> > >>>>>>>> build-package:cros. I think it would also make sense to use the same\n> > >>>>>>>> build options for the amd64 and arm64 packages (beside possibly the\n> > >>>>>>>> selected pipeline handlers, although the 'auto' option may work for\n> > >>>>>>>> both).\n> > >>>>>>>>\n> > >>>>>>>>> extends:\n> > >>>>>>>>> - .fdo.distribution-image@debian\n> > >>>>>>>>> - .libcamera-ci.debian:12\n> > >>>>>>>>> - .libcamera-ci.scripts\n> > >>>>>>>>> - stage: test\n> > >>>>>>>>> + stage: build\n> > >>>>>>>>> needs:\n> > >>>>>>>>> - job: container-debian:12\n> > >>>>>>>>> artifacts: false\n> > >>>>>>>>> - tags:\n> > >>>>>>>>> - - kvm\n> > >>>>>>>>> script:\n> > >>>>>>>>> - $CI_PROJECT_DIR/.gitlab-ci/build-libcamera.sh\n> > >>>>>>>>> - - $CI_PROJECT_DIR/.gitlab-ci/test-libcamera-qemu.sh\n> > >>>>>>>>> - artifacts:\n> > >>>>>>>>> - name: libcamera-unit-tests-${CI_COMMIT_SHA}\n> > >>>>>>>>> - when: always\n> > >>>>>>>>> - expire_in: 1 week\n> > >>>>>>>>> - paths:\n> > >>>>>>>>> - - build/meson-logs/\n> > >>>>>>>>> variables:\n> > >>>>>>>>> BUILD_TYPE: debug\n> > >>>>>>>>> MESON_OPTIONS: >-\n> > >>>>>>>>> @@ -399,6 +389,30 @@ test-unit:\n> > >>>>>>>>> -D qcam=disabled\n> > >>>>>>>>> -D test=true\n> > >>>>>>>>> -D v4l2=true\n> > >>>>>>>>> + artifacts:\n> > >>>>>>>>> + paths:\n> > >>>>>>>>> + - build/\n> > >>>>>>>>\n> > >>>>>>>> The whole build directory can be very large. Can't we do the same as\n> > >>>>>>>> build-package:arm64 and run package-libcamera.sh to only package what we\n> > >>>>>>>> need ? We'll need probably need an unpackage script for the test-unit\n> > >>>>>>>> job.\n> > >>>>>>>\n> > >>>>>>> But of course the unit test binaries don't get installed... Can we fix\n> > >>>>>>> that and install them ? You can specify \"install_tag : 'tests'\" in\n> > >>>>>>> meson.build so they won't be installed by default (an appropriate\n> > >>>>>>> install_dir is also needed). This in turn requires bumping the minimum\n> > >>>>>>> meson version from 0.63.0 to 0.64.0, which shouldn't be an issue.\n> > >>>>>>\n> > >>>>>> I've been told on IRC that the motivation for the \"tests\" install tag in\n> > >>>>>> meson is https://gitlab.gnome.org/GNOME/gnome-desktop-testing. I don't\n> > >>>>>> think we should switch to a separate runner for unit tests (the pain is\n> > >>>>>> not worth the gain at this point in my opinion), but it could be useful\n> > >>>>>> to tag lc-compliance with install_tag = 'tests'.\n> > >>>>>>\n> > >>>>>>> And now that I've said this, I realize we wouldn't be able to run \"meson\n> > >>>>>>> test\" to run the tests :-/ I'm not sure there's an appropriate solution\n> > >>>>>>> for this. If not, given the size of the build directory, and to avoid\n> > >>>>>>> transferring a large amount of data between runners, we may need to keep\n> > >>>>>>> building libcamera within the test-unit job :-(\n> > >>>>>>>\n> > >>>>>>> A separate build-package target for lc-compliance would still make\n> > >>>>>>> sense.\n> > >>>>>\n> > >>>>> I think it would be unfortunate to give up the usage `meson test` as you\n> > >>>>> mentioned.\n> > >>>>\n> > >>>> We could work on a replacement, but it would require a significant\n> > >>>> amount of work and I think there are better things to do.\n> > >>>>\n> > >>>>> I have not noticed that these build artifacts would put any\n> > >>>>> appreciable strain on the infrastructure. The compressed build directory\n> > >>>>> comes out to around 167 MiB; I am not sure if I would consider that a\n> > >>>>> large amount of data. It is definitely cheaper, in terms of time, than\n> > >>>>> building libcamera twice. Clearing the object files could be another\n> > >>>>> option. With `artifacts:exclude: build/**/*.o` we can seemingly\n> > >>>>> remove more than half of the uncompressed size, and about 1/4 of\n> > >>>>> the compressed size. Does this look acceptable?\n> > >>>>\n> > >>>> Possibly. We should probably ask the fdo sysadmins about what is\n> > >>>> acceptable.\n> > >>>>\n> > >>>> I gave it a try locally though, and deleting all *.o files in the build\n> > >>>> directory results in \"meson test\" rebuilding everything.\n> > >>>\n> > >>> As far as I can see that does not happen with the `--no-rebuild` option,\n> > >>> which is already used in `test-libcamera-qemu.sh`.\n> > > \n> > > Ah, good point, I missed that.\n> > > \n> > >>>> For other uses of the artifacts (in particular deployment on real\n> > >>>> devices), I would still prefer minimizing the bandwidth, creating a\n> > >>>> package similarly to what build-package:arm64 does. How about keeping\n> > >>>> test-unit as-is, at the cost of a recompilation, and creating a\n> > >>>> build-package:amd64 that will be used by the lc-compliance test job ? We\n> > >>>> can try to improve on top when/if needed.\n> > >>>\n> > >>> Couple observations:\n> > >>>\n> > >>> 1. The virtual pipeline handler configuration is not installed, so\n> > >>> that needs to be addressed. (Was this omitted intentionally?)\n> > > \n> > > I don't know. We have to be careful here, we don't want to virtual\n> > > pipeline handler to end up being enabled with a default valid\n> > > configuration in distributions, otherwise people will all of a sudden\n> > > see virtual cameras poluting their devices list.\n> > \n> > I am not sure what should be done. Maybe a separate meson `install_tag`.\n> > Or I suppose the configuration file could be created right before\n> > running it?\n> \n> Hmmmm... If this is a sample configuration file that we don't want to\n> install by default in the location where it will be lookup up, how about\n> installing it as an example in doc_install_dir (e.g.\n> /usr/share/doc/libcamera/) and name it virtual.yaml.example ? That seems\n> to be a fairly common pattern.\n\nAnd then in CI we would take that file and copy it to the expected\nlocation, renaming it to virtual.yaml.\n\nThere's also the question of what install_tag to use. Other data files\nuse 'runtime', but those are installed to their runtime location. 'doc'\ncould be an option, we can use that even if we disable documentation\nbuild in CI, but it may feel a bit weird, and possibly later install\nother types of documentation that we way not want (although I suppose\ninstallation of future \"real\" documentation could be conditioned by the\ndocumentation meson option).\n\nAnother option is to create it from scratch in CI (possibly with the\nfile just added to the CI repository and copied in the test job).\n\n> You can move the doc_install_dir definition from\n> Documentation/meson.build to src/meson.build and rename it to\n> libcamera_docdir.\n> \n> > >>> 2. I am not a fan of the extra `tar` and `ldconfig` calls that\n> > >>> need to be sprinkled in. I think this would be much better\n> > >>> if the package was not a mere tar archive but a proper deb/etc.\n> > >>> package. I imagine that is a prerequisite of deploying on real\n> > >>> hardware in any case, correct?\n> > >>\n> > >> Having the server build a 'real' deb would be a real benefit IMO, and\n> > >> indeed could help with installation/set up on real targets for testing\n> > >> in defined environments.\n> > >>\n> > >> I've wanted to tackle that for a while but never got time.\n> > > \n> > > Same, I think it's probably worth it, but it will require some effort\n> > > and I didn't have enough time when I implemented build-package:arm64.","headers":{"Return-Path":"","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id 8D4D4C32F6\n\tfor ;\n\tTue, 17 Dec 2024 17:32:30 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id AD76B67FF3;\n\tTue, 17 Dec 2024 18:32:29 +0100 (CET)","from perceval.ideasonboard.com (perceval.ideasonboard.com\n\t[IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id 6BBCD67FD3\n\tfor ;\n\tTue, 17 Dec 2024 18:32:28 +0100 (CET)","from pendragon.ideasonboard.com (81-175-209-231.bb.dnainternet.fi\n\t[81.175.209.231])\n\tby perceval.ideasonboard.com (Postfix) with ESMTPSA id ABBEB3E;\n\tTue, 17 Dec 2024 18:31:50 +0100 (CET)"],"Authentication-Results":"lancelot.ideasonboard.com; dkim=pass (1024-bit key;\n\tunprotected) header.d=ideasonboard.com header.i=@ideasonboard.com\n\theader.b=\"SJTnzA+Y\"; dkim-atps=neutral","DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com;\n\ts=mail; t=1734456710;\n\tbh=+dnG0P31OGhwcYw1UmhxhRgF1LhEST1gQd2KozkZWYo=;\n\th=Date:From:To:Cc:Subject:References:In-Reply-To:From;\n\tb=SJTnzA+YSCcLoJVEowa7/S2vGT7TgaY+y4p7Gi/c76GHtDDaA5DDoS5mSQQzG584f\n\tVe3EeyFltqWH5EH8MVfnO9pNT4HLdssoC/oYcmndX5ZjESJJ1ZdHVW+Lk9KtAdCXUp\n\thKos3vftvHdfr8l+AkGUBNJrq9S2FawBTc6wDTU8=","Date":"Tue, 17 Dec 2024 19:32:25 +0200","From":"Laurent Pinchart ","To":"=?utf-8?q?Barnab=C3=A1s_P=C5=91cze?= ","Cc":"Kieran Bingham ,\n\tlibcamera-devel@lists.libcamera.org","Subject":"Re: [libcamera-ci] [RFC PATCH v1 2/3] Separate the building and\n\trunning of unit tests","Message-ID":"<20241217173225.GN20432@pendragon.ideasonboard.com>","References":"<20241215190408.GL9975@pendragon.ideasonboard.com>\n\t<20241215194320.GN9975@pendragon.ideasonboard.com>\n\t<20241215204350.GP9975@pendragon.ideasonboard.com>\n\t<7948749f-b11a-4397-8332-29c681f188d0@ideasonboard.com>\n\t<20241216110442.GH32204@pendragon.ideasonboard.com>\n\t<5c441936-60e0-40e5-b542-a02ccf0739ca@ideasonboard.com>\n\t<173437357450.543771.15768411058221148888@ping.linuxembedded.co.uk>\n\t<20241216200141.GC1304@pendragon.ideasonboard.com>\n\t<2d23688e-7770-4a44-9495-ec31363fb769@ideasonboard.com>\n\t<20241217172459.GM20432@pendragon.ideasonboard.com>","MIME-Version":"1.0","Content-Type":"text/plain; charset=utf-8","Content-Disposition":"inline","Content-Transfer-Encoding":"8bit","In-Reply-To":"<20241217172459.GM20432@pendragon.ideasonboard.com>","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" "}},{"id":32845,"web_url":"https://patchwork.libcamera.org/comment/32845/","msgid":"<173445708372.4145007.5052263089528641829@ping.linuxembedded.co.uk>","date":"2024-12-17T17:38:03","subject":"Re: [libcamera-ci] [RFC PATCH v1 2/3] Separate the building and\n\trunning of unit tests","submitter":{"id":4,"url":"https://patchwork.libcamera.org/api/people/4/","name":"Kieran Bingham","email":"kieran.bingham@ideasonboard.com"},"content":"Quoting Laurent Pinchart (2024-12-17 17:24:59)\n> Hi Barnabás,\n> \n> On Tue, Dec 17, 2024 at 05:09:45PM +0100, Barnabás Pőcze wrote:\n> > 2024. 12. 16. 21:01 keltezéssel, Laurent Pinchart írta:\n> > > On Mon, Dec 16, 2024 at 06:26:14PM +0000, Kieran Bingham wrote:\n> > >> Quoting Barnabás Pőcze (2024-12-16 17:28:46)\n> > >>> 2024. 12. 16. 12:04 keltezéssel, Laurent Pinchart írta:\n> > >>>> On Mon, Dec 16, 2024 at 10:13:54AM +0100, Barnabás Pőcze wrote:\n> > >>>>> 2024. 12. 15. 21:43 keltezéssel, Laurent Pinchart írta:\n> > >>>>>> On Sun, Dec 15, 2024 at 09:43:20PM +0200, Laurent Pinchart wrote:\n> > >>>>>>> On Sun, Dec 15, 2024 at 09:04:08PM +0200, Laurent Pinchart wrote:\n> > >>>>>>>> On Thu, Dec 12, 2024 at 07:16:54PM +0100, Barnabás Pőcze wrote:\n> > >>>>>>>>> The built artifacts will be reused in a later job, so split\n> > >>>>>>>>> the \"test-unit\" into the \"build-test\" and \"test-unit\" jobs.\n> > >>>>>>>>>\n> > >>>>>>>>> The `libevent` development package cannot be installed in the container\n> > >>>>>>>>\n> > >>>>>>>> I've write `libevent-dev` here to avoid ambiguities.\n> > >>>>>>>>\n> > >>>>>>>>> directly because it is not multiarch compatible. It is, however, installed\n> > >>>>>>>>> in the architecture specific build jobs, right before building. To ensure\n> > >>>>>>>>> that the it is available for already built executables in different jobs,\n> > >>>>>>>>\n> > >>>>>>>> \"that the it is\" ?\n> > >>>>>>>>\n> > >>>>>>>>> install just the libraries in the container.\n> > >>>>>>>>\n> > >>>>>>>> And name here `libevent`.\n> > >>>>>>>>\n> > >>>>>>>>>\n> > >>>>>>>>> Signed-off-by: Barnabás Pőcze \n> > >>>>>>>>> ---\n> > >>>>>>>>> .gitlab-ci/setup-container.sh | 3 +++\n> > >>>>>>>>> gitlab-ci.yml | 42 +++++++++++++++++++++++------------\n> > >>>>>>>>> 2 files changed, 31 insertions(+), 14 deletions(-)\n> > >>>>>>>>>\n> > >>>>>>>>> diff --git a/.gitlab-ci/setup-container.sh b/.gitlab-ci/setup-container.sh\n> > >>>>>>>>> index d2909c7..0658368 100755\n> > >>>>>>>>> --- a/.gitlab-ci/setup-container.sh\n> > >>>>>>>>> +++ b/.gitlab-ci/setup-container.sh\n> > >>>>>>>>> @@ -103,6 +103,9 @@ case $FDO_DISTRIBUTION_VERSION in\n> > >>>>>>>>> 'bookworm')\n> > >>>>>>>>> # libclang-rt-dev for the clang ASan runtime.\n> > >>>>>>>>> PKGS_LIBCAMERA_RUNTIME_MULTIARCH+=( libclang-rt-dev )\n> > >>>>>>>>> + # For cam and lc-compliance\n> > >>>>>>>>> + # libevent-dev cannot be used here, see build-libcamera-common.sh\n> > >>>>>>>>> + PKGS_LIBCAMERA_RUNTIME_MULTIARCH+=( libevent-2.1-7 libevent-pthreads-2.1-7 )\n> > >>>>>>>>> ;;\n> > >>>>>>>>> 'trixie')\n> > >>>>>>>>> # gcc 13 to expand compilation testing coverage.\n> > >>>>>>>>> diff --git a/gitlab-ci.yml b/gitlab-ci.yml\n> > >>>>>>>>> index 8bc8bc2..c7448b8 100644\n> > >>>>>>>>> --- a/gitlab-ci.yml\n> > >>>>>>>>> +++ b/gitlab-ci.yml\n> > >>>>>>>>> @@ -64,7 +64,7 @@ include:\n> > >>>>>>>>> .libcamera-ci.debian:12:\n> > >>>>>>>>> variables:\n> > >>>>>>>>> FDO_DISTRIBUTION_VERSION: 'bookworm'\n> > >>>>>>>>> - FDO_DISTRIBUTION_TAG: '2024-12-12.1'\n> > >>>>>>>>> + FDO_DISTRIBUTION_TAG: '2024-12-12.2'\n> > >>>>>>>>>\n> > >>>>>>>>> .libcamera-ci.debian:13:\n> > >>>>>>>>> variables:\n> > >>>>>>>>> @@ -363,28 +363,18 @@ test-soraka:\n> > >>>>>>>>> script:\n> > >>>>>>>>> - submit .gitlab-ci/lava/soraka-camera-test.yml\n> > >>>>>>>>>\n> > >>>>>>>>> -# Run the unit tests in a virtual machine. Enable only the options exercised by\n> > >>>>>>>>> -# the unit tests.\n> > >>>>>>>>> -test-unit:\n> > >>>>>>>>> +# Enable only the options exercised by the unit tests.\n> > >>>>>>>>> +build-test:debug:\n> > >>>>>>>>\n> > >>>>>>>> I'd call this build-package:amd64, as we have build-package:arm64 and\n> > >>>>>>>> build-package:cros. I think it would also make sense to use the same\n> > >>>>>>>> build options for the amd64 and arm64 packages (beside possibly the\n> > >>>>>>>> selected pipeline handlers, although the 'auto' option may work for\n> > >>>>>>>> both).\n> > >>>>>>>>\n> > >>>>>>>>> extends:\n> > >>>>>>>>> - .fdo.distribution-image@debian\n> > >>>>>>>>> - .libcamera-ci.debian:12\n> > >>>>>>>>> - .libcamera-ci.scripts\n> > >>>>>>>>> - stage: test\n> > >>>>>>>>> + stage: build\n> > >>>>>>>>> needs:\n> > >>>>>>>>> - job: container-debian:12\n> > >>>>>>>>> artifacts: false\n> > >>>>>>>>> - tags:\n> > >>>>>>>>> - - kvm\n> > >>>>>>>>> script:\n> > >>>>>>>>> - $CI_PROJECT_DIR/.gitlab-ci/build-libcamera.sh\n> > >>>>>>>>> - - $CI_PROJECT_DIR/.gitlab-ci/test-libcamera-qemu.sh\n> > >>>>>>>>> - artifacts:\n> > >>>>>>>>> - name: libcamera-unit-tests-${CI_COMMIT_SHA}\n> > >>>>>>>>> - when: always\n> > >>>>>>>>> - expire_in: 1 week\n> > >>>>>>>>> - paths:\n> > >>>>>>>>> - - build/meson-logs/\n> > >>>>>>>>> variables:\n> > >>>>>>>>> BUILD_TYPE: debug\n> > >>>>>>>>> MESON_OPTIONS: >-\n> > >>>>>>>>> @@ -399,6 +389,30 @@ test-unit:\n> > >>>>>>>>> -D qcam=disabled\n> > >>>>>>>>> -D test=true\n> > >>>>>>>>> -D v4l2=true\n> > >>>>>>>>> + artifacts:\n> > >>>>>>>>> + paths:\n> > >>>>>>>>> + - build/\n> > >>>>>>>>\n> > >>>>>>>> The whole build directory can be very large. Can't we do the same as\n> > >>>>>>>> build-package:arm64 and run package-libcamera.sh to only package what we\n> > >>>>>>>> need ? We'll need probably need an unpackage script for the test-unit\n> > >>>>>>>> job.\n> > >>>>>>>\n> > >>>>>>> But of course the unit test binaries don't get installed... Can we fix\n> > >>>>>>> that and install them ? You can specify \"install_tag : 'tests'\" in\n> > >>>>>>> meson.build so they won't be installed by default (an appropriate\n> > >>>>>>> install_dir is also needed). This in turn requires bumping the minimum\n> > >>>>>>> meson version from 0.63.0 to 0.64.0, which shouldn't be an issue.\n> > >>>>>>\n> > >>>>>> I've been told on IRC that the motivation for the \"tests\" install tag in\n> > >>>>>> meson is https://gitlab.gnome.org/GNOME/gnome-desktop-testing. I don't\n> > >>>>>> think we should switch to a separate runner for unit tests (the pain is\n> > >>>>>> not worth the gain at this point in my opinion), but it could be useful\n> > >>>>>> to tag lc-compliance with install_tag = 'tests'.\n> > >>>>>>\n> > >>>>>>> And now that I've said this, I realize we wouldn't be able to run \"meson\n> > >>>>>>> test\" to run the tests :-/ I'm not sure there's an appropriate solution\n> > >>>>>>> for this. If not, given the size of the build directory, and to avoid\n> > >>>>>>> transferring a large amount of data between runners, we may need to keep\n> > >>>>>>> building libcamera within the test-unit job :-(\n> > >>>>>>>\n> > >>>>>>> A separate build-package target for lc-compliance would still make\n> > >>>>>>> sense.\n> > >>>>>\n> > >>>>> I think it would be unfortunate to give up the usage `meson test` as you\n> > >>>>> mentioned.\n> > >>>>\n> > >>>> We could work on a replacement, but it would require a significant\n> > >>>> amount of work and I think there are better things to do.\n> > >>>>\n> > >>>>> I have not noticed that these build artifacts would put any\n> > >>>>> appreciable strain on the infrastructure. The compressed build directory\n> > >>>>> comes out to around 167 MiB; I am not sure if I would consider that a\n> > >>>>> large amount of data. It is definitely cheaper, in terms of time, than\n> > >>>>> building libcamera twice. Clearing the object files could be another\n> > >>>>> option. With `artifacts:exclude: build/**/*.o` we can seemingly\n> > >>>>> remove more than half of the uncompressed size, and about 1/4 of\n> > >>>>> the compressed size. Does this look acceptable?\n> > >>>>\n> > >>>> Possibly. We should probably ask the fdo sysadmins about what is\n> > >>>> acceptable.\n> > >>>>\n> > >>>> I gave it a try locally though, and deleting all *.o files in the build\n> > >>>> directory results in \"meson test\" rebuilding everything.\n> > >>>\n> > >>> As far as I can see that does not happen with the `--no-rebuild` option,\n> > >>> which is already used in `test-libcamera-qemu.sh`.\n> > > \n> > > Ah, good point, I missed that.\n> > > \n> > >>>> For other uses of the artifacts (in particular deployment on real\n> > >>>> devices), I would still prefer minimizing the bandwidth, creating a\n> > >>>> package similarly to what build-package:arm64 does. How about keeping\n> > >>>> test-unit as-is, at the cost of a recompilation, and creating a\n> > >>>> build-package:amd64 that will be used by the lc-compliance test job ? We\n> > >>>> can try to improve on top when/if needed.\n> > >>>\n> > >>> Couple observations:\n> > >>>\n> > >>> 1. The virtual pipeline handler configuration is not installed, so\n> > >>> that needs to be addressed. (Was this omitted intentionally?)\n> > > \n> > > I don't know. We have to be careful here, we don't want to virtual\n> > > pipeline handler to end up being enabled with a default valid\n> > > configuration in distributions, otherwise people will all of a sudden\n> > > see virtual cameras poluting their devices list.\n> > \n> > I am not sure what should be done. Maybe a separate meson `install_tag`.\n> > Or I suppose the configuration file could be created right before\n> > running it?\n> \n> Hmmmm... If this is a sample configuration file that we don't want to\n> install by default in the location where it will be lookup up, how about\n> installing it as an example in doc_install_dir (e.g.\n> /usr/share/doc/libcamera/) and name it virtual.yaml.example ? That seems\n> to be a fairly common pattern.\n\n\nThat sounds fairly reasonable.\n\n> You can move the doc_install_dir definition from\n> Documentation/meson.build to src/meson.build and rename it to\n> libcamera_docdir.\n\nI'm fine with that. It would only get installed there if the virtual\npipeline handler is enabled, and I suspect distro's won't include that ?\n\n\nWill the CI define it's own 'virtual camera config' file to support it's\nown virtual requirements?\n\n--\nKieran\n\n\n> \n> > >>> 2. I am not a fan of the extra `tar` and `ldconfig` calls that\n> > >>> need to be sprinkled in. I think this would be much better\n> > >>> if the package was not a mere tar archive but a proper deb/etc.\n> > >>> package. I imagine that is a prerequisite of deploying on real\n> > >>> hardware in any case, correct?\n> > >>\n> > >> Having the server build a 'real' deb would be a real benefit IMO, and\n> > >> indeed could help with installation/set up on real targets for testing\n> > >> in defined environments.\n> > >>\n> > >> I've wanted to tackle that for a while but never got time.\n> > > \n> > > Same, I think it's probably worth it, but it will require some effort\n> > > and I didn't have enough time when I implemented build-package:arm64.\n> \n> -- \n> Regards,\n> \n> Laurent Pinchart","headers":{"Return-Path":"","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id 85BBBBD1F1\n\tfor ;\n\tTue, 17 Dec 2024 17:38:08 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id C043067FF4;\n\tTue, 17 Dec 2024 18:38:07 +0100 (CET)","from perceval.ideasonboard.com (perceval.ideasonboard.com\n\t[213.167.242.64])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id C0A4867FE6\n\tfor ;\n\tTue, 17 Dec 2024 18:38:06 +0100 (CET)","from pendragon.ideasonboard.com\n\t(cpc89244-aztw30-2-0-cust6594.18-1.cable.virginm.net [86.31.185.195])\n\tby perceval.ideasonboard.com (Postfix) with ESMTPSA id 4C1923E;\n\tTue, 17 Dec 2024 18:37:29 +0100 (CET)"],"Authentication-Results":"lancelot.ideasonboard.com; dkim=pass (1024-bit key;\n\tunprotected) header.d=ideasonboard.com header.i=@ideasonboard.com\n\theader.b=\"cwi+CcNx\"; dkim-atps=neutral","DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com;\n\ts=mail; t=1734457049;\n\tbh=n2mr3CmQVK4fbcayoLmusVJAKF0/qGEwnYTjG4Buu24=;\n\th=In-Reply-To:References:Subject:From:Cc:To:Date:From;\n\tb=cwi+CcNxqVqzYMZzJ6xonzw5Sw7DkgIl6gwa6dKdWc7M9rBx36tEiv57fBP/MrJyU\n\twi5X9nieaVs20BVBEdqRzKdDjcKRyGZDI7Kby6GA1+dUze8mQael9dXRGZyGp6KEHm\n\t6eTMeETQNKt72eRp9RB76Ca/tP9ZSJ+OQ+m3gJFw=","Content-Type":"text/plain; charset=\"utf-8\"","MIME-Version":"1.0","Content-Transfer-Encoding":"quoted-printable","In-Reply-To":"<20241217172459.GM20432@pendragon.ideasonboard.com>","References":"<20241212181655.112958-2-barnabas.pocze@ideasonboard.com>\n\t<20241215194320.GN9975@pendragon.ideasonboard.com>\n\t<20241215204350.GP9975@pendragon.ideasonboard.com>\n\t<7948749f-b11a-4397-8332-29c681f188d0@ideasonboard.com>\n\t<20241216110442.GH32204@pendragon.ideasonboard.com>\n\t<5c441936-60e0-40e5-b542-a02ccf0739ca@ideasonboard.com>\n\t<173437357450.543771.15768411058221148888@ping.linuxembedded.co.uk>\n\t<20241216200141.GC1304@pendragon.ideasonboard.com>\n\t<2d23688e-7770-4a44-9495-ec31363fb769@ideasonboard.com>\n\t<20241217172459.GM20432@pendragon.ideasonboard.com>","Subject":"Re: [libcamera-ci] [RFC PATCH v1 2/3] Separate the building and\n\trunning of unit tests","From":"Kieran Bingham ","Cc":"libcamera-devel@lists.libcamera.org","To":"=?utf-8?q?Barnab=C3=A1s_P=C5=91cze?= ,\n\tLaurent Pinchart ","Date":"Tue, 17 Dec 2024 17:38:03 +0000","Message-ID":"<173445708372.4145007.5052263089528641829@ping.linuxembedded.co.uk>","User-Agent":"alot/0.10","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" "}},{"id":32847,"web_url":"https://patchwork.libcamera.org/comment/32847/","msgid":"<173445716854.4145007.14278755417112074362@ping.linuxembedded.co.uk>","date":"2024-12-17T17:39:28","subject":"Re: [libcamera-ci] [RFC PATCH v1 2/3] Separate the building and\n\trunning of unit tests","submitter":{"id":4,"url":"https://patchwork.libcamera.org/api/people/4/","name":"Kieran Bingham","email":"kieran.bingham@ideasonboard.com"},"content":"Quoting Laurent Pinchart (2024-12-17 17:32:25)\n> On Tue, Dec 17, 2024 at 07:25:00PM +0200, Laurent Pinchart wrote:\n> > > >>> Couple observations:\n> > > >>>\n> > > >>> 1. The virtual pipeline handler configuration is not installed, so\n> > > >>> that needs to be addressed. (Was this omitted intentionally?)\n> > > > \n> > > > I don't know. We have to be careful here, we don't want to virtual\n> > > > pipeline handler to end up being enabled with a default valid\n> > > > configuration in distributions, otherwise people will all of a sudden\n> > > > see virtual cameras poluting their devices list.\n> > > \n> > > I am not sure what should be done. Maybe a separate meson `install_tag`.\n> > > Or I suppose the configuration file could be created right before\n> > > running it?\n> > \n> > Hmmmm... If this is a sample configuration file that we don't want to\n> > install by default in the location where it will be lookup up, how about\n> > installing it as an example in doc_install_dir (e.g.\n> > /usr/share/doc/libcamera/) and name it virtual.yaml.example ? That seems\n> > to be a fairly common pattern.\n> \n> And then in CI we would take that file and copy it to the expected\n> location, renaming it to virtual.yaml.\n> \n> There's also the question of what install_tag to use. Other data files\n> use 'runtime', but those are installed to their runtime location. 'doc'\n> could be an option, we can use that even if we disable documentation\n> build in CI, but it may feel a bit weird, and possibly later install\n> other types of documentation that we way not want (although I suppose\n> installation of future \"real\" documentation could be conditioned by the\n> documentation meson option).\n> \n> Another option is to create it from scratch in CI (possibly with the\n> file just added to the CI repository and copied in the test job).\n\nUltimately that's perhaps the 'easiest' option. But it would be nice to\nkeep the CI aligned with any updates to the example virtual\nconfiguration too.\n\n--\nKieran","headers":{"Return-Path":"","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id 8994DBD1F1\n\tfor ;\n\tTue, 17 Dec 2024 17:39:33 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id 48AC367FFB;\n\tTue, 17 Dec 2024 18:39:33 +0100 (CET)","from perceval.ideasonboard.com (perceval.ideasonboard.com\n\t[213.167.242.64])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id 9A88067FE6\n\tfor ;\n\tTue, 17 Dec 2024 18:39:31 +0100 (CET)","from pendragon.ideasonboard.com\n\t(cpc89244-aztw30-2-0-cust6594.18-1.cable.virginm.net [86.31.185.195])\n\tby perceval.ideasonboard.com (Postfix) with ESMTPSA id 462133E;\n\tTue, 17 Dec 2024 18:38:54 +0100 (CET)"],"Authentication-Results":"lancelot.ideasonboard.com; dkim=pass (1024-bit key;\n\tunprotected) header.d=ideasonboard.com header.i=@ideasonboard.com\n\theader.b=\"BIgPwwCU\"; dkim-atps=neutral","DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com;\n\ts=mail; t=1734457134;\n\tbh=qY7DG8BmZT/CZh630xtYQqgS16P4GZH/kYvBQLGr/Rg=;\n\th=In-Reply-To:References:Subject:From:Cc:To:Date:From;\n\tb=BIgPwwCUozjLA9jjHhq6O/idbnH/Rw0L8tDF3DFdqLVKykOMfX6E34RQeub1isCFV\n\tIQga1BqpweooDadQCXFRYjkh+Pn2ZU6ZxDACqodwJIp1Me8ca+6eM2W/RD3JmePmdx\n\tZzQ2yMXzqiBhlfPxSpYED6PSfGDz5GDRPAMjcTws=","Content-Type":"text/plain; charset=\"utf-8\"","MIME-Version":"1.0","Content-Transfer-Encoding":"quoted-printable","In-Reply-To":"<20241217173225.GN20432@pendragon.ideasonboard.com>","References":"<20241215190408.GL9975@pendragon.ideasonboard.com>\n\t<20241215204350.GP9975@pendragon.ideasonboard.com>\n\t<7948749f-b11a-4397-8332-29c681f188d0@ideasonboard.com>\n\t<20241216110442.GH32204@pendragon.ideasonboard.com>\n\t<5c441936-60e0-40e5-b542-a02ccf0739ca@ideasonboard.com>\n\t<173437357450.543771.15768411058221148888@ping.linuxembedded.co.uk>\n\t<20241216200141.GC1304@pendragon.ideasonboard.com>\n\t<2d23688e-7770-4a44-9495-ec31363fb769@ideasonboard.com>\n\t<20241217172459.GM20432@pendragon.ideasonboard.com>\n\t<20241217173225.GN20432@pendragon.ideasonboard.com>","Subject":"Re: [libcamera-ci] [RFC PATCH v1 2/3] Separate the building and\n\trunning of unit tests","From":"Kieran Bingham ","Cc":"libcamera-devel@lists.libcamera.org","To":"=?utf-8?q?Barnab=C3=A1s_P=C5=91cze?= ,\n\tLaurent Pinchart ","Date":"Tue, 17 Dec 2024 17:39:28 +0000","Message-ID":"<173445716854.4145007.14278755417112074362@ping.linuxembedded.co.uk>","User-Agent":"alot/0.10","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" "}},{"id":32857,"web_url":"https://patchwork.libcamera.org/comment/32857/","msgid":"<20241218010720.GV23470@pendragon.ideasonboard.com>","date":"2024-12-18T01:07:20","subject":"Re: [libcamera-ci] [RFC PATCH v1 2/3] Separate the building and\n\trunning of unit tests","submitter":{"id":2,"url":"https://patchwork.libcamera.org/api/people/2/","name":"Laurent Pinchart","email":"laurent.pinchart@ideasonboard.com"},"content":"On Tue, Dec 17, 2024 at 05:38:03PM +0000, Kieran Bingham wrote:\n> Quoting Laurent Pinchart (2024-12-17 17:24:59)\n> > On Tue, Dec 17, 2024 at 05:09:45PM +0100, Barnabás Pőcze wrote:\n> > > 2024. 12. 16. 21:01 keltezéssel, Laurent Pinchart írta:\n> > > > On Mon, Dec 16, 2024 at 06:26:14PM +0000, Kieran Bingham wrote:\n> > > >> Quoting Barnabás Pőcze (2024-12-16 17:28:46)\n> > > >>> 2024. 12. 16. 12:04 keltezéssel, Laurent Pinchart írta:\n> > > >>>> On Mon, Dec 16, 2024 at 10:13:54AM +0100, Barnabás Pőcze wrote:\n> > > >>>>> 2024. 12. 15. 21:43 keltezéssel, Laurent Pinchart írta:\n> > > >>>>>> On Sun, Dec 15, 2024 at 09:43:20PM +0200, Laurent Pinchart wrote:\n> > > >>>>>>> On Sun, Dec 15, 2024 at 09:04:08PM +0200, Laurent Pinchart wrote:\n> > > >>>>>>>> On Thu, Dec 12, 2024 at 07:16:54PM +0100, Barnabás Pőcze wrote:\n> > > >>>>>>>>> The built artifacts will be reused in a later job, so split\n> > > >>>>>>>>> the \"test-unit\" into the \"build-test\" and \"test-unit\" jobs.\n> > > >>>>>>>>>\n> > > >>>>>>>>> The `libevent` development package cannot be installed in the container\n> > > >>>>>>>>\n> > > >>>>>>>> I've write `libevent-dev` here to avoid ambiguities.\n> > > >>>>>>>>\n> > > >>>>>>>>> directly because it is not multiarch compatible. It is, however, installed\n> > > >>>>>>>>> in the architecture specific build jobs, right before building. To ensure\n> > > >>>>>>>>> that the it is available for already built executables in different jobs,\n> > > >>>>>>>>\n> > > >>>>>>>> \"that the it is\" ?\n> > > >>>>>>>>\n> > > >>>>>>>>> install just the libraries in the container.\n> > > >>>>>>>>\n> > > >>>>>>>> And name here `libevent`.\n> > > >>>>>>>>\n> > > >>>>>>>>>\n> > > >>>>>>>>> Signed-off-by: Barnabás Pőcze \n> > > >>>>>>>>> ---\n> > > >>>>>>>>> .gitlab-ci/setup-container.sh | 3 +++\n> > > >>>>>>>>> gitlab-ci.yml | 42 +++++++++++++++++++++++------------\n> > > >>>>>>>>> 2 files changed, 31 insertions(+), 14 deletions(-)\n> > > >>>>>>>>>\n> > > >>>>>>>>> diff --git a/.gitlab-ci/setup-container.sh b/.gitlab-ci/setup-container.sh\n> > > >>>>>>>>> index d2909c7..0658368 100755\n> > > >>>>>>>>> --- a/.gitlab-ci/setup-container.sh\n> > > >>>>>>>>> +++ b/.gitlab-ci/setup-container.sh\n> > > >>>>>>>>> @@ -103,6 +103,9 @@ case $FDO_DISTRIBUTION_VERSION in\n> > > >>>>>>>>> 'bookworm')\n> > > >>>>>>>>> # libclang-rt-dev for the clang ASan runtime.\n> > > >>>>>>>>> PKGS_LIBCAMERA_RUNTIME_MULTIARCH+=( libclang-rt-dev )\n> > > >>>>>>>>> + # For cam and lc-compliance\n> > > >>>>>>>>> + # libevent-dev cannot be used here, see build-libcamera-common.sh\n> > > >>>>>>>>> + PKGS_LIBCAMERA_RUNTIME_MULTIARCH+=( libevent-2.1-7 libevent-pthreads-2.1-7 )\n> > > >>>>>>>>> ;;\n> > > >>>>>>>>> 'trixie')\n> > > >>>>>>>>> # gcc 13 to expand compilation testing coverage.\n> > > >>>>>>>>> diff --git a/gitlab-ci.yml b/gitlab-ci.yml\n> > > >>>>>>>>> index 8bc8bc2..c7448b8 100644\n> > > >>>>>>>>> --- a/gitlab-ci.yml\n> > > >>>>>>>>> +++ b/gitlab-ci.yml\n> > > >>>>>>>>> @@ -64,7 +64,7 @@ include:\n> > > >>>>>>>>> .libcamera-ci.debian:12:\n> > > >>>>>>>>> variables:\n> > > >>>>>>>>> FDO_DISTRIBUTION_VERSION: 'bookworm'\n> > > >>>>>>>>> - FDO_DISTRIBUTION_TAG: '2024-12-12.1'\n> > > >>>>>>>>> + FDO_DISTRIBUTION_TAG: '2024-12-12.2'\n> > > >>>>>>>>>\n> > > >>>>>>>>> .libcamera-ci.debian:13:\n> > > >>>>>>>>> variables:\n> > > >>>>>>>>> @@ -363,28 +363,18 @@ test-soraka:\n> > > >>>>>>>>> script:\n> > > >>>>>>>>> - submit .gitlab-ci/lava/soraka-camera-test.yml\n> > > >>>>>>>>>\n> > > >>>>>>>>> -# Run the unit tests in a virtual machine. Enable only the options exercised by\n> > > >>>>>>>>> -# the unit tests.\n> > > >>>>>>>>> -test-unit:\n> > > >>>>>>>>> +# Enable only the options exercised by the unit tests.\n> > > >>>>>>>>> +build-test:debug:\n> > > >>>>>>>>\n> > > >>>>>>>> I'd call this build-package:amd64, as we have build-package:arm64 and\n> > > >>>>>>>> build-package:cros. I think it would also make sense to use the same\n> > > >>>>>>>> build options for the amd64 and arm64 packages (beside possibly the\n> > > >>>>>>>> selected pipeline handlers, although the 'auto' option may work for\n> > > >>>>>>>> both).\n> > > >>>>>>>>\n> > > >>>>>>>>> extends:\n> > > >>>>>>>>> - .fdo.distribution-image@debian\n> > > >>>>>>>>> - .libcamera-ci.debian:12\n> > > >>>>>>>>> - .libcamera-ci.scripts\n> > > >>>>>>>>> - stage: test\n> > > >>>>>>>>> + stage: build\n> > > >>>>>>>>> needs:\n> > > >>>>>>>>> - job: container-debian:12\n> > > >>>>>>>>> artifacts: false\n> > > >>>>>>>>> - tags:\n> > > >>>>>>>>> - - kvm\n> > > >>>>>>>>> script:\n> > > >>>>>>>>> - $CI_PROJECT_DIR/.gitlab-ci/build-libcamera.sh\n> > > >>>>>>>>> - - $CI_PROJECT_DIR/.gitlab-ci/test-libcamera-qemu.sh\n> > > >>>>>>>>> - artifacts:\n> > > >>>>>>>>> - name: libcamera-unit-tests-${CI_COMMIT_SHA}\n> > > >>>>>>>>> - when: always\n> > > >>>>>>>>> - expire_in: 1 week\n> > > >>>>>>>>> - paths:\n> > > >>>>>>>>> - - build/meson-logs/\n> > > >>>>>>>>> variables:\n> > > >>>>>>>>> BUILD_TYPE: debug\n> > > >>>>>>>>> MESON_OPTIONS: >-\n> > > >>>>>>>>> @@ -399,6 +389,30 @@ test-unit:\n> > > >>>>>>>>> -D qcam=disabled\n> > > >>>>>>>>> -D test=true\n> > > >>>>>>>>> -D v4l2=true\n> > > >>>>>>>>> + artifacts:\n> > > >>>>>>>>> + paths:\n> > > >>>>>>>>> + - build/\n> > > >>>>>>>>\n> > > >>>>>>>> The whole build directory can be very large. Can't we do the same as\n> > > >>>>>>>> build-package:arm64 and run package-libcamera.sh to only package what we\n> > > >>>>>>>> need ? We'll need probably need an unpackage script for the test-unit\n> > > >>>>>>>> job.\n> > > >>>>>>>\n> > > >>>>>>> But of course the unit test binaries don't get installed... Can we fix\n> > > >>>>>>> that and install them ? You can specify \"install_tag : 'tests'\" in\n> > > >>>>>>> meson.build so they won't be installed by default (an appropriate\n> > > >>>>>>> install_dir is also needed). This in turn requires bumping the minimum\n> > > >>>>>>> meson version from 0.63.0 to 0.64.0, which shouldn't be an issue.\n> > > >>>>>>\n> > > >>>>>> I've been told on IRC that the motivation for the \"tests\" install tag in\n> > > >>>>>> meson is https://gitlab.gnome.org/GNOME/gnome-desktop-testing. I don't\n> > > >>>>>> think we should switch to a separate runner for unit tests (the pain is\n> > > >>>>>> not worth the gain at this point in my opinion), but it could be useful\n> > > >>>>>> to tag lc-compliance with install_tag = 'tests'.\n> > > >>>>>>\n> > > >>>>>>> And now that I've said this, I realize we wouldn't be able to run \"meson\n> > > >>>>>>> test\" to run the tests :-/ I'm not sure there's an appropriate solution\n> > > >>>>>>> for this. If not, given the size of the build directory, and to avoid\n> > > >>>>>>> transferring a large amount of data between runners, we may need to keep\n> > > >>>>>>> building libcamera within the test-unit job :-(\n> > > >>>>>>>\n> > > >>>>>>> A separate build-package target for lc-compliance would still make\n> > > >>>>>>> sense.\n> > > >>>>>\n> > > >>>>> I think it would be unfortunate to give up the usage `meson test` as you\n> > > >>>>> mentioned.\n> > > >>>>\n> > > >>>> We could work on a replacement, but it would require a significant\n> > > >>>> amount of work and I think there are better things to do.\n> > > >>>>\n> > > >>>>> I have not noticed that these build artifacts would put any\n> > > >>>>> appreciable strain on the infrastructure. The compressed build directory\n> > > >>>>> comes out to around 167 MiB; I am not sure if I would consider that a\n> > > >>>>> large amount of data. It is definitely cheaper, in terms of time, than\n> > > >>>>> building libcamera twice. Clearing the object files could be another\n> > > >>>>> option. With `artifacts:exclude: build/**/*.o` we can seemingly\n> > > >>>>> remove more than half of the uncompressed size, and about 1/4 of\n> > > >>>>> the compressed size. Does this look acceptable?\n> > > >>>>\n> > > >>>> Possibly. We should probably ask the fdo sysadmins about what is\n> > > >>>> acceptable.\n> > > >>>>\n> > > >>>> I gave it a try locally though, and deleting all *.o files in the build\n> > > >>>> directory results in \"meson test\" rebuilding everything.\n> > > >>>\n> > > >>> As far as I can see that does not happen with the `--no-rebuild` option,\n> > > >>> which is already used in `test-libcamera-qemu.sh`.\n> > > > \n> > > > Ah, good point, I missed that.\n> > > > \n> > > >>>> For other uses of the artifacts (in particular deployment on real\n> > > >>>> devices), I would still prefer minimizing the bandwidth, creating a\n> > > >>>> package similarly to what build-package:arm64 does. How about keeping\n> > > >>>> test-unit as-is, at the cost of a recompilation, and creating a\n> > > >>>> build-package:amd64 that will be used by the lc-compliance test job ? We\n> > > >>>> can try to improve on top when/if needed.\n> > > >>>\n> > > >>> Couple observations:\n> > > >>>\n> > > >>> 1. The virtual pipeline handler configuration is not installed, so\n> > > >>> that needs to be addressed. (Was this omitted intentionally?)\n> > > > \n> > > > I don't know. We have to be careful here, we don't want to virtual\n> > > > pipeline handler to end up being enabled with a default valid\n> > > > configuration in distributions, otherwise people will all of a sudden\n> > > > see virtual cameras poluting their devices list.\n> > > \n> > > I am not sure what should be done. Maybe a separate meson `install_tag`.\n> > > Or I suppose the configuration file could be created right before\n> > > running it?\n> > \n> > Hmmmm... If this is a sample configuration file that we don't want to\n> > install by default in the location where it will be lookup up, how about\n> > installing it as an example in doc_install_dir (e.g.\n> > /usr/share/doc/libcamera/) and name it virtual.yaml.example ? That seems\n> > to be a fairly common pattern.\n> \n> That sounds fairly reasonable.\n> \n> > You can move the doc_install_dir definition from\n> > Documentation/meson.build to src/meson.build and rename it to\n> > libcamera_docdir.\n> \n> I'm fine with that. It would only get installed there if the virtual\n> pipeline handler is enabled, and I suspect distro's won't include that ?\n\nDistros may, but as long as the example file is installed as an example\nin the documentation directory, and not where the virtual pipeline\nhandler looks for it, it won't disturb users.\n\n> Will the CI define it's own 'virtual camera config' file to support it's\n> own virtual requirements?\n\nI'm increasingly thinking that should be the way, in which case we don't\nneed to install the example. We still could though.\n\n> > > >>> 2. I am not a fan of the extra `tar` and `ldconfig` calls that\n> > > >>> need to be sprinkled in. I think this would be much better\n> > > >>> if the package was not a mere tar archive but a proper deb/etc.\n> > > >>> package. I imagine that is a prerequisite of deploying on real\n> > > >>> hardware in any case, correct?\n> > > >>\n> > > >> Having the server build a 'real' deb would be a real benefit IMO, and\n> > > >> indeed could help with installation/set up on real targets for testing\n> > > >> in defined environments.\n> > > >>\n> > > >> I've wanted to tackle that for a while but never got time.\n> > > > \n> > > > Same, I think it's probably worth it, but it will require some effort\n> > > > and I didn't have enough time when I implemented build-package:arm64.","headers":{"Return-Path":"","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id ED97EC3301\n\tfor ;\n\tWed, 18 Dec 2024 01:07:25 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id 14C3A6805E;\n\tWed, 18 Dec 2024 02:07:25 +0100 (CET)","from perceval.ideasonboard.com (perceval.ideasonboard.com\n\t[IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id 0A8A361898\n\tfor ;\n\tWed, 18 Dec 2024 02:07:23 +0100 (CET)","from pendragon.ideasonboard.com (81-175-209-231.bb.dnainternet.fi\n\t[81.175.209.231])\n\tby perceval.ideasonboard.com (Postfix) with ESMTPSA id 2A4EA514;\n\tWed, 18 Dec 2024 02:06:45 +0100 (CET)"],"Authentication-Results":"lancelot.ideasonboard.com; dkim=pass (1024-bit key;\n\tunprotected) header.d=ideasonboard.com header.i=@ideasonboard.com\n\theader.b=\"fq936C7w\"; dkim-atps=neutral","DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com;\n\ts=mail; t=1734484005;\n\tbh=ldpzvT+tuevB1YimTx8+GT7DElpoPDt7pUzzX5N/QfI=;\n\th=Date:From:To:Cc:Subject:References:In-Reply-To:From;\n\tb=fq936C7wq77+BIyvjRx2XE+BR9sfQ0rABkVSR3Bt2nuYmBcBkJ1ASeWgulyfMASAs\n\tBnf+I1EPJ6N+01XGWfNJOUF2ZYIHqQ1SE5L8DUBRl718g7cXMkaxIMdnjs9JdWp5J2\n\tkFR08CMKBq5mwQyLvfN+AnaChcsjVLuXUftHidig=","Date":"Wed, 18 Dec 2024 03:07:20 +0200","From":"Laurent Pinchart ","To":"Kieran Bingham ","Cc":"=?utf-8?q?Barnab=C3=A1s_P=C5=91cze?= ,\n\tlibcamera-devel@lists.libcamera.org","Subject":"Re: [libcamera-ci] [RFC PATCH v1 2/3] Separate the building and\n\trunning of unit tests","Message-ID":"<20241218010720.GV23470@pendragon.ideasonboard.com>","References":"<20241215194320.GN9975@pendragon.ideasonboard.com>\n\t<20241215204350.GP9975@pendragon.ideasonboard.com>\n\t<7948749f-b11a-4397-8332-29c681f188d0@ideasonboard.com>\n\t<20241216110442.GH32204@pendragon.ideasonboard.com>\n\t<5c441936-60e0-40e5-b542-a02ccf0739ca@ideasonboard.com>\n\t<173437357450.543771.15768411058221148888@ping.linuxembedded.co.uk>\n\t<20241216200141.GC1304@pendragon.ideasonboard.com>\n\t<2d23688e-7770-4a44-9495-ec31363fb769@ideasonboard.com>\n\t<20241217172459.GM20432@pendragon.ideasonboard.com>\n\t<173445708372.4145007.5052263089528641829@ping.linuxembedded.co.uk>","MIME-Version":"1.0","Content-Type":"text/plain; charset=utf-8","Content-Disposition":"inline","Content-Transfer-Encoding":"8bit","In-Reply-To":"<173445708372.4145007.5052263089528641829@ping.linuxembedded.co.uk>","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" "}},{"id":32858,"web_url":"https://patchwork.libcamera.org/comment/32858/","msgid":"<20241218011009.GW23470@pendragon.ideasonboard.com>","date":"2024-12-18T01:10:09","subject":"Re: [libcamera-ci] [RFC PATCH v1 2/3] Separate the building and\n\trunning of unit tests","submitter":{"id":2,"url":"https://patchwork.libcamera.org/api/people/2/","name":"Laurent Pinchart","email":"laurent.pinchart@ideasonboard.com"},"content":"On Tue, Dec 17, 2024 at 05:39:28PM +0000, Kieran Bingham wrote:\n> Quoting Laurent Pinchart (2024-12-17 17:32:25)\n> > On Tue, Dec 17, 2024 at 07:25:00PM +0200, Laurent Pinchart wrote:\n> > > > >>> Couple observations:\n> > > > >>>\n> > > > >>> 1. The virtual pipeline handler configuration is not installed, so\n> > > > >>> that needs to be addressed. (Was this omitted intentionally?)\n> > > > > \n> > > > > I don't know. We have to be careful here, we don't want to virtual\n> > > > > pipeline handler to end up being enabled with a default valid\n> > > > > configuration in distributions, otherwise people will all of a sudden\n> > > > > see virtual cameras poluting their devices list.\n> > > > \n> > > > I am not sure what should be done. Maybe a separate meson `install_tag`.\n> > > > Or I suppose the configuration file could be created right before\n> > > > running it?\n> > > \n> > > Hmmmm... If this is a sample configuration file that we don't want to\n> > > install by default in the location where it will be lookup up, how about\n> > > installing it as an example in doc_install_dir (e.g.\n> > > /usr/share/doc/libcamera/) and name it virtual.yaml.example ? That seems\n> > > to be a fairly common pattern.\n> > \n> > And then in CI we would take that file and copy it to the expected\n> > location, renaming it to virtual.yaml.\n> > \n> > There's also the question of what install_tag to use. Other data files\n> > use 'runtime', but those are installed to their runtime location. 'doc'\n> > could be an option, we can use that even if we disable documentation\n> > build in CI, but it may feel a bit weird, and possibly later install\n> > other types of documentation that we way not want (although I suppose\n> > installation of future \"real\" documentation could be conditioned by the\n> > documentation meson option).\n> > \n> > Another option is to create it from scratch in CI (possibly with the\n> > file just added to the CI repository and copied in the test job).\n> \n> Ultimately that's perhaps the 'easiest' option. But it would be nice to\n> keep the CI aligned with any updates to the example virtual\n> configuration too.\n\nYes, but if we consider that the virtual pipeline handler configuration\nwe use in CI has to be written for the needs of CI, then it should be\npart of the libcamera-ci repository, and updated as the virtual pipeline\nhandler evolves.\n\nOn the other hand, the pipeline handler should pass lc-compliance with\nany valid configuration, so it makes sense to test the example\nconfiguration. Installing it to the documentation directory and then\ncopying it from there in the CI lc-compliance job sounds like a good\noption (which contradicts what I wrote in a separate e-mail 2 minutes\nago :-)). Barnabás, what do you think ?","headers":{"Return-Path":"","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id 6CEA0BD1F1\n\tfor ;\n\tWed, 18 Dec 2024 01:10:14 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id 81DA96805E;\n\tWed, 18 Dec 2024 02:10:13 +0100 (CET)","from perceval.ideasonboard.com (perceval.ideasonboard.com\n\t[IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id 2F15061898\n\tfor ;\n\tWed, 18 Dec 2024 02:10:12 +0100 (CET)","from pendragon.ideasonboard.com (81-175-209-231.bb.dnainternet.fi\n\t[81.175.209.231])\n\tby perceval.ideasonboard.com (Postfix) with ESMTPSA id 565F2514;\n\tWed, 18 Dec 2024 02:09:34 +0100 (CET)"],"Authentication-Results":"lancelot.ideasonboard.com; dkim=pass (1024-bit key;\n\tunprotected) header.d=ideasonboard.com header.i=@ideasonboard.com\n\theader.b=\"kiBkZj3C\"; dkim-atps=neutral","DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com;\n\ts=mail; t=1734484174;\n\tbh=IYbyGPe9n+urPJZW2Ej0ueGmW19FCOlQtMGIoaTbLZc=;\n\th=Date:From:To:Cc:Subject:References:In-Reply-To:From;\n\tb=kiBkZj3CUnk8aesLaY9DovwH2uzon8UALZpF+u56kQUWG2JjG6oL8gCiJkYRcFDUj\n\tJ6f2hc4GrhEsG6eKZGNNNe28JaxVDliIpUUS7dpS/i//hi+becHYb0r9Zb+E/ax+dP\n\tG3J4j4apLhVWC/+l7zIGz7Jij+KJ5LbViHMlimN8=","Date":"Wed, 18 Dec 2024 03:10:09 +0200","From":"Laurent Pinchart ","To":"Kieran Bingham ","Cc":"=?utf-8?q?Barnab=C3=A1s_P=C5=91cze?= ,\n\tlibcamera-devel@lists.libcamera.org","Subject":"Re: [libcamera-ci] [RFC PATCH v1 2/3] Separate the building and\n\trunning of unit tests","Message-ID":"<20241218011009.GW23470@pendragon.ideasonboard.com>","References":"<20241215204350.GP9975@pendragon.ideasonboard.com>\n\t<7948749f-b11a-4397-8332-29c681f188d0@ideasonboard.com>\n\t<20241216110442.GH32204@pendragon.ideasonboard.com>\n\t<5c441936-60e0-40e5-b542-a02ccf0739ca@ideasonboard.com>\n\t<173437357450.543771.15768411058221148888@ping.linuxembedded.co.uk>\n\t<20241216200141.GC1304@pendragon.ideasonboard.com>\n\t<2d23688e-7770-4a44-9495-ec31363fb769@ideasonboard.com>\n\t<20241217172459.GM20432@pendragon.ideasonboard.com>\n\t<20241217173225.GN20432@pendragon.ideasonboard.com>\n\t<173445716854.4145007.14278755417112074362@ping.linuxembedded.co.uk>","MIME-Version":"1.0","Content-Type":"text/plain; charset=utf-8","Content-Disposition":"inline","Content-Transfer-Encoding":"8bit","In-Reply-To":"<173445716854.4145007.14278755417112074362@ping.linuxembedded.co.uk>","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" "}},{"id":32877,"web_url":"https://patchwork.libcamera.org/comment/32877/","msgid":"<81d7a53a-4f31-446f-8ae7-ed27838f8f66@ideasonboard.com>","date":"2024-12-18T21:24:50","subject":"Re: [libcamera-ci] [RFC PATCH v1 2/3] Separate the building and\n\trunning of unit tests","submitter":{"id":216,"url":"https://patchwork.libcamera.org/api/people/216/","name":"Barnabás Pőcze","email":"barnabas.pocze@ideasonboard.com"},"content":"2024. 12. 18. 2:10 keltezéssel, Laurent Pinchart írta:\n> On Tue, Dec 17, 2024 at 05:39:28PM +0000, Kieran Bingham wrote:\n>> Quoting Laurent Pinchart (2024-12-17 17:32:25)\n>>> On Tue, Dec 17, 2024 at 07:25:00PM +0200, Laurent Pinchart wrote:\n>>>>>>>> Couple observations:\n>>>>>>>>\n>>>>>>>> 1. The virtual pipeline handler configuration is not installed, so\n>>>>>>>> that needs to be addressed. (Was this omitted intentionally?)\n>>>>>>\n>>>>>> I don't know. We have to be careful here, we don't want to virtual\n>>>>>> pipeline handler to end up being enabled with a default valid\n>>>>>> configuration in distributions, otherwise people will all of a sudden\n>>>>>> see virtual cameras poluting their devices list.\n>>>>>\n>>>>> I am not sure what should be done. Maybe a separate meson `install_tag`.\n>>>>> Or I suppose the configuration file could be created right before\n>>>>> running it?\n>>>>\n>>>> Hmmmm... If this is a sample configuration file that we don't want to\n>>>> install by default in the location where it will be lookup up, how about\n>>>> installing it as an example in doc_install_dir (e.g.\n>>>> /usr/share/doc/libcamera/) and name it virtual.yaml.example ? That seems\n>>>> to be a fairly common pattern.\n>>>\n>>> And then in CI we would take that file and copy it to the expected\n>>> location, renaming it to virtual.yaml.\n>>>\n>>> There's also the question of what install_tag to use. Other data files\n>>> use 'runtime', but those are installed to their runtime location. 'doc'\n>>> could be an option, we can use that even if we disable documentation\n>>> build in CI, but it may feel a bit weird, and possibly later install\n>>> other types of documentation that we way not want (although I suppose\n>>> installation of future \"real\" documentation could be conditioned by the\n>>> documentation meson option).\n>>>\n>>> Another option is to create it from scratch in CI (possibly with the\n>>> file just added to the CI repository and copied in the test job).\n>>\n>> Ultimately that's perhaps the 'easiest' option. But it would be nice to\n>> keep the CI aligned with any updates to the example virtual\n>> configuration too.\n> \n> Yes, but if we consider that the virtual pipeline handler configuration\n> we use in CI has to be written for the needs of CI, then it should be\n> part of the libcamera-ci repository, and updated as the virtual pipeline\n> handler evolves.\n> \n> On the other hand, the pipeline handler should pass lc-compliance with\n> any valid configuration, so it makes sense to test the example\n> configuration. Installing it to the documentation directory and then\n> copying it from there in the CI lc-compliance job sounds like a good\n> option (which contradicts what I wrote in a separate e-mail 2 minutes\n> ago :-)). Barnabás, what do you think ?\n> \n\nNot sure. To be honest I am not a fan of either solution, but I don't\nreally ave an alternative idea. I suppose we could have a \"special\"\ncamera(s) named something like \"Virtual-CI\" in the example configuration,\nwhich would be the one used in the CI test. Between these two options,\nkeeping the configuration in the main repository seems better to me.\n\nRegards,\nBarnabás Pőcze","headers":{"Return-Path":"","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id 9E557C32FE\n\tfor ;\n\tWed, 18 Dec 2024 21:24:56 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id A40D3680B6;\n\tWed, 18 Dec 2024 22:24:55 +0100 (CET)","from perceval.ideasonboard.com (perceval.ideasonboard.com\n\t[213.167.242.64])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id 408D1680AE\n\tfor ;\n\tWed, 18 Dec 2024 22:24:54 +0100 (CET)","from [192.168.33.16] (185.221.140.157.nat.pool.zt.hu\n\t[185.221.140.157])\n\tby perceval.ideasonboard.com (Postfix) with ESMTPSA id BBFE834D;\n\tWed, 18 Dec 2024 22:24:15 +0100 (CET)"],"Authentication-Results":"lancelot.ideasonboard.com; dkim=pass (1024-bit key;\n\tunprotected) header.d=ideasonboard.com header.i=@ideasonboard.com\n\theader.b=\"Dwo/Py1v\"; dkim-atps=neutral","DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com;\n\ts=mail; t=1734557055;\n\tbh=MRxQeoI2it8KuiHpGKGNWcg7Bp+PtYtr8LON8PlEbgw=;\n\th=Date:From:Subject:To:Cc:References:In-Reply-To:From;\n\tb=Dwo/Py1vzYXTLnp2050B/FA8yggLWZwzBAjTgG4/YON9px3szlrENb9Tv5u0zEKRa\n\t96fFF4T3Di+ybXxav548V/dVqxiesxeiK3moaexhAvVrVAp+b4hI4xXyBAQXhXYu15\n\tq37rA87YuPyZghNX6rMrwYypqHE+OSL5Jb7FbWcQ=","Message-ID":"<81d7a53a-4f31-446f-8ae7-ed27838f8f66@ideasonboard.com>","Date":"Wed, 18 Dec 2024 22:24:50 +0100","MIME-Version":"1.0","User-Agent":"Mozilla Thunderbird","From":"=?utf-8?q?Barnab=C3=A1s_P=C5=91cze?= ","Subject":"Re: [libcamera-ci] [RFC PATCH v1 2/3] Separate the building and\n\trunning of unit tests","To":"Laurent Pinchart ,\n\tKieran Bingham ","Cc":"libcamera-devel@lists.libcamera.org","References":"<20241215204350.GP9975@pendragon.ideasonboard.com>\n\t<7948749f-b11a-4397-8332-29c681f188d0@ideasonboard.com>\n\t<20241216110442.GH32204@pendragon.ideasonboard.com>\n\t<5c441936-60e0-40e5-b542-a02ccf0739ca@ideasonboard.com>\n\t<173437357450.543771.15768411058221148888@ping.linuxembedded.co.uk>\n\t<20241216200141.GC1304@pendragon.ideasonboard.com>\n\t<2d23688e-7770-4a44-9495-ec31363fb769@ideasonboard.com>\n\t<20241217172459.GM20432@pendragon.ideasonboard.com>\n\t<20241217173225.GN20432@pendragon.ideasonboard.com>\n\t<173445716854.4145007.14278755417112074362@ping.linuxembedded.co.uk>\n\t<20241218011009.GW23470@pendragon.ideasonboard.com>","Content-Language":"en-US, hu-HU","In-Reply-To":"<20241218011009.GW23470@pendragon.ideasonboard.com>","Content-Type":"text/plain; charset=UTF-8; format=flowed","Content-Transfer-Encoding":"8bit","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" "}},{"id":32878,"web_url":"https://patchwork.libcamera.org/comment/32878/","msgid":"<20241218232719.GE5518@pendragon.ideasonboard.com>","date":"2024-12-18T23:27:19","subject":"Re: [libcamera-ci] [RFC PATCH v1 2/3] Separate the building and\n\trunning of unit tests","submitter":{"id":2,"url":"https://patchwork.libcamera.org/api/people/2/","name":"Laurent Pinchart","email":"laurent.pinchart@ideasonboard.com"},"content":"Hi Barnabás,\n\nOn Wed, Dec 18, 2024 at 10:24:50PM +0100, Barnabás Pőcze wrote:\n> 2024. 12. 18. 2:10 keltezéssel, Laurent Pinchart írta:\n> > On Tue, Dec 17, 2024 at 05:39:28PM +0000, Kieran Bingham wrote:\n> >> Quoting Laurent Pinchart (2024-12-17 17:32:25)\n> >>> On Tue, Dec 17, 2024 at 07:25:00PM +0200, Laurent Pinchart wrote:\n> >>>>>>>> Couple observations:\n> >>>>>>>>\n> >>>>>>>> 1. The virtual pipeline handler configuration is not installed, so\n> >>>>>>>> that needs to be addressed. (Was this omitted intentionally?)\n> >>>>>>\n> >>>>>> I don't know. We have to be careful here, we don't want to virtual\n> >>>>>> pipeline handler to end up being enabled with a default valid\n> >>>>>> configuration in distributions, otherwise people will all of a sudden\n> >>>>>> see virtual cameras poluting their devices list.\n> >>>>>\n> >>>>> I am not sure what should be done. Maybe a separate meson `install_tag`.\n> >>>>> Or I suppose the configuration file could be created right before\n> >>>>> running it?\n> >>>>\n> >>>> Hmmmm... If this is a sample configuration file that we don't want to\n> >>>> install by default in the location where it will be lookup up, how about\n> >>>> installing it as an example in doc_install_dir (e.g.\n> >>>> /usr/share/doc/libcamera/) and name it virtual.yaml.example ? That seems\n> >>>> to be a fairly common pattern.\n> >>>\n> >>> And then in CI we would take that file and copy it to the expected\n> >>> location, renaming it to virtual.yaml.\n> >>>\n> >>> There's also the question of what install_tag to use. Other data files\n> >>> use 'runtime', but those are installed to their runtime location. 'doc'\n> >>> could be an option, we can use that even if we disable documentation\n> >>> build in CI, but it may feel a bit weird, and possibly later install\n> >>> other types of documentation that we way not want (although I suppose\n> >>> installation of future \"real\" documentation could be conditioned by the\n> >>> documentation meson option).\n> >>>\n> >>> Another option is to create it from scratch in CI (possibly with the\n> >>> file just added to the CI repository and copied in the test job).\n> >>\n> >> Ultimately that's perhaps the 'easiest' option. But it would be nice to\n> >> keep the CI aligned with any updates to the example virtual\n> >> configuration too.\n> > \n> > Yes, but if we consider that the virtual pipeline handler configuration\n> > we use in CI has to be written for the needs of CI, then it should be\n> > part of the libcamera-ci repository, and updated as the virtual pipeline\n> > handler evolves.\n> > \n> > On the other hand, the pipeline handler should pass lc-compliance with\n> > any valid configuration, so it makes sense to test the example\n> > configuration. Installing it to the documentation directory and then\n> > copying it from there in the CI lc-compliance job sounds like a good\n> > option (which contradicts what I wrote in a separate e-mail 2 minutes\n> > ago :-)). Barnabás, what do you think ?\n> \n> Not sure. To be honest I am not a fan of either solution, but I don't\n> really ave an alternative idea. I suppose we could have a \"special\"\n> camera(s) named something like \"Virtual-CI\" in the example configuration,\n> which would be the one used in the CI test. Between these two options,\n> keeping the configuration in the main repository seems better to me.\n\nAgreed. Let's go for that, installing the configuration file as an\nexample, and copying it as part of the CI job.","headers":{"Return-Path":"","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id C80F7C3273\n\tfor ;\n\tWed, 18 Dec 2024 23:27:25 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id DD825680B8;\n\tThu, 19 Dec 2024 00:27:24 +0100 (CET)","from perceval.ideasonboard.com (perceval.ideasonboard.com\n\t[IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id 22D8368000\n\tfor ;\n\tThu, 19 Dec 2024 00:27:23 +0100 (CET)","from pendragon.ideasonboard.com (81-175-209-231.bb.dnainternet.fi\n\t[81.175.209.231])\n\tby perceval.ideasonboard.com (Postfix) with ESMTPSA id B49DE4D4;\n\tThu, 19 Dec 2024 00:26:44 +0100 (CET)"],"Authentication-Results":"lancelot.ideasonboard.com; dkim=pass (1024-bit key;\n\tunprotected) header.d=ideasonboard.com header.i=@ideasonboard.com\n\theader.b=\"AekyQ/0o\"; dkim-atps=neutral","DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com;\n\ts=mail; t=1734564404;\n\tbh=gujB4MQYm4BX++VFf9xgNosdvutmlniuNQIKrtZ/98U=;\n\th=Date:From:To:Cc:Subject:References:In-Reply-To:From;\n\tb=AekyQ/0o5rKcInlGq4HxppFx6t+icU1rkDiHYDVI0aE6Ofww63u5O9PR+w4vrAGXp\n\tHdinTefvo1N7h47DsnNLt4PK8akGgtLVGm6T8yimuA7N5fWy7WJoBC6xpRMWVON7nC\n\tnnYp6LAV1/8TPqeWSwsqOH6kmC6WOauJPX5fbPSo=","Date":"Thu, 19 Dec 2024 01:27:19 +0200","From":"Laurent Pinchart ","To":"=?utf-8?q?Barnab=C3=A1s_P=C5=91cze?= ","Cc":"Kieran Bingham ,\n\tlibcamera-devel@lists.libcamera.org","Subject":"Re: [libcamera-ci] [RFC PATCH v1 2/3] Separate the building and\n\trunning of unit tests","Message-ID":"<20241218232719.GE5518@pendragon.ideasonboard.com>","References":"<20241216110442.GH32204@pendragon.ideasonboard.com>\n\t<5c441936-60e0-40e5-b542-a02ccf0739ca@ideasonboard.com>\n\t<173437357450.543771.15768411058221148888@ping.linuxembedded.co.uk>\n\t<20241216200141.GC1304@pendragon.ideasonboard.com>\n\t<2d23688e-7770-4a44-9495-ec31363fb769@ideasonboard.com>\n\t<20241217172459.GM20432@pendragon.ideasonboard.com>\n\t<20241217173225.GN20432@pendragon.ideasonboard.com>\n\t<173445716854.4145007.14278755417112074362@ping.linuxembedded.co.uk>\n\t<20241218011009.GW23470@pendragon.ideasonboard.com>\n\t<81d7a53a-4f31-446f-8ae7-ed27838f8f66@ideasonboard.com>","MIME-Version":"1.0","Content-Type":"text/plain; charset=utf-8","Content-Disposition":"inline","Content-Transfer-Encoding":"8bit","In-Reply-To":"<81d7a53a-4f31-446f-8ae7-ed27838f8f66@ideasonboard.com>","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" "}}]