[{"id":32734,"web_url":"https://patchwork.libcamera.org/comment/32734/","msgid":"<20241215192108.GM9975@pendragon.ideasonboard.com>","date":"2024-12-15T19:21:08","subject":"Re: [libcamera-ci] [RFC PATCH v1 3/3] Add job to run lc-compliance\n\ton the virtual pipeline handler","submitter":{"id":2,"url":"https://patchwork.libcamera.org/api/people/2/","name":"Laurent Pinchart","email":"laurent.pinchart@ideasonboard.com"},"content":"Hi Barnabás,\n\nThank you for the patch.\n\nOn Thu, Dec 12, 2024 at 07:16:55PM +0100, Barnabás Pőcze wrote:\n> Add a new job \"lc-compliance:virtual\" that uses the build artifacts\n> produced by the \"build-test:debug\" job to run lc-compliance in a\n> virtual machine on the \"Virtual0\" camera.\n> \n> The `force_fallback_for=gtest` option is needed because `cpp_debustl`\n> makes ABI incompatible changes in the STL containers, so googletest\n> also needs to be compiled with these changes.\n> \n> Signed-off-by: Barnabás Pőcze \n> ---\n> .gitlab-ci/test-lc-compliance.sh | 27 +++++++++++++++++++++++++++\n> gitlab-ci.yml | 18 ++++++++++++++++--\n> 2 files changed, 43 insertions(+), 2 deletions(-)\n> create mode 100755 .gitlab-ci/test-lc-compliance.sh\n> \n> diff --git a/.gitlab-ci/test-lc-compliance.sh b/.gitlab-ci/test-lc-compliance.sh\n> new file mode 100755\n> index 0000000..e6697a6\n> --- /dev/null\n> +++ b/.gitlab-ci/test-lc-compliance.sh\n> @@ -0,0 +1,27 @@\n> +#!/bin/bash\n\nPlease add an SPDX comment block, see other scripts for examples.\n\n> +\n> +set -e\n> +\n> +source \"$(dirname \"$0\")/lib.sh\"\n> +\n> +libcamera_compliance() {\n> +\techo \"Running libcamera compliance tests in a qemu VM\"\n> +\n> +\tvirtme-ng \\\n> +\t\t--verbose \\\n> +\t\t--skip-modules \\\n> +\t\t--force-9p \\\n> +\t\t--rwdir \"$PWD/build\" \\\n> +\t\t--run /opt/linux/bzImage \\\n> +\t\t--exec \"LIBCAMERA_LOG_LEVELS=*:DEBUG ./build/src/apps/lc-compliance/lc-compliance -c Virtual0; echo \\\\\\$? > ./build/.test-status\"\n> +\n> +\tlocal status=$(cat build/.test-status)\n> +\techo \"Test result exit state: $status\"\n> +\trm build/.test-status\n> +\n> +\tif [[ $status != 0 ]] ; then\n> +\t\texit $status\n> +\tfi\n> +}\n> +\n> +run libcamera_compliance\n> diff --git a/gitlab-ci.yml b/gitlab-ci.yml\n> index c7448b8..5e9e309 100644\n> --- a/gitlab-ci.yml\n> +++ b/gitlab-ci.yml\n> @@ -382,9 +382,10 @@ build-test:debug:\n> -D cam=disabled\n> -D cpp_debugstl=true\n> -D documentation=disabled\n> + -D force_fallback_for=gtest\n> -D gstreamer=enabled\n> - -D lc-compliance=disabled\n> - -D pipelines=[]\n> + -D lc-compliance=enabled\n> + -D pipelines=virtual\n\n -D pipelines=[virtual]\n\nbut I think that's not strictly needed, as the virtual pipeline handler\nshould be selected automatically when the test option is enabled.\n\n> -D pycamera=enabled\n> -D qcam=disabled\n> -D test=true\n> @@ -420,3 +421,16 @@ test-unit:\n> # artifacts:\n> # reports:\n> # junit: build/meson-logs/testlog.junit.xml\n> +\n> +lc-compliance:virtual:\n> + extends:\n> + - .fdo.distribution-image@debian\n> + - .libcamera-ci.debian:12\n> + - .libcamera-ci.scripts\n> + stage: test\n> + needs:\n> + - job: build-test:debug\n> + tags:\n> + - kvm\n> + script:\n> + - $CI_PROJECT_DIR/.gitlab-ci/test-lc-compliance.sh\n\nI think gtest support generating junit reports. Can we do that, and\npackage the report in artifacts ?","headers":{"Return-Path":"","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id 2C618C32F6\n\tfor ;\n\tSun, 15 Dec 2024 19:21:28 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id 2106467F0D;\n\tSun, 15 Dec 2024 20:21:27 +0100 (CET)","from perceval.ideasonboard.com (perceval.ideasonboard.com\n\t[213.167.242.64])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id CEC6767EEE\n\tfor ;\n\tSun, 15 Dec 2024 20:21:25 +0100 (CET)","from pendragon.ideasonboard.com (81-175-209-231.bb.dnainternet.fi\n\t[81.175.209.231])\n\tby perceval.ideasonboard.com (Postfix) with ESMTPSA id 90B9099F;\n\tSun, 15 Dec 2024 20:20:49 +0100 (CET)"],"Authentication-Results":"lancelot.ideasonboard.com; dkim=pass (1024-bit key;\n\tunprotected) header.d=ideasonboard.com header.i=@ideasonboard.com\n\theader.b=\"mRugBJ8q\"; dkim-atps=neutral","DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com;\n\ts=mail; t=1734290449;\n\tbh=j6eYtCaqtwOvQWapmLatJuewZeavBEfwNKqtmWVBPdE=;\n\th=Date:From:To:Cc:Subject:References:In-Reply-To:From;\n\tb=mRugBJ8qHwV8NwfRHwgDIqSCmGH0gVwtxRCs95qkAN6CJkDMpMG+F3B3RkLMGh8yW\n\t3r5C2SvqgJsHCvpNAwlPrYg1Kzbwkm3yhriKU023dQhFIQvpuBSEqLJGTdn/JCdnZw\n\tvUMSbtvDIJiOVdeUKzEtAD8kaE5uroJOHuKD3+DM=","Date":"Sun, 15 Dec 2024 21:21:08 +0200","From":"Laurent Pinchart ","To":"=?utf-8?q?Barnab=C3=A1s_P=C5=91cze?= ","Cc":"libcamera-devel@lists.libcamera.org","Subject":"Re: [libcamera-ci] [RFC PATCH v1 3/3] Add job to run lc-compliance\n\ton the virtual pipeline handler","Message-ID":"<20241215192108.GM9975@pendragon.ideasonboard.com>","References":"<20241212181655.112958-1-barnabas.pocze@ideasonboard.com>\n\t<20241212181655.112958-3-barnabas.pocze@ideasonboard.com>","MIME-Version":"1.0","Content-Type":"text/plain; charset=utf-8","Content-Disposition":"inline","Content-Transfer-Encoding":"8bit","In-Reply-To":"<20241212181655.112958-3-barnabas.pocze@ideasonboard.com>","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" "}},{"id":32756,"web_url":"https://patchwork.libcamera.org/comment/32756/","msgid":"","date":"2024-12-16T10:15:33","subject":"Re: [libcamera-ci] [RFC PATCH v1 3/3] Add job to run lc-compliance\n\ton the virtual pipeline handler","submitter":{"id":216,"url":"https://patchwork.libcamera.org/api/people/216/","name":"Barnabás Pőcze","email":"barnabas.pocze@ideasonboard.com"},"content":"Hi\n\n\n2024. 12. 15. 20:21 keltezéssel, Laurent Pinchart írta:\n> Hi Barnabás,\n> \n> Thank you for the patch.\n> \n> On Thu, Dec 12, 2024 at 07:16:55PM +0100, Barnabás Pőcze wrote:\n>> Add a new job \"lc-compliance:virtual\" that uses the build artifacts\n>> produced by the \"build-test:debug\" job to run lc-compliance in a\n>> virtual machine on the \"Virtual0\" camera.\n>>\n>> The `force_fallback_for=gtest` option is needed because `cpp_debustl`\n>> makes ABI incompatible changes in the STL containers, so googletest\n>> also needs to be compiled with these changes.\n>>\n>> Signed-off-by: Barnabás Pőcze \n>> ---\n>> .gitlab-ci/test-lc-compliance.sh | 27 +++++++++++++++++++++++++++\n>> gitlab-ci.yml | 18 ++++++++++++++++--\n>> 2 files changed, 43 insertions(+), 2 deletions(-)\n>> create mode 100755 .gitlab-ci/test-lc-compliance.sh\n>>\n>> diff --git a/.gitlab-ci/test-lc-compliance.sh b/.gitlab-ci/test-lc-compliance.sh\n>> new file mode 100755\n>> index 0000000..e6697a6\n>> --- /dev/null\n>> +++ b/.gitlab-ci/test-lc-compliance.sh\n>> @@ -0,0 +1,27 @@\n>> +#!/bin/bash\n> \n> Please add an SPDX comment block, see other scripts for examples.\n> \n>> +\n>> +set -e\n>> +\n>> +source \"$(dirname \"$0\")/lib.sh\"\n>> +\n>> +libcamera_compliance() {\n>> +\techo \"Running libcamera compliance tests in a qemu VM\"\n>> +\n>> +\tvirtme-ng \\\n>> +\t\t--verbose \\\n>> +\t\t--skip-modules \\\n>> +\t\t--force-9p \\\n>> +\t\t--rwdir \"$PWD/build\" \\\n>> +\t\t--run /opt/linux/bzImage \\\n>> +\t\t--exec \"LIBCAMERA_LOG_LEVELS=*:DEBUG ./build/src/apps/lc-compliance/lc-compliance -c Virtual0; echo \\\\\\$? > ./build/.test-status\"\n>> +\n>> +\tlocal status=$(cat build/.test-status)\n>> +\techo \"Test result exit state: $status\"\n>> +\trm build/.test-status\n>> +\n>> +\tif [[ $status != 0 ]] ; then\n>> +\t\texit $status\n>> +\tfi\n>> +}\n>> +\n>> +run libcamera_compliance\n>> diff --git a/gitlab-ci.yml b/gitlab-ci.yml\n>> index c7448b8..5e9e309 100644\n>> --- a/gitlab-ci.yml\n>> +++ b/gitlab-ci.yml\n>> @@ -382,9 +382,10 @@ build-test:debug:\n>> -D cam=disabled\n>> -D cpp_debugstl=true\n>> -D documentation=disabled\n>> + -D force_fallback_for=gtest\n>> -D gstreamer=enabled\n>> - -D lc-compliance=disabled\n>> - -D pipelines=[]\n>> + -D lc-compliance=enabled\n>> + -D pipelines=virtual\n> \n> -D pipelines=[virtual]\n\nI am fairly certain `pipelines=virtual` should work, or is this\na coding style preference? As far as I see tings are not consistent\nunfortunately, `build-libcamera-cros.sh` does not use the full\narray notation, while the other occurrences do.\n\n\n> \n> but I think that's not strictly needed, as the virtual pipeline handler\n> should be selected automatically when the test option is enabled.\n\nYou're right. So should I remove it?\n\n\n> \n>> -D pycamera=enabled\n>> -D qcam=disabled\n>> -D test=true\n>> @@ -420,3 +421,16 @@ test-unit:\n>> # artifacts:\n>> # reports:\n>> # junit: build/meson-logs/testlog.junit.xml\n>> +\n>> +lc-compliance:virtual:\n>> + extends:\n>> + - .fdo.distribution-image@debian\n>> + - .libcamera-ci.debian:12\n>> + - .libcamera-ci.scripts\n>> + stage: test\n>> + needs:\n>> + - job: build-test:debug\n>> + tags:\n>> + - kvm\n>> + script:\n>> + - $CI_PROJECT_DIR/.gitlab-ci/test-lc-compliance.sh\n> \n> I think gtest support generating junit reports. Can we do that, and\n> package the report in artifacts ?\n> \n\nIndeed, it can:\nhttps://gitlab.freedesktop.org/pobrn/libcamera/-/pipelines/1330595/test_report?job_name=lc-compliance%3Avirtual\n\n\nRegards,\nBarnabás Pőcze","headers":{"Return-Path":"","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id D5E01C32F9\n\tfor ;\n\tMon, 16 Dec 2024 10:15:38 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id 019DF67F4B;\n\tMon, 16 Dec 2024 11:15:38 +0100 (CET)","from perceval.ideasonboard.com (perceval.ideasonboard.com\n\t[IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id 4E92167F37\n\tfor ;\n\tMon, 16 Dec 2024 11:15:37 +0100 (CET)","from [192.168.33.16] (185.221.140.157.nat.pool.zt.hu\n\t[185.221.140.157])\n\tby perceval.ideasonboard.com (Postfix) with ESMTPSA id 9C01A22F;\n\tMon, 16 Dec 2024 11:15:00 +0100 (CET)"],"Authentication-Results":"lancelot.ideasonboard.com; dkim=pass (1024-bit key;\n\tunprotected) header.d=ideasonboard.com header.i=@ideasonboard.com\n\theader.b=\"FLihuukx\"; dkim-atps=neutral","DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com;\n\ts=mail; t=1734344100;\n\tbh=wgRo090Ycl5onqr+IUORarfp3MLTDZjNa9BZEXL5W0I=;\n\th=Date:Subject:To:Cc:References:From:In-Reply-To:From;\n\tb=FLihuukxfzMjT6e7/klkiXkeWSHVFT3jEWN7dTvBnQQMh7Ml9LrVCjlozyV3B5I+q\n\tkmG2YNW+80tNOCFgXUpuEMI8SF0Q01amR1PaQ8y4FOffy78QaIMtUBiVrHtTMF/F3t\n\tjC3U88G6tebfjW2kcW/WF7jN/GmVsFWMf5bAWwPc=","Message-ID":"","Date":"Mon, 16 Dec 2024 11:15:33 +0100","MIME-Version":"1.0","User-Agent":"Mozilla Thunderbird","Subject":"Re: [libcamera-ci] [RFC PATCH v1 3/3] Add job to run lc-compliance\n\ton the virtual pipeline handler","To":"Laurent Pinchart ","Cc":"libcamera-devel@lists.libcamera.org","References":"<20241212181655.112958-1-barnabas.pocze@ideasonboard.com>\n\t<20241212181655.112958-3-barnabas.pocze@ideasonboard.com>\n\t<20241215192108.GM9975@pendragon.ideasonboard.com>","Content-Language":"en-US, hu-HU","From":"=?utf-8?q?Barnab=C3=A1s_P=C5=91cze?= ","In-Reply-To":"<20241215192108.GM9975@pendragon.ideasonboard.com>","Content-Type":"text/plain; charset=UTF-8; format=flowed","Content-Transfer-Encoding":"8bit","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" "}},{"id":32760,"web_url":"https://patchwork.libcamera.org/comment/32760/","msgid":"<20241216110610.GI32204@pendragon.ideasonboard.com>","date":"2024-12-16T11:06:10","subject":"Re: [libcamera-ci] [RFC PATCH v1 3/3] Add job to run lc-compliance\n\ton the virtual pipeline handler","submitter":{"id":2,"url":"https://patchwork.libcamera.org/api/people/2/","name":"Laurent Pinchart","email":"laurent.pinchart@ideasonboard.com"},"content":"On Mon, Dec 16, 2024 at 11:15:33AM +0100, Barnabás Pőcze wrote:\n> 2024. 12. 15. 20:21 keltezéssel, Laurent Pinchart írta:\n> > On Thu, Dec 12, 2024 at 07:16:55PM +0100, Barnabás Pőcze wrote:\n> >> Add a new job \"lc-compliance:virtual\" that uses the build artifacts\n> >> produced by the \"build-test:debug\" job to run lc-compliance in a\n> >> virtual machine on the \"Virtual0\" camera.\n> >>\n> >> The `force_fallback_for=gtest` option is needed because `cpp_debustl`\n> >> makes ABI incompatible changes in the STL containers, so googletest\n> >> also needs to be compiled with these changes.\n> >>\n> >> Signed-off-by: Barnabás Pőcze \n> >> ---\n> >> .gitlab-ci/test-lc-compliance.sh | 27 +++++++++++++++++++++++++++\n> >> gitlab-ci.yml | 18 ++++++++++++++++--\n> >> 2 files changed, 43 insertions(+), 2 deletions(-)\n> >> create mode 100755 .gitlab-ci/test-lc-compliance.sh\n> >>\n> >> diff --git a/.gitlab-ci/test-lc-compliance.sh b/.gitlab-ci/test-lc-compliance.sh\n> >> new file mode 100755\n> >> index 0000000..e6697a6\n> >> --- /dev/null\n> >> +++ b/.gitlab-ci/test-lc-compliance.sh\n> >> @@ -0,0 +1,27 @@\n> >> +#!/bin/bash\n> > \n> > Please add an SPDX comment block, see other scripts for examples.\n> > \n> >> +\n> >> +set -e\n> >> +\n> >> +source \"$(dirname \"$0\")/lib.sh\"\n> >> +\n> >> +libcamera_compliance() {\n> >> +\techo \"Running libcamera compliance tests in a qemu VM\"\n> >> +\n> >> +\tvirtme-ng \\\n> >> +\t\t--verbose \\\n> >> +\t\t--skip-modules \\\n> >> +\t\t--force-9p \\\n> >> +\t\t--rwdir \"$PWD/build\" \\\n> >> +\t\t--run /opt/linux/bzImage \\\n> >> +\t\t--exec \"LIBCAMERA_LOG_LEVELS=*:DEBUG ./build/src/apps/lc-compliance/lc-compliance -c Virtual0; echo \\\\\\$? > ./build/.test-status\"\n> >> +\n> >> +\tlocal status=$(cat build/.test-status)\n> >> +\techo \"Test result exit state: $status\"\n> >> +\trm build/.test-status\n> >> +\n> >> +\tif [[ $status != 0 ]] ; then\n> >> +\t\texit $status\n> >> +\tfi\n> >> +}\n> >> +\n> >> +run libcamera_compliance\n> >> diff --git a/gitlab-ci.yml b/gitlab-ci.yml\n> >> index c7448b8..5e9e309 100644\n> >> --- a/gitlab-ci.yml\n> >> +++ b/gitlab-ci.yml\n> >> @@ -382,9 +382,10 @@ build-test:debug:\n> >> -D cam=disabled\n> >> -D cpp_debugstl=true\n> >> -D documentation=disabled\n> >> + -D force_fallback_for=gtest\n> >> -D gstreamer=enabled\n> >> - -D lc-compliance=disabled\n> >> - -D pipelines=[]\n> >> + -D lc-compliance=enabled\n> >> + -D pipelines=virtual\n> > \n> > -D pipelines=[virtual]\n> \n> I am fairly certain `pipelines=virtual` should work, or is this\n> a coding style preference?\n\nIt's a style preference, as the option is an array. I should have made\nthat clearer, sorry.\n\n> As far as I see tings are not consistent\n> unfortunately, `build-libcamera-cros.sh` does not use the full\n> array notation, while the other occurrences do.\n\nMaybe something to fix ? :-)\n\n> > but I think that's not strictly needed, as the virtual pipeline handler\n> > should be selected automatically when the test option is enabled.\n> \n> You're right. So should I remove it?\n\nI'm fine dropping it (or to be exact keeping \"-D pipelines=[]\").\n\n> >> -D pycamera=enabled\n> >> -D qcam=disabled\n> >> -D test=true\n> >> @@ -420,3 +421,16 @@ test-unit:\n> >> # artifacts:\n> >> # reports:\n> >> # junit: build/meson-logs/testlog.junit.xml\n> >> +\n> >> +lc-compliance:virtual:\n> >> + extends:\n> >> + - .fdo.distribution-image@debian\n> >> + - .libcamera-ci.debian:12\n> >> + - .libcamera-ci.scripts\n> >> + stage: test\n> >> + needs:\n> >> + - job: build-test:debug\n> >> + tags:\n> >> + - kvm\n> >> + script:\n> >> + - $CI_PROJECT_DIR/.gitlab-ci/test-lc-compliance.sh\n> > \n> > I think gtest support generating junit reports. Can we do that, and\n> > package the report in artifacts ?\n> \n> Indeed, it can:\n> https://gitlab.freedesktop.org/pobrn/libcamera/-/pipelines/1330595/test_report?job_name=lc-compliance%3Avirtual","headers":{"Return-Path":"","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id 84762C32F6\n\tfor ;\n\tMon, 16 Dec 2024 11:06:28 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id 33A5B67F5B;\n\tMon, 16 Dec 2024 12:06:28 +0100 (CET)","from perceval.ideasonboard.com (perceval.ideasonboard.com\n\t[213.167.242.64])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id 7665D67F37\n\tfor ;\n\tMon, 16 Dec 2024 12:06:26 +0100 (CET)","from pendragon.ideasonboard.com (81-175-209-231.bb.dnainternet.fi\n\t[81.175.209.231])\n\tby perceval.ideasonboard.com (Postfix) with ESMTPSA id E114813C;\n\tMon, 16 Dec 2024 12:05:49 +0100 (CET)"],"Authentication-Results":"lancelot.ideasonboard.com; dkim=pass (1024-bit key;\n\tunprotected) header.d=ideasonboard.com header.i=@ideasonboard.com\n\theader.b=\"eGKw8uXt\"; dkim-atps=neutral","DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com;\n\ts=mail; t=1734347150;\n\tbh=g83VO/GZg3Avog90DeZbFexrRGsDp1ijf1OLhcfAqFg=;\n\th=Date:From:To:Cc:Subject:References:In-Reply-To:From;\n\tb=eGKw8uXtKPhzgRYYZH6hMIzOOdmLlTQWG18IFnOcTBYku4uCjy12Wi/x8nw8iSRkk\n\tJoLi59Uj7upDwQ82PWOMl0RR/Gdzcb5A7e5FtDtxDnpq6TYUAhHRr1qorjePr+HlNy\n\t0TyxIt4PQS7HUEU5W5Cc1WFxVTHiO22+AP65PoE0=","Date":"Mon, 16 Dec 2024 13:06:10 +0200","From":"Laurent Pinchart ","To":"=?utf-8?q?Barnab=C3=A1s_P=C5=91cze?= ","Cc":"libcamera-devel@lists.libcamera.org","Subject":"Re: [libcamera-ci] [RFC PATCH v1 3/3] Add job to run lc-compliance\n\ton the virtual pipeline handler","Message-ID":"<20241216110610.GI32204@pendragon.ideasonboard.com>","References":"<20241212181655.112958-1-barnabas.pocze@ideasonboard.com>\n\t<20241212181655.112958-3-barnabas.pocze@ideasonboard.com>\n\t<20241215192108.GM9975@pendragon.ideasonboard.com>\n\t","MIME-Version":"1.0","Content-Type":"text/plain; charset=utf-8","Content-Disposition":"inline","Content-Transfer-Encoding":"8bit","In-Reply-To":"","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" "}},{"id":32761,"web_url":"https://patchwork.libcamera.org/comment/32761/","msgid":"<173434748449.543771.15850414210516484730@ping.linuxembedded.co.uk>","date":"2024-12-16T11:11:24","subject":"Re: [libcamera-ci] [RFC PATCH v1 3/3] Add job to run lc-compliance\n\ton the virtual pipeline handler","submitter":{"id":4,"url":"https://patchwork.libcamera.org/api/people/4/","name":"Kieran Bingham","email":"kieran.bingham@ideasonboard.com"},"content":"Quoting Laurent Pinchart (2024-12-16 11:06:10)\n> On Mon, Dec 16, 2024 at 11:15:33AM +0100, Barnabás Pőcze wrote:\n> > 2024. 12. 15. 20:21 keltezéssel, Laurent Pinchart írta:\n> > > On Thu, Dec 12, 2024 at 07:16:55PM +0100, Barnabás Pőcze wrote:\n> > >> Add a new job \"lc-compliance:virtual\" that uses the build artifacts\n> > >> produced by the \"build-test:debug\" job to run lc-compliance in a\n> > >> virtual machine on the \"Virtual0\" camera.\n> > >>\n> > >> The `force_fallback_for=gtest` option is needed because `cpp_debustl`\n> > >> makes ABI incompatible changes in the STL containers, so googletest\n> > >> also needs to be compiled with these changes.\n> > >>\n> > >> Signed-off-by: Barnabás Pőcze \n> > >> ---\n> > >> .gitlab-ci/test-lc-compliance.sh | 27 +++++++++++++++++++++++++++\n> > >> gitlab-ci.yml | 18 ++++++++++++++++--\n> > >> 2 files changed, 43 insertions(+), 2 deletions(-)\n> > >> create mode 100755 .gitlab-ci/test-lc-compliance.sh\n> > >>\n> > >> diff --git a/.gitlab-ci/test-lc-compliance.sh b/.gitlab-ci/test-lc-compliance.sh\n> > >> new file mode 100755\n> > >> index 0000000..e6697a6\n> > >> --- /dev/null\n> > >> +++ b/.gitlab-ci/test-lc-compliance.sh\n> > >> @@ -0,0 +1,27 @@\n> > >> +#!/bin/bash\n> > > \n> > > Please add an SPDX comment block, see other scripts for examples.\n> > > \n> > >> +\n> > >> +set -e\n> > >> +\n> > >> +source \"$(dirname \"$0\")/lib.sh\"\n> > >> +\n> > >> +libcamera_compliance() {\n> > >> + echo \"Running libcamera compliance tests in a qemu VM\"\n> > >> +\n> > >> + virtme-ng \\\n> > >> + --verbose \\\n> > >> + --skip-modules \\\n> > >> + --force-9p \\\n> > >> + --rwdir \"$PWD/build\" \\\n> > >> + --run /opt/linux/bzImage \\\n> > >> + --exec \"LIBCAMERA_LOG_LEVELS=*:DEBUG ./build/src/apps/lc-compliance/lc-compliance -c Virtual0; echo \\\\\\$? > ./build/.test-status\"\n> > >> +\n> > >> + local status=$(cat build/.test-status)\n> > >> + echo \"Test result exit state: $status\"\n> > >> + rm build/.test-status\n> > >> +\n> > >> + if [[ $status != 0 ]] ; then\n> > >> + exit $status\n> > >> + fi\n> > >> +}\n> > >> +\n> > >> +run libcamera_compliance\n> > >> diff --git a/gitlab-ci.yml b/gitlab-ci.yml\n> > >> index c7448b8..5e9e309 100644\n> > >> --- a/gitlab-ci.yml\n> > >> +++ b/gitlab-ci.yml\n> > >> @@ -382,9 +382,10 @@ build-test:debug:\n> > >> -D cam=disabled\n> > >> -D cpp_debugstl=true\n> > >> -D documentation=disabled\n> > >> + -D force_fallback_for=gtest\n> > >> -D gstreamer=enabled\n> > >> - -D lc-compliance=disabled\n> > >> - -D pipelines=[]\n> > >> + -D lc-compliance=enabled\n> > >> + -D pipelines=virtual\n> > > \n> > > -D pipelines=[virtual]\n> > \n> > I am fairly certain `pipelines=virtual` should work, or is this\n> > a coding style preference?\n> \n> It's a style preference, as the option is an array. I should have made\n> that clearer, sorry.\n> \n> > As far as I see tings are not consistent\n> > unfortunately, `build-libcamera-cros.sh` does not use the full\n> > array notation, while the other occurrences do.\n> \n> Maybe something to fix ? :-)\n> \n> > > but I think that's not strictly needed, as the virtual pipeline handler\n> > > should be selected automatically when the test option is enabled.\n> > \n> > You're right. So should I remove it?\n> \n> I'm fine dropping it (or to be exact keeping \"-D pipelines=[]\").\n> \n> > >> -D pycamera=enabled\n> > >> -D qcam=disabled\n> > >> -D test=true\n> > >> @@ -420,3 +421,16 @@ test-unit:\n> > >> # artifacts:\n> > >> # reports:\n> > >> # junit: build/meson-logs/testlog.junit.xml\n> > >> +\n> > >> +lc-compliance:virtual:\n> > >> + extends:\n> > >> + - .fdo.distribution-image@debian\n> > >> + - .libcamera-ci.debian:12\n> > >> + - .libcamera-ci.scripts\n> > >> + stage: test\n> > >> + needs:\n> > >> + - job: build-test:debug\n> > >> + tags:\n> > >> + - kvm\n> > >> + script:\n> > >> + - $CI_PROJECT_DIR/.gitlab-ci/test-lc-compliance.sh\n> > > \n> > > I think gtest support generating junit reports. Can we do that, and\n> > > package the report in artifacts ?\n> > \n> > Indeed, it can:\n> > https://gitlab.freedesktop.org/pobrn/libcamera/-/pipelines/1330595/test_report?job_name=lc-compliance%3Avirtual\n\nOh - wow - Is there any way we can get the CTS results pulled into this\nin the future too ?\n\n--\nKieran","headers":{"Return-Path":"","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id 4F51EC32F9\n\tfor ;\n\tMon, 16 Dec 2024 11:11:29 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id 90CC767F5B;\n\tMon, 16 Dec 2024 12:11:28 +0100 (CET)","from perceval.ideasonboard.com (perceval.ideasonboard.com\n\t[213.167.242.64])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id 0A87267F37\n\tfor ;\n\tMon, 16 Dec 2024 12:11:27 +0100 (CET)","from pendragon.ideasonboard.com\n\t(cpc89244-aztw30-2-0-cust6594.18-1.cable.virginm.net [86.31.185.195])\n\tby perceval.ideasonboard.com (Postfix) with ESMTPSA id 842C813C;\n\tMon, 16 Dec 2024 12:10:50 +0100 (CET)"],"Authentication-Results":"lancelot.ideasonboard.com; dkim=pass (1024-bit key;\n\tunprotected) header.d=ideasonboard.com header.i=@ideasonboard.com\n\theader.b=\"Y7i/5m3o\"; dkim-atps=neutral","DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com;\n\ts=mail; t=1734347450;\n\tbh=GsNFwK4uV+NxW1F4WhzotzUaCnRKol/lSqhVAx8rSp4=;\n\th=In-Reply-To:References:Subject:From:Cc:To:Date:From;\n\tb=Y7i/5m3ogWvBRlx+rVvk5N24p+kvTClyz0rEvd4JQok2wiMganVwRZecuUL9wSRDk\n\tmfGXtFIGaKQCKYzZTg5I8ip0X8OagUxw+iB/nllRsSkhC5wwNIPt77t3YvJeumeBbq\n\tW8hfoQwAQER3IBEYrgwOZC9IcX8yGs+HLun99ckY=","Content-Type":"text/plain; charset=\"utf-8\"","MIME-Version":"1.0","Content-Transfer-Encoding":"quoted-printable","In-Reply-To":"<20241216110610.GI32204@pendragon.ideasonboard.com>","References":"<20241212181655.112958-1-barnabas.pocze@ideasonboard.com>\n\t<20241212181655.112958-3-barnabas.pocze@ideasonboard.com>\n\t<20241215192108.GM9975@pendragon.ideasonboard.com>\n\t\n\t<20241216110610.GI32204@pendragon.ideasonboard.com>","Subject":"Re: [libcamera-ci] [RFC PATCH v1 3/3] Add job to run lc-compliance\n\ton the virtual pipeline handler","From":"Kieran Bingham ","Cc":"libcamera-devel@lists.libcamera.org","To":"=?utf-8?q?Barnab=C3=A1s_P=C5=91cze?= ,\n\tLaurent Pinchart ","Date":"Mon, 16 Dec 2024 11:11:24 +0000","Message-ID":"<173434748449.543771.15850414210516484730@ping.linuxembedded.co.uk>","User-Agent":"alot/0.10","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" "}}]