[libcamera-ci,RFC,v3,4/4] Add job to run lc-compliance on the virtual pipeline handler
diff mbox series

Message ID 20241217153153.140417-4-barnabas.pocze@ideasonboard.com
State Superseded
Headers show
Series
  • [libcamera-ci,RFC,v3,1/4] Enable `UDMABUF` in the kernel
Related show

Commit Message

Barnabás Pőcze Dec. 17, 2024, 3:31 p.m. UTC 
Add a new job named `test-lc-compliance:virtual` that uses the
build artifacts created by the `build-package:debug` job
on amd64 to run `lc-compliance` on the "Virtual0" camera
in a virtual machine.

Signed-off-by: Barnabás Pőcze <barnabas.pocze@ideasonboard.com>
---

This needs libcamera changes to work and to run in reasonable time:
  * https://patchwork.libcamera.org/project/libcamera/list/?series=4874
  * https://patchwork.libcamera.org/patch/22379/
  * https://gitlab.freedesktop.org/pobrn/libcamera/-/commit/ed4b5bc179ea405f2f01b85b4114720f67ce3595

Changes in v2:
  * publish lc-compliance results to gitlab
  * add and use script to install libcamera binary package

---
 .gitlab-ci/test-lc-compliance.sh  | 37 +++++++++++++++++++++++++++++++
 .gitlab-ci/unpackage-libcamera.sh | 17 ++++++++++++++
 gitlab-ci.yml                     | 22 +++++++++++++++++-
 3 files changed, 75 insertions(+), 1 deletion(-)
 create mode 100755 .gitlab-ci/test-lc-compliance.sh
 create mode 100755 .gitlab-ci/unpackage-libcamera.sh

--
2.47.1

Comments

Laurent Pinchart Dec. 17, 2024, 4:41 p.m. UTC | #1 
Hi Barnabás,

Thank you for the patch.

On Tue, Dec 17, 2024 at 04:31:53PM +0100, Barnabás Pőcze wrote:
> Add a new job named `test-lc-compliance:virtual` that uses the
> build artifacts created by the `build-package:debug` job
> on amd64 to run `lc-compliance` on the "Virtual0" camera
> in a virtual machine.
> 
> Signed-off-by: Barnabás Pőcze <barnabas.pocze@ideasonboard.com>

Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>

> ---
> 
> This needs libcamera changes to work and to run in reasonable time:
>   * https://patchwork.libcamera.org/project/libcamera/list/?series=4874
>   * https://patchwork.libcamera.org/patch/22379/
>   * https://gitlab.freedesktop.org/pobrn/libcamera/-/commit/ed4b5bc179ea405f2f01b85b4114720f67ce3595

I've reviewed the first two links. I think you can already push patches
1/4 to 3/4 from this series, until we finish discussing how to handle
the configuration file.

> Changes in v2:
>   * publish lc-compliance results to gitlab
>   * add and use script to install libcamera binary package
> 
> ---
>  .gitlab-ci/test-lc-compliance.sh  | 37 +++++++++++++++++++++++++++++++
>  .gitlab-ci/unpackage-libcamera.sh | 17 ++++++++++++++
>  gitlab-ci.yml                     | 22 +++++++++++++++++-
>  3 files changed, 75 insertions(+), 1 deletion(-)
>  create mode 100755 .gitlab-ci/test-lc-compliance.sh
>  create mode 100755 .gitlab-ci/unpackage-libcamera.sh
> 
> diff --git a/.gitlab-ci/test-lc-compliance.sh b/.gitlab-ci/test-lc-compliance.sh
> new file mode 100755
> index 0000000..8c726fa
> --- /dev/null
> +++ b/.gitlab-ci/test-lc-compliance.sh
> @@ -0,0 +1,37 @@
> +#!/bin/bash
> +
> +# SPDX-License-Identifier: GPL-2.0-or-later
> +# SPDX-FileCopyrightText: © 2024 Google Inc.
> +
> +set -e
> +
> +source "$(dirname "$0")/lib.sh"
> +
> +libcamera_compliance() {
> +	echo "Running libcamera compliance tests in a qemu VM"
> +
> +	virtme-ng \
> +		--verbose \
> +		--skip-modules \
> +		--force-9p \
> +		--rwdir "$PWD/build" \
> +		--run /opt/linux/bzImage \
> +		--exec " \
> +			LIBCAMERA_LOG_LEVELS=*:DEBUG \
> +			ASAN_OPTIONS=halt_on_error=1:abort_on_error=1:print_summary=1 \
> +			UBSAN_OPTIONS=halt_on_error=1:abort_on_error=1:print_summary=1:print_stacktrace=1 \
> +			GTEST_OUTPUT=xml:./build/lc-compliance-report.xml \
> +			lc-compliance -c Virtual0; \
> +			echo \\\$? > ./build/.test-status; \
> +		"
> +
> +	local status=$(cat build/.test-status)
> +	echo "Test result exit state: $status"
> +	rm build/.test-status
> +
> +	if [[ $status != 0 ]] ; then
> +		exit $status
> +	fi
> +}
> +
> +run libcamera_compliance
> diff --git a/.gitlab-ci/unpackage-libcamera.sh b/.gitlab-ci/unpackage-libcamera.sh
> new file mode 100755
> index 0000000..9eb3192
> --- /dev/null
> +++ b/.gitlab-ci/unpackage-libcamera.sh
> @@ -0,0 +1,17 @@
> +#!/bin/bash
> +
> +# SPDX-License-Identifier: GPL-2.0-or-later
> +# SPDX-FileCopyrightText: © 2024 Google Inc.
> +
> +set -e
> +
> +source "$(dirname "$0")/lib.sh"
> +
> +libcamera_unpackage() {
> +	echo "Unpackage libcamera binaries"
> +
> +	tar -xvf libcamera-${CI_COMMIT_SHA}.tar.xz -C /
> +	ldconfig -v
> +}
> +
> +run libcamera_unpackage
> diff --git a/gitlab-ci.yml b/gitlab-ci.yml
> index 205c6ff..701d02f 100644
> --- a/gitlab-ci.yml
> +++ b/gitlab-ci.yml
> @@ -273,7 +273,7 @@ build-package:debug:
>        -D b_sanitize=address,undefined
>        -D cpp_debugstl=true
>        -D force_fallback_for=['gtest']
> -      -D pipelines=['auto']
> +      -D pipelines=['auto','virtual']
>        -D auto_features=disabled
>        -D lc-compliance=enabled
>        -D test=false
> @@ -408,3 +408,23 @@ test-unit:
>    # artifacts:
>    #   reports:
>    #     junit: build/meson-logs/testlog.junit.xml
> +
> +test-lc-compliance:virtual:
> +  extends:
> +    - .fdo.distribution-image@debian
> +    - .libcamera-ci.debian:12
> +    - .libcamera-ci.scripts
> +  stage: test
> +  needs:
> +    - job: build-package:debug
> +      parallel:
> +        matrix:
> +          - ARCH: amd64
> +  tags:
> +    - kvm
> +  script:
> +    - $CI_PROJECT_DIR/.gitlab-ci/unpackage-libcamera.sh
> +    - $CI_PROJECT_DIR/.gitlab-ci/test-lc-compliance.sh
> +  artifacts:
> +    reports:
> +      junit: build/lc-compliance-report.xml
Barnabás Pőcze Dec. 18, 2024, 12:51 p.m. UTC | #2 
2024. 12. 17. 17:41 keltezéssel, Laurent Pinchart írta:
> Hi Barnabás,
> 
> Thank you for the patch.
> 
> On Tue, Dec 17, 2024 at 04:31:53PM +0100, Barnabás Pőcze wrote:
>> Add a new job named `test-lc-compliance:virtual` that uses the
>> build artifacts created by the `build-package:debug` job
>> on amd64 to run `lc-compliance` on the "Virtual0" camera
>> in a virtual machine.
>>
>> Signed-off-by: Barnabás Pőcze <barnabas.pocze@ideasonboard.com>
> 
> Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
> 
>> ---
>>
>> This needs libcamera changes to work and to run in reasonable time:
>>    * https://patchwork.libcamera.org/project/libcamera/list/?series=4874
>>    * https://patchwork.libcamera.org/patch/22379/
>>    * https://gitlab.freedesktop.org/pobrn/libcamera/-/commit/ed4b5bc179ea405f2f01b85b4114720f67ce3595
> 
> I've reviewed the first two links. I think you can already push patches
> 1/4 to 3/4 from this series, until we finish discussing how to handle
> the configuration file.

Done.


Regards,
Barnabás Pőcze

> 
>> Changes in v2:
>>    * publish lc-compliance results to gitlab
>>    * add and use script to install libcamera binary package
>>
>> ---
>>   .gitlab-ci/test-lc-compliance.sh  | 37 +++++++++++++++++++++++++++++++
>>   .gitlab-ci/unpackage-libcamera.sh | 17 ++++++++++++++
>>   gitlab-ci.yml                     | 22 +++++++++++++++++-
>>   3 files changed, 75 insertions(+), 1 deletion(-)
>>   create mode 100755 .gitlab-ci/test-lc-compliance.sh
>>   create mode 100755 .gitlab-ci/unpackage-libcamera.sh
>>
>> diff --git a/.gitlab-ci/test-lc-compliance.sh b/.gitlab-ci/test-lc-compliance.sh
>> new file mode 100755
>> index 0000000..8c726fa
>> --- /dev/null
>> +++ b/.gitlab-ci/test-lc-compliance.sh
>> @@ -0,0 +1,37 @@
>> +#!/bin/bash
>> +
>> +# SPDX-License-Identifier: GPL-2.0-or-later
>> +# SPDX-FileCopyrightText: © 2024 Google Inc.
>> +
>> +set -e
>> +
>> +source "$(dirname "$0")/lib.sh"
>> +
>> +libcamera_compliance() {
>> +	echo "Running libcamera compliance tests in a qemu VM"
>> +
>> +	virtme-ng \
>> +		--verbose \
>> +		--skip-modules \
>> +		--force-9p \
>> +		--rwdir "$PWD/build" \
>> +		--run /opt/linux/bzImage \
>> +		--exec " \
>> +			LIBCAMERA_LOG_LEVELS=*:DEBUG \
>> +			ASAN_OPTIONS=halt_on_error=1:abort_on_error=1:print_summary=1 \
>> +			UBSAN_OPTIONS=halt_on_error=1:abort_on_error=1:print_summary=1:print_stacktrace=1 \
>> +			GTEST_OUTPUT=xml:./build/lc-compliance-report.xml \
>> +			lc-compliance -c Virtual0; \
>> +			echo \\\$? > ./build/.test-status; \
>> +		"
>> +
>> +	local status=$(cat build/.test-status)
>> +	echo "Test result exit state: $status"
>> +	rm build/.test-status
>> +
>> +	if [[ $status != 0 ]] ; then
>> +		exit $status
>> +	fi
>> +}
>> +
>> +run libcamera_compliance
>> diff --git a/.gitlab-ci/unpackage-libcamera.sh b/.gitlab-ci/unpackage-libcamera.sh
>> new file mode 100755
>> index 0000000..9eb3192
>> --- /dev/null
>> +++ b/.gitlab-ci/unpackage-libcamera.sh
>> @@ -0,0 +1,17 @@
>> +#!/bin/bash
>> +
>> +# SPDX-License-Identifier: GPL-2.0-or-later
>> +# SPDX-FileCopyrightText: © 2024 Google Inc.
>> +
>> +set -e
>> +
>> +source "$(dirname "$0")/lib.sh"
>> +
>> +libcamera_unpackage() {
>> +	echo "Unpackage libcamera binaries"
>> +
>> +	tar -xvf libcamera-${CI_COMMIT_SHA}.tar.xz -C /
>> +	ldconfig -v
>> +}
>> +
>> +run libcamera_unpackage
>> diff --git a/gitlab-ci.yml b/gitlab-ci.yml
>> index 205c6ff..701d02f 100644
>> --- a/gitlab-ci.yml
>> +++ b/gitlab-ci.yml
>> @@ -273,7 +273,7 @@ build-package:debug:
>>         -D b_sanitize=address,undefined
>>         -D cpp_debugstl=true
>>         -D force_fallback_for=['gtest']
>> -      -D pipelines=['auto']
>> +      -D pipelines=['auto','virtual']
>>         -D auto_features=disabled
>>         -D lc-compliance=enabled
>>         -D test=false
>> @@ -408,3 +408,23 @@ test-unit:
>>     # artifacts:
>>     #   reports:
>>     #     junit: build/meson-logs/testlog.junit.xml
>> +
>> +test-lc-compliance:virtual:
>> +  extends:
>> +    - .fdo.distribution-image@debian
>> +    - .libcamera-ci.debian:12
>> +    - .libcamera-ci.scripts
>> +  stage: test
>> +  needs:
>> +    - job: build-package:debug
>> +      parallel:
>> +        matrix:
>> +          - ARCH: amd64
>> +  tags:
>> +    - kvm
>> +  script:
>> +    - $CI_PROJECT_DIR/.gitlab-ci/unpackage-libcamera.sh
>> +    - $CI_PROJECT_DIR/.gitlab-ci/test-lc-compliance.sh
>> +  artifacts:
>> +    reports:
>> +      junit: build/lc-compliance-report.xml
>

Patch
diff mbox series 

diff --git a/.gitlab-ci/test-lc-compliance.sh b/.gitlab-ci/test-lc-compliance.sh
new file mode 100755
index 0000000..8c726fa
--- /dev/null
+++ b/.gitlab-ci/test-lc-compliance.sh
@@ -0,0 +1,37 @@ 
+#!/bin/bash
+
+# SPDX-License-Identifier: GPL-2.0-or-later
+# SPDX-FileCopyrightText: © 2024 Google Inc.
+
+set -e
+
+source "$(dirname "$0")/lib.sh"
+
+libcamera_compliance() {
+	echo "Running libcamera compliance tests in a qemu VM"
+
+	virtme-ng \
+		--verbose \
+		--skip-modules \
+		--force-9p \
+		--rwdir "$PWD/build" \
+		--run /opt/linux/bzImage \
+		--exec " \
+			LIBCAMERA_LOG_LEVELS=*:DEBUG \
+			ASAN_OPTIONS=halt_on_error=1:abort_on_error=1:print_summary=1 \
+			UBSAN_OPTIONS=halt_on_error=1:abort_on_error=1:print_summary=1:print_stacktrace=1 \
+			GTEST_OUTPUT=xml:./build/lc-compliance-report.xml \
+			lc-compliance -c Virtual0; \
+			echo \\\$? > ./build/.test-status; \
+		"
+
+	local status=$(cat build/.test-status)
+	echo "Test result exit state: $status"
+	rm build/.test-status
+
+	if [[ $status != 0 ]] ; then
+		exit $status
+	fi
+}
+
+run libcamera_compliance
diff --git a/.gitlab-ci/unpackage-libcamera.sh b/.gitlab-ci/unpackage-libcamera.sh
new file mode 100755
index 0000000..9eb3192
--- /dev/null
+++ b/.gitlab-ci/unpackage-libcamera.sh
@@ -0,0 +1,17 @@ 
+#!/bin/bash
+
+# SPDX-License-Identifier: GPL-2.0-or-later
+# SPDX-FileCopyrightText: © 2024 Google Inc.
+
+set -e
+
+source "$(dirname "$0")/lib.sh"
+
+libcamera_unpackage() {
+	echo "Unpackage libcamera binaries"
+
+	tar -xvf libcamera-${CI_COMMIT_SHA}.tar.xz -C /
+	ldconfig -v
+}
+
+run libcamera_unpackage
diff --git a/gitlab-ci.yml b/gitlab-ci.yml
index 205c6ff..701d02f 100644
--- a/gitlab-ci.yml
+++ b/gitlab-ci.yml
@@ -273,7 +273,7 @@  build-package:debug:
       -D b_sanitize=address,undefined
       -D cpp_debugstl=true
       -D force_fallback_for=['gtest']
-      -D pipelines=['auto']
+      -D pipelines=['auto','virtual']
       -D auto_features=disabled
       -D lc-compliance=enabled
       -D test=false
@@ -408,3 +408,23 @@  test-unit:
   # artifacts:
   #   reports:
   #     junit: build/meson-logs/testlog.junit.xml
+
+test-lc-compliance:virtual:
+  extends:
+    - .fdo.distribution-image@debian
+    - .libcamera-ci.debian:12
+    - .libcamera-ci.scripts
+  stage: test
+  needs:
+    - job: build-package:debug
+      parallel:
+        matrix:
+          - ARCH: amd64
+  tags:
+    - kvm
+  script:
+    - $CI_PROJECT_DIR/.gitlab-ci/unpackage-libcamera.sh
+    - $CI_PROJECT_DIR/.gitlab-ci/test-lc-compliance.sh
+  artifacts:
+    reports:
+      junit: build/lc-compliance-report.xml