[libcamera-devel,3/3] ci: gitlab: Provide initial automated testing

Message ID 20200120125544.12562-4-kieran.bingham@ideasonboard.com
State Superseded
Delegated to: Kieran Bingham
Headers show
Series
  • CI: Initial gitlab ci builder
Related show

Commit Message

Kieran Bingham Jan. 20, 2020, 12:55 p.m. UTC
Integrate compile testing and the ninja test framework.
Any unit tests registered with the Meson Build system will automatically
be run with 'ninja test'.

Documentation will be built, and the resulting package will be installed
to 'libcamera/package'

Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>

---
v2:
 - CI: Add Documentation to build artifacts
 - CI: Convert to alpine image
 - CI: test install phase
---
 .gitlab-ci.yml | 18 ++++++++++++++++++
 1 file changed, 18 insertions(+)
 create mode 100644 .gitlab-ci.yml

Comments

Laurent Pinchart Jan. 20, 2020, 5:08 p.m. UTC | #1
Hi Kieran,

Thank you for the patch.

On Mon, Jan 20, 2020 at 12:55:44PM +0000, Kieran Bingham wrote:
> Integrate compile testing and the ninja test framework.
> Any unit tests registered with the Meson Build system will automatically
> be run with 'ninja test'.

I'm not sure when you mean here. The script below doesn't run tests,
does it ?

> Documentation will be built, and the resulting package will be installed
> to 'libcamera/package'
> 
> Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
> 
> ---
> v2:
>  - CI: Add Documentation to build artifacts
>  - CI: Convert to alpine image
>  - CI: test install phase
> ---
>  .gitlab-ci.yml | 18 ++++++++++++++++++
>  1 file changed, 18 insertions(+)
>  create mode 100644 .gitlab-ci.yml
> 
> diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml
> new file mode 100644
> index 000000000000..5851af90e22c
> --- /dev/null
> +++ b/.gitlab-ci.yml
> @@ -0,0 +1,18 @@
> +image: alpine:edge

Any specific reason for this choice ? I suppose it's fine to start with,
and we can then extend to more distributions.

> +
> +build:
> +  stage: build
> +  before_script:
> +    - apk add --no-cache gcc g++ musl-dev meson py3-yaml py3-sphinx linux-headers pkgconfig eudev eudev-dev

Should we also depend on Qt to test compilation of qcam ?

Is there an easy way to also test compilation with clang ?

> +  script:
> +    - mkdir libcamera && cd libcamera && meson .. && ninja

How about enabling the android and V4L2 compatibility layers ?

> +    - ninja Documentation/linkcheck
> +    - DESTDIR=$PWD/package ninja install
> +  artifacts:
> +    expire_in: 6 hrs
> +    paths:
> +      - libcamera/Documentation/*
> +      - libcamera/src/libcamera/libcamera.so
> +      - libcamera/test/test_init
> +      - libcamera/meson-logs/testlog.txt
> +      - libcamera/package/*
Kieran Bingham Jan. 20, 2020, 5:16 p.m. UTC | #2
Hi Laurent,

On 20/01/2020 17:08, Laurent Pinchart wrote:
> Hi Kieran,
> 
> Thank you for the patch.
> 
> On Mon, Jan 20, 2020 at 12:55:44PM +0000, Kieran Bingham wrote:
>> Integrate compile testing and the ninja test framework.
>> Any unit tests registered with the Meson Build system will automatically
>> be run with 'ninja test'.
> 
> I'm not sure when you mean here. The script below doesn't run tests,
> does it ?

Argh, an earlier version of this file did :-)


>> Documentation will be built, and the resulting package will be installed
>> to 'libcamera/package'
>>
>> Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
>>
>> ---
>> v2:
>>  - CI: Add Documentation to build artifacts
>>  - CI: Convert to alpine image
>>  - CI: test install phase
>> ---
>>  .gitlab-ci.yml | 18 ++++++++++++++++++
>>  1 file changed, 18 insertions(+)
>>  create mode 100644 .gitlab-ci.yml
>>
>> diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml
>> new file mode 100644
>> index 000000000000..5851af90e22c
>> --- /dev/null
>> +++ b/.gitlab-ci.yml
>> @@ -0,0 +1,18 @@
>> +image: alpine:edge
> 
> Any specific reason for this choice ? I suppose it's fine to start with,
> and we can then extend to more distributions.

I chose it (a year ago?) to extend coverage for c-libraries. I (we?)
usually build on a glibc target. The alpine image uses Musl. It's also a
smaller image suitable for containers, so it speeds up the build.


I want the build to extend to multiple distributions, and multiple
toolchains. The pushback I received last time I looked at this stopped
me from continuing this path of development/investigation.

Hopefully we can continue to get some extended build coverage.


>> +
>> +build:
>> +  stage: build
>> +  before_script:
>> +    - apk add --no-cache gcc g++ musl-dev meson py3-yaml py3-sphinx linux-headers pkgconfig eudev eudev-dev
> 
> Should we also depend on Qt to test compilation of qcam ?

Yes, we should also build qcam.

I expect this script was created before we had qcam, and I've only
half-maintained it to keep it building what was there.


> Is there an easy way to also test compilation with clang ?

That comes under the category of multiple build matrix support.


>> +  script:
>> +    - mkdir libcamera && cd libcamera && meson .. && ninja
> 
> How about enabling the android and V4L2 compatibility layers ?

Yes, they would be useful too.
Are there external requirements to build the android layer?


>> +    - ninja Documentation/linkcheck
>> +    - DESTDIR=$PWD/package ninja install
>> +  artifacts:
>> +    expire_in: 6 hrs
>> +    paths:
>> +      - libcamera/Documentation/*
>> +      - libcamera/src/libcamera/libcamera.so
>> +      - libcamera/test/test_init

test/test_init looks like an old artefact that should be removed too.

>> +      - libcamera/meson-logs/testlog.txt
>> +      - libcamera/package/*
>
Laurent Pinchart Jan. 20, 2020, 5:21 p.m. UTC | #3
Hi Kieran,

On Mon, Jan 20, 2020 at 05:16:57PM +0000, Kieran Bingham wrote:
> On 20/01/2020 17:08, Laurent Pinchart wrote:
> > On Mon, Jan 20, 2020 at 12:55:44PM +0000, Kieran Bingham wrote:
> >> Integrate compile testing and the ninja test framework.
> >> Any unit tests registered with the Meson Build system will automatically
> >> be run with 'ninja test'.
> > 
> > I'm not sure when you mean here. The script below doesn't run tests,
> > does it ?
> 
> Argh, an earlier version of this file did :-)
> 
> >> Documentation will be built, and the resulting package will be installed
> >> to 'libcamera/package'
> >>
> >> Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
> >>
> >> ---
> >> v2:
> >>  - CI: Add Documentation to build artifacts
> >>  - CI: Convert to alpine image
> >>  - CI: test install phase
> >> ---
> >>  .gitlab-ci.yml | 18 ++++++++++++++++++
> >>  1 file changed, 18 insertions(+)
> >>  create mode 100644 .gitlab-ci.yml
> >>
> >> diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml
> >> new file mode 100644
> >> index 000000000000..5851af90e22c
> >> --- /dev/null
> >> +++ b/.gitlab-ci.yml
> >> @@ -0,0 +1,18 @@
> >> +image: alpine:edge
> > 
> > Any specific reason for this choice ? I suppose it's fine to start with,
> > and we can then extend to more distributions.
> 
> I chose it (a year ago?) to extend coverage for c-libraries. I (we?)
> usually build on a glibc target. The alpine image uses Musl. It's also a
> smaller image suitable for containers, so it speeds up the build.
> 
> I want the build to extend to multiple distributions, and multiple
> toolchains. The pushback I received last time I looked at this stopped
> me from continuing this path of development/investigation.
> 
> Hopefully we can continue to get some extended build coverage.
> 
> >> +
> >> +build:
> >> +  stage: build
> >> +  before_script:
> >> +    - apk add --no-cache gcc g++ musl-dev meson py3-yaml py3-sphinx linux-headers pkgconfig eudev eudev-dev
> > 
> > Should we also depend on Qt to test compilation of qcam ?
> 
> Yes, we should also build qcam.
> 
> I expect this script was created before we had qcam, and I've only
> half-maintained it to keep it building what was there.
> 
> > Is there an easy way to also test compilation with clang ?
> 
> That comes under the category of multiple build matrix support.

OK.

> >> +  script:
> >> +    - mkdir libcamera && cd libcamera && meson .. && ninja
> > 
> > How about enabling the android and V4L2 compatibility layers ?
> 
> Yes, they would be useful too.
> Are there external requirements to build the android layer?

No, it's self-contained.

> >> +    - ninja Documentation/linkcheck
> >> +    - DESTDIR=$PWD/package ninja install
> >> +  artifacts:
> >> +    expire_in: 6 hrs
> >> +    paths:
> >> +      - libcamera/Documentation/*
> >> +      - libcamera/src/libcamera/libcamera.so
> >> +      - libcamera/test/test_init
> 
> test/test_init looks like an old artefact that should be removed too.
> 
> >> +      - libcamera/meson-logs/testlog.txt
> >> +      - libcamera/package/*
Nicolas Dufresne Jan. 20, 2020, 7:22 p.m. UTC | #4
Le lundi 20 janvier 2020 à 12:55 +0000, Kieran Bingham a écrit :
> Integrate compile testing and the ninja test framework.
> Any unit tests registered with the Meson Build system will automatically
> be run with 'ninja test'.
> 
> Documentation will be built, and the resulting package will be installed
> to 'libcamera/package'
> 
> Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
> 
> ---
> v2:
>  - CI: Add Documentation to build artifacts
>  - CI: Convert to alpine image
>  - CI: test install phase
> ---
>  .gitlab-ci.yml | 18 ++++++++++++++++++
>  1 file changed, 18 insertions(+)
>  create mode 100644 .gitlab-ci.yml
> 
> diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml
> new file mode 100644
> index 000000000000..5851af90e22c
> --- /dev/null
> +++ b/.gitlab-ci.yml
> @@ -0,0 +1,18 @@
> +image: alpine:edge
> +
> +build:
> +  stage: build
> +  before_script:
> +    - apk add --no-cache gcc g++ musl-dev meson py3-yaml py3-sphinx linux-

How long does that take and how much network is required each time ? That being
said, it will be fixed if you use ci-templates, as there you can gen and save
your docker.

> headers pkgconfig eudev eudev-dev
> +  script:
> +    - mkdir libcamera && cd libcamera && meson .. && ninja

I'm not sure I like the idea of usign "libcamera" as a build directory. But that
does not seems a big deal. It you be cleaner to use the following invocation
though:

  - meson libcamera
  - ninja -C libcamera

Meson will create that directory for you (alternatively you can use the long
explicit way, meson setup <builddir>). This splits the configure and the build
step properly. It will be easier to add configure options later.

> +    - ninja Documentation/linkcheck
> +    - DESTDIR=$PWD/package ninja install
> +  artifacts:
> +    expire_in: 6 hrs
> +    paths:
> +      - libcamera/Documentation/*
> +      - libcamera/src/libcamera/libcamera.so
> +      - libcamera/test/test_init
> +      - libcamera/meson-logs/testlog.txt
> +      - libcamera/package/*
Laurent Pinchart Jan. 20, 2020, 7:31 p.m. UTC | #5
On Mon, Jan 20, 2020 at 02:22:52PM -0500, Nicolas Dufresne wrote:
> Le lundi 20 janvier 2020 à 12:55 +0000, Kieran Bingham a écrit :
> > Integrate compile testing and the ninja test framework.
> > Any unit tests registered with the Meson Build system will automatically
> > be run with 'ninja test'.
> > 
> > Documentation will be built, and the resulting package will be installed
> > to 'libcamera/package'
> > 
> > Signed-off-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
> > 
> > ---
> > v2:
> >  - CI: Add Documentation to build artifacts
> >  - CI: Convert to alpine image
> >  - CI: test install phase
> > ---
> >  .gitlab-ci.yml | 18 ++++++++++++++++++
> >  1 file changed, 18 insertions(+)
> >  create mode 100644 .gitlab-ci.yml
> > 
> > diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml
> > new file mode 100644
> > index 000000000000..5851af90e22c
> > --- /dev/null
> > +++ b/.gitlab-ci.yml
> > @@ -0,0 +1,18 @@
> > +image: alpine:edge
> > +
> > +build:
> > +  stage: build
> > +  before_script:
> > +    - apk add --no-cache gcc g++ musl-dev meson py3-yaml py3-sphinx linux-
> 
> How long does that take and how much network is required each time ? That being
> said, it will be fixed if you use ci-templates, as there you can gen and save
> your docker.
> 
> > headers pkgconfig eudev eudev-dev
> > +  script:
> > +    - mkdir libcamera && cd libcamera && meson .. && ninja
> 
> I'm not sure I like the idea of usign "libcamera" as a build directory. But that
> does not seems a big deal. It you be cleaner to use the following invocation

I would name it build indeed.

> though:
> 
>   - meson libcamera
>   - ninja -C libcamera
> 
> Meson will create that directory for you (alternatively you can use the long
> explicit way, meson setup <builddir>). This splits the configure and the build
> step properly. It will be easier to add configure options later.

Agreed, I think that's better.

> > +    - ninja Documentation/linkcheck
> > +    - DESTDIR=$PWD/package ninja install
> > +  artifacts:
> > +    expire_in: 6 hrs
> > +    paths:
> > +      - libcamera/Documentation/*
> > +      - libcamera/src/libcamera/libcamera.so
> > +      - libcamera/test/test_init
> > +      - libcamera/meson-logs/testlog.txt
> > +      - libcamera/package/*

Patch

diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml
new file mode 100644
index 000000000000..5851af90e22c
--- /dev/null
+++ b/.gitlab-ci.yml
@@ -0,0 +1,18 @@ 
+image: alpine:edge
+
+build:
+  stage: build
+  before_script:
+    - apk add --no-cache gcc g++ musl-dev meson py3-yaml py3-sphinx linux-headers pkgconfig eudev eudev-dev
+  script:
+    - mkdir libcamera && cd libcamera && meson .. && ninja
+    - ninja Documentation/linkcheck
+    - DESTDIR=$PWD/package ninja install
+  artifacts:
+    expire_in: 6 hrs
+    paths:
+      - libcamera/Documentation/*
+      - libcamera/src/libcamera/libcamera.so
+      - libcamera/test/test_init
+      - libcamera/meson-logs/testlog.txt
+      - libcamera/package/*