[libcamera-ci,RFC,v2,0/5] on-device-testing proof of concept
mbox series

Message ID 20260130160254.1770742-1-barnabas.pocze@ideasonboard.com
Headers show
Series
  • on-device-testing proof of concept
Related show

Message

Barnabás Pőcze Jan. 30, 2026, 4:02 p.m. UTC
This adds the necessary gitlab parts for running lc-compliance via
lava on a "real" device, specifically a debix-a board with an imx219
sensor.

There are three parts to the process:

1. libcamera is cross-compiled to the target architecture.
   This uses a separate container, and does not use setup-container.sh.

2. A debian root file system is built, appropriate for running
   libcamera as compiled in (1).

3. LAVA job is submitted to the lava worker to run `test-libcamera.sh`
   on the device.

Couple points:

 * Deploying the debian root filesystem is a manual process. It must be built and
   uploaded to the lava worker's local container registry. Specifying which container
   image the device should use is not implemented at the moment, it is hard-coded on
   the device itself.

   I think be very nice if this could be fully automatic.

 * lava-gitlab-runner can only use files from artifacts and not from
   the repository (https://github.com/collabora/lava-gitlab-runner/issues/16),
   so there is a job `lava-jobs-as-artifacts` for that. Although even if it
   could, the libcamera ci definitions are in a separate repository, so that
   would most likely still not work.

 * The cross compilation jobs saves the url of the debian package artifact
   in a `dotenv` artifact report, for lava-gitlab-runner to access it, and for
   it to be accessible on the device to download and install the package.

   This is needed because there does not appear to be a way to download job
   artifacts based on the (pipeline id, job name) tuple, same applies to getting
   the job id from another job. At least without more complex use of the gitlab api.

   But this means that the lava job must get the artifacts from the cross compilation
   job, which unfortunately means downloading the deb package unnecessarily.

 * The camera id is hard-coded in the lava job definition. Maybe it could be better
   to extend lc-compliance to support camera indices.

 * dependencies:
     - https://patchwork.libcamera.org/project/libcamera/list/?series=5756
     - https://patchwork.libcamera.org/patch/26038/

---
changes in v2:
  * use existing containers
  * manual rootfs workflow

v1: https://patchwork.libcamera.org/cover/25839/
---
Barnabás Pőcze (5):
  Install `fpm` in the debian 13 container
  Add job to build deb package for libcamera
  Add job to run lc-compliance via lava
  Guard hardware testing jobs
  Add description about debian rootfs setup for testing

 .gitlab-ci/on-device-testing/lava-debix-a.yml | 40 ++++++++
 .gitlab-ci/setup-container.sh                 |  6 ++
 .../debian-rootfs/build-debian-rootfs.sh      | 62 ++++++++++++
 containers/debian-rootfs/build.sh             |  9 ++
 .../overlay/opt/test-libcamera.sh             | 18 ++++
 doc/debian-rootfs.rst                         | 47 +++++++++
 gitlab-ci.yml                                 | 95 ++++++++++++++++++-
 7 files changed, 276 insertions(+), 1 deletion(-)
 create mode 100644 .gitlab-ci/on-device-testing/lava-debix-a.yml
 create mode 100755 containers/debian-rootfs/build-debian-rootfs.sh
 create mode 100755 containers/debian-rootfs/build.sh
 create mode 100755 containers/debian-rootfs/overlay/opt/test-libcamera.sh
 create mode 100644 doc/debian-rootfs.rst

--
2.52.0

Comments

Laurent Pinchart Feb. 5, 2026, 1:28 p.m. UTC | #1
On Fri, Jan 30, 2026 at 05:02:49PM +0100, Barnabás Pőcze wrote:
> This adds the necessary gitlab parts for running lc-compliance via
> lava on a "real" device, specifically a debix-a board with an imx219
> sensor.
> 
> There are three parts to the process:
> 
> 1. libcamera is cross-compiled to the target architecture.
>    This uses a separate container, and does not use setup-container.sh.
> 
> 2. A debian root file system is built, appropriate for running
>    libcamera as compiled in (1).
> 
> 3. LAVA job is submitted to the lava worker to run `test-libcamera.sh`
>    on the device.
> 
> Couple points:
> 
>  * Deploying the debian root filesystem is a manual process. It must be built and
>    uploaded to the lava worker's local container registry. Specifying which container
>    image the device should use is not implemented at the moment, it is hard-coded on
>    the device itself.
> 
>    I think be very nice if this could be fully automatic.
> 
>  * lava-gitlab-runner can only use files from artifacts and not from
>    the repository (https://github.com/collabora/lava-gitlab-runner/issues/16),
>    so there is a job `lava-jobs-as-artifacts` for that. Although even if it
>    could, the libcamera ci definitions are in a separate repository, so that
>    would most likely still not work.
> 
>  * The cross compilation jobs saves the url of the debian package artifact
>    in a `dotenv` artifact report, for lava-gitlab-runner to access it, and for
>    it to be accessible on the device to download and install the package.
> 
>    This is needed because there does not appear to be a way to download job
>    artifacts based on the (pipeline id, job name) tuple, same applies to getting
>    the job id from another job. At least without more complex use of the gitlab api.
> 
>    But this means that the lava job must get the artifacts from the cross compilation
>    job, which unfortunately means downloading the deb package unnecessarily.

Yes that's annoying. The lava runner has some serious limitations, I
don't understand how this behaviour hasn't annoyed other people enough
to fix the problem :-)

>  * The camera id is hard-coded in the lava job definition. Maybe it could be better
>    to extend lc-compliance to support camera indices.

Or possibly pipeline handlers, if the goal is to test a particular
pipeline handler.

>  * dependencies:
>      - https://patchwork.libcamera.org/project/libcamera/list/?series=5756
>      - https://patchwork.libcamera.org/patch/26038/
> 
> ---
> changes in v2:
>   * use existing containers
>   * manual rootfs workflow
> 
> v1: https://patchwork.libcamera.org/cover/25839/
> ---
> Barnabás Pőcze (5):
>   Install `fpm` in the debian 13 container
>   Add job to build deb package for libcamera
>   Add job to run lc-compliance via lava
>   Guard hardware testing jobs
>   Add description about debian rootfs setup for testing
> 
>  .gitlab-ci/on-device-testing/lava-debix-a.yml | 40 ++++++++
>  .gitlab-ci/setup-container.sh                 |  6 ++
>  .../debian-rootfs/build-debian-rootfs.sh      | 62 ++++++++++++
>  containers/debian-rootfs/build.sh             |  9 ++
>  .../overlay/opt/test-libcamera.sh             | 18 ++++
>  doc/debian-rootfs.rst                         | 47 +++++++++
>  gitlab-ci.yml                                 | 95 ++++++++++++++++++-
>  7 files changed, 276 insertions(+), 1 deletion(-)
>  create mode 100644 .gitlab-ci/on-device-testing/lava-debix-a.yml
>  create mode 100755 containers/debian-rootfs/build-debian-rootfs.sh
>  create mode 100755 containers/debian-rootfs/build.sh
>  create mode 100755 containers/debian-rootfs/overlay/opt/test-libcamera.sh
>  create mode 100644 doc/debian-rootfs.rst
Barnabás Pőcze Feb. 9, 2026, 10:39 a.m. UTC | #2
2026. 02. 05. 14:28 keltezéssel, Laurent Pinchart írta:
> On Fri, Jan 30, 2026 at 05:02:49PM +0100, Barnabás Pőcze wrote:
>> This adds the necessary gitlab parts for running lc-compliance via
>> lava on a "real" device, specifically a debix-a board with an imx219
>> sensor.
>>
>> There are three parts to the process:
>>
>> 1. libcamera is cross-compiled to the target architecture.
>>     This uses a separate container, and does not use setup-container.sh.
>>
>> 2. A debian root file system is built, appropriate for running
>>     libcamera as compiled in (1).
>>
>> 3. LAVA job is submitted to the lava worker to run `test-libcamera.sh`
>>     on the device.
>>
>> Couple points:
>>
>>   * Deploying the debian root filesystem is a manual process. It must be built and
>>     uploaded to the lava worker's local container registry. Specifying which container
>>     image the device should use is not implemented at the moment, it is hard-coded on
>>     the device itself.
>>
>>     I think be very nice if this could be fully automatic.
>>
>>   * lava-gitlab-runner can only use files from artifacts and not from
>>     the repository (https://github.com/collabora/lava-gitlab-runner/issues/16),
>>     so there is a job `lava-jobs-as-artifacts` for that. Although even if it
>>     could, the libcamera ci definitions are in a separate repository, so that
>>     would most likely still not work.
>>
>>   * The cross compilation jobs saves the url of the debian package artifact
>>     in a `dotenv` artifact report, for lava-gitlab-runner to access it, and for
>>     it to be accessible on the device to download and install the package.
>>
>>     This is needed because there does not appear to be a way to download job
>>     artifacts based on the (pipeline id, job name) tuple, same applies to getting
>>     the job id from another job. At least without more complex use of the gitlab api.
>>
>>     But this means that the lava job must get the artifacts from the cross compilation
>>     job, which unfortunately means downloading the deb package unnecessarily.
> 
> Yes that's annoying. The lava runner has some serious limitations, I
> don't understand how this behaviour hasn't annoyed other people enough
> to fix the problem :-)

I suppose the question is: does it annoy us enough?


> 
>>   * The camera id is hard-coded in the lava job definition. Maybe it could be better
>>     to extend lc-compliance to support camera indices.
> 
> Or possibly pipeline handlers, if the goal is to test a particular
> pipeline handler.

That information is not available yet, no? So it would be a an API addition as well?


> 
>>   * dependencies:
>>       - https://patchwork.libcamera.org/project/libcamera/list/?series=5756
>>       - https://patchwork.libcamera.org/patch/26038/
>>
>> ---
>> changes in v2:
>>    * use existing containers
>>    * manual rootfs workflow
>>
>> v1: https://patchwork.libcamera.org/cover/25839/
>> ---
>> Barnabás Pőcze (5):
>>    Install `fpm` in the debian 13 container
>>    Add job to build deb package for libcamera
>>    Add job to run lc-compliance via lava
>>    Guard hardware testing jobs
>>    Add description about debian rootfs setup for testing
>>
>>   .gitlab-ci/on-device-testing/lava-debix-a.yml | 40 ++++++++
>>   .gitlab-ci/setup-container.sh                 |  6 ++
>>   .../debian-rootfs/build-debian-rootfs.sh      | 62 ++++++++++++
>>   containers/debian-rootfs/build.sh             |  9 ++
>>   .../overlay/opt/test-libcamera.sh             | 18 ++++
>>   doc/debian-rootfs.rst                         | 47 +++++++++
>>   gitlab-ci.yml                                 | 95 ++++++++++++++++++-
>>   7 files changed, 276 insertions(+), 1 deletion(-)
>>   create mode 100644 .gitlab-ci/on-device-testing/lava-debix-a.yml
>>   create mode 100755 containers/debian-rootfs/build-debian-rootfs.sh
>>   create mode 100755 containers/debian-rootfs/build.sh
>>   create mode 100755 containers/debian-rootfs/overlay/opt/test-libcamera.sh
>>   create mode 100644 doc/debian-rootfs.rst
>
Laurent Pinchart Feb. 9, 2026, 11:03 a.m. UTC | #3
On Mon, Feb 09, 2026 at 11:39:03AM +0100, Barnabás Pőcze wrote:
> 2026. 02. 05. 14:28 keltezéssel, Laurent Pinchart írta:
> > On Fri, Jan 30, 2026 at 05:02:49PM +0100, Barnabás Pőcze wrote:
> >> This adds the necessary gitlab parts for running lc-compliance via
> >> lava on a "real" device, specifically a debix-a board with an imx219
> >> sensor.
> >>
> >> There are three parts to the process:
> >>
> >> 1. libcamera is cross-compiled to the target architecture.
> >>     This uses a separate container, and does not use setup-container.sh.
> >>
> >> 2. A debian root file system is built, appropriate for running
> >>     libcamera as compiled in (1).
> >>
> >> 3. LAVA job is submitted to the lava worker to run `test-libcamera.sh`
> >>     on the device.
> >>
> >> Couple points:
> >>
> >>   * Deploying the debian root filesystem is a manual process. It must be built and
> >>     uploaded to the lava worker's local container registry. Specifying which container
> >>     image the device should use is not implemented at the moment, it is hard-coded on
> >>     the device itself.
> >>
> >>     I think be very nice if this could be fully automatic.
> >>
> >>   * lava-gitlab-runner can only use files from artifacts and not from
> >>     the repository (https://github.com/collabora/lava-gitlab-runner/issues/16),
> >>     so there is a job `lava-jobs-as-artifacts` for that. Although even if it
> >>     could, the libcamera ci definitions are in a separate repository, so that
> >>     would most likely still not work.
> >>
> >>   * The cross compilation jobs saves the url of the debian package artifact
> >>     in a `dotenv` artifact report, for lava-gitlab-runner to access it, and for
> >>     it to be accessible on the device to download and install the package.
> >>
> >>     This is needed because there does not appear to be a way to download job
> >>     artifacts based on the (pipeline id, job name) tuple, same applies to getting
> >>     the job id from another job. At least without more complex use of the gitlab api.
> >>
> >>     But this means that the lava job must get the artifacts from the cross compilation
> >>     job, which unfortunately means downloading the deb package unnecessarily.
> > 
> > Yes that's annoying. The lava runner has some serious limitations, I
> > don't understand how this behaviour hasn't annoyed other people enough
> > to fix the problem :-)
> 
> I suppose the question is: does it annoy us enough?

Maybe this falls in the category of those issues that everybody would
love to see being fixed by someone else.

> >>   * The camera id is hard-coded in the lava job definition. Maybe it could be better
> >>     to extend lc-compliance to support camera indices.
> > 
> > Or possibly pipeline handlers, if the goal is to test a particular
> > pipeline handler.
> 
> That information is not available yet, no? So it would be a an API addition as well?

Yes, it would be an API addition.

> >>   * dependencies:
> >>       - https://patchwork.libcamera.org/project/libcamera/list/?series=5756
> >>       - https://patchwork.libcamera.org/patch/26038/
> >>
> >> ---
> >> changes in v2:
> >>    * use existing containers
> >>    * manual rootfs workflow
> >>
> >> v1: https://patchwork.libcamera.org/cover/25839/
> >> ---
> >> Barnabás Pőcze (5):
> >>    Install `fpm` in the debian 13 container
> >>    Add job to build deb package for libcamera
> >>    Add job to run lc-compliance via lava
> >>    Guard hardware testing jobs
> >>    Add description about debian rootfs setup for testing
> >>
> >>   .gitlab-ci/on-device-testing/lava-debix-a.yml | 40 ++++++++
> >>   .gitlab-ci/setup-container.sh                 |  6 ++
> >>   .../debian-rootfs/build-debian-rootfs.sh      | 62 ++++++++++++
> >>   containers/debian-rootfs/build.sh             |  9 ++
> >>   .../overlay/opt/test-libcamera.sh             | 18 ++++
> >>   doc/debian-rootfs.rst                         | 47 +++++++++
> >>   gitlab-ci.yml                                 | 95 ++++++++++++++++++-
> >>   7 files changed, 276 insertions(+), 1 deletion(-)
> >>   create mode 100644 .gitlab-ci/on-device-testing/lava-debix-a.yml
> >>   create mode 100755 containers/debian-rootfs/build-debian-rootfs.sh
> >>   create mode 100755 containers/debian-rootfs/build.sh
> >>   create mode 100755 containers/debian-rootfs/overlay/opt/test-libcamera.sh
> >>   create mode 100644 doc/debian-rootfs.rst