[0/7] tests: Separate suites and check software ISP crops
mbox series

Message ID 20261016202916.1405813-1-halfline@gmail.com
Headers show
Series
  • tests: Separate suites and check software ISP crops
Related show

Message

Ray Strode Oct. 16, 2026, 8:29 p.m. UTC
Hi all,

Following Hans's suggestion, this series adds opt-in integration tests for
the software ISP's rendered output and pixel statistics. It applies on top
of the crop and ScalerCrop metadata v3 [1].

The first three patches classify the existing tests into unit and
integration suites, add matching test-unit and test-integration build
options, and document their use. The existing test option enables both.

The remaining patches add software ISP backend checks. The rendering test
compares resized output with a centered crop of the native render. The
statistics test checks sample counts, symmetry, and pixel inclusion and
exclusion at crop boundaries. Both cover 8-bit and packed RAW10 input with
two Bayer patterns.

A llvmpipe setup runs the rendering check with surfaceless Mesa software
rendering and treats an unavailable EGL display as a failure. The guide
includes the build and test commands for CI runners.

These tests exercise the rendering and statistics components. Camera
capture, DMA-BUF transport, synchronization and request metadata need
separate integration coverage.

[1] https://patchwork.libcamera.org/project/libcamera/list/?series=6258

Thanks,
Ray

Ray Strode (7):
  tests: Separate unit and integration suites
  tests: Select unit and integration suites independently
  docs: Describe unit and integration test selection
  software-isp: Allow drawing into existing EGL textures
  tests: Validate rendered crops in the integration suite
  tests: Validate pixel statistics for the visible GPU crop
  docs: Describe software ISP integration testing

 Documentation/index.rst                       |   1 +
 Documentation/meson.build                     |   1 +
 Documentation/testing.rst                     |  88 +++++++
 meson.build                                   |  12 +-
 meson_options.txt                             |  12 +-
 src/ipa/meson.build                           |   6 +-
 src/libcamera/software_isp/debayer_egl.cpp    |  25 +-
 src/libcamera/software_isp/debayer_egl.h      |   3 +
 test/camera/meson.build                       |   6 +-
 test/controls/meson.build                     |  10 +-
 test/gstreamer/meson.build                    |   6 +-
 test/integration/meson.build                  |   3 +
 test/integration/software_isp/gpu-crop.cpp    | 222 ++++++++++++++++++
 test/integration/software_isp/gpu-stats.cpp   | 208 ++++++++++++++++
 test/integration/software_isp/meson.build     |  29 +++
 test/ipa/libipa/meson.build                   |   6 +-
 test/ipa/meson.build                          |   6 +-
 test/ipc/meson.build                          |   8 +-
 test/log/meson.build                          |   6 +-
 test/media_device/meson.build                 |   6 +-
 test/meson.build                              |  45 +++-
 test/process/meson.build                      |   6 +-
 test/py/meson.build                           |   6 +-
 .../generated_serializer/meson.build          |   6 +-
 test/serialization/meson.build                |   6 +-
 test/software_isp/meson.build                 |   4 +-
 test/stream/meson.build                       |   6 +-
 test/v4l2_compat/meson.build                  |   6 +-
 test/v4l2_subdevice/meson.build               |   6 +-
 test/v4l2_videodevice/meson.build             |   6 +-
 30 files changed, 720 insertions(+), 41 deletions(-)
 create mode 100644 Documentation/testing.rst
 create mode 100644 test/integration/meson.build
 create mode 100644 test/integration/software_isp/gpu-crop.cpp
 create mode 100644 test/integration/software_isp/gpu-stats.cpp
 create mode 100644 test/integration/software_isp/meson.build


base-commit: a92285e9c8f8d5aba24c6f72f49efc73074002e1
prerequisite-patch-id: 0d61bd9859ee3467227b46e4efe6e57af196a936
prerequisite-patch-id: b3c35fc8290ae5bbf07b94c3b7ecfe9c935b3b81
prerequisite-patch-id: 4ffb815e04da5a91649cce459957e4b958a52241
prerequisite-patch-id: 428d5ac131fc74e9290f8023c08e39b7b38574cc
prerequisite-patch-id: c69c25c0fd6999abf23cec7d2f06e19ecde53d14
prerequisite-patch-id: 42fadde8a27a403757b7e7905c6506b950e22392
prerequisite-patch-id: f9034f957c59d9942b59ac6fcafc5170c4dca8e8
prerequisite-patch-id: a78fb89885d8f2ab927ba46ee0f154a4f81d8e4e
prerequisite-patch-id: 7529c539479ef6110d882c2fba231e6ce9a0044e
prerequisite-patch-id: 50d4c66d468591ccc351d53f56e1a660d22ac80a
prerequisite-patch-id: 89101d95b5934715c5d325c8e23e9b3b99aa1903

Comments

Laurent Pinchart Oct. 16, 2026, 8:36 p.m. UTC | #1
On Fri, Oct 16, 2026 at 04:29:09PM -0400, Ray Strode wrote:
> Hi all,
> 
> Following Hans's suggestion, this series adds opt-in integration tests for
> the software ISP's rendered output and pixel statistics. It applies on top
> of the crop and ScalerCrop metadata v3 [1].

Nack to AI slop, sorry.

> The first three patches classify the existing tests into unit and
> integration suites, add matching test-unit and test-integration build
> options, and document their use. The existing test option enables both.
> 
> The remaining patches add software ISP backend checks. The rendering test
> compares resized output with a centered crop of the native render. The
> statistics test checks sample counts, symmetry, and pixel inclusion and
> exclusion at crop boundaries. Both cover 8-bit and packed RAW10 input with
> two Bayer patterns.
> 
> A llvmpipe setup runs the rendering check with surfaceless Mesa software
> rendering and treats an unavailable EGL display as a failure. The guide
> includes the build and test commands for CI runners.
> 
> These tests exercise the rendering and statistics components. Camera
> capture, DMA-BUF transport, synchronization and request metadata need
> separate integration coverage.
> 
> [1] https://patchwork.libcamera.org/project/libcamera/list/?series=6258
> 
> Thanks,
> Ray
> 
> Ray Strode (7):
>   tests: Separate unit and integration suites
>   tests: Select unit and integration suites independently
>   docs: Describe unit and integration test selection
>   software-isp: Allow drawing into existing EGL textures
>   tests: Validate rendered crops in the integration suite
>   tests: Validate pixel statistics for the visible GPU crop
>   docs: Describe software ISP integration testing
> 
>  Documentation/index.rst                       |   1 +
>  Documentation/meson.build                     |   1 +
>  Documentation/testing.rst                     |  88 +++++++
>  meson.build                                   |  12 +-
>  meson_options.txt                             |  12 +-
>  src/ipa/meson.build                           |   6 +-
>  src/libcamera/software_isp/debayer_egl.cpp    |  25 +-
>  src/libcamera/software_isp/debayer_egl.h      |   3 +
>  test/camera/meson.build                       |   6 +-
>  test/controls/meson.build                     |  10 +-
>  test/gstreamer/meson.build                    |   6 +-
>  test/integration/meson.build                  |   3 +
>  test/integration/software_isp/gpu-crop.cpp    | 222 ++++++++++++++++++
>  test/integration/software_isp/gpu-stats.cpp   | 208 ++++++++++++++++
>  test/integration/software_isp/meson.build     |  29 +++
>  test/ipa/libipa/meson.build                   |   6 +-
>  test/ipa/meson.build                          |   6 +-
>  test/ipc/meson.build                          |   8 +-
>  test/log/meson.build                          |   6 +-
>  test/media_device/meson.build                 |   6 +-
>  test/meson.build                              |  45 +++-
>  test/process/meson.build                      |   6 +-
>  test/py/meson.build                           |   6 +-
>  .../generated_serializer/meson.build          |   6 +-
>  test/serialization/meson.build                |   6 +-
>  test/software_isp/meson.build                 |   4 +-
>  test/stream/meson.build                       |   6 +-
>  test/v4l2_compat/meson.build                  |   6 +-
>  test/v4l2_subdevice/meson.build               |   6 +-
>  test/v4l2_videodevice/meson.build             |   6 +-
>  30 files changed, 720 insertions(+), 41 deletions(-)
>  create mode 100644 Documentation/testing.rst
>  create mode 100644 test/integration/meson.build
>  create mode 100644 test/integration/software_isp/gpu-crop.cpp
>  create mode 100644 test/integration/software_isp/gpu-stats.cpp
>  create mode 100644 test/integration/software_isp/meson.build
> 
> 
> base-commit: a92285e9c8f8d5aba24c6f72f49efc73074002e1
> prerequisite-patch-id: 0d61bd9859ee3467227b46e4efe6e57af196a936
> prerequisite-patch-id: b3c35fc8290ae5bbf07b94c3b7ecfe9c935b3b81
> prerequisite-patch-id: 4ffb815e04da5a91649cce459957e4b958a52241
> prerequisite-patch-id: 428d5ac131fc74e9290f8023c08e39b7b38574cc
> prerequisite-patch-id: c69c25c0fd6999abf23cec7d2f06e19ecde53d14
> prerequisite-patch-id: 42fadde8a27a403757b7e7905c6506b950e22392
> prerequisite-patch-id: f9034f957c59d9942b59ac6fcafc5170c4dca8e8
> prerequisite-patch-id: a78fb89885d8f2ab927ba46ee0f154a4f81d8e4e
> prerequisite-patch-id: 7529c539479ef6110d882c2fba231e6ce9a0044e
> prerequisite-patch-id: 50d4c66d468591ccc351d53f56e1a660d22ac80a
> prerequisite-patch-id: 89101d95b5934715c5d325c8e23e9b3b99aa1903
Ray Strode Oct. 16, 2026, 9:10 p.m. UTC | #2
Hi,

On Fri, Oct 16, 2026 at 4:36 PM Laurent Pinchart
<laurent.pinchart@ideasonboard.com> wrote:
> > Following Hans's suggestion, this series adds opt-in integration tests for
> > the software ISP's rendered output and pixel statistics. It applies on top
> > of the crop and ScalerCrop metadata v3 [1].
>
> Nack ...

I took your earlier "not a unit test" feedback as a reason to separate
the existing non-unit
tests into an integration suite, but I recognize that's a broader
change to the test infrastructure
than we'd discussed and it may seem like a bit of an overcorrection.

I'm happy to drop this series for now and revisit imaging tests later
once a design is worked out
for the best way to do those kinds of tests.

--Ray