[RFC,v3,0/2] libcamera: pipeline: virtual: Add raw Bayer frame support

26720 mbox series

Changes since v2: 
- Use std::get_if instead of holds_alternative + std::get
- Extracted the file collection into collectFiles() helper
- Use BayerFormat::toPixelFormat()
- Drop SoftISP-specific controls and the mention of it

The virtual pipeline handler currently only produces NV12 output, either
from test patterns or JPEG files. This means that SoftISP, which
processes raw Bayer data, cannot be tested without physical camera
hardware. A first step to achieving this would be to support raw streams
in the virtual pipeline handler. This series introduces a RawFrameGenerator
that reads binary raw Bayer files from disk and serves them through the virtual
pipeline. The introduced changes are opt-in through the YAML configuration. If
virtual cameras are configured with raw_frames, the new code path is
chosen, paths for the existing test_patterns and frames are unaffected.

The SoftISP part of this patch's motivation is complemented by David Plowman's
series, which introduced the CameraSensorMemory class
(https://patchwork.libcamera.org/patch/25468/), since SoftISP needs an
actual CameraSensor for correct construction. The idea is that in the
future it can be used together to serve Raw frames through the
virtual pipeline and process them with SoftISP, for example
for CI tests or more easily reproducible benchmarks.

Testing: I have used `dd` to create zero filled raw frames to
test things such as format negotiation, buffer sizing, general frame
flow, config parsing (both right/wrong, e.g. rejecting invalid input)
and bit depth handling. I've written a small bash script to essentially
automate this for all n : { 8, 10, 12, 14, 16 } and the four Bayer
formats in accordance to `src/libcamera/formats.yaml`. I then used numpy to
test the generateFrame() functionality by generating a known frame with
content, capturing it through the pipeline to disk with the cam application
and then reading the output back again to verify that all the pixels match
the expected values.

I noticed that the pipeline handler components currently have no
dedicated test coverage, if I missed them or if there is some preferred
approach for adding additional testing I'd be open to any suggestions.
For now I can also share my scripts if that's wanted.

Ran `git clang-format` as well as the `checkstyle.py`. I've run the
Meson tests both with and without the changes and get the same:
Ok:                 51
Expected Fail:      1
Fail:               1
Unexpected Pass:    0
Skipped:            29
Timeout:            0

Open questions:
- The sensor properties set in match() (ColorFilterArrangement,
  PixelArraySize, UnitCellSize) are currently hardcoded, and I've
  left them with a \todo. Should I move these directly to the yaml as well or proceed differently?
- Currently every frame is read into framesDatas_ immediately at
  startup. Depending on the number and the size of the raw files, this
  could take up quite a lot of memory. For the use cases I thought of
  (e.g. CI) this would be fine, since the number of files would probably
  be comparatively small there. Still I wanted to bring it up.
- We weren't sure if the SPDX license header is fine like this. For the
  moment I've just placed my intern company mail, is this fine as is?

Documentation:
I've added documentation according to the coding-style.rst,
if my comments are not sufficient I'd be happy to add more of course.

AI Disclosure:
I've used DeepWiki to familiarize myself with the libcamera repository
and to point me at reference implementations and patterns to use as orientation.

Thank you all for your time!

Max Bretschneider (2):
  libcamera: pipeline: virtual: Add RawFrameGenerator
  libcamera: pipeline: virtual: Add raw_frames config and Bayer format
    support

 .../pipeline/virtual/config_parser.cpp        | 156 ++++++++++++++----
 src/libcamera/pipeline/virtual/meson.build    |   1 +
 .../pipeline/virtual/raw_frame_generator.cpp  | 131 +++++++++++++++
 .../pipeline/virtual/raw_frame_generator.h    |  47 ++++++
 src/libcamera/pipeline/virtual/virtual.cpp    |  99 +++++++++--
 src/libcamera/pipeline/virtual/virtual.h      |   3 +-
 6 files changed, 389 insertions(+), 48 deletions(-)
 create mode 100644 src/libcamera/pipeline/virtual/raw_frame_generator.cpp
 create mode 100644 src/libcamera/pipeline/virtual/raw_frame_generator.h