[v5,00/12] Enable raw streams with software ISP
mbox series

Message ID 20250520123158.44237-1-mzamazal@redhat.com
Headers show
Series
  • Enable raw streams with software ISP
Related show

Message

Milan Zamazal May 20, 2025, 12:31 p.m. UTC 
This makes raw streams working again in ‘simple’ pipeline when software
ISP is enabled for the given device.  At most one raw stream and one
processed stream (possibly both at once) are supported.

An example ‘cam’ invocation requesting a raw stream rather than a debayered stream:

  cam -c1 -C8 -s role=raw,width=1920,height=1080 -Ffile#.raw

Or for both raw and processed streams:

  cam -c1 -C8 -Ffile# \
    -s role=viewfinder,width=1920,height=1080,pixelformat=RGB888 \
    -s role=raw,width=3280,height=2464,pixelformat=SRGGB8 \

When only a raw stream is requested, there are no exposure/gain
adjustments applied.  This could be improved in future, once software
ISP gets a mechanism to gather image statistics without processing and
using them to make the adjustments, or once manual exposure controls are
added to software ISP.  In the meantime, exposure must be changed
externally.

Tested-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Tested-by: Robert Mader<robert.mader@collabora.com>

Changes in v5:
- Possible temporary segfault in the patch adding plain output
  configurations avoided.
- PixelFormatInfo::isRaw() helper added.
- SimplePipelineHandler::setUpFormatSizes replaced with a lambda
  function.
- SimpleCameraData::{rawRequested_,processedRequested_} are set in
  SimpleCameraConfiguration first and copied to SimpleCameraData only
  after successful configure().

Changes in v4:
- Broken range pruning due to passing a value rather than a reference
  fixed.
- New common function isFormatRaw introduced.
- The patch assigning colour spaces in the simple pipeline, previously
  posted separately, included in this series, as the first patch.  It
  can still be handled separately; in any case the rest sort of depends
  on it.
- Setting metadataRequired to false where needed to prevent freezes and
  assertion errors; related to metadata reporting support merged to
  master since v3.

Changes in v3:
- Significantly reworked, with both functional and clarity improvements.
  The level of guesswork and confusion is hopefully reduced enough now
  to drop the RFC prefix.
- The number of streams is set to 2 only with software ISP.
- SimpleCameraData::pipeConfig_ nullptr check patch dropped.
- PPM/raw file output patch dropped from this series.  Let’s handle this
  separately as the patch series is already complex enough.

Changes in v2:
- Completely reworked.
- Extended to be able to produce a raw stream together with a processed
  stream.

Milan Zamazal (12):
  libcamera: software_isp: Assign colour spaces in configurations
  libcamera: simple: Set the number of software ISP streams to 2
  libcamera: formats: Add a helper to check for a raw pixel format
  libcamera: formats: Add a helper to check for a raw pixel info
  libcamera: simple: Don't use raw output formats with conversions
  libcamera: simple: Add plain output configurations
  libcamera: simple: Identify requested stream roles
  libcamera: simple: Consider raw output configurations
  libcamera: simple: Validate raw stream configurations
  libcamera: simple: Don't enforce conversion with an added raw stream
  libcamera: simple: Require metadata only when software ISP is used
  libcamera: simple: Make raw streams working

 include/libcamera/internal/formats.h          |   3 +
 src/libcamera/formats.cpp                     |  16 ++
 src/libcamera/pipeline/imx8-isi/imx8-isi.cpp  |  11 +-
 src/libcamera/pipeline/ipu3/ipu3.cpp          |   4 +-
 src/libcamera/pipeline/mali-c55/mali-c55.cpp  |  10 -
 src/libcamera/pipeline/rkisp1/rkisp1.cpp      |  11 +-
 src/libcamera/pipeline/rkisp1/rkisp1_path.cpp |   7 +-
 src/libcamera/pipeline/simple/simple.cpp      | 261 +++++++++++++-----
 8 files changed, 231 insertions(+), 92 deletions(-)