[v3,0/8] Enable raw streams with software ISP
mbox series

Message ID 20250305192617.18361-1-mzamazal@redhat.com
Headers show
Series
  • Enable raw streams with software ISP
Related show

Message

Milan Zamazal March 5, 2025, 7:26 p.m. UTC
This makes raw streams working again in ‘simple’ pipeline when software
ISP is enabled for the given device.  At most one raw stream and one
processed stream (possibly both at once) are supported.

An example ‘cam’ invocation requesting a raw stream rather than a debayered stream:

  cam -c1 -C8 -s role=raw,width=1920,height=1080 -Ffile#.raw

Or for both raw and processed streams:

  cam -c1 -C8 -Ffile# \
    -s role=viewfinder,width=1920,height=1080,pixelformat=RGB888 \
    -s role=raw,width=3280,height=2464,pixelformat=SRGGB8 \

When only a raw stream is requested, there are no exposure/gain
adjustments applied.  This could be improved in future, once software
ISP gets a mechanism to gather image statistics without processing and
using them to make the adjustments, or once manual exposure controls are
added to software ISP.  In the meantime, exposure must be changed
externally.

Changes in v3:
- Significantly reworked, with both functional and clarity improvements.
  The level of guesswork and confusion is hopefully reduced enough now
  to drop the RFC prefix.
- The number of streams is set to 2 only with software ISP.
- SimpleCameraData::pipeConfig_ nullptr check patch dropped.
- PPM/raw file output patch dropped from this series.  Let’s handle this
  separately as the patch series is already complex enough.

Changes in v2:
- Completely reworked.
- Extended to be able to produce a raw stream together with a processed
  stream.

Milan Zamazal (8):
  libcamera: simple: Set the number of software ISP streams to 2
  libcamera: simple: Don't use raw output formats with conversions
  libcamera: simple: Add plain output configurations
  libcamera: simple: Identify requested stream roles
  libcamera: simple: Consider raw output configurations
  libcamera: simple: Validate raw stream configurations
  libcamera: simple: Don't enforce conversion with an added raw stream
  libcamera: simple: Make raw streams working

 src/libcamera/pipeline/simple/simple.cpp | 231 +++++++++++++++++------
 1 file changed, 173 insertions(+), 58 deletions(-)

Comments

Robert Mader March 26, 2025, 3:17 p.m. UTC | #1
This series does not apply on master any more - can you send out a 
rebased version? Thanks!

FTR.: now that we have the CCM support in, we'll need this series to 
capture raw reference images (with camshark) in order to create tuning 
files.

On 05.03.25 20:26, Milan Zamazal wrote:
> This makes raw streams working again in ‘simple’ pipeline when software
> ISP is enabled for the given device.  At most one raw stream and one
> processed stream (possibly both at once) are supported.
>
> An example ‘cam’ invocation requesting a raw stream rather than a debayered stream:
>
>    cam -c1 -C8 -s role=raw,width=1920,height=1080 -Ffile#.raw
>
> Or for both raw and processed streams:
>
>    cam -c1 -C8 -Ffile# \
>      -s role=viewfinder,width=1920,height=1080,pixelformat=RGB888 \
>      -s role=raw,width=3280,height=2464,pixelformat=SRGGB8 \
>
> When only a raw stream is requested, there are no exposure/gain
> adjustments applied.  This could be improved in future, once software
> ISP gets a mechanism to gather image statistics without processing and
> using them to make the adjustments, or once manual exposure controls are
> added to software ISP.  In the meantime, exposure must be changed
> externally.
>
> Changes in v3:
> - Significantly reworked, with both functional and clarity improvements.
>    The level of guesswork and confusion is hopefully reduced enough now
>    to drop the RFC prefix.
> - The number of streams is set to 2 only with software ISP.
> - SimpleCameraData::pipeConfig_ nullptr check patch dropped.
> - PPM/raw file output patch dropped from this series.  Let’s handle this
>    separately as the patch series is already complex enough.
>
> Changes in v2:
> - Completely reworked.
> - Extended to be able to produce a raw stream together with a processed
>    stream.
>
> Milan Zamazal (8):
>    libcamera: simple: Set the number of software ISP streams to 2
>    libcamera: simple: Don't use raw output formats with conversions
>    libcamera: simple: Add plain output configurations
>    libcamera: simple: Identify requested stream roles
>    libcamera: simple: Consider raw output configurations
>    libcamera: simple: Validate raw stream configurations
>    libcamera: simple: Don't enforce conversion with an added raw stream
>    libcamera: simple: Make raw streams working
>
>   src/libcamera/pipeline/simple/simple.cpp | 231 +++++++++++++++++------
>   1 file changed, 173 insertions(+), 58 deletions(-)
>
Milan Zamazal March 26, 2025, 4:38 p.m. UTC | #2
Robert Mader <robert.mader@collabora.com> writes:

> This series does not apply on master any more - can you send out a rebased version? Thanks!

Hi Robert,

are you sure?  It (https://patchwork.libcamera.org/series/5047/mbox/)
applies cleanly for me on master with `git am'.

(I can post a rebased version anyway but in order to not spam the list
I'd prefer first looking at the problem Kieran reported and also see
what happens with the colour space patch, which is somewhat related.)

> FTR.: now that we have the CCM support in, we'll need this series to capture raw reference images (with
> camshark) in order to create tuning files.
>
> On 05.03.25 20:26, Milan Zamazal wrote:
>> This makes raw streams working again in ‘simple’ pipeline when software
>> ISP is enabled for the given device.  At most one raw stream and one
>> processed stream (possibly both at once) are supported.
>>
>> An example ‘cam’ invocation requesting a raw stream rather than a debayered stream:
>>
>>    cam -c1 -C8 -s role=raw,width=1920,height=1080 -Ffile#.raw
>>
>> Or for both raw and processed streams:
>>
>>    cam -c1 -C8 -Ffile# \
>>      -s role=viewfinder,width=1920,height=1080,pixelformat=RGB888 \
>>      -s role=raw,width=3280,height=2464,pixelformat=SRGGB8 \
>>
>> When only a raw stream is requested, there are no exposure/gain
>> adjustments applied.  This could be improved in future, once software
>> ISP gets a mechanism to gather image statistics without processing and
>> using them to make the adjustments, or once manual exposure controls are
>> added to software ISP.  In the meantime, exposure must be changed
>> externally.
>>
>> Changes in v3:
>> - Significantly reworked, with both functional and clarity improvements.
>>    The level of guesswork and confusion is hopefully reduced enough now
>>    to drop the RFC prefix.
>> - The number of streams is set to 2 only with software ISP.
>> - SimpleCameraData::pipeConfig_ nullptr check patch dropped.
>> - PPM/raw file output patch dropped from this series.  Let’s handle this
>>    separately as the patch series is already complex enough.
>>
>> Changes in v2:
>> - Completely reworked.
>> - Extended to be able to produce a raw stream together with a processed
>>    stream.
>>
>> Milan Zamazal (8):
>>    libcamera: simple: Set the number of software ISP streams to 2
>>    libcamera: simple: Don't use raw output formats with conversions
>>    libcamera: simple: Add plain output configurations
>>    libcamera: simple: Identify requested stream roles
>>    libcamera: simple: Consider raw output configurations
>>    libcamera: simple: Validate raw stream configurations
>>    libcamera: simple: Don't enforce conversion with an added raw stream
>>    libcamera: simple: Make raw streams working
>>
>>   src/libcamera/pipeline/simple/simple.cpp | 231 +++++++++++++++++------
>>   1 file changed, 173 insertions(+), 58 deletions(-)
>>
Robert Mader March 26, 2025, 4:57 p.m. UTC | #3
Whops, you are right, my branch wasn't clean 🤦

Sorry for the noise, will give it a try now :)

On 26.03.25 17:38, Milan Zamazal wrote:
> Robert Mader <robert.mader@collabora.com> writes:
>
>> This series does not apply on master any more - can you send out a rebased version? Thanks!
> Hi Robert,
>
> are you sure?  It (https://patchwork.libcamera.org/series/5047/mbox/)
> applies cleanly for me on master with `git am'.
>
> (I can post a rebased version anyway but in order to not spam the list
> I'd prefer first looking at the problem Kieran reported and also see
> what happens with the colour space patch, which is somewhat related.)
>
>> FTR.: now that we have the CCM support in, we'll need this series to capture raw reference images (with
>> camshark) in order to create tuning files.
>>
>> On 05.03.25 20:26, Milan Zamazal wrote:
>>> This makes raw streams working again in ‘simple’ pipeline when software
>>> ISP is enabled for the given device.  At most one raw stream and one
>>> processed stream (possibly both at once) are supported.
>>>
>>> An example ‘cam’ invocation requesting a raw stream rather than a debayered stream:
>>>
>>>     cam -c1 -C8 -s role=raw,width=1920,height=1080 -Ffile#.raw
>>>
>>> Or for both raw and processed streams:
>>>
>>>     cam -c1 -C8 -Ffile# \
>>>       -s role=viewfinder,width=1920,height=1080,pixelformat=RGB888 \
>>>       -s role=raw,width=3280,height=2464,pixelformat=SRGGB8 \
>>>
>>> When only a raw stream is requested, there are no exposure/gain
>>> adjustments applied.  This could be improved in future, once software
>>> ISP gets a mechanism to gather image statistics without processing and
>>> using them to make the adjustments, or once manual exposure controls are
>>> added to software ISP.  In the meantime, exposure must be changed
>>> externally.
>>>
>>> Changes in v3:
>>> - Significantly reworked, with both functional and clarity improvements.
>>>     The level of guesswork and confusion is hopefully reduced enough now
>>>     to drop the RFC prefix.
>>> - The number of streams is set to 2 only with software ISP.
>>> - SimpleCameraData::pipeConfig_ nullptr check patch dropped.
>>> - PPM/raw file output patch dropped from this series.  Let’s handle this
>>>     separately as the patch series is already complex enough.
>>>
>>> Changes in v2:
>>> - Completely reworked.
>>> - Extended to be able to produce a raw stream together with a processed
>>>     stream.
>>>
>>> Milan Zamazal (8):
>>>     libcamera: simple: Set the number of software ISP streams to 2
>>>     libcamera: simple: Don't use raw output formats with conversions
>>>     libcamera: simple: Add plain output configurations
>>>     libcamera: simple: Identify requested stream roles
>>>     libcamera: simple: Consider raw output configurations
>>>     libcamera: simple: Validate raw stream configurations
>>>     libcamera: simple: Don't enforce conversion with an added raw stream
>>>     libcamera: simple: Make raw streams working
>>>
>>>    src/libcamera/pipeline/simple/simple.cpp | 231 +++++++++++++++++------
>>>    1 file changed, 173 insertions(+), 58 deletions(-)
>>>