[libcamera-devel,v3,00/22] libcamera: pipeline: rkisp1: Extend to support two streams

9798 mbox series

Hi,

This series extends the RkISP1 pipeline handler to work with 2 streams.
It allows cameras backed by this pipeline to expose a main path stream,
same as the one before this series. And a new self path stream that can
operate in addition or independently from the main path.

In previous versions of this series RGB formats where added to the self 
path (new formats not supported by the main path) but further 
examination of those formats resulted in that the captured frames looks 
weird. Pending future investigation drop that work from this series.

Furthermore this series does not extend support for RAW formats as this
is such a special case for this pipeline. Only the main path may be
active and no ISP parameters can be processed. This may be added on top
but functionality have been proven as a prototype.

New in this version are patches 15/22 to 21/22 that adds a new 
RkISP1Path class which abstracts the two paths into a single helper 
class with shared logic. The shared logic can then be leveraged to 
simplify the pipeline handler.

Patch 22/22 is an attempt to use a lambda function to further simplify 
error logic in start(). It is primary provided as a proof-of-concept 
and should be regarded as RFC.

To be able to use both main and self path simultaneously must run a
kernel that includes [1]. The development of the series is tested on the
media-tree [2] with out-of-tree DTS patches for the test Scarlet
platform to enable the camera sensors.

1. 60e915354c270357 ("media: staging: rkisp1: cap: fix return values from pm functions")
2. f45882cfb152f5d3 ("media: camss: fix memory leaks on error handling paths in probe")

Niklas Söderlund (22):
  libcamera: pipeline: rkisp1: Set number of planes based on format
  libcamera: pipeline: rkisp1: Remove redundant check of buffer in
    Request
  libcamera: pipeline: rkisp1: Breakout mainpath size and format
    constraints
  libcamera: pipeline: rkisp1: Setup links as part of configuration
  libcamera: pipeline: rkisp1: Prepare buffer ready handlers for
    multiple streams
  libcamera: pipeline: rkisp1: Create RkISP1Frames from camera data
  libcamera: pipeline: rkisp1: Export stream formats to applications
  libcamera: pipeline: rkisp1: Set the crop rectangle
  libcamera: pipeline: rkisp1: Prefix main path video and resizer
  libcamera: pipeline: rkisp1: Add self path devices
  libcamera: pipeline: rkisp1: Configure self path
  libcamera: pipeline: rkisp1: Track buffers for self path
  libcamera: pipeline: rkisp1: Add format validation for self path
  libcamera: pipeline: rkisp1: Expose self path stream
  libcamera: pipeline: rkisp1: Breakout basic path handling to own class
  libcamera: pipeline: rkisp1: Move path configuration to RkISP1Path
  libcamera: pipeline: rkisp1: Move path configuration generation and
    validation to RkISP1Path
  libcamera: pipeline: rkisp1: Add wrappers for accessing the path video
    device
  libcamera: pipeline: rkisp1: Move start and stop of path to RkISP1Path
  libcamera: pipeline: rkisp1: Move path link handling to RkISP1Path
  libcamera: pipeline: rkisp1: Use the media link to track if a path is
    enabled
  libcamera: pipeline: rkisp1: Use cleanup error paths for start()

 src/libcamera/pipeline/rkisp1/meson.build    |   1 +
 src/libcamera/pipeline/rkisp1/rkisp1.cpp     | 555 +++++++++++--------
 src/libcamera/pipeline/rkisp1/rkisp1path.cpp | 237 ++++++++
 src/libcamera/pipeline/rkisp1/rkisp1path.h   |  83 +++
 4 files changed, 633 insertions(+), 243 deletions(-)
 create mode 100644 src/libcamera/pipeline/rkisp1/rkisp1path.cpp
 create mode 100644 src/libcamera/pipeline/rkisp1/rkisp1path.h

Hi Niklas,

Thank you for the series.

On Fri, Sep 25, 2020 at 03:41:45AM +0200, Niklas Söderlund wrote:
> Hi,
> 
> This series extends the RkISP1 pipeline handler to work with 2 streams.
> It allows cameras backed by this pipeline to expose a main path stream,
> same as the one before this series. And a new self path stream that can
> operate in addition or independently from the main path.
> 
> In previous versions of this series RGB formats where added to the self 
> path (new formats not supported by the main path) but further 
> examination of those formats resulted in that the captured frames looks 
> weird. Pending future investigation drop that work from this series.

Just for my own curiosity, how weird ?

> Furthermore this series does not extend support for RAW formats as this
> is such a special case for this pipeline. Only the main path may be
> active and no ISP parameters can be processed. This may be added on top
> but functionality have been proven as a prototype.

I think adding support for the second ISP would then be an interesting
experiment. Clearly for (much) later :-)

> New in this version are patches 15/22 to 21/22 that adds a new 
> RkISP1Path class which abstracts the two paths into a single helper 
> class with shared logic. The shared logic can then be leveraged to 
> simplify the pipeline handler.

Very nice cleanups, thanks.

> Patch 22/22 is an attempt to use a lambda function to further simplify 
> error logic in start(). It is primary provided as a proof-of-concept 
> and should be regarded as RFC.
> 
> To be able to use both main and self path simultaneously must run a
> kernel that includes [1]. The development of the series is tested on the
> media-tree [2] with out-of-tree DTS patches for the test Scarlet
> platform to enable the camera sensors.
> 
> 1. 60e915354c270357 ("media: staging: rkisp1: cap: fix return values from pm functions")
> 2. f45882cfb152f5d3 ("media: camss: fix memory leaks on error handling paths in probe")
> 
> Niklas Söderlund (22):
>   libcamera: pipeline: rkisp1: Set number of planes based on format
>   libcamera: pipeline: rkisp1: Remove redundant check of buffer in
>     Request
>   libcamera: pipeline: rkisp1: Breakout mainpath size and format
>     constraints
>   libcamera: pipeline: rkisp1: Setup links as part of configuration
>   libcamera: pipeline: rkisp1: Prepare buffer ready handlers for
>     multiple streams
>   libcamera: pipeline: rkisp1: Create RkISP1Frames from camera data
>   libcamera: pipeline: rkisp1: Export stream formats to applications
>   libcamera: pipeline: rkisp1: Set the crop rectangle
>   libcamera: pipeline: rkisp1: Prefix main path video and resizer
>   libcamera: pipeline: rkisp1: Add self path devices
>   libcamera: pipeline: rkisp1: Configure self path
>   libcamera: pipeline: rkisp1: Track buffers for self path
>   libcamera: pipeline: rkisp1: Add format validation for self path
>   libcamera: pipeline: rkisp1: Expose self path stream
>   libcamera: pipeline: rkisp1: Breakout basic path handling to own class
>   libcamera: pipeline: rkisp1: Move path configuration to RkISP1Path
>   libcamera: pipeline: rkisp1: Move path configuration generation and
>     validation to RkISP1Path
>   libcamera: pipeline: rkisp1: Add wrappers for accessing the path video
>     device
>   libcamera: pipeline: rkisp1: Move start and stop of path to RkISP1Path
>   libcamera: pipeline: rkisp1: Move path link handling to RkISP1Path
>   libcamera: pipeline: rkisp1: Use the media link to track if a path is
>     enabled
>   libcamera: pipeline: rkisp1: Use cleanup error paths for start()
> 
>  src/libcamera/pipeline/rkisp1/meson.build    |   1 +
>  src/libcamera/pipeline/rkisp1/rkisp1.cpp     | 555 +++++++++++--------
>  src/libcamera/pipeline/rkisp1/rkisp1path.cpp | 237 ++++++++
>  src/libcamera/pipeline/rkisp1/rkisp1path.h   |  83 +++
>  4 files changed, 633 insertions(+), 243 deletions(-)
>  create mode 100644 src/libcamera/pipeline/rkisp1/rkisp1path.cpp
>  create mode 100644 src/libcamera/pipeline/rkisp1/rkisp1path.h
>