[libcamera-devel,v2,00/11] libcamera: ipu3: Attach to an skeleton IPA
mbox series

Message ID 20201229160318.77536-1-niklas.soderlund@ragnatech.se
Headers show
Series
  • libcamera: ipu3: Attach to an skeleton IPA
Related show

Message

Niklas Söderlund Dec. 29, 2020, 4:03 p.m. UTC 
Hello,

This series adds an IPA skeleton for the IPU3 pipeline. The IPA itself
supports mapping of parameter and statistic buffers as well as pushing
V4L2 controls to the camera sensor. It does however not read or write
anything into the buffers given to it during the capture process. It
also adds all the plumbing inside IPU3 pipeline to function with an IPA
while of of course retaining all the different capture setups previously
supported.

The idea is to prepare for adding algorithms in the IPA on-top without
having to worry about all the plumbing inside the pipeline handler. This
will likely happen in two steps. A first where fake algorithms are added
to generate metadata to be used in further development of cam and the
Android HAL. Then a follow up step where the fake ones are replaced with
real ones.

This series depends on the DelayedControls work.

Niklas Söderlund (11):
  include: linux: Add ipu3 kernel header and format definitions
  libcamera: ipu3: imgu: Configure the stat video device as part of
    configure()
  libcamera: ipu3: imgu: Add parameters video device
  libcamera: ipu3: cio2: Return the FrameBuffer pointer used
  libcamera: ipu3: cio2: Generate start of frame event
  libcamera: ipa: ipu3: Add an IPA skeleton for the IPU3 pipeline
  libcamera: ipu3: Attach to an IPA and allow it to set sensor controls
  libcamera: ipu3: imgu: Allocate buffers for stats and param
  libcamera: ipu3: Map buffers in IPA
  libcamera: ipu3: Add helper for parameter and statistic buffers
  libcamera: ipu3: Share parameter and statistic buffers with IPA

 include/libcamera/ipa/ipu3.h            |   23 +
 include/linux/intel-ipu3.h              | 2785 +++++++++++++++++++++++
 src/ipa/ipu3/ipu3.cpp                   |  236 ++
 src/ipa/ipu3/meson.build                |   21 +
 src/ipa/meson.build                     |    2 +-
 src/libcamera/pipeline/ipu3/cio2.cpp    |   28 +-
 src/libcamera/pipeline/ipu3/cio2.h      |    5 +-
 src/libcamera/pipeline/ipu3/frames.cpp  |  141 ++
 src/libcamera/pipeline/ipu3/frames.h    |   63 +
 src/libcamera/pipeline/ipu3/imgu.cpp    |   60 +-
 src/libcamera/pipeline/ipu3/imgu.h      |   13 +-
 src/libcamera/pipeline/ipu3/ipu3.cpp    |  261 ++-
 src/libcamera/pipeline/ipu3/meson.build |    1 +
 13 files changed, 3590 insertions(+), 49 deletions(-)
 create mode 100644 include/libcamera/ipa/ipu3.h
 create mode 100644 include/linux/intel-ipu3.h
 create mode 100644 src/ipa/ipu3/ipu3.cpp
 create mode 100644 src/ipa/ipu3/meson.build
 create mode 100644 src/libcamera/pipeline/ipu3/frames.cpp
 create mode 100644 src/libcamera/pipeline/ipu3/frames.h

Comments

Jean-Michel Hautbois Dec. 30, 2020, 4:38 p.m. UTC | #1 
Hi Niklas,

Thanks for the patchset !

On 29/12/2020 17:03, Niklas Söderlund wrote:
> Hello,
> 
> This series adds an IPA skeleton for the IPU3 pipeline. The IPA itself
> supports mapping of parameter and statistic buffers as well as pushing
> V4L2 controls to the camera sensor. It does however not read or write
> anything into the buffers given to it during the capture process. It
> also adds all the plumbing inside IPU3 pipeline to function with an IPA
> while of of course retaining all the different capture setups previously
> supported.
> 
> The idea is to prepare for adding algorithms in the IPA on-top without
> having to worry about all the plumbing inside the pipeline handler. This
> will likely happen in two steps. A first where fake algorithms are added
> to generate metadata to be used in further development of cam and the
> Android HAL. Then a follow up step where the fake ones are replaced with
> real ones.
> 
> This series depends on the DelayedControls work.

Can you tell what differs between v1 and v2 ?

> Niklas Söderlund (11):
>   include: linux: Add ipu3 kernel header and format definitions
>   libcamera: ipu3: imgu: Configure the stat video device as part of
>     configure()
>   libcamera: ipu3: imgu: Add parameters video device
>   libcamera: ipu3: cio2: Return the FrameBuffer pointer used
>   libcamera: ipu3: cio2: Generate start of frame event
>   libcamera: ipa: ipu3: Add an IPA skeleton for the IPU3 pipeline
>   libcamera: ipu3: Attach to an IPA and allow it to set sensor controls
>   libcamera: ipu3: imgu: Allocate buffers for stats and param
>   libcamera: ipu3: Map buffers in IPA
>   libcamera: ipu3: Add helper for parameter and statistic buffers
>   libcamera: ipu3: Share parameter and statistic buffers with IPA
> 
>  include/libcamera/ipa/ipu3.h            |   23 +
>  include/linux/intel-ipu3.h              | 2785 +++++++++++++++++++++++
>  src/ipa/ipu3/ipu3.cpp                   |  236 ++
>  src/ipa/ipu3/meson.build                |   21 +
>  src/ipa/meson.build                     |    2 +-
>  src/libcamera/pipeline/ipu3/cio2.cpp    |   28 +-
>  src/libcamera/pipeline/ipu3/cio2.h      |    5 +-
>  src/libcamera/pipeline/ipu3/frames.cpp  |  141 ++
>  src/libcamera/pipeline/ipu3/frames.h    |   63 +
>  src/libcamera/pipeline/ipu3/imgu.cpp    |   60 +-
>  src/libcamera/pipeline/ipu3/imgu.h      |   13 +-
>  src/libcamera/pipeline/ipu3/ipu3.cpp    |  261 ++-
>  src/libcamera/pipeline/ipu3/meson.build |    1 +
>  13 files changed, 3590 insertions(+), 49 deletions(-)
>  create mode 100644 include/libcamera/ipa/ipu3.h
>  create mode 100644 include/linux/intel-ipu3.h
>  create mode 100644 src/ipa/ipu3/ipu3.cpp
>  create mode 100644 src/ipa/ipu3/meson.build
>  create mode 100644 src/libcamera/pipeline/ipu3/frames.cpp
>  create mode 100644 src/libcamera/pipeline/ipu3/frames.h
> 

--
JM
Niklas Söderlund Dec. 30, 2020, 6:24 p.m. UTC | #2 
Hi Jean-Michel,

Thanks for your feedback.

On 2020-12-30 17:38:55 +0100, Jean-Michel Hautbois wrote:
> Hi Niklas,
> 
> Thanks for the patchset !
> 
> On 29/12/2020 17:03, Niklas Söderlund wrote:
> > Hello,
> > 
> > This series adds an IPA skeleton for the IPU3 pipeline. The IPA itself
> > supports mapping of parameter and statistic buffers as well as pushing
> > V4L2 controls to the camera sensor. It does however not read or write
> > anything into the buffers given to it during the capture process. It
> > also adds all the plumbing inside IPU3 pipeline to function with an IPA
> > while of of course retaining all the different capture setups previously
> > supported.
> > 
> > The idea is to prepare for adding algorithms in the IPA on-top without
> > having to worry about all the plumbing inside the pipeline handler. This
> > will likely happen in two steps. A first where fake algorithms are added
> > to generate metadata to be used in further development of cam and the
> > Android HAL. Then a follow up step where the fake ones are replaced with
> > real ones.
> > 
> > This series depends on the DelayedControls work.
> 
> Can you tell what differs between v1 and v2 ?

Differences are recorded in each individual patch.

> 
> > Niklas Söderlund (11):
> >   include: linux: Add ipu3 kernel header and format definitions
> >   libcamera: ipu3: imgu: Configure the stat video device as part of
> >     configure()
> >   libcamera: ipu3: imgu: Add parameters video device
> >   libcamera: ipu3: cio2: Return the FrameBuffer pointer used
> >   libcamera: ipu3: cio2: Generate start of frame event
> >   libcamera: ipa: ipu3: Add an IPA skeleton for the IPU3 pipeline
> >   libcamera: ipu3: Attach to an IPA and allow it to set sensor controls
> >   libcamera: ipu3: imgu: Allocate buffers for stats and param
> >   libcamera: ipu3: Map buffers in IPA
> >   libcamera: ipu3: Add helper for parameter and statistic buffers
> >   libcamera: ipu3: Share parameter and statistic buffers with IPA
> > 
> >  include/libcamera/ipa/ipu3.h            |   23 +
> >  include/linux/intel-ipu3.h              | 2785 +++++++++++++++++++++++
> >  src/ipa/ipu3/ipu3.cpp                   |  236 ++
> >  src/ipa/ipu3/meson.build                |   21 +
> >  src/ipa/meson.build                     |    2 +-
> >  src/libcamera/pipeline/ipu3/cio2.cpp    |   28 +-
> >  src/libcamera/pipeline/ipu3/cio2.h      |    5 +-
> >  src/libcamera/pipeline/ipu3/frames.cpp  |  141 ++
> >  src/libcamera/pipeline/ipu3/frames.h    |   63 +
> >  src/libcamera/pipeline/ipu3/imgu.cpp    |   60 +-
> >  src/libcamera/pipeline/ipu3/imgu.h      |   13 +-
> >  src/libcamera/pipeline/ipu3/ipu3.cpp    |  261 ++-
> >  src/libcamera/pipeline/ipu3/meson.build |    1 +
> >  13 files changed, 3590 insertions(+), 49 deletions(-)
> >  create mode 100644 include/libcamera/ipa/ipu3.h
> >  create mode 100644 include/linux/intel-ipu3.h
> >  create mode 100644 src/ipa/ipu3/ipu3.cpp
> >  create mode 100644 src/ipa/ipu3/meson.build
> >  create mode 100644 src/libcamera/pipeline/ipu3/frames.cpp
> >  create mode 100644 src/libcamera/pipeline/ipu3/frames.h
> > 
> 
> --
> JM