[v2,0/2] Add Renesas R-Car Gen4 pipeline with IPA support
mbox series

Message ID 20250911090547.1860863-1-niklas.soderlund+renesas@ragnatech.se
Headers show
Series
  • Add Renesas R-Car Gen4 pipeline with IPA support
Related show

Message

Niklas Söderlund Sept. 11, 2025, 9:05 a.m. UTC
Hello,

These two patches adds support for Renesas R-Car Gen4 devices. A new
R-Car Gen4 specific pipeline handler is needed to support capture and
DMA to/from the ISP. The IPA functionality can however (in user-space)
piggy back on the existing RkISP1 IPA implementation.

The architect of the hardware consist of two different blocks. There is
one capture block (VIN) that is responsible for the interactions with
the sensor and get RAW frames to user-space. And a ISP block acting in
mem-to-mem mode responsible to process the RAW frames into statistic's
and debayerd images. One notable thing about the hardware is that there
are no scaler available anywhere in either of the blocks (that can scale
RAW, VIN can scale non-RAW image data).

The ISP is based on Dream Chip RPP which is similar to the V10 variant
of RkISP1. For this reason the kernel driver was designed to reuse the
RK1S and RK1E pixel formats. For libcamera this means the RkISP1 IPA can
be re-used (almost) as is.

Patch 1/2 adds a ID for the Dream Chip RPP ISP for the RkISP1 IPA. It's
very similar to V10, but the histogram have more buckets so a distinct
ID was needed. Patch 2/2 adds the new pipeline handler and wires it up
to the RkISP1 IPA.

This work have been tested with IMX219 and IMX462 on R-Car V4H. All
libcamera compliance test pass. For IPA extensive testing with camshark
have been done, and the RkISP1 uncalibrated.yaml hardware blocks
function as expected and gives a good image.

Please note, not all RK1E blocks are yet supported by the kernel driver,
so using imx219.yaml will enable some parts of the IPA the kernel driver
do not yet support. The plan on the kernel side is to add support for
these blocks. Doing so will (hopefully) not require any changes to
libcamera.

This series depends on a patch from Hans de Goede [1] to allow more then
one pipeline handler to use the same IPA. While on the kernel side this
works depends on the still out-of-tree patches for the R-Car ISP [2].

Patches to add RkISP1 IPA debug meta data from Stefan have been used to
verify proper operation of the hardware blocks supported by the kernel
driver. Thanks for all your nice utilities Stefan, it made this work so
much more enjoyable!

Largest change since v1 is updating internal interfaces as v1 no longer 
compiles on-top of master. See individual patches for changelog.

1. [PATCH v2 7/8] libcamera: ipa_manager: createIPA: Allow matching by IPA name instead of by pipeline
   20250510141220.54872-8-hdegoede@redhat.com

2. [PATCH 0/7] media: Add support for R-Car ISP using Dreamchip RPPX1 ISP
   https://lore.kernel.org/linux-renesas-soc/20250617103201.1594152-1-niklas.soderlund%2Brenesas@ragnatech.se/

Niklas Söderlund (2):
  ipa: rkisp1: Add settings for DreamChip RPPX1 ISP
  libcamera: pipeline: Add R-Car Gen4 ISP pipeline

 include/libcamera/ipa/meson.build             |   1 +
 include/libcamera/ipa/rkisp1.mojom            |   8 +
 meson.build                                   |   1 +
 meson_options.txt                             |   1 +
 src/ipa/rkisp1/rkisp1.cpp                     |  11 +
 src/libcamera/pipeline/rcar-gen4/frames.cpp   | 280 ++++++
 src/libcamera/pipeline/rcar-gen4/frames.h     |  85 ++
 src/libcamera/pipeline/rcar-gen4/isp.cpp      | 227 +++++
 src/libcamera/pipeline/rcar-gen4/isp.h        |  44 +
 src/libcamera/pipeline/rcar-gen4/meson.build  |   8 +
 .../pipeline/rcar-gen4/rcar-gen4.cpp          | 814 ++++++++++++++++++
 src/libcamera/pipeline/rcar-gen4/vin.cpp      | 391 +++++++++
 src/libcamera/pipeline/rcar-gen4/vin.h        |  66 ++
 13 files changed, 1937 insertions(+)
 create mode 100644 src/libcamera/pipeline/rcar-gen4/frames.cpp
 create mode 100644 src/libcamera/pipeline/rcar-gen4/frames.h
 create mode 100644 src/libcamera/pipeline/rcar-gen4/isp.cpp
 create mode 100644 src/libcamera/pipeline/rcar-gen4/isp.h
 create mode 100644 src/libcamera/pipeline/rcar-gen4/meson.build
 create mode 100644 src/libcamera/pipeline/rcar-gen4/rcar-gen4.cpp
 create mode 100644 src/libcamera/pipeline/rcar-gen4/vin.cpp
 create mode 100644 src/libcamera/pipeline/rcar-gen4/vin.h

Comments

Kieran Bingham Sept. 13, 2025, 2:41 p.m. UTC | #1
Quoting Niklas Söderlund (2025-09-11 10:05:45)
> Hello,
> 
> These two patches adds support for Renesas R-Car Gen4 devices. A new
> R-Car Gen4 specific pipeline handler is needed to support capture and
> DMA to/from the ISP. The IPA functionality can however (in user-space)
> piggy back on the existing RkISP1 IPA implementation.
> 
> The architect of the hardware consist of two different blocks. There is
> one capture block (VIN) that is responsible for the interactions with
> the sensor and get RAW frames to user-space. And a ISP block acting in
> mem-to-mem mode responsible to process the RAW frames into statistic's
> and debayerd images. One notable thing about the hardware is that there
> are no scaler available anywhere in either of the blocks (that can scale
> RAW, VIN can scale non-RAW image data).
> 
> The ISP is based on Dream Chip RPP which is similar to the V10 variant
> of RkISP1. For this reason the kernel driver was designed to reuse the
> RK1S and RK1E pixel formats. For libcamera this means the RkISP1 IPA can
> be re-used (almost) as is.
> 
> Patch 1/2 adds a ID for the Dream Chip RPP ISP for the RkISP1 IPA. It's
> very similar to V10, but the histogram have more buckets so a distinct
> ID was needed. Patch 2/2 adds the new pipeline handler and wires it up
> to the RkISP1 IPA.
> 
> This work have been tested with IMX219 and IMX462 on R-Car V4H. All
> libcamera compliance test pass. For IPA extensive testing with camshark
> have been done, and the RkISP1 uncalibrated.yaml hardware blocks
> function as expected and gives a good image.
> 
> Please note, not all RK1E blocks are yet supported by the kernel driver,
> so using imx219.yaml will enable some parts of the IPA the kernel driver
> do not yet support. The plan on the kernel side is to add support for
> these blocks. Doing so will (hopefully) not require any changes to
> libcamera.
> 
> This series depends on a patch from Hans de Goede [1] to allow more then
> one pipeline handler to use the same IPA. While on the kernel side this
> works depends on the still out-of-tree patches for the R-Car ISP [2].

Can you include [1] in your series for v3 ? Otherwise the CI just simply
reports that this is broken.

I think that series is watiing on a new revision too and there's no
point being blocked on that forever if you can just include that one
patch here.

> Patches to add RkISP1 IPA debug meta data from Stefan have been used to
> verify proper operation of the hardware blocks supported by the kernel
> driver. Thanks for all your nice utilities Stefan, it made this work so
> much more enjoyable!
> 
> Largest change since v1 is updating internal interfaces as v1 no longer 
> compiles on-top of master. See individual patches for changelog.
> 
> 1. [PATCH v2 7/8] libcamera: ipa_manager: createIPA: Allow matching by IPA name instead of by pipeline
>    20250510141220.54872-8-hdegoede@redhat.com
> 
> 2. [PATCH 0/7] media: Add support for R-Car ISP using Dreamchip RPPX1 ISP
>    https://lore.kernel.org/linux-renesas-soc/20250617103201.1594152-1-niklas.soderlund%2Brenesas@ragnatech.se/
> 
> Niklas Söderlund (2):
>   ipa: rkisp1: Add settings for DreamChip RPPX1 ISP
>   libcamera: pipeline: Add R-Car Gen4 ISP pipeline
> 
>  include/libcamera/ipa/meson.build             |   1 +
>  include/libcamera/ipa/rkisp1.mojom            |   8 +
>  meson.build                                   |   1 +
>  meson_options.txt                             |   1 +
>  src/ipa/rkisp1/rkisp1.cpp                     |  11 +
>  src/libcamera/pipeline/rcar-gen4/frames.cpp   | 280 ++++++
>  src/libcamera/pipeline/rcar-gen4/frames.h     |  85 ++
>  src/libcamera/pipeline/rcar-gen4/isp.cpp      | 227 +++++
>  src/libcamera/pipeline/rcar-gen4/isp.h        |  44 +
>  src/libcamera/pipeline/rcar-gen4/meson.build  |   8 +
>  .../pipeline/rcar-gen4/rcar-gen4.cpp          | 814 ++++++++++++++++++
>  src/libcamera/pipeline/rcar-gen4/vin.cpp      | 391 +++++++++
>  src/libcamera/pipeline/rcar-gen4/vin.h        |  66 ++
>  13 files changed, 1937 insertions(+)
>  create mode 100644 src/libcamera/pipeline/rcar-gen4/frames.cpp
>  create mode 100644 src/libcamera/pipeline/rcar-gen4/frames.h
>  create mode 100644 src/libcamera/pipeline/rcar-gen4/isp.cpp
>  create mode 100644 src/libcamera/pipeline/rcar-gen4/isp.h
>  create mode 100644 src/libcamera/pipeline/rcar-gen4/meson.build
>  create mode 100644 src/libcamera/pipeline/rcar-gen4/rcar-gen4.cpp
>  create mode 100644 src/libcamera/pipeline/rcar-gen4/vin.cpp
>  create mode 100644 src/libcamera/pipeline/rcar-gen4/vin.h
> 
> -- 
> 2.51.0
>