| Message ID | 20250617104642.1607118-1-niklas.soderlund+renesas@ragnatech.se |
|---|---|
| Headers | show |
| Series |
|
| Related | show |
Hi Niklas, Quoting Niklas Söderlund (2025-06-17 11:46:40) > Hello, > > These two patches adds support for Renesas R-Car Gen4 devices. A new > R-Car Gen4 specific pipeline handler is needed to support capture and > DMA to/from the ISP. The IPA functionality can however (in user-space) > piggy back on the existing RkISP1 IPA implementation. Thanks - it's awesome to see a new platform getting supported in libcamera. Unfortunately - there are a lot of CI build failures on this patch series: https://gitlab.freedesktop.org/camera/libcamera/-/pipelines/1451951 -- Regards Kieran > > The architect of the hardware consist of two different blocks. There is > one capture block (VIN) that is responsible for the interactions with > the sensor and get RAW frames to user-space. And a ISP block acting in > mem-to-mem mode responsible to process the RAW frames into statistic's > and debayerd images. One notable thing about the hardware is that there > are no scaler available anywhere in either of the blocks (that can scale > RAW, VIN can scale non-RAW image data). > > The ISP is based on Dream Chip RPP which is similar to the V10 variant > of RkISP1. For this reason the kernel driver was designed to reuse the > RK1S and RK1E pixel formats. For libcamera this means the RkISP1 IPA can > be re-used (almost) as is. > > Patch 1/2 adds a ID for the Dream Chip RPP ISP for the RkISP1 IPA. It's > very similar to V10, but the histogram have more buckets so a distinct > ID was needed. Patch 2/2 adds the new pipeline handler and wires it up > to the RkISP1 IPA. > > This work have been tested with IMX219 and IMX462 on R-Car V4H. All > libcamera compliance test pass. For IPA extensive testing with camshark > have been done, and the RkISP1 uncalibrated.yaml hardware blocks > function as expected and gives a good image. > > Please note, not all RK1E blocks are yet supported by the kernel driver, > so using imx219.yaml will enable some parts of the IPA the kernel driver > do not yet support. The plan on the kernel side is to add support for > these blocks. Doing so will (hopefully) not require any changes to > libcamera. > > This series depends on a patch from Hans de Goede [1] to allow more then > one pipeline handler to use the same IPA. While on the kernel side this > works depends on the still out-of-tree patches for the R-Car ISP [2]. > > Patches to add RkISP1 IPA debug meta data from Stefan have been used to > verify proper operation of the hardware blocks supported by the kernel > driver. Thanks for all your nice utilities Stefan, it made this work so > much more enjoyable! > > 1. [PATCH v2 7/8] libcamera: ipa_manager: createIPA: Allow matching by IPA name instead of by pipeline > 20250510141220.54872-8-hdegoede@redhat.com > > 2. [PATCH 0/7] media: Add support for R-Car ISP using Dreamchip RPPX1 ISP > https://lore.kernel.org/linux-renesas-soc/20250617103201.1594152-1-niklas.soderlund%2Brenesas@ragnatech.se/ > > Niklas Söderlund (2): > ipa: rkisp1: Add settings for DreamChip RPPX1 ISP > libcamera: pipeline: Add R-Car Gen4 ISP pipeline > > include/libcamera/ipa/rkisp1.mojom | 8 + > meson.build | 1 + > meson_options.txt | 1 + > src/ipa/rkisp1/rkisp1.cpp | 11 + > src/libcamera/pipeline/rcar-gen4/frames.cpp | 283 ++++++ > src/libcamera/pipeline/rcar-gen4/frames.h | 87 ++ > src/libcamera/pipeline/rcar-gen4/isp.cpp | 227 +++++ > src/libcamera/pipeline/rcar-gen4/isp.h | 44 + > src/libcamera/pipeline/rcar-gen4/meson.build | 8 + > .../pipeline/rcar-gen4/rcar-gen4.cpp | 816 ++++++++++++++++++ > src/libcamera/pipeline/rcar-gen4/vin.cpp | 386 +++++++++ > src/libcamera/pipeline/rcar-gen4/vin.h | 68 ++ > 12 files changed, 1940 insertions(+) > create mode 100644 src/libcamera/pipeline/rcar-gen4/frames.cpp > create mode 100644 src/libcamera/pipeline/rcar-gen4/frames.h > create mode 100644 src/libcamera/pipeline/rcar-gen4/isp.cpp > create mode 100644 src/libcamera/pipeline/rcar-gen4/isp.h > create mode 100644 src/libcamera/pipeline/rcar-gen4/meson.build > create mode 100644 src/libcamera/pipeline/rcar-gen4/rcar-gen4.cpp > create mode 100644 src/libcamera/pipeline/rcar-gen4/vin.cpp > create mode 100644 src/libcamera/pipeline/rcar-gen4/vin.h > > -- > 2.49.0 >
Hi Kieran, On 2025-06-17 12:54:33 +0100, Kieran Bingham wrote: > Hi Niklas, > > Quoting Niklas Söderlund (2025-06-17 11:46:40) > > Hello, > > > > These two patches adds support for Renesas R-Car Gen4 devices. A new > > R-Car Gen4 specific pipeline handler is needed to support capture and > > DMA to/from the ISP. The IPA functionality can however (in user-space) > > piggy back on the existing RkISP1 IPA implementation. > > > Thanks - it's awesome to see a new platform getting supported in > libcamera. Indeed. > > Unfortunately - there are a lot of CI build failures on this patch > series: > > https://gitlab.freedesktop.org/camera/libcamera/-/pipelines/1451951 The build failures are due to the decency listed (Hans IPA multi match work) is not in the upstream tree ;-) > > -- > Regards > > Kieran > > > > > The architect of the hardware consist of two different blocks. There is > > one capture block (VIN) that is responsible for the interactions with > > the sensor and get RAW frames to user-space. And a ISP block acting in > > mem-to-mem mode responsible to process the RAW frames into statistic's > > and debayerd images. One notable thing about the hardware is that there > > are no scaler available anywhere in either of the blocks (that can scale > > RAW, VIN can scale non-RAW image data). > > > > The ISP is based on Dream Chip RPP which is similar to the V10 variant > > of RkISP1. For this reason the kernel driver was designed to reuse the > > RK1S and RK1E pixel formats. For libcamera this means the RkISP1 IPA can > > be re-used (almost) as is. > > > > Patch 1/2 adds a ID for the Dream Chip RPP ISP for the RkISP1 IPA. It's > > very similar to V10, but the histogram have more buckets so a distinct > > ID was needed. Patch 2/2 adds the new pipeline handler and wires it up > > to the RkISP1 IPA. > > > > This work have been tested with IMX219 and IMX462 on R-Car V4H. All > > libcamera compliance test pass. For IPA extensive testing with camshark > > have been done, and the RkISP1 uncalibrated.yaml hardware blocks > > function as expected and gives a good image. > > > > Please note, not all RK1E blocks are yet supported by the kernel driver, > > so using imx219.yaml will enable some parts of the IPA the kernel driver > > do not yet support. The plan on the kernel side is to add support for > > these blocks. Doing so will (hopefully) not require any changes to > > libcamera. > > > > This series depends on a patch from Hans de Goede [1] to allow more then > > one pipeline handler to use the same IPA. While on the kernel side this > > works depends on the still out-of-tree patches for the R-Car ISP [2]. > > > > Patches to add RkISP1 IPA debug meta data from Stefan have been used to > > verify proper operation of the hardware blocks supported by the kernel > > driver. Thanks for all your nice utilities Stefan, it made this work so > > much more enjoyable! > > > > 1. [PATCH v2 7/8] libcamera: ipa_manager: createIPA: Allow matching by IPA name instead of by pipeline > > 20250510141220.54872-8-hdegoede@redhat.com > > > > 2. [PATCH 0/7] media: Add support for R-Car ISP using Dreamchip RPPX1 ISP > > https://lore.kernel.org/linux-renesas-soc/20250617103201.1594152-1-niklas.soderlund%2Brenesas@ragnatech.se/ > > > > Niklas Söderlund (2): > > ipa: rkisp1: Add settings for DreamChip RPPX1 ISP > > libcamera: pipeline: Add R-Car Gen4 ISP pipeline > > > > include/libcamera/ipa/rkisp1.mojom | 8 + > > meson.build | 1 + > > meson_options.txt | 1 + > > src/ipa/rkisp1/rkisp1.cpp | 11 + > > src/libcamera/pipeline/rcar-gen4/frames.cpp | 283 ++++++ > > src/libcamera/pipeline/rcar-gen4/frames.h | 87 ++ > > src/libcamera/pipeline/rcar-gen4/isp.cpp | 227 +++++ > > src/libcamera/pipeline/rcar-gen4/isp.h | 44 + > > src/libcamera/pipeline/rcar-gen4/meson.build | 8 + > > .../pipeline/rcar-gen4/rcar-gen4.cpp | 816 ++++++++++++++++++ > > src/libcamera/pipeline/rcar-gen4/vin.cpp | 386 +++++++++ > > src/libcamera/pipeline/rcar-gen4/vin.h | 68 ++ > > 12 files changed, 1940 insertions(+) > > create mode 100644 src/libcamera/pipeline/rcar-gen4/frames.cpp > > create mode 100644 src/libcamera/pipeline/rcar-gen4/frames.h > > create mode 100644 src/libcamera/pipeline/rcar-gen4/isp.cpp > > create mode 100644 src/libcamera/pipeline/rcar-gen4/isp.h > > create mode 100644 src/libcamera/pipeline/rcar-gen4/meson.build > > create mode 100644 src/libcamera/pipeline/rcar-gen4/rcar-gen4.cpp > > create mode 100644 src/libcamera/pipeline/rcar-gen4/vin.cpp > > create mode 100644 src/libcamera/pipeline/rcar-gen4/vin.h > > > > -- > > 2.49.0 > >
Quoting Niklas Söderlund (2025-06-17 13:20:09) > Hi Kieran, > > On 2025-06-17 12:54:33 +0100, Kieran Bingham wrote: > > Hi Niklas, > > > > Quoting Niklas Söderlund (2025-06-17 11:46:40) > > > Hello, > > > > > > These two patches adds support for Renesas R-Car Gen4 devices. A new > > > R-Car Gen4 specific pipeline handler is needed to support capture and > > > DMA to/from the ISP. The IPA functionality can however (in user-space) > > > piggy back on the existing RkISP1 IPA implementation. > > > > > > Thanks - it's awesome to see a new platform getting supported in > > libcamera. > > Indeed. > > > > > Unfortunately - there are a lot of CI build failures on this patch > > series: > > > > https://gitlab.freedesktop.org/camera/libcamera/-/pipelines/1451951 > > The build failures are due to the decency listed (Hans IPA multi match > work) is not in the upstream tree ;-) aha, thanks - ok - that clears that up! I thought it was a bit too many ;-) I'll ignore the CI for now then. -- Kieran > > > > > -- > > Regards > > > > Kieran > > > > > > > > The architect of the hardware consist of two different blocks. There is > > > one capture block (VIN) that is responsible for the interactions with > > > the sensor and get RAW frames to user-space. And a ISP block acting in > > > mem-to-mem mode responsible to process the RAW frames into statistic's > > > and debayerd images. One notable thing about the hardware is that there > > > are no scaler available anywhere in either of the blocks (that can scale > > > RAW, VIN can scale non-RAW image data). > > > > > > The ISP is based on Dream Chip RPP which is similar to the V10 variant > > > of RkISP1. For this reason the kernel driver was designed to reuse the > > > RK1S and RK1E pixel formats. For libcamera this means the RkISP1 IPA can > > > be re-used (almost) as is. > > > > > > Patch 1/2 adds a ID for the Dream Chip RPP ISP for the RkISP1 IPA. It's > > > very similar to V10, but the histogram have more buckets so a distinct > > > ID was needed. Patch 2/2 adds the new pipeline handler and wires it up > > > to the RkISP1 IPA. > > > > > > This work have been tested with IMX219 and IMX462 on R-Car V4H. All > > > libcamera compliance test pass. For IPA extensive testing with camshark > > > have been done, and the RkISP1 uncalibrated.yaml hardware blocks > > > function as expected and gives a good image. > > > > > > Please note, not all RK1E blocks are yet supported by the kernel driver, > > > so using imx219.yaml will enable some parts of the IPA the kernel driver > > > do not yet support. The plan on the kernel side is to add support for > > > these blocks. Doing so will (hopefully) not require any changes to > > > libcamera. > > > > > > This series depends on a patch from Hans de Goede [1] to allow more then > > > one pipeline handler to use the same IPA. While on the kernel side this > > > works depends on the still out-of-tree patches for the R-Car ISP [2]. > > > > > > Patches to add RkISP1 IPA debug meta data from Stefan have been used to > > > verify proper operation of the hardware blocks supported by the kernel > > > driver. Thanks for all your nice utilities Stefan, it made this work so > > > much more enjoyable! > > > > > > 1. [PATCH v2 7/8] libcamera: ipa_manager: createIPA: Allow matching by IPA name instead of by pipeline > > > 20250510141220.54872-8-hdegoede@redhat.com > > > > > > 2. [PATCH 0/7] media: Add support for R-Car ISP using Dreamchip RPPX1 ISP > > > https://lore.kernel.org/linux-renesas-soc/20250617103201.1594152-1-niklas.soderlund%2Brenesas@ragnatech.se/ > > > > > > Niklas Söderlund (2): > > > ipa: rkisp1: Add settings for DreamChip RPPX1 ISP > > > libcamera: pipeline: Add R-Car Gen4 ISP pipeline > > > > > > include/libcamera/ipa/rkisp1.mojom | 8 + > > > meson.build | 1 + > > > meson_options.txt | 1 + > > > src/ipa/rkisp1/rkisp1.cpp | 11 + > > > src/libcamera/pipeline/rcar-gen4/frames.cpp | 283 ++++++ > > > src/libcamera/pipeline/rcar-gen4/frames.h | 87 ++ > > > src/libcamera/pipeline/rcar-gen4/isp.cpp | 227 +++++ > > > src/libcamera/pipeline/rcar-gen4/isp.h | 44 + > > > src/libcamera/pipeline/rcar-gen4/meson.build | 8 + > > > .../pipeline/rcar-gen4/rcar-gen4.cpp | 816 ++++++++++++++++++ > > > src/libcamera/pipeline/rcar-gen4/vin.cpp | 386 +++++++++ > > > src/libcamera/pipeline/rcar-gen4/vin.h | 68 ++ > > > 12 files changed, 1940 insertions(+) > > > create mode 100644 src/libcamera/pipeline/rcar-gen4/frames.cpp > > > create mode 100644 src/libcamera/pipeline/rcar-gen4/frames.h > > > create mode 100644 src/libcamera/pipeline/rcar-gen4/isp.cpp > > > create mode 100644 src/libcamera/pipeline/rcar-gen4/isp.h > > > create mode 100644 src/libcamera/pipeline/rcar-gen4/meson.build > > > create mode 100644 src/libcamera/pipeline/rcar-gen4/rcar-gen4.cpp > > > create mode 100644 src/libcamera/pipeline/rcar-gen4/vin.cpp > > > create mode 100644 src/libcamera/pipeline/rcar-gen4/vin.h > > > > > > -- > > > 2.49.0 > > > > > -- > Kind Regards, > Niklas Söderlund