[v5,0/7] libcamera: mali-c55: Add support for memory-to-memory

26291 mbox series

Add support for m2m camera operations in the mali-c55 pipeline handler.

The series targets SoCs like the Renesas RZ/V2H(P) one where the
Mali-C55 ISP is integrated in m2m mode. The CSI-2 receiver (CRU) saves
images to memory and the ISP is fed through a dedicated DMA engine that
provides memory interfacing capabilities.

The series starts by adding support for the RZG2LCRU class (named after
the media device name) and then plumbs memory-to-memory operations in
the pipeline handler.

Tested on RZ/V2H(P) EVK with imx708 camera sensor.

Dan, could you re-test with Inline mode please ?
I should have collected your fix that restored proper operations with
Inline.

Signed-off-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
---
Changes in v5:
- Use overloaded std::visit when possible
- Adjust indentation

Changes in v4:
- Move the overloaded visitor helper to utils.h

Changes in v3:
- Drop dynamic polymorphism and use overloaded visitor in
  MaliC55CameraData. The result is neater in my opinion.
- Address Barnabas' comments I missed in v2

Changes in v2:
- Almost a complete rework of the CameraData which is now a class
  hierarchy. I'm not sure the result is that much cleaner, but hey, I'm
  using templates and pattern with weird names, so it ought be better
  for sure

---
Daniel Scally (3):
      libcamera: mali-c55: Add RZG2LCRU class
      libcamera: mali-c55: Register memory input camera
      libcamera: mali-c55: Implement capture for memory-to-memory

Jacopo Mondi (4):
      libcamera: utils: Add overloaded visitor helpers
      libcamera: mali-c55: Split TPG and Inline camera handling
      libcamera: mali-c55: Configure camera in memory-to-memory
      libcamera: mali-c55: Fix sensor size computation

 include/libcamera/base/utils.h                |   7 +
 src/libcamera/base/utils.cpp                  |  35 ++
 src/libcamera/pipeline/mali-c55/mali-c55.cpp  | 623 ++++++++++++++++++++------
 src/libcamera/pipeline/mali-c55/meson.build   |   3 +-
 src/libcamera/pipeline/mali-c55/rzg2l-cru.cpp | 260 +++++++++++
 src/libcamera/pipeline/mali-c55/rzg2l-cru.h   |  73 +++
 src/libcamera/pipeline/virtual/virtual.cpp    |  10 +-
 7 files changed, 867 insertions(+), 144 deletions(-)
---
base-commit: 02277d4c1a5ae7fee582f635936877435a12db64
change-id: 20251205-mali-cru-e212f8ab332e

Best regards,

Hi

2026. 03. 13. 17:14 keltezéssel, Jacopo Mondi írta:
> Add support for m2m camera operations in the mali-c55 pipeline handler.
> 
> The series targets SoCs like the Renesas RZ/V2H(P) one where the
> Mali-C55 ISP is integrated in m2m mode. The CSI-2 receiver (CRU) saves
> images to memory and the ISP is fed through a dedicated DMA engine that
> provides memory interfacing capabilities.
> 
> The series starts by adding support for the RZG2LCRU class (named after
> the media device name) and then plumbs memory-to-memory operations in
> the pipeline handler.
> 
> Tested on RZ/V2H(P) EVK with imx708 camera sensor.
> 
> Dan, could you re-test with Inline mode please ?
> I should have collected your fix that restored proper operations with
> Inline.
> 
> Signed-off-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
> ---

Please check the following branch: https://gitlab.freedesktop.org/pobrn/libcamera/-/commits/mali-c55-mem
I have made some small changes, mainly to the variant handling.

I have also modified the `RZG2LCRU` class to use an `std::vector` instead of
an `std::queue`, since it does not really need to be a queue. And as somewhat
expected, this brings out the raw buffer reuse issue much more consistently,
so I think `frameInfo.rawBuffer = nullptr` should be added to `PipelineHandlerMaliC55::paramsComputed()`
as discussed. Or alternatively maybe `RZG2LCRU::cruReturnBuffer()` could be
called from `PipelineHandlerMaliC55::tryComplete()` like it is done for the
other buffers. Although I think reusing it as soon as possible is probably
the better option.

And the test pattern generator does not seem to work, I don't know why, yet.
But it is not working even without these changes, so probably unrelated.


> Changes in v5:
> - Use overloaded std::visit when possible
> - Adjust indentation
> 
> Changes in v4:
> - Move the overloaded visitor helper to utils.h
> 
> Changes in v3:
> - Drop dynamic polymorphism and use overloaded visitor in
>    MaliC55CameraData. The result is neater in my opinion.
> - Address Barnabas' comments I missed in v2
> 
> Changes in v2:
> - Almost a complete rework of the CameraData which is now a class
>    hierarchy. I'm not sure the result is that much cleaner, but hey, I'm
>    using templates and pattern with weird names, so it ought be better
>    for sure
> 
> ---
> Daniel Scally (3):
>        libcamera: mali-c55: Add RZG2LCRU class
>        libcamera: mali-c55: Register memory input camera
>        libcamera: mali-c55: Implement capture for memory-to-memory
> 
> Jacopo Mondi (4):
>        libcamera: utils: Add overloaded visitor helpers
>        libcamera: mali-c55: Split TPG and Inline camera handling
>        libcamera: mali-c55: Configure camera in memory-to-memory
>        libcamera: mali-c55: Fix sensor size computation
> 
>   include/libcamera/base/utils.h                |   7 +
>   src/libcamera/base/utils.cpp                  |  35 ++
>   src/libcamera/pipeline/mali-c55/mali-c55.cpp  | 623 ++++++++++++++++++++------
>   src/libcamera/pipeline/mali-c55/meson.build   |   3 +-
>   src/libcamera/pipeline/mali-c55/rzg2l-cru.cpp | 260 +++++++++++
>   src/libcamera/pipeline/mali-c55/rzg2l-cru.h   |  73 +++
>   src/libcamera/pipeline/virtual/virtual.cpp    |  10 +-
>   7 files changed, 867 insertions(+), 144 deletions(-)
> ---
> base-commit: 02277d4c1a5ae7fee582f635936877435a12db64
> change-id: 20251205-mali-cru-e212f8ab332e
> 
> Best regards,

Hi Barnabás

On Wed, Mar 18, 2026 at 05:10:42PM +0100, Barnabás Pőcze wrote:
> Hi
>
> 2026. 03. 13. 17:14 keltezéssel, Jacopo Mondi írta:
> > Add support for m2m camera operations in the mali-c55 pipeline handler.
> >
> > The series targets SoCs like the Renesas RZ/V2H(P) one where the
> > Mali-C55 ISP is integrated in m2m mode. The CSI-2 receiver (CRU) saves
> > images to memory and the ISP is fed through a dedicated DMA engine that
> > provides memory interfacing capabilities.
> >
> > The series starts by adding support for the RZG2LCRU class (named after
> > the media device name) and then plumbs memory-to-memory operations in
> > the pipeline handler.
> >
> > Tested on RZ/V2H(P) EVK with imx708 camera sensor.
> >
> > Dan, could you re-test with Inline mode please ?
> > I should have collected your fix that restored proper operations with
> > Inline.
> >
> > Signed-off-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
> > ---
>
> Please check the following branch: https://gitlab.freedesktop.org/pobrn/libcamera/-/commits/mali-c55-mem
> I have made some small changes, mainly to the variant handling.

Thank you! I apreciate them.

>
> I have also modified the `RZG2LCRU` class to use an `std::vector` instead of
> an `std::queue`, since it does not really need to be a queue. And as somewhat

A vector uses contiguous memory, and I think that's preferred yeah.

With this change availableBuffers_ is now used as a LIFO, which might
also have an effect on the caching as buffers exported from the CRU
are cache coherent.

> expected, this brings out the raw buffer reuse issue much more consistently,
> so I think `frameInfo.rawBuffer = nullptr` should be added to `PipelineHandlerMaliC55::paramsComputed()`
> as discussed. Or alternatively maybe `RZG2LCRU::cruReturnBuffer()` could be

Thanks, I see you have added it to "fixup! libcamera: mali-c55:
Implement capture for memory-to-memory"

> called from `PipelineHandlerMaliC55::tryComplete()` like it is done for the
> other buffers. Although I think reusing it as soon as possible is probably
> the better option.
>
> And the test pattern generator does not seem to work, I don't know why, yet.
> But it is not working even without these changes, so probably unrelated.

No, it didn't so at least we're not making things worse :)

>
>
> > Changes in v5:
> > - Use overloaded std::visit when possible
> > - Adjust indentation
> >
> > Changes in v4:
> > - Move the overloaded visitor helper to utils.h
> >
> > Changes in v3:
> > - Drop dynamic polymorphism and use overloaded visitor in
> >    MaliC55CameraData. The result is neater in my opinion.
> > - Address Barnabas' comments I missed in v2
> >
> > Changes in v2:
> > - Almost a complete rework of the CameraData which is now a class
> >    hierarchy. I'm not sure the result is that much cleaner, but hey, I'm
> >    using templates and pattern with weird names, so it ought be better
> >    for sure
> >
> > ---
> > Daniel Scally (3):
> >        libcamera: mali-c55: Add RZG2LCRU class
> >        libcamera: mali-c55: Register memory input camera
> >        libcamera: mali-c55: Implement capture for memory-to-memory
> >
> > Jacopo Mondi (4):
> >        libcamera: utils: Add overloaded visitor helpers
> >        libcamera: mali-c55: Split TPG and Inline camera handling
> >        libcamera: mali-c55: Configure camera in memory-to-memory
> >        libcamera: mali-c55: Fix sensor size computation
> >
> >   include/libcamera/base/utils.h                |   7 +
> >   src/libcamera/base/utils.cpp                  |  35 ++
> >   src/libcamera/pipeline/mali-c55/mali-c55.cpp  | 623 ++++++++++++++++++++------
> >   src/libcamera/pipeline/mali-c55/meson.build   |   3 +-
> >   src/libcamera/pipeline/mali-c55/rzg2l-cru.cpp | 260 +++++++++++
> >   src/libcamera/pipeline/mali-c55/rzg2l-cru.h   |  73 +++
> >   src/libcamera/pipeline/virtual/virtual.cpp    |  10 +-
> >   7 files changed, 867 insertions(+), 144 deletions(-)
> > ---
> > base-commit: 02277d4c1a5ae7fee582f635936877435a12db64
> > change-id: 20251205-mali-cru-e212f8ab332e
> >
> > Best regards,
>