| Message ID | 20261118140719.46849-1-david.plowman@raspberrypi.com |
|---|---|
| Headers | show |
| Series |
|
| Related | show |
Hi David, On Wed, Nov 18, 2026 at 01:15:16PM +0000, David Plowman wrote: > Hi everyone > > This is a small preparatory set of patches for "pixel peeping" IPAs, > meaning IPAs that want to have a look at some of the raw pixel data. > > There are 3 patches: > > 1. Pass the PixelFormat to the IPAs. > > I didn't turn the PixelFormat into a first class mojom object as it > would have required me to write more code (!), but I could do that if > we thought others might want to do the same at some point, or if > anyone generally felt strongly about it. > > 2. Memory map raw buffers and track their fds. > > Naush - I've done this a bit differently to your original version, > mostly to keep the extra mapping calls out of the platform files > (vc4.cpp and pisp.cpp), and only in the shared files. Perhaps you > could check you're still happy with this... > > 3. Pass raw buffer to IPAs. > > This is just an update to our prepare() methods to accept the raw > buffer, even though no one is using it yet. I wanted to keep all this > fluff out of the previous commit to make that one easier to digest. > > And in case folks are curious, one of the first possible IPAs on my > hitlist might be a global motion detection algorithm. The CPU-based soft ISP performs both pixel processing and statistics calculation in the pipeline handler, and GPU acceleration is also handled there. Is there something fundamentally different in motion detection to require processing on the IPA module side ? > David Plowman (3): > pipeline: rpi: Pass the raw buffer pixel format to the IPA > ipa: rpi: Memory map raw stream buffers > ipa: rpi: Pass the raw stream buffer to the control algorithms > > include/libcamera/ipa/raspberrypi.mojom | 2 + > src/ipa/rpi/common/ipa_base.cpp | 16 ++++--- > src/ipa/rpi/common/ipa_base.h | 2 + > src/ipa/rpi/controller/algorithm.cpp | 3 +- > src/ipa/rpi/controller/algorithm.h | 2 +- > src/ipa/rpi/controller/camera_mode.h | 3 ++ > src/ipa/rpi/controller/controller.cpp | 4 +- > src/ipa/rpi/controller/controller.h | 11 ++++- > src/ipa/rpi/controller/rpi/af.cpp | 3 +- > src/ipa/rpi/controller/rpi/af.h | 2 +- > src/ipa/rpi/controller/rpi/agc.cpp | 3 +- > src/ipa/rpi/controller/rpi/agc.h | 2 +- > src/ipa/rpi/controller/rpi/alsc.cpp | 3 +- > src/ipa/rpi/controller/rpi/alsc.h | 2 +- > src/ipa/rpi/controller/rpi/awb.cpp | 3 +- > src/ipa/rpi/controller/rpi/awb.h | 2 +- > src/ipa/rpi/controller/rpi/black_level.cpp | 3 +- > src/ipa/rpi/controller/rpi/black_level.h | 2 +- > src/ipa/rpi/controller/rpi/cac.cpp | 3 +- > src/ipa/rpi/controller/rpi/cac.h | 2 +- > src/ipa/rpi/controller/rpi/ccm.cpp | 3 +- > src/ipa/rpi/controller/rpi/ccm.h | 2 +- > src/ipa/rpi/controller/rpi/contrast.cpp | 3 +- > src/ipa/rpi/controller/rpi/contrast.h | 2 +- > src/ipa/rpi/controller/rpi/decompand.cpp | 3 +- > src/ipa/rpi/controller/rpi/decompand.h | 2 +- > src/ipa/rpi/controller/rpi/denoise.cpp | 3 +- > src/ipa/rpi/controller/rpi/denoise.h | 2 +- > src/ipa/rpi/controller/rpi/dpc.cpp | 3 +- > src/ipa/rpi/controller/rpi/dpc.h | 2 +- > src/ipa/rpi/controller/rpi/geq.cpp | 3 +- > src/ipa/rpi/controller/rpi/geq.h | 2 +- > src/ipa/rpi/controller/rpi/hdr.cpp | 3 +- > src/ipa/rpi/controller/rpi/hdr.h | 2 +- > src/ipa/rpi/controller/rpi/lux.cpp | 3 +- > src/ipa/rpi/controller/rpi/lux.h | 2 +- > src/ipa/rpi/controller/rpi/noise.cpp | 3 +- > src/ipa/rpi/controller/rpi/noise.h | 2 +- > src/ipa/rpi/controller/rpi/saturation.cpp | 3 +- > src/ipa/rpi/controller/rpi/saturation.h | 2 +- > src/ipa/rpi/controller/rpi/sdn.cpp | 3 +- > src/ipa/rpi/controller/rpi/sdn.h | 2 +- > src/ipa/rpi/controller/rpi/sharpen.cpp | 3 +- > src/ipa/rpi/controller/rpi/sharpen.h | 2 +- > src/ipa/rpi/controller/rpi/tonemap.cpp | 3 +- > src/ipa/rpi/controller/rpi/tonemap.h | 2 +- > .../pipeline/rpi/common/pipeline_base.cpp | 44 ++++++++++++++++++- > .../pipeline/rpi/common/pipeline_base.h | 1 + > 48 files changed, 133 insertions(+), 50 deletions(-) > > -- > 2.47.3 >
Hi Laurent On Wed, 18 Nov 2026 at 14:21, Laurent Pinchart <laurent.pinchart@ideasonboard.com> wrote: > > Hi David, > > On Wed, Nov 18, 2026 at 01:15:16PM +0000, David Plowman wrote: > > Hi everyone > > > > This is a small preparatory set of patches for "pixel peeping" IPAs, > > meaning IPAs that want to have a look at some of the raw pixel data. > > > > There are 3 patches: > > > > 1. Pass the PixelFormat to the IPAs. > > > > I didn't turn the PixelFormat into a first class mojom object as it > > would have required me to write more code (!), but I could do that if > > we thought others might want to do the same at some point, or if > > anyone generally felt strongly about it. > > > > 2. Memory map raw buffers and track their fds. > > > > Naush - I've done this a bit differently to your original version, > > mostly to keep the extra mapping calls out of the platform files > > (vc4.cpp and pisp.cpp), and only in the shared files. Perhaps you > > could check you're still happy with this... > > > > 3. Pass raw buffer to IPAs. > > > > This is just an update to our prepare() methods to accept the raw > > buffer, even though no one is using it yet. I wanted to keep all this > > fluff out of the previous commit to make that one easier to digest. > > > > And in case folks are curious, one of the first possible IPAs on my > > hitlist might be a global motion detection algorithm. > > The CPU-based soft ISP performs both pixel processing and statistics > calculation in the pipeline handler, and GPU acceleration is also > handled there. Is there something fundamentally different in motion > detection to require processing on the IPA module side ? Pragmatically, I think we can expect to have different algorithms and tuning parameters for them for different sensors and different use cases. The information produced by such algorithms might be of interest to other IPAs, for example, so it all fits very naturally there using all the mechanisms that we already have. More abstractly, I think would could make arguments about the soft ISP really being the core part of its pipeline handler, so it doesn't make sense to push it somewhere else as if it were an accessory or optional function. However, I feel that these "image processing algorithms" fall much more into that category, so treating them as our other IPAs is reasonable as well. David > > > David Plowman (3): > > pipeline: rpi: Pass the raw buffer pixel format to the IPA > > ipa: rpi: Memory map raw stream buffers > > ipa: rpi: Pass the raw stream buffer to the control algorithms > > > > include/libcamera/ipa/raspberrypi.mojom | 2 + > > src/ipa/rpi/common/ipa_base.cpp | 16 ++++--- > > src/ipa/rpi/common/ipa_base.h | 2 + > > src/ipa/rpi/controller/algorithm.cpp | 3 +- > > src/ipa/rpi/controller/algorithm.h | 2 +- > > src/ipa/rpi/controller/camera_mode.h | 3 ++ > > src/ipa/rpi/controller/controller.cpp | 4 +- > > src/ipa/rpi/controller/controller.h | 11 ++++- > > src/ipa/rpi/controller/rpi/af.cpp | 3 +- > > src/ipa/rpi/controller/rpi/af.h | 2 +- > > src/ipa/rpi/controller/rpi/agc.cpp | 3 +- > > src/ipa/rpi/controller/rpi/agc.h | 2 +- > > src/ipa/rpi/controller/rpi/alsc.cpp | 3 +- > > src/ipa/rpi/controller/rpi/alsc.h | 2 +- > > src/ipa/rpi/controller/rpi/awb.cpp | 3 +- > > src/ipa/rpi/controller/rpi/awb.h | 2 +- > > src/ipa/rpi/controller/rpi/black_level.cpp | 3 +- > > src/ipa/rpi/controller/rpi/black_level.h | 2 +- > > src/ipa/rpi/controller/rpi/cac.cpp | 3 +- > > src/ipa/rpi/controller/rpi/cac.h | 2 +- > > src/ipa/rpi/controller/rpi/ccm.cpp | 3 +- > > src/ipa/rpi/controller/rpi/ccm.h | 2 +- > > src/ipa/rpi/controller/rpi/contrast.cpp | 3 +- > > src/ipa/rpi/controller/rpi/contrast.h | 2 +- > > src/ipa/rpi/controller/rpi/decompand.cpp | 3 +- > > src/ipa/rpi/controller/rpi/decompand.h | 2 +- > > src/ipa/rpi/controller/rpi/denoise.cpp | 3 +- > > src/ipa/rpi/controller/rpi/denoise.h | 2 +- > > src/ipa/rpi/controller/rpi/dpc.cpp | 3 +- > > src/ipa/rpi/controller/rpi/dpc.h | 2 +- > > src/ipa/rpi/controller/rpi/geq.cpp | 3 +- > > src/ipa/rpi/controller/rpi/geq.h | 2 +- > > src/ipa/rpi/controller/rpi/hdr.cpp | 3 +- > > src/ipa/rpi/controller/rpi/hdr.h | 2 +- > > src/ipa/rpi/controller/rpi/lux.cpp | 3 +- > > src/ipa/rpi/controller/rpi/lux.h | 2 +- > > src/ipa/rpi/controller/rpi/noise.cpp | 3 +- > > src/ipa/rpi/controller/rpi/noise.h | 2 +- > > src/ipa/rpi/controller/rpi/saturation.cpp | 3 +- > > src/ipa/rpi/controller/rpi/saturation.h | 2 +- > > src/ipa/rpi/controller/rpi/sdn.cpp | 3 +- > > src/ipa/rpi/controller/rpi/sdn.h | 2 +- > > src/ipa/rpi/controller/rpi/sharpen.cpp | 3 +- > > src/ipa/rpi/controller/rpi/sharpen.h | 2 +- > > src/ipa/rpi/controller/rpi/tonemap.cpp | 3 +- > > src/ipa/rpi/controller/rpi/tonemap.h | 2 +- > > .../pipeline/rpi/common/pipeline_base.cpp | 44 ++++++++++++++++++- > > .../pipeline/rpi/common/pipeline_base.h | 1 + > > 48 files changed, 133 insertions(+), 50 deletions(-) > > > > -- > > 2.47.3 > > > > -- > Regards, > > Laurent Pinchart