[v2,0/8] Raspberry Pi: Update VC4 pipeline for mainline
mbox series

Message ID 20260722-pi4-upstream-v2-0-eb44d3c6fd92@ideasonboard.com
Headers show
Series
  • Raspberry Pi: Update VC4 pipeline for mainline
Related show

Message

Jai Luthra July 22, 2026, 11:23 a.m. UTC 
Hi,

This series has updates to the Raspberry Pi VC4-based image pipeline to
work with the mainline variants of the capture (unicam) and ISP (VC4)
drivers.

The unicam driver is already merged in mainline. The latest revision of
the ISP driver is posted here:
	https://lore.kernel.org/all/20260717-b4-vchiq-isp-v3-0-fb8235e15c68@ideasonboard.com/

For testing this series you can use this kernel branch (with overlays,
VCHIQ dependencies and a temporary firmware clock fix):
	https://github.com/jailuthra/linux/commits/b4/rpi-isp

As this series breaks backward-compatibility with older downstream
kernels, and modifies a lot of different things, I've tried to split the
migration into multiple patches for ease of review:

	PATCH 1-4 are preparation patches, and should still work with a
	downstream kernel.
	PATCH 5-8 transition to the mainline kernel.

All of these should be 'bisectable' w.r.t. compilation, but actually
running libcamera on a mainline kernel in between the transition of
PATCH 5 and PATCH 7 might not work.

Signed-off-by: Jai Luthra <jai.luthra@ideasonboard.com>
---
Changes in v2:
- Populate the LS table's DMABUF descriptor in the pipeline handler
- Split and reorganize migration patches to ease review
- Cleanup unused PrepareParams argument in platformPrepareIsp()
- Link to v1: https://lore.kernel.org/r/20260210-pi4-upstream-v1-0-279841c15fba@ideasonboard.com

---
Jai Luthra (7):
      ipa: rpi: Drop unused params argument from platformPrepareIsp()
      include: uapi: bcm2835-isp: Add extensible parameter buffer interface
      ipa: rpi: vc4: Add Bcm2835Params helper
      ipa: rpi: Allow sharing params buffer to the IPA
      pipeline: rpi: vc4: Switch to mainline media topology for bcm2835-isp
      pipeline/ipa: rpi: vc4: Switch configuration from controls to param buffer
      include: uapi: bcm2835-isp: Drop legacy V4L2 controls

Laurent Pinchart (1):
      pipeline: rpi: vc4: Configure format on Unicam subdev

 include/libcamera/ipa/raspberrypi.mojom        |  15 +-
 include/linux/bcm2835-isp.h                    | 274 +++++++++++++++++++++----
 include/linux/v4l2-controls.h                  |   3 -
 include/linux/videodev2.h                      |   3 +-
 src/ipa/rpi/common/ipa_base.cpp                |  13 +-
 src/ipa/rpi/common/ipa_base.h                  |   7 +-
 src/ipa/rpi/pisp/pisp.cpp                      |   6 +-
 src/ipa/rpi/vc4/params.h                       |  79 +++++++
 src/ipa/rpi/vc4/vc4.cpp                        | 225 +++++++++-----------
 src/libcamera/pipeline/rpi/common/rpi_stream.h |   1 +
 src/libcamera/pipeline/rpi/pisp/pisp.cpp       |   6 +-
 src/libcamera/pipeline/rpi/vc4/vc4.cpp         | 238 ++++++++++++++++-----
 12 files changed, 622 insertions(+), 248 deletions(-)
---
base-commit: 8c453beec03e428cb63c79f588fbb12b59baddef
change-id: 20260210-pi4-upstream-869c057a4067

Best regards,

Comments

Paul Elder Sept. 1, 2026, 6:58 a.m. UTC | #1 
Hi Jai,

Thanks for the series.

Quoting Jai Luthra (2026-07-22 20:23:16)
> Hi,
> 
> This series has updates to the Raspberry Pi VC4-based image pipeline to
> work with the mainline variants of the capture (unicam) and ISP (VC4)
> drivers.
> 
> The unicam driver is already merged in mainline. The latest revision of
> the ISP driver is posted here:
>         https://lore.kernel.org/all/20260717-b4-vchiq-isp-v3-0-fb8235e15c68@ideasonboard.com/
> 
> For testing this series you can use this kernel branch (with overlays,
> VCHIQ dependencies and a temporary firmware clock fix):
>         https://github.com/jailuthra/linux/commits/b4/rpi-isp

It works! \o/

Tested on Raspberry Pi 4B with imx219.

# cam -c1 --capture=10
[42:53:29.157032279] [4146]  INFO Camera camera_manager.cpp:340 libcamera v0.7.2+10-0de3e405
[42:53:29.182984113] [4149]  INFO IPAProxy ipa_proxy.cpp:184 Using tuning file /usr/share/libcamera/ipa/rpi/vc4/imx219.json
[42:53:29.191749464] [4149]  WARN RPI vc4.cpp:377 Mismatch between Unicam and CamHelper for embedded data usage!
[42:53:29.192265409] [4149]  INFO Camera camera_manager.cpp:223 Adding camera '/base/i2c-mux0/i2c@1/camera@10' for pipeline handler rpi/vc4
[42:53:29.192290964] [4149]  INFO RPI vc4.cpp:434 Registered camera /base/i2c-mux0/i2c@1/camera@10 to Unicam device /dev/media0 and ISP device /dev/media1
Using camera /base/i2c-mux0/i2c@1/camera@10 as cam0
[42:53:29.192825798] [4146]  INFO Camera camera.cpp:1216 configuring streams: (0) 800x600-XRGB8888/sRGB
[42:53:29.192997261] [4149]  INFO RPI vc4.cpp:644 Sensor: /base/i2c-mux0/i2c@1/camera@10 - Selected sensor format: 1640x1232-SBGGR10_1X10/RAW - Selected unicam format: 1640x1232-pBAA/RAW
cam0: Capture 10 frames
154409.246635 (0.00 fps) cam0-stream0 seq: 000000 bytesused: 1920000
154409.279952 (30.01 fps) cam0-stream0 seq: 000001 bytesused: 1920000
154409.313269 (30.01 fps) cam0-stream0 seq: 000002 bytesused: 1920000
154409.380002 (14.99 fps) cam0-stream0 seq: 000003 bytesused: 1920000
154409.446734 (14.99 fps) cam0-stream0 seq: 000004 bytesused: 1920000
154409.513466 (14.99 fps) cam0-stream0 seq: 000005 bytesused: 1920000
154409.580197 (14.99 fps) cam0-stream0 seq: 000006 bytesused: 1920000
154409.646930 (14.99 fps) cam0-stream0 seq: 000007 bytesused: 1920000
154409.713658 (14.99 fps) cam0-stream0 seq: 000008 bytesused: 1920000
154409.780390 (14.99 fps) cam0-stream0 seq: 000009 bytesused: 1920000

Tested-by: Paul Elder <paul.elder@ideasonboard.com>

> 
> As this series breaks backward-compatibility with older downstream
> kernels, and modifies a lot of different things, I've tried to split the
> migration into multiple patches for ease of review:
> 
>         PATCH 1-4 are preparation patches, and should still work with a
>         downstream kernel.
>         PATCH 5-8 transition to the mainline kernel.
> 
> All of these should be 'bisectable' w.r.t. compilation, but actually
> running libcamera on a mainline kernel in between the transition of
> PATCH 5 and PATCH 7 might not work.
> 
> Signed-off-by: Jai Luthra <jai.luthra@ideasonboard.com>
> ---
> Changes in v2:
> - Populate the LS table's DMABUF descriptor in the pipeline handler
> - Split and reorganize migration patches to ease review
> - Cleanup unused PrepareParams argument in platformPrepareIsp()
> - Link to v1: https://lore.kernel.org/r/20260210-pi4-upstream-v1-0-279841c15fba@ideasonboard.com
> 
> ---
> Jai Luthra (7):
>       ipa: rpi: Drop unused params argument from platformPrepareIsp()
>       include: uapi: bcm2835-isp: Add extensible parameter buffer interface
>       ipa: rpi: vc4: Add Bcm2835Params helper
>       ipa: rpi: Allow sharing params buffer to the IPA
>       pipeline: rpi: vc4: Switch to mainline media topology for bcm2835-isp
>       pipeline/ipa: rpi: vc4: Switch configuration from controls to param buffer
>       include: uapi: bcm2835-isp: Drop legacy V4L2 controls
> 
> Laurent Pinchart (1):
>       pipeline: rpi: vc4: Configure format on Unicam subdev
> 
>  include/libcamera/ipa/raspberrypi.mojom        |  15 +-
>  include/linux/bcm2835-isp.h                    | 274 +++++++++++++++++++++----
>  include/linux/v4l2-controls.h                  |   3 -
>  include/linux/videodev2.h                      |   3 +-
>  src/ipa/rpi/common/ipa_base.cpp                |  13 +-
>  src/ipa/rpi/common/ipa_base.h                  |   7 +-
>  src/ipa/rpi/pisp/pisp.cpp                      |   6 +-
>  src/ipa/rpi/vc4/params.h                       |  79 +++++++
>  src/ipa/rpi/vc4/vc4.cpp                        | 225 +++++++++-----------
>  src/libcamera/pipeline/rpi/common/rpi_stream.h |   1 +
>  src/libcamera/pipeline/rpi/pisp/pisp.cpp       |   6 +-
>  src/libcamera/pipeline/rpi/vc4/vc4.cpp         | 238 ++++++++++++++++-----
>  12 files changed, 622 insertions(+), 248 deletions(-)
> ---
> base-commit: 8c453beec03e428cb63c79f588fbb12b59baddef
> change-id: 20260210-pi4-upstream-869c057a4067
> 
> Best regards,
> -- 
> Jai Luthra <jai.luthra@ideasonboard.com>
>