| Message ID | 20220801000543.3501-1-laurent.pinchart@ideasonboard.com |
|---|---|
| Headers | show |
| Series |
|
| Related | show |
Hi Laurent Just a note to record I tested on a non-routed pipeline (pinephone) and I've seen no regressions there Tested-by: Jacopo Mondi <jacopo@jmondi.org> On Mon, Aug 01, 2022 at 03:05:30AM +0300, Laurent Pinchart via libcamera-devel wrote: > Hello, > > This patch series adds support for the NXP i.MX8 ISI to the simple > pipeline handler. > > The ISI (which stands for Image Sensor Interface) is an IP core found in > multiple SoCs from the NXP i.MX8 family. It combines a crossbar switch > with processing pipelines that each include a scaler, a colorspace > converter and a DMA engine to capture frames from multiple inputs to > memory. As data flows linearly from the source to the DMA engine, the > ISI is a good candidate for the simple pipeline handler (while the ISI > can be used with Bayer sensors by bypassing the processing pipeline to > capture raw data, most use cases will use RGB or YUV sources). > > The ISI driver has been posted to the linux-media mailing list, > currently as a v2 ([1]). It depends on the V4L2 routing API to control > the crossbar switch, and as the API is still under development ([2]), > this blocks merging the driver for the time being. However, the routing > API is now at v12, and the development effort isn't slowing down, we may > consider merging support for the API in libcamera before it hits > upstream. I would appreciate feedback on this. > > The series starts with three patches that update the Linux kernel > header, with 01/13 moving to headers from kernel v5.19, and patches > 02/13 and 03/13 adding new pixel formats that have been merged in > subsystem trees and will be present in v5.20. Patch 04/13 then adds > support for those formats to libcamera. If we decide that the routing > API shouldn't be merged in libcamera yet, these first four patches are > still candidates for upstreaming without delay. > > Patch 05/13 then adds the V4L2 routing API to the kernel headers, and > patches 06/13 to 08/13 extend the V4L2Subdevice class to support the > API. Patches 09/13 to 12/13 extend the simple pipeline handler to take > routes into account during pipeline discovery and configuration. > Finally, patch 13/13 enables support for the ISI in the simple pipeline > handler. > > [1] https://lore.kernel.org/linux-media/20220712000251.13607-1-laurent.pinchart@ideasonboard.com/ > [2] https://lore.kernel.org/linux-media/20220301161156.1119557-1-tomi.valkeinen@ideasonboard.com/ > > Jacopo Mondi (4): > libcamera: v4l2_subdevice: Change V4L2Subdevice::Whence > libcamera: v4l2_subdevice: Collect subdev capabilities > libcamera: v4l2_subdevice: Add support for the V4L2 subdev routing API > libcamera: pipeline: simple: Reset routing table of subdevs > > Laurent Pinchart (7): > include: linux: Update kernel headers to version v5.19 > include: drm_fourcc: Add AVUY and XVUY 4:4:4 packet formats > include: linux: Add V4L2 YUVA32 and YUVX32 pixel formats > libcamera: formats: Add AVUY8888 and XVUY8888 formats > include: linux: Add V4L2 subdev internal routing API > libcamera: pipeline: simple: Setup links in the context of sink > entities > libcamera: pipeline: simple: Add support for NXP ISI > > Phi-Bang Nguyen (2): > libcamera: pipeline: simple: Walk pipeline using subdev internal > routing > libcamera: pipeline: simple: Don't disable links carrying other > streams > > include/libcamera/internal/v4l2_subdevice.h | 28 +- > include/linux/README | 2 +- > include/linux/dma-buf.h | 4 +- > include/linux/drm_fourcc.h | 88 +++++- > include/linux/intel-ipu3.h | 35 ++- > include/linux/v4l2-controls.h | 301 +++++++++++++++++++- > include/linux/v4l2-subdev.h | 88 +++++- > include/linux/videodev2.h | 15 +- > src/libcamera/formats.cpp | 26 ++ > src/libcamera/formats.yaml | 4 + > src/libcamera/pipeline/simple/simple.cpp | 173 ++++++++++- > src/libcamera/v4l2_pixelformat.cpp | 4 + > src/libcamera/v4l2_subdevice.cpp | 161 ++++++++++- > 13 files changed, 877 insertions(+), 52 deletions(-) > > > base-commit: 69ae75b0cc211f82665b3e92fb3de64a9852b403 > -- > Regards, > > Laurent Pinchart >
Hello, This patch series adds support for the NXP i.MX8 ISI to the simple pipeline handler. The ISI (which stands for Image Sensor Interface) is an IP core found in multiple SoCs from the NXP i.MX8 family. It combines a crossbar switch with processing pipelines that each include a scaler, a colorspace converter and a DMA engine to capture frames from multiple inputs to memory. As data flows linearly from the source to the DMA engine, the ISI is a good candidate for the simple pipeline handler (while the ISI can be used with Bayer sensors by bypassing the processing pipeline to capture raw data, most use cases will use RGB or YUV sources). The ISI driver has been posted to the linux-media mailing list, currently as a v2 ([1]). It depends on the V4L2 routing API to control the crossbar switch, and as the API is still under development ([2]), this blocks merging the driver for the time being. However, the routing API is now at v12, and the development effort isn't slowing down, we may consider merging support for the API in libcamera before it hits upstream. I would appreciate feedback on this. The series starts with three patches that update the Linux kernel header, with 01/13 moving to headers from kernel v5.19, and patches 02/13 and 03/13 adding new pixel formats that have been merged in subsystem trees and will be present in v5.20. Patch 04/13 then adds support for those formats to libcamera. If we decide that the routing API shouldn't be merged in libcamera yet, these first four patches are still candidates for upstreaming without delay. Patch 05/13 then adds the V4L2 routing API to the kernel headers, and patches 06/13 to 08/13 extend the V4L2Subdevice class to support the API. Patches 09/13 to 12/13 extend the simple pipeline handler to take routes into account during pipeline discovery and configuration. Finally, patch 13/13 enables support for the ISI in the simple pipeline handler. [1] https://lore.kernel.org/linux-media/20220712000251.13607-1-laurent.pinchart@ideasonboard.com/ [2] https://lore.kernel.org/linux-media/20220301161156.1119557-1-tomi.valkeinen@ideasonboard.com/ Jacopo Mondi (4): libcamera: v4l2_subdevice: Change V4L2Subdevice::Whence libcamera: v4l2_subdevice: Collect subdev capabilities libcamera: v4l2_subdevice: Add support for the V4L2 subdev routing API libcamera: pipeline: simple: Reset routing table of subdevs Laurent Pinchart (7): include: linux: Update kernel headers to version v5.19 include: drm_fourcc: Add AVUY and XVUY 4:4:4 packet formats include: linux: Add V4L2 YUVA32 and YUVX32 pixel formats libcamera: formats: Add AVUY8888 and XVUY8888 formats include: linux: Add V4L2 subdev internal routing API libcamera: pipeline: simple: Setup links in the context of sink entities libcamera: pipeline: simple: Add support for NXP ISI Phi-Bang Nguyen (2): libcamera: pipeline: simple: Walk pipeline using subdev internal routing libcamera: pipeline: simple: Don't disable links carrying other streams include/libcamera/internal/v4l2_subdevice.h | 28 +- include/linux/README | 2 +- include/linux/dma-buf.h | 4 +- include/linux/drm_fourcc.h | 88 +++++- include/linux/intel-ipu3.h | 35 ++- include/linux/v4l2-controls.h | 301 +++++++++++++++++++- include/linux/v4l2-subdev.h | 88 +++++- include/linux/videodev2.h | 15 +- src/libcamera/formats.cpp | 26 ++ src/libcamera/formats.yaml | 4 + src/libcamera/pipeline/simple/simple.cpp | 173 ++++++++++- src/libcamera/v4l2_pixelformat.cpp | 4 + src/libcamera/v4l2_subdevice.cpp | 161 ++++++++++- 13 files changed, 877 insertions(+), 52 deletions(-) base-commit: 69ae75b0cc211f82665b3e92fb3de64a9852b403