| Message ID | 20251114154341.654850-1-antoine.bouyer@nxp.com |
|---|---|
| Headers | show |
| Series |
|
| Related | show |
Hi Antoine, Thank you for the patches! Quoting Antoine Bouyer (2025-11-14 15:43:39) > Submit this imx8-isi rework on behalf of Andrei. This series is about > using libcamera MediaPipeline class to simplify imx8-isi pipeline > configuration. > > Instead of going over each pipeline subdevices during imx8-isi match(), > all is handled by MediaPipeline class. It helps supporting complex > topologies, where subdevice(s) could be optional, typically on i.MX95 > SoC which has a formatter, while other i.MX SoCs don't have it. > > It reuses the simple pipeline's locateSensors method, so external ISP are > also supported then. ISP is considered as the 'sensor' element. > > Tested on i.MX8MP SoC. > I'm trying to test this patch series by re-routing my pipeline through the ISI instead of the ISP, and I'm having some issues with getting it to work. My media graph is as below: ``` - entity 1: crossbar (5 pads, 4 links, 2 routes) type V4L2 subdev subtype Unknown flags 0 device node name /dev/v4l-subdev0 routes: 0/0 -> 3/0 [ACTIVE] 0/0 -> 4/0 [ACTIVE] pad0: SINK,MUST_CONNECT [stream:0 fmt:SBGGR10_1X10/400x400 field:none colorspace:raw] <- "csis-32e40000.csi":1 [ENABLED,IMMUTABLE] pad1: SINK,MUST_CONNECT pad2: SINK,MUST_CONNECT <- "mxc_isi.output":0 [ENABLED,IMMUTABLE] pad3: SOURCE [stream:0 fmt:SBGGR10_1X10/400x400 field:none colorspace:raw] -> "mxc_isi.0":0 [ENABLED,IMMUTABLE] pad4: SOURCE [stream:0 fmt:SBGGR10_1X10/400x400 field:none colorspace:raw] -> "mxc_isi.1":0 [ENABLED,IMMUTABLE] - entity 7: mxc_isi.0 (2 pads, 2 links, 0 routes) type V4L2 subdev subtype Unknown flags 0 device node name /dev/v4l-subdev1 pad0: SINK [stream:0 fmt:SBGGR10_1X10/400x400 field:none colorspace:raw compose.bounds:(0,0)/400x400 compose:(0,0)/400x400] <- "crossbar":3 [ENABLED,IMMUTABLE] pad1: SOURCE [stream:0 fmt:SBGGR10_1X10/400x400 field:none crop.bounds:(0,0)/400x400 crop:(0,0)/400x400] -> "mxc_isi.0.capture":0 [ENABLED,IMMUTABLE] - entity 10: mxc_isi.0.capture (1 pad, 1 link) type Node subtype V4L flags 0 device node name /dev/video1 pad0: SINK <- "mxc_isi.0":1 [ENABLED,IMMUTABLE] - entity 18: mxc_isi.1 (2 pads, 2 links, 0 routes) type V4L2 subdev subtype Unknown flags 0 device node name /dev/v4l-subdev2 pad0: SINK [stream:0 fmt:UYVY8_2X8/1920x1080 field:none colorspace:jpeg xfer:srgb ycbcr:601 quantization:full-range compose.bounds:(0,0)/1920x1080 compose:(0,0)/1920x1080] <- "crossbar":4 [ENABLED,IMMUTABLE] pad1: SOURCE [stream:0 fmt:YUV8_1X24/1920x1080 field:none colorspace:jpeg xfer:srgb ycbcr:601 quantization:full-range crop.bounds:(0,0)/1920x1080 crop:(0,0)/1920x1080] -> "mxc_isi.1.capture":0 [ENABLED,IMMUTABLE] - entity 21: mxc_isi.1.capture (1 pad, 1 link) type Node subtype V4L flags 0 device node name /dev/video2 pad0: SINK <- "mxc_isi.1":1 [ENABLED,IMMUTABLE] - entity 29: mxc_isi.output (1 pad, 1 link) type Node subtype V4L flags 0 pad0: SOURCE -> "crossbar":2 [ENABLED,IMMUTABLE] - entity 36: csis-32e40000.csi (2 pads, 2 links, 0 routes) type V4L2 subdev subtype Unknown flags 0 device node name /dev/v4l-subdev3 pad0: SINK,MUST_CONNECT [stream:0 fmt:SBGGR10_1X10/400x400 field:none colorspace:raw] <- "fpga-1":2 [ENABLED] pad1: SOURCE,MUST_CONNECT [stream:0 fmt:SBGGR10_1X10/400x400 field:none colorspace:raw] -> "crossbar":0 [ENABLED,IMMUTABLE] - entity 41: fpga-1 (3 pads, 3 links, 0 routes) type V4L2 subdev subtype Unknown flags 0 device node name /dev/v4l-subdev4 pad0: SINK [stream:0 fmt:SBGGR10_1X10/400x400 field:none colorspace:raw] <- "camera 11-0036":0 [ENABLED,DYNAMIC] pad1: SINK [stream:0 fmt:Y8_1X8/1x1 field:none] <- "bridge 11-0060":0 [DYNAMIC] pad2: SOURCE [stream:0 fmt:SBGGR10_1X10/400x400 field:none colorspace:raw] -> "csis-32e40000.csi":0 [ENABLED] - entity 47: camera 11-0036 (1 pad, 1 link, 0 routes) type V4L2 subdev subtype Sensor flags 0 device node name /dev/v4l-subdev5 pad0: SOURCE [stream:0 fmt:SBGGR10_1X10/400x400 field:none colorspace:raw xfer:none ycbcr:601 quantization:full-range crop.bounds:(8,8)/400x400 crop:(8,8)/400x400] -> "fpga-1":0 [ENABLED,DYNAMIC] - entity 51: camera_torch_1 (0 pad, 0 link, 0 routes) type V4L2 subdev subtype Flash flags 0 device node name /dev/v4l-subdev6 - entity 53: bridge 11-0060 (1 pad, 1 link, 0 routes) type V4L2 subdev subtype Sensor flags 0 device node name /dev/v4l-subdev7 pad0: SOURCE [stream:0 fmt:SBGGR10_1X10/400x400 crop.bounds:(0,0)/640x480 crop:(0,0)/400x400] -> "fpga-1":1 [DYNAMIC] - entity 57: bridge_11-0060_flash (0 pad, 0 link, 0 routes) type V4L2 subdev subtype Flash flags 0 device node name /dev/v4l-subdev8 ``` The camera that is actively connected to the system is camera 11-0036, and the formats have been correctly configured for that camera as 400x400 SBGGR10, has propagated through as expected to mxc_isi.0. However, when I attempt to start the stream, I get EPIPE because mxc_isi.1 is configured to 1080p UYVY8, which doesn't match: ``` [ 40.833904] mxc-isi 32e00000.isi: v4l2_subdev_link_validate_default: width does not match (source 400, sink 1920) [ 40.833944] mxc-isi 32e00000.isi: v4l2_subdev_link_validate_default: height does not match (source 400, sink 1080) [ 40.833950] mxc-isi 32e00000.isi: v4l2_subdev_link_validate_default: media bus code does not match (source 0x00003007, sink 0x00002006) [ 40.833955] mxc-isi 32e00000.isi: v4l2_subdev_link_validate_default: link was "crossbar":4 -> "mxc_isi.1":0 ``` My question is: Should the link between crossbar:4 and mxc_isi.1 be enabled when it is not used? Perhaps I'm misunderstanding something here. Best wishes, Isaac > --- > Changes in v2: > - Add missing documentation as suggested by Barnabás > - Move Entity parameters documentation from .h to .cpp file. > - Replace 'video' by 'last' node in source descriptions, because in > imx8-isi pipeline case, the last MediaPipeline entity (i.e. crossbar) is > not a video node. > - Apply review comments from Jacopo: move Entity definition to beginning > of public section; move entities() to const-callable as well. > - link to v1: https://patchwork.libcamera.org/patch/25010/ > > --- > Andrei Gansari (2): > libamera: media_pipeline: Add accessor for MediaPipeline list of > entities > pipeline: imx8-isi: Integrating MediaPipeline class > > include/libcamera/internal/media_pipeline.h | 29 +--- > src/libcamera/media_pipeline.cpp | 44 +++++ > src/libcamera/pipeline/imx8-isi/imx8-isi.cpp | 159 ++++++++++++------- > 3 files changed, 149 insertions(+), 83 deletions(-) > > -- > 2.34.1 >
Hi Isaac Thanks for your feedback On 11/18/25 12:14 PM, Isaac Scott wrote: > Caution: This is an external email. Please take care when clicking links or opening attachments. When in doubt, report the message using the 'Report this email' button > > > Hi Antoine, > > Thank you for the patches! > > Quoting Antoine Bouyer (2025-11-14 15:43:39) >> Submit this imx8-isi rework on behalf of Andrei. This series is about >> using libcamera MediaPipeline class to simplify imx8-isi pipeline >> configuration. >> >> Instead of going over each pipeline subdevices during imx8-isi match(), >> all is handled by MediaPipeline class. It helps supporting complex >> topologies, where subdevice(s) could be optional, typically on i.MX95 >> SoC which has a formatter, while other i.MX SoCs don't have it. >> >> It reuses the simple pipeline's locateSensors method, so external ISP are >> also supported then. ISP is considered as the 'sensor' element. >> >> Tested on i.MX8MP SoC. >> > > I'm trying to test this patch series by re-routing my pipeline through > the ISI instead of the ISP, and I'm having some issues with getting it > to work. > > My media graph is as below: > > ``` > - entity 1: crossbar (5 pads, 4 links, 2 routes) > type V4L2 subdev subtype Unknown flags 0 > device node name /dev/v4l-subdev0 > routes: > 0/0 -> 3/0 [ACTIVE] > 0/0 -> 4/0 [ACTIVE] > pad0: SINK,MUST_CONNECT > [stream:0 fmt:SBGGR10_1X10/400x400 field:none colorspace:raw] > <- "csis-32e40000.csi":1 [ENABLED,IMMUTABLE] > pad1: SINK,MUST_CONNECT > pad2: SINK,MUST_CONNECT > <- "mxc_isi.output":0 [ENABLED,IMMUTABLE] > pad3: SOURCE > [stream:0 fmt:SBGGR10_1X10/400x400 field:none colorspace:raw] > -> "mxc_isi.0":0 [ENABLED,IMMUTABLE] > pad4: SOURCE > [stream:0 fmt:SBGGR10_1X10/400x400 field:none colorspace:raw] > -> "mxc_isi.1":0 [ENABLED,IMMUTABLE] > > - entity 7: mxc_isi.0 (2 pads, 2 links, 0 routes) > type V4L2 subdev subtype Unknown flags 0 > device node name /dev/v4l-subdev1 > pad0: SINK > [stream:0 fmt:SBGGR10_1X10/400x400 field:none colorspace:raw > compose.bounds:(0,0)/400x400 > compose:(0,0)/400x400] > <- "crossbar":3 [ENABLED,IMMUTABLE] > pad1: SOURCE > [stream:0 fmt:SBGGR10_1X10/400x400 field:none > crop.bounds:(0,0)/400x400 > crop:(0,0)/400x400] > -> "mxc_isi.0.capture":0 [ENABLED,IMMUTABLE] > > - entity 10: mxc_isi.0.capture (1 pad, 1 link) > type Node subtype V4L flags 0 > device node name /dev/video1 > pad0: SINK > <- "mxc_isi.0":1 [ENABLED,IMMUTABLE] > > - entity 18: mxc_isi.1 (2 pads, 2 links, 0 routes) > type V4L2 subdev subtype Unknown flags 0 > device node name /dev/v4l-subdev2 > pad0: SINK > [stream:0 fmt:UYVY8_2X8/1920x1080 field:none colorspace:jpeg xfer:srgb ycbcr:601 quantization:full-range > compose.bounds:(0,0)/1920x1080 > compose:(0,0)/1920x1080] > <- "crossbar":4 [ENABLED,IMMUTABLE] > pad1: SOURCE > [stream:0 fmt:YUV8_1X24/1920x1080 field:none colorspace:jpeg xfer:srgb ycbcr:601 quantization:full-range > crop.bounds:(0,0)/1920x1080 > crop:(0,0)/1920x1080] > -> "mxc_isi.1.capture":0 [ENABLED,IMMUTABLE] > > - entity 21: mxc_isi.1.capture (1 pad, 1 link) > type Node subtype V4L flags 0 > device node name /dev/video2 > pad0: SINK > <- "mxc_isi.1":1 [ENABLED,IMMUTABLE] > > - entity 29: mxc_isi.output (1 pad, 1 link) > type Node subtype V4L flags 0 > pad0: SOURCE > -> "crossbar":2 [ENABLED,IMMUTABLE] > > - entity 36: csis-32e40000.csi (2 pads, 2 links, 0 routes) > type V4L2 subdev subtype Unknown flags 0 > device node name /dev/v4l-subdev3 > pad0: SINK,MUST_CONNECT > [stream:0 fmt:SBGGR10_1X10/400x400 field:none colorspace:raw] > <- "fpga-1":2 [ENABLED] > pad1: SOURCE,MUST_CONNECT > [stream:0 fmt:SBGGR10_1X10/400x400 field:none colorspace:raw] > -> "crossbar":0 [ENABLED,IMMUTABLE] > > - entity 41: fpga-1 (3 pads, 3 links, 0 routes) > type V4L2 subdev subtype Unknown flags 0 > device node name /dev/v4l-subdev4 > pad0: SINK > [stream:0 fmt:SBGGR10_1X10/400x400 field:none colorspace:raw] > <- "camera 11-0036":0 [ENABLED,DYNAMIC] > pad1: SINK > [stream:0 fmt:Y8_1X8/1x1 field:none] > <- "bridge 11-0060":0 [DYNAMIC] > pad2: SOURCE > [stream:0 fmt:SBGGR10_1X10/400x400 field:none colorspace:raw] > -> "csis-32e40000.csi":0 [ENABLED] > > - entity 47: camera 11-0036 (1 pad, 1 link, 0 routes) > type V4L2 subdev subtype Sensor flags 0 > device node name /dev/v4l-subdev5 > pad0: SOURCE > [stream:0 fmt:SBGGR10_1X10/400x400 field:none colorspace:raw xfer:none ycbcr:601 quantization:full-range > crop.bounds:(8,8)/400x400 > crop:(8,8)/400x400] > -> "fpga-1":0 [ENABLED,DYNAMIC] > > - entity 51: camera_torch_1 (0 pad, 0 link, 0 routes) > type V4L2 subdev subtype Flash flags 0 > device node name /dev/v4l-subdev6 > > - entity 53: bridge 11-0060 (1 pad, 1 link, 0 routes) > type V4L2 subdev subtype Sensor flags 0 > device node name /dev/v4l-subdev7 > pad0: SOURCE > [stream:0 fmt:SBGGR10_1X10/400x400 > crop.bounds:(0,0)/640x480 > crop:(0,0)/400x400] > -> "fpga-1":1 [DYNAMIC] > > - entity 57: bridge_11-0060_flash (0 pad, 0 link, 0 routes) > type V4L2 subdev subtype Flash flags 0 > device node name /dev/v4l-subdev8 > ``` > > The camera that is actively connected to the system is camera 11-0036, > and the formats have been correctly configured for that camera as > 400x400 SBGGR10, has propagated through as expected to mxc_isi.0. > > However, when I attempt to start the stream, I get EPIPE because > mxc_isi.1 is configured to 1080p UYVY8, which doesn't match: Indeed this is odd. > > ``` > [ 40.833904] mxc-isi 32e00000.isi: v4l2_subdev_link_validate_default: width does not match (source 400, sink 1920) > [ 40.833944] mxc-isi 32e00000.isi: v4l2_subdev_link_validate_default: height does not match (source 400, sink 1080) > [ 40.833950] mxc-isi 32e00000.isi: v4l2_subdev_link_validate_default: media bus code does not match (source 0x00003007, sink 0x00002006) > [ 40.833955] mxc-isi 32e00000.isi: v4l2_subdev_link_validate_default: link was "crossbar":4 -> "mxc_isi.1":0 > ``` > > My question is: Should the link between crossbar:4 and mxc_isi.1 be > enabled when it is not used? Perhaps I'm misunderstanding something > here. The link is immutable, so we cannot disable it, but format should be aligned indeed. Could you share your test command/sequence please ? So I can try to reproduce on my side too. Did the same sequence pass before this patch series on your side ? Here is an example of test command for multiple stream case: $ export LIBCAMERA_PIPELINES_MATCH_LIST='imx8-isi' $ cam -c1 -s width=400,height=400,pixelformat=NV12 -C5 --file=/tmp/frame-#.nv12 -s width=1280,height=1024,pixelformat=NV12 -C5 --file=/tmp/frame-#.nv12 Best regards Antoine > > Best wishes, > > Isaac > >> --- >> Changes in v2: >> - Add missing documentation as suggested by Barnabás >> - Move Entity parameters documentation from .h to .cpp file. >> - Replace 'video' by 'last' node in source descriptions, because in >> imx8-isi pipeline case, the last MediaPipeline entity (i.e. crossbar) is >> not a video node. >> - Apply review comments from Jacopo: move Entity definition to beginning >> of public section; move entities() to const-callable as well. >> - link to v1: https://patchwork.libcamera.org/patch/25010/ >> >> --- >> Andrei Gansari (2): >> libamera: media_pipeline: Add accessor for MediaPipeline list of >> entities >> pipeline: imx8-isi: Integrating MediaPipeline class >> >> include/libcamera/internal/media_pipeline.h | 29 +--- >> src/libcamera/media_pipeline.cpp | 44 +++++ >> src/libcamera/pipeline/imx8-isi/imx8-isi.cpp | 159 ++++++++++++------- >> 3 files changed, 149 insertions(+), 83 deletions(-) >> >> -- >> 2.34.1 >>
Submit this imx8-isi rework on behalf of Andrei. This series is about using libcamera MediaPipeline class to simplify imx8-isi pipeline configuration. Instead of going over each pipeline subdevices during imx8-isi match(), all is handled by MediaPipeline class. It helps supporting complex topologies, where subdevice(s) could be optional, typically on i.MX95 SoC which has a formatter, while other i.MX SoCs don't have it. It reuses the simple pipeline's locateSensors method, so external ISP are also supported then. ISP is considered as the 'sensor' element. Tested on i.MX8MP SoC. --- Changes in v2: - Add missing documentation as suggested by Barnabás - Move Entity parameters documentation from .h to .cpp file. - Replace 'video' by 'last' node in source descriptions, because in imx8-isi pipeline case, the last MediaPipeline entity (i.e. crossbar) is not a video node. - Apply review comments from Jacopo: move Entity definition to beginning of public section; move entities() to const-callable as well. - link to v1: https://patchwork.libcamera.org/patch/25010/ --- Andrei Gansari (2): libamera: media_pipeline: Add accessor for MediaPipeline list of entities pipeline: imx8-isi: Integrating MediaPipeline class include/libcamera/internal/media_pipeline.h | 29 +--- src/libcamera/media_pipeline.cpp | 44 +++++ src/libcamera/pipeline/imx8-isi/imx8-isi.cpp | 159 ++++++++++++------- 3 files changed, 149 insertions(+), 83 deletions(-)