[v3,2/2] pipeline: imx8-isi: Add multicamera support
diff mbox series

Message ID 20250630150128.2435445-3-antoine.bouyer@nxp.com
State Superseded
Headers show
Series
  • pipeline: imx8-isi: Add multicamera support
Related show

Commit Message

Antoine Bouyer June 30, 2025, 3:01 p.m. UTC 
ISI can be used to support multiple cameras at the same time in the media
device. A dedicated pad is assigned to each camera by HW (dts). We then
assign one (or more) source pad(s) depending on ISI constraints in the SoC.

Since ISI may have different number of pipes depending on SoC, the number
of cameras is computed _before_ going through all pipeline's components,
to limit number of pipes that could be assigned to a camera.

For instance, 3 is targeted as maximum amount of streams per camera as
defined by 'kNumStreams' constant. If 2 cameras are connected, with
only 5 ISI pipes, this target cannot be achieved. In such case, 2 streams
are assigned to each camera.

On the other hand, if ISI has 8 source pads (from index 6 to 13) and 2
cameras, first three source pads [6:8] are assigned to first camera, and
the three next source pads [9:11] are assigned to the second camera. All
these pads (sink and sources) must have same format to prevent configuration
mismatch during start.

However, since ISI routing must be performed _before_ any stream becomes
active, 'setRouting' is now executed during the 'match' step, instead of
'configure'. Indeed, it is up to the application to decide when a camera
'start' is executed: this could happen before or after the other camera
configuration is executed. Moving routing into the 'match' step makes sure
the routing is correctly applied before any 'start' is executed.

Signed-off-by: Antoine Bouyer <antoine.bouyer@nxp.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
---
 src/libcamera/pipeline/imx8-isi/imx8-isi.cpp | 112 ++++++++++++-------
 1 file changed, 74 insertions(+), 38 deletions(-)

Patch
diff mbox series 

diff --git a/src/libcamera/pipeline/imx8-isi/imx8-isi.cpp b/src/libcamera/pipeline/imx8-isi/imx8-isi.cpp
index 186b623df186..a29e361571a4 100644
--- a/src/libcamera/pipeline/imx8-isi/imx8-isi.cpp
+++ b/src/libcamera/pipeline/imx8-isi/imx8-isi.cpp
@@ -55,7 +55,7 @@  public:
 
 	unsigned int pipeIndex(const Stream *stream)
 	{
-		return stream - &*streams_.begin();
+		return stream - &*streams_.begin() + xbarSourceOffset_;
 	}
 
 	unsigned int getRawMediaBusFormat(PixelFormat *pixelFormat) const;
@@ -69,7 +69,8 @@  public:
 
 	std::vector<Stream *> enabledStreams_;
 
-	unsigned int xbarSink_;
+	unsigned int xbarSink_ = 0;
+	unsigned int xbarSourceOffset_ = 0;
 };
 
 class ISICameraConfiguration : public CameraConfiguration
@@ -809,34 +810,9 @@  int PipelineHandlerISI::configure(Camera *camera, CameraConfiguration *c)
 	ISICameraConfiguration *camConfig = static_cast<ISICameraConfiguration *>(c);
 	ISICameraData *data = cameraData(camera);
 
-	/* All links are immutable except the sensor -> csis link. */
-	const MediaPad *sensorSrc = data->sensor_->entity()->getPadByIndex(0);
-	sensorSrc->links()[0]->setEnabled(true);
-
-	/*
-	 * Reset the crossbar switch routing and enable one route for each
-	 * requested stream configuration.
-	 *
-	 * \todo Handle concurrent usage of multiple cameras by adjusting the
-	 * routing table instead of resetting it.
-	 */
-	V4L2Subdevice::Routing routing = {};
-	unsigned int xbarFirstSource = crossbar_->entity()->pads().size() / 2 + 1;
-
-	for (const auto &[idx, config] : utils::enumerate(*c)) {
-		uint32_t sourcePad = xbarFirstSource + idx;
-		routing.emplace_back(V4L2Subdevice::Stream{ data->xbarSink_, 0 },
-				     V4L2Subdevice::Stream{ sourcePad, 0 },
-				     V4L2_SUBDEV_ROUTE_FL_ACTIVE);
-	}
-
-	int ret = crossbar_->setRouting(&routing, V4L2Subdevice::ActiveFormat);
-	if (ret)
-		return ret;
-
-	/* Apply format to the sensor and CSIS receiver. */
+	/* Apply format to the sensor, CSIS receiver and crossbar sink pad. */
 	V4L2SubdeviceFormat format = camConfig->sensorFormat_;
-	ret = data->sensor_->setFormat(&format);
+	int ret = data->sensor_->setFormat(&format);
 	if (ret)
 		return ret;
 
@@ -848,10 +824,17 @@  int PipelineHandlerISI::configure(Camera *camera, CameraConfiguration *c)
 	if (ret)
 		return ret;
 
-	/* Now configure the ISI and video node instances, one per stream. */
-	data->enabledStreams_.clear();
-	for (const auto &config : *c) {
-		Pipe *pipe = pipeFromStream(camera, config.stream());
+	/*
+	 * As links on the output of the crossbar switch are immutable, the
+	 * routing table configured at match() time creates a media pipeline
+	 * that includes all the ISI pipelines corresponding to streams of this
+	 * camera, regardless of whether or not the streams are used in the
+	 * camera configuration. Set the format on the sink pad of all
+	 * corresponding ISI pipelines to avoid link validation failures when
+	 * starting streaming on the media pipeline.
+	 */
+	for (unsigned i = 0; i < data->streams_.size(); i++) {
+		Pipe *pipe = &pipes_.at(data->xbarSourceOffset_ + i);
 
 		/*
 		 * Set the format on the ISI sink pad: it must match what is
@@ -860,6 +843,15 @@  int PipelineHandlerISI::configure(Camera *camera, CameraConfiguration *c)
 		ret = pipe->isi->setFormat(0, &format);
 		if (ret)
 			return ret;
+	}
+
+	/*
+	 * Now configure the ISI pipeline source pad and video node instances,
+	 * one per enabled stream.
+	 */
+	data->enabledStreams_.clear();
+	for (const auto &config : *c) {
+		Pipe *pipe = pipeFromStream(camera, config.stream());
 
 		/*
 		 * Configure the ISI sink compose rectangle to downscale the
@@ -971,6 +963,20 @@  bool PipelineHandlerISI::match(DeviceEnumerator *enumerator)
 	if (!isiDev_)
 		return false;
 
+	/* Count the number of sensors, to create one camera per sensor. */
+	unsigned cameraCount = 0;
+	for (MediaEntity *entity : isiDev_->entities()) {
+		if (entity->function() != MEDIA_ENT_F_CAM_SENSOR)
+			continue;
+
+		cameraCount++;
+	}
+
+	if (!cameraCount) {
+		LOG(ISI, Error) << "No camera sensor found";
+		return false;
+	}
+
 	/*
 	 * Acquire the subdevs and video nodes for the crossbar switch and the
 	 * processing pipelines.
@@ -1014,12 +1020,24 @@  bool PipelineHandlerISI::match(DeviceEnumerator *enumerator)
 		return false;
 	}
 
+	if (cameraCount > pipes_.size()) {
+		LOG(ISI, Error) << "Too many cameras";
+		return false;
+	}
+
 	/*
 	 * Loop over all the crossbar switch sink pads to find connected CSI-2
 	 * receivers and camera sensors.
+	 *
+	 * In multicamera case, limit maximum amount of streams to allow all
+	 * sensors to get at least one dedicated pipe.
 	 */
 	unsigned int numCameras = 0;
 	unsigned int numSinks = 0;
+	const unsigned int xbarFirstSource = crossbar_->entity()->pads().size() - pipes_.size();
+	const unsigned int maxStreams = pipes_.size() / cameraCount;
+	V4L2Subdevice::Routing routing = {};
+
 	for (MediaPad *pad : crossbar_->entity()->pads()) {
 		unsigned int sink = numSinks;
 
@@ -1050,17 +1068,23 @@  bool PipelineHandlerISI::match(DeviceEnumerator *enumerator)
 			continue;
 		}
 
+		/* All links are immutable except the sensor -> csis link. */
+		const MediaPad *sensorSrc = sensor->getPadByIndex(0);
+		sensorSrc->links()[0]->setEnabled(true);
+
 		/* Create the camera data. */
-		/*
-		 * \todo compute available pipes per camera instead of using
-		 * pipes_.size() for multi cameras case.
-		 */
 		std::unique_ptr<ISICameraData> data =
-			std::make_unique<ISICameraData>(this, pipes_.size());
+			std::make_unique<ISICameraData>(this, maxStreams);
 
 		data->sensor_ = CameraSensorFactoryBase::create(sensor);
 		data->csis_ = std::make_unique<V4L2Subdevice>(csi);
 		data->xbarSink_ = sink;
+		data->xbarSourceOffset_ = numCameras * data->streams_.size();
+
+		LOG(ISI, Debug)
+			<< "cam" << numCameras
+			<< " streams " << data->streams_.size()
+			<< " offset " << data->xbarSourceOffset_;
 
 		ret = data->init();
 		if (ret) {
@@ -1075,6 +1099,14 @@  bool PipelineHandlerISI::match(DeviceEnumerator *enumerator)
 			       std::inserter(streams, streams.end()),
 			       [](Stream &s) { return &s; });
 
+		/*  Add routes to the crossbar switch routing table. */
+		for (unsigned i = 0; i < data->streams_.size(); i++) {
+			unsigned int sourcePad = xbarFirstSource + data->xbarSourceOffset_ + i;
+			routing.emplace_back(V4L2Subdevice::Stream{ data->xbarSink_, 0 },
+					     V4L2Subdevice::Stream{ sourcePad, 0 },
+					     V4L2_SUBDEV_ROUTE_FL_ACTIVE);
+		}
+
 		std::shared_ptr<Camera> camera =
 			Camera::create(std::move(data), id, streams);
 
@@ -1082,6 +1114,10 @@  bool PipelineHandlerISI::match(DeviceEnumerator *enumerator)
 		numCameras++;
 	}
 
+	ret = crossbar_->setRouting(&routing, V4L2Subdevice::ActiveFormat);
+	if (ret)
+		return false;
+
 	return numCameras > 0;
 }