[libcamera-devel,2/5] libcamera: imx8-isi: Break out YUV format selection
diff mbox series

Message ID 20230129135830.27490-3-jacopo.mondi@ideasonboard.com
State Accepted
Headers show
Series
  • libcamera: imx8-isi: Remove pixelformat-2-media-bus map
Related show

Commit Message

Jacopo Mondi Jan. 29, 2023, 1:58 p.m. UTC 
As per the RAW format selection, the media bus format selection
procedure relies on the direct association of PixelFormat and media
bus code in the formatsMap_ map.

As the ISI can generate YUV and RGB formats from any non-Bayer media
bus format, break out the YUV/RGB media bus format selection to a
separate function.

The newly introduced getYuvMediaBusFormat() tests a list of
known-supported media bus formats against the list of media bus
formats supported by the sensor and tries to prefer media bus
codes with the same encoding as the requested PixelFormat.

Use the newly introduced function in
ISICameraConfiguration::validateYuv() to make sure the sensor can
produce a YUV/RGB media bus format.

Signed-off-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>
---
 src/libcamera/pipeline/imx8-isi/imx8-isi.cpp | 76 ++++++++++++++++++++
 1 file changed, 76 insertions(+)

Comments

Paul Elder March 2, 2023, 11:35 a.m. UTC | #1 
On Sun, Jan 29, 2023 at 02:58:27PM +0100, Jacopo Mondi via libcamera-devel wrote:
> As per the RAW format selection, the media bus format selection
> procedure relies on the direct association of PixelFormat and media
> bus code in the formatsMap_ map.
> 
> As the ISI can generate YUV and RGB formats from any non-Bayer media
> bus format, break out the YUV/RGB media bus format selection to a
> separate function.
> 
> The newly introduced getYuvMediaBusFormat() tests a list of
> known-supported media bus formats against the list of media bus
> formats supported by the sensor and tries to prefer media bus
> codes with the same encoding as the requested PixelFormat.
> 
> Use the newly introduced function in
> ISICameraConfiguration::validateYuv() to make sure the sensor can
> produce a YUV/RGB media bus format.
> 
> Signed-off-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>

Reviewed-by: Paul Elder <paul.elder@ideasonboard.com>

> ---
>  src/libcamera/pipeline/imx8-isi/imx8-isi.cpp | 76 ++++++++++++++++++++
>  1 file changed, 76 insertions(+)
> 
> diff --git a/src/libcamera/pipeline/imx8-isi/imx8-isi.cpp b/src/libcamera/pipeline/imx8-isi/imx8-isi.cpp
> index 72bc310d80ec..445fad32656c 100644
> --- a/src/libcamera/pipeline/imx8-isi/imx8-isi.cpp
> +++ b/src/libcamera/pipeline/imx8-isi/imx8-isi.cpp
> @@ -60,6 +60,7 @@ public:
>  	}
>  
>  	unsigned int getRawMediaBusFormat(PixelFormat *pixelFormat) const;
> +	unsigned int getYuvMediaBusFormat(PixelFormat *pixelFormat) const;
>  
>  	std::unique_ptr<CameraSensor> sensor_;
>  	std::unique_ptr<V4L2Subdevice> csis_;
> @@ -255,6 +256,65 @@ unsigned int ISICameraData::getRawMediaBusFormat(PixelFormat *pixelFormat) const
>  	return sensorCode;
>  }
>  
> +/*
> + * Get a YUV/RGB media bus format from which the ISI can produce a processed
> + * stream, preferring codes with the same colour encoding as the requested
> + * pixelformat.
> + *
> + * If the sensor does not provide any YUV/RGB media bus format the ISI cannot
> + * generate any processed pixel format as it cannot debayer.
> + */
> +unsigned int ISICameraData::getYuvMediaBusFormat(PixelFormat *pixelFormat) const
> +{
> +	std::vector<unsigned int> mbusCodes = sensor_->mbusCodes();
> +
> +	/*
> +	 * The ISI can produce YUV/RGB pixel formats from any non-RAW Bayer
> +	 * media bus formats.
> +	 *
> +	 * Keep the list in sync with the mxc_isi_bus_formats[] array in
> +	 * the ISI driver.
> +	 */
> +	std::vector<unsigned int> yuvCodes = {
> +		MEDIA_BUS_FMT_UYVY8_1X16,
> +		MEDIA_BUS_FMT_YUV8_1X24,
> +		MEDIA_BUS_FMT_RGB565_1X16,
> +		MEDIA_BUS_FMT_RGB888_1X24,
> +	};
> +
> +	std::sort(mbusCodes.begin(), mbusCodes.end());
> +	std::sort(yuvCodes.begin(), yuvCodes.end());
> +
> +	std::vector<unsigned int> supportedCodes;
> +	std::set_intersection(mbusCodes.begin(), mbusCodes.end(),
> +			      yuvCodes.begin(), yuvCodes.end(),
> +			      std::back_inserter(supportedCodes));
> +
> +	if (supportedCodes.empty()) {
> +		LOG(ISI, Warning) << "Cannot find a supported YUV/RGB format";
> +		*pixelFormat = {};
> +
> +		return 0;
> +	}
> +
> +	/* Prefer codes with the same encoding as the requested pixel format. */
> +	const PixelFormatInfo &info = PixelFormatInfo::info(*pixelFormat);
> +	for (unsigned int code : supportedCodes) {
> +		if (info.colourEncoding == PixelFormatInfo::ColourEncodingYUV &&
> +		    (code == MEDIA_BUS_FMT_UYVY8_1X16 ||
> +		     code == MEDIA_BUS_FMT_YUV8_1X24))
> +			return code;
> +
> +		if (info.colourEncoding == PixelFormatInfo::ColourEncodingRGB &&
> +		    (code == MEDIA_BUS_FMT_RGB565_1X16 ||
> +		     code == MEDIA_BUS_FMT_RGB888_1X24))
> +			return code;
> +	}
> +
> +	/* Otherwise return the first found code. */
> +	return supportedCodes[0];
> +}
> +
>  /* -----------------------------------------------------------------------------
>   * Camera Configuration
>   */
> @@ -455,6 +515,22 @@ ISICameraConfiguration::validateYuv(std::set<Stream *> &availableStreams,
>  {
>  	CameraConfiguration::Status status = Valid;
>  
> +	StreamConfiguration &yuvConfig = config_[0];
> +	PixelFormat yuvPixelFormat = yuvConfig.pixelFormat;
> +
> +	/*
> +	 * Make sure the sensor can produce a compatible YUV/RGB media bus
> +	 * format. If the sensor can only produce RAW Bayer we can only fail
> +	 * here as we can't adjust to anything but RAW.
> +	 */
> +	unsigned int yuvMediaBusCode = data_->getYuvMediaBusFormat(&yuvPixelFormat);
> +	if (!yuvMediaBusCode) {
> +		LOG(ISI, Error) << "Cannot adjust pixelformat "
> +				<< yuvConfig.pixelFormat;
> +		return Invalid;
> +	}
> +
> +	/* Adjust all the other streams. */
>  	for (const auto &[i, cfg] : utils::enumerate(config_)) {
>  
>  		LOG(ISI, Debug) << "Stream " << i << ": " << cfg.toString();
> -- 
> 2.39.0
>
Laurent Pinchart March 12, 2023, 4:33 p.m. UTC | #2 
Hi Jacopo,

Thank you for the patch.

On Sun, Jan 29, 2023 at 02:58:27PM +0100, Jacopo Mondi via libcamera-devel wrote:
> As per the RAW format selection, the media bus format selection
> procedure relies on the direct association of PixelFormat and media
> bus code in the formatsMap_ map.
> 
> As the ISI can generate YUV and RGB formats from any non-Bayer media
> bus format, break out the YUV/RGB media bus format selection to a
> separate function.
> 
> The newly introduced getYuvMediaBusFormat() tests a list of
> known-supported media bus formats against the list of media bus
> formats supported by the sensor and tries to prefer media bus
> codes with the same encoding as the requested PixelFormat.
> 
> Use the newly introduced function in
> ISICameraConfiguration::validateYuv() to make sure the sensor can
> produce a YUV/RGB media bus format.
> 
> Signed-off-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>

Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>

> ---
>  src/libcamera/pipeline/imx8-isi/imx8-isi.cpp | 76 ++++++++++++++++++++
>  1 file changed, 76 insertions(+)
> 
> diff --git a/src/libcamera/pipeline/imx8-isi/imx8-isi.cpp b/src/libcamera/pipeline/imx8-isi/imx8-isi.cpp
> index 72bc310d80ec..445fad32656c 100644
> --- a/src/libcamera/pipeline/imx8-isi/imx8-isi.cpp
> +++ b/src/libcamera/pipeline/imx8-isi/imx8-isi.cpp
> @@ -60,6 +60,7 @@ public:
>  	}
>  
>  	unsigned int getRawMediaBusFormat(PixelFormat *pixelFormat) const;
> +	unsigned int getYuvMediaBusFormat(PixelFormat *pixelFormat) const;
>  
>  	std::unique_ptr<CameraSensor> sensor_;
>  	std::unique_ptr<V4L2Subdevice> csis_;
> @@ -255,6 +256,65 @@ unsigned int ISICameraData::getRawMediaBusFormat(PixelFormat *pixelFormat) const
>  	return sensorCode;
>  }
>  
> +/*
> + * Get a YUV/RGB media bus format from which the ISI can produce a processed
> + * stream, preferring codes with the same colour encoding as the requested
> + * pixelformat.
> + *
> + * If the sensor does not provide any YUV/RGB media bus format the ISI cannot
> + * generate any processed pixel format as it cannot debayer.
> + */
> +unsigned int ISICameraData::getYuvMediaBusFormat(PixelFormat *pixelFormat) const
> +{
> +	std::vector<unsigned int> mbusCodes = sensor_->mbusCodes();
> +
> +	/*
> +	 * The ISI can produce YUV/RGB pixel formats from any non-RAW Bayer
> +	 * media bus formats.
> +	 *
> +	 * Keep the list in sync with the mxc_isi_bus_formats[] array in
> +	 * the ISI driver.
> +	 */
> +	std::vector<unsigned int> yuvCodes = {
> +		MEDIA_BUS_FMT_UYVY8_1X16,
> +		MEDIA_BUS_FMT_YUV8_1X24,
> +		MEDIA_BUS_FMT_RGB565_1X16,
> +		MEDIA_BUS_FMT_RGB888_1X24,
> +	};
> +
> +	std::sort(mbusCodes.begin(), mbusCodes.end());
> +	std::sort(yuvCodes.begin(), yuvCodes.end());
> +
> +	std::vector<unsigned int> supportedCodes;
> +	std::set_intersection(mbusCodes.begin(), mbusCodes.end(),
> +			      yuvCodes.begin(), yuvCodes.end(),
> +			      std::back_inserter(supportedCodes));
> +
> +	if (supportedCodes.empty()) {
> +		LOG(ISI, Warning) << "Cannot find a supported YUV/RGB format";
> +		*pixelFormat = {};
> +
> +		return 0;
> +	}
> +
> +	/* Prefer codes with the same encoding as the requested pixel format. */
> +	const PixelFormatInfo &info = PixelFormatInfo::info(*pixelFormat);
> +	for (unsigned int code : supportedCodes) {
> +		if (info.colourEncoding == PixelFormatInfo::ColourEncodingYUV &&
> +		    (code == MEDIA_BUS_FMT_UYVY8_1X16 ||
> +		     code == MEDIA_BUS_FMT_YUV8_1X24))
> +			return code;
> +
> +		if (info.colourEncoding == PixelFormatInfo::ColourEncodingRGB &&
> +		    (code == MEDIA_BUS_FMT_RGB565_1X16 ||
> +		     code == MEDIA_BUS_FMT_RGB888_1X24))
> +			return code;
> +	}
> +
> +	/* Otherwise return the first found code. */
> +	return supportedCodes[0];
> +}
> +
>  /* -----------------------------------------------------------------------------
>   * Camera Configuration
>   */
> @@ -455,6 +515,22 @@ ISICameraConfiguration::validateYuv(std::set<Stream *> &availableStreams,
>  {
>  	CameraConfiguration::Status status = Valid;
>  
> +	StreamConfiguration &yuvConfig = config_[0];
> +	PixelFormat yuvPixelFormat = yuvConfig.pixelFormat;
> +
> +	/*
> +	 * Make sure the sensor can produce a compatible YUV/RGB media bus
> +	 * format. If the sensor can only produce RAW Bayer we can only fail
> +	 * here as we can't adjust to anything but RAW.
> +	 */
> +	unsigned int yuvMediaBusCode = data_->getYuvMediaBusFormat(&yuvPixelFormat);
> +	if (!yuvMediaBusCode) {
> +		LOG(ISI, Error) << "Cannot adjust pixelformat "
> +				<< yuvConfig.pixelFormat;
> +		return Invalid;
> +	}
> +
> +	/* Adjust all the other streams. */
>  	for (const auto &[i, cfg] : utils::enumerate(config_)) {
>  
>  		LOG(ISI, Debug) << "Stream " << i << ": " << cfg.toString();

Patch
diff mbox series 

diff --git a/src/libcamera/pipeline/imx8-isi/imx8-isi.cpp b/src/libcamera/pipeline/imx8-isi/imx8-isi.cpp
index 72bc310d80ec..445fad32656c 100644
--- a/src/libcamera/pipeline/imx8-isi/imx8-isi.cpp
+++ b/src/libcamera/pipeline/imx8-isi/imx8-isi.cpp
@@ -60,6 +60,7 @@  public:
 	}
 
 	unsigned int getRawMediaBusFormat(PixelFormat *pixelFormat) const;
+	unsigned int getYuvMediaBusFormat(PixelFormat *pixelFormat) const;
 
 	std::unique_ptr<CameraSensor> sensor_;
 	std::unique_ptr<V4L2Subdevice> csis_;
@@ -255,6 +256,65 @@  unsigned int ISICameraData::getRawMediaBusFormat(PixelFormat *pixelFormat) const
 	return sensorCode;
 }
 
+/*
+ * Get a YUV/RGB media bus format from which the ISI can produce a processed
+ * stream, preferring codes with the same colour encoding as the requested
+ * pixelformat.
+ *
+ * If the sensor does not provide any YUV/RGB media bus format the ISI cannot
+ * generate any processed pixel format as it cannot debayer.
+ */
+unsigned int ISICameraData::getYuvMediaBusFormat(PixelFormat *pixelFormat) const
+{
+	std::vector<unsigned int> mbusCodes = sensor_->mbusCodes();
+
+	/*
+	 * The ISI can produce YUV/RGB pixel formats from any non-RAW Bayer
+	 * media bus formats.
+	 *
+	 * Keep the list in sync with the mxc_isi_bus_formats[] array in
+	 * the ISI driver.
+	 */
+	std::vector<unsigned int> yuvCodes = {
+		MEDIA_BUS_FMT_UYVY8_1X16,
+		MEDIA_BUS_FMT_YUV8_1X24,
+		MEDIA_BUS_FMT_RGB565_1X16,
+		MEDIA_BUS_FMT_RGB888_1X24,
+	};
+
+	std::sort(mbusCodes.begin(), mbusCodes.end());
+	std::sort(yuvCodes.begin(), yuvCodes.end());
+
+	std::vector<unsigned int> supportedCodes;
+	std::set_intersection(mbusCodes.begin(), mbusCodes.end(),
+			      yuvCodes.begin(), yuvCodes.end(),
+			      std::back_inserter(supportedCodes));
+
+	if (supportedCodes.empty()) {
+		LOG(ISI, Warning) << "Cannot find a supported YUV/RGB format";
+		*pixelFormat = {};
+
+		return 0;
+	}
+
+	/* Prefer codes with the same encoding as the requested pixel format. */
+	const PixelFormatInfo &info = PixelFormatInfo::info(*pixelFormat);
+	for (unsigned int code : supportedCodes) {
+		if (info.colourEncoding == PixelFormatInfo::ColourEncodingYUV &&
+		    (code == MEDIA_BUS_FMT_UYVY8_1X16 ||
+		     code == MEDIA_BUS_FMT_YUV8_1X24))
+			return code;
+
+		if (info.colourEncoding == PixelFormatInfo::ColourEncodingRGB &&
+		    (code == MEDIA_BUS_FMT_RGB565_1X16 ||
+		     code == MEDIA_BUS_FMT_RGB888_1X24))
+			return code;
+	}
+
+	/* Otherwise return the first found code. */
+	return supportedCodes[0];
+}
+
 /* -----------------------------------------------------------------------------
  * Camera Configuration
  */
@@ -455,6 +515,22 @@  ISICameraConfiguration::validateYuv(std::set<Stream *> &availableStreams,
 {
 	CameraConfiguration::Status status = Valid;
 
+	StreamConfiguration &yuvConfig = config_[0];
+	PixelFormat yuvPixelFormat = yuvConfig.pixelFormat;
+
+	/*
+	 * Make sure the sensor can produce a compatible YUV/RGB media bus
+	 * format. If the sensor can only produce RAW Bayer we can only fail
+	 * here as we can't adjust to anything but RAW.
+	 */
+	unsigned int yuvMediaBusCode = data_->getYuvMediaBusFormat(&yuvPixelFormat);
+	if (!yuvMediaBusCode) {
+		LOG(ISI, Error) << "Cannot adjust pixelformat "
+				<< yuvConfig.pixelFormat;
+		return Invalid;
+	}
+
+	/* Adjust all the other streams. */
 	for (const auto &[i, cfg] : utils::enumerate(config_)) {
 
 		LOG(ISI, Debug) << "Stream " << i << ": " << cfg.toString();