[libcamera-devel,v6,2/6] pipeline: raspberrypi: Set the ISP Output1 to 1/4 resolution if unused

11192 diff mbox series

In preparation for fast colour denoise, set the low resolution ISP
output stream (Output1) to a 1/4 resolution of the application requested
stream (Output0). This only happens if the application has not requested
an additional YUV or RGB stream.

We also constrain this 1/4 resolution to at most 1200 pixels in the
largest dimension to avoid being too taxing on memory usage and system
bandwidth.

Also switch the default StreamRole::VideoRecording to YUV420 to allow
fast colour denoise to run.

Signed-off-by: Naushir Patuck <naush@raspberrypi.com>
Reviewed-by: David Plowman <david.plowman@raspberrypi.com>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
Signed-off-by: Naushir Patuck <naush@raspberrypi.com>
---
 .../pipeline/raspberrypi/raspberrypi.cpp      | 46 ++++++++++++++++++-
 1 file changed, 45 insertions(+), 1 deletion(-)

Hi Naush,

Thank you for the patch.

On Mon, Feb 08, 2021 at 03:07:34PM +0000, Naushir Patuck wrote:
> In preparation for fast colour denoise, set the low resolution ISP
> output stream (Output1) to a 1/4 resolution of the application requested
> stream (Output0). This only happens if the application has not requested
> an additional YUV or RGB stream.
> 
> We also constrain this 1/4 resolution to at most 1200 pixels in the
> largest dimension to avoid being too taxing on memory usage and system
> bandwidth.
> 
> Also switch the default StreamRole::VideoRecording to YUV420 to allow
> fast colour denoise to run.
> 
> Signed-off-by: Naushir Patuck <naush@raspberrypi.com>
> Reviewed-by: David Plowman <david.plowman@raspberrypi.com>
> Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
> Signed-off-by: Naushir Patuck <naush@raspberrypi.com>

A single SoB line is enough ;-)

> ---
>  .../pipeline/raspberrypi/raspberrypi.cpp      | 46 ++++++++++++++++++-
>  1 file changed, 45 insertions(+), 1 deletion(-)
> 
> diff --git a/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp b/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp
> index 183a690dbca1..7ecb080f62d3 100644
> --- a/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp
> +++ b/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp
> @@ -494,8 +494,16 @@ CameraConfiguration *PipelineHandlerRPi::generateConfiguration(Camera *camera,
>  			break;
>  
>  		case StreamRole::VideoRecording:
> +			/*
> +			 * The colour denoise algorithm require the analysis

s/require/requires/

I'll fix when applying.

Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>

> +			 * image, produced by the second ISP output, to be in
> +			 * YUV420 format. Select this format as the default, to
> +			 * maximize chances that it will be picked by
> +			 * applications and enable usage of the colour denoise
> +			 * algorithm.
> +			 */
>  			fmts = data->isp_[Isp::Output0].dev()->formats();
> -			pixelFormat = formats::NV12;
> +			pixelFormat = formats::YUV420;
>  			size = { 1920, 1080 };
>  			bufferCount = 4;
>  			outCount++;
> @@ -607,6 +615,7 @@ int PipelineHandlerRPi::configure(Camera *camera, CameraConfiguration *config)
>  	 * StreamConfiguration appropriately.
>  	 */
>  	V4L2DeviceFormat format;
> +	bool output1Set = false;
>  	for (unsigned i = 0; i < config->size(); i++) {
>  		StreamConfiguration &cfg = config->at(i);
>  
> @@ -631,6 +640,9 @@ int PipelineHandlerRPi::configure(Camera *camera, CameraConfiguration *config)
>  		format.size = cfg.size;
>  		format.fourcc = fourcc;
>  
> +		LOG(RPI, Debug) << "Setting " << stream->name() << " to "
> +				<< format.toString();
> +
>  		ret = stream->dev()->setFormat(&format);
>  		if (ret)
>  			return -EINVAL;
> @@ -644,6 +656,38 @@ int PipelineHandlerRPi::configure(Camera *camera, CameraConfiguration *config)
>  
>  		cfg.setStream(stream);
>  		stream->setExternal(true);
> +
> +		if (i != maxIndex)
> +			output1Set = true;
> +	}
> +
> +	/*
> +	 * If ISP::Output1 stream has not been requested by the application, we
> +	 * set it up for internal use now. This second stream will be used for
> +	 * fast colour denoise, and must be a quarter resolution of the ISP::Output0
> +	 * stream. However, also limit the maximum size to 1200 pixels in the
> +	 * larger dimension, just to avoid being wasteful with buffer allocations
> +	 * and memory bandwidth.
> +	 *
> +	 * \todo If Output 1 format is not YUV420, Output 1 ought to be disabled as
> +	 * colour denoise will not run.
> +	 */
> +	if (!output1Set) {
> +		V4L2DeviceFormat output1Format = format;
> +		constexpr Size maxDimensions(1200, 1200);
> +		const Size limit = maxDimensions.boundedToAspectRatio(format.size);
> +
> +		output1Format.size = (format.size / 2).boundedTo(limit).alignedDownTo(2, 2);
> +
> +		LOG(RPI, Debug) << "Setting ISP Output1 (internal) to "
> +				<< output1Format.toString();
> +
> +		ret = data->isp_[Isp::Output1].dev()->setFormat(&output1Format);
> +		if (ret) {
> +			LOG(RPI, Error) << "Failed to set format on ISP Output1: "
> +					<< ret;
> +			return -EINVAL;
> +		}
>  	}
>  
>  	/* ISP statistics output format. */