From patchwork Mon Feb 8 15:07:34 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Naushir Patuck X-Patchwork-Id: 11192 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id CEBC9BD160 for ; Mon, 8 Feb 2021 15:07:58 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 9DCA860D2D; Mon, 8 Feb 2021 16:07:58 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="BkQsrYjE"; dkim-atps=neutral Received: from mail-wr1-x431.google.com (mail-wr1-x431.google.com [IPv6:2a00:1450:4864:20::431]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id C7EA760D18 for ; Mon, 8 Feb 2021 16:07:56 +0100 (CET) Received: by mail-wr1-x431.google.com with SMTP id n6so4707359wrv.8 for ; Mon, 08 Feb 2021 07:07:56 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=ryNStR4waILijeEpS9cXHD4lYAxZzMUYzfSprSf3nS0=; b=BkQsrYjE5ZtzOYWQdJqvDSYgHhp8cxRaugY5P/+4+zCLuJMrzr1k9CQPiR7TLk1KeT F1YlMpWTBMn9s1zlyXl4oeR7E7t81oGpDe4hAAtiY1JbZzbtVd3Yo/tjyVJx4P33sMLb CFbSeac0p36JNMSQ5NVb8gKNwDjdH+Fs0mLpO3mhfa+DstU8itqcoaSngWaZIMrKEUJw ldwcu99/J5fLXlT3yrzJ0aNMjQh+TADy8ezbq73rM5hhfxzJZMv+EIUtiKOHDABAly47 PcJUeMOfqTiYKWWJnvsF/OEW73ZLUiLD410V4Vq2jEhsiZLFfhnPvU6/ICE3cg2yLs/w 4mQw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=ryNStR4waILijeEpS9cXHD4lYAxZzMUYzfSprSf3nS0=; b=i6kBR927EZtEhowE8sp0XF9S7BvESUsigMLVq7p5PqOSfiHC5wwDxHjSXQ394O/9RX GXuYqpzpPuLVQJOirwlzRU2jOM/a915rSYLuEEd1JzQxf4NqOmopyITX+VcMU6tBdeav XdX6cBg48m75xMPuTjViO7OYzbZHb48SK33aiE/m/4d9HJEg2Tf4NwGZNncWPbLtKWsF CEO2fasP5WHL11CCodKv7lOqv3A8KEfiiDpjbXITlX8/jsQSMHkdt7ahlqa4woMu09BF cSpU0JVnBRWp4L+rVXONp9QLgn+HV5du0t+UDVyp9z3Xaa+aEQ8jUDSmLJfqz8MwZD9S usMg== X-Gm-Message-State: AOAM530GnJBp6UBxHPcpzD47F6IJQ8TSor+qanghyZ4GkWRoj2Am/ClS JGd7M4PfGIZiRf+W7/p5tRhCvDdIEwkdagU9 X-Google-Smtp-Source: ABdhPJwyEaWzfsa+PvS4FLqnrzPtZq9iJB/WYez6RQDZ1he6LsnZSI/EYoHEOUiCRf/cYJAcB6u/GQ== X-Received: by 2002:adf:ed45:: with SMTP id u5mr20235183wro.358.1612796874100; Mon, 08 Feb 2021 07:07:54 -0800 (PST) Received: from naush-laptop.pitowers.org (cust51-dsl50.idnet.net. [212.69.50.51]) by smtp.gmail.com with ESMTPSA id a17sm31261389wrx.63.2021.02.08.07.07.53 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 08 Feb 2021 07:07:53 -0800 (PST) From: Naushir Patuck To: libcamera-devel@lists.libcamera.org Date: Mon, 8 Feb 2021 15:07:34 +0000 Message-Id: <20210208150738.2293520-3-naush@raspberrypi.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210208150738.2293520-1-naush@raspberrypi.com> References: <20210208150738.2293520-1-naush@raspberrypi.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v6 2/6] pipeline: raspberrypi: Set the ISP Output1 to 1/4 resolution if unused X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" In preparation for fast colour denoise, set the low resolution ISP output stream (Output1) to a 1/4 resolution of the application requested stream (Output0). This only happens if the application has not requested an additional YUV or RGB stream. We also constrain this 1/4 resolution to at most 1200 pixels in the largest dimension to avoid being too taxing on memory usage and system bandwidth. Also switch the default StreamRole::VideoRecording to YUV420 to allow fast colour denoise to run. Signed-off-by: Naushir Patuck Reviewed-by: David Plowman Reviewed-by: Kieran Bingham Signed-off-by: Naushir Patuck Reviewed-by: Laurent Pinchart --- .../pipeline/raspberrypi/raspberrypi.cpp | 46 ++++++++++++++++++- 1 file changed, 45 insertions(+), 1 deletion(-) diff --git a/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp b/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp index 183a690dbca1..7ecb080f62d3 100644 --- a/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp +++ b/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp @@ -494,8 +494,16 @@ CameraConfiguration *PipelineHandlerRPi::generateConfiguration(Camera *camera, break; case StreamRole::VideoRecording: + /* + * The colour denoise algorithm require the analysis + * image, produced by the second ISP output, to be in + * YUV420 format. Select this format as the default, to + * maximize chances that it will be picked by + * applications and enable usage of the colour denoise + * algorithm. + */ fmts = data->isp_[Isp::Output0].dev()->formats(); - pixelFormat = formats::NV12; + pixelFormat = formats::YUV420; size = { 1920, 1080 }; bufferCount = 4; outCount++; @@ -607,6 +615,7 @@ int PipelineHandlerRPi::configure(Camera *camera, CameraConfiguration *config) * StreamConfiguration appropriately. */ V4L2DeviceFormat format; + bool output1Set = false; for (unsigned i = 0; i < config->size(); i++) { StreamConfiguration &cfg = config->at(i); @@ -631,6 +640,9 @@ int PipelineHandlerRPi::configure(Camera *camera, CameraConfiguration *config) format.size = cfg.size; format.fourcc = fourcc; + LOG(RPI, Debug) << "Setting " << stream->name() << " to " + << format.toString(); + ret = stream->dev()->setFormat(&format); if (ret) return -EINVAL; @@ -644,6 +656,38 @@ int PipelineHandlerRPi::configure(Camera *camera, CameraConfiguration *config) cfg.setStream(stream); stream->setExternal(true); + + if (i != maxIndex) + output1Set = true; + } + + /* + * If ISP::Output1 stream has not been requested by the application, we + * set it up for internal use now. This second stream will be used for + * fast colour denoise, and must be a quarter resolution of the ISP::Output0 + * stream. However, also limit the maximum size to 1200 pixels in the + * larger dimension, just to avoid being wasteful with buffer allocations + * and memory bandwidth. + * + * \todo If Output 1 format is not YUV420, Output 1 ought to be disabled as + * colour denoise will not run. + */ + if (!output1Set) { + V4L2DeviceFormat output1Format = format; + constexpr Size maxDimensions(1200, 1200); + const Size limit = maxDimensions.boundedToAspectRatio(format.size); + + output1Format.size = (format.size / 2).boundedTo(limit).alignedDownTo(2, 2); + + LOG(RPI, Debug) << "Setting ISP Output1 (internal) to " + << output1Format.toString(); + + ret = data->isp_[Isp::Output1].dev()->setFormat(&output1Format); + if (ret) { + LOG(RPI, Error) << "Failed to set format on ISP Output1: " + << ret; + return -EINVAL; + } } /* ISP statistics output format. */