[{"id":28375,"web_url":"https://patchwork.libcamera.org/comment/28375/","msgid":"<20240104223453.GB14342@pendragon.ideasonboard.com>","date":"2024-01-04T22:34:53","subject":"Re: [libcamera-devel] [PATCH v2] ipa: rpi: Add hardware line rate\n\tconstraints","submitter":{"id":2,"url":"https://patchwork.libcamera.org/api/people/2/","name":"Laurent Pinchart","email":"laurent.pinchart@ideasonboard.com"},"content":"Hi Naush,\n\nThank you for the patch.\n\nOn Thu, Jan 04, 2024 at 11:38:55AM +0000, Naushir Patuck via libcamera-devel wrote:\n> Advertise hardware constraints on the pixel processing rate through the\n> Controller::HardwareConfig structure. When calculating the minimum line\n> length during a configure() operation, ensure that we don't exceed this\n> constraint.\n> \n> If we do exceed the hardware constraints, increase the modes's minimum\n> line length so the pixel processing rate falls below the hardware limit.\n> If this is not possible, throw a loud error message in the logs.\n> \n> Signed-off-by: Naushir Patuck \n> Reviewed-by: David Plowman \n> Reviewed-by: Nick Hollinghurst \n\nReviewed-by: Laurent Pinchart \n\n> ---\n> src/ipa/rpi/common/ipa_base.cpp | 27 +++++++++++++++++++++++++++\n> src/ipa/rpi/controller/controller.cpp | 20 ++++++++++++++++++++\n> src/ipa/rpi/controller/controller.h | 2 ++\n> 3 files changed, 49 insertions(+)\n> \n> diff --git a/src/ipa/rpi/common/ipa_base.cpp b/src/ipa/rpi/common/ipa_base.cpp\n> index 6ac9d5de2f88..6ec9157561cf 100644\n> --- a/src/ipa/rpi/common/ipa_base.cpp\n> +++ b/src/ipa/rpi/common/ipa_base.cpp\n> @@ -531,6 +531,33 @@ void IpaBase::setMode(const IPACameraSensorInfo &sensorInfo)\n> \tmode_.minLineLength = sensorInfo.minLineLength * (1.0s / sensorInfo.pixelRate);\n> \tmode_.maxLineLength = sensorInfo.maxLineLength * (1.0s / sensorInfo.pixelRate);\n> \n> +\t/*\n> +\t * Ensure that the maximum pixel processing rate does not exceed the ISP\n> +\t * hardware capabilities. If it does, try adjusting the minimum line\n> +\t * length to compensate if possible.\n> +\t */\n> +\tDuration minPixelTime = controller_.getHardwareConfig().minPixelProcessingTime;\n> +\tDuration pixelTime = mode_.minLineLength / mode_.width;\n> +\tif (minPixelTime && pixelTime < minPixelTime) {\n> +\t\tDuration adjustedLineLength = minPixelTime * mode_.width;\n> +\t\tif (adjustedLineLength <= mode_.maxLineLength) {\n> +\t\t\tLOG(IPARPI, Info)\n> +\t\t\t\t<< \"Adjusting mode minimum line length from \" << mode_.minLineLength\n> +\t\t\t\t<< \" to \" << adjustedLineLength << \" because of ISP constraints.\";\n> +\t\t\tmode_.minLineLength = adjustedLineLength;\n> +\t\t} else {\n> +\t\t\tLOG(IPARPI, Error)\n> +\t\t\t\t<< \"Sensor minimum line length of \" << pixelTime * mode_.width\n> +\t\t\t\t<< \" (\" << 1us / pixelTime << \" MPix/s)\"\n> +\t\t\t\t<< \" is below the minimum allowable ISP limit of \"\n> +\t\t\t\t<< adjustedLineLength\n> +\t\t\t\t<< \" (\" << 1us / minPixelTime << \" MPix/s) \";\n> +\t\t\tLOG(IPARPI, Error)\n> +\t\t\t\t<< \"THIS WILL CAUSE IMAGE CORRUPTION!!! \"\n> +\t\t\t\t<< \"Please update the camera sensor driver to allow more horizontal blanking control.\";\n> +\t\t}\n> +\t}\n> +\n> \t/*\n> \t * Set the frame length limits for the mode to ensure exposure and\n> \t * framerate calculations are clipped appropriately.\n> diff --git a/src/ipa/rpi/controller/controller.cpp b/src/ipa/rpi/controller/controller.cpp\n> index e62becd87e85..5ca98b989740 100644\n> --- a/src/ipa/rpi/controller/controller.cpp\n> +++ b/src/ipa/rpi/controller/controller.cpp\n> @@ -17,6 +17,7 @@\n> \n> using namespace RPiController;\n> using namespace libcamera;\n> +using namespace std::literals::chrono_literals;\n> \n> LOG_DEFINE_CATEGORY(RPiController)\n> \n> @@ -37,6 +38,7 @@ static const std::map HardwareConfigMap\n> \t\t\t.numGammaPoints = 33,\n> \t\t\t.pipelineWidth = 13,\n> \t\t\t.statsInline = false,\n> +\t\t\t.minPixelProcessingTime = 0s,\n> \t\t}\n> \t},\n> \t{\n> @@ -51,6 +53,24 @@ static const std::map HardwareConfigMap\n> \t\t\t.numGammaPoints = 64,\n> \t\t\t.pipelineWidth = 16,\n> \t\t\t.statsInline = true,\n> +\n> +\t\t\t/*\n> +\t\t\t * The constraint below is on the rate of pixels going\n> +\t\t\t * from CSI2 peripheral to ISP-FE (400Mpix/s, plus tiny\n> +\t\t\t * overheads per scanline, for which 380Mpix/s is a\n> +\t\t\t * conservative bound).\n> +\t\t\t *\n> +\t\t\t * There is a 64kbit data FIFO before the bottleneck,\n> +\t\t\t * which means that in all reasonable cases the\n> +\t\t\t * constraint applies at a timescale >= 1 scanline, so\n> +\t\t\t * adding horizontal blanking can prevent loss.\n> +\t\t\t *\n> +\t\t\t * If the backlog were to grow beyond 64kbit during a\n> +\t\t\t * single scanline, there could still be loss. This\n> +\t\t\t * could happen using 4 lanes at 1.5Gbps at 10bpp with\n> +\t\t\t * frames wider than ~16,000 pixels.\n> +\t\t\t */\n> +\t\t\t.minPixelProcessingTime = 1.0us / 380,\n> \t\t}\n> \t},\n> };\n> diff --git a/src/ipa/rpi/controller/controller.h b/src/ipa/rpi/controller/controller.h\n> index 6e5f595284fd..170aea740789 100644\n> --- a/src/ipa/rpi/controller/controller.h\n> +++ b/src/ipa/rpi/controller/controller.h\n> @@ -15,6 +15,7 @@\n> #include \n> #include \n> \n> +#include \n> #include \"libcamera/internal/yaml_parser.h\"\n> \n> #include \"camera_mode.h\"\n> @@ -47,6 +48,7 @@ public:\n> \t\tunsigned int numGammaPoints;\n> \t\tunsigned int pipelineWidth;\n> \t\tbool statsInline;\n> +\t\tlibcamera::utils::Duration minPixelProcessingTime;\n> \t};\n> \n> \tController();","headers":{"Return-Path":"","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id 71A0BC323E\n\tfor ;\n\tThu, 4 Jan 2024 22:34:47 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id F2E8762B41;\n\tThu, 4 Jan 2024 23:34:46 +0100 (CET)","from perceval.ideasonboard.com (perceval.ideasonboard.com\n\t[IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id 1FBFF61D82\n\tfor ;\n\tThu, 4 Jan 2024 23:34:45 +0100 (CET)","from pendragon.ideasonboard.com (213-243-189-158.bb.dnainternet.fi\n\t[213.243.189.158])\n\tby perceval.ideasonboard.com (Postfix) with ESMTPSA id 0E310B3;\n\tThu, 4 Jan 2024 23:33:43 +0100 (CET)"],"DKIM-Signature":["v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org;\n\ts=mail; t=1704407687;\n\tbh=Dr71nxYgxSsvZwogNVtDXn1zjxZz3BzDAhee72gEzT0=;\n\th=Date:To:References:In-Reply-To:Subject:List-Id:List-Unsubscribe:\n\tList-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To:Cc:\n\tFrom;\n\tb=CwcZCKyRGQ+J8Porgmw3Zr+gpXe3EJuNlFHdcw2KMSkmcfnh4dP/OhG7ABxMZT+E4\n\twQsHKLasv/1rO4Bkpxf8NNpn04hFFhFRUzGEtxj18MaDaBEl5SCp1SdxeCteDuGq+I\n\tCO4BgCiwFGNy/hGTQ9vTV97lNa6tTOybIsDqtWIdPPY9cF5cAl3SpVtwYofrm9YvPd\n\tn0dKrE8vPC7DQXYGMfhowVdWmRIYsK6ZH+R97RkN7OdwcSeDH451ufFrq8MZLM4T+Y\n\t+DDzTcWtiNTgKBXCHkO2z9WckNMJTq0okGP/exK6HGnVnDz6CLOSP2pYAH0usiaUQJ\n\tfU6N34ZJlHQWA==","v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com;\n\ts=mail; t=1704407624;\n\tbh=Dr71nxYgxSsvZwogNVtDXn1zjxZz3BzDAhee72gEzT0=;\n\th=Date:From:To:Cc:Subject:References:In-Reply-To:From;\n\tb=Yfb8LYf1s9VhIXg0rw3/0VGezgQOlJBXK0MGt0G1lIMzoFjeii/TVA9VHiKOL2mdx\n\tuZ5I7j0pUeMkGcquomd/ZMvrj6oUCRP6t21Vf064WybyrqK8QLOrfEQEdmbEKkByt9\n\tiwFY2Pys6Vr7t1gDQ/6wjBldB6ioamxKiXoFyj34="],"Authentication-Results":"lancelot.ideasonboard.com; dkim=pass (1024-bit key; \n\tunprotected) header.d=ideasonboard.com\n\theader.i=@ideasonboard.com\n\theader.b=\"Yfb8LYf1\"; dkim-atps=neutral","Date":"Fri, 5 Jan 2024 00:34:53 +0200","To":"Naushir Patuck ","Message-ID":"<20240104223453.GB14342@pendragon.ideasonboard.com>","References":"<20240104113855.23865-1-naush@raspberrypi.com>","MIME-Version":"1.0","Content-Type":"text/plain; charset=utf-8","Content-Disposition":"inline","In-Reply-To":"<20240104113855.23865-1-naush@raspberrypi.com>","Subject":"Re: [libcamera-devel] [PATCH v2] ipa: rpi: Add hardware line rate\n\tconstraints","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","From":"Laurent Pinchart via libcamera-devel\n\t","Reply-To":"Laurent Pinchart ","Cc":"libcamera-devel@lists.libcamera.org","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" "}}]