From patchwork Thu Jan 4 11:38:55 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Naushir Patuck X-Patchwork-Id: 19360 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id F2F4BBDB1D for ; Thu, 4 Jan 2024 11:39:04 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 5C49562B41; Thu, 4 Jan 2024 12:39:04 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1704368344; bh=caPkXhcQMHaYW1zTzID4GH9oXuOQGwWLlhWB5p0T8o4=; h=To:Date:Subject:List-Id:List-Unsubscribe:List-Archive:List-Post: List-Help:List-Subscribe:From:Reply-To:From; b=3qQveGb5QvW+0BtHlr/3f6auYd+qlsM8cw3EL2/83BWgywY1rUeag1d9l5wFpOE4L iur2JHUOWE0f3N5T7/Gq91QfSBfC4c6mMVMMbixLP1ZznrFqUU0s7O+iitqoZps8Z7 czMTkRpd/a5aY/N5w8dRn3BmEmaTEBR3ojB40SkcXBNYpI6C78QwQ8Klev4dl7PrJq /RhKdSZR9qjfReU9TSEstvJequR+/JJTp3gKtX8wrSwgTE8DzvXo4gpmWx/TZL2i+m B7APYctqcJIzOGd50EUshnRddxwMg1b2M9o7zq5SkDQc49Q1UUybbAsBzrXYuG0QRp hBDbRf1Ub1XPQ== Received: from mail-wm1-x329.google.com (mail-wm1-x329.google.com [IPv6:2a00:1450:4864:20::329]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 03E8F61D82 for ; Thu, 4 Jan 2024 12:39:02 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="Lm2ENOnQ"; dkim-atps=neutral Received: by mail-wm1-x329.google.com with SMTP id 5b1f17b1804b1-40d88fff7faso3498005e9.3 for ; Thu, 04 Jan 2024 03:39:01 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; t=1704368341; x=1704973141; darn=lists.libcamera.org; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:from:to:cc:subject:date:message-id:reply-to; bh=KQ2XdtIJKFUdHvKJJqI57au9cLadPk5h2KB/ovL+qAk=; b=Lm2ENOnQ1RWhT/gHYZkgSoxdqahAS6v18nmzp8Br5/4MI2NDO164pcL+m9BDvhZfug thPwT5ejjyd0nioMx9vkkaGvvpDqDKG49GzFZHHvyfIL4Zwj2mdKRtbK9OElWMPtucww 9+XOaBif2MnAtKWYUSOXj6C6Otnxe5boZnfEG0TqhZvFauQpZ5qcqwv6Fou3u2VdhWfz 68rmVD2NzmGs0VPO/HPKs6qRcazQaB2ZY6Qr1pJ+e/g2JYX+Awtp1yDKA0a97TuSd/9i eJ4X2CzPFn2OdLgH/Dk0/D17AksBjbEqPuX2RjP8xZChJm4z61z1lUlpohDpogf2iQvO oMzg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1704368341; x=1704973141; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=KQ2XdtIJKFUdHvKJJqI57au9cLadPk5h2KB/ovL+qAk=; b=jNQwhu8sHF3s71XfVbIzsz1X8NgsqBy0+jG2X4ADFUp+cR15srFmT+oZnfxSuG5AOK QQ7rNkE1OEnzWP/Hz8h4AUeqRCbflAgpean7xiJ/Hf/HGzqlrTczpZtikFD9TMiNVUg0 sGz+aYsJP2qBwgvq486HwlMNfolJJIb7KIuenKf20Ft0tegp0kfNbm8CStc9k7fbc2ol G/PIHjAJY+NQfpaXPr8ESGUzBE/Rd9iBEKQIqjtx1SddElZXt0vHVGcl0aInvIfaQe4v mhWhfBEpzJpcuLvZN4DIuQ3/lD0THXlNX77jV9qbQeK1+Dqi4v+gZnPfJujzeLwgDrUG dgzw== X-Gm-Message-State: AOJu0Yx1EwK9sEMhUbM1AefGLyhxaJFHQ6XzcDiibUkA6bqqIOcK6tVm YNJB+Bfrf4LtsyfY5Oqkjl/y4KmPV2MalKdYSVBtWylu1f8= X-Google-Smtp-Source: AGHT+IH32Bjl1AJY7J6sytMrzTsyYgbpiJ7t7Xfomir/F7z9nPr620tvPcL8kLIckTxncOQGZpQX6Q== X-Received: by 2002:a7b:c391:0:b0:40d:71f3:d744 with SMTP id s17-20020a7bc391000000b0040d71f3d744mr132835wmj.343.1704368341133; Thu, 04 Jan 2024 03:39:01 -0800 (PST) Received: from localhost.localdomain ([188.39.149.98]) by smtp.gmail.com with ESMTPSA id l27-20020a05600c1d1b00b0040d6e07a147sm5458117wms.23.2024.01.04.03.39.00 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 04 Jan 2024 03:39:00 -0800 (PST) To: libcamera-devel@lists.libcamera.org Date: Thu, 4 Jan 2024 11:38:55 +0000 Message-Id: <20240104113855.23865-1-naush@raspberrypi.com> X-Mailer: git-send-email 2.34.1 MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v2] ipa: rpi: Add hardware line rate constraints X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Naushir Patuck via libcamera-devel From: Naushir Patuck Reply-To: Naushir Patuck Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Advertise hardware constraints on the pixel processing rate through the Controller::HardwareConfig structure. When calculating the minimum line length during a configure() operation, ensure that we don't exceed this constraint. If we do exceed the hardware constraints, increase the modes's minimum line length so the pixel processing rate falls below the hardware limit. If this is not possible, throw a loud error message in the logs. Signed-off-by: Naushir Patuck Reviewed-by: David Plowman Reviewed-by: Nick Hollinghurst Reviewed-by: Laurent Pinchart --- src/ipa/rpi/common/ipa_base.cpp | 27 +++++++++++++++++++++++++++ src/ipa/rpi/controller/controller.cpp | 20 ++++++++++++++++++++ src/ipa/rpi/controller/controller.h | 2 ++ 3 files changed, 49 insertions(+) diff --git a/src/ipa/rpi/common/ipa_base.cpp b/src/ipa/rpi/common/ipa_base.cpp index 6ac9d5de2f88..6ec9157561cf 100644 --- a/src/ipa/rpi/common/ipa_base.cpp +++ b/src/ipa/rpi/common/ipa_base.cpp @@ -531,6 +531,33 @@ void IpaBase::setMode(const IPACameraSensorInfo &sensorInfo) mode_.minLineLength = sensorInfo.minLineLength * (1.0s / sensorInfo.pixelRate); mode_.maxLineLength = sensorInfo.maxLineLength * (1.0s / sensorInfo.pixelRate); + /* + * Ensure that the maximum pixel processing rate does not exceed the ISP + * hardware capabilities. If it does, try adjusting the minimum line + * length to compensate if possible. + */ + Duration minPixelTime = controller_.getHardwareConfig().minPixelProcessingTime; + Duration pixelTime = mode_.minLineLength / mode_.width; + if (minPixelTime && pixelTime < minPixelTime) { + Duration adjustedLineLength = minPixelTime * mode_.width; + if (adjustedLineLength <= mode_.maxLineLength) { + LOG(IPARPI, Info) + << "Adjusting mode minimum line length from " << mode_.minLineLength + << " to " << adjustedLineLength << " because of ISP constraints."; + mode_.minLineLength = adjustedLineLength; + } else { + LOG(IPARPI, Error) + << "Sensor minimum line length of " << pixelTime * mode_.width + << " (" << 1us / pixelTime << " MPix/s)" + << " is below the minimum allowable ISP limit of " + << adjustedLineLength + << " (" << 1us / minPixelTime << " MPix/s) "; + LOG(IPARPI, Error) + << "THIS WILL CAUSE IMAGE CORRUPTION!!! " + << "Please update the camera sensor driver to allow more horizontal blanking control."; + } + } + /* * Set the frame length limits for the mode to ensure exposure and * framerate calculations are clipped appropriately. diff --git a/src/ipa/rpi/controller/controller.cpp b/src/ipa/rpi/controller/controller.cpp index e62becd87e85..5ca98b989740 100644 --- a/src/ipa/rpi/controller/controller.cpp +++ b/src/ipa/rpi/controller/controller.cpp @@ -17,6 +17,7 @@ using namespace RPiController; using namespace libcamera; +using namespace std::literals::chrono_literals; LOG_DEFINE_CATEGORY(RPiController) @@ -37,6 +38,7 @@ static const std::map HardwareConfigMap .numGammaPoints = 33, .pipelineWidth = 13, .statsInline = false, + .minPixelProcessingTime = 0s, } }, { @@ -51,6 +53,24 @@ static const std::map HardwareConfigMap .numGammaPoints = 64, .pipelineWidth = 16, .statsInline = true, + + /* + * The constraint below is on the rate of pixels going + * from CSI2 peripheral to ISP-FE (400Mpix/s, plus tiny + * overheads per scanline, for which 380Mpix/s is a + * conservative bound). + * + * There is a 64kbit data FIFO before the bottleneck, + * which means that in all reasonable cases the + * constraint applies at a timescale >= 1 scanline, so + * adding horizontal blanking can prevent loss. + * + * If the backlog were to grow beyond 64kbit during a + * single scanline, there could still be loss. This + * could happen using 4 lanes at 1.5Gbps at 10bpp with + * frames wider than ~16,000 pixels. + */ + .minPixelProcessingTime = 1.0us / 380, } }, }; diff --git a/src/ipa/rpi/controller/controller.h b/src/ipa/rpi/controller/controller.h index 6e5f595284fd..170aea740789 100644 --- a/src/ipa/rpi/controller/controller.h +++ b/src/ipa/rpi/controller/controller.h @@ -15,6 +15,7 @@ #include #include +#include #include "libcamera/internal/yaml_parser.h" #include "camera_mode.h" @@ -47,6 +48,7 @@ public: unsigned int numGammaPoints; unsigned int pipelineWidth; bool statsInline; + libcamera::utils::Duration minPixelProcessingTime; }; Controller();