From patchwork Wed Jan 7 11:05:24 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Plowman X-Patchwork-Id: 25680 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id D7186BE08B for ; Wed, 7 Jan 2026 11:05:49 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 8F43061FA3; Wed, 7 Jan 2026 12:05:49 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="tYvUrLnd"; dkim-atps=neutral Received: from mail-wm1-x336.google.com (mail-wm1-x336.google.com [IPv6:2a00:1450:4864:20::336]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id D049161FA0 for ; Wed, 7 Jan 2026 12:05:47 +0100 (CET) Received: by mail-wm1-x336.google.com with SMTP id 5b1f17b1804b1-477a219dbcaso16147165e9.3 for ; Wed, 07 Jan 2026 03:05:47 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; t=1767783947; x=1768388747; darn=lists.libcamera.org; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:from:to:cc:subject:date:message-id:reply-to; bh=l+3MD26GUc2WJc9Bt7MR6kF382OZwAZWQpNe9SKDouY=; b=tYvUrLndeO/5LNIt/YC1LOlTLg1/0ZGBFFqbttAn6vEWXFmbOjDNu0FCJxFEUICPBu GAYNQIG2nUNoepWN853KCEk9oN8TdOCEic/p/j3uK5DhEgZAJdA274JA3fMaFXCiJnB3 X4Hpvo8I3eVf9Sl65XaDnfmVqIbJ2dmfKtGGHvVHsUXTdjvVxU3I1f7orFCIiEJSp5VD qakIpCu4jxmksvXLByxcBYK4mMVPqm+2frgtfD1s1C4WaoEFlFvbAviWx6953Flo6OdV uyR7rsXvyceG8ULIG4NuUSFB8a4dYsr4gfEFvldc7o9E0P1qjqMHxqgnuV5J2SNsKSOn kCrA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1767783947; x=1768388747; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:x-gm-gg:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=l+3MD26GUc2WJc9Bt7MR6kF382OZwAZWQpNe9SKDouY=; b=sIwSXxmgUUmIArO0xnhoMrJrZBP1Jj+wtamybSLt6aQNk7EgP39DiSw4UAPbR4szsh ej5w39Rpdo6Cjh4Yv85A0e0ShxC+glCGS798NA5tCHxuPqFmAU1Q94Cy2LPytrYzrk6u 5cgW1jjxbyNVg+/RP/V1L1KdD7k83ygk2xQYLEMX4AxU9VDfJbiaesUQzp1JMn3kz9m3 ba7YdQQENno8SYbE2H2Ad1GgrDa0AVjO2q2KKuYdnLvyYi0Hl1953QWhhSFDnBtTK83Q XFtZRk+ReiJcXEORbiMkvYy20fMrdfCg9Qw0Z96759p0orqNkM0aBhEUQM2Do7JvohjO dBvw== X-Gm-Message-State: AOJu0YxhqJ5fZ+HSrg1yLvw2usRHVveLqXTMyqqEe2STVr7iAtvTnHb8 tdlsmdCVV+LYJ6wFC5BJBzl4Qz7chmLMlpBZ4s3Kt4nts18RqTlpGoTgFPi8B2V0TAKdh6qQ7LS WzuDOLBk= X-Gm-Gg: AY/fxX52sYveDlRz9g9uZ28YR2/SfixmjcmJRzjKnjIkcJvmat3EcrFTGJxX7w6PKYv dM/5BMI6h3i9TnfHryC5fPolzSKH+KTA/5pKlRzofPnJ425jLz6DZOAOoVN/9a7JGlQTaKVpfo3 zDBRSkJ7hRSiDgv968P5sjf7evfOHOsTTwrEEgXd04D4mF/9yuUlex2lbGio/M3EMGqTvhtbIo9 NaqsYRGCc8HOdnmOJZVxhyf6ZIJIl82NdEEIi8xhheU20wI3oyK0Olb4FTIoq+A4gn7Wg820EfH vH3rfyxr5REDDIiRfN3BtT3az5EvxCe9DBGo6icBERJPHVS/Wh0GldaTaOK3es8ODowAUjhvFV4 Fu/GgefYQB+/AJuP9mwbbZMpV8PoaH5u1sd8pjgZRIWoYYayIcy1+DJlkMODQXbU87eTmE+T6Dk Hn3pBs9SJwXdNsukqVqcEqgTk0NVn5YB6YLQerFuJUURbaHfepI5iPQ5HXCVKy/Fd59YuVY+KVi ROrdzJxbMSAb5sTSfXLABBETfzqKg== X-Google-Smtp-Source: AGHT+IGM+NgmRmxi14vaJ+qDYabGDhs64v67CokKL1ZIatwyRQTimy4wbs/8Qq9Y0CWOHB3nyvtWgQ== X-Received: by 2002:a05:600c:1d14:b0:477:7b16:5fb1 with SMTP id 5b1f17b1804b1-47d84b0a96emr20505125e9.7.1767783946919; Wed, 07 Jan 2026 03:05:46 -0800 (PST) Received: from davidp-pi5.pitowers.org ([2a00:1098:3142:1f:88ea:c658:5b20:5e46]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-47d7f410c86sm93821345e9.3.2026.01.07.03.05.46 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 07 Jan 2026 03:05:46 -0800 (PST) From: David Plowman To: libcamera-devel@lists.libcamera.org Cc: David Plowman Subject: [PATCH v2] libcamera: rpi: Make the controller min frame duration configurable Date: Wed, 7 Jan 2026 11:05:24 +0000 Message-ID: <20260107110544.16422-1-david.plowman@raspberrypi.com> X-Mailer: git-send-email 2.47.3 MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" The controller min frame duration is used to rate limit how often we run IPAs. Historically this has been set to 33333us, meaning that the algorithms effectively skip frames when the camera is running faster than 30fps. This patch adds a small amount of plumbing that allows this value to be set in the Raspberry Pi configuration file. Some applications or platforms (such as Pi 5) are easily capable of running these more often, should there be a need to do so. Signed-off-by: David Plowman --- include/libcamera/ipa/raspberrypi.mojom | 1 + src/ipa/rpi/common/ipa_base.cpp | 10 +++++++- src/ipa/rpi/common/ipa_base.h | 2 ++ .../pipeline/rpi/common/pipeline_base.cpp | 24 ++++++++++++++----- .../pipeline/rpi/common/pipeline_base.h | 5 ++++ .../pipeline/rpi/pisp/data/example.yaml | 6 +++++ .../pipeline/rpi/vc4/data/example.yaml | 6 +++++ 7 files changed, 47 insertions(+), 7 deletions(-) diff --git a/include/libcamera/ipa/raspberrypi.mojom b/include/libcamera/ipa/raspberrypi.mojom index 12b083e9..1b7e0358 100644 --- a/include/libcamera/ipa/raspberrypi.mojom +++ b/include/libcamera/ipa/raspberrypi.mojom @@ -18,6 +18,7 @@ struct SensorConfig { struct InitParams { bool lensPresent; libcamera.IPACameraSensorInfo sensorInfo; + float controllerMinFrameDurationUs; /* PISP specific */ libcamera.SharedFD fe; libcamera.SharedFD be; diff --git a/src/ipa/rpi/common/ipa_base.cpp b/src/ipa/rpi/common/ipa_base.cpp index 14aba450..0a1fc0e9 100644 --- a/src/ipa/rpi/common/ipa_base.cpp +++ b/src/ipa/rpi/common/ipa_base.cpp @@ -184,6 +184,14 @@ int32_t IpaBase::init(const IPASettings &settings, const InitParams ¶ms, Ini result->controlInfo = ControlInfoMap(std::move(ctrlMap), controls::controls); + /* + * This determines the minimum allowable inter-frame duration to run the + * controller algorithms. If the pipeline handler provider frames at a + * rate higher than this, we rate-limit the controller Prepare() and + * Process() calls to lower than or equal to this rate. + */ + controllerMinFrameDuration_ = params.controllerMinFrameDurationUs * 1us; + return platformInit(params, result); } @@ -465,7 +473,7 @@ void IpaBase::prepareIsp(const PrepareParams ¶ms) /* Allow a 10% margin on the comparison below. */ Duration delta = (frameTimestamp - lastRunTimestamp_) * 1.0ns; if (lastRunTimestamp_ && frameCount_ > invalidCount_ && - delta < controllerMinFrameDuration * 0.9 && !hdrChange) { + delta < controllerMinFrameDuration_ * 0.9 && !hdrChange) { /* * Ensure we merge the previous frame's metadata with the current * frame. This will not overwrite exposure/gain values for the diff --git a/src/ipa/rpi/common/ipa_base.h b/src/ipa/rpi/common/ipa_base.h index 5348f2ea..90f018b2 100644 --- a/src/ipa/rpi/common/ipa_base.h +++ b/src/ipa/rpi/common/ipa_base.h @@ -142,6 +142,8 @@ private: } flickerState_; bool awbEnabled_; + + utils::Duration controllerMinFrameDuration_; }; } /* namespace ipa::RPi */ diff --git a/src/libcamera/pipeline/rpi/common/pipeline_base.cpp b/src/libcamera/pipeline/rpi/common/pipeline_base.cpp index fb8e466f..b7655d8d 100644 --- a/src/libcamera/pipeline/rpi/common/pipeline_base.cpp +++ b/src/libcamera/pipeline/rpi/common/pipeline_base.cpp @@ -33,6 +33,12 @@ LOG_DEFINE_CATEGORY(RPI) using StreamFlag = RPi::Stream::StreamFlag; +/* + * The IPA's algorithms will not be called more often than this many + * microseconds. The default corresponds to 30fps. + */ +constexpr float defaultControllerMinimumFrameDurationUs = 1000000.0 / 30.0; + namespace { constexpr unsigned int defaultRawBitDepth = 12; @@ -800,6 +806,12 @@ int PipelineHandlerBase::registerCamera(std::unique_ptr &camera if (!data->sensor_) return -EINVAL; + ret = data->loadPipelineConfiguration(); + if (ret) { + LOG(RPI, Error) << "Unable to load pipeline configuration"; + return ret; + } + /* Populate the map of sensor supported formats and sizes. */ for (auto const mbusCode : data->sensor_->mbusCodes()) data->sensorFormats_.emplace(mbusCode, @@ -859,12 +871,6 @@ int PipelineHandlerBase::registerCamera(std::unique_ptr &camera if (ret) return ret; - ret = data->loadPipelineConfiguration(); - if (ret) { - LOG(RPI, Error) << "Unable to load pipeline configuration"; - return ret; - } - /* Setup the general IPA signal handlers. */ data->frontendDevice()->dequeueTimeout.connect(data, &RPi::CameraData::cameraTimeout); data->frontendDevice()->frameStart.connect(data, &RPi::CameraData::frameStarted); @@ -1096,6 +1102,7 @@ int CameraData::loadPipelineConfiguration() { config_ = { .cameraTimeoutValue = 0, + .controllerMinFrameDurationUs = defaultControllerMinimumFrameDurationUs, }; /* Initial configuration of the platform, in case no config file is present */ @@ -1145,6 +1152,9 @@ int CameraData::loadPipelineConfiguration() frontendDevice()->setDequeueTimeout(config_.cameraTimeoutValue * 1ms); } + config_.controllerMinFrameDurationUs = + phConfig["controller_min_frame_duration_us"].get(config_.controllerMinFrameDurationUs); + return platformPipelineConfigure(root); } @@ -1173,6 +1183,8 @@ int CameraData::loadIPA(ipa::RPi::InitResult *result) } params.lensPresent = !!sensor_->focusLens(); + params.controllerMinFrameDurationUs = config_.controllerMinFrameDurationUs; + ret = platformInitIpa(params); if (ret) return ret; diff --git a/src/libcamera/pipeline/rpi/common/pipeline_base.h b/src/libcamera/pipeline/rpi/common/pipeline_base.h index 6257a934..597eb587 100644 --- a/src/libcamera/pipeline/rpi/common/pipeline_base.h +++ b/src/libcamera/pipeline/rpi/common/pipeline_base.h @@ -169,6 +169,11 @@ public: * on frame durations. */ unsigned int cameraTimeoutValue; + /* + * The minimum frame duration between the IPA's calls to the + * algorithms themselves (in microseconds). + */ + float controllerMinFrameDurationUs; }; Config config_; diff --git a/src/libcamera/pipeline/rpi/pisp/data/example.yaml b/src/libcamera/pipeline/rpi/pisp/data/example.yaml index baf03be7..c5edbba0 100644 --- a/src/libcamera/pipeline/rpi/pisp/data/example.yaml +++ b/src/libcamera/pipeline/rpi/pisp/data/example.yaml @@ -36,5 +36,11 @@ # framebuffers required for its operation. # # "disable_hdr": false, + + # Limits the rate at which IPAs are called. The algorithms will + # be skipped until this many microseconds have elapsed since + # the last call. The default value represents a 30fps limit. + # + # "controller_min_frame_duration_us": 33333.333, } } diff --git a/src/libcamera/pipeline/rpi/vc4/data/example.yaml b/src/libcamera/pipeline/rpi/vc4/data/example.yaml index 27e54348..2ee2b864 100644 --- a/src/libcamera/pipeline/rpi/vc4/data/example.yaml +++ b/src/libcamera/pipeline/rpi/vc4/data/example.yaml @@ -37,5 +37,11 @@ # timeout value. # # "camera_timeout_value_ms": 0, + + # Limits the rate at which IPAs are called. The algorithms will + # be skipped until this many microseconds have elapsed since + # the last call. The default value represents a 30fps limit. + # + # "controller_min_frame_duration_us": 33333.333, } }