From patchwork Wed Jan 7 09:37:36 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Plowman X-Patchwork-Id: 25678 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 34350BE08B for ; Wed, 7 Jan 2026 09:38:29 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 4202C61FC1; Wed, 7 Jan 2026 10:38:28 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="VeAtPhGG"; dkim-atps=neutral Received: from mail-wm1-x333.google.com (mail-wm1-x333.google.com [IPv6:2a00:1450:4864:20::333]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id C72FA61F61 for ; Wed, 7 Jan 2026 10:38:25 +0100 (CET) Received: by mail-wm1-x333.google.com with SMTP id 5b1f17b1804b1-477a219dbcaso15250235e9.3 for ; Wed, 07 Jan 2026 01:38:25 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; t=1767778705; x=1768383505; darn=lists.libcamera.org; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:from:to:cc:subject:date:message-id:reply-to; bh=5ynmxmxJ7YUm/yGx64w2BfC9qQ0Z4qqdhAUa19x8DEU=; b=VeAtPhGGjDn07BMIu5S/CdaCYlCm0PW/zTp8OZsmqTq0SqrlP4VTAk5eADGY5AzmVE zui7S+4qdFuuS8FZKML69yJ2ytcjdRzj7h6l8m3rWek9ORS/s4otK1U7Eax1S21KH4vE KWVbMEHlJoyGBVyNKH41+JqqQ/o0cs5HSBRCFdrEet5zuUA/vKi3OO14vA+7/kuF5aNA DwpVVprtoWVJgoWMGAYqWml5yYe5rXKYBGXbLtJ746qEScU8bLdY5LQHYNJM3a4rmC37 WVLRmAc6+DdvDW4Vo/LjV5bvzJfvK9CwHEt/0ce2ZLY97+nEhNNXaQhXENUrw3yzxlVK y7zg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1767778705; x=1768383505; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:x-gm-gg:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=5ynmxmxJ7YUm/yGx64w2BfC9qQ0Z4qqdhAUa19x8DEU=; b=kJBoXO7D9xTpZmZIJ7HkkdUAL3Ivsh5xiFZYvl/rNJl5MuMOphdvUyyvYWxPdodWXy 5tg89217HyfC5A1/19yphUC6ILPjnN8nPJ0KyqRWhYAED0GFLwjdLudSooXotE9UqY15 TX5iM/KVKHvjkQ2Cw6EyAKVZtPd3Fvio2/5/1aRXkGFL9qVKwGZkFZun763mBzj9CG+E 4ZxzWB9PGwCHoFzttMkrmFCe9c0TFQJ7x4dxsAgMlrdyPB715q6vMYBXJFIC+GWTSp5p PUgvChV0Iv84wog/sBYDSLtDg/4hFu7yIiGdR32rPWaDa9jwiKCr6+TS6YzqVeJh6c0O HWpw== X-Gm-Message-State: AOJu0YzvfsLxgSzlZ/kTbvOtPtmhOvWxgWF2VY1GxezoL3MxeV56tKKg WWPVs69ZsB4/YSsruC/gOhsMGjblXCy9GSz+GIDkL11LQkEGTHqUr8iOm6DKxIRbRiWbVi956rh 561xGjJE= X-Gm-Gg: AY/fxX5n1ZLujEKHjJoiSke6ltmgaDEyux1zmaM/xijyYBhxVXi3lBOobH4hvdvLTFN pI7ymEsyo+yrWNnbapTlhA+9pjEPqs5rSxxqylYK5qGFZmU8KsSc8HsAQnqWshWFnWXwRsZqAqh mHNdQ6vuQ9BwSAogIwpRFZHS3oNRhJOdForfLK9fNUSFTHDUoaddPqsTQBAVVFKkEpyyUcp+Xy9 Q2pZ+QiG//NcRjyX9QGgKUMDgC4xNbFaqWDDRsU8ZCmKHNdwv9EKkqTZS+fy7D3cqGwldUeH9x3 sWWq+bsxMvUpOLElLJ43dDAuWTsHjRiRiakO1yb8RluLwXJaMxb/4ws3geUkSv7qnUEOBx02R82 VbJ5f2d/7JzbevVHDGCX8i1S67R/Y7PAiFdorL4yTLv2k3nBFwJ3Yc3yQrLbYA3piaSE9TdA+48 urxpD62qibjJwuml/i5NtiWJ+8shOU2LmsWX78nHTDmQJ5Q3zDzaDaRvkXvavGVe5lKYbcDWvLG fopyGZl+aNDxE6G4dLP63uDLTguDw== X-Google-Smtp-Source: AGHT+IExf18Z4bW6i8sWXM3C7vWDRka4CLkIHXjr9zOf0h6ujOSWCWJ7CVLOdJ0cu6oqtr1XMo+iZA== X-Received: by 2002:a05:600c:470c:b0:477:557b:691d with SMTP id 5b1f17b1804b1-47d84b39d1cmr17666925e9.25.1767778704686; Wed, 07 Jan 2026 01:38:24 -0800 (PST) Received: from davidp-pi5.pitowers.org ([2a00:1098:3142:1f:88ea:c658:5b20:5e46]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-47d8717d78fsm7445015e9.9.2026.01.07.01.38.23 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 07 Jan 2026 01:38:24 -0800 (PST) From: David Plowman To: libcamera-devel@lists.libcamera.org Cc: David Plowman Subject: [PATCH] libcamera: rpi: Make the controller min frame duration configurable Date: Wed, 7 Jan 2026 09:37:36 +0000 Message-ID: <20260107093821.14600-1-david.plowman@raspberrypi.com> X-Mailer: git-send-email 2.47.3 MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" The controller min frame duration is used to rate limit how often we run IPAs. Historically this has been set to 33333us, meaning that the algorithms effectively skip frames when the camera is running faster than 30fps. This patch adds a small amount of plumbing that allows this value to be set in the Raspberry Pi configuration file. Some applications or platforms (such as Pi 5) are easily capable of running these more often, should there be a need to do so. Signed-off-by: David Plowman --- include/libcamera/ipa/raspberrypi.mojom | 1 + src/ipa/rpi/common/ipa_base.cpp | 11 ++++++++- src/ipa/rpi/common/ipa_base.h | 2 ++ .../pipeline/rpi/common/pipeline_base.cpp | 24 ++++++++++++++----- .../pipeline/rpi/common/pipeline_base.h | 5 ++++ .../pipeline/rpi/pisp/data/example.yaml | 6 +++++ .../pipeline/rpi/vc4/data/example.yaml | 6 +++++ 7 files changed, 48 insertions(+), 7 deletions(-) diff --git a/include/libcamera/ipa/raspberrypi.mojom b/include/libcamera/ipa/raspberrypi.mojom index 12b083e9..1b7e0358 100644 --- a/include/libcamera/ipa/raspberrypi.mojom +++ b/include/libcamera/ipa/raspberrypi.mojom @@ -18,6 +18,7 @@ struct SensorConfig { struct InitParams { bool lensPresent; libcamera.IPACameraSensorInfo sensorInfo; + float controllerMinFrameDurationUs; /* PISP specific */ libcamera.SharedFD fe; libcamera.SharedFD be; diff --git a/src/ipa/rpi/common/ipa_base.cpp b/src/ipa/rpi/common/ipa_base.cpp index 14aba450..2fd101da 100644 --- a/src/ipa/rpi/common/ipa_base.cpp +++ b/src/ipa/rpi/common/ipa_base.cpp @@ -184,6 +184,15 @@ int32_t IpaBase::init(const IPASettings &settings, const InitParams ¶ms, Ini result->controlInfo = ControlInfoMap(std::move(ctrlMap), controls::controls); + /* + * This determines the minimum allowable inter-frame duration to run the + * controller algorithms. If the pipeline handler provider frames at a + * rate higher than this, we rate-limit the controller Prepare() and + * Process() calls to lower than or equal to this rate. + */ + double dur_us = params.controllerMinFrameDurationUs; + controllerMinFrameDuration_ = std::chrono::duration(dur_us); + return platformInit(params, result); } @@ -465,7 +474,7 @@ void IpaBase::prepareIsp(const PrepareParams ¶ms) /* Allow a 10% margin on the comparison below. */ Duration delta = (frameTimestamp - lastRunTimestamp_) * 1.0ns; if (lastRunTimestamp_ && frameCount_ > invalidCount_ && - delta < controllerMinFrameDuration * 0.9 && !hdrChange) { + delta < controllerMinFrameDuration_ * 0.9 && !hdrChange) { /* * Ensure we merge the previous frame's metadata with the current * frame. This will not overwrite exposure/gain values for the diff --git a/src/ipa/rpi/common/ipa_base.h b/src/ipa/rpi/common/ipa_base.h index 5348f2ea..90f018b2 100644 --- a/src/ipa/rpi/common/ipa_base.h +++ b/src/ipa/rpi/common/ipa_base.h @@ -142,6 +142,8 @@ private: } flickerState_; bool awbEnabled_; + + utils::Duration controllerMinFrameDuration_; }; } /* namespace ipa::RPi */ diff --git a/src/libcamera/pipeline/rpi/common/pipeline_base.cpp b/src/libcamera/pipeline/rpi/common/pipeline_base.cpp index fb8e466f..b7655d8d 100644 --- a/src/libcamera/pipeline/rpi/common/pipeline_base.cpp +++ b/src/libcamera/pipeline/rpi/common/pipeline_base.cpp @@ -33,6 +33,12 @@ LOG_DEFINE_CATEGORY(RPI) using StreamFlag = RPi::Stream::StreamFlag; +/* + * The IPA's algorithms will not be called more often than this many + * microseconds. The default corresponds to 30fps. + */ +constexpr float defaultControllerMinimumFrameDurationUs = 1000000.0 / 30.0; + namespace { constexpr unsigned int defaultRawBitDepth = 12; @@ -800,6 +806,12 @@ int PipelineHandlerBase::registerCamera(std::unique_ptr &camera if (!data->sensor_) return -EINVAL; + ret = data->loadPipelineConfiguration(); + if (ret) { + LOG(RPI, Error) << "Unable to load pipeline configuration"; + return ret; + } + /* Populate the map of sensor supported formats and sizes. */ for (auto const mbusCode : data->sensor_->mbusCodes()) data->sensorFormats_.emplace(mbusCode, @@ -859,12 +871,6 @@ int PipelineHandlerBase::registerCamera(std::unique_ptr &camera if (ret) return ret; - ret = data->loadPipelineConfiguration(); - if (ret) { - LOG(RPI, Error) << "Unable to load pipeline configuration"; - return ret; - } - /* Setup the general IPA signal handlers. */ data->frontendDevice()->dequeueTimeout.connect(data, &RPi::CameraData::cameraTimeout); data->frontendDevice()->frameStart.connect(data, &RPi::CameraData::frameStarted); @@ -1096,6 +1102,7 @@ int CameraData::loadPipelineConfiguration() { config_ = { .cameraTimeoutValue = 0, + .controllerMinFrameDurationUs = defaultControllerMinimumFrameDurationUs, }; /* Initial configuration of the platform, in case no config file is present */ @@ -1145,6 +1152,9 @@ int CameraData::loadPipelineConfiguration() frontendDevice()->setDequeueTimeout(config_.cameraTimeoutValue * 1ms); } + config_.controllerMinFrameDurationUs = + phConfig["controller_min_frame_duration_us"].get(config_.controllerMinFrameDurationUs); + return platformPipelineConfigure(root); } @@ -1173,6 +1183,8 @@ int CameraData::loadIPA(ipa::RPi::InitResult *result) } params.lensPresent = !!sensor_->focusLens(); + params.controllerMinFrameDurationUs = config_.controllerMinFrameDurationUs; + ret = platformInitIpa(params); if (ret) return ret; diff --git a/src/libcamera/pipeline/rpi/common/pipeline_base.h b/src/libcamera/pipeline/rpi/common/pipeline_base.h index 6257a934..597eb587 100644 --- a/src/libcamera/pipeline/rpi/common/pipeline_base.h +++ b/src/libcamera/pipeline/rpi/common/pipeline_base.h @@ -169,6 +169,11 @@ public: * on frame durations. */ unsigned int cameraTimeoutValue; + /* + * The minimum frame duration between the IPA's calls to the + * algorithms themselves (in microseconds). + */ + float controllerMinFrameDurationUs; }; Config config_; diff --git a/src/libcamera/pipeline/rpi/pisp/data/example.yaml b/src/libcamera/pipeline/rpi/pisp/data/example.yaml index baf03be7..c5edbba0 100644 --- a/src/libcamera/pipeline/rpi/pisp/data/example.yaml +++ b/src/libcamera/pipeline/rpi/pisp/data/example.yaml @@ -36,5 +36,11 @@ # framebuffers required for its operation. # # "disable_hdr": false, + + # Limits the rate at which IPAs are called. The algorithms will + # be skipped until this many microseconds have elapsed since + # the last call. The default value represents a 30fps limit. + # + # "controller_min_frame_duration_us": 33333.333, } } diff --git a/src/libcamera/pipeline/rpi/vc4/data/example.yaml b/src/libcamera/pipeline/rpi/vc4/data/example.yaml index 27e54348..2ee2b864 100644 --- a/src/libcamera/pipeline/rpi/vc4/data/example.yaml +++ b/src/libcamera/pipeline/rpi/vc4/data/example.yaml @@ -37,5 +37,11 @@ # timeout value. # # "camera_timeout_value_ms": 0, + + # Limits the rate at which IPAs are called. The algorithms will + # be skipped until this many microseconds have elapsed since + # the last call. The default value represents a 30fps limit. + # + # "controller_min_frame_duration_us": 33333.333, } }