From patchwork Wed Jan 7 18:14:26 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Plowman X-Patchwork-Id: 25681 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 97D31BDCBF for ; Wed, 7 Jan 2026 18:14:51 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 01BE161FA0; Wed, 7 Jan 2026 19:14:51 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="PZxYYzue"; dkim-atps=neutral Received: from mail-wm1-x32f.google.com (mail-wm1-x32f.google.com [IPv6:2a00:1450:4864:20::32f]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 9E80D61F84 for ; Wed, 7 Jan 2026 19:14:49 +0100 (CET) Received: by mail-wm1-x32f.google.com with SMTP id 5b1f17b1804b1-47a8195e515so17241655e9.0 for ; Wed, 07 Jan 2026 10:14:49 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; t=1767809689; x=1768414489; darn=lists.libcamera.org; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:from:to:cc:subject:date:message-id:reply-to; bh=Dbzz5Vq4ie1o0ZsZ8fdxcCU8jIGu2XxUyMpV4pNGyfQ=; b=PZxYYzue4z5ajGAC49NLLf5xI1WmlIhUO0gm9pPs/QcM338WR5Ll8CMIUJ5y+zTw/a CiSZoKMfY6OH02AgGHnyZiFeBnoqNTe6XJ5XrelvmHz08hLIZuL14jwoNH7n7YxEddG3 qSprt9fv6hq8zlY4UYV1ZhNl6i9CXfRgcYlEECHb1m23Y3eVHu+T8Ah1mUa4ih4fAKwH SwKhnvNjooELqYW2TfDxP/TK84AxuKVF2YIxbBSDPM8TKiG7p6liJhsWvbHrS5yIYxyB Eo0Mq37XuS1q2CG/G6mXRf8umGMUGV3kgPFAf2FMorqDIUteMPIiRO4IeIwuYHhII8cJ BIUw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1767809689; x=1768414489; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:x-gm-gg:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=Dbzz5Vq4ie1o0ZsZ8fdxcCU8jIGu2XxUyMpV4pNGyfQ=; b=GU/91kCnx9Bdwb4kARI5RyO667X2d71rocvpgYMZ8B/RZyTp9Eh8oA8q5liq2RtVBE WEPkPwXgFpCQJGWG19GMV2PFl64cTkjPGmtJ17FMo2lQckF0tSYoc6CCjlquxu9r/GwA ZK0kjOSRnqcVV2VVN5TpLZM5bx4qvDepvncaWYXe2YUjbc5MUw27TISHkobNpohSm6Cb RJl39nYCr6d3T+b75fEqwciJ6cUVIJ+j9ZwjUyNbGzyvBiXvwOAvPPKXxDB0D1c5Sl1T mMBJ7eG5vqnq17SOzKjts7YUzIq0mS+PDX9p/ch6aHCDoEumtHKSbhLCV17Nwl1/paeS Ucpw== X-Gm-Message-State: AOJu0YxOk3MI9p7RRfbTj2t4aKwtJySPkXZCcQ8FSo7jxkLFERFreXYQ nx0QbdsWthF57VhMvKia4z73T6nLqfYUxXvLogZlplat1k8bJfMnaWV29hGhrIoen/TUUfh4PrU cnV8uytc= X-Gm-Gg: AY/fxX5y49vqTZzgTOoRqSv2vOM0v2O3dPVaNQ1XYa3lfd4wK6+/TpKx3Ytl+rvTk0x l52RY27QfTCWy6ySa1k7o0QQXndo9krOADzp7JyWdjL+Uu1fkU9FXA9l7UXDxrG5LGAOBLZ+ZUq flWq1SQT+N7ozYVOT4sueS7Xi5+WWAclqCVrfTJsCo8Wb+hK0QT/GJA0pd8p1/Ln0dc9zWRtxnj yVax1mIbAljt/gqcnPH9AGO1qT4V/uzUGf+C4B7y7MWK0bs7sIlgEDL0lojNWBEouv1mjihUEmz 4DLaHK3ch4FwjrYdPs3LqRcPueiQnhK6ecCBNRIJBXIL3inLuKl2XJcRWQOco8Pm4xPR8IyqOqT QKBxLlVFFpIsnn+J4h4nQgndYQ5nVdCEGipiLe4P0jCQZdm2mi3OMGfY6PuuLSrCjrajCUuTHRC UwFMvZmd+Vm98OBfYx+FxFZAlp461P8kP+Ke841jalPZnKJyXqvL6qv40OXFlrqnIQiHpFLeqXt g8+leKiM9dWy4wHlso+x2w/MrdbFLQB4EhR7PMw X-Google-Smtp-Source: AGHT+IFHp4Htm5WVJiuiaZo9YcPgwjwzoYnnY/U1mvc7Ywsl0zz5gGOIWoIuSg8yHfIXrd3VKPU+sA== X-Received: by 2002:a05:600c:4e8a:b0:477:2f7c:314f with SMTP id 5b1f17b1804b1-47d84b17dc7mr47223265e9.10.1767809688582; Wed, 07 Jan 2026 10:14:48 -0800 (PST) Received: from davidp-pi5.pitowers.org ([2a00:1098:3142:1f:88ea:c658:5b20:5e46]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-47d865e3dfasm18526165e9.2.2026.01.07.10.14.47 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 07 Jan 2026 10:14:48 -0800 (PST) From: David Plowman To: libcamera-devel@lists.libcamera.org Cc: David Plowman Subject: [PATCH v3] libcamera: rpi: Make the controller min frame duration configurable Date: Wed, 7 Jan 2026 18:14:26 +0000 Message-ID: <20260107181445.18005-1-david.plowman@raspberrypi.com> X-Mailer: git-send-email 2.47.3 MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" The controller min frame duration is used to rate limit how often we run IPAs. Historically this has been set to 33333us, meaning that the algorithms effectively skip frames when the camera is running faster than 30fps. This patch adds a small amount of plumbing that allows this value to be set in the Raspberry Pi configuration file. Some applications or platforms (such as Pi 5) are easily capable of running these more often, should there be a need to do so. Signed-off-by: David Plowman --- include/libcamera/ipa/raspberrypi.mojom | 1 + src/ipa/rpi/common/ipa_base.cpp | 18 +++++++------- src/ipa/rpi/common/ipa_base.h | 2 ++ .../pipeline/rpi/common/pipeline_base.cpp | 24 ++++++++++++++----- .../pipeline/rpi/common/pipeline_base.h | 5 ++++ .../pipeline/rpi/pisp/data/example.yaml | 6 +++++ .../pipeline/rpi/vc4/data/example.yaml | 6 +++++ 7 files changed, 47 insertions(+), 15 deletions(-) diff --git a/include/libcamera/ipa/raspberrypi.mojom b/include/libcamera/ipa/raspberrypi.mojom index 12b083e9..1b7e0358 100644 --- a/include/libcamera/ipa/raspberrypi.mojom +++ b/include/libcamera/ipa/raspberrypi.mojom @@ -18,6 +18,7 @@ struct SensorConfig { struct InitParams { bool lensPresent; libcamera.IPACameraSensorInfo sensorInfo; + float controllerMinFrameDurationUs; /* PISP specific */ libcamera.SharedFD fe; libcamera.SharedFD be; diff --git a/src/ipa/rpi/common/ipa_base.cpp b/src/ipa/rpi/common/ipa_base.cpp index 14aba450..c0f69b79 100644 --- a/src/ipa/rpi/common/ipa_base.cpp +++ b/src/ipa/rpi/common/ipa_base.cpp @@ -46,14 +46,6 @@ constexpr Duration defaultExposureTime = 20.0ms; constexpr Duration defaultMinFrameDuration = 1.0s / 30.0; constexpr Duration defaultMaxFrameDuration = 250.0s; -/* - * Determine the minimum allowable inter-frame duration to run the controller - * algorithms. If the pipeline handler provider frames at a rate higher than this, - * we rate-limit the controller Prepare() and Process() calls to lower than or - * equal to this rate. - */ -constexpr Duration controllerMinFrameDuration = 1.0s / 30.0; - /* List of controls handled by the Raspberry Pi IPA */ const ControlInfoMap::Map ipaControls{ /* \todo Move this to the Camera class */ @@ -184,6 +176,14 @@ int32_t IpaBase::init(const IPASettings &settings, const InitParams ¶ms, Ini result->controlInfo = ControlInfoMap(std::move(ctrlMap), controls::controls); + /* + * This determines the minimum allowable inter-frame duration to run the + * controller algorithms. If the pipeline handler provider frames at a + * rate higher than this, we rate-limit the controller Prepare() and + * Process() calls to lower than or equal to this rate. + */ + controllerMinFrameDuration_ = params.controllerMinFrameDurationUs * 1us; + return platformInit(params, result); } @@ -465,7 +465,7 @@ void IpaBase::prepareIsp(const PrepareParams ¶ms) /* Allow a 10% margin on the comparison below. */ Duration delta = (frameTimestamp - lastRunTimestamp_) * 1.0ns; if (lastRunTimestamp_ && frameCount_ > invalidCount_ && - delta < controllerMinFrameDuration * 0.9 && !hdrChange) { + delta < controllerMinFrameDuration_ * 0.9 && !hdrChange) { /* * Ensure we merge the previous frame's metadata with the current * frame. This will not overwrite exposure/gain values for the diff --git a/src/ipa/rpi/common/ipa_base.h b/src/ipa/rpi/common/ipa_base.h index 5348f2ea..90f018b2 100644 --- a/src/ipa/rpi/common/ipa_base.h +++ b/src/ipa/rpi/common/ipa_base.h @@ -142,6 +142,8 @@ private: } flickerState_; bool awbEnabled_; + + utils::Duration controllerMinFrameDuration_; }; } /* namespace ipa::RPi */ diff --git a/src/libcamera/pipeline/rpi/common/pipeline_base.cpp b/src/libcamera/pipeline/rpi/common/pipeline_base.cpp index fb8e466f..b7655d8d 100644 --- a/src/libcamera/pipeline/rpi/common/pipeline_base.cpp +++ b/src/libcamera/pipeline/rpi/common/pipeline_base.cpp @@ -33,6 +33,12 @@ LOG_DEFINE_CATEGORY(RPI) using StreamFlag = RPi::Stream::StreamFlag; +/* + * The IPA's algorithms will not be called more often than this many + * microseconds. The default corresponds to 30fps. + */ +constexpr float defaultControllerMinimumFrameDurationUs = 1000000.0 / 30.0; + namespace { constexpr unsigned int defaultRawBitDepth = 12; @@ -800,6 +806,12 @@ int PipelineHandlerBase::registerCamera(std::unique_ptr &camera if (!data->sensor_) return -EINVAL; + ret = data->loadPipelineConfiguration(); + if (ret) { + LOG(RPI, Error) << "Unable to load pipeline configuration"; + return ret; + } + /* Populate the map of sensor supported formats and sizes. */ for (auto const mbusCode : data->sensor_->mbusCodes()) data->sensorFormats_.emplace(mbusCode, @@ -859,12 +871,6 @@ int PipelineHandlerBase::registerCamera(std::unique_ptr &camera if (ret) return ret; - ret = data->loadPipelineConfiguration(); - if (ret) { - LOG(RPI, Error) << "Unable to load pipeline configuration"; - return ret; - } - /* Setup the general IPA signal handlers. */ data->frontendDevice()->dequeueTimeout.connect(data, &RPi::CameraData::cameraTimeout); data->frontendDevice()->frameStart.connect(data, &RPi::CameraData::frameStarted); @@ -1096,6 +1102,7 @@ int CameraData::loadPipelineConfiguration() { config_ = { .cameraTimeoutValue = 0, + .controllerMinFrameDurationUs = defaultControllerMinimumFrameDurationUs, }; /* Initial configuration of the platform, in case no config file is present */ @@ -1145,6 +1152,9 @@ int CameraData::loadPipelineConfiguration() frontendDevice()->setDequeueTimeout(config_.cameraTimeoutValue * 1ms); } + config_.controllerMinFrameDurationUs = + phConfig["controller_min_frame_duration_us"].get(config_.controllerMinFrameDurationUs); + return platformPipelineConfigure(root); } @@ -1173,6 +1183,8 @@ int CameraData::loadIPA(ipa::RPi::InitResult *result) } params.lensPresent = !!sensor_->focusLens(); + params.controllerMinFrameDurationUs = config_.controllerMinFrameDurationUs; + ret = platformInitIpa(params); if (ret) return ret; diff --git a/src/libcamera/pipeline/rpi/common/pipeline_base.h b/src/libcamera/pipeline/rpi/common/pipeline_base.h index 6257a934..597eb587 100644 --- a/src/libcamera/pipeline/rpi/common/pipeline_base.h +++ b/src/libcamera/pipeline/rpi/common/pipeline_base.h @@ -169,6 +169,11 @@ public: * on frame durations. */ unsigned int cameraTimeoutValue; + /* + * The minimum frame duration between the IPA's calls to the + * algorithms themselves (in microseconds). + */ + float controllerMinFrameDurationUs; }; Config config_; diff --git a/src/libcamera/pipeline/rpi/pisp/data/example.yaml b/src/libcamera/pipeline/rpi/pisp/data/example.yaml index baf03be7..c5edbba0 100644 --- a/src/libcamera/pipeline/rpi/pisp/data/example.yaml +++ b/src/libcamera/pipeline/rpi/pisp/data/example.yaml @@ -36,5 +36,11 @@ # framebuffers required for its operation. # # "disable_hdr": false, + + # Limits the rate at which IPAs are called. The algorithms will + # be skipped until this many microseconds have elapsed since + # the last call. The default value represents a 30fps limit. + # + # "controller_min_frame_duration_us": 33333.333, } } diff --git a/src/libcamera/pipeline/rpi/vc4/data/example.yaml b/src/libcamera/pipeline/rpi/vc4/data/example.yaml index 27e54348..2ee2b864 100644 --- a/src/libcamera/pipeline/rpi/vc4/data/example.yaml +++ b/src/libcamera/pipeline/rpi/vc4/data/example.yaml @@ -37,5 +37,11 @@ # timeout value. # # "camera_timeout_value_ms": 0, + + # Limits the rate at which IPAs are called. The algorithms will + # be skipped until this many microseconds have elapsed since + # the last call. The default value represents a 30fps limit. + # + # "controller_min_frame_duration_us": 33333.333, } }