From patchwork Tue Oct 20 13:24:09 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Robert Mader X-Patchwork-Id: 28533 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id BBEC9C324C for ; Tue, 20 Oct 2026 13:24:28 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 0717E68EFB; Tue, 20 Oct 2026 15:24:28 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=collabora.com header.i=robert.mader@collabora.com header.b="GSybJbdK"; dkim-atps=neutral Received: from sender6-op-o11.zoho.com (sender6-op-o11.zoho.com [165.173.180.11]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 44E5568E58 for ; Tue, 20 Oct 2026 15:24:25 +0200 (CEST) ARC-Seal: i=1; a=rsa-sha256; t=1792502661; cv=none; d=zohomail.com; s=zohoarc; b=JdHTZ9LFDcjRh3rKCjWQ2zZxVOodc5sN3ovhvqWood13k9uEQAWs10uF8AbBYLNYGBfM9ED1ZAiFt2xS2rKTNelnYCSpoDI4F3CCht547O+oup9cW6WDy3QjWCec9FXIvhJ/n4JfDj8Y5388gpILW8V7mVrMQUBVNj08scuumbY= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1792502661; h=Content-Type:Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:MIME-Version:Message-ID:Subject:Subject:To:To:Message-Id:Reply-To; bh=3jmJGMjwmF1Cl3ekdOnRzUkyYntKAZpoV9LzqocpqJc=; b=e67VWQvCo+HdsBw/idE8jNXJDklPvm4maMGcWjJtU2vW4YbPR4DH9sSqitbwMwjJoyKXYVBpKciSpq+iquEldMeysdpoGxG39KQNyotz4XmCANfF+ImUeg+bRCrHlJNdfXbFm/Kc5bLmtRCXRGgkL6+3H3jbIl+wFRbGtu2v1pw= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass header.i=collabora.com; spf=pass smtp.mailfrom=robert.mader@collabora.com; dmarc=pass header.from= DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; t=1792502661; s=zohomail; d=collabora.com; i=robert.mader@collabora.com; h=From:From:To:To:Cc:Cc:Subject:Subject:Date:Date:Message-ID:MIME-Version:Content-Type:Content-Transfer-Encoding:Message-Id:Reply-To; bh=3jmJGMjwmF1Cl3ekdOnRzUkyYntKAZpoV9LzqocpqJc=; b=GSybJbdKNXIQ5RSXUL22w5q0RBTrdGDpvyNM47uW9mGoEneSSRlmKz+HJz80/m9I WqqDq7X8Pz7vX3m2i9Bxc3Ie+9MdYrIpdd+Z7CfhkWMPQS8yipQZWYwxejmHY++nWTX ulSiOsi4pMxa9T2yolRP9nSTljlSogH7G28xNN8c= Received: by smtp.zohomail.com with SMTPS id 1792502658825434.42158138909883; Tue, 20 Oct 2026 06:24:18 -0700 (PDT) From: Robert Mader To: libcamera-devel@lists.libcamera.org Cc: Robert Mader , Kieran Bingham Subject: [PATCH v7] virtual: Implement frame rate handling Date: Tue, 20 Oct 2026 15:24:09 +0200 Message-ID: <20261020132409.37650-1-robert.mader@collabora.com> X-Mailer: git-send-email 2.56.0 MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Throttle requests to the minimal frame duration in order to honor the configured frame rates. Signed-off-by: Robert Mader Reviewed-by: Kieran Bingham --- Changes in v7: - Address feedback from Laurent - Replace lastTimestamp_ with targetTimestamp_ and simplify calculation Changes in v6: - Use std::clamp (Kieran) and minor code reshuffle in start() - Use "frame rate" consistently - Added R-B from Kieran (assuming the changes are ok) Changes in v5: - Also clamp frame duration limit to highest one allowed by the config so clients can't set a unsupported frame rate. - Added comments for that calculations using frame rate logic, as that is what the config files use. - Successfully tested the patch against the following Pipewire MR: https://gitlab.freedesktop.org/pipewire/pipewire/-/merge_requests/3059 Changes in v4: - Calculate minFrameDuration_ once in start() using FDLs from camera and, if present, from client provided controls. The later is not yet tested. - Assert that the camera has FDLs set by the config parser. - Set FrameDuration in request metadata. Changes in v3: - Removed a leftover debug change Changes in v2: - Remove debug log in order to avoid CI issues. - Simplify code in queueRequestDevice(), hopefully making it easier to follow. Changes in v1 (from RFC version): - Use std::this_thread::sleep_for() instead usleep() (Barnabás) - Sleep only once per request, after generating all frames - Remove control handling in start() (defer that for later) - Reset state in stop() instead of start() --- src/libcamera/pipeline/virtual/virtual.cpp | 58 ++++++++++++++++++---- src/libcamera/pipeline/virtual/virtual.h | 5 +- 2 files changed, 53 insertions(+), 10 deletions(-) diff --git a/src/libcamera/pipeline/virtual/virtual.cpp b/src/libcamera/pipeline/virtual/virtual.cpp index f35af97b103d..13a96e6a9412 100644 --- a/src/libcamera/pipeline/virtual/virtual.cpp +++ b/src/libcamera/pipeline/virtual/virtual.cpp @@ -135,8 +135,14 @@ VirtualCameraData::VirtualCameraData(PipelineHandler *pipe, moveToThread(this); } -void VirtualCameraData::processRequest(Request *request) +void VirtualCameraData::processRequest(Request *request, + uint64_t targetTimestamp) { + const auto targetTime = std::chrono::steady_clock::time_point{ + std::chrono::nanoseconds(targetTimestamp) + }; + std::this_thread::sleep_until(targetTime); + for (const auto &[stream, buffer] : request->buffers()) { bool found = false; /* map buffer and fill test patterns */ @@ -146,7 +152,7 @@ void VirtualCameraData::processRequest(Request *request) fmd.status = FrameMetadata::Status::FrameSuccess; fmd.sequence = streamConfig.seq++; - fmd.timestamp = currentTimestamp(); + fmd.timestamp = targetTimestamp; std::span planes = buffer->planes(); for (const auto [i, p] : utils::enumerate(planes)) @@ -348,11 +354,34 @@ int PipelineHandlerVirtual::exportFrameBuffers([[maybe_unused]] Camera *camera, return dmaBufAllocator_.exportBuffers(config.bufferCount, planeSizes, buffers); } -int PipelineHandlerVirtual::start([[maybe_unused]] Camera *camera, - [[maybe_unused]] const ControlList *controls) +int PipelineHandlerVirtual::start(Camera *camera, const ControlList *controls) { VirtualCameraData *data = cameraData(camera); + auto cameraFrameDurationLimits = + camera->controls().find(&controls::FrameDurationLimits); + ASSERT(cameraFrameDurationLimits != camera->controls().end()); + + /* By default target the highest frame rate allowed by the config. */ + data->minFrameDuration_ = + cameraFrameDurationLimits->second.min().get(); + + if (controls) { + if (auto clientFrameDurationLimits = + controls->get(controls::FrameDurationLimits)) { + /* + * Target the highest frame rate allowed by the client, + * clamped to the values allowed by the config. + */ + data->minFrameDuration_ = std::clamp( + clientFrameDurationLimits->front(), + data->minFrameDuration_, + cameraFrameDurationLimits->second.max().get()); + } + } + + data->targetTimestamp_ = 0; + for (auto &s : data->streamConfigs_) s.seq = 0; @@ -379,15 +408,26 @@ void PipelineHandlerVirtual::stopDevice(Camera *camera) cancelRequest(data->queuedRequests_.front()); } -int PipelineHandlerVirtual::queueRequestDevice([[maybe_unused]] Camera *camera, - Request *request) +int PipelineHandlerVirtual::queueRequestDevice(Camera *camera, Request *request) { VirtualCameraData *data = cameraData(camera); - const auto timestamp = currentTimestamp(); - request->_d()->metadata().set(controls::SensorTimestamp, timestamp); + /* + * TODO: Skip frames if we can't keep up with the maximum frame + * duration. + */ + data->targetTimestamp_ = + std::max(currentTimestamp(), + data->targetTimestamp_ + data->minFrameDuration_ * 1000); + + request->_d()->metadata().set(controls::SensorTimestamp, + data->targetTimestamp_); + request->_d()->metadata().set(controls::FrameDuration, + data->minFrameDuration_); + data->invokeMethod(&VirtualCameraData::processRequest, - ConnectionTypeQueued, request); + ConnectionTypeQueued, request, + data->targetTimestamp_); return 0; } diff --git a/src/libcamera/pipeline/virtual/virtual.h b/src/libcamera/pipeline/virtual/virtual.h index 215e56fa3b9d..40e9095c0c4f 100644 --- a/src/libcamera/pipeline/virtual/virtual.h +++ b/src/libcamera/pipeline/virtual/virtual.h @@ -60,12 +60,15 @@ public: ~VirtualCameraData() = default; - void processRequest(Request *request); + void processRequest(Request *request, uint64_t targetTimestamp); Configuration config_; std::vector streamConfigs_; Signal bufferCompleted; + + int64_t minFrameDuration_; + uint64_t targetTimestamp_; }; } /* namespace libcamera */