From patchwork Thu Oct 15 10:36:04 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Robert Mader X-Patchwork-Id: 28455 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 3EE7EC32F4 for ; Thu, 15 Oct 2026 10:36:20 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 562F568E01; Thu, 15 Oct 2026 12:36:19 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=collabora.com header.i=robert.mader@collabora.com header.b="QWbRd4xI"; dkim-atps=neutral Received: from sender5-op-o11.zoho.com (sender5-op-o11.zoho.com [165.173.182.11]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 22A9268A73 for ; Thu, 15 Oct 2026 12:36:16 +0200 (CEST) ARC-Seal: i=1; a=rsa-sha256; t=1792060572; cv=none; d=zohomail.com; s=zohoarc; b=Vtw9Xdd3u8x8bQDQfNIRm8Ihbt4hw5ZwlaUnkuaSrGgfz/XUIXLN8/fwMfljcZSBAJQcPdtT6rSzqKXhMsHlTFjoiKJDEplHrDuArI4fuYAWkDWHnTyIruYzTGmtl9FMIrsnYxQG1ZXmwS5j/34UE32KVJUzahOZ2VJAsbSQodk= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1792060572; h=Content-Type:Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:MIME-Version:Message-ID:Subject:Subject:To:To:Message-Id:Reply-To; bh=6iXmiQG4l3KGeVYEny+mJUqbglbrmtPlYi7BLTBZoIQ=; b=WX5B0aFYCZ3+jfHNPW+88i3Isn1eMDylRvc2ZcX5QF/LowGrKaB2xNKu4ysvaG1YnM5nxNqNUg9/3FtK2Fx+z5k1dToAsWeeuX0OCBakQmJ/0BIlG4gsZMdRvPCz0dD/qmcZhgi8AP2oneJTR7EoyYpFX5sC/DZMH1BfZqULjyw= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass header.i=collabora.com; spf=pass smtp.mailfrom=robert.mader@collabora.com; dmarc=pass header.from= DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; t=1792060572; s=zohomail; d=collabora.com; i=robert.mader@collabora.com; h=From:From:To:To:Cc:Cc:Subject:Subject:Date:Date:Message-ID:MIME-Version:Content-Type:Content-Transfer-Encoding:Message-Id:Reply-To; bh=6iXmiQG4l3KGeVYEny+mJUqbglbrmtPlYi7BLTBZoIQ=; b=QWbRd4xIL7A2m0DoUDZhpWluo50hXzEu1/2ASgXqUulHq3GGpPpU5UUteOOZPQzQ tk9hJ0cZoxxEe1G7N6emAl+pSp+x9xNGj5LodjpyGfhOou/j34XCgBYrdjz1ksh6YWA 8MJKiV0yALhgKyNudW28hjgn3Adqe9aJrXiE39NM= Received: by smtp.zohomail.com with SMTPS id 1792060569472613.1948297180335; Thu, 15 Oct 2026 03:36:09 -0700 (PDT) From: Robert Mader To: libcamera-devel@lists.libcamera.org Cc: Robert Mader , Kieran Bingham Subject: [PATCH v6] virtual: Implement frame rate handling Date: Thu, 15 Oct 2026 12:36:04 +0200 Message-ID: <20261015103604.283600-1-robert.mader@collabora.com> X-Mailer: git-send-email 2.56.0 MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Throttle requests to the minimal frame duration in order to honor the configured frame rates. Signed-off-by: Robert Mader Reviewed-by: Kieran Bingham --- Changes in v6: - Use std::clamp (Kieran) and minor code reshuffle in start() - Use "frame rate" consistently - Added R-B from Kieran (assuming the changes are ok) Changes in v5: - Also clamp frame duration limit to highest one allowed by the config so clients can't set a unsupported frame rate. - Added comments for that calculations using frame rate logic, as that is what the config files use. - Successfully tested the patch against the following Pipewire MR: https://gitlab.freedesktop.org/pipewire/pipewire/-/merge_requests/3059 Changes in v4: - Calculate minFrameDuration_ once in start() using FDLs from camera and, if present, from client provided controls. The later is not yet tested. - Assert that the camera has FDLs set by the config parser. - Set FrameDuration in request metadata. Changes in v3: - Removed a leftover debug change Changes in v2: - Remove debug log in order to avoid CI issues. - Simplify code in queueRequestDevice(), hopefully making it easier to follow. Changes in v1 (from RFC version): - Use std::this_thread::sleep_for() instead usleep() (Barnabás) - Sleep only once per request, after generating all frames - Remove control handling in start() (defer that for later) - Reset state in stop() instead of start() --- src/libcamera/pipeline/virtual/virtual.cpp | 62 +++++++++++++++++++--- src/libcamera/pipeline/virtual/virtual.h | 3 ++ 2 files changed, 57 insertions(+), 8 deletions(-) diff --git a/src/libcamera/pipeline/virtual/virtual.cpp b/src/libcamera/pipeline/virtual/virtual.cpp index e25dd85a7391..994f01c1b224 100644 --- a/src/libcamera/pipeline/virtual/virtual.cpp +++ b/src/libcamera/pipeline/virtual/virtual.cpp @@ -130,6 +130,9 @@ VirtualCameraData::VirtualCameraData(PipelineHandler *pipe, void VirtualCameraData::processRequest(Request *request) { + int64_t sensorTimestamp = *request->metadata().get(controls::SensorTimestamp); + std::vector buffers; + for (const auto &[stream, buffer] : request->buffers()) { bool found = false; /* map buffer and fill test patterns */ @@ -139,7 +142,7 @@ void VirtualCameraData::processRequest(Request *request) fmd.status = FrameMetadata::Status::FrameSuccess; fmd.sequence = streamConfig.seq++; - fmd.timestamp = currentTimestamp(); + fmd.timestamp = sensorTimestamp; std::span planes = buffer->planes(); for (const auto [i, p] : utils::enumerate(planes)) @@ -151,12 +154,21 @@ void VirtualCameraData::processRequest(Request *request) stream->configuration().size, buffer)) fmd.status = FrameMetadata::Status::FrameError; - bufferCompleted.emit(buffer); + buffers.push_back(buffer); break; } } ASSERT(found); } + + const uint64_t now = currentTimestamp(); + if (now < static_cast(sensorTimestamp)) { + std::chrono::nanoseconds duration(sensorTimestamp - now); + std::this_thread::sleep_for(duration); + } + + for (auto buffer : buffers) + bufferCompleted.emit(buffer); } VirtualCameraConfiguration::VirtualCameraConfiguration(VirtualCameraData *data) @@ -323,11 +335,33 @@ int PipelineHandlerVirtual::exportFrameBuffers([[maybe_unused]] Camera *camera, return dmaBufAllocator_.exportBuffers(config.bufferCount, planeSizes, buffers); } -int PipelineHandlerVirtual::start([[maybe_unused]] Camera *camera, - [[maybe_unused]] const ControlList *controls) +int PipelineHandlerVirtual::start(Camera *camera, const ControlList *controls) { VirtualCameraData *data = cameraData(camera); + ASSERT(data->minFrameDuration_ == 0); + + auto cameraFrameDurationLimits = + camera->controls().find(&controls::FrameDurationLimits); + ASSERT(cameraFrameDurationLimits != camera->controls().end()); + + if (controls) { + if (auto clientFrameDurationLimits = + controls->get(controls::FrameDurationLimits)) { + /* Target the highest frame rate allowed by the client, + * clamped to the values allow by the config. */ + data->minFrameDuration_ = std::clamp( + clientFrameDurationLimits->front(), + cameraFrameDurationLimits->second.min().get(), + cameraFrameDurationLimits->second.max().get()); + } + } + + /* By default target the highest frame rate allowed by the config. */ + if (data->minFrameDuration_ == 0) + data->minFrameDuration_ = + cameraFrameDurationLimits->second.min().get(); + for (auto &s : data->streamConfigs_) s.seq = 0; @@ -349,18 +383,30 @@ void PipelineHandlerVirtual::stopDevice(Camera *camera) /* Process pending `bufferCompleted` signals. */ thread()->dispatchMessages(Message::Type::InvokeMessage, this); data->bufferCompleted.disconnect(this); + data->minFrameDuration_ = 0; + data->lastTimestamp_ = 0; while (!data->queuedRequests_.empty()) cancelRequest(data->queuedRequests_.front()); } -int PipelineHandlerVirtual::queueRequestDevice([[maybe_unused]] Camera *camera, - Request *request) +int PipelineHandlerVirtual::queueRequestDevice(Camera *camera, Request *request) { VirtualCameraData *data = cameraData(camera); - const auto timestamp = currentTimestamp(); - request->_d()->metadata().set(controls::SensorTimestamp, timestamp); + const uint64_t now = currentTimestamp(); + uint64_t targetTimestamp = + data->lastTimestamp_ + data->minFrameDuration_ * 1000; + + if (targetTimestamp < now) + targetTimestamp = now; + data->lastTimestamp_ = targetTimestamp; + + request->_d()->metadata().set(controls::SensorTimestamp, + targetTimestamp); + request->_d()->metadata().set(controls::FrameDuration, + data->minFrameDuration_); + data->invokeMethod(&VirtualCameraData::processRequest, ConnectionTypeQueued, request); diff --git a/src/libcamera/pipeline/virtual/virtual.h b/src/libcamera/pipeline/virtual/virtual.h index 215e56fa3b9d..477f5b15a88f 100644 --- a/src/libcamera/pipeline/virtual/virtual.h +++ b/src/libcamera/pipeline/virtual/virtual.h @@ -66,6 +66,9 @@ public: std::vector streamConfigs_; Signal bufferCompleted; + + int64_t minFrameDuration_ = 0; + uint64_t lastTimestamp_ = 0; }; } /* namespace libcamera */