From patchwork Thu Jun 19 10:05:56 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Naushir Patuck X-Patchwork-Id: 23605 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 02661C3240 for ; Thu, 19 Jun 2025 10:09:13 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 495D268DEB; Thu, 19 Jun 2025 12:09:10 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="QCAiWwYw"; dkim-atps=neutral Received: from mail-wm1-x332.google.com (mail-wm1-x332.google.com [IPv6:2a00:1450:4864:20::332]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 48A3368DEB for ; Thu, 19 Jun 2025 12:09:04 +0200 (CEST) Received: by mail-wm1-x332.google.com with SMTP id 5b1f17b1804b1-4519dd6523dso338225e9.1 for ; Thu, 19 Jun 2025 03:09:04 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; t=1750327743; x=1750932543; darn=lists.libcamera.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=UsJG0sUjwmdzpZDVOl09h+1erDYt2a8aPHUb1k2opsw=; b=QCAiWwYwu9D+PTr80zQCcp1mhpNPW+Dj1ZpbJ5XlxTK2+7T8RJlLaQCqZnUB9Jxqb5 JLjCyw19h65ZCfDP2qJ+mKrJOefknlN6Bf9EZu6x44Umc7w4RnX3wOrwEByFAXMDw5nL 4e0Q/74VQX2qNjObRq+szCwmK1KdOLxMTML/v+k0aCFdXag0ttZ/PMpApVj7o4OAZb4N 5CH3tY5kdqfwobqmshb7BiyfA1nJMPFbHMOz5sBWy1f0HGXDew3vN2gabrM5EJJ6Yp8+ /zTxp2yA4uLNdSmn6UkozjUkoyTA0eCHGqslV61JHH9oTTn4wDnjYeGsCw7EQNd1X9ye POpA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1750327743; x=1750932543; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=UsJG0sUjwmdzpZDVOl09h+1erDYt2a8aPHUb1k2opsw=; b=QDD86eQQ4Sx1WmUVTOVm50gmUSoMR3qGGvLgG/mjHQWgnVUZexW9BjEPEGNogVwtUh tEFs2kzkd9WDfPGVuD+ixA7df0nvQn8OaWC2lrCsqx4dbxsVefX0lxTSSuXjqedhBu+U UVcOMKgXXimQ3QL4rTJx8ttYMJ9mqJs1SlhNw4xjEd1ZoTwNi0sz1G/o8ra4odoqjhhH BXlLgdHz5tUOs6hiTIp1oGlEk7HuyrDH9HxkEoX25zIphyBk5hZa4IJW993OlSRblf5M Swp8sHq3lXERZl4WpaflazkmjB4HFTLan7DmnvLQ3UzuvblttAOODgGi8rcAST1lsQVT HCBw== X-Gm-Message-State: AOJu0Yw0WhQBLQPT0JnKcclFhpej08CJaRvPrpv8zYNCdpovNsijpaWu JZrxDwodHkuXEN0/3I6KFobaWXTSF6i5rdLHw3qH24j5FwnwoQFgQK2v49pvjDWnpWMPE+U1x16 Evw6J X-Gm-Gg: ASbGncv7k9fz54HQYb/B2Leh44c0ACN8qbPRCEPdPIbZpZZRnXua7DrjbwXu05K60rj qyIEbj5zWpOT90+L7n1Vh8yuJLQN1/MN/7us5Nl+NYPZ+yIcTnMT5c8oG4s+8h9OF/f8s9rRZbu LgP0p/wse8AJBC6f2oIoQWRW1eLO1CoMbm+AZLRty/yjOVZoySNqXvNrZScXGUXfKbYxx98xLcW djbrn+BV3ZtE+UV4/LYRAE2T0sVCkuN9TK/wmVAJttNuF6rLQT2K23yx6TbEoGqdnX0JcSvPhh2 AP0DA6uhY8VprazNoC60OLIR7NA1TNrGibOXTFxmh8aIR/b07eEsPE7GaKzjwOFH95bCz5nQ1N7 CPL5B3A== X-Google-Smtp-Source: AGHT+IHgI0VtbGGbbHClENwshKCGdK1RN+jS9GJWJ7LFEJt/F6GFyT7tKbxuPjOg+H+e/zVa8yio0w== X-Received: by 2002:a05:600c:b95:b0:43e:94fa:4aef with SMTP id 5b1f17b1804b1-4535efa1e0cmr8297005e9.8.1750327743248; Thu, 19 Jun 2025 03:09:03 -0700 (PDT) Received: from NAUSH-P-DELL.pitowers.org ([93.93.133.154]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-4535ebcecb5sm23926555e9.37.2025.06.19.03.09.02 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 19 Jun 2025 03:09:02 -0700 (PDT) From: Naushir Patuck To: libcamera-devel@lists.libcamera.org Cc: kieran.bingham@ideasonboard.com, David Plowman , Naushir Patuck Subject: [PATCH 4/4] pipeline: rpi: Add wallclock timestamp support Date: Thu, 19 Jun 2025 11:05:56 +0100 Message-ID: <20250619100857.124809-5-naush@raspberrypi.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20250619100857.124809-1-naush@raspberrypi.com> References: <20250619100857.124809-1-naush@raspberrypi.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" From: David Plowman A ClockRecovery object is added for derived classes to use, and wallclock timestamps are copied into the request metadata for applications. Wallclock timestamps are derived corresponding to the sensor timestamp, and made available to the base pipeline handler class and to IPAs, for both vc4 and pisp platforms. Signed-off-by: David Plowman Reviewed-by: Kieran Bingham Reviewed-by: Naushir Patuck Signed-off-by: Naushir Patuck --- src/libcamera/pipeline/rpi/common/pipeline_base.cpp | 5 +++++ src/libcamera/pipeline/rpi/common/pipeline_base.h | 3 +++ src/libcamera/pipeline/rpi/pisp/pisp.cpp | 10 ++++++++-- src/libcamera/pipeline/rpi/vc4/vc4.cpp | 10 ++++++++-- 4 files changed, 24 insertions(+), 4 deletions(-) diff --git a/src/libcamera/pipeline/rpi/common/pipeline_base.cpp b/src/libcamera/pipeline/rpi/common/pipeline_base.cpp index d8c7ca9352de..e14d3b913aaa 100644 --- a/src/libcamera/pipeline/rpi/common/pipeline_base.cpp +++ b/src/libcamera/pipeline/rpi/common/pipeline_base.cpp @@ -685,6 +685,9 @@ int PipelineHandlerBase::start(Camera *camera, const ControlList *controls) return ret; } + /* A good moment to add an initial clock sample. */ + data->wallClockRecovery_.addSample(); + /* * Reset the delayed controls with the gain and exposure values set by * the IPA. @@ -1482,6 +1485,8 @@ void CameraData::fillRequestMetadata(const ControlList &bufferControls, Request { request->metadata().set(controls::SensorTimestamp, bufferControls.get(controls::SensorTimestamp).value_or(0)); + request->metadata().set(controls::FrameWallClock, + bufferControls.get(controls::FrameWallClock).value_or(0)); if (cropParams_.size()) { std::vector crops; diff --git a/src/libcamera/pipeline/rpi/common/pipeline_base.h b/src/libcamera/pipeline/rpi/common/pipeline_base.h index 898f31577059..4c5743e04f86 100644 --- a/src/libcamera/pipeline/rpi/common/pipeline_base.h +++ b/src/libcamera/pipeline/rpi/common/pipeline_base.h @@ -20,6 +20,7 @@ #include "libcamera/internal/bayer_format.h" #include "libcamera/internal/camera.h" #include "libcamera/internal/camera_sensor.h" +#include "libcamera/internal/clock_recovery.h" #include "libcamera/internal/framebuffer.h" #include "libcamera/internal/media_device.h" #include "libcamera/internal/media_object.h" @@ -172,6 +173,8 @@ public: Config config_; + ClockRecovery wallClockRecovery_; + protected: void fillRequestMetadata(const ControlList &bufferControls, Request *request); diff --git a/src/libcamera/pipeline/rpi/pisp/pisp.cpp b/src/libcamera/pipeline/rpi/pisp/pisp.cpp index ccf135c3d8ce..2df91bacf3be 100644 --- a/src/libcamera/pipeline/rpi/pisp/pisp.cpp +++ b/src/libcamera/pipeline/rpi/pisp/pisp.cpp @@ -1755,9 +1755,15 @@ void PiSPCameraData::cfeBufferDequeue(FrameBuffer *buffer) auto [ctrl, delayContext] = delayedCtrls_->get(buffer->metadata().sequence); /* * Add the frame timestamp to the ControlList for the IPA to use - * as it does not receive the FrameBuffer object. + * as it does not receive the FrameBuffer object. Also derive a + * corresponding wallclock value. */ - ctrl.set(controls::SensorTimestamp, buffer->metadata().timestamp); + wallClockRecovery_.addSample(); + uint64_t sensorTimestamp = buffer->metadata().timestamp; + uint64_t wallClockTimestamp = wallClockRecovery_.getOutput(sensorTimestamp / 1000); + + ctrl.set(controls::SensorTimestamp, sensorTimestamp); + ctrl.set(controls::FrameWallClock, wallClockTimestamp); job.sensorControls = std::move(ctrl); job.delayContext = delayContext; } else if (stream == &cfe_[Cfe::Config]) { diff --git a/src/libcamera/pipeline/rpi/vc4/vc4.cpp b/src/libcamera/pipeline/rpi/vc4/vc4.cpp index ac6dab814d35..e99a7edf809c 100644 --- a/src/libcamera/pipeline/rpi/vc4/vc4.cpp +++ b/src/libcamera/pipeline/rpi/vc4/vc4.cpp @@ -773,9 +773,15 @@ void Vc4CameraData::unicamBufferDequeue(FrameBuffer *buffer) auto [ctrl, delayContext] = delayedCtrls_->get(buffer->metadata().sequence); /* * Add the frame timestamp to the ControlList for the IPA to use - * as it does not receive the FrameBuffer object. + * as it does not receive the FrameBuffer object. Also derive a + * corresponding wallclock value. */ - ctrl.set(controls::SensorTimestamp, buffer->metadata().timestamp); + wallClockRecovery_.addSample(); + uint64_t sensorTimestamp = buffer->metadata().timestamp; + uint64_t wallClockTimestamp = wallClockRecovery_.getOutput(sensorTimestamp / 1000); + + ctrl.set(controls::SensorTimestamp, sensorTimestamp); + ctrl.set(controls::FrameWallClock, wallClockTimestamp); bayerQueue_.push({ buffer, std::move(ctrl), delayContext }); } else { embeddedQueue_.push(buffer);