{"id":23605,"url":"https://patchwork.libcamera.org/api/patches/23605/?format=json","web_url":"https://patchwork.libcamera.org/patch/23605/","project":{"id":1,"url":"https://patchwork.libcamera.org/api/projects/1/?format=json","name":"libcamera","link_name":"libcamera","list_id":"libcamera_core","list_email":"libcamera-devel@lists.libcamera.org","web_url":"","scm_url":"","webscm_url":""},"msgid":"<20250619100857.124809-5-naush@raspberrypi.com>","date":"2025-06-19T10:05:56","name":"[4/4] pipeline: rpi: Add wallclock timestamp support","commit_ref":"1537da74427791bb3b5880e7d002daf8ea42db31","pull_url":null,"state":"accepted","archived":false,"hash":"afd671234110ee7e7bf96174eac2cd2aa2216ea8","submitter":{"id":34,"url":"https://patchwork.libcamera.org/api/people/34/?format=json","name":"Naushir Patuck","email":"naush@raspberrypi.com"},"delegate":null,"mbox":"https://patchwork.libcamera.org/patch/23605/mbox/","series":[{"id":5232,"url":"https://patchwork.libcamera.org/api/series/5232/?format=json","web_url":"https://patchwork.libcamera.org/project/libcamera/list/?series=5232","date":"2025-06-19T10:05:52","name":"Wallclock support","version":1,"mbox":"https://patchwork.libcamera.org/series/5232/mbox/"}],"comments":"https://patchwork.libcamera.org/api/patches/23605/comments/","check":"pending","checks":"https://patchwork.libcamera.org/api/patches/23605/checks/","tags":{},"headers":{"Return-Path":"","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id 02661C3240\n\tfor ;\n\tThu, 19 Jun 2025 10:09:13 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id 495D268DEB;\n\tThu, 19 Jun 2025 12:09:10 +0200 (CEST)","from mail-wm1-x332.google.com (mail-wm1-x332.google.com\n\t[IPv6:2a00:1450:4864:20::332])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id 48A3368DEB\n\tfor ;\n\tThu, 19 Jun 2025 12:09:04 +0200 (CEST)","by mail-wm1-x332.google.com with SMTP id\n\t5b1f17b1804b1-4519dd6523dso338225e9.1\n\tfor ;\n\tThu, 19 Jun 2025 03:09:04 -0700 (PDT)","from NAUSH-P-DELL.pitowers.org ([93.93.133.154])\n\tby smtp.gmail.com with ESMTPSA id\n\t5b1f17b1804b1-4535ebcecb5sm23926555e9.37.2025.06.19.03.09.02\n\t(version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);\n\tThu, 19 Jun 2025 03:09:02 -0700 (PDT)"],"Authentication-Results":"lancelot.ideasonboard.com; dkim=pass (2048-bit key;\n\tunprotected) header.d=raspberrypi.com header.i=@raspberrypi.com\n\theader.b=\"QCAiWwYw\"; dkim-atps=neutral","DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed;\n\td=raspberrypi.com; s=google; t=1750327743; x=1750932543;\n\tdarn=lists.libcamera.org; \n\th=content-transfer-encoding:mime-version:references:in-reply-to\n\t:message-id:date:subject:cc:to:from:from:to:cc:subject:date\n\t:message-id:reply-to;\n\tbh=UsJG0sUjwmdzpZDVOl09h+1erDYt2a8aPHUb1k2opsw=;\n\tb=QCAiWwYwu9D+PTr80zQCcp1mhpNPW+Dj1ZpbJ5XlxTK2+7T8RJlLaQCqZnUB9Jxqb5\n\tJLjCyw19h65ZCfDP2qJ+mKrJOefknlN6Bf9EZu6x44Umc7w4RnX3wOrwEByFAXMDw5nL\n\t4e0Q/74VQX2qNjObRq+szCwmK1KdOLxMTML/v+k0aCFdXag0ttZ/PMpApVj7o4OAZb4N\n\t5CH3tY5kdqfwobqmshb7BiyfA1nJMPFbHMOz5sBWy1f0HGXDew3vN2gabrM5EJJ6Yp8+\n\t/zTxp2yA4uLNdSmn6UkozjUkoyTA0eCHGqslV61JHH9oTTn4wDnjYeGsCw7EQNd1X9ye\n\tPOpA==","X-Google-DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed;\n\td=1e100.net; s=20230601; t=1750327743; x=1750932543;\n\th=content-transfer-encoding:mime-version:references:in-reply-to\n\t:message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc\n\t:subject:date:message-id:reply-to;\n\tbh=UsJG0sUjwmdzpZDVOl09h+1erDYt2a8aPHUb1k2opsw=;\n\tb=QDD86eQQ4Sx1WmUVTOVm50gmUSoMR3qGGvLgG/mjHQWgnVUZexW9BjEPEGNogVwtUh\n\ttEFs2kzkd9WDfPGVuD+ixA7df0nvQn8OaWC2lrCsqx4dbxsVefX0lxTSSuXjqedhBu+U\n\tUVcOMKgXXimQ3QL4rTJx8ttYMJ9mqJs1SlhNw4xjEd1ZoTwNi0sz1G/o8ra4odoqjhhH\n\tBXlLgdHz5tUOs6hiTIp1oGlEk7HuyrDH9HxkEoX25zIphyBk5hZa4IJW993OlSRblf5M\n\tSwp8sHq3lXERZl4WpaflazkmjB4HFTLan7DmnvLQ3UzuvblttAOODgGi8rcAST1lsQVT\n\tHCBw==","X-Gm-Message-State":"AOJu0Yw0WhQBLQPT0JnKcclFhpej08CJaRvPrpv8zYNCdpovNsijpaWu\n\tJZrxDwodHkuXEN0/3I6KFobaWXTSF6i5rdLHw3qH24j5FwnwoQFgQK2v49pvjDWnpWMPE+U1x16\n\tEvw6J","X-Gm-Gg":"ASbGncv7k9fz54HQYb/B2Leh44c0ACN8qbPRCEPdPIbZpZZRnXua7DrjbwXu05K60rj\n\tqyIEbj5zWpOT90+L7n1Vh8yuJLQN1/MN/7us5Nl+NYPZ+yIcTnMT5c8oG4s+8h9OF/f8s9rRZbu\n\tLgP0p/wse8AJBC6f2oIoQWRW1eLO1CoMbm+AZLRty/yjOVZoySNqXvNrZScXGUXfKbYxx98xLcW\n\tdjbrn+BV3ZtE+UV4/LYRAE2T0sVCkuN9TK/wmVAJttNuF6rLQT2K23yx6TbEoGqdnX0JcSvPhh2\n\tAP0DA6uhY8VprazNoC60OLIR7NA1TNrGibOXTFxmh8aIR/b07eEsPE7GaKzjwOFH95bCz5nQ1N7\n\tCPL5B3A==","X-Google-Smtp-Source":"AGHT+IHgI0VtbGGbbHClENwshKCGdK1RN+jS9GJWJ7LFEJt/F6GFyT7tKbxuPjOg+H+e/zVa8yio0w==","X-Received":"by 2002:a05:600c:b95:b0:43e:94fa:4aef with SMTP id\n\t5b1f17b1804b1-4535efa1e0cmr8297005e9.8.1750327743248; \n\tThu, 19 Jun 2025 03:09:03 -0700 (PDT)","From":"Naushir Patuck ","To":"libcamera-devel@lists.libcamera.org","Cc":"kieran.bingham@ideasonboard.com,\n\tDavid Plowman ,\n\tNaushir Patuck ","Subject":"[PATCH 4/4] pipeline: rpi: Add wallclock timestamp support","Date":"Thu, 19 Jun 2025 11:05:56 +0100","Message-ID":"<20250619100857.124809-5-naush@raspberrypi.com>","X-Mailer":"git-send-email 2.43.0","In-Reply-To":"<20250619100857.124809-1-naush@raspberrypi.com>","References":"<20250619100857.124809-1-naush@raspberrypi.com>","MIME-Version":"1.0","Content-Transfer-Encoding":"8bit","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" "},"content":"From: David Plowman \n\nA ClockRecovery object is added for derived classes to use, and\nwallclock timestamps are copied into the request metadata for\napplications.\n\nWallclock timestamps are derived corresponding to the sensor\ntimestamp, and made available to the base pipeline handler class and\nto IPAs, for both vc4 and pisp platforms.\n\nSigned-off-by: David Plowman \nReviewed-by: Kieran Bingham \nReviewed-by: Naushir Patuck \nSigned-off-by: Naushir Patuck \n---\n src/libcamera/pipeline/rpi/common/pipeline_base.cpp | 5 +++++\n src/libcamera/pipeline/rpi/common/pipeline_base.h | 3 +++\n src/libcamera/pipeline/rpi/pisp/pisp.cpp | 10 ++++++++--\n src/libcamera/pipeline/rpi/vc4/vc4.cpp | 10 ++++++++--\n 4 files changed, 24 insertions(+), 4 deletions(-)","diff":"diff --git a/src/libcamera/pipeline/rpi/common/pipeline_base.cpp b/src/libcamera/pipeline/rpi/common/pipeline_base.cpp\nindex d8c7ca9352de..e14d3b913aaa 100644\n--- a/src/libcamera/pipeline/rpi/common/pipeline_base.cpp\n+++ b/src/libcamera/pipeline/rpi/common/pipeline_base.cpp\n@@ -685,6 +685,9 @@ int PipelineHandlerBase::start(Camera *camera, const ControlList *controls)\n \t\treturn ret;\n \t}\n \n+\t/* A good moment to add an initial clock sample. */\n+\tdata->wallClockRecovery_.addSample();\n+\n \t/*\n \t * Reset the delayed controls with the gain and exposure values set by\n \t * the IPA.\n@@ -1482,6 +1485,8 @@ void CameraData::fillRequestMetadata(const ControlList &bufferControls, Request\n {\n \trequest->metadata().set(controls::SensorTimestamp,\n \t\t\t\tbufferControls.get(controls::SensorTimestamp).value_or(0));\n+\trequest->metadata().set(controls::FrameWallClock,\n+\t\t\t\tbufferControls.get(controls::FrameWallClock).value_or(0));\n \n \tif (cropParams_.size()) {\n \t\tstd::vector crops;\ndiff --git a/src/libcamera/pipeline/rpi/common/pipeline_base.h b/src/libcamera/pipeline/rpi/common/pipeline_base.h\nindex 898f31577059..4c5743e04f86 100644\n--- a/src/libcamera/pipeline/rpi/common/pipeline_base.h\n+++ b/src/libcamera/pipeline/rpi/common/pipeline_base.h\n@@ -20,6 +20,7 @@\n #include \"libcamera/internal/bayer_format.h\"\n #include \"libcamera/internal/camera.h\"\n #include \"libcamera/internal/camera_sensor.h\"\n+#include \"libcamera/internal/clock_recovery.h\"\n #include \"libcamera/internal/framebuffer.h\"\n #include \"libcamera/internal/media_device.h\"\n #include \"libcamera/internal/media_object.h\"\n@@ -172,6 +173,8 @@ public:\n \n \tConfig config_;\n \n+\tClockRecovery wallClockRecovery_;\n+\n protected:\n \tvoid fillRequestMetadata(const ControlList &bufferControls,\n \t\t\t\t Request *request);\ndiff --git a/src/libcamera/pipeline/rpi/pisp/pisp.cpp b/src/libcamera/pipeline/rpi/pisp/pisp.cpp\nindex ccf135c3d8ce..2df91bacf3be 100644\n--- a/src/libcamera/pipeline/rpi/pisp/pisp.cpp\n+++ b/src/libcamera/pipeline/rpi/pisp/pisp.cpp\n@@ -1755,9 +1755,15 @@ void PiSPCameraData::cfeBufferDequeue(FrameBuffer *buffer)\n \t\tauto [ctrl, delayContext] = delayedCtrls_->get(buffer->metadata().sequence);\n \t\t/*\n \t\t * Add the frame timestamp to the ControlList for the IPA to use\n-\t\t * as it does not receive the FrameBuffer object.\n+\t\t * as it does not receive the FrameBuffer object. Also derive a\n+\t\t * corresponding wallclock value.\n \t\t */\n-\t\tctrl.set(controls::SensorTimestamp, buffer->metadata().timestamp);\n+\t\twallClockRecovery_.addSample();\n+\t\tuint64_t sensorTimestamp = buffer->metadata().timestamp;\n+\t\tuint64_t wallClockTimestamp = wallClockRecovery_.getOutput(sensorTimestamp / 1000);\n+\n+\t\tctrl.set(controls::SensorTimestamp, sensorTimestamp);\n+\t\tctrl.set(controls::FrameWallClock, wallClockTimestamp);\n \t\tjob.sensorControls = std::move(ctrl);\n \t\tjob.delayContext = delayContext;\n \t} else if (stream == &cfe_[Cfe::Config]) {\ndiff --git a/src/libcamera/pipeline/rpi/vc4/vc4.cpp b/src/libcamera/pipeline/rpi/vc4/vc4.cpp\nindex ac6dab814d35..e99a7edf809c 100644\n--- a/src/libcamera/pipeline/rpi/vc4/vc4.cpp\n+++ b/src/libcamera/pipeline/rpi/vc4/vc4.cpp\n@@ -773,9 +773,15 @@ void Vc4CameraData::unicamBufferDequeue(FrameBuffer *buffer)\n \t\tauto [ctrl, delayContext] = delayedCtrls_->get(buffer->metadata().sequence);\n \t\t/*\n \t\t * Add the frame timestamp to the ControlList for the IPA to use\n-\t\t * as it does not receive the FrameBuffer object.\n+\t\t * as it does not receive the FrameBuffer object. Also derive a\n+\t\t * corresponding wallclock value.\n \t\t */\n-\t\tctrl.set(controls::SensorTimestamp, buffer->metadata().timestamp);\n+\t\twallClockRecovery_.addSample();\n+\t\tuint64_t sensorTimestamp = buffer->metadata().timestamp;\n+\t\tuint64_t wallClockTimestamp = wallClockRecovery_.getOutput(sensorTimestamp / 1000);\n+\n+\t\tctrl.set(controls::SensorTimestamp, sensorTimestamp);\n+\t\tctrl.set(controls::FrameWallClock, wallClockTimestamp);\n \t\tbayerQueue_.push({ buffer, std::move(ctrl), delayContext });\n \t} else {\n \t\tembeddedQueue_.push(buffer);\n","prefixes":["4/4"]}