{"id":22685,"url":"https://patchwork.libcamera.org/api/1.1/patches/22685/?format=json","web_url":"https://patchwork.libcamera.org/patch/22685/","project":{"id":1,"url":"https://patchwork.libcamera.org/api/1.1/projects/1/?format=json","name":"libcamera","link_name":"libcamera","list_id":"libcamera_core","list_email":"libcamera-devel@lists.libcamera.org","web_url":"","scm_url":"","webscm_url":""},"msgid":"<20250122145349.7220-5-david.plowman@raspberrypi.com>","date":"2025-01-22T14:53:46","name":"[v4,4/7] pipeline: rpi: common: Add wallclock timestamp support","commit_ref":null,"pull_url":null,"state":"accepted","archived":false,"hash":"c91e7c64dcff96672311913c0d42279be040fdb2","submitter":{"id":42,"url":"https://patchwork.libcamera.org/api/1.1/people/42/?format=json","name":"David Plowman","email":"david.plowman@raspberrypi.com"},"delegate":null,"mbox":"https://patchwork.libcamera.org/patch/22685/mbox/","series":[{"id":4964,"url":"https://patchwork.libcamera.org/api/1.1/series/4964/?format=json","web_url":"https://patchwork.libcamera.org/project/libcamera/list/?series=4964","date":"2025-01-22T14:53:47","name":"Camera synchronisation","version":4,"mbox":"https://patchwork.libcamera.org/series/4964/mbox/"}],"comments":"https://patchwork.libcamera.org/api/patches/22685/comments/","check":"pending","checks":"https://patchwork.libcamera.org/api/patches/22685/checks/","tags":{},"headers":{"Return-Path":"","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":["parsemail@patchwork.libcamera.org","kbingham@ideasonboard.com"],"Received":["from perceval.ideasonboard.com (perceval.ideasonboard.com\n\t[213.167.242.64])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id D7F29BD808\n\tfor ;\n\tThu, 30 Jan 2025 11:52:33 +0000 (UTC)","from pendragon.ideasonboard.com\n\t(cpc89244-aztw30-2-0-cust6594.18-1.cable.virginm.net\n\t[86.31.185.195])\n\tby perceval.ideasonboard.com (Postfix) with ESMTPSA id 6AC9F886\n\tfor ;\n\tThu, 30 Jan 2025 12:51:25 +0100 (CET)","from perceval.ideasonboard.com\n\tby perceval.ideasonboard.com with LMTP id MO1/JkUGkWeNRjQA4E0KoQ\n\t(envelope-from )\n\tfor ; Wed, 22 Jan 2025 15:52:53 +0100","from mail-wm1-x329.google.com (mail-wm1-x329.google.com\n\t[IPv6:2a00:1450:4864:20::329])\tby perceval.ideasonboard.com (Postfix)\n\twith ESMTPS id 847EDC16\tfor ;\n\tWed, 22 Jan 2025 15:52:53 +0100 (CET)","by mail-wm1-x329.google.com with SMTP id\n\t5b1f17b1804b1-436326dcb1cso49406445e9.0 for\n\t;\n\tWed, 22 Jan 2025 06:53:56 -0800 (PST)","from raspberrypi.pitowers.org\n\t([2a00:1098:3142:1f:ffc9:aff6:7f7f:893b]) by smtp.gmail.com with\n\tESMTPSA id\n\t5b1f17b1804b1-438b66dc08bsm11551395e9.37.2025.01.22.06.53.54\n\t(version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); \n\tWed, 22 Jan 2025 06:53:55 -0800 (PST)"],"Authentication-Results":"perceval.ideasonboard.com;\n\tdkim=permerror header.d=raspberrypi.com header.i=@raspberrypi.com\n\theader.a=rsa-sha1 header.s=google header.b=K0obU6lg; \n\tdkim-atps=neutral","DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed; \n\td=raspberrypi.com; s=google; t=1737557635; x=1738162435;\n\tdarn=ideasonboard.com; \n\th=content-transfer-encoding:mime-version:references:in-reply-to \n\t:message-id:date:subject:cc:to:from:from:to:cc:subject:date \n\t:message-id:reply-to; bh=/Th3SY4pThmSp8bZ5uSgh71EJfe/F3FqF6YTxuYoO50=;\n\tb=K0obU6lgltexfCSJcFYheH14o3ov3PV3eCvMi3BoH4A+VQXNkz1bAoILmi21j/scii\n\tMBhW44yWBc6w8if58qx2vX1divQvpT3RHonOgkAOrEhEBaxnYHcJ5y/w7/avqB7CSxQI\n\t3FE8YKZBNpwUVEa47FpZBDpnpo+aBnXK1SJz0chTKkFYvaNW2D/KuhRoOGYNi/avjG6q\n\tHRPpO+gY+3Jq4BG3ZPKfI/NiHDfnpx/pmAnnvwF/kKTwBIXdd2UnmWqcDh3KHRFgFOZT\n\t/76tUuImPxuipGWmhZeo1mKqFVWj4fjYoZtnAptqCyhnlQkq6vSngudhzeztVt7jp1SP\n\t0d7A==","X-Google-DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed;\n\td=1e100.net; s=20230601; t=1737557635; x=1738162435;\n\th=content-transfer-encoding:mime-version:references:in-reply-to\n\t:message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc\n\t:subject:date:message-id:reply-to;\n\tbh=/Th3SY4pThmSp8bZ5uSgh71EJfe/F3FqF6YTxuYoO50=;\n\tb=hW7BxMVmOC9LrETCg51dOncOW4/Uyr6GmCe33Q57dQz6ceheNyShZZwHLFJOdQoeTO\n\t0meyNAsZAE+G/KDUjfSW9ZM9xxvwSBRjRBa2W50CmLlGCCdingbkIWGkvroBVff/fAB/\n\tMW5ao3X25tT2E4IkWZ6EDiKmUpQrdFb1RtI6lI8/urcIhrpjMFc8Sn2mEIhtgkTpL1gC\n\t9ahmHd/jBP9LGHASYe9AfKiwZV8L+dRpEyutmbvRWhOu1W1PsnjFxZybs4xY8/ZtIGBY\n\tAnE+pPjJBOxV2A8Ces4uNqLGyIJDKuJoWn9PMHik94MUOmsBpRE5sxJgPUy3XpQxFMyR\n\tiWDg==","X-Forwarded-Encrypted":"i=1; =?utf-8?q?AJvYcCV5U+7RxCS9SazbwgmVq2J6zTdmN2ex?=\n\t=?utf-8?q?Q1hI7TsZjPWta5GDRa8Js2bo3XhxMjOs6WvD1t0yUL0IFiZMtS6PCw?=\n\t=?utf-8?q?=3D=3D=40ideasonboard=2Ecom?=","X-Gm-Message-State":"AOJu0YzPeEjeK6qrN6jGyWU+xp/zVw/lj1ZmXXgX0GCs6y9uvXVe7OD3\n\tCQOBvHEhsKvNpvFRErr+/xACl7z4364+DKik+kHzNwdRwTOEX0eJ23YEmr+L0bg=","X-Gm-Gg":"ASbGncsctvXDukdrOhiOjmQEsm7J1P41bMK9L26QScyQb5z03tBnphaXMC28aOs3j+A\n\tP6Tz3QZG7Duedv774X8RfS90iNKOGKe0PSo7oPDxObnxQ2rge0oUkjwYdiRbSWlLdcUQamyIb+W\n\tKjxVtr0btIP8dP8+lg/6rLkgUiYVio7g1grJ+EZ9td6CLeec4B3i8e7ZRUfDVXi8YCJYyVCKt5a\n\tQnNQ/LkyJeS0vf/mcYe2clPqLdbmzEOFMPedCWCIWscmZfoLvPBMjTuz3a2T79mq9iDUBz0xsED\n\tMbV2SrsVAlxcipts87YZwqNtkA==","X-Google-Smtp-Source":"=?utf-8?q?AGHT+IE7Jc5DUGLCCBpThrzWHE90sZRjygCl0pvs7?=\n\t=?utf-8?q?j/LJG7u9axhNRJhYJlDh/IaeNcBVyVhuz1w0g=3D=3D?=","X-Received":"by 2002:a05:600c:1e02:b0:438:ad4d:cf09 with SMTP id\n\t5b1f17b1804b1-438ad4dd134mr84433575e9.9.1737557635422; Wed, 22 Jan\n\t2025 06:53:55 -0800 (PST)","From":"David Plowman ","To":"libcamera-devel@lists.libcamera.org","Cc":"David Plowman ,\n\tKieran Bingham ,\n\tNaushir Patuck ","Subject":"[PATCH v4 4/7] pipeline: rpi: common: Add wallclock timestamp\n\tsupport","Date":"Wed, 22 Jan 2025 14:53:46 +0000","Message-Id":"<20250122145349.7220-5-david.plowman@raspberrypi.com>","X-Mailer":"git-send-email 2.39.5","In-Reply-To":"<20250122145349.7220-1-david.plowman@raspberrypi.com>","References":"<20250122145349.7220-1-david.plowman@raspberrypi.com>","MIME-Version":"1.0","Content-Transfer-Encoding":"8bit","X-TUID":"uwkgrETFaAFz","Resent-From":"Kieran Bingham ","Resent-To":"parsemail@patchwork.libcamera.org"},"content":"A ClockRecovery object is added for derived classes to use, and\nwallclock timestamps are copied into the request metadata for\napplications.\n\nWallclock timestamps are derived corresponding to the sensor\ntimestamp, and made available to the base pipeline handler class and\nto IPAs.\n\nSigned-off-by: David Plowman \nReviewed-by: Kieran Bingham \nReviewed-by: Naushir Patuck \n---\n src/libcamera/pipeline/rpi/common/pipeline_base.cpp | 5 +++++\n src/libcamera/pipeline/rpi/common/pipeline_base.h | 3 +++\n src/libcamera/pipeline/rpi/vc4/vc4.cpp | 10 ++++++++--\n 3 files changed, 16 insertions(+), 2 deletions(-)","diff":"diff --git a/src/libcamera/pipeline/rpi/common/pipeline_base.cpp b/src/libcamera/pipeline/rpi/common/pipeline_base.cpp\nindex 4b147fdb..56f3c4df 100644\n--- a/src/libcamera/pipeline/rpi/common/pipeline_base.cpp\n+++ b/src/libcamera/pipeline/rpi/common/pipeline_base.cpp\n@@ -686,6 +686,9 @@ int PipelineHandlerBase::start(Camera *camera, const ControlList *controls)\n \t\treturn ret;\n \t}\n \n+\t/* A good moment to add an initial clock sample. */\n+\tdata->wallClockRecovery_.addSample();\n+\n \t/*\n \t * Reset the delayed controls with the gain and exposure values set by\n \t * the IPA.\n@@ -1510,6 +1513,8 @@ void CameraData::fillRequestMetadata(const ControlList &bufferControls, Request\n {\n \trequest->metadata().set(controls::SensorTimestamp,\n \t\t\t\tbufferControls.get(controls::SensorTimestamp).value_or(0));\n+\trequest->metadata().set(controls::FrameWallClock,\n+\t\t\t\tbufferControls.get(controls::FrameWallClock).value_or(0));\n \n \tif (cropParams_.size()) {\n \t\tstd::vector crops;\ndiff --git a/src/libcamera/pipeline/rpi/common/pipeline_base.h b/src/libcamera/pipeline/rpi/common/pipeline_base.h\nindex aae0c2f3..ca706bf3 100644\n--- a/src/libcamera/pipeline/rpi/common/pipeline_base.h\n+++ b/src/libcamera/pipeline/rpi/common/pipeline_base.h\n@@ -20,6 +20,7 @@\n #include \"libcamera/internal/bayer_format.h\"\n #include \"libcamera/internal/camera.h\"\n #include \"libcamera/internal/camera_sensor.h\"\n+#include \"libcamera/internal/clock_recovery.h\"\n #include \"libcamera/internal/framebuffer.h\"\n #include \"libcamera/internal/media_device.h\"\n #include \"libcamera/internal/media_object.h\"\n@@ -177,6 +178,8 @@ public:\n \n \tConfig config_;\n \n+\tClockRecovery wallClockRecovery_;\n+\n protected:\n \tvoid fillRequestMetadata(const ControlList &bufferControls,\n \t\t\t\t Request *request);\ndiff --git a/src/libcamera/pipeline/rpi/vc4/vc4.cpp b/src/libcamera/pipeline/rpi/vc4/vc4.cpp\nindex fd8d84b1..3d8c65c6 100644\n--- a/src/libcamera/pipeline/rpi/vc4/vc4.cpp\n+++ b/src/libcamera/pipeline/rpi/vc4/vc4.cpp\n@@ -781,9 +781,15 @@ void Vc4CameraData::unicamBufferDequeue(FrameBuffer *buffer)\n \t\tauto [ctrl, delayContext] = delayedCtrls_->get(buffer->metadata().sequence);\n \t\t/*\n \t\t * Add the frame timestamp to the ControlList for the IPA to use\n-\t\t * as it does not receive the FrameBuffer object.\n+\t\t * as it does not receive the FrameBuffer object. Also derive a\n+\t\t * corresponding wallclock value.\n \t\t */\n-\t\tctrl.set(controls::SensorTimestamp, buffer->metadata().timestamp);\n+\t\twallClockRecovery_.addSample();\n+\t\tuint64_t sensorTimestamp = buffer->metadata().timestamp;\n+\t\tuint64_t wallClockTimestamp = wallClockRecovery_.getOutput(sensorTimestamp / 1000);\n+\n+\t\tctrl.set(controls::SensorTimestamp, sensorTimestamp);\n+\t\tctrl.set(controls::FrameWallClock, wallClockTimestamp);\n \t\tbayerQueue_.push({ buffer, std::move(ctrl), delayContext });\n \t} else {\n \t\tembeddedQueue_.push(buffer);\n","prefixes":["v4","4/7"]}