From patchwork Wed Jan 22 14:53:46 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Plowman X-Patchwork-Id: 22685 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by patchwork.libcamera.org (Postfix) with ESMTPS id D7F29BD808 for ; Thu, 30 Jan 2025 11:52:33 +0000 (UTC) Received: from pendragon.ideasonboard.com (cpc89244-aztw30-2-0-cust6594.18-1.cable.virginm.net [86.31.185.195]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 6AC9F886 for ; Thu, 30 Jan 2025 12:51:25 +0100 (CET) Authentication-Results: perceval.ideasonboard.com; dkim=permerror header.d=raspberrypi.com header.i=@raspberrypi.com header.a=rsa-sha1 header.s=google header.b=K0obU6lg; dkim-atps=neutral Delivered-To: kbingham@ideasonboard.com Received: from perceval.ideasonboard.com by perceval.ideasonboard.com with LMTP id MO1/JkUGkWeNRjQA4E0KoQ (envelope-from ) for ; Wed, 22 Jan 2025 15:52:53 +0100 Received: from mail-wm1-x329.google.com (mail-wm1-x329.google.com [IPv6:2a00:1450:4864:20::329]) by perceval.ideasonboard.com (Postfix) with ESMTPS id 847EDC16 for ; Wed, 22 Jan 2025 15:52:53 +0100 (CET) Received: by mail-wm1-x329.google.com with SMTP id 5b1f17b1804b1-436326dcb1cso49406445e9.0 for ; Wed, 22 Jan 2025 06:53:56 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; t=1737557635; x=1738162435; darn=ideasonboard.com; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=/Th3SY4pThmSp8bZ5uSgh71EJfe/F3FqF6YTxuYoO50=; b=K0obU6lgltexfCSJcFYheH14o3ov3PV3eCvMi3BoH4A+VQXNkz1bAoILmi21j/scii MBhW44yWBc6w8if58qx2vX1divQvpT3RHonOgkAOrEhEBaxnYHcJ5y/w7/avqB7CSxQI 3FE8YKZBNpwUVEa47FpZBDpnpo+aBnXK1SJz0chTKkFYvaNW2D/KuhRoOGYNi/avjG6q HRPpO+gY+3Jq4BG3ZPKfI/NiHDfnpx/pmAnnvwF/kKTwBIXdd2UnmWqcDh3KHRFgFOZT /76tUuImPxuipGWmhZeo1mKqFVWj4fjYoZtnAptqCyhnlQkq6vSngudhzeztVt7jp1SP 0d7A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1737557635; x=1738162435; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=/Th3SY4pThmSp8bZ5uSgh71EJfe/F3FqF6YTxuYoO50=; b=hW7BxMVmOC9LrETCg51dOncOW4/Uyr6GmCe33Q57dQz6ceheNyShZZwHLFJOdQoeTO 0meyNAsZAE+G/KDUjfSW9ZM9xxvwSBRjRBa2W50CmLlGCCdingbkIWGkvroBVff/fAB/ MW5ao3X25tT2E4IkWZ6EDiKmUpQrdFb1RtI6lI8/urcIhrpjMFc8Sn2mEIhtgkTpL1gC 9ahmHd/jBP9LGHASYe9AfKiwZV8L+dRpEyutmbvRWhOu1W1PsnjFxZybs4xY8/ZtIGBY AnE+pPjJBOxV2A8Ces4uNqLGyIJDKuJoWn9PMHik94MUOmsBpRE5sxJgPUy3XpQxFMyR iWDg== X-Forwarded-Encrypted: i=1; =?utf-8?q?AJvYcCV5U+7RxCS9SazbwgmVq2J6zTdmN2ex?= =?utf-8?q?Q1hI7TsZjPWta5GDRa8Js2bo3XhxMjOs6WvD1t0yUL0IFiZMtS6PCw?= =?utf-8?q?=3D=3D=40ideasonboard=2Ecom?= X-Gm-Message-State: AOJu0YzPeEjeK6qrN6jGyWU+xp/zVw/lj1ZmXXgX0GCs6y9uvXVe7OD3 CQOBvHEhsKvNpvFRErr+/xACl7z4364+DKik+kHzNwdRwTOEX0eJ23YEmr+L0bg= X-Gm-Gg: ASbGncsctvXDukdrOhiOjmQEsm7J1P41bMK9L26QScyQb5z03tBnphaXMC28aOs3j+A P6Tz3QZG7Duedv774X8RfS90iNKOGKe0PSo7oPDxObnxQ2rge0oUkjwYdiRbSWlLdcUQamyIb+W KjxVtr0btIP8dP8+lg/6rLkgUiYVio7g1grJ+EZ9td6CLeec4B3i8e7ZRUfDVXi8YCJYyVCKt5a QnNQ/LkyJeS0vf/mcYe2clPqLdbmzEOFMPedCWCIWscmZfoLvPBMjTuz3a2T79mq9iDUBz0xsED MbV2SrsVAlxcipts87YZwqNtkA== X-Google-Smtp-Source: =?utf-8?q?AGHT+IE7Jc5DUGLCCBpThrzWHE90sZRjygCl0pvs7?= =?utf-8?q?j/LJG7u9axhNRJhYJlDh/IaeNcBVyVhuz1w0g=3D=3D?= X-Received: by 2002:a05:600c:1e02:b0:438:ad4d:cf09 with SMTP id 5b1f17b1804b1-438ad4dd134mr84433575e9.9.1737557635422; Wed, 22 Jan 2025 06:53:55 -0800 (PST) Received: from raspberrypi.pitowers.org ([2a00:1098:3142:1f:ffc9:aff6:7f7f:893b]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-438b66dc08bsm11551395e9.37.2025.01.22.06.53.54 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 22 Jan 2025 06:53:55 -0800 (PST) From: David Plowman To: libcamera-devel@lists.libcamera.org Cc: David Plowman , Kieran Bingham , Naushir Patuck Subject: [PATCH v4 4/7] pipeline: rpi: common: Add wallclock timestamp support Date: Wed, 22 Jan 2025 14:53:46 +0000 Message-Id: <20250122145349.7220-5-david.plowman@raspberrypi.com> X-Mailer: git-send-email 2.39.5 In-Reply-To: <20250122145349.7220-1-david.plowman@raspberrypi.com> References: <20250122145349.7220-1-david.plowman@raspberrypi.com> MIME-Version: 1.0 X-TUID: uwkgrETFaAFz Resent-From: Kieran Bingham Resent-To: parsemail@patchwork.libcamera.org A ClockRecovery object is added for derived classes to use, and wallclock timestamps are copied into the request metadata for applications. Wallclock timestamps are derived corresponding to the sensor timestamp, and made available to the base pipeline handler class and to IPAs. Signed-off-by: David Plowman Reviewed-by: Kieran Bingham Reviewed-by: Naushir Patuck --- src/libcamera/pipeline/rpi/common/pipeline_base.cpp | 5 +++++ src/libcamera/pipeline/rpi/common/pipeline_base.h | 3 +++ src/libcamera/pipeline/rpi/vc4/vc4.cpp | 10 ++++++++-- 3 files changed, 16 insertions(+), 2 deletions(-) diff --git a/src/libcamera/pipeline/rpi/common/pipeline_base.cpp b/src/libcamera/pipeline/rpi/common/pipeline_base.cpp index 4b147fdb..56f3c4df 100644 --- a/src/libcamera/pipeline/rpi/common/pipeline_base.cpp +++ b/src/libcamera/pipeline/rpi/common/pipeline_base.cpp @@ -686,6 +686,9 @@ int PipelineHandlerBase::start(Camera *camera, const ControlList *controls) return ret; } + /* A good moment to add an initial clock sample. */ + data->wallClockRecovery_.addSample(); + /* * Reset the delayed controls with the gain and exposure values set by * the IPA. @@ -1510,6 +1513,8 @@ void CameraData::fillRequestMetadata(const ControlList &bufferControls, Request { request->metadata().set(controls::SensorTimestamp, bufferControls.get(controls::SensorTimestamp).value_or(0)); + request->metadata().set(controls::FrameWallClock, + bufferControls.get(controls::FrameWallClock).value_or(0)); if (cropParams_.size()) { std::vector crops; diff --git a/src/libcamera/pipeline/rpi/common/pipeline_base.h b/src/libcamera/pipeline/rpi/common/pipeline_base.h index aae0c2f3..ca706bf3 100644 --- a/src/libcamera/pipeline/rpi/common/pipeline_base.h +++ b/src/libcamera/pipeline/rpi/common/pipeline_base.h @@ -20,6 +20,7 @@ #include "libcamera/internal/bayer_format.h" #include "libcamera/internal/camera.h" #include "libcamera/internal/camera_sensor.h" +#include "libcamera/internal/clock_recovery.h" #include "libcamera/internal/framebuffer.h" #include "libcamera/internal/media_device.h" #include "libcamera/internal/media_object.h" @@ -177,6 +178,8 @@ public: Config config_; + ClockRecovery wallClockRecovery_; + protected: void fillRequestMetadata(const ControlList &bufferControls, Request *request); diff --git a/src/libcamera/pipeline/rpi/vc4/vc4.cpp b/src/libcamera/pipeline/rpi/vc4/vc4.cpp index fd8d84b1..3d8c65c6 100644 --- a/src/libcamera/pipeline/rpi/vc4/vc4.cpp +++ b/src/libcamera/pipeline/rpi/vc4/vc4.cpp @@ -781,9 +781,15 @@ void Vc4CameraData::unicamBufferDequeue(FrameBuffer *buffer) auto [ctrl, delayContext] = delayedCtrls_->get(buffer->metadata().sequence); /* * Add the frame timestamp to the ControlList for the IPA to use - * as it does not receive the FrameBuffer object. + * as it does not receive the FrameBuffer object. Also derive a + * corresponding wallclock value. */ - ctrl.set(controls::SensorTimestamp, buffer->metadata().timestamp); + wallClockRecovery_.addSample(); + uint64_t sensorTimestamp = buffer->metadata().timestamp; + uint64_t wallClockTimestamp = wallClockRecovery_.getOutput(sensorTimestamp / 1000); + + ctrl.set(controls::SensorTimestamp, sensorTimestamp); + ctrl.set(controls::FrameWallClock, wallClockTimestamp); bayerQueue_.push({ buffer, std::move(ctrl), delayContext }); } else { embeddedQueue_.push(buffer);