From patchwork Thu Jan 9 14:32:08 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Plowman X-Patchwork-Id: 22502 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id BFE9EC32EA for ; Thu, 9 Jan 2025 14:32:26 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 678F468536; Thu, 9 Jan 2025 15:32:26 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="AvJDToAo"; dkim-atps=neutral Received: from mail-wr1-x42a.google.com (mail-wr1-x42a.google.com [IPv6:2a00:1450:4864:20::42a]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id EF5F5684E2 for ; Thu, 9 Jan 2025 15:32:19 +0100 (CET) Received: by mail-wr1-x42a.google.com with SMTP id ffacd0b85a97d-3862b40a6e0so571433f8f.0 for ; Thu, 09 Jan 2025 06:32:19 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; t=1736433139; x=1737037939; darn=lists.libcamera.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=MyqtSsrjam7Je3P1fuyUgnWz9ifuQumfixpMt1SesR8=; b=AvJDToAoHZSSO8mAaGWWBClu+O2VO6im+UE5Iau8nrVQKPdvcOw0hu588fzi5HJaT4 +YO35Q4vAGVNG3xhUm6PXtJxH7HClxo5CwG0F3wcABBC4mEMl79d0KCETyfmmKTw0+rt RWSpLJZcPDtvQE9AybcY2nFQGUPbMfiJjnuM6FbX51dzBVTzfJY5Bh2UriDDMJsnCXEP AbyE1z4XPw0bATJd4UH/fnbjm9z5qQ7X8ce4R7c4qDJ7HP2f1JXlXJinqHLS4fLLtkha UjUni+zFaF6TlN9b5zynQAHj/g79R75iA3b5NdVV+h5Bl+1GMC3of/NEd0kpQpl9PdSh QEsg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1736433139; x=1737037939; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=MyqtSsrjam7Je3P1fuyUgnWz9ifuQumfixpMt1SesR8=; b=o/FofA25uSJwUgqfj57cbG09lDSNU+byEA4O0f/yk9vwE6vXUt/Z2PjUlpACuE0PYJ Nb8qrYyl/yJHw5NvpJ9WueXJ19GflRwvVdrymP3Gq/pVInTjvaxobT/AJRfrDSeu32TS W3lfX21OLJhwjP8YrsKZFj3EGze9r+j9GmQcd/9h8XFkhYkNqXHO5lclhN3S3vhRj8Pb VaoqQ8sEPksWkNjioII45NMIn4qLy75X33DoRRIw0uISKg8MKJkz+Lbaa1pzkv5qQNTP rLtccg6NkaQyUUt1Er1V/7MFUvb9oI7MFHwhsANwet23rdRT4o6Ob+3s7aFlKZcO5QSl b3lg== X-Gm-Message-State: AOJu0YzGU5sc4387VxI8aRuk2ON5R3sBqP84I8tJ0iGA4SVDWYeRN+KZ Vx8MHpeRMvcMGic5KU+EEYvD2ULSnWgQFvyhIoedEc2kmuUT4MXKJdMG944koC4F98OJvbl9x9E l X-Gm-Gg: ASbGncsmC3AhbB8JPyrt5xBvFLWR+m8i1SdGoOfovx2IdskxZJa/uDfHPJ4hjj0AUEm 2Vivc69O3Zs/pA98drpQWHD1YyBRSU6+n5V9Tbh/jA2w591hogCdQPxT0pFTXbYPO0ZOay8dYri SAnYa/Ya5oHANHvzJnd1zuX+2taDZ/NUaCqoZhGSH1k8n8YqCU6K2HUlF0b2dJMwcS1gwD+HGMh HGaW2W3Hl7aA6RSOonWV+QXrvxDJ182yWbfXbxJsIzEt9hfde8GgFndspKOhxl3blt03chOQ6nU YWcSEv+r2gXC X-Google-Smtp-Source: AGHT+IEIIp/CBeq9Y5yD9yFHh7cQhsebEOG/gj/u8RRQhR/Xpq/nx7Mog1Bh2vaL9BTZPcUVKlJoXQ== X-Received: by 2002:a05:6000:18a8:b0:388:c790:1dff with SMTP id ffacd0b85a97d-38a8733fd71mr4965527f8f.47.1736433139182; Thu, 09 Jan 2025 06:32:19 -0800 (PST) Received: from raspberrypi.pitowers.org ([2a00:1098:3142:1f:ffc9:aff6:7f7f:893b]) by smtp.gmail.com with ESMTPSA id ffacd0b85a97d-38a8e4b80b2sm1952569f8f.80.2025.01.09.06.32.18 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 09 Jan 2025 06:32:18 -0800 (PST) From: David Plowman To: libcamera-devel@lists.libcamera.org Cc: David Plowman Subject: [PATCH v3 4/7] pipeline: rpi:: Add wallclock timestamp support Date: Thu, 9 Jan 2025 14:32:08 +0000 Message-Id: <20250109143211.11939-5-david.plowman@raspberrypi.com> X-Mailer: git-send-email 2.39.5 In-Reply-To: <20250109143211.11939-1-david.plowman@raspberrypi.com> References: <20250109143211.11939-1-david.plowman@raspberrypi.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" A ClockRecovery object is added for derived classes to use, and wallclock timetamps are copied into the request metadata for applications. In the derived class, wallclock timestamps are derived corresponding to the sensor timestamp, and made available to the base pipeline handler class. Signed-off-by: David Plowman Reviewed-by: Kieran Bingham Reviewed-by: Naushir Patuck --- src/libcamera/pipeline/rpi/common/pipeline_base.cpp | 5 +++++ src/libcamera/pipeline/rpi/common/pipeline_base.h | 3 +++ src/libcamera/pipeline/rpi/vc4/vc4.cpp | 10 ++++++++-- 3 files changed, 16 insertions(+), 2 deletions(-) diff --git a/src/libcamera/pipeline/rpi/common/pipeline_base.cpp b/src/libcamera/pipeline/rpi/common/pipeline_base.cpp index 4b147fdb..56f3c4df 100644 --- a/src/libcamera/pipeline/rpi/common/pipeline_base.cpp +++ b/src/libcamera/pipeline/rpi/common/pipeline_base.cpp @@ -686,6 +686,9 @@ int PipelineHandlerBase::start(Camera *camera, const ControlList *controls) return ret; } + /* A good moment to add an initial clock sample. */ + data->wallClockRecovery_.addSample(); + /* * Reset the delayed controls with the gain and exposure values set by * the IPA. @@ -1510,6 +1513,8 @@ void CameraData::fillRequestMetadata(const ControlList &bufferControls, Request { request->metadata().set(controls::SensorTimestamp, bufferControls.get(controls::SensorTimestamp).value_or(0)); + request->metadata().set(controls::FrameWallClock, + bufferControls.get(controls::FrameWallClock).value_or(0)); if (cropParams_.size()) { std::vector crops; diff --git a/src/libcamera/pipeline/rpi/common/pipeline_base.h b/src/libcamera/pipeline/rpi/common/pipeline_base.h index aae0c2f3..ca706bf3 100644 --- a/src/libcamera/pipeline/rpi/common/pipeline_base.h +++ b/src/libcamera/pipeline/rpi/common/pipeline_base.h @@ -20,6 +20,7 @@ #include "libcamera/internal/bayer_format.h" #include "libcamera/internal/camera.h" #include "libcamera/internal/camera_sensor.h" +#include "libcamera/internal/clock_recovery.h" #include "libcamera/internal/framebuffer.h" #include "libcamera/internal/media_device.h" #include "libcamera/internal/media_object.h" @@ -177,6 +178,8 @@ public: Config config_; + ClockRecovery wallClockRecovery_; + protected: void fillRequestMetadata(const ControlList &bufferControls, Request *request); diff --git a/src/libcamera/pipeline/rpi/vc4/vc4.cpp b/src/libcamera/pipeline/rpi/vc4/vc4.cpp index fd8d84b1..3d8c65c6 100644 --- a/src/libcamera/pipeline/rpi/vc4/vc4.cpp +++ b/src/libcamera/pipeline/rpi/vc4/vc4.cpp @@ -781,9 +781,15 @@ void Vc4CameraData::unicamBufferDequeue(FrameBuffer *buffer) auto [ctrl, delayContext] = delayedCtrls_->get(buffer->metadata().sequence); /* * Add the frame timestamp to the ControlList for the IPA to use - * as it does not receive the FrameBuffer object. + * as it does not receive the FrameBuffer object. Also derive a + * corresponding wallclock value. */ - ctrl.set(controls::SensorTimestamp, buffer->metadata().timestamp); + wallClockRecovery_.addSample(); + uint64_t sensorTimestamp = buffer->metadata().timestamp; + uint64_t wallClockTimestamp = wallClockRecovery_.getOutput(sensorTimestamp / 1000); + + ctrl.set(controls::SensorTimestamp, sensorTimestamp); + ctrl.set(controls::FrameWallClock, wallClockTimestamp); bayerQueue_.push({ buffer, std::move(ctrl), delayContext }); } else { embeddedQueue_.push(buffer);