From patchwork Fri Dec 6 14:27:38 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Plowman X-Patchwork-Id: 22212 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 8D419BE173 for ; Fri, 6 Dec 2024 14:27:54 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 966B366175; Fri, 6 Dec 2024 15:27:50 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="bAbjHXHO"; dkim-atps=neutral Received: from mail-wm1-x329.google.com (mail-wm1-x329.google.com [IPv6:2a00:1450:4864:20::329]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 95CAA66158 for ; Fri, 6 Dec 2024 15:27:46 +0100 (CET) Received: by mail-wm1-x329.google.com with SMTP id 5b1f17b1804b1-434a9f2da82so14413385e9.2 for ; Fri, 06 Dec 2024 06:27:46 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; t=1733495266; x=1734100066; darn=lists.libcamera.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=Ix5wJI5o5aG4F8QgGrowVLLKPUwwpcwGZhwknVZBHnc=; b=bAbjHXHOelFxx712eez0mE6q6u5ghnyWS0rBbhzV+cnno9tTFu0gADZ4nMEQ6hxY4g me0MtSvKaVHLYG6CwqATvYOVu6CxejJjm6gtHwL6zPT53hnDC81JFT93Mod02xt4Gsqj Q8veguoWF/70Xavz6Dy1KipTgWGds7L59GVazCc4H+N4W4w4pz7j7oe/jFaQnGFhNwC3 R8KPPi4wNd8I3S5J3iwbBiQj5vfCwj6BBgYRdRK7KRbeWUPYqA23Cv/WbAJmiBab7JVh OCteCsTTNL2DicohIE+fnCoQuqTrW9pw1TfqHbJp03cCz2yvpDetSa5811MsKAM9LIGf HrXA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1733495266; x=1734100066; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=Ix5wJI5o5aG4F8QgGrowVLLKPUwwpcwGZhwknVZBHnc=; b=aOA+4Dt4fCepfpGTpZbHTnuEzosiyZPhHoIzl5x3djKDJzIiZ2l2fa0H6j0uKjn1hy 2FHCF3mmU6Zvzf+ypw8DUEa+zCYmhmoaxJXYJqNoY+4BpQwC+KlJiv3Mw1GeaMGFXeHk /V6Z9Dn8nnsEdbXetH606OZOzXvLx6wyb6FJhdXTGMHysn5AvqGKFZM2RRd/ZG0SkPji IkdMnG54DWA5mBsbzyuGMSvCFPt6DKmhLkqUA0i01/isFefPEDW+BEehr4K3KJPPWJQM vonxxtk+uEMIgGCL8+Btr1ceakYg1SP+dzcZpdE1Rk8laTiVkdalhCapVPig5ZlKVKYc ichw== X-Gm-Message-State: AOJu0YwBK9ElVogO2Fq20s1BlrXdOJgurorFm8+RWMsTtGa7Skh63V+/ qUhZQUsgki1OhYpZVHFOWfQeSJo0ztp9VhXYXmyXwnVpwqLDT1KH5vL1pCJneve59JMYnRB0SAr c X-Gm-Gg: ASbGncuZzidmLEErD89My1ptOp93vS2ZFZBDrr719edIpYD3uG28/jAMpkUieFvHzgf pBsUrhH9s/WYcZCalZXF31oyOCFhFsm3dVWyOkRCgDOSvMnKrxu+ppjaFYaic68M+d/UDKqAZgz SUPw+2XoSTZR1MUlPY0f9/GATP/8wijct/7tSdj82Rzka6ZVyof+XaqGFgmZuxpiLsOLyl+NRzi LMPdz9nQV4mqBs+Ld9p2nmkEfkgTVYHOcEif/uav1lu35TKdi9wFp16DBxWWAJM9bVtff3apyA= X-Google-Smtp-Source: AGHT+IF0Udg/FhpCltpUlBqjSU3EGJbhseUhuu/cEkZHmRB+cMRxy/aIkD9Kn5DHsETac8i8rrMIWA== X-Received: by 2002:a05:600c:1d21:b0:434:9cc4:5139 with SMTP id 5b1f17b1804b1-434ddeb0492mr33102595e9.17.1733495265845; Fri, 06 Dec 2024 06:27:45 -0800 (PST) Received: from localhost.localdomain ([88.202.252.90]) by smtp.gmail.com with ESMTPSA id ffacd0b85a97d-386220b071dsm4608219f8f.101.2024.12.06.06.27.45 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 06 Dec 2024 06:27:45 -0800 (PST) From: David Plowman To: libcamera-devel@lists.libcamera.org Cc: David Plowman Subject: [PATCH 1/5] controls: Add FrameWallClock control Date: Fri, 6 Dec 2024 14:27:38 +0000 Message-Id: <20241206142742.7931-2-david.plowman@raspberrypi.com> X-Mailer: git-send-email 2.39.5 In-Reply-To: <20241206142742.7931-1-david.plowman@raspberrypi.com> References: <20241206142742.7931-1-david.plowman@raspberrypi.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Add a FrameWallClock control that reports the same moment as the frame's SensorTimestamp, but in wallclock units. Signed-off-by: David Plowman Reviewed-by: Kieran Bingham Reviewed-by: Naushir Patuck Reviewed-by: Laurent Pinchart --- src/libcamera/control_ids_core.yaml | 14 ++++++++++++++ 1 file changed, 14 insertions(+) diff --git a/src/libcamera/control_ids_core.yaml b/src/libcamera/control_ids_core.yaml index d45cf8e5..d951b2a3 100644 --- a/src/libcamera/control_ids_core.yaml +++ b/src/libcamera/control_ids_core.yaml @@ -971,4 +971,18 @@ controls: description: | Enable or disable the debug metadata. + - FrameWallClock: + type: int64_t + description: | + This timestamp corresponds to the same moment in time as the + SensorTimestamp, but is represented as a wall clock time as measured by + the Linux-specific CLOCK_REALTIME clock id. + + Being a wall clock measurement, it can be used to synchronise timing + across different devices. + + \sa SensorTimestamp + + The FrameWallClock control can only be returned in metadata. + ... From patchwork Fri Dec 6 14:27:39 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Plowman X-Patchwork-Id: 22213 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id AA77FBE173 for ; Fri, 6 Dec 2024 14:27:56 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 8987566164; Fri, 6 Dec 2024 15:27:51 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="UTq9lmfL"; dkim-atps=neutral Received: from mail-wm1-x331.google.com (mail-wm1-x331.google.com [IPv6:2a00:1450:4864:20::331]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 2BC7B66159 for ; Fri, 6 Dec 2024 15:27:47 +0100 (CET) Received: by mail-wm1-x331.google.com with SMTP id 5b1f17b1804b1-434a0fd9778so21152865e9.0 for ; Fri, 06 Dec 2024 06:27:47 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; t=1733495266; x=1734100066; darn=lists.libcamera.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=8WZjyOLa+u+y5Ber4pcSec/2+A8y3D3TjO9Bb/8OAx0=; b=UTq9lmfLPGaSnkgn6w7TafDdJsIvWgJetPfTlURHkdp6Y4l/wBbGIdbVcdQrQI2QAx qpvBJgkmeiyx8YnCPgS5y2YwjlbXJfXW5QfICujq64wSLvv3x1PrJ2aAZLitQnBuXr15 CqZ3pS8EJ1JFhTfJv+yIKYKk5620C0ubZh70fx1LBy4Ejri3I2vR/ifl/BUUDl4XXdi8 jleLWplzM44RAlyyyKrq+RRo7HElJqCBRyERLArVn5z85BU/3BEbyyEr1nviyZMiPwqo GkjrzaCKL/bU32QSqi33TZoHH02ucwHmqMX+OhtmnyliNKce4lZUalhYjAjL5lgIlUhE 8ONA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1733495266; x=1734100066; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=8WZjyOLa+u+y5Ber4pcSec/2+A8y3D3TjO9Bb/8OAx0=; b=GhbuNgL2+iCoSMnYDz1D+Qpg9PLtCkDvoepBLKjSKLqiD6el4jGqO4r/XJRNNMOTFW MhCe806rIph576gjLO8aZpgzaiN4XUQ/bfNVpalwPe4znAge2vcbK6rt7sAyynPsx1DB t/GTuGndgzWLqoZqBcx5nQxvSSspUMjWCSrY8JiMHDVRPWKWyQiH0NBhTO7EoNTHuRSm /LZjqLXWI3YNzbx+jTHV+U+vdFjDb0VdcNoXfyskV7eRLPsOQbHKTLfgzaTLAuC7BiPm VrU3hPxm7qIvnvGO+igfsfxHWxFJLsD6v1nUBzVcY+U5O6FpTB2Yog20QwxDSM1sx9se RjhQ== X-Gm-Message-State: AOJu0Yxdu5OZkd4VLGYGFyhse2Y3eZPtGnqGB2htF3JsLGHLIgZix1dF ORSMnLj88LZ4Oh/y95A8vfuQ88pTPOa42iFKyN4zMaW0MxTkO5Nljx42/SHyG9vkZ+dbbW1AUF9 s X-Gm-Gg: ASbGncuqf4oLewveWOxCl7MccmSK94ZZKMhg1lUmJEBYXzpKrBWVW0P8HLFozXNltD1 WKgz3gqO0eTdvLr3rOu0bO6cbOqSSg32xadUThpL9ThJQLewYFybbOmG72v2VcBQjabWiL5z04p X5DI4aVo/YTRPWrlb+17jqqlr5+H8VCwa+J+RwIzB9hr7d3/5eFfA7xXae+nikoGmcGQbWe+kfy As7tOsRWB4k87LS55jZ4Y9EOyhT1bb2+/3wlHHAtMgC2pNe+eDMoQvKsh4UtZOBE+Z+96tz7ys= X-Google-Smtp-Source: AGHT+IGeUUymyuM5VgCWS2WsDoPLK/kQA4ky10VF7TVFC70fyerYZ3AP+FMnIuT6yxeoDorN3WD7jA== X-Received: by 2002:a05:600c:1d85:b0:434:a902:4168 with SMTP id 5b1f17b1804b1-434ddeb8c74mr29804615e9.18.1733495266410; Fri, 06 Dec 2024 06:27:46 -0800 (PST) Received: from localhost.localdomain ([88.202.252.90]) by smtp.gmail.com with ESMTPSA id ffacd0b85a97d-386220b071dsm4608219f8f.101.2024.12.06.06.27.45 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 06 Dec 2024 06:27:46 -0800 (PST) From: David Plowman To: libcamera-devel@lists.libcamera.org Cc: David Plowman Subject: [PATCH 2/5] libcamera: Add ClockRecovery class to generate wallclock timestamps Date: Fri, 6 Dec 2024 14:27:39 +0000 Message-Id: <20241206142742.7931-3-david.plowman@raspberrypi.com> X-Mailer: git-send-email 2.39.5 In-Reply-To: <20241206142742.7931-1-david.plowman@raspberrypi.com> References: <20241206142742.7931-1-david.plowman@raspberrypi.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" The ClockRecovery class takes pairs of timestamps from two different clocks, and models the second ("output") clock from the first ("input") clock. We can use it, in particular, to get a good wallclock estimate for a frame's SensorTimestamp. Signed-off-by: David Plowman --- include/libcamera/internal/clock_recovery.h | 72 +++++++ include/libcamera/internal/meson.build | 1 + src/libcamera/clock_recovery.cpp | 207 ++++++++++++++++++++ src/libcamera/meson.build | 1 + 4 files changed, 281 insertions(+) create mode 100644 include/libcamera/internal/clock_recovery.h create mode 100644 src/libcamera/clock_recovery.cpp diff --git a/include/libcamera/internal/clock_recovery.h b/include/libcamera/internal/clock_recovery.h new file mode 100644 index 00000000..c874574e --- /dev/null +++ b/include/libcamera/internal/clock_recovery.h @@ -0,0 +1,72 @@ +/* SPDX-License-Identifier: BSD-2-Clause */ +/* + * Copyright (C) 2024, Raspberry Pi Ltd + * + * Camera recovery algorithm + */ +#pragma once + +#include + +namespace libcamera { + +class ClockRecovery +{ +public: + ClockRecovery(); + + /* Set configuration parameters. */ + void configure(unsigned int numPts = 100, unsigned int maxJitter = 2000, unsigned int minPts = 10, + unsigned int errorThreshold = 50000); + /* Erase all history and restart the fitting process. */ + void reset(); + + /* + * Add a new input clock / output clock sample, taking the input from the Linux + * CLOCK_BOOTTIME and the output from the CLOCK_REALTIME. + */ + void addSample(); + /* + * Add a new input clock / output clock sample, specifying the clock times exactly. Use this + * when you want to use clocks other than the ones described above. + */ + void addSample(uint64_t input, uint64_t output); + /* Calculate the output clock value for this input. */ + uint64_t getOutput(uint64_t input); + +private: + unsigned int numPts_; /* how many samples contribute to the history */ + unsigned int maxJitter_; /* smooth out any jitter larger than this immediately */ + unsigned int minPts_; /* number of samples below which we treat clocks as 1:1 */ + unsigned int errorThreshold_; /* reset everything when the error exceeds this */ + + unsigned int count_; /* how many samples seen (up to numPts_) */ + uint64_t inputBase_; /* subtract this from all input values, just to make the numbers easier */ + uint64_t outputBase_; /* as above, for the output */ + + uint64_t lastInput_; /* the previous input sample */ + uint64_t lastOutput_; /* the previous output sample */ + + /* + * We do a linear regression of y against x, where: + * x is the value input - inputBase_, and + * y is the value output - outputBase_ - x. + * We additionally subtract x from y so that y "should" be zero, again making the numnbers easier. + */ + double xAve_; /* average x value seen so far */ + double yAve_; /* average y value seen so far */ + double x2Ave_; /* average x^2 value seen so far */ + double xyAve_; /* average x*y value seen so far */ + + /* + * Once we've seen more than minPts_ samples, we recalculate the slope and offset according + * to the linear regression normal equations. + */ + double slope_; /* latest slope value */ + double offset_; /* latest offset value */ + + /* We use this cumulative error to monitor spontaneous system clock updates. */ + double error_; +}; + +} /* namespace libcamera */ diff --git a/include/libcamera/internal/meson.build b/include/libcamera/internal/meson.build index 7d6aa8b7..41500636 100644 --- a/include/libcamera/internal/meson.build +++ b/include/libcamera/internal/meson.build @@ -11,6 +11,7 @@ libcamera_internal_headers = files([ 'camera_manager.h', 'camera_sensor.h', 'camera_sensor_properties.h', + 'clock_recovery.h', 'control_serializer.h', 'control_validator.h', 'converter.h', diff --git a/src/libcamera/clock_recovery.cpp b/src/libcamera/clock_recovery.cpp new file mode 100644 index 00000000..966599ee --- /dev/null +++ b/src/libcamera/clock_recovery.cpp @@ -0,0 +1,207 @@ +/* SPDX-License-Identifier: BSD-2-Clause */ +/* + * Copyright (C) 2024, Raspberry Pi Ltd + * + * Clock recovery algorithm + */ + +#include "libcamera/internal/clock_recovery.h" + +#include + +#include + +/** + * \file clock_recovery.h + * \brief Clock recovery - deriving one clock from another independent clock + */ + +namespace libcamera { + +LOG_DEFINE_CATEGORY(ClockRec) + +/** + * \class ClockRecovery + * \brief Recover an output clock from an input clock + * + * The ClockRecovery class derives an output clock from an input clock, + * modelling the output clock as being linearly related to the input clock. + * For example, we may use it to derive wall clock timestamps from timestamps + * measured by the internal system clock which counts local time since boot. + * + * When pairs of corresponding input and output timestamps are available, + * they should be submitted to the model with addSample(). The model will + * update, and output clock values for known input clock values can be + * obtained using getOutput(). + * + * As a convenience, if the input clock is indeed the time since boot, and the + * output clock represents a real wallclock time, then addSample() can be + * called with no arguments, and a pair of timestamps will be captured at + * that moment. + * + * The configure() function accepts some configuration parameters to control + * the linear fitting process. + */ + +/** + * \brief Construct a ClockRecovery + */ +ClockRecovery::ClockRecovery() +{ + configure(); + reset(); +} + +/** + * \brief Set configuration parameters + * \param[in] numPts The approximate duration for which the state of the model + * is persistent, measured in samples + * \param[in] maxJitter New output samples are clamped to no more than this + * amount of jitter, to prevent sudden swings from having a large effect + * \param[in] minPts The fitted clock model is not used to generate outputs + * until this many samples have been received + * \param[in] errorThreshold If the accumulated differences between input and + * output clocks reaches this amount over a few frames, the model is reset + */ +void ClockRecovery::configure(unsigned int numPts, unsigned int maxJitter, unsigned int minPts, + unsigned int errorThreshold) +{ + LOG(ClockRec, Debug) + << "configure " << numPts << " " << maxJitter << " " << minPts << " " << errorThreshold; + + numPts_ = numPts; + maxJitter_ = maxJitter; + minPts_ = minPts; + errorThreshold_ = errorThreshold; +} + +/** + * \brief Reset the clock recovery model and start again from scratch + */ +void ClockRecovery::reset() +{ + LOG(ClockRec, Debug) << "reset"; + + lastInput_ = 0; + lastOutput_ = 0; + xAve_ = 0; + yAve_ = 0; + x2Ave_ = 0; + xyAve_ = 0; + count_ = 0; + slope_ = 0.0; + offset_ = 0.0; + error_ = 0.0; +} + +/** + * \brief Add a sample point to the clock recovery model, for recovering a wall + * clock value from the internal system time since boot + * + * This is a convenience function to make it easy to derive a wall clock value + * (using the Linux CLOCK_REALTIME) from the time since the system started + * (measured by CLOCK_BOOTTIME). + */ +void ClockRecovery::addSample() +{ + LOG(ClockRec, Debug) << "addSample"; + + struct timespec bootTime; + struct timespec wallTime; + + /* Get boot and wall clocks in microseconds. */ + clock_gettime(CLOCK_BOOTTIME, &bootTime); + clock_gettime(CLOCK_REALTIME, &wallTime); + uint64_t boot = bootTime.tv_sec * 1000000ULL + bootTime.tv_nsec / 1000; + uint64_t wall = wallTime.tv_sec * 1000000ULL + wallTime.tv_nsec / 1000; + + addSample(boot, wall); +} + +/** + * \brief Add a sample point to the clock recovery model, specifying the exact + * input and output clock values + * + * This function should be used for corresponding clocks other than the Linux + * BOOTTIME and REALTIME clocks. + */ +void ClockRecovery::addSample(uint64_t input, uint64_t output) +{ + LOG(ClockRec, Debug) << "addSample " << input << " " << output; + + if (count_ == 0) { + inputBase_ = input; + outputBase_ = output; + } + + /* + * We keep an eye on cumulative drift over the last several frames. If this exceeds a + * threshold, then probably the system clock has been updated and we're going to have to + * reset everything and start over. + */ + if (lastOutput_) { + int64_t inputDiff = getOutput(input) - getOutput(lastInput_); + int64_t outputDiff = output - lastOutput_; + error_ = error_ * 0.95 + (outputDiff - inputDiff); + if (std::abs(error_) > errorThreshold_) { + reset(); + inputBase_ = input; + outputBase_ = output; + } + } + lastInput_ = input; + lastOutput_ = output; + + /* + * Never let the new output value be more than maxJitter_ away from what we would have expected. + * This is just to reduce the effect of sudden large delays in the measured output. + */ + uint64_t expectedOutput = getOutput(input); + output = std::clamp(output, expectedOutput - maxJitter_, expectedOutput + maxJitter_); + + /* + * We use x, y, x^2 and x*y sums to calculate the best fit line. Here we update them by + * pretending we have count_ samples at the previous fit, and now one new one. Gradually + * the effect of the older values gets lost. This is a very simple way of updating the + * fit (there are much more complicated ones!), but it works well enough. Using averages + * instead of sums makes the relative effect of old values and the new sample clearer. + */ + double x = static_cast(input - inputBase_); + double y = static_cast(output - outputBase_) - x; + unsigned int count1 = count_ + 1; + xAve_ = (count_ * xAve_ + x) / count1; + yAve_ = (count_ * yAve_ + y) / count1; + x2Ave_ = (count_ * x2Ave_ + x * x) / count1; + xyAve_ = (count_ * xyAve_ + x * y) / count1; + + /* Don't update slope and offset until we've seen "enough" sample points. */ + if (count_ > minPts_) { + /* These are the standard equations for least squares linear regression. */ + slope_ = (count1 * count1 * xyAve_ - count1 * xAve_ * count1 * yAve_) / + (count1 * count1 * x2Ave_ - count1 * xAve_ * count1 * xAve_); + offset_ = yAve_ - slope_ * xAve_; + } + + /* Don't increase count_ above numPts_, as this controls the long-term amount of the residual fit. */ + if (count1 < numPts_) + count_++; +} + +/** + * \brief Calculate the output clock value according to the model from an input + * clock value + * + * \return Output clock value + */ +uint64_t ClockRecovery::getOutput(uint64_t input) +{ + double x = static_cast(input - inputBase_); + double y = slope_ * x + offset_; + uint64_t output = y + x + outputBase_; + + LOG(ClockRec, Debug) << "getOutput " << input << " " << output; + + return output; +} + +} /* namespace libcamera */ diff --git a/src/libcamera/meson.build b/src/libcamera/meson.build index 57fde8a8..4eaa1c8e 100644 --- a/src/libcamera/meson.build +++ b/src/libcamera/meson.build @@ -21,6 +21,7 @@ libcamera_internal_sources = files([ 'byte_stream_buffer.cpp', 'camera_controls.cpp', 'camera_lens.cpp', + 'clock_recovery.cpp', 'control_serializer.cpp', 'control_validator.cpp', 'converter.cpp', From patchwork Fri Dec 6 14:27:40 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Plowman X-Patchwork-Id: 22214 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id E8252BE173 for ; Fri, 6 Dec 2024 14:27:57 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id CFC3A6616B; Fri, 6 Dec 2024 15:27:52 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="qJOVOanT"; dkim-atps=neutral Received: from mail-wr1-x435.google.com (mail-wr1-x435.google.com [IPv6:2a00:1450:4864:20::435]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id D6964618B3 for ; Fri, 6 Dec 2024 15:27:47 +0100 (CET) Received: by mail-wr1-x435.google.com with SMTP id ffacd0b85a97d-385e3621518so1580564f8f.1 for ; Fri, 06 Dec 2024 06:27:47 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; t=1733495267; x=1734100067; darn=lists.libcamera.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=oAcvVu9/nphoPJeM1gayrL8QAg+lnY0IAKNaVkIKf7A=; b=qJOVOanTCDD+m8M+e3GDldf3NtTRsNcOXJJxH5vTgInNTbyXakbneTTau0jJQJVXUb 7u0Dc4PZ6PsYnXw5Wx5s9YhYe4dEdEcLd+p3SBhUMHcRTM7yr7J+RU0Cc+iGfaHutTN4 1opTccEOXvaoq2geHaq9lKG4kzoaJedeTONu04nyE3dts8FIKVIGRrJp+Nnds9X7DUVt imrz4u6yBbdXrjxgb/x5WOHnkjF09cUvrAka3qHiwiOYYY+eDuzAd66xlW/chp3iXBQ4 p3LdRkSlGYJyaezQHcD61PkbC7XIWa9lz/qKgvJutylrVQDXWPGhZaakfESlSts86J6h Oxgg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1733495267; x=1734100067; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=oAcvVu9/nphoPJeM1gayrL8QAg+lnY0IAKNaVkIKf7A=; b=TSx666v22bdnPVydcV5GUX8S1oeunokVFgXQuVmVFTgwYWUdDIqAnDcKlGcpaxSd0O IvZ8La30isbDdCDQLSyLleNnKmYQRfv5cchP+xXYIE/dk9nLt+kh4GHcm5g8S9GR+EvT YwxCHDoqEzIEero6bpKehm58LQxZKnhc4hk0XSdnn27HMBIt3vfgwe6qK0kd4zfx9aYq Y+EmtZ9byo2U8uRQNkWjOBPWksR/K0YveExR2h3DcC3+cm+fdlJuDXWDw3DA3drGK2LJ WczD5pk6SeXH12DCZIASE0EvXm+jeQQime4fo7HZFBrRYDklRyG+hk85kvrscLiXPwkz oPhA== X-Gm-Message-State: AOJu0Yzc+IRPKUiBPgfESQbfWckNYfvqKPPRH8Rv4GaP2mzGkBbtP/wU qko8jZ0at3UrO4EkEYzWX5vDnfz2jJZKq+e6dWd9AfWxGw9e5V8Cw/NtEe+al8vow/ZZrPQGP8E f X-Gm-Gg: ASbGnct/WZ1rn9yEK2AOlxmqHOvzbnLsha3jV1xQ0nxPMC87dpPmb0ljofnTOZoFELv 93uOTSz3lZSu2d8p9R3bZl3qW6ef1atduvzj1aCFg9YHqGD7rnXweajMg1Ahzeax483HJm7SyIF ZK8anipuum03CxTDQueYP7c3JB6m1+O6UBObtx6qI7oE0HBg6ey7KOFUYlbkW+FVgwlH4IgEVnt d7zrB8HHRZdvnI/Hh1Ea2rip9HLNQtg+JW2P7gPGoMHEL1IYsmG+XrRcGOVxUmzmxIY5JAF5GU= X-Google-Smtp-Source: AGHT+IFRtD17/etU97O2m2ykLDQVmAa7qfsAw76noXUeAIF4ce7P+5BSMmzss4DB32GAfb9qYcfHMQ== X-Received: by 2002:a05:6000:717:b0:385:f4b0:596 with SMTP id ffacd0b85a97d-3862b3632ecmr2536499f8f.18.1733495267051; Fri, 06 Dec 2024 06:27:47 -0800 (PST) Received: from localhost.localdomain ([88.202.252.90]) by smtp.gmail.com with ESMTPSA id ffacd0b85a97d-386220b071dsm4608219f8f.101.2024.12.06.06.27.46 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 06 Dec 2024 06:27:46 -0800 (PST) From: David Plowman To: libcamera-devel@lists.libcamera.org Cc: David Plowman Subject: [PATCH 3/5] libcamera: v4l2: Add wallclock metadata to video devices Date: Fri, 6 Dec 2024 14:27:40 +0000 Message-Id: <20241206142742.7931-4-david.plowman@raspberrypi.com> X-Mailer: git-send-email 2.39.5 In-Reply-To: <20241206142742.7931-1-david.plowman@raspberrypi.com> References: <20241206142742.7931-1-david.plowman@raspberrypi.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" FrameMetadata is extended to include a wallclock timestamp. When a frame is dequeued, we use the ClockRecovery class to generate a good estimate of a wallclock timestamp to correspond to the frame's SensorTimestamp. Pipeline handlers must enable wallclocks for chosen devices by passing an appropriate ClockRecovery object. Signed-off-by: David Plowman --- include/libcamera/framebuffer.h | 1 + include/libcamera/internal/v4l2_videodevice.h | 5 +++ src/libcamera/v4l2_videodevice.cpp | 36 ++++++++++++++++++- 3 files changed, 41 insertions(+), 1 deletion(-) diff --git a/include/libcamera/framebuffer.h b/include/libcamera/framebuffer.h index ff839243..4579d0c6 100644 --- a/include/libcamera/framebuffer.h +++ b/include/libcamera/framebuffer.h @@ -35,6 +35,7 @@ struct FrameMetadata { Status status; unsigned int sequence; uint64_t timestamp; + uint64_t wallClock; Span planes() { return planes_; } Span planes() const { return planes_; } diff --git a/include/libcamera/internal/v4l2_videodevice.h b/include/libcamera/internal/v4l2_videodevice.h index f021c2a0..5327c112 100644 --- a/include/libcamera/internal/v4l2_videodevice.h +++ b/include/libcamera/internal/v4l2_videodevice.h @@ -37,6 +37,7 @@ namespace libcamera { +class ClockRecovery; class EventNotifier; class MediaDevice; class MediaEntity; @@ -232,6 +233,8 @@ public: V4L2PixelFormat toV4L2PixelFormat(const PixelFormat &pixelFormat) const; + void enableWallClock(ClockRecovery *wallClockRecovery); + protected: std::string logPrefix() const override; @@ -290,6 +293,8 @@ private: Timer watchdog_; utils::Duration watchdogDuration_; + + ClockRecovery *wallClockRecovery_; }; class V4L2M2MDevice diff --git a/src/libcamera/v4l2_videodevice.cpp b/src/libcamera/v4l2_videodevice.cpp index a5cf6784..2007dffc 100644 --- a/src/libcamera/v4l2_videodevice.cpp +++ b/src/libcamera/v4l2_videodevice.cpp @@ -26,6 +26,7 @@ #include #include +#include "libcamera/internal/clock_recovery.h" #include "libcamera/internal/formats.h" #include "libcamera/internal/framebuffer.h" #include "libcamera/internal/media_device.h" @@ -534,7 +535,7 @@ std::ostream &operator<<(std::ostream &out, const V4L2DeviceFormat &f) V4L2VideoDevice::V4L2VideoDevice(const std::string &deviceNode) : V4L2Device(deviceNode), formatInfo_(nullptr), cache_(nullptr), fdBufferNotifier_(nullptr), state_(State::Stopped), - watchdogDuration_(0.0) + watchdogDuration_(0.0), wallClockRecovery_(nullptr) { /* * We default to an MMAP based CAPTURE video device, however this will @@ -1865,6 +1866,17 @@ FrameBuffer *V4L2VideoDevice::dequeueBuffer() metadata.timestamp = buf.timestamp.tv_sec * 1000000000ULL + buf.timestamp.tv_usec * 1000ULL; + metadata.wallClock = 0; + if (wallClockRecovery_) { + /* + * Sample the internal (CLOCK_BOOTTIME) and realtime (CLOCK_REALTIME) clocks and + * update the clock recovery model. Then derive the wallclock estimate for the + * frame timestamp. + */ + wallClockRecovery_->addSample(); + metadata.wallClock = wallClockRecovery_->getOutput(metadata.timestamp / 1000); + } + if (V4L2_TYPE_IS_OUTPUT(buf.type)) return buffer; @@ -1965,6 +1977,11 @@ int V4L2VideoDevice::streamOn() return ret; } + if (wallClockRecovery_) { + /* A good moment to sample the clocks to improve the clock recovery model. */ + wallClockRecovery_->addSample(); + } + state_ = State::Streaming; if (watchdogDuration_ && !queuedBuffers_.empty()) watchdog_.start(std::chrono::duration_cast(watchdogDuration_)); @@ -2120,6 +2137,23 @@ V4L2PixelFormat V4L2VideoDevice::toV4L2PixelFormat(const PixelFormat &pixelForma return {}; } +/** + * \brief Enable wall clock timestamps for this device + * \param[in] wallClockRecovery an appropriately configured ClockRecovery, or + * nullptr to disable wall clocks + * + * When buffers are dequeued, wall clock timestamps will be generated that + * correspond to the frame's timestamp, as returned by V4l2. + */ +void V4L2VideoDevice::enableWallClock(ClockRecovery *wallClockRecovery) +{ + wallClockRecovery_ = wallClockRecovery; + + /* Also a reasonable moment to sample the two clocks. */ + if (wallClockRecovery_) + wallClockRecovery_->addSample(); +} + /** * \class V4L2M2MDevice * \brief Memory-to-Memory video device From patchwork Fri Dec 6 14:27:41 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Plowman X-Patchwork-Id: 22215 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id ECD5CBE173 for ; Fri, 6 Dec 2024 14:27:58 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 848F766176; Fri, 6 Dec 2024 15:27:53 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="RRatNyea"; dkim-atps=neutral Received: from mail-wm1-x336.google.com (mail-wm1-x336.google.com [IPv6:2a00:1450:4864:20::336]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 71FC866166 for ; Fri, 6 Dec 2024 15:27:48 +0100 (CET) Received: by mail-wm1-x336.google.com with SMTP id 5b1f17b1804b1-434e3953b65so892865e9.1 for ; Fri, 06 Dec 2024 06:27:48 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; t=1733495268; x=1734100068; darn=lists.libcamera.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=1BggXtC9eS3Pe4g0NtueKve30BwBdwUvqTaJYJVa6/M=; b=RRatNyea6Em4dta/RFj+dngdEc1LRNia//ut4tB6Gj8BaSFXdag3y/aj291mGt4YLK 1m946IpMnLnm+RFuBqIati12kYGBatJKq1fXkaXqFRSXozH3417YrVv7TTucKMdg2/4F 0jC9uYAIIxV2p7jSOlwcyorMhrd2AmbyEwvBvhzOsjSQB27HoSAukRq6baak1vy1OuL9 /LBGnTLqm6qqzOfd0nttSih8fHxRI9NLXDLK5oSOAbJ8bV4XgjJpkAGhly6Si0IYLxv6 R4ioFF6X3hCzDIodLSppfiw6pnMOfuw8QPTV6v4j2zZHJHc4ZUxeaClvSYbzraMXyDpR Rlfw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1733495268; x=1734100068; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=1BggXtC9eS3Pe4g0NtueKve30BwBdwUvqTaJYJVa6/M=; b=QlnPgSwxo00qfZRfhlEgVY41mt/BlzvtKrEygCWR3fAUU7V6YKCCrnHZWqpuQu6Sw9 ce+ZQm8i2jv0gDEOG7gcuEcjl1iRRMtnnp8w17bDAvha66cpvijoLhRbKB1Q1ihSaTRp u9+EpDingTzApIhMduxuEpfHLeF4Rsqdvql0gFU03I3TWvolin2q3DRWu5T7jNm8mkDA GMMe+LSYN0p/HFTJIhWg+5x/g37W8q1C9GnlGmpW3WcuMhwb2Qb+dizGpkkarLIBmw5U u2uQ1I92wD+uvd6JJRgOrPVWOPEQG4KO7YQt1/MMPuMRa6ueFBp0f9cAKGa9+kwMX5ii lriw== X-Gm-Message-State: AOJu0YwWJUC2qMJIQ4aH9Yvz9w2/yw28d9Z4g1Wx9dNW+sRIM4FG6coZ laIdwSsuGVZtesCYmFGVnzS6HwMPeQWSUSdlQQ7pVvugPJuUr/dDDC6bID2bWDx0rClmNwIeEd2 m X-Gm-Gg: ASbGnctQMctdyxFz21QhzGdVXQHZ0uxu+FLBKgN6hAG0cCVG+f5b0Jt3t0yhDEuk4pv M/52fYGC96oF7MBgSWAokUU/wx2y1xSlKCsDm3W1nq+kJ54h4ow34CqqiBd5YGCi2ce4KxtFmEv WcgEAetf2u1QcIOvpCKOi4gZu1P1yFvQOV3+JxL1WfAN0DrDRqykoyoRBGQi35XIWKq3A1DSZiB 5ECVmXNwD3k4yYCQmTT2H5IOjmRfM2hzEeEpUHEg32tnV79FFXtNch5HUx8JViT0MuErCeLRs8= X-Google-Smtp-Source: AGHT+IEXkMvfVT8xbVSxt6RfxVlUst44WgdaWvUFvMVNhgXZDhYeuw2c7MRK0aHrZGrbOa3UQgoQBg== X-Received: by 2002:a05:6000:178a:b0:385:f417:ee46 with SMTP id ffacd0b85a97d-3862b33b786mr2368320f8f.2.1733495267690; Fri, 06 Dec 2024 06:27:47 -0800 (PST) Received: from localhost.localdomain ([88.202.252.90]) by smtp.gmail.com with ESMTPSA id ffacd0b85a97d-386220b071dsm4608219f8f.101.2024.12.06.06.27.47 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 06 Dec 2024 06:27:47 -0800 (PST) From: David Plowman To: libcamera-devel@lists.libcamera.org Cc: David Plowman Subject: [PATCH 4/5] pipeline: rpi: common: Add wallclock timestamp support Date: Fri, 6 Dec 2024 14:27:41 +0000 Message-Id: <20241206142742.7931-5-david.plowman@raspberrypi.com> X-Mailer: git-send-email 2.39.5 In-Reply-To: <20241206142742.7931-1-david.plowman@raspberrypi.com> References: <20241206142742.7931-1-david.plowman@raspberrypi.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" A ClockRecovery object is added for derived classes to use, and wallclock timetamps are copied into the request metadata for applications. Signed-off-by: David Plowman --- src/libcamera/pipeline/rpi/common/pipeline_base.cpp | 2 ++ src/libcamera/pipeline/rpi/common/pipeline_base.h | 3 +++ 2 files changed, 5 insertions(+) diff --git a/src/libcamera/pipeline/rpi/common/pipeline_base.cpp b/src/libcamera/pipeline/rpi/common/pipeline_base.cpp index 6f278b29..34971973 100644 --- a/src/libcamera/pipeline/rpi/common/pipeline_base.cpp +++ b/src/libcamera/pipeline/rpi/common/pipeline_base.cpp @@ -1509,6 +1509,8 @@ void CameraData::fillRequestMetadata(const ControlList &bufferControls, Request { request->metadata().set(controls::SensorTimestamp, bufferControls.get(controls::SensorTimestamp).value_or(0)); + request->metadata().set(controls::FrameWallClock, + bufferControls.get(controls::FrameWallClock).value_or(0)); if (cropParams_.size()) { std::vector crops; diff --git a/src/libcamera/pipeline/rpi/common/pipeline_base.h b/src/libcamera/pipeline/rpi/common/pipeline_base.h index aae0c2f3..9aa591d6 100644 --- a/src/libcamera/pipeline/rpi/common/pipeline_base.h +++ b/src/libcamera/pipeline/rpi/common/pipeline_base.h @@ -20,6 +20,7 @@ #include "libcamera/internal/bayer_format.h" #include "libcamera/internal/camera.h" #include "libcamera/internal/camera_sensor.h" +#include "libcamera/internal/clock_recovery.h" #include "libcamera/internal/framebuffer.h" #include "libcamera/internal/media_device.h" #include "libcamera/internal/media_object.h" @@ -237,6 +238,8 @@ protected: virtual int platformRegister(std::unique_ptr &cameraData, MediaDevice *unicam, MediaDevice *isp) = 0; + ClockRecovery wallClockRecoery_; + private: CameraData *cameraData(Camera *camera) { From patchwork Fri Dec 6 14:27:42 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Plowman X-Patchwork-Id: 22216 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id B7DF2BE173 for ; Fri, 6 Dec 2024 14:27:59 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id A737F67E19; Fri, 6 Dec 2024 15:27:54 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="JrGxnO9r"; dkim-atps=neutral Received: from mail-wr1-x433.google.com (mail-wr1-x433.google.com [IPv6:2a00:1450:4864:20::433]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 145DF66159 for ; Fri, 6 Dec 2024 15:27:49 +0100 (CET) Received: by mail-wr1-x433.google.com with SMTP id ffacd0b85a97d-3862f32a33eso214222f8f.3 for ; Fri, 06 Dec 2024 06:27:49 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; t=1733495268; x=1734100068; darn=lists.libcamera.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=RwXAjeZBjmTaRhBl0Lt3tdMya7DG8/0UHFFSELomyPA=; b=JrGxnO9r8vRtVXtg38yr2OjKSNeIt3ZJvFUJdmE6aIy9KPd3v/f/OjVvns90uwcgvg D7Eub6ePOrNK5/EnLhULASxLLsLeqWCNIfifbLfs+IKgFwIbYf+wQfxEea3FF1udn0+A Z3hmEgYXj61kiLHSEI2q+o8JKBKg3vuLCWiXm7ru/SDSW8KThuyUW6Z2fSdHFZIRZ07i 46tQWpSI4dGalTeLTMctFlNeGKifZvS7zQxH3gn7A+V7QG9qYOlOO+GETb1zJG0OTd9P khRyuEL1Kg4fLA+vDbfrEmKU4QL7eyAdkbdcok7Sp/DQ+tTUtpnQKy8Shj9XbiKoxbtH 1wjg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1733495268; x=1734100068; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=RwXAjeZBjmTaRhBl0Lt3tdMya7DG8/0UHFFSELomyPA=; b=WSdg6iZXVy3J9b7TjTLXld8yeHLeneAWWjsGmg7YwE/TaGhDUAuuezsQ0rgqPNW5Zu yW8aEFpr37ptB9v27a8gKJ33s5F/S232+CxN4kzd/SU1dQxdmHKsDYGKw731FHNuKEFB gC3iUyPGbR2gMQC5F1wWqQSKVrRNMiWKJHYvxxoi+2HNZVNOztfUqZgIk6e4uGx6alfr vzU8t3LAkXEJ3hHnSv6GV2ldKjstBFwj9hEvxWyf+x8Kq2ttFHBqxMRMM2utqHfxMGy9 svc1DHaE6BrmtX6WopYf9g7xD3e11EUiSGwSyzlGCouTuUMGBxTW/I55kqzB0W2Du5kN 4l5w== X-Gm-Message-State: AOJu0YwmzAAJLBb5kO44DdU3k4saGI1Rjr9IqgyvG9LXZUg46qmbZIiY m2H8MqIZDDW0eSO6zkyZKSz3WP8vf4g6uLG8l3c75/XAqfChEuCwNf92rWz/l7nCMD8JvakACND y X-Gm-Gg: ASbGnctY0Zo2fdh8YSMeOnr1QVxkqMMQy7VKPDVcrN7QBq/C28N911f3DpeyBY7GR6C U72ZquUAmm0lP3X4tspOabgXx9D7ngCZm/5sWKLRPKRrttlQEoGjqQiyH1Gt9vc/BgUQKvePxTu ZnqeSAZq6ZcE3TOLUMRgZbYep4Pakd7cqET6MZas3BZXJd9RHQ5WkVSRAqd321jwNeS+uIUCMdR 3miHTzm92Oe4Q4iI/NFYY98KzIiiLh5/SUhc1uihnkWwS3Ar78V3dgShHI7x1PSTBtKpU+QTLc= X-Google-Smtp-Source: AGHT+IH8m0qAB5srEnDm2xrQP0MnazAqKbyVam/9EjJQdOZ2r57U+05ysS8jVb3A9UpRysrUN/UL9w== X-Received: by 2002:a05:6000:1fa4:b0:385:f677:8594 with SMTP id ffacd0b85a97d-3862b3e5d6dmr2562804f8f.43.1733495268327; Fri, 06 Dec 2024 06:27:48 -0800 (PST) Received: from localhost.localdomain ([88.202.252.90]) by smtp.gmail.com with ESMTPSA id ffacd0b85a97d-386220b071dsm4608219f8f.101.2024.12.06.06.27.47 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 06 Dec 2024 06:27:48 -0800 (PST) From: David Plowman To: libcamera-devel@lists.libcamera.org Cc: David Plowman Subject: [PATCH 5/5] pipeline: rpi: vc4: Add wallclock timestamp support Date: Fri, 6 Dec 2024 14:27:42 +0000 Message-Id: <20241206142742.7931-6-david.plowman@raspberrypi.com> X-Mailer: git-send-email 2.39.5 In-Reply-To: <20241206142742.7931-1-david.plowman@raspberrypi.com> References: <20241206142742.7931-1-david.plowman@raspberrypi.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Wallclocks are enabled for the unicam output, and the values returned in frame metadata are made available to the base pipeline handler class. Signed-off-by: David Plowman --- src/libcamera/pipeline/rpi/vc4/vc4.cpp | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/src/libcamera/pipeline/rpi/vc4/vc4.cpp b/src/libcamera/pipeline/rpi/vc4/vc4.cpp index fd8d84b1..eb98c56e 100644 --- a/src/libcamera/pipeline/rpi/vc4/vc4.cpp +++ b/src/libcamera/pipeline/rpi/vc4/vc4.cpp @@ -398,6 +398,9 @@ int PipelineHandlerVc4::platformRegister(std::unique_ptr &camer Camera::create(std::move(cameraData), id, streams); PipelineHandler::registerCamera(std::move(camera)); + /* Enable wall clock timestamps for the unicam output. */ + data->unicam_[Unicam::Image].dev()->enableWallClock(&wallClockRecoery_); + LOG(RPI, Info) << "Registered camera " << id << " to Unicam device " << unicam->deviceNode() << " and ISP device " << isp->deviceNode(); @@ -784,6 +787,7 @@ void Vc4CameraData::unicamBufferDequeue(FrameBuffer *buffer) * as it does not receive the FrameBuffer object. */ ctrl.set(controls::SensorTimestamp, buffer->metadata().timestamp); + ctrl.set(controls::FrameWallClock, buffer->metadata().wallClock); bayerQueue_.push({ buffer, std::move(ctrl), delayContext }); } else { embeddedQueue_.push(buffer);