From patchwork Fri Dec 6 14:27:37 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Plowman X-Patchwork-Id: 22211 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 982D2BE173 for ; Fri, 6 Dec 2024 14:27:50 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id BCEAB6616B; Fri, 6 Dec 2024 15:27:48 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="gWkV+uUt"; dkim-atps=neutral Received: from mail-wm1-x333.google.com (mail-wm1-x333.google.com [IPv6:2a00:1450:4864:20::333]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 1CE6F618B3 for ; Fri, 6 Dec 2024 15:27:46 +0100 (CET) Received: by mail-wm1-x333.google.com with SMTP id 5b1f17b1804b1-434acf1f9abso20953565e9.2 for ; Fri, 06 Dec 2024 06:27:46 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; t=1733495265; x=1734100065; darn=lists.libcamera.org; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:from:to:cc:subject:date:message-id:reply-to; bh=Ym57vKKbuIG9G5jAdrl1HOlEgNSspU9Ui0lCCQ1OTRE=; b=gWkV+uUtrbceLVR+2ecccRuc4NU6VWYrXeceexx2bogq2fmUlSZxK84By5N9DJ15iT 2sFC5lVRqUnvPHS7uM6KDEkyfuKHADu+do0vJXoZ2Q18yYJTVnYL5ZGm/JJmfXYFqULY 4+qX5O10CN159QwVwyiKnp6GMkaZEDM7ON29QAoTXMlo+Hi/g0T6E0TtbzGXpn6HMVLG e4NW4SVZe+VRZaIMblbjOm6sL0YXlPMJ+XqmT760l8nAeuuCAO4Sqc0NFoU6vOhPb3rO kIHsx12tgZSoI/LqpSE5x34NXqYH5wlp012cCCYL4k6pp44oeuX3EsTPYbPd1BYtUTEZ K9/A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1733495265; x=1734100065; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=Ym57vKKbuIG9G5jAdrl1HOlEgNSspU9Ui0lCCQ1OTRE=; b=a9r7tv9Tifq1ufBOWfsdE9WaOlvM9eGM0t7jWJKa3AzUBy5P+zUbEAe8/rFCpne2Ls gUgMochY+c9uitjImD5zazhPSbG8qLC6qhWQiDmWj5xwWAnLmIYCDViuIRGVCziwHAdb +ZNToqfL03l+nfzSSjMLUm0NWn2nyp3yIqFbqgc9/AYoWNSMKR2f8vRr3wY+mS79QwAe e1NGuxqWIy8jV9sUJQKN9QlBC5K5Eq31uaY79r1c/GwT0iR28FM7e0D8pV441YoZ5hUC t4x+BhVTqVIxjsXXAaj+281j05UtB6F4tQDBOc6RSesQXJF+FyEe4TYRgXZLlw4+6AS5 ckrg== X-Gm-Message-State: AOJu0Yw380jiRvIBKtMUN4dDUPwIZ3enMtAnlP0PJZoX5QODBf7/5cux 2Gomp1H9VotQTUW39JPTl6xPE/4cK/9KS+gnxfakOzBnPXpr5TU/0NZaCkUiXfcTbO2rh7JyyrR 6 X-Gm-Gg: ASbGncuIWCs/xEBJD/n7YEKBMIV26qs8pAB2tQ1ZWcgHUlbUBr1NI/gGj/wFD5iDfn0 O39P2+FLvfggbOzBVyfn+UmiMH2zna20lVCmIfl/tjbLpkUFzdayTNcTTg3NtEIHYdroSue00Yd Gd9lxwVEvMPLResjHkAjTFI+Bb70UO55sRovNpQJCqchue0HrP5//CdghTwc74JqT81itofsK+j 3Ic8Bcu/oB7uMjcJDJkeNrSQ3R66gClFOP1exY7AKgVmlpKXaE5lBRjEsaT+F7adFImnhCDjSo= X-Google-Smtp-Source: AGHT+IFicjW+emi2tS0wywbsLr2u5FjinQ2DkFV0cbBEyIunl1nEL0Ennwfw7IzSd3A9hd74mnuJJg== X-Received: by 2002:a05:6000:784:b0:385:f220:f789 with SMTP id ffacd0b85a97d-3862b3f50fbmr2377573f8f.48.1733495265296; Fri, 06 Dec 2024 06:27:45 -0800 (PST) Received: from localhost.localdomain ([88.202.252.90]) by smtp.gmail.com with ESMTPSA id ffacd0b85a97d-386220b071dsm4608219f8f.101.2024.12.06.06.27.44 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 06 Dec 2024 06:27:44 -0800 (PST) From: David Plowman To: libcamera-devel@lists.libcamera.org Cc: David Plowman Subject: [PATCH 0/5] Implement wallclock timestamps for frames Date: Fri, 6 Dec 2024 14:27:37 +0000 Message-Id: <20241206142742.7931-1-david.plowman@raspberrypi.com> X-Mailer: git-send-email 2.39.5 MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Hi everyone Here's a version that incorporates the various comments and decisions from yesterday's useful discussions. So thanks to everyone for taking the time yesterday! To summarise what has changed: * It's been simplified by sampling both clocks in dequeueBuffer. It works better like this too, as there's less chance of context switches intervening. (I've also provided an option to have the ClockRecovery class sample the timestamps for you, for convenience.) * I've stopped talking about "steady clocks" and "system clocks" in the C++ fashion. It doesn't seem 100% explicit exactly what these are, so I've gone over to Linux CLOCK_BOOTTIME and CLOCK_REALTIME instead (which do appear to be equivalent, at least). * I've removed the raw wallclock entirely. It's not really useful on its own, because of the other changes, and I couldn't really decide what to do. I've left some logging in the clock recovery code, though, so at least there's that. * Pipeline handlers have to enable wallclocks for a device, by passing in a ClockRecovery (which they could tweak, if they don't like the default configuration). * There's a near-duplicate of the final patch for the Pi 5 too. Looking forward to being able to submit that!! But lots still to talk about, I'm sure - looking forward to it! Thanks David David Plowman (5): controls: Add FrameWallClock control libcamera: Add ClockRecovery class to generate wallclock timestamps libcamera: v4l2: Add wallclock metadata to video devices pipeline: rpi: common: Add wallclock timestamp support pipeline: rpi: vc4: Add wallclock timestamp support include/libcamera/framebuffer.h | 1 + include/libcamera/internal/clock_recovery.h | 72 ++++++ include/libcamera/internal/meson.build | 1 + include/libcamera/internal/v4l2_videodevice.h | 5 + src/libcamera/clock_recovery.cpp | 207 ++++++++++++++++++ src/libcamera/control_ids_core.yaml | 14 ++ src/libcamera/meson.build | 1 + .../pipeline/rpi/common/pipeline_base.cpp | 2 + .../pipeline/rpi/common/pipeline_base.h | 3 + src/libcamera/pipeline/rpi/vc4/vc4.cpp | 4 + src/libcamera/v4l2_videodevice.cpp | 36 ++- 11 files changed, 345 insertions(+), 1 deletion(-) create mode 100644 include/libcamera/internal/clock_recovery.h create mode 100644 src/libcamera/clock_recovery.cpp