{"id":22687,"url":"https://patchwork.libcamera.org/api/1.1/patches/22687/?format=json","web_url":"https://patchwork.libcamera.org/patch/22687/","project":{"id":1,"url":"https://patchwork.libcamera.org/api/1.1/projects/1/?format=json","name":"libcamera","link_name":"libcamera","list_id":"libcamera_core","list_email":"libcamera-devel@lists.libcamera.org","web_url":"","scm_url":"","webscm_url":""},"msgid":"<20250122145349.7220-7-david.plowman@raspberrypi.com>","date":"2025-01-22T14:53:48","name":"[v4,6/7] ipa: rpi: sync: Add an implementation of the camera sync algorithm","commit_ref":null,"pull_url":null,"state":"new","archived":false,"hash":"15ac0c4aaf04e671f6b05c2337213f9832a6a6f6","submitter":{"id":42,"url":"https://patchwork.libcamera.org/api/1.1/people/42/?format=json","name":"David Plowman","email":"david.plowman@raspberrypi.com"},"delegate":null,"mbox":"https://patchwork.libcamera.org/patch/22687/mbox/","series":[{"id":4964,"url":"https://patchwork.libcamera.org/api/1.1/series/4964/?format=json","web_url":"https://patchwork.libcamera.org/project/libcamera/list/?series=4964","date":"2025-01-22T14:53:47","name":"Camera synchronisation","version":4,"mbox":"https://patchwork.libcamera.org/series/4964/mbox/"}],"comments":"https://patchwork.libcamera.org/api/patches/22687/comments/","check":"pending","checks":"https://patchwork.libcamera.org/api/patches/22687/checks/","tags":{},"headers":{"Return-Path":"","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":["parsemail@patchwork.libcamera.org","kbingham@ideasonboard.com"],"Received":["from perceval.ideasonboard.com (perceval.ideasonboard.com\n\t[213.167.242.64])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id 09E9DBD808\n\tfor ;\n\tThu, 30 Jan 2025 11:53:04 +0000 (UTC)","from pendragon.ideasonboard.com\n\t(cpc89244-aztw30-2-0-cust6594.18-1.cable.virginm.net\n\t[86.31.185.195])\n\tby perceval.ideasonboard.com (Postfix) with ESMTPSA id 8B0AB886\n\tfor ;\n\tThu, 30 Jan 2025 12:51:55 +0100 (CET)","from perceval.ideasonboard.com\n\tby perceval.ideasonboard.com with LMTP id 2AU9DVUGkWe3RjQA4E0KoQ\n\t(envelope-from )\n\tfor ; Wed, 22 Jan 2025 15:53:09 +0100","from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\tby perceval.ideasonboard.com (Postfix) with ESMTPS\n\tid 0B0CC1081;\tWed, 22 Jan 2025 15:53:09 +0100 (CET)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id 8ED4C6856A;\n\tWed, 22 Jan 2025 15:54:10 +0100 (CET)","from mail-wm1-x332.google.com (mail-wm1-x332.google.com\n\t[IPv6:2a00:1450:4864:20::332])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id 9D6C06855F\n\tfor ;\n\tWed, 22 Jan 2025 15:53:57 +0100 (CET)","by mail-wm1-x332.google.com with SMTP id\n\t5b1f17b1804b1-4361f664af5so80664165e9.1\n\tfor ;\n\tWed, 22 Jan 2025 06:53:57 -0800 (PST)","from raspberrypi.pitowers.org\n\t([2a00:1098:3142:1f:ffc9:aff6:7f7f:893b])\n\tby smtp.gmail.com with ESMTPSA id\n\t5b1f17b1804b1-438b66dc08bsm11551395e9.37.2025.01.22.06.53.56\n\t(version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);\n\tWed, 22 Jan 2025 06:53:56 -0800 (PST)"],"Authentication-Results":["perceval.ideasonboard.com;\n\tdkim=permerror header.d=raspberrypi.com header.i=@raspberrypi.com\n\theader.a=rsa-sha1 header.s=google header.b=OxUYv8dI; \n\tdkim-atps=neutral","lancelot.ideasonboard.com; dkim=pass (2048-bit key; \n\tunprotected) header.d=raspberrypi.com\n\theader.i=@raspberrypi.com\n\theader.b=\"OxUYv8dI\";\tdkim-atps=neutral"],"DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed;\n\td=raspberrypi.com; s=google; t=1737557637; x=1738162437;\n\tdarn=lists.libcamera.org; \n\th=content-transfer-encoding:mime-version:references:in-reply-to\n\t:message-id:date:subject:cc:to:from:from:to:cc:subject:date\n\t:message-id:reply-to;\n\tbh=FDvnN7ryORqQHcoAh5tIXt4yh5I/D3MEokFNcVWq958=;\n\tb=OxUYv8dIe59obmcvhMfm6epzWw7rYCiwo4UKK20Bmu8mdwPHxt3+W/E86N/WYIQbRb\n\tMTRfGpYx39reSGL5YZQQ6bjCLJcDF6lLdcefr8a9SECLVR/NgOenhd96ByPdzPxLpHxg\n\tSh6gB1OEXkJ+OrBGtUnyKHB9+Su0SOb2B6N+NAyKJWWbBM6p4Oxto11qQh48cIYGfQKK\n\tGgmOkmSWwySHYnZv9d+4KGEqSBiVS693Cub2kmwfhkIxOJ3OEqAH8tZ7cS+xYakv8Ch0\n\tfwaGLjwChgIhKspcf7KYFnU8jJR2/NX/AkfyYVvnKDF8oJElzoGTfhd1kQXH6aLGZheM\n\tCfPQ==","X-Google-DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed;\n\td=1e100.net; s=20230601; t=1737557637; x=1738162437;\n\th=content-transfer-encoding:mime-version:references:in-reply-to\n\t:message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc\n\t:subject:date:message-id:reply-to;\n\tbh=FDvnN7ryORqQHcoAh5tIXt4yh5I/D3MEokFNcVWq958=;\n\tb=Qh00Y/xUl5W5BJFfvHxgft/ALp3O5LVmW/LXey6NDPCRm/MMrqbn2FfoNnFm/13mlV\n\t63NkWDAz/4AglAMbiK1NBQoCkBZl4I+9kdSqpjRLrshgfhCGre01hfrL1Ni3HhR8SVwY\n\twAVRlKHPoWOBJHXXD+fgCHze98OPRPc0HE1OG0vD707syWcWurhC04DOJ7T0J6w1y2rB\n\t+cP9HIoZy4iz9n3SD9jpCd7BRvnaqkZ6mGhWvaMyyp7/7hhN40KhCGSdpCCuw7F+77AE\n\tf0DTZ0bHq2lk7FP7veMJkuf2Vl74lKn9i1Duf7emD4CAqNrC29MOYUbS1Ws+zvTFvIqr\n\tDG6A==","X-Gm-Message-State":"AOJu0YyZl7C8pQThPAzazB34715zjZoAz/3R/wvMjshHq0PUKa1KQ1Hm\n\tuBDFkKXpRiD5Ura3P1FFA/ytiDqdTJWImEf/d6L2hR/AioQhJk/Z4cja4vFOUzl/5D5D9u04nuo\n\tb","X-Gm-Gg":"ASbGnctWS5ixcS45lAoah1zRj+n37+MyfboIFYnptdJZVEtn7VO01um+LvPZ2ryCIs4\n\tqe9v1VWjrUR8qzZPcAVZC37wCNNX9vaJrj1Gstyvp7eFj2mvIx2CxFD5V8EF0qO3U+Nrxwvl6xC\n\thkvUhOaHlLVnEurWECsq7wr+P849Fysyr80luuwWLgtTT4A6vMG6Y/jLndv+XfF+i/lW9PskmOO\n\to+t7Kc35NcbDVE36W+Uq7Ydd/MTpCuGV8cR902wwm8g3i186OCz9J1gPFsU4+uCaUzbaUSx1lfY\n\t76OR5BJn+rwvCcYocRQF03PpTw==","X-Google-Smtp-Source":"=?utf-8?q?AGHT+IHAr4swNtEMo57x1rOuUCjo0oK5q7/nE2ALz?=\n\t=?utf-8?q?JyesrRVQu159QuWxGSD/Pv8XjUUly/1aOQewg=3D=3D?=","X-Received":"by 2002:a05:600c:3b02:b0:434:a781:f5d5 with SMTP id\n\t5b1f17b1804b1-4389143ba53mr213022325e9.30.1737557636728; \n\tWed, 22 Jan 2025 06:53:56 -0800 (PST)","From":"David Plowman ","To":"libcamera-devel@lists.libcamera.org","Cc":"David Plowman ,\n\tArsen Mikovic ,\n\tNaushir Patuck ","Subject":"[PATCH v4 6/7] ipa: rpi: sync: Add an implementation of the camera\n\tsync algorithm","Date":"Wed, 22 Jan 2025 14:53:48 +0000","Message-Id":"<20250122145349.7220-7-david.plowman@raspberrypi.com>","X-Mailer":"git-send-email 2.39.5","In-Reply-To":"<20250122145349.7220-1-david.plowman@raspberrypi.com>","References":"<20250122145349.7220-1-david.plowman@raspberrypi.com>","MIME-Version":"1.0","Content-Transfer-Encoding":"8bit","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" ","X-TUID":"aDncQKUh8YKg","Resent-From":"Kieran Bingham ","Resent-To":"parsemail@patchwork.libcamera.org"},"content":"In this implementation, the server sends data packets out onto the\nnetwork every 30 frames or so.\n\nClients listening for this packet will send frame length deltas back\nto the pipeline handler to match the synchronisation of the server.\n\nWe use wallclock timestamps, passed to us from the pipeline handler,\nthat have been de-jittered appropriately, meaning that the\nsynchronisation will actually work across networked devices.\n\nWhen the server's advertised \"ready time\" is reached, both client and\nserver will signal this through metadata back to their respective\ncontrolling applications.\n\nSigned-off-by: David Plowman \nSigned-off-by: Arsen Mikovic \nSigned-off-by: Naushir Patuck \n---\n src/ipa/rpi/controller/meson.build | 1 +\n src/ipa/rpi/controller/rpi/sync.cpp | 329 ++++++++++++++++++++++++++++\n src/ipa/rpi/controller/rpi/sync.h | 68 ++++++\n 3 files changed, 398 insertions(+)\n create mode 100644 src/ipa/rpi/controller/rpi/sync.cpp\n create mode 100644 src/ipa/rpi/controller/rpi/sync.h","diff":"diff --git a/src/ipa/rpi/controller/meson.build b/src/ipa/rpi/controller/meson.build\nindex 74b74888..dde4ac12 100644\n--- a/src/ipa/rpi/controller/meson.build\n+++ b/src/ipa/rpi/controller/meson.build\n@@ -23,6 +23,7 @@ rpi_ipa_controller_sources = files([\n 'rpi/saturation.cpp',\n 'rpi/sdn.cpp',\n 'rpi/sharpen.cpp',\n+ 'rpi/sync.cpp',\n 'rpi/tonemap.cpp',\n ])\n \ndiff --git a/src/ipa/rpi/controller/rpi/sync.cpp b/src/ipa/rpi/controller/rpi/sync.cpp\nnew file mode 100644\nindex 00000000..39bf5de7\n--- /dev/null\n+++ b/src/ipa/rpi/controller/rpi/sync.cpp\n@@ -0,0 +1,329 @@\n+/* SPDX-License-Identifier: BSD-2-Clause */\n+/*\n+ * Copyright (C) 2024, Raspberry Pi Ltd\n+ *\n+ * sync.cpp - sync algorithm\n+ */\n+#include \"sync.h\"\n+\n+#include \n+#include \n+#include \n+#include \n+#include \n+\n+#include \n+\n+#include \n+\n+#include \"sync_status.h\"\n+\n+using namespace std;\n+using namespace std::chrono_literals;\n+using namespace RPiController;\n+using namespace libcamera;\n+\n+LOG_DEFINE_CATEGORY(RPiSync)\n+\n+#define NAME \"rpi.sync\"\n+\n+Sync::Sync(Controller *controller)\n+\t: SyncAlgorithm(controller), mode_(Mode::Off), socket_(-1), frameDuration_(0s), frameCount_(0)\n+{\n+}\n+\n+Sync::~Sync()\n+{\n+\tif (socket_ >= 0)\n+\t\tclose(socket_);\n+}\n+\n+char const *Sync::name() const\n+{\n+\treturn NAME;\n+}\n+\n+/* This reads from json file and intitiaises server and client */\n+int Sync::read(const libcamera::YamlObject ¶ms)\n+{\n+\tconst char *kDefaultGroup = \"239.255.255.250\";\n+\tconstexpr unsigned int kDefaultPort = 10000;\n+\tconstexpr unsigned int kDefaultSyncPeriod = 30;\n+\tconstexpr unsigned int kDefaultReadyFrame = 100;\n+\tconstexpr unsigned int kDefaultMinAdjustment = 50;\n+\n+\t/* Socket on which to communicate. */\n+\tgroup_ = params[\"group\"].get(kDefaultGroup);\n+\tport_ = params[\"port\"].get(kDefaultPort);\n+\t/* Send a sync message every this many frames. */\n+\tsyncPeriod_ = params[\"sync_period\"].get(kDefaultSyncPeriod);\n+\t/* Application will be told we're ready after this many frames. */\n+\treadyFrame_ = params[\"ready_frame\"].get(kDefaultReadyFrame);\n+\t/* Don't change client frame length unless the change exceeds this amount (microseconds). */\n+\tminAdjustment_ = params[\"min_adjustment\"].get(kDefaultMinAdjustment);\n+\n+\treturn 0;\n+}\n+\n+void Sync::initialiseSocket()\n+{\n+\tsocket_ = socket(AF_INET, SOCK_DGRAM, 0);\n+\tif (socket_ < 0) {\n+\t\tLOG(RPiSync, Error) << \"Unable to create socket\";\n+\t\treturn;\n+\t}\n+\n+\tmemset(&addr_, 0, sizeof(addr_));\n+\taddr_.sin_family = AF_INET;\n+\taddr_.sin_addr.s_addr = mode_ == Mode::Client ? htonl(INADDR_ANY) : inet_addr(group_.c_str());\n+\taddr_.sin_port = htons(port_);\n+\n+\tif (mode_ == Mode::Client) {\n+\t\t/* Set to non-blocking. */\n+\t\tint flags = fcntl(socket_, F_GETFL, 0);\n+\t\tfcntl(socket_, F_SETFL, flags | O_NONBLOCK);\n+\n+\t\tunsigned int en = 1;\n+\t\tif (setsockopt(socket_, SOL_SOCKET, SO_REUSEADDR, &en, sizeof(en)) < 0) {\n+\t\t\tLOG(RPiSync, Error) << \"Unable to set socket options\";\n+\t\t\tgoto err;\n+\t\t}\n+\n+\t\tstruct ip_mreq mreq {};\n+\t\tmreq.imr_multiaddr.s_addr = inet_addr(group_.c_str());\n+\t\tmreq.imr_interface.s_addr = htonl(INADDR_ANY);\n+\t\tif (setsockopt(socket_, IPPROTO_IP, IP_ADD_MEMBERSHIP, &mreq, sizeof(mreq)) < 0) {\n+\t\t\tLOG(RPiSync, Error) << \"Unable to set socket options\";\n+\t\t\tgoto err;\n+\t\t}\n+\n+\t\tif (bind(socket_, (struct sockaddr *)&addr_, sizeof(addr_)) < 0) {\n+\t\t\tLOG(RPiSync, Error) << \"Unable to bind client socket\";\n+\t\t\tgoto err;\n+\t\t}\n+\t}\n+\n+\treturn;\n+\n+err:\n+\tclose(socket_);\n+\tsocket_ = -1;\n+}\n+\n+void Sync::switchMode([[maybe_unused]] CameraMode const &cameraMode, [[maybe_unused]] Metadata *metadata)\n+{\n+\t/*\n+\t * A mode switch means the camera has stopped, so synchronisation will be lost.\n+\t * Reset all the internal state so that we start over.\n+\t */\n+\treset();\n+}\n+\n+/*\n+ * Camera sync algorithm.\n+ * Server - there is a single server that sends framerate timing information over the network to any\n+ * clients that are listening. It also signals when it will send a \"everything is synchronised, now go\"\n+ * message back to the algorithm.\n+ * Client - there may be many clients, either on the same Pi or different ones. They match their\n+ * framerates to the server, and indicate when to \"go\" at the same instant as the server.\n+ */\n+void Sync::process([[maybe_unused]] StatisticsPtr &stats, Metadata *imageMetadata)\n+{\n+\tSyncPayload payload;\n+\tSyncParams local{};\n+\tSyncStatus status{};\n+\tbool timerKnown = true;\n+\n+\tif (mode_ == Mode::Off)\n+\t\treturn;\n+\n+\tif (!frameDuration_) {\n+\t\tLOG(RPiSync, Error) << \"Sync frame duration not set!\";\n+\t\treturn;\n+\t}\n+\n+\tif (socket_ < 0) {\n+\t\tinitialiseSocket();\n+\n+\t\tif (socket_ < 0)\n+\t\t\treturn;\n+\n+\t\t/*\n+\t\t * For the client, flush anything in the socket. It might be stale from a previous sync run,\n+\t\t * or we might get another packet in a frame to two before the adjustment caused by this (old)\n+\t\t * packet, although correct, had taken effect. So this keeps things simpler.\n+\t\t */\n+\t\tif (mode_ == Mode::Client) {\n+\t\t\tsocklen_t addrlen = sizeof(addr_);\n+\t\t\tint ret = 0;\n+\t\t\twhile (ret >= 0)\n+\t\t\t\tret = recvfrom(socket_, &payload, sizeof(payload), 0, (struct sockaddr *)&addr_, &addrlen);\n+\t\t}\n+\t}\n+\n+\timageMetadata->get(\"sync.params\", local);\n+\n+\t/* The wallclock has already been de-jittered for us. */\n+\tuint64_t wallClockFrameTimestamp = local.wallClock;\n+\n+\t/*\n+\t * This is the headline frame duration in microseconds as programmed into the sensor. Strictly,\n+\t * the sensor might not quite match the system clock, but this shouldn't matter for the calculations\n+\t * we'll do with it, unless it's a very very long way out!\n+\t */\n+\tuint32_t frameDuration = frameDuration_.get();\n+\n+\t/* Timestamps tell us if we've dropped any frames, but we still want to count them. */\n+\tint droppedFrames = 0;\n+\tif (frameCount_) {\n+\t\t/*\n+\t\t * Round down here, because frameCount_ gets incremented at the end of the function. Also\n+\t\t * ensure droppedFrames can't go negative. It shouldn't, but things would go badly wrong\n+\t\t * if it did.\n+\t\t */\n+\t\twallClockFrameTimestamp = std::max(wallClockFrameTimestamp, lastWallClockFrameTimestamp_ + frameDuration / 2);\n+\t\tdroppedFrames = (wallClockFrameTimestamp - lastWallClockFrameTimestamp_ - frameDuration / 2) / frameDuration;\n+\t\tframeCount_ += droppedFrames;\n+\t}\n+\n+\tif (mode_ == Mode::Server) {\n+\t\t/*\n+\t\t * Server sends a packet every syncPeriod_ frames, or as soon after as possible (if any\n+\t\t * frames were dropped).\n+\t\t */\n+\t\tserverFrameCountPeriod_ += droppedFrames;\n+\n+\t\t/*\n+\t\t * The client may want a better idea of the true frame duration. Any error would feed straight\n+\t\t * into the correction term because of how it uses it to get the \"nearest\" frame.\n+\t\t */\n+\t\tif (frameCount_ == 0)\n+\t\t\tframeDurationEstimated_ = frameDuration;\n+\t\telse {\n+\t\t\tdouble diff = (wallClockFrameTimestamp - lastWallClockFrameTimestamp_) / (1 + droppedFrames);\n+\t\t\tint N = std::min(frameCount_, 99U);\n+\t\t\tframeDurationEstimated_ = frameCount_ == 1 ? diff : (N * frameDurationEstimated_ + diff) / (N + 1);\n+\t\t}\n+\n+\t\t/* Calculate frames remaining, and therefore \"time left until ready\". */\n+\t\tint framesRemaining = readyFrame_ - frameCount_;\n+\t\tuint64_t wallClockReadyTime = wallClockFrameTimestamp + (int64_t)framesRemaining * frameDurationEstimated_;\n+\n+\t\tif (serverFrameCountPeriod_ >= syncPeriod_) {\n+\t\t\tserverFrameCountPeriod_ = 0;\n+\n+\t\t\tpayload.frameDuration = frameDurationEstimated_ + .5; /* round to nearest */\n+\t\t\tpayload.wallClockFrameTimestamp = wallClockFrameTimestamp;\n+\t\t\tpayload.wallClockReadyTime = wallClockReadyTime;\n+\n+\t\t\tLOG(RPiSync, Debug) << \"Send packet (frameNumber \" << frameCount_ << \"):\";\n+\t\t\tLOG(RPiSync, Debug) << \" frameDuration \" << payload.frameDuration;\n+\t\t\tLOG(RPiSync, Debug) << \" wallClockFrameTimestamp \" << wallClockFrameTimestamp\n+\t\t\t\t\t << \" (\" << wallClockFrameTimestamp - lastWallClockFrameTimestamp_ << \")\";\n+\t\t\tLOG(RPiSync, Debug) << \" wallClockReadyTime \" << wallClockReadyTime;\n+\n+\t\t\tif (sendto(socket_, &payload, sizeof(payload), 0, (const sockaddr *)&addr_, sizeof(addr_)) < 0)\n+\t\t\t\tLOG(RPiSync, Error) << \"Send error! \" << strerror(errno);\n+\t\t}\n+\n+\t\ttimerValue_ = static_cast(wallClockReadyTime - wallClockFrameTimestamp);\n+\t\tif (!syncReady_ && wallClockFrameTimestamp + frameDurationEstimated_ / 2 > wallClockReadyTime) {\n+\t\t\tsyncReady_ = true;\n+\t\t\tLOG(RPiSync, Info) << \"*** Sync achieved! Difference \" << timerValue_ << \"us\";\n+\t\t}\n+\n+\t\tserverFrameCountPeriod_ += 1;\n+\n+\t} else if (mode_ == Mode::Client) {\n+\t\tuint64_t serverFrameTimestamp = 0;\n+\n+\t\tbool packetReceived = false;\n+\t\twhile (true) {\n+\t\t\tsocklen_t addrlen = sizeof(addr_);\n+\t\t\tint ret = recvfrom(socket_, &payload, sizeof(payload), 0, (struct sockaddr *)&addr_, &addrlen);\n+\n+\t\t\tif (ret < 0)\n+\t\t\t\tbreak;\n+\t\t\tpacketReceived = (ret > 0);\n+\t\t\tclientSeenPacket_ = true;\n+\n+\t\t\tframeDurationEstimated_ = payload.frameDuration;\n+\t\t\tserverFrameTimestamp = payload.wallClockFrameTimestamp;\n+\t\t\tserverReadyTime_ = payload.wallClockReadyTime;\n+\t\t}\n+\n+\t\tif (packetReceived) {\n+\t\t\tuint64_t clientFrameTimestamp = wallClockFrameTimestamp;\n+\t\t\tint64_t clientServerDelta = clientFrameTimestamp - serverFrameTimestamp;\n+\t\t\t/* \"A few frames ago\" may have better matched the server's frame. Calculate when it was. */\n+\t\t\tint framePeriodErrors = (clientServerDelta + frameDurationEstimated_ / 2) / frameDurationEstimated_;\n+\t\t\tint64_t clientFrameTimestampNearest = clientFrameTimestamp - framePeriodErrors * frameDurationEstimated_;\n+\t\t\t/* We must shorten a single client frame by this amount if it exceeds the minimum: */\n+\t\t\tint32_t correction = clientFrameTimestampNearest - serverFrameTimestamp;\n+\t\t\tif (std::abs(correction) < minAdjustment_)\n+\t\t\t\tcorrection = 0;\n+\n+\t\t\tLOG(RPiSync, Debug) << \"Received packet (frameNumber \" << frameCount_ << \"):\";\n+\t\t\tLOG(RPiSync, Debug) << \" serverFrameTimestamp \" << serverFrameTimestamp;\n+\t\t\tLOG(RPiSync, Debug) << \" serverReadyTime \" << serverReadyTime_;\n+\t\t\tLOG(RPiSync, Debug) << \" clientFrameTimestamp \" << clientFrameTimestamp;\n+\t\t\tLOG(RPiSync, Debug) << \" clientFrameTimestampNearest \" << clientFrameTimestampNearest\n+\t\t\t\t\t << \" (\" << framePeriodErrors << \")\";\n+\t\t\tLOG(RPiSync, Debug) << \" correction \" << correction;\n+\n+\t\t\tstatus.frameDurationOffset = correction * 1us;\n+\t\t}\n+\n+\t\ttimerValue_ = static_cast(serverReadyTime_ - wallClockFrameTimestamp);\n+\t\ttimerKnown = clientSeenPacket_; /* client must receive a packet before the timer value is correct */\n+\t\tif (clientSeenPacket_ && !syncReady_ && wallClockFrameTimestamp + frameDurationEstimated_ / 2 > serverReadyTime_) {\n+\t\t\tsyncReady_ = true;\n+\t\t\tLOG(RPiSync, Info) << \"*** Sync achieved! Difference \" << timerValue_ << \"us\";\n+\t\t}\n+\t}\n+\n+\tlastWallClockFrameTimestamp_ = wallClockFrameTimestamp;\n+\n+\tstatus.ready = syncReady_;\n+\tstatus.timerValue = timerValue_;\n+\tstatus.timerKnown = timerKnown;\n+\timageMetadata->set(\"sync.status\", status);\n+\tframeCount_++;\n+}\n+\n+void Sync::reset()\n+{\n+\t/* This resets the state so that the synchronisation procedure will start over. */\n+\tsyncReady_ = false;\n+\tframeCount_ = 0;\n+\ttimerValue_ = 0;\n+\tserverFrameCountPeriod_ = 0;\n+\tserverReadyTime_ = 0;\n+\tclientSeenPacket_ = false;\n+}\n+\n+void Sync::setMode(Mode mode)\n+{\n+\tmode_ = mode;\n+\n+\t/* Another \"sync session\" can be started by turning it off and on again. */\n+\tif (mode == Mode::Off)\n+\t\treset();\n+}\n+\n+void Sync::setFrameDuration(libcamera::utils::Duration frameDuration)\n+{\n+\tframeDuration_ = frameDuration;\n+};\n+\n+void Sync::setReadyFrame(unsigned int frame)\n+{\n+\treadyFrame_ = frame;\n+};\n+\n+/* Register algorithm with the system. */\n+static Algorithm *create(Controller *controller)\n+{\n+\treturn (Algorithm *)new Sync(controller);\n+}\n+static RegisterAlgorithm reg(NAME, &create);\ndiff --git a/src/ipa/rpi/controller/rpi/sync.h b/src/ipa/rpi/controller/rpi/sync.h\nnew file mode 100644\nindex 00000000..d3c79b7a\n--- /dev/null\n+++ b/src/ipa/rpi/controller/rpi/sync.h\n@@ -0,0 +1,68 @@\n+/* SPDX-License-Identifier: BSD-2-Clause */\n+/*\n+ * Copyright (C) 2024, Raspberry Pi Ltd\n+ *\n+ * sync.h - sync algorithm\n+ */\n+#pragma once\n+\n+#include \n+\n+#include \"../sync_algorithm.h\"\n+\n+namespace RPiController {\n+\n+struct SyncPayload {\n+\t/* Frame duration in microseconds. */\n+\tuint32_t frameDuration;\n+\t/* Server system (kernel) frame timestamp. */\n+\tuint64_t systemFrameTimestamp;\n+\t/* Server wall clock version of the frame timestamp. */\n+\tuint64_t wallClockFrameTimestamp;\n+\t/* Server system (kernel) sync time (the time at which frames are marked ready). */\n+\tuint64_t systemReadyTime;\n+\t/* Server wall clock version of the sync time. */\n+\tuint64_t wallClockReadyTime;\n+};\n+\n+class Sync : public SyncAlgorithm\n+{\n+public:\n+\tSync(Controller *controller);\n+\t~Sync();\n+\tchar const *name() const override;\n+\tint read(const libcamera::YamlObject ¶ms) override;\n+\tvoid setMode(Mode mode) override;\n+\tvoid initialiseSocket();\n+\tvoid switchMode(CameraMode const &cameraMode, Metadata *metadata) override;\n+\tvoid process(StatisticsPtr &stats, Metadata *imageMetadata) override;\n+\tvoid setFrameDuration(libcamera::utils::Duration frameDuration) override;\n+\tvoid setReadyFrame(unsigned int frame) override;\n+\n+private:\n+\tvoid reset(); /* reset internal state and start over */\n+\n+\tMode mode_; /* server or client */\n+\tstd::string group_; /* IP group address for sync messages */\n+\tuint16_t port_; /* port number for messages */\n+\tuint32_t syncPeriod_; /* send a sync message every this many frames */\n+\tuint32_t readyFrame_; /* tell the application we're ready after this many frames */\n+\tuint32_t minAdjustment_; /* don't adjust the client frame length by less than this */\n+\n+\tstruct sockaddr_in addr_;\n+\tint socket_ = -1;\n+\tlibcamera::utils::Duration frameDuration_;\n+\tunsigned int frameCount_;\n+\tbool syncReady_;\n+\tint64_t timerValue_ = 0; /* time until \"ready time\" */\n+\n+\tdouble frameDurationEstimated_ = 0; /* estimate the true frame duration of the sensor */\n+\tuint64_t lastWallClockFrameTimestamp_; /* wall clock timestamp of previous frame */\n+\n+\tuint32_t serverFrameCountPeriod_ = 0; /* send the next packet when this reaches syncPeriod_ */\n+\n+\tbool clientSeenPacket_ = false; /* whether the client has received a packet yet */\n+\tuint64_t serverReadyTime_ = 0; /* the client's latest value for when the server will be \"ready\" */\n+};\n+\n+} /* namespace RPiController */\n","prefixes":["v4","6/7"]}