[{"id":32672,"web_url":"https://patchwork.libcamera.org/comment/32672/","msgid":"","date":"2024-12-11T10:22:02","subject":"Re: [PATCH 1/1] controls: Add camera synchronisation controls","submitter":{"id":42,"url":"https://patchwork.libcamera.org/api/people/42/","name":"David Plowman","email":"david.plowman@raspberrypi.com"},"content":"Replying to my own patch...\n\nOn Mon, 9 Dec 2024 at 16:27, David Plowman\n wrote:\n>\n> New controls are added to control the camera \"sync\" algorithm, which\n> allows different cameras to synchronise their frames.\n>\n> Signed-off-by: David Plowman \n> ---\n> src/libcamera/control_ids_core.yaml | 104 ++++++++++++++++++++++++++++\n> 1 file changed, 104 insertions(+)\n>\n> diff --git a/src/libcamera/control_ids_core.yaml b/src/libcamera/control_ids_core.yaml\n> index d45cf8e5..6d45b757 100644\n> --- a/src/libcamera/control_ids_core.yaml\n> +++ b/src/libcamera/control_ids_core.yaml\n> @@ -971,4 +971,108 @@ controls:\n> description: |\n> Enable or disable the debug metadata.\n>\n> + - SyncMode:\n> + type: int32_t\n> + description: |\n> + Enable or disable camera synchronisation (\"sync\") mode.\n> +\n> + When sync mode is enabled, a camera will synchronise frames temporally\n> + with other cameras, either attached to the same device or a different\n> + one. There should be one \"server\" device, which broadcasts timing\n> + information to one or more \"clients\".\n> +\n> + Sync mode requires all cameras to be running at (as far as possible) the\n> + same fixed framerate, and may continue to make adjustments to keep the\n> + cameras together for the duration of the session.\n> +\n> + enum:\n> + - name: SyncModeOff\n> + value: 0\n> + description: Disable sync mode.\n> + - name: SyncModeServer\n> + value: 1\n> + description: |\n> + Enable sync mode, act as server. The server broadcasts timing\n> + messages to any clients that are listening, so that the clients can\n> + synchronise their camera frames with the server's.\n> + - name: SyncModeClient\n> + value: 2\n> + description: |\n> + Enable sync mode, act as client. A client listens for any server\n> + messages, and arranges for its camera frames to synchronise as\n> + closely as possible with the server's. Many clients can listen out\n> + for the same server. Clients can also be started ahead of any\n> + servers, causing them merely to wait for the server to start.\n> +\n> + \\sa SyncReady\n> + \\sa SyncTimer\n> + \\sa SyncFrames\n\nThese three \"\\sa\" lines need to go above the \"enum:\", of course. Will\nfix that in v2.\n\nThanks, David! :)\n\n> +\n> + - SyncReady:\n> + type: bool\n> + description: |\n> + When using the camera synchronisation algorithm, the server broadcasts\n> + timing information to the clients. This also includes the time (some\n> + number of frames in the future, called the \"ready time\") at which the\n> + server will signal its controlling application, using this control, to\n> + start using the image frames.\n> +\n> + The client receives the \"ready time\" from the server, and will signal\n> + its application to start using the frames at this same moment.\n> +\n> + While this control value is false, applications (on both client and\n> + server) should continue to wait, and not use the frames.\n> +\n> + Once this value becomes true, it means that this is the first frame\n> + where client and server have agreed that they will both be synchronised\n> + and that applications should begin consuming frames.\n> +\n> + \\sa SyncMode\n> + \\sa SyncTImer\n> + \\sa SyncFrames\n> +\n> + - SyncTimer:\n> + type: int64_t\n> + description: |\n> + This reports the amount of time, in microseconds, until the \"ready\n> + time\", at which the server and client will signal their controlling\n> + applications that the frames are now synchronised and should be\n> + used. The value may be refined slightly over time, becoming more precise\n> + as the \"ready time\" approaches.\n> +\n> + Servers always report this value, whereas clients will omit this control\n> + until they have received a message from the server that enables them to\n> + calculate it.\n> +\n> + Normally the value will start positive (the \"ready time\" is in the\n> + future), and decrease towards zero, before becoming negative (the \"ready\n> + time\" has elapsed). So there should be just one frame where the timer\n> + value is, or is very close to, zero - the one for which the SyncReady\n> + control becomes true. At this moment, the value indicates how closely\n> + synchronised the client believes it is with the server.\n> +\n> + But note that if frames are being dropped, then the \"near zero\" valued\n> + frame, or indeed any other, could be skipped. In these cases the timer\n> + value allows an application to deduce that this has happened.\n> +\n> + \\sa SyncMode\n> + \\sa SyncReady\n> + \\sa SyncFrames\n> +\n> + - SyncFrames:\n> + type: int32_t\n> + description: |\n> + The number of frames the server should wait, after enabling\n> + SyncModeServer, before signalling (via the SyncReady control) that\n> + frames should be used. This therefore determines the \"ready time\" for\n> + all synchronised cameras.\n> +\n> + This control value should be set only for the device that is to act as\n> + the server, before or at the same moment that SyncModeServer is\n> + enabled.\n> +\n> + \\sa SyncMode\n> + \\sa SyncReady\n> + \\sa SyncTimer\n> +\n> ...\n> --\n> 2.39.5\n>","headers":{"Return-Path":"","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id 96F6BBD80A\n\tfor ;\n\tWed, 11 Dec 2024 10:22:18 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id 8C7F967EA9;\n\tWed, 11 Dec 2024 11:22:17 +0100 (CET)","from mail-qt1-x82e.google.com (mail-qt1-x82e.google.com\n\t[IPv6:2607:f8b0:4864:20::82e])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id 7CE1666136\n\tfor ;\n\tWed, 11 Dec 2024 11:22:14 +0100 (CET)","by mail-qt1-x82e.google.com with SMTP id\n\td75a77b69052e-46788c32a69so8169261cf.2\n\tfor ;\n\tWed, 11 Dec 2024 02:22:14 -0800 (PST)"],"Authentication-Results":"lancelot.ideasonboard.com; dkim=pass (2048-bit key;\n\tunprotected) header.d=raspberrypi.com header.i=@raspberrypi.com\n\theader.b=\"BMgRdAg8\"; dkim-atps=neutral","DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed;\n\td=raspberrypi.com; s=google; t=1733912533; x=1734517333;\n\tdarn=lists.libcamera.org; \n\th=to:subject:message-id:date:from:in-reply-to:references:mime-version\n\t:from:to:cc:subject:date:message-id:reply-to;\n\tbh=zpbGcvSsW020O138mz45Oc3fVd8Hl7CxnUmAlN0JkLk=;\n\tb=BMgRdAg8hiWwBFOjhXLzuDW6yCYUv+0/ZH3yel32eq3Llsy/yH24UYvZuelEqieqlL\n\tbi+Ij82qmmgGvG9itYybEolCrfpgkk92hy9am9fNUSUcoDuEb8oQlTEGcd09ceR6pEje\n\tcqlNsPXVH2fZATE6InQEkndv0KUlGJsrKl9h1xOcqOWn6XldLjgqWMvlCy2mtTajIMnk\n\tbQcSHPYipZJaYHH3xZTYmf0OBc//0I0aKbPaxMGFihLwkSILynK/IhTyihdeEzsKM0s2\n\tKIB7PkDgeR1MOpm9v4XJ3NsOS2HbmaOqUdaXj9sRHx2mtVW18AFgfU2MTQJ4DL3eqvtm\n\t9OEQ==","X-Google-DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed;\n\td=1e100.net; s=20230601; t=1733912533; x=1734517333;\n\th=to:subject:message-id:date:from:in-reply-to:references:mime-version\n\t:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;\n\tbh=zpbGcvSsW020O138mz45Oc3fVd8Hl7CxnUmAlN0JkLk=;\n\tb=Ilkp4wUJV72lN+PqBDXNbHD9ZWHTfgwTkXggee6P1ZRS/FM+rczNfGewIbQLfCYSmr\n\t6urs3tzDh48an880RDK27RWnP+QvnP7tgHXlilzv1GLEYPNz4Qd+E1EalSfJj8DcxyoJ\n\tOPW0EGdvQlP4F/wEstv+UCtabVSyyH+woDdNLtlJVJqvAVf/hFdem1PpvJQdcALzYHtd\n\tSspmdwZ0IlYUbbMQPYeALjGGCOEAiVVL0hHsraCaTIucdwtZVwO7ufe1UecUtTB4Cz6l\n\tpMc3hM7S4b95js1fysxVwczLLVmdsL0D/kRN+H4Z8w15NiDKuPPHqHgZvI6wNgVVj/n+\n\ts0fw==","X-Gm-Message-State":"AOJu0YwoeHY3lEddfD8kmB1dLCxn7jhf7WK5ktmdc2MXO6i4IYACI6kg\n\tCjGebGyfp/22FcYIi+XH3zc0/sjf9/Rw3z4WFKqqXlUTldLJDmZdFHtFo7t0vm8P8iYUCao2Kk4\n\tRx7ObOhwNU+ynELvPUr2ZcqtLRXqLKo8ezA2ZoNE/1m7mYsXO","X-Gm-Gg":"ASbGnct1s/audPS6ZQu+4i9QOL3Z/GOxdxYyfiONi1tGox3c5WcnPwRU7KVFra66FSB\n\tNelGYqU41Vec++00qiTI7DTTsja86oh1pjxmYnV7sduxS3vC+MMq6JRAZJfPi5kzTGcg=","X-Google-Smtp-Source":"AGHT+IGvbv7IcLhmDYKV+88rn8UCgys7jJqUmtHA9BYrpbQVbr8QHoyjTUOVLnVd6c9yJDUz9eckdCQ5gSQ5O/cbFIY=","X-Received":"by 2002:a05:622a:11cc:b0:467:48f3:3452 with SMTP id\n\td75a77b69052e-467893a01fcmr32522091cf.56.1733912533208;\n\tWed, 11 Dec 2024 02:22:13 -0800 (PST)","MIME-Version":"1.0","References":"<20241209162747.2961-1-david.plowman@raspberrypi.com>\n\t<20241209162747.2961-2-david.plowman@raspberrypi.com>","In-Reply-To":"<20241209162747.2961-2-david.plowman@raspberrypi.com>","From":"David Plowman ","Date":"Wed, 11 Dec 2024 10:22:02 +0000","Message-ID":"","Subject":"Re: [PATCH 1/1] controls: Add camera synchronisation controls","To":"libcamera-devel@lists.libcamera.org","Content-Type":"text/plain; charset=\"UTF-8\"","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" "}},{"id":32689,"web_url":"https://patchwork.libcamera.org/comment/32689/","msgid":"","date":"2024-12-12T09:47:06","subject":"Re: [PATCH 1/1] controls: Add camera synchronisation controls","submitter":{"id":34,"url":"https://patchwork.libcamera.org/api/people/34/","name":"Naushir Patuck","email":"naush@raspberrypi.com"},"content":"Hi David,\n\nOn Mon, 9 Dec 2024 at 16:27, David Plowman\n wrote:\n>\n> New controls are added to control the camera \"sync\" algorithm, which\n> allows different cameras to synchronise their frames.\n>\n> Signed-off-by: David Plowman \n> ---\n> src/libcamera/control_ids_core.yaml | 104 ++++++++++++++++++++++++++++\n> 1 file changed, 104 insertions(+)\n>\n> diff --git a/src/libcamera/control_ids_core.yaml b/src/libcamera/control_ids_core.yaml\n> index d45cf8e5..6d45b757 100644\n> --- a/src/libcamera/control_ids_core.yaml\n> +++ b/src/libcamera/control_ids_core.yaml\n> @@ -971,4 +971,108 @@ controls:\n> description: |\n> Enable or disable the debug metadata.\n>\n> + - SyncMode:\n> + type: int32_t\n> + description: |\n> + Enable or disable camera synchronisation (\"sync\") mode.\n> +\n> + When sync mode is enabled, a camera will synchronise frames temporally\n> + with other cameras, either attached to the same device or a different\n> + one. There should be one \"server\" device, which broadcasts timing\n> + information to one or more \"clients\".\n> +\n> + Sync mode requires all cameras to be running at (as far as possible) the\n> + same fixed framerate, and may continue to make adjustments to keep the\n> + cameras together for the duration of the session.\n\nPerhaps worth noting that only the \"client\" will make adjustments in\nthe above paragraph?\n\n> +\n> + enum:\n> + - name: SyncModeOff\n> + value: 0\n> + description: Disable sync mode.\n> + - name: SyncModeServer\n> + value: 1\n> + description: |\n> + Enable sync mode, act as server. The server broadcasts timing\n> + messages to any clients that are listening, so that the clients can\n> + synchronise their camera frames with the server's.\n> + - name: SyncModeClient\n> + value: 2\n> + description: |\n> + Enable sync mode, act as client. A client listens for any server\n> + messages, and arranges for its camera frames to synchronise as\n> + closely as possible with the server's. Many clients can listen out\n> + for the same server. Clients can also be started ahead of any\n> + servers, causing them merely to wait for the server to start.\n> +\n> + \\sa SyncReady\n> + \\sa SyncTimer\n> + \\sa SyncFrames\n> +\n> + - SyncReady:\n> + type: bool\n> + description: |\n> + When using the camera synchronisation algorithm, the server broadcasts\n> + timing information to the clients. This also includes the time (some\n> + number of frames in the future, called the \"ready time\") at which the\n> + server will signal its controlling application, using this control, to\n> + start using the image frames.\n> +\n> + The client receives the \"ready time\" from the server, and will signal\n> + its application to start using the frames at this same moment.\n> +\n> + While this control value is false, applications (on both client and\n> + server) should continue to wait, and not use the frames.\n> +\n> + Once this value becomes true, it means that this is the first frame\n> + where client and server have agreed that they will both be synchronised\n\ns/client/clients/\n\nPerhaps we can clarify if SyncReady remains latched to true after\nready time or if it's a single-shot?\n\nWith these minor edits:\n\nReviewed-by: Naushir Patuck \n\n> + and that applications should begin consuming frames.\n> +\n> + \\sa SyncMode\n> + \\sa SyncTImer\n> + \\sa SyncFrames\n> +\n> + - SyncTimer:\n> + type: int64_t\n> + description: |\n> + This reports the amount of time, in microseconds, until the \"ready\n> + time\", at which the server and client will signal their controlling\n> + applications that the frames are now synchronised and should be\n> + used. The value may be refined slightly over time, becoming more precise\n> + as the \"ready time\" approaches.\n> +\n> + Servers always report this value, whereas clients will omit this control\n> + until they have received a message from the server that enables them to\n> + calculate it.\n> +\n> + Normally the value will start positive (the \"ready time\" is in the\n> + future), and decrease towards zero, before becoming negative (the \"ready\n> + time\" has elapsed). So there should be just one frame where the timer\n> + value is, or is very close to, zero - the one for which the SyncReady\n> + control becomes true. At this moment, the value indicates how closely\n> + synchronised the client believes it is with the server.\n> +\n> + But note that if frames are being dropped, then the \"near zero\" valued\n> + frame, or indeed any other, could be skipped. In these cases the timer\n> + value allows an application to deduce that this has happened.\n> +\n> + \\sa SyncMode\n> + \\sa SyncReady\n> + \\sa SyncFrames\n> +\n> + - SyncFrames:\n> + type: int32_t\n> + description: |\n> + The number of frames the server should wait, after enabling\n> + SyncModeServer, before signalling (via the SyncReady control) that\n> + frames should be used. This therefore determines the \"ready time\" for\n> + all synchronised cameras.\n> +\n> + This control value should be set only for the device that is to act as\n> + the server, before or at the same moment that SyncModeServer is\n> + enabled.\n> +\n> + \\sa SyncMode\n> + \\sa SyncReady\n> + \\sa SyncTimer\n> +\n> ...\n> --\n> 2.39.5\n>","headers":{"Return-Path":"","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id B85F3BD80A\n\tfor ;\n\tThu, 12 Dec 2024 09:47:46 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id B674A67EBF;\n\tThu, 12 Dec 2024 10:47:45 +0100 (CET)","from mail-yb1-xb36.google.com (mail-yb1-xb36.google.com\n\t[IPv6:2607:f8b0:4864:20::b36])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id 21F7267E6D\n\tfor ;\n\tThu, 12 Dec 2024 10:47:43 +0100 (CET)","by mail-yb1-xb36.google.com with SMTP id\n\t3f1490d57ef6-e28ebd78792so42111276.3\n\tfor ;\n\tThu, 12 Dec 2024 01:47:43 -0800 (PST)"],"Authentication-Results":"lancelot.ideasonboard.com; dkim=pass (2048-bit key;\n\tunprotected) header.d=raspberrypi.com header.i=@raspberrypi.com\n\theader.b=\"lPPCxr+B\"; dkim-atps=neutral","DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed;\n\td=raspberrypi.com; s=google; t=1733996862; x=1734601662;\n\tdarn=lists.libcamera.org; \n\th=cc:to:subject:message-id:date:from:in-reply-to:references\n\t:mime-version:from:to:cc:subject:date:message-id:reply-to;\n\tbh=SN7eOBMUK8g8L7dC8chZ9depkNv1WJMsE/ylgaJD07M=;\n\tb=lPPCxr+Bl06DkfLOy9dP2HXzztm6Yzxqcgv10MUCgCiyGEZTuC8nf43CyXK3D1RQHD\n\tSVlwgM14JaXxnU5J5nnIoLtHIlG+QF976hs5LkjUbmZEBpdOKzy9E1oozS3AphnvsCeL\n\tVoCurKq7pWbqbFQE8cIGaVk+0gvvQWexVjZfDSnwjPhRhZOchR/LTXVwa67VRA5L/OZN\n\tJ4D0DIjrV342WJbj9WLV4bEQ/QQXkbYw6uGbu7hbQ8AcBUVEoBPy24nPfinSNeUZvuug\n\tpNiaACyBoxbM+EOo/R7wuGrepewnaktUll+y5oU6M0LjC46z2Mle1PS6TLQojWWuXB5I\n\tN7sQ==","X-Google-DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed;\n\td=1e100.net; s=20230601; t=1733996862; x=1734601662;\n\th=cc:to:subject:message-id:date:from:in-reply-to:references\n\t:mime-version:x-gm-message-state:from:to:cc:subject:date:message-id\n\t:reply-to;\n\tbh=SN7eOBMUK8g8L7dC8chZ9depkNv1WJMsE/ylgaJD07M=;\n\tb=vFyHIeSCWilu5fSVUIy9BXJOduzbpCG1az8+6+sn2pq0dLE73HA3/ZRIvCNsGbGvEU\n\tq0FC1H5ngvvb5hOqY4SAt3etWjN1z9mBTTMNWq857EYZ8zjLgJtxMjT7Z9Ic038k440p\n\tueSfTnh2yrpFo7+j0sGp1hcTa9l9omW1rpXx16GBtddwbLpnaAKiGRQDLJXVLTe17rer\n\tysM9MGMFUA8oShda/Wxu/GDKm27XGMpjhe7xpDNjcfK/W46z6fPOAKdZhYfemCe9Il6h\n\tiPhiB+WrYo26bgkdqeNJImuYJlbWfV6w3ct3vqSLGBKAgqTQLrZiODsqddPLnszRXEnf\n\tuFDw==","X-Gm-Message-State":"AOJu0YxxdHVqvx4dcYvUUDgoBG3PPXxo3iixrkLjLYoocehkX29qVrxw\n\tiOBZoowTUMugEc/yYdlgDeDKYN2oLqufnjM+eOUC2o2yEbQ4BZineq4SWIUg4Tl3rK0vt7vQxyA\n\tzWm7+vkZxTOpByuH+cB7nBxPYSaY5aYIJl/4MVQ==","X-Gm-Gg":"ASbGncvcOCw5AiVBq53jJJ12kRYjYzQ1F6qygl+AZxxO0cm7xbd8psYWklT4W0ODYBF\n\tk5Mcx6L6igwcQow7aYn6VdDaNc3Tl8q/ROem7wfNuq8oqJMu7DVRDlPtDkgEVt9zAEIUrrw==","X-Google-Smtp-Source":"AGHT+IF3KJcolRjZycHygfc+yPZkdVvmleZilXHr5fE4a1dIDjrpDXqDk9mqIe9ZNLkw+U8oHHmwseGMqiIctBOfsxA=","X-Received":"by 2002:a05:690c:3388:b0:6ea:8a23:7679 with SMTP id\n\t00721157ae682-6f198a377b6mr11081077b3.1.1733996861651;\n\tThu, 12 Dec 2024 01:47:41 -0800 (PST)","MIME-Version":"1.0","References":"<20241209162747.2961-1-david.plowman@raspberrypi.com>\n\t<20241209162747.2961-2-david.plowman@raspberrypi.com>","In-Reply-To":"<20241209162747.2961-2-david.plowman@raspberrypi.com>","From":"Naushir Patuck ","Date":"Thu, 12 Dec 2024 09:47:06 +0000","Message-ID":"","Subject":"Re: [PATCH 1/1] controls: Add camera synchronisation controls","To":"David Plowman ","Cc":"libcamera-devel@lists.libcamera.org","Content-Type":"text/plain; charset=\"UTF-8\"","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" "}}]