From patchwork Wed Jan 22 14:53:45 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Plowman X-Patchwork-Id: 22684 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by patchwork.libcamera.org (Postfix) with ESMTPS id A24DDBD808 for ; Thu, 30 Jan 2025 11:52:25 +0000 (UTC) Received: from pendragon.ideasonboard.com (cpc89244-aztw30-2-0-cust6594.18-1.cable.virginm.net [86.31.185.195]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 2C93B886 for ; Thu, 30 Jan 2025 12:51:17 +0100 (CET) Authentication-Results: perceval.ideasonboard.com; dkim=permerror header.d=raspberrypi.com header.i=@raspberrypi.com header.a=rsa-sha1 header.s=google header.b=eFK2Mi5v; dkim-atps=neutral Delivered-To: kbingham@ideasonboard.com Received: from perceval.ideasonboard.com by perceval.ideasonboard.com with LMTP id 6GGVKk8GkWffRjQA4E0KoQ (envelope-from ) for ; Wed, 22 Jan 2025 15:53:03 +0100 Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by perceval.ideasonboard.com (Postfix) with ESMTPS id 843721081; Wed, 22 Jan 2025 15:53:03 +0100 (CET) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 6573D68549; Wed, 22 Jan 2025 15:54:03 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="eFK2Mi5v"; dkim-atps=neutral Received: from mail-wm1-x32e.google.com (mail-wm1-x32e.google.com [IPv6:2a00:1450:4864:20::32e]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id DDEB96855B for ; Wed, 22 Jan 2025 15:53:55 +0100 (CET) Received: by mail-wm1-x32e.google.com with SMTP id 5b1f17b1804b1-436345cc17bso50869475e9.0 for ; Wed, 22 Jan 2025 06:53:55 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; t=1737557635; x=1738162435; darn=lists.libcamera.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=O7R7laaCtTqG8D2rZUcpq6OOb09xjPw1X7M9gCXqZtw=; b=eFK2Mi5vXwnBjvUTtg1eCszTu6K9NyabtvPKYJM2iUXZJGN+DrhJDU1diVw6SyJ/Lg zumlk67UX/ax72N/0YrUsulaa63mUqDLp+Jg/tZqRsswi6JgDrp5UeSozatBgXWpcpny LAFUcX4xWWgHOHo2npfT/sSi14Vq2mapB35zBiRaeD0XULDBM6vnJx9QZNIfcZTVni5R hhSXX5gawe4wn49ckYpF/UaXmEVLWlEeCleS4ROzsJ4PtwOBxbUZnypXDfibD+lKdJLk q6pvwefxQKswnCCdtBbeey7i2jEHPXe/3lb9ExbOze8/vUicC5AAWwLTKZcnZui77OfL B95g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1737557635; x=1738162435; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=O7R7laaCtTqG8D2rZUcpq6OOb09xjPw1X7M9gCXqZtw=; b=rk3efMevmONCneAzZAb8kQ+6oH1xRRrgBFq1XX6v3Zz/DExNbpaFgLJpYRO8DtlQEc MVa561/CQFnQceivqfurPuKnNIO0N0Sl8VxzK6Gi/7UigI6pbPHDaGz+ATFjTC/lwIGV piDz5InEKe7lN97xgKgWYnACo3sLchlwnpHQbFCTkeBjteWyQ6IqZoYRk6Q3z9/Vpx+w k9pAi+fP97XAf4sMtvejoebu60Li9Mc48RI4u3yiF89tj3zmIYHUSeDGr5vFmYKCB2O7 2DRoIKCL3bjS2AbaqOe/ElfUyBKSxyf4Bty/UmloRwTtAKrfJVc7de68+q765fjk0Cm1 d+Nw== X-Gm-Message-State: AOJu0YwJ6ZuRklPoO+zJ3umHynYYikdkdSrmK6zRqMVDXRs/AIgxEdjy MZMOyyUN9OaRCZy8F4L26fEEIEKLCu6AcCH6A0ScVxtQB1UQb0EK+UBVGCyNIt7iaLa8wY1pBtu A X-Gm-Gg: ASbGncuaYfGpLVJmFQAt4VtGdO5VgHZICIz0O68znayjMx9/q7ui+mIAe07/QqVU+Uo fJ8nSuDs9c9nwX6ThV/G/mSW0fIHFOCmbihqO4V/grGM67t6sur337ce8GKRUAuqFm0GOza7JuU 5cyXOrej8dgwfG58NdbABcK/5jOmWaMBDYKVEuFUzN6gj7N7xGYJwql2Se0MZp7VDvwAg2G9s7b 5xjfxRebSJDoi24zkEmRrAkl1XE3YQWun1RgLjPoeVZ+HVqGPUzAGOuv6eRaSXO4EQAx8kQmjhy H8898qKkNyQ7jP4dFj5PWcqk3Q== X-Google-Smtp-Source: =?utf-8?q?AGHT+IGpi0cXeCh2+UhsBDCXrY57k/541gnFyeV4o?= =?utf-8?q?4FVEN6ZPl7qu/PUKk8BIsjRUuqLyL8B+4ZpCw=3D=3D?= X-Received: by 2002:a05:600c:3593:b0:431:55c1:f440 with SMTP id 5b1f17b1804b1-4389144eea8mr240611075e9.30.1737557634830; Wed, 22 Jan 2025 06:53:54 -0800 (PST) Received: from raspberrypi.pitowers.org ([2a00:1098:3142:1f:ffc9:aff6:7f7f:893b]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-438b66dc08bsm11551395e9.37.2025.01.22.06.53.54 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 22 Jan 2025 06:53:54 -0800 (PST) From: David Plowman To: libcamera-devel@lists.libcamera.org Cc: David Plowman , Naushir Patuck Subject: [PATCH v4 3/7] controls: Add camera synchronisation controls for Raspberry Pi Date: Wed, 22 Jan 2025 14:53:45 +0000 Message-Id: <20250122145349.7220-4-david.plowman@raspberrypi.com> X-Mailer: git-send-email 2.39.5 In-Reply-To: <20250122145349.7220-1-david.plowman@raspberrypi.com> References: <20250122145349.7220-1-david.plowman@raspberrypi.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" X-TUID: r1puB9Q1dEfW Resent-From: Kieran Bingham Resent-To: parsemail@patchwork.libcamera.org New controls are added to control the camera "sync" algorithm, which allows different cameras to synchronise their frames. For the time being, the controls are Raspberry Pi specific, though this is expected to change in future. Signed-off-by: David Plowman Reviewed-by: Naushir Patuck --- src/libcamera/control_ids_rpi.yaml | 113 +++++++++++++++++++++++++++++ 1 file changed, 113 insertions(+) diff --git a/src/libcamera/control_ids_rpi.yaml b/src/libcamera/control_ids_rpi.yaml index 7524c5d2..5c2721c0 100644 --- a/src/libcamera/control_ids_rpi.yaml +++ b/src/libcamera/control_ids_rpi.yaml @@ -58,4 +58,117 @@ controls: official libcamera API support for per-stream controls in the future. \sa ScalerCrop + + - SyncMode: + type: int32_t + direction: in + description: | + Enable or disable camera synchronisation ("sync") mode. + + When sync mode is enabled, a camera will synchronise frames temporally + with other cameras, either attached to the same device or a different + one. There should be one "server" device, which broadcasts timing + information to one or more "clients". Communication is one-way, from + server to clients only, and it is only clients that adjust their frame + timings to match the server. + + Sync mode requires all cameras to be running at (as far as possible) the + same fixed framerate. Clients may continue to make adjustments to keep + their cameras synchronised with the server for the duration of the + session, though any updates after the initial ones should remain small. + + \sa SyncReady + \sa SyncTimer + \sa SyncFrames + + enum: + - name: SyncModeOff + value: 0 + description: Disable sync mode. + - name: SyncModeServer + value: 1 + description: | + Enable sync mode, act as server. The server broadcasts timing + messages to any clients that are listening, so that the clients can + synchronise their camera frames with the server's. + - name: SyncModeClient + value: 2 + description: | + Enable sync mode, act as client. A client listens for any server + messages, and arranges for its camera frames to synchronise as + closely as possible with the server's. Many clients can listen out + for the same server. Clients can also be started ahead of any + servers, causing them merely to wait for the server to start. + + - SyncReady: + type: bool + direction: out + description: | + When using the camera synchronisation algorithm, the server broadcasts + timing information to the clients. This also includes the time (some + number of frames in the future, called the "ready time") at which the + server will signal its controlling application, using this control, to + start using the image frames. + + The client receives the "ready time" from the server, and will signal + its application to start using the frames at this same moment. + + While this control value is false, applications (on both client and + server) should continue to wait, and not use the frames. + + Once this value becomes true, it means that this is the first frame + where the server and its clients have agreed that they will both be + synchronised and that applications should begin consuming frames. + Thereafter, this control will continue to signal the value true for + the rest of the session. + + \sa SyncMode + \sa SyncTimer + \sa SyncFrames + + - SyncTimer: + type: int64_t + direction: out + description: | + This reports the amount of time, in microseconds, until the "ready + time", at which the server and client will signal their controlling + applications that the frames are now synchronised and should be + used. The value may be refined slightly over time, becoming more precise + as the "ready time" approaches. + + Servers always report this value, whereas clients will omit this control + until they have received a message from the server that enables them to + calculate it. + + Normally the value will start positive (the "ready time" is in the + future), and decrease towards zero, before becoming negative (the "ready + time" has elapsed). So there should be just one frame where the timer + value is, or is very close to, zero - the one for which the SyncReady + control becomes true. At this moment, the value indicates how closely + synchronised the client believes it is with the server. + + But note that if frames are being dropped, then the "near zero" valued + frame, or indeed any other, could be skipped. In these cases the timer + value allows an application to deduce that this has happened. + + \sa SyncMode + \sa SyncReady + \sa SyncFrames + + - SyncFrames: + type: int32_t + direction: in + description: | + The number of frames the server should wait, after enabling + SyncModeServer, before signalling (via the SyncReady control) that + frames should be used. This therefore determines the "ready time" for + all synchronised cameras. + + This control value should be set only for the device that is to act as + the server, before or at the same moment at which SyncModeServer is + enabled. + + \sa SyncMode + \sa SyncReady + \sa SyncTimer ...