From patchwork Thu Jan 9 14:32:07 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Plowman X-Patchwork-Id: 22503 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id AE24EC32EA for ; Thu, 9 Jan 2025 14:32:28 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id E8B8E68549; Thu, 9 Jan 2025 15:32:27 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="hAJqMt+F"; dkim-atps=neutral Received: from mail-wr1-x42c.google.com (mail-wr1-x42c.google.com [IPv6:2a00:1450:4864:20::42c]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id C1040684E4 for ; Thu, 9 Jan 2025 15:32:20 +0100 (CET) Received: by mail-wr1-x42c.google.com with SMTP id ffacd0b85a97d-3862d161947so538006f8f.3 for ; Thu, 09 Jan 2025 06:32:20 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; t=1736433140; x=1737037940; darn=lists.libcamera.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=O7R7laaCtTqG8D2rZUcpq6OOb09xjPw1X7M9gCXqZtw=; b=hAJqMt+FHYhhU4Y+eilTljb/iRaAN4wesE7ralJM5ZnbjKb8wwJ7GW2BthPZByp6Yo HfT396V8B5gLyqfeThO5LvI0gMV8xezfrPnm90YyHxFBc3VTzdrklSDG9oQKIo99Vay3 4uMQa6KE+V9DOGRMJ4CQ7jEhyWnmOpI2sfJLozDdBAYtZashMj+t0kUP2VQO4uc11igI IMUnGWs/SEHA6Co5OybJDiHaqjP99EsGIdUuuzyj3SPr7IxfMjI2n3GXQj8G7hxqE+gl 2sQU1A2jBCheta75SCbW+Yo0rygEIPI1K4YM0T5ySvZxm735E1CDZ6/fE9JiolSGgs+x IA0A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1736433140; x=1737037940; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=O7R7laaCtTqG8D2rZUcpq6OOb09xjPw1X7M9gCXqZtw=; b=RcyzCGYTn48giDek+tdsnqvSV7IQLPJf6gAG22vPm6ZZgIfKExoagTtLgQzzZ7eWPm gt+gyeUTII7HL2G9rCo2rmii+DwdwzyLbB0OCx2rNkCsEgtee6I3qGC13IPiOXwChP8E KixpU7xhlGo2i7QHJN/dXCOySA8y2L8jnppxneISOqx5PeU4SeQ47uo118N3D7PRt47O WlTc6ZGH/o8p06t4OQ4CKoCgXEaGx4IhRCgRB7q1YJKFwSuvL+m/XkZ+q6/hkx0TZiCo lRkHdd0XY+piFyae4tgMpp8wJ/2lW1Nhvx5ciQWN4JVGzNUds/OtAaAu3P1laNegUvBQ KqJg== X-Gm-Message-State: AOJu0Yw9vukROhIJzdUkVogCLfjDwOJhXS4bP4L8tQ9uMldLDMTfG194 Nd04YKyHor5RdTwuwqkh0sEqP5aUO6DNhFsP1suigI+yi1m0hQmRJafg9IgXwkZ2xp3srrCNqjC 8 X-Gm-Gg: ASbGncvaJ+dzR5W7R6+qA5l37SejQrXHhAqQ69UjraInNGJPWw7tAKrU4oN0SqkXfSv b55UbC/D0X5mA7hhjlb+lcPKc+TXVHWAoNJ8Q3zfCQ/jWTgK1U7ds0YYRV/GpBELRitZ9X5YL5c 6Bix7xcB3gHTUnno7CUk1ZjKCdAL2iNRF8tXejn6NUmo0ldCMOdqLtRMIca6QX71BSC7qYwYAzD n7r7GZPAjwLEvXyi+uznb6PM4knMP1AoOBI8fpAQtxxQ53P2AQqoV0Ljp0U3F98Mxl84e2imk7o H8LVv5dqYF++ X-Google-Smtp-Source: AGHT+IEbXmlWX0F2jBQLlopT9Vq/ymdnV3x4n7BJ+aoon3seQwb0gOP2XIO0+VPWbceYbojkgfubog== X-Received: by 2002:a5d:6d8e:0:b0:38a:88be:bcb4 with SMTP id ffacd0b85a97d-38a88bebd23mr5435050f8f.29.1736433138555; Thu, 09 Jan 2025 06:32:18 -0800 (PST) Received: from raspberrypi.pitowers.org ([2a00:1098:3142:1f:ffc9:aff6:7f7f:893b]) by smtp.gmail.com with ESMTPSA id ffacd0b85a97d-38a8e4b80b2sm1952569f8f.80.2025.01.09.06.32.17 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 09 Jan 2025 06:32:18 -0800 (PST) From: David Plowman To: libcamera-devel@lists.libcamera.org Cc: David Plowman , Naushir Patuck Subject: [PATCH v3 3/7] controls: Add camera synchronisation controls for Raspberry Pi Date: Thu, 9 Jan 2025 14:32:07 +0000 Message-Id: <20250109143211.11939-4-david.plowman@raspberrypi.com> X-Mailer: git-send-email 2.39.5 In-Reply-To: <20250109143211.11939-1-david.plowman@raspberrypi.com> References: <20250109143211.11939-1-david.plowman@raspberrypi.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" New controls are added to control the camera "sync" algorithm, which allows different cameras to synchronise their frames. For the time being, the controls are Raspberry Pi specific, though this is expected to change in future. Signed-off-by: David Plowman Reviewed-by: Naushir Patuck Acked-by: Kieran Bingham --- src/libcamera/control_ids_rpi.yaml | 113 +++++++++++++++++++++++++++++ 1 file changed, 113 insertions(+) diff --git a/src/libcamera/control_ids_rpi.yaml b/src/libcamera/control_ids_rpi.yaml index 7524c5d2..5c2721c0 100644 --- a/src/libcamera/control_ids_rpi.yaml +++ b/src/libcamera/control_ids_rpi.yaml @@ -58,4 +58,117 @@ controls: official libcamera API support for per-stream controls in the future. \sa ScalerCrop + + - SyncMode: + type: int32_t + direction: in + description: | + Enable or disable camera synchronisation ("sync") mode. + + When sync mode is enabled, a camera will synchronise frames temporally + with other cameras, either attached to the same device or a different + one. There should be one "server" device, which broadcasts timing + information to one or more "clients". Communication is one-way, from + server to clients only, and it is only clients that adjust their frame + timings to match the server. + + Sync mode requires all cameras to be running at (as far as possible) the + same fixed framerate. Clients may continue to make adjustments to keep + their cameras synchronised with the server for the duration of the + session, though any updates after the initial ones should remain small. + + \sa SyncReady + \sa SyncTimer + \sa SyncFrames + + enum: + - name: SyncModeOff + value: 0 + description: Disable sync mode. + - name: SyncModeServer + value: 1 + description: | + Enable sync mode, act as server. The server broadcasts timing + messages to any clients that are listening, so that the clients can + synchronise their camera frames with the server's. + - name: SyncModeClient + value: 2 + description: | + Enable sync mode, act as client. A client listens for any server + messages, and arranges for its camera frames to synchronise as + closely as possible with the server's. Many clients can listen out + for the same server. Clients can also be started ahead of any + servers, causing them merely to wait for the server to start. + + - SyncReady: + type: bool + direction: out + description: | + When using the camera synchronisation algorithm, the server broadcasts + timing information to the clients. This also includes the time (some + number of frames in the future, called the "ready time") at which the + server will signal its controlling application, using this control, to + start using the image frames. + + The client receives the "ready time" from the server, and will signal + its application to start using the frames at this same moment. + + While this control value is false, applications (on both client and + server) should continue to wait, and not use the frames. + + Once this value becomes true, it means that this is the first frame + where the server and its clients have agreed that they will both be + synchronised and that applications should begin consuming frames. + Thereafter, this control will continue to signal the value true for + the rest of the session. + + \sa SyncMode + \sa SyncTimer + \sa SyncFrames + + - SyncTimer: + type: int64_t + direction: out + description: | + This reports the amount of time, in microseconds, until the "ready + time", at which the server and client will signal their controlling + applications that the frames are now synchronised and should be + used. The value may be refined slightly over time, becoming more precise + as the "ready time" approaches. + + Servers always report this value, whereas clients will omit this control + until they have received a message from the server that enables them to + calculate it. + + Normally the value will start positive (the "ready time" is in the + future), and decrease towards zero, before becoming negative (the "ready + time" has elapsed). So there should be just one frame where the timer + value is, or is very close to, zero - the one for which the SyncReady + control becomes true. At this moment, the value indicates how closely + synchronised the client believes it is with the server. + + But note that if frames are being dropped, then the "near zero" valued + frame, or indeed any other, could be skipped. In these cases the timer + value allows an application to deduce that this has happened. + + \sa SyncMode + \sa SyncReady + \sa SyncFrames + + - SyncFrames: + type: int32_t + direction: in + description: | + The number of frames the server should wait, after enabling + SyncModeServer, before signalling (via the SyncReady control) that + frames should be used. This therefore determines the "ready time" for + all synchronised cameras. + + This control value should be set only for the device that is to act as + the server, before or at the same moment at which SyncModeServer is + enabled. + + \sa SyncMode + \sa SyncReady + \sa SyncTimer ...