From patchwork Wed Jan 22 14:53:43 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Plowman X-Patchwork-Id: 22675 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by patchwork.libcamera.org (Postfix) with ESMTPS id D6A23BDB1C for ; Thu, 30 Jan 2025 11:51:42 +0000 (UTC) Received: from pendragon.ideasonboard.com (cpc89244-aztw30-2-0-cust6594.18-1.cable.virginm.net [86.31.185.195]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 38986886 for ; Thu, 30 Jan 2025 12:50:34 +0100 (CET) Authentication-Results: perceval.ideasonboard.com; dkim=permerror header.d=raspberrypi.com header.i=@raspberrypi.com header.a=rsa-sha1 header.s=google header.b=cvhHQZmy; dkim-atps=neutral Delivered-To: kbingham@ideasonboard.com Received: from perceval.ideasonboard.com by perceval.ideasonboard.com with LMTP id 8PRvMkMGkWdTRjQA4E0KoQ (envelope-from ) for ; Wed, 22 Jan 2025 15:52:51 +0100 Received: from mail-wm1-x32e.google.com (mail-wm1-x32e.google.com [IPv6:2a00:1450:4864:20::32e]) by perceval.ideasonboard.com (Postfix) with ESMTPS id B1054F91 for ; Wed, 22 Jan 2025 15:52:51 +0100 (CET) Received: by mail-wm1-x32e.google.com with SMTP id 5b1f17b1804b1-4361b0ec57aso72574415e9.0 for ; Wed, 22 Jan 2025 06:53:54 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; t=1737557633; x=1738162433; darn=ideasonboard.com; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=wWS+qkr4e2HeK4Yl6e1wMBaJ6VR8PI7E4/+klrfz1lo=; b=cvhHQZmywiNokFHaWKFDxvF1fSDkhukts6ExdZtdrJ0bIkZkop4nMDmCvlJFEAmaFC iUUW4LRndObRbaRkKgMx7PhRrWyBEr4BTvZLgUZXBWoLM1F0F66xmRgcGorF1q01ksJr 4+yaei5TlZ0ZeFfgGXcXh6AO4P8GgvG0X5I36cSPreMMSZ8rKUjdzqv/3bKn2jfoj9Nw 2zi/QChoiMthfiWRI1q16WSOBjAZRyEI/+8L2O1kFA9OW92+BcV0w593Vs/v6/rC0Mbi XxeOLYDlguD9nFxlxQyvSFFUQnEjKDMvRzqN8AglffWlmFvebI79+G50dIRv2UMZ7Kft zReg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1737557633; x=1738162433; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=wWS+qkr4e2HeK4Yl6e1wMBaJ6VR8PI7E4/+klrfz1lo=; b=ghWscfu6b/vq03e0EMv1Equte04wMwk7lNz0hxwDQrUBuVO0aAlxFEPbQ/WL5f7EuJ ME/EHcSVbSquKIdWa0BQD7HdK9Xucnys4GjhQIpOp4gxN4kKGPBG244lT/Yl7+YbzEyu oPka98FcVAKsMapyM4zZR+TN1PGxpfiQ58mu2L7mm6DtU95UNcT3QI//9d1awvMBu/rL IPdK/bzsLDUJ39eOLBRVHk6UNLXdAblWrUeAmgI7VB5XW+elvyqr/PnGXbH5JOBqmHYU cLk1EwctT5qNck+fFQi94CxTh20i6L3e9URYwEo0bjVSwSXFb1TZ4jn5n7LqhEF55bIu QQUg== X-Forwarded-Encrypted: i=1; =?utf-8?q?AJvYcCUiBgqJyS5ov1bkO5jtgKvBHcINIAf8?= =?utf-8?q?tAWIT1Zb5uQdl8TX+C946/qDzw7LA8uVfwSaqR2kD/vTzhy0migKyw?= =?utf-8?q?=3D=3D=40ideasonboard=2Ecom?= X-Gm-Message-State: AOJu0YwveOLqnzVHUiX44+X+OOsoPLSQmP7oIU/6Mj3kKwnT5NzAL5dc K+OCyO2pb/V2FR9FlwnjTErQWot+FsjA28uRHh6r1yVWNfVHb+pJuQwNuBkri+I= X-Gm-Gg: ASbGncuDsYzqzqUlUGt4r6kDIvZmKCxXLQ9q/pZkp/O37l+4qj+9xDWB5ToPL2ox0mp Ld+Z4XPfex7yt/ie+FbVS03RzTFSZW7nIVZjAN+h1vr50nQPQm1EZi/s7gD2pm6W81k5F+pFXj6 HSWg/VIYkwKHf1Skes3Mf00Tg6zvEzWHfpskpyhex7IqHVwL2R8ScJs3BY05mHY8ii/O9xjm9qj V7SIp6rV50Xev4fQBRZI2rOtPCGoRDJ3gbf+e03NtVCad04rqjZFiQwhYTVe4RdaTrdrnYE1f8J pypkkldsNW/Iy7+wQQoQHZ7tYw== X-Google-Smtp-Source: =?utf-8?q?AGHT+IEUwzPO8tTUiCeAD0fO+J3bUitlNQubQuvJO?= =?utf-8?q?Y/6k24gRJevylhbFRRXFr4FMh5rQGWAvz0fXA=3D=3D?= X-Received: by 2002:a05:600c:19cc:b0:436:fb9e:26c with SMTP id 5b1f17b1804b1-438913de937mr203978545e9.17.1737557633585; Wed, 22 Jan 2025 06:53:53 -0800 (PST) Received: from raspberrypi.pitowers.org ([2a00:1098:3142:1f:ffc9:aff6:7f7f:893b]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-438b66dc08bsm11551395e9.37.2025.01.22.06.53.53 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 22 Jan 2025 06:53:53 -0800 (PST) From: David Plowman To: libcamera-devel@lists.libcamera.org Cc: David Plowman , Kieran Bingham , Naushir Patuck , Laurent Pinchart Subject: [PATCH v4 1/7] controls: Add FrameWallClock control Date: Wed, 22 Jan 2025 14:53:43 +0000 Message-Id: <20250122145349.7220-2-david.plowman@raspberrypi.com> X-Mailer: git-send-email 2.39.5 In-Reply-To: <20250122145349.7220-1-david.plowman@raspberrypi.com> References: <20250122145349.7220-1-david.plowman@raspberrypi.com> MIME-Version: 1.0 X-TUID: UiUOg+CTxXPG Resent-From: Kieran Bingham Resent-To: parsemail@patchwork.libcamera.org Add a FrameWallClock control that reports the same moment as the frame's SensorTimestamp, but in wallclock units. Signed-off-by: David Plowman Reviewed-by: Kieran Bingham Reviewed-by: Naushir Patuck Reviewed-by: Laurent Pinchart --- src/libcamera/control_ids_core.yaml | 19 ++++++++++++++++--- 1 file changed, 16 insertions(+), 3 deletions(-) diff --git a/src/libcamera/control_ids_core.yaml b/src/libcamera/control_ids_core.yaml index aa744864..028919ef 100644 --- a/src/libcamera/control_ids_core.yaml +++ b/src/libcamera/control_ids_core.yaml @@ -1,6 +1,4 @@ -# SPDX-License-Identifier: LGPL-2.1-or-later -# -# Copyright (C) 2019, Google Inc. + # %YAML 1.1 --- @@ -1268,4 +1266,19 @@ controls: description: | Enable or disable the debug metadata. + - FrameWallClock: + type: int64_t + direction: out + description: | + This timestamp corresponds to the same moment in time as the + SensorTimestamp, but is represented as a wall clock time as measured by + the CLOCK_REALTIME clock. + + Being a wall clock measurement, it can be used to synchronise timing + across different devices. + + \sa SensorTimestamp + + The FrameWallClock control can only be returned in metadata. + ... From patchwork Wed Jan 22 14:53:45 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Plowman X-Patchwork-Id: 22684 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by patchwork.libcamera.org (Postfix) with ESMTPS id A24DDBD808 for ; Thu, 30 Jan 2025 11:52:25 +0000 (UTC) Received: from pendragon.ideasonboard.com (cpc89244-aztw30-2-0-cust6594.18-1.cable.virginm.net [86.31.185.195]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 2C93B886 for ; Thu, 30 Jan 2025 12:51:17 +0100 (CET) Authentication-Results: perceval.ideasonboard.com; dkim=permerror header.d=raspberrypi.com header.i=@raspberrypi.com header.a=rsa-sha1 header.s=google header.b=eFK2Mi5v; dkim-atps=neutral Delivered-To: kbingham@ideasonboard.com Received: from perceval.ideasonboard.com by perceval.ideasonboard.com with LMTP id 6GGVKk8GkWffRjQA4E0KoQ (envelope-from ) for ; Wed, 22 Jan 2025 15:53:03 +0100 Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by perceval.ideasonboard.com (Postfix) with ESMTPS id 843721081; Wed, 22 Jan 2025 15:53:03 +0100 (CET) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 6573D68549; Wed, 22 Jan 2025 15:54:03 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="eFK2Mi5v"; dkim-atps=neutral Received: from mail-wm1-x32e.google.com (mail-wm1-x32e.google.com [IPv6:2a00:1450:4864:20::32e]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id DDEB96855B for ; Wed, 22 Jan 2025 15:53:55 +0100 (CET) Received: by mail-wm1-x32e.google.com with SMTP id 5b1f17b1804b1-436345cc17bso50869475e9.0 for ; Wed, 22 Jan 2025 06:53:55 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; t=1737557635; x=1738162435; darn=lists.libcamera.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=O7R7laaCtTqG8D2rZUcpq6OOb09xjPw1X7M9gCXqZtw=; b=eFK2Mi5vXwnBjvUTtg1eCszTu6K9NyabtvPKYJM2iUXZJGN+DrhJDU1diVw6SyJ/Lg zumlk67UX/ax72N/0YrUsulaa63mUqDLp+Jg/tZqRsswi6JgDrp5UeSozatBgXWpcpny LAFUcX4xWWgHOHo2npfT/sSi14Vq2mapB35zBiRaeD0XULDBM6vnJx9QZNIfcZTVni5R hhSXX5gawe4wn49ckYpF/UaXmEVLWlEeCleS4ROzsJ4PtwOBxbUZnypXDfibD+lKdJLk q6pvwefxQKswnCCdtBbeey7i2jEHPXe/3lb9ExbOze8/vUicC5AAWwLTKZcnZui77OfL B95g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1737557635; x=1738162435; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=O7R7laaCtTqG8D2rZUcpq6OOb09xjPw1X7M9gCXqZtw=; b=rk3efMevmONCneAzZAb8kQ+6oH1xRRrgBFq1XX6v3Zz/DExNbpaFgLJpYRO8DtlQEc MVa561/CQFnQceivqfurPuKnNIO0N0Sl8VxzK6Gi/7UigI6pbPHDaGz+ATFjTC/lwIGV piDz5InEKe7lN97xgKgWYnACo3sLchlwnpHQbFCTkeBjteWyQ6IqZoYRk6Q3z9/Vpx+w k9pAi+fP97XAf4sMtvejoebu60Li9Mc48RI4u3yiF89tj3zmIYHUSeDGr5vFmYKCB2O7 2DRoIKCL3bjS2AbaqOe/ElfUyBKSxyf4Bty/UmloRwTtAKrfJVc7de68+q765fjk0Cm1 d+Nw== X-Gm-Message-State: AOJu0YwJ6ZuRklPoO+zJ3umHynYYikdkdSrmK6zRqMVDXRs/AIgxEdjy MZMOyyUN9OaRCZy8F4L26fEEIEKLCu6AcCH6A0ScVxtQB1UQb0EK+UBVGCyNIt7iaLa8wY1pBtu A X-Gm-Gg: ASbGncuaYfGpLVJmFQAt4VtGdO5VgHZICIz0O68znayjMx9/q7ui+mIAe07/QqVU+Uo fJ8nSuDs9c9nwX6ThV/G/mSW0fIHFOCmbihqO4V/grGM67t6sur337ce8GKRUAuqFm0GOza7JuU 5cyXOrej8dgwfG58NdbABcK/5jOmWaMBDYKVEuFUzN6gj7N7xGYJwql2Se0MZp7VDvwAg2G9s7b 5xjfxRebSJDoi24zkEmRrAkl1XE3YQWun1RgLjPoeVZ+HVqGPUzAGOuv6eRaSXO4EQAx8kQmjhy H8898qKkNyQ7jP4dFj5PWcqk3Q== X-Google-Smtp-Source: =?utf-8?q?AGHT+IGpi0cXeCh2+UhsBDCXrY57k/541gnFyeV4o?= =?utf-8?q?4FVEN6ZPl7qu/PUKk8BIsjRUuqLyL8B+4ZpCw=3D=3D?= X-Received: by 2002:a05:600c:3593:b0:431:55c1:f440 with SMTP id 5b1f17b1804b1-4389144eea8mr240611075e9.30.1737557634830; Wed, 22 Jan 2025 06:53:54 -0800 (PST) Received: from raspberrypi.pitowers.org ([2a00:1098:3142:1f:ffc9:aff6:7f7f:893b]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-438b66dc08bsm11551395e9.37.2025.01.22.06.53.54 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 22 Jan 2025 06:53:54 -0800 (PST) From: David Plowman To: libcamera-devel@lists.libcamera.org Cc: David Plowman , Naushir Patuck Subject: [PATCH v4 3/7] controls: Add camera synchronisation controls for Raspberry Pi Date: Wed, 22 Jan 2025 14:53:45 +0000 Message-Id: <20250122145349.7220-4-david.plowman@raspberrypi.com> X-Mailer: git-send-email 2.39.5 In-Reply-To: <20250122145349.7220-1-david.plowman@raspberrypi.com> References: <20250122145349.7220-1-david.plowman@raspberrypi.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" X-TUID: r1puB9Q1dEfW Resent-From: Kieran Bingham Resent-To: parsemail@patchwork.libcamera.org New controls are added to control the camera "sync" algorithm, which allows different cameras to synchronise their frames. For the time being, the controls are Raspberry Pi specific, though this is expected to change in future. Signed-off-by: David Plowman Reviewed-by: Naushir Patuck --- src/libcamera/control_ids_rpi.yaml | 113 +++++++++++++++++++++++++++++ 1 file changed, 113 insertions(+) diff --git a/src/libcamera/control_ids_rpi.yaml b/src/libcamera/control_ids_rpi.yaml index 7524c5d2..5c2721c0 100644 --- a/src/libcamera/control_ids_rpi.yaml +++ b/src/libcamera/control_ids_rpi.yaml @@ -58,4 +58,117 @@ controls: official libcamera API support for per-stream controls in the future. \sa ScalerCrop + + - SyncMode: + type: int32_t + direction: in + description: | + Enable or disable camera synchronisation ("sync") mode. + + When sync mode is enabled, a camera will synchronise frames temporally + with other cameras, either attached to the same device or a different + one. There should be one "server" device, which broadcasts timing + information to one or more "clients". Communication is one-way, from + server to clients only, and it is only clients that adjust their frame + timings to match the server. + + Sync mode requires all cameras to be running at (as far as possible) the + same fixed framerate. Clients may continue to make adjustments to keep + their cameras synchronised with the server for the duration of the + session, though any updates after the initial ones should remain small. + + \sa SyncReady + \sa SyncTimer + \sa SyncFrames + + enum: + - name: SyncModeOff + value: 0 + description: Disable sync mode. + - name: SyncModeServer + value: 1 + description: | + Enable sync mode, act as server. The server broadcasts timing + messages to any clients that are listening, so that the clients can + synchronise their camera frames with the server's. + - name: SyncModeClient + value: 2 + description: | + Enable sync mode, act as client. A client listens for any server + messages, and arranges for its camera frames to synchronise as + closely as possible with the server's. Many clients can listen out + for the same server. Clients can also be started ahead of any + servers, causing them merely to wait for the server to start. + + - SyncReady: + type: bool + direction: out + description: | + When using the camera synchronisation algorithm, the server broadcasts + timing information to the clients. This also includes the time (some + number of frames in the future, called the "ready time") at which the + server will signal its controlling application, using this control, to + start using the image frames. + + The client receives the "ready time" from the server, and will signal + its application to start using the frames at this same moment. + + While this control value is false, applications (on both client and + server) should continue to wait, and not use the frames. + + Once this value becomes true, it means that this is the first frame + where the server and its clients have agreed that they will both be + synchronised and that applications should begin consuming frames. + Thereafter, this control will continue to signal the value true for + the rest of the session. + + \sa SyncMode + \sa SyncTimer + \sa SyncFrames + + - SyncTimer: + type: int64_t + direction: out + description: | + This reports the amount of time, in microseconds, until the "ready + time", at which the server and client will signal their controlling + applications that the frames are now synchronised and should be + used. The value may be refined slightly over time, becoming more precise + as the "ready time" approaches. + + Servers always report this value, whereas clients will omit this control + until they have received a message from the server that enables them to + calculate it. + + Normally the value will start positive (the "ready time" is in the + future), and decrease towards zero, before becoming negative (the "ready + time" has elapsed). So there should be just one frame where the timer + value is, or is very close to, zero - the one for which the SyncReady + control becomes true. At this moment, the value indicates how closely + synchronised the client believes it is with the server. + + But note that if frames are being dropped, then the "near zero" valued + frame, or indeed any other, could be skipped. In these cases the timer + value allows an application to deduce that this has happened. + + \sa SyncMode + \sa SyncReady + \sa SyncFrames + + - SyncFrames: + type: int32_t + direction: in + description: | + The number of frames the server should wait, after enabling + SyncModeServer, before signalling (via the SyncReady control) that + frames should be used. This therefore determines the "ready time" for + all synchronised cameras. + + This control value should be set only for the device that is to act as + the server, before or at the same moment at which SyncModeServer is + enabled. + + \sa SyncMode + \sa SyncReady + \sa SyncTimer ... From patchwork Wed Jan 22 14:53:46 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Plowman X-Patchwork-Id: 22685 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by patchwork.libcamera.org (Postfix) with ESMTPS id D7F29BD808 for ; Thu, 30 Jan 2025 11:52:33 +0000 (UTC) Received: from pendragon.ideasonboard.com (cpc89244-aztw30-2-0-cust6594.18-1.cable.virginm.net [86.31.185.195]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 6AC9F886 for ; Thu, 30 Jan 2025 12:51:25 +0100 (CET) Authentication-Results: perceval.ideasonboard.com; dkim=permerror header.d=raspberrypi.com header.i=@raspberrypi.com header.a=rsa-sha1 header.s=google header.b=K0obU6lg; dkim-atps=neutral Delivered-To: kbingham@ideasonboard.com Received: from perceval.ideasonboard.com by perceval.ideasonboard.com with LMTP id MO1/JkUGkWeNRjQA4E0KoQ (envelope-from ) for ; Wed, 22 Jan 2025 15:52:53 +0100 Received: from mail-wm1-x329.google.com (mail-wm1-x329.google.com [IPv6:2a00:1450:4864:20::329]) by perceval.ideasonboard.com (Postfix) with ESMTPS id 847EDC16 for ; Wed, 22 Jan 2025 15:52:53 +0100 (CET) Received: by mail-wm1-x329.google.com with SMTP id 5b1f17b1804b1-436326dcb1cso49406445e9.0 for ; Wed, 22 Jan 2025 06:53:56 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; t=1737557635; x=1738162435; darn=ideasonboard.com; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=/Th3SY4pThmSp8bZ5uSgh71EJfe/F3FqF6YTxuYoO50=; b=K0obU6lgltexfCSJcFYheH14o3ov3PV3eCvMi3BoH4A+VQXNkz1bAoILmi21j/scii MBhW44yWBc6w8if58qx2vX1divQvpT3RHonOgkAOrEhEBaxnYHcJ5y/w7/avqB7CSxQI 3FE8YKZBNpwUVEa47FpZBDpnpo+aBnXK1SJz0chTKkFYvaNW2D/KuhRoOGYNi/avjG6q HRPpO+gY+3Jq4BG3ZPKfI/NiHDfnpx/pmAnnvwF/kKTwBIXdd2UnmWqcDh3KHRFgFOZT /76tUuImPxuipGWmhZeo1mKqFVWj4fjYoZtnAptqCyhnlQkq6vSngudhzeztVt7jp1SP 0d7A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1737557635; x=1738162435; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=/Th3SY4pThmSp8bZ5uSgh71EJfe/F3FqF6YTxuYoO50=; b=hW7BxMVmOC9LrETCg51dOncOW4/Uyr6GmCe33Q57dQz6ceheNyShZZwHLFJOdQoeTO 0meyNAsZAE+G/KDUjfSW9ZM9xxvwSBRjRBa2W50CmLlGCCdingbkIWGkvroBVff/fAB/ MW5ao3X25tT2E4IkWZ6EDiKmUpQrdFb1RtI6lI8/urcIhrpjMFc8Sn2mEIhtgkTpL1gC 9ahmHd/jBP9LGHASYe9AfKiwZV8L+dRpEyutmbvRWhOu1W1PsnjFxZybs4xY8/ZtIGBY AnE+pPjJBOxV2A8Ces4uNqLGyIJDKuJoWn9PMHik94MUOmsBpRE5sxJgPUy3XpQxFMyR iWDg== X-Forwarded-Encrypted: i=1; =?utf-8?q?AJvYcCV5U+7RxCS9SazbwgmVq2J6zTdmN2ex?= =?utf-8?q?Q1hI7TsZjPWta5GDRa8Js2bo3XhxMjOs6WvD1t0yUL0IFiZMtS6PCw?= =?utf-8?q?=3D=3D=40ideasonboard=2Ecom?= X-Gm-Message-State: AOJu0YzPeEjeK6qrN6jGyWU+xp/zVw/lj1ZmXXgX0GCs6y9uvXVe7OD3 CQOBvHEhsKvNpvFRErr+/xACl7z4364+DKik+kHzNwdRwTOEX0eJ23YEmr+L0bg= X-Gm-Gg: ASbGncsctvXDukdrOhiOjmQEsm7J1P41bMK9L26QScyQb5z03tBnphaXMC28aOs3j+A P6Tz3QZG7Duedv774X8RfS90iNKOGKe0PSo7oPDxObnxQ2rge0oUkjwYdiRbSWlLdcUQamyIb+W KjxVtr0btIP8dP8+lg/6rLkgUiYVio7g1grJ+EZ9td6CLeec4B3i8e7ZRUfDVXi8YCJYyVCKt5a QnNQ/LkyJeS0vf/mcYe2clPqLdbmzEOFMPedCWCIWscmZfoLvPBMjTuz3a2T79mq9iDUBz0xsED MbV2SrsVAlxcipts87YZwqNtkA== X-Google-Smtp-Source: =?utf-8?q?AGHT+IE7Jc5DUGLCCBpThrzWHE90sZRjygCl0pvs7?= =?utf-8?q?j/LJG7u9axhNRJhYJlDh/IaeNcBVyVhuz1w0g=3D=3D?= X-Received: by 2002:a05:600c:1e02:b0:438:ad4d:cf09 with SMTP id 5b1f17b1804b1-438ad4dd134mr84433575e9.9.1737557635422; Wed, 22 Jan 2025 06:53:55 -0800 (PST) Received: from raspberrypi.pitowers.org ([2a00:1098:3142:1f:ffc9:aff6:7f7f:893b]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-438b66dc08bsm11551395e9.37.2025.01.22.06.53.54 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 22 Jan 2025 06:53:55 -0800 (PST) From: David Plowman To: libcamera-devel@lists.libcamera.org Cc: David Plowman , Kieran Bingham , Naushir Patuck Subject: [PATCH v4 4/7] pipeline: rpi: common: Add wallclock timestamp support Date: Wed, 22 Jan 2025 14:53:46 +0000 Message-Id: <20250122145349.7220-5-david.plowman@raspberrypi.com> X-Mailer: git-send-email 2.39.5 In-Reply-To: <20250122145349.7220-1-david.plowman@raspberrypi.com> References: <20250122145349.7220-1-david.plowman@raspberrypi.com> MIME-Version: 1.0 X-TUID: uwkgrETFaAFz Resent-From: Kieran Bingham Resent-To: parsemail@patchwork.libcamera.org A ClockRecovery object is added for derived classes to use, and wallclock timestamps are copied into the request metadata for applications. Wallclock timestamps are derived corresponding to the sensor timestamp, and made available to the base pipeline handler class and to IPAs. Signed-off-by: David Plowman Reviewed-by: Kieran Bingham Reviewed-by: Naushir Patuck --- src/libcamera/pipeline/rpi/common/pipeline_base.cpp | 5 +++++ src/libcamera/pipeline/rpi/common/pipeline_base.h | 3 +++ src/libcamera/pipeline/rpi/vc4/vc4.cpp | 10 ++++++++-- 3 files changed, 16 insertions(+), 2 deletions(-) diff --git a/src/libcamera/pipeline/rpi/common/pipeline_base.cpp b/src/libcamera/pipeline/rpi/common/pipeline_base.cpp index 4b147fdb..56f3c4df 100644 --- a/src/libcamera/pipeline/rpi/common/pipeline_base.cpp +++ b/src/libcamera/pipeline/rpi/common/pipeline_base.cpp @@ -686,6 +686,9 @@ int PipelineHandlerBase::start(Camera *camera, const ControlList *controls) return ret; } + /* A good moment to add an initial clock sample. */ + data->wallClockRecovery_.addSample(); + /* * Reset the delayed controls with the gain and exposure values set by * the IPA. @@ -1510,6 +1513,8 @@ void CameraData::fillRequestMetadata(const ControlList &bufferControls, Request { request->metadata().set(controls::SensorTimestamp, bufferControls.get(controls::SensorTimestamp).value_or(0)); + request->metadata().set(controls::FrameWallClock, + bufferControls.get(controls::FrameWallClock).value_or(0)); if (cropParams_.size()) { std::vector crops; diff --git a/src/libcamera/pipeline/rpi/common/pipeline_base.h b/src/libcamera/pipeline/rpi/common/pipeline_base.h index aae0c2f3..ca706bf3 100644 --- a/src/libcamera/pipeline/rpi/common/pipeline_base.h +++ b/src/libcamera/pipeline/rpi/common/pipeline_base.h @@ -20,6 +20,7 @@ #include "libcamera/internal/bayer_format.h" #include "libcamera/internal/camera.h" #include "libcamera/internal/camera_sensor.h" +#include "libcamera/internal/clock_recovery.h" #include "libcamera/internal/framebuffer.h" #include "libcamera/internal/media_device.h" #include "libcamera/internal/media_object.h" @@ -177,6 +178,8 @@ public: Config config_; + ClockRecovery wallClockRecovery_; + protected: void fillRequestMetadata(const ControlList &bufferControls, Request *request); diff --git a/src/libcamera/pipeline/rpi/vc4/vc4.cpp b/src/libcamera/pipeline/rpi/vc4/vc4.cpp index fd8d84b1..3d8c65c6 100644 --- a/src/libcamera/pipeline/rpi/vc4/vc4.cpp +++ b/src/libcamera/pipeline/rpi/vc4/vc4.cpp @@ -781,9 +781,15 @@ void Vc4CameraData::unicamBufferDequeue(FrameBuffer *buffer) auto [ctrl, delayContext] = delayedCtrls_->get(buffer->metadata().sequence); /* * Add the frame timestamp to the ControlList for the IPA to use - * as it does not receive the FrameBuffer object. + * as it does not receive the FrameBuffer object. Also derive a + * corresponding wallclock value. */ - ctrl.set(controls::SensorTimestamp, buffer->metadata().timestamp); + wallClockRecovery_.addSample(); + uint64_t sensorTimestamp = buffer->metadata().timestamp; + uint64_t wallClockTimestamp = wallClockRecovery_.getOutput(sensorTimestamp / 1000); + + ctrl.set(controls::SensorTimestamp, sensorTimestamp); + ctrl.set(controls::FrameWallClock, wallClockTimestamp); bayerQueue_.push({ buffer, std::move(ctrl), delayContext }); } else { embeddedQueue_.push(buffer); From patchwork Wed Jan 22 14:53:47 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Plowman X-Patchwork-Id: 22613 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 3E9C5C3319 for ; Wed, 22 Jan 2025 14:54:09 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id E982C68567; Wed, 22 Jan 2025 15:54:07 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="lC/b6pRD"; dkim-atps=neutral Received: from mail-wr1-x42c.google.com (mail-wr1-x42c.google.com [IPv6:2a00:1450:4864:20::42c]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id E09FA68559 for ; Wed, 22 Jan 2025 15:53:56 +0100 (CET) Received: by mail-wr1-x42c.google.com with SMTP id ffacd0b85a97d-38634c35129so5941096f8f.3 for ; Wed, 22 Jan 2025 06:53:56 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; t=1737557636; x=1738162436; darn=lists.libcamera.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=dZRsfHRIhVt9X4suCupUrPLxrY9XKlNXZm04oJrTHek=; b=lC/b6pRDQc5GBYRQxFMdA3LuenWBkRtCRnnn9klPHISXSJbFCJDI9hBysmFRJqqiag 0XXdcfhX8zJm8Q53sfaJX5mwtkErcCyqnDT7ah2tiV05O0xi5EsfMIPFobzsl63G1l1w 9VKPRKBOPPZc/G7D76H4GTOpX6uPD5+KPlts04bjWcg875C1jEHS9uEkDahvJpHE7FRs HTNRYFQBZw5gHM35UzNK9yzrI3VTDt+d8j+OjpN1Sf4p1rgM6Z3w79dzqQFMFhpafHGe B3fd4yUe73mzyUizLTuslFwCSIdR3bBTibdcf0B3y1AvghQYr931cuEP15L4sNWHz6we usVQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1737557636; x=1738162436; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=dZRsfHRIhVt9X4suCupUrPLxrY9XKlNXZm04oJrTHek=; b=AvBQe6sw/aIlVuNOqcT24WGKuG3blgVR59JgkPwJNI+MxWocBmmvQCGJ+6fOHtdZy5 EQbgoaXUNXhKgC0naFR9SLpdrhO41MzboRuDb2ucgYGqkJzMIreUfTq2YXcIzv7y75aV ZiCI8CkFfnSW9W+QcwHCkd485j46tzLFdxvzH8hAqxXyFX7PyL0M9SNK7o+2pm0ZKndb gcUEX+KLNy4jyQchylA96543RYvnaV+6bxYNOIR2tiM6QRy7oBVbgzOtN13+Z+36SYSl osjuRskYIIzAaUhoW5bCqkfMJfQEUlTJNiwsDOAP9q55k1HFl1JoA6upznW36zrKetZA aBSQ== X-Gm-Message-State: AOJu0YxsO3WbpFKTdOoy6fbkT7yNwQYLfaqlo+e5o3ma1lZl5IYOUymj R/vCwacT4daEGNg436927h9xUpARUlVVmxA3M01Wo2IPlL8EpsuHuLaYvAYoL6E7Ms8SOzZeX6B s X-Gm-Gg: ASbGncskZg21wIMvl46r3joKM/W+ii7UnVPk9AICgFlv05aC4T44MihR/QsGx+7WZnm 7CaTOeA59LbufdOFauYhJlcipr8L+ST5b0uk3IRhTVMzFVYkjy/KV3gjuGNnHvgpW7oPBZDHJ3y 7WTzj5iurGo7YfV1Gd4dSz9VBw0r1Hgyet47lLpCeiz9VGIp/OPfv9cdDSZViIxJr65lerwbVxj WvHGtIQUJ/FI9jSh32BhHRECfMtIIPE+5E4OT7duNzdhhyvBk25HZXIfSLp940g6lP9ySsnZt4D qgW3afrssMXiIXVFwinaOTYvcw== X-Google-Smtp-Source: AGHT+IEm1yvPzu9FMPWb+e6Pqx1T79DY1cxylGpCoBXikxCqCCijptf6vT9/iW6s60rPQjVFjIsudg== X-Received: by 2002:adf:f141:0:b0:38a:418e:1171 with SMTP id ffacd0b85a97d-38bf58e88admr14678658f8f.37.1737557636073; Wed, 22 Jan 2025 06:53:56 -0800 (PST) Received: from raspberrypi.pitowers.org ([2a00:1098:3142:1f:ffc9:aff6:7f7f:893b]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-438b66dc08bsm11551395e9.37.2025.01.22.06.53.55 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 22 Jan 2025 06:53:55 -0800 (PST) From: David Plowman To: libcamera-devel@lists.libcamera.org Cc: Naushir Patuck , David Plowman Subject: [PATCH v4 5/7] ipa: rpi: Add base classes and plumbing for sync algorithm Date: Wed, 22 Jan 2025 14:53:47 +0000 Message-Id: <20250122145349.7220-6-david.plowman@raspberrypi.com> X-Mailer: git-send-email 2.39.5 In-Reply-To: <20250122145349.7220-1-david.plowman@raspberrypi.com> References: <20250122145349.7220-1-david.plowman@raspberrypi.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" From: Naushir Patuck We add a base class for a "sync algorithm", and define its inputs and outputs in the SyncStatus class. We add the necessary plumbing to the base IPA code so as to arrange for the necessary parameters to be made available to such an algorithm, and also to handle the return values, passing them back as necessary to the pipeline handler. Signed-off-by: Naushir Patuck Signed-off-by: David Plowman --- src/ipa/rpi/common/ipa_base.cpp | 61 +++++++++++++++++++++++-- src/ipa/rpi/common/ipa_base.h | 4 +- src/ipa/rpi/controller/sync_algorithm.h | 31 +++++++++++++ src/ipa/rpi/controller/sync_status.h | 27 +++++++++++ 4 files changed, 118 insertions(+), 5 deletions(-) create mode 100644 src/ipa/rpi/controller/sync_algorithm.h create mode 100644 src/ipa/rpi/controller/sync_status.h diff --git a/src/ipa/rpi/common/ipa_base.cpp b/src/ipa/rpi/common/ipa_base.cpp index bd3c2200..528e163c 100644 --- a/src/ipa/rpi/common/ipa_base.cpp +++ b/src/ipa/rpi/common/ipa_base.cpp @@ -28,6 +28,8 @@ #include "controller/lux_status.h" #include "controller/sharpen_algorithm.h" #include "controller/statistics.h" +#include "controller/sync_algorithm.h" +#include "controller/sync_status.h" namespace libcamera { @@ -81,6 +83,8 @@ const ControlInfoMap::Map ipaControls{ { &controls::Sharpness, ControlInfo(0.0f, 16.0f, 1.0f) }, { &controls::ScalerCrop, ControlInfo(Rectangle{}, Rectangle(65535, 65535, 65535, 65535), Rectangle{}) }, { &controls::FrameDurationLimits, ControlInfo(INT64_C(33333), INT64_C(120000)) }, + { &controls::rpi::SyncMode, ControlInfo(controls::rpi::SyncModeValues) }, + { &controls::rpi::SyncFrames, ControlInfo(1, 1000000, 100) }, { &controls::draft::NoiseReductionMode, ControlInfo(controls::draft::NoiseReductionModeValues) }, { &controls::rpi::StatsOutputEnable, ControlInfo(false, true, false) }, }; @@ -399,6 +403,7 @@ void IpaBase::prepareIsp(const PrepareParams ¶ms) rpiMetadata.clear(); fillDeviceStatus(params.sensorControls, ipaContext); + fillSyncParams(params, ipaContext); if (params.buffers.embedded) { /* @@ -497,10 +502,24 @@ void IpaBase::processStats(const ProcessParams ¶ms) helper_->process(statistics, rpiMetadata); controller_.process(statistics, &rpiMetadata); + /* Send any sync algorithm outputs back to the pipeline handler */ + Duration offset(0s); + struct SyncStatus syncStatus; + if (rpiMetadata.get("sync.status", syncStatus) == 0) { + if (minFrameDuration_ != maxFrameDuration_) + LOG(IPARPI, Error) << "Sync algorithm enabled with variable framerate. " + << minFrameDuration_ << " " << maxFrameDuration_; + offset = syncStatus.frameDurationOffset; + + libcameraMetadata_.set(controls::rpi::SyncReady, syncStatus.ready); + if (syncStatus.timerKnown) + libcameraMetadata_.set(controls::rpi::SyncTimer, syncStatus.timerValue); + } + struct AgcStatus agcStatus; if (rpiMetadata.get("agc.status", agcStatus) == 0) { ControlList ctrls(sensorCtrls_); - applyAGC(&agcStatus, ctrls); + applyAGC(&agcStatus, ctrls, offset); setDelayedControls.emit(ctrls, ipaContext); setCameraTimeoutValue(); } @@ -737,6 +756,7 @@ void IpaBase::applyControls(const ControlList &controls) using RPiController::ContrastAlgorithm; using RPiController::DenoiseAlgorithm; using RPiController::HdrAlgorithm; + using RPiController::SyncAlgorithm; /* Clear the return metadata buffer. */ libcameraMetadata_.clear(); @@ -1328,6 +1348,17 @@ void IpaBase::applyControls(const ControlList &controls) statsMetadataOutput_ = ctrl.second.get(); break; + case controls::rpi::SYNC_FRAMES: { + SyncAlgorithm *sync = dynamic_cast(controller_.getAlgorithm("sync")); + + if (sync) { + int frames = ctrl.second.get(); + if (frames > 0) + sync->setReadyFrame(frames); + } + break; + } + default: LOG(IPARPI, Warning) << "Ctrl " << controls::controls.at(ctrl.first)->name() @@ -1364,6 +1395,19 @@ void IpaBase::fillDeviceStatus(const ControlList &sensorControls, unsigned int i rpiMetadata_[ipaContext].set("device.status", deviceStatus); } +void IpaBase::fillSyncParams(const PrepareParams ¶ms, unsigned int ipaContext) +{ + RPiController::SyncAlgorithm *sync = dynamic_cast( + controller_.getAlgorithm("sync")); + if (!sync) + return; + + SyncParams syncParams; + syncParams.wallClock = *params.sensorControls.get(controls::FrameWallClock); + syncParams.sensorTimestamp = *params.sensorControls.get(controls::SensorTimestamp); + rpiMetadata_[ipaContext].set("sync.params", syncParams); +} + void IpaBase::reportMetadata(unsigned int ipaContext) { RPiController::Metadata &rpiMetadata = rpiMetadata_[ipaContext]; @@ -1542,14 +1586,22 @@ void IpaBase::applyFrameDurations(Duration minFrameDuration, Duration maxFrameDu * value possible. */ Duration maxExposureTime = Duration::max(); - helper_->getBlanking(maxExposureTime, minFrameDuration_, maxFrameDuration_); + auto [vblank, hblank] = helper_->getBlanking(maxExposureTime, minFrameDuration_, maxFrameDuration_); RPiController::AgcAlgorithm *agc = dynamic_cast( controller_.getAlgorithm("agc")); agc->setMaxExposureTime(maxExposureTime); + + RPiController::SyncAlgorithm *sync = dynamic_cast( + controller_.getAlgorithm("sync")); + if (sync) { + Duration duration = (mode_.height + vblank) * ((mode_.width + hblank) * 1.0s / mode_.pixelRate); + LOG(IPARPI, Debug) << "setting sync frame duration to " << duration; + sync->setFrameDuration(duration); + } } -void IpaBase::applyAGC(const struct AgcStatus *agcStatus, ControlList &ctrls) +void IpaBase::applyAGC(const struct AgcStatus *agcStatus, ControlList &ctrls, Duration frameDurationOffset) { const int32_t minGainCode = helper_->gainCode(mode_.minAnalogueGain); const int32_t maxGainCode = helper_->gainCode(mode_.maxAnalogueGain); @@ -1564,7 +1616,8 @@ void IpaBase::applyAGC(const struct AgcStatus *agcStatus, ControlList &ctrls) /* getBlanking might clip exposure time to the fps limits. */ Duration exposure = agcStatus->exposureTime; - auto [vblank, hblank] = helper_->getBlanking(exposure, minFrameDuration_, maxFrameDuration_); + auto [vblank, hblank] = helper_->getBlanking(exposure, minFrameDuration_ - frameDurationOffset, + maxFrameDuration_ - frameDurationOffset); int32_t exposureLines = helper_->exposureLines(exposure, helper_->hblankToLineLength(hblank)); diff --git a/src/ipa/rpi/common/ipa_base.h b/src/ipa/rpi/common/ipa_base.h index 1a811beb..b53d0bfb 100644 --- a/src/ipa/rpi/common/ipa_base.h +++ b/src/ipa/rpi/common/ipa_base.h @@ -95,9 +95,11 @@ private: void applyControls(const ControlList &controls); virtual void handleControls(const ControlList &controls) = 0; void fillDeviceStatus(const ControlList &sensorControls, unsigned int ipaContext); + void fillSyncParams(const PrepareParams ¶ms, unsigned int ipaContext); void reportMetadata(unsigned int ipaContext); void applyFrameDurations(utils::Duration minFrameDuration, utils::Duration maxFrameDuration); - void applyAGC(const struct AgcStatus *agcStatus, ControlList &ctrls); + void applyAGC(const struct AgcStatus *agcStatus, ControlList &ctrls, + utils::Duration frameDurationOffset = utils::Duration(0)); std::map buffers_; diff --git a/src/ipa/rpi/controller/sync_algorithm.h b/src/ipa/rpi/controller/sync_algorithm.h new file mode 100644 index 00000000..c242def6 --- /dev/null +++ b/src/ipa/rpi/controller/sync_algorithm.h @@ -0,0 +1,31 @@ +/* SPDX-License-Identifier: BSD-2-Clause */ +/* + * Copyright (C) 2024, Raspberry Pi Ltd + * + * sync_algorithm.h - Camera sync algorithm interface + */ +#pragma once + +#include + +#include "algorithm.h" + +namespace RPiController { + +class SyncAlgorithm : public Algorithm +{ +public: + enum class Mode { + Off, + Server, + Client, + }; + + SyncAlgorithm(Controller *controller) + : Algorithm(controller) {} + virtual void setFrameDuration(libcamera::utils::Duration frameDuration) = 0; + virtual void setReadyFrame(unsigned int frame) = 0; + virtual void setMode(Mode mode) = 0; +}; + +} /* namespace RPiController */ diff --git a/src/ipa/rpi/controller/sync_status.h b/src/ipa/rpi/controller/sync_status.h new file mode 100644 index 00000000..10f97502 --- /dev/null +++ b/src/ipa/rpi/controller/sync_status.h @@ -0,0 +1,27 @@ +/* SPDX-License-Identifier: BSD-2-Clause */ +/* + * Copyright (C) 2024, Raspberry Pi Ltd + * + * sync_status.h - Sync algorithm params and status structures + */ +#pragma once + +#include + +struct SyncParams { + /* Wall clock time for this frame */ + uint64_t wallClock; + /* Kernel timestamp for this frame */ + uint64_t sensorTimestamp; +}; + +struct SyncStatus { + /* Frame length correction to apply */ + libcamera::utils::Duration frameDurationOffset; + /* Whether the "ready time" has been reached */ + bool ready; + /* Time until the "ready time" */ + int64_t timerValue; + /* Whether timerValue is known (client has to wait for a server message) */ + bool timerKnown; +}; From patchwork Wed Jan 22 14:53:48 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Plowman X-Patchwork-Id: 22687 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by patchwork.libcamera.org (Postfix) with ESMTPS id 09E9DBD808 for ; Thu, 30 Jan 2025 11:53:04 +0000 (UTC) Received: from pendragon.ideasonboard.com (cpc89244-aztw30-2-0-cust6594.18-1.cable.virginm.net [86.31.185.195]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 8B0AB886 for ; Thu, 30 Jan 2025 12:51:55 +0100 (CET) Authentication-Results: perceval.ideasonboard.com; dkim=permerror header.d=raspberrypi.com header.i=@raspberrypi.com header.a=rsa-sha1 header.s=google header.b=OxUYv8dI; dkim-atps=neutral Delivered-To: kbingham@ideasonboard.com Received: from perceval.ideasonboard.com by perceval.ideasonboard.com with LMTP id 2AU9DVUGkWe3RjQA4E0KoQ (envelope-from ) for ; Wed, 22 Jan 2025 15:53:09 +0100 Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by perceval.ideasonboard.com (Postfix) with ESMTPS id 0B0CC1081; Wed, 22 Jan 2025 15:53:09 +0100 (CET) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 8ED4C6856A; Wed, 22 Jan 2025 15:54:10 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="OxUYv8dI"; dkim-atps=neutral Received: from mail-wm1-x332.google.com (mail-wm1-x332.google.com [IPv6:2a00:1450:4864:20::332]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 9D6C06855F for ; Wed, 22 Jan 2025 15:53:57 +0100 (CET) Received: by mail-wm1-x332.google.com with SMTP id 5b1f17b1804b1-4361f664af5so80664165e9.1 for ; Wed, 22 Jan 2025 06:53:57 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; t=1737557637; x=1738162437; darn=lists.libcamera.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=FDvnN7ryORqQHcoAh5tIXt4yh5I/D3MEokFNcVWq958=; b=OxUYv8dIe59obmcvhMfm6epzWw7rYCiwo4UKK20Bmu8mdwPHxt3+W/E86N/WYIQbRb MTRfGpYx39reSGL5YZQQ6bjCLJcDF6lLdcefr8a9SECLVR/NgOenhd96ByPdzPxLpHxg Sh6gB1OEXkJ+OrBGtUnyKHB9+Su0SOb2B6N+NAyKJWWbBM6p4Oxto11qQh48cIYGfQKK GgmOkmSWwySHYnZv9d+4KGEqSBiVS693Cub2kmwfhkIxOJ3OEqAH8tZ7cS+xYakv8Ch0 fwaGLjwChgIhKspcf7KYFnU8jJR2/NX/AkfyYVvnKDF8oJElzoGTfhd1kQXH6aLGZheM CfPQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1737557637; x=1738162437; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=FDvnN7ryORqQHcoAh5tIXt4yh5I/D3MEokFNcVWq958=; b=Qh00Y/xUl5W5BJFfvHxgft/ALp3O5LVmW/LXey6NDPCRm/MMrqbn2FfoNnFm/13mlV 63NkWDAz/4AglAMbiK1NBQoCkBZl4I+9kdSqpjRLrshgfhCGre01hfrL1Ni3HhR8SVwY wAVRlKHPoWOBJHXXD+fgCHze98OPRPc0HE1OG0vD707syWcWurhC04DOJ7T0J6w1y2rB +cP9HIoZy4iz9n3SD9jpCd7BRvnaqkZ6mGhWvaMyyp7/7hhN40KhCGSdpCCuw7F+77AE f0DTZ0bHq2lk7FP7veMJkuf2Vl74lKn9i1Duf7emD4CAqNrC29MOYUbS1Ws+zvTFvIqr DG6A== X-Gm-Message-State: AOJu0YyZl7C8pQThPAzazB34715zjZoAz/3R/wvMjshHq0PUKa1KQ1Hm uBDFkKXpRiD5Ura3P1FFA/ytiDqdTJWImEf/d6L2hR/AioQhJk/Z4cja4vFOUzl/5D5D9u04nuo b X-Gm-Gg: ASbGnctWS5ixcS45lAoah1zRj+n37+MyfboIFYnptdJZVEtn7VO01um+LvPZ2ryCIs4 qe9v1VWjrUR8qzZPcAVZC37wCNNX9vaJrj1Gstyvp7eFj2mvIx2CxFD5V8EF0qO3U+Nrxwvl6xC hkvUhOaHlLVnEurWECsq7wr+P849Fysyr80luuwWLgtTT4A6vMG6Y/jLndv+XfF+i/lW9PskmOO o+t7Kc35NcbDVE36W+Uq7Ydd/MTpCuGV8cR902wwm8g3i186OCz9J1gPFsU4+uCaUzbaUSx1lfY 76OR5BJn+rwvCcYocRQF03PpTw== X-Google-Smtp-Source: =?utf-8?q?AGHT+IHAr4swNtEMo57x1rOuUCjo0oK5q7/nE2ALz?= =?utf-8?q?JyesrRVQu159QuWxGSD/Pv8XjUUly/1aOQewg=3D=3D?= X-Received: by 2002:a05:600c:3b02:b0:434:a781:f5d5 with SMTP id 5b1f17b1804b1-4389143ba53mr213022325e9.30.1737557636728; Wed, 22 Jan 2025 06:53:56 -0800 (PST) Received: from raspberrypi.pitowers.org ([2a00:1098:3142:1f:ffc9:aff6:7f7f:893b]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-438b66dc08bsm11551395e9.37.2025.01.22.06.53.56 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 22 Jan 2025 06:53:56 -0800 (PST) From: David Plowman To: libcamera-devel@lists.libcamera.org Cc: David Plowman , Arsen Mikovic , Naushir Patuck Subject: [PATCH v4 6/7] ipa: rpi: sync: Add an implementation of the camera sync algorithm Date: Wed, 22 Jan 2025 14:53:48 +0000 Message-Id: <20250122145349.7220-7-david.plowman@raspberrypi.com> X-Mailer: git-send-email 2.39.5 In-Reply-To: <20250122145349.7220-1-david.plowman@raspberrypi.com> References: <20250122145349.7220-1-david.plowman@raspberrypi.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" X-TUID: aDncQKUh8YKg Resent-From: Kieran Bingham Resent-To: parsemail@patchwork.libcamera.org In this implementation, the server sends data packets out onto the network every 30 frames or so. Clients listening for this packet will send frame length deltas back to the pipeline handler to match the synchronisation of the server. We use wallclock timestamps, passed to us from the pipeline handler, that have been de-jittered appropriately, meaning that the synchronisation will actually work across networked devices. When the server's advertised "ready time" is reached, both client and server will signal this through metadata back to their respective controlling applications. Signed-off-by: David Plowman Signed-off-by: Arsen Mikovic Signed-off-by: Naushir Patuck --- src/ipa/rpi/controller/meson.build | 1 + src/ipa/rpi/controller/rpi/sync.cpp | 329 ++++++++++++++++++++++++++++ src/ipa/rpi/controller/rpi/sync.h | 68 ++++++ 3 files changed, 398 insertions(+) create mode 100644 src/ipa/rpi/controller/rpi/sync.cpp create mode 100644 src/ipa/rpi/controller/rpi/sync.h diff --git a/src/ipa/rpi/controller/meson.build b/src/ipa/rpi/controller/meson.build index 74b74888..dde4ac12 100644 --- a/src/ipa/rpi/controller/meson.build +++ b/src/ipa/rpi/controller/meson.build @@ -23,6 +23,7 @@ rpi_ipa_controller_sources = files([ 'rpi/saturation.cpp', 'rpi/sdn.cpp', 'rpi/sharpen.cpp', + 'rpi/sync.cpp', 'rpi/tonemap.cpp', ]) diff --git a/src/ipa/rpi/controller/rpi/sync.cpp b/src/ipa/rpi/controller/rpi/sync.cpp new file mode 100644 index 00000000..39bf5de7 --- /dev/null +++ b/src/ipa/rpi/controller/rpi/sync.cpp @@ -0,0 +1,329 @@ +/* SPDX-License-Identifier: BSD-2-Clause */ +/* + * Copyright (C) 2024, Raspberry Pi Ltd + * + * sync.cpp - sync algorithm + */ +#include "sync.h" + +#include +#include +#include +#include +#include + +#include + +#include + +#include "sync_status.h" + +using namespace std; +using namespace std::chrono_literals; +using namespace RPiController; +using namespace libcamera; + +LOG_DEFINE_CATEGORY(RPiSync) + +#define NAME "rpi.sync" + +Sync::Sync(Controller *controller) + : SyncAlgorithm(controller), mode_(Mode::Off), socket_(-1), frameDuration_(0s), frameCount_(0) +{ +} + +Sync::~Sync() +{ + if (socket_ >= 0) + close(socket_); +} + +char const *Sync::name() const +{ + return NAME; +} + +/* This reads from json file and intitiaises server and client */ +int Sync::read(const libcamera::YamlObject ¶ms) +{ + const char *kDefaultGroup = "239.255.255.250"; + constexpr unsigned int kDefaultPort = 10000; + constexpr unsigned int kDefaultSyncPeriod = 30; + constexpr unsigned int kDefaultReadyFrame = 100; + constexpr unsigned int kDefaultMinAdjustment = 50; + + /* Socket on which to communicate. */ + group_ = params["group"].get(kDefaultGroup); + port_ = params["port"].get(kDefaultPort); + /* Send a sync message every this many frames. */ + syncPeriod_ = params["sync_period"].get(kDefaultSyncPeriod); + /* Application will be told we're ready after this many frames. */ + readyFrame_ = params["ready_frame"].get(kDefaultReadyFrame); + /* Don't change client frame length unless the change exceeds this amount (microseconds). */ + minAdjustment_ = params["min_adjustment"].get(kDefaultMinAdjustment); + + return 0; +} + +void Sync::initialiseSocket() +{ + socket_ = socket(AF_INET, SOCK_DGRAM, 0); + if (socket_ < 0) { + LOG(RPiSync, Error) << "Unable to create socket"; + return; + } + + memset(&addr_, 0, sizeof(addr_)); + addr_.sin_family = AF_INET; + addr_.sin_addr.s_addr = mode_ == Mode::Client ? htonl(INADDR_ANY) : inet_addr(group_.c_str()); + addr_.sin_port = htons(port_); + + if (mode_ == Mode::Client) { + /* Set to non-blocking. */ + int flags = fcntl(socket_, F_GETFL, 0); + fcntl(socket_, F_SETFL, flags | O_NONBLOCK); + + unsigned int en = 1; + if (setsockopt(socket_, SOL_SOCKET, SO_REUSEADDR, &en, sizeof(en)) < 0) { + LOG(RPiSync, Error) << "Unable to set socket options"; + goto err; + } + + struct ip_mreq mreq {}; + mreq.imr_multiaddr.s_addr = inet_addr(group_.c_str()); + mreq.imr_interface.s_addr = htonl(INADDR_ANY); + if (setsockopt(socket_, IPPROTO_IP, IP_ADD_MEMBERSHIP, &mreq, sizeof(mreq)) < 0) { + LOG(RPiSync, Error) << "Unable to set socket options"; + goto err; + } + + if (bind(socket_, (struct sockaddr *)&addr_, sizeof(addr_)) < 0) { + LOG(RPiSync, Error) << "Unable to bind client socket"; + goto err; + } + } + + return; + +err: + close(socket_); + socket_ = -1; +} + +void Sync::switchMode([[maybe_unused]] CameraMode const &cameraMode, [[maybe_unused]] Metadata *metadata) +{ + /* + * A mode switch means the camera has stopped, so synchronisation will be lost. + * Reset all the internal state so that we start over. + */ + reset(); +} + +/* + * Camera sync algorithm. + * Server - there is a single server that sends framerate timing information over the network to any + * clients that are listening. It also signals when it will send a "everything is synchronised, now go" + * message back to the algorithm. + * Client - there may be many clients, either on the same Pi or different ones. They match their + * framerates to the server, and indicate when to "go" at the same instant as the server. + */ +void Sync::process([[maybe_unused]] StatisticsPtr &stats, Metadata *imageMetadata) +{ + SyncPayload payload; + SyncParams local{}; + SyncStatus status{}; + bool timerKnown = true; + + if (mode_ == Mode::Off) + return; + + if (!frameDuration_) { + LOG(RPiSync, Error) << "Sync frame duration not set!"; + return; + } + + if (socket_ < 0) { + initialiseSocket(); + + if (socket_ < 0) + return; + + /* + * For the client, flush anything in the socket. It might be stale from a previous sync run, + * or we might get another packet in a frame to two before the adjustment caused by this (old) + * packet, although correct, had taken effect. So this keeps things simpler. + */ + if (mode_ == Mode::Client) { + socklen_t addrlen = sizeof(addr_); + int ret = 0; + while (ret >= 0) + ret = recvfrom(socket_, &payload, sizeof(payload), 0, (struct sockaddr *)&addr_, &addrlen); + } + } + + imageMetadata->get("sync.params", local); + + /* The wallclock has already been de-jittered for us. */ + uint64_t wallClockFrameTimestamp = local.wallClock; + + /* + * This is the headline frame duration in microseconds as programmed into the sensor. Strictly, + * the sensor might not quite match the system clock, but this shouldn't matter for the calculations + * we'll do with it, unless it's a very very long way out! + */ + uint32_t frameDuration = frameDuration_.get(); + + /* Timestamps tell us if we've dropped any frames, but we still want to count them. */ + int droppedFrames = 0; + if (frameCount_) { + /* + * Round down here, because frameCount_ gets incremented at the end of the function. Also + * ensure droppedFrames can't go negative. It shouldn't, but things would go badly wrong + * if it did. + */ + wallClockFrameTimestamp = std::max(wallClockFrameTimestamp, lastWallClockFrameTimestamp_ + frameDuration / 2); + droppedFrames = (wallClockFrameTimestamp - lastWallClockFrameTimestamp_ - frameDuration / 2) / frameDuration; + frameCount_ += droppedFrames; + } + + if (mode_ == Mode::Server) { + /* + * Server sends a packet every syncPeriod_ frames, or as soon after as possible (if any + * frames were dropped). + */ + serverFrameCountPeriod_ += droppedFrames; + + /* + * The client may want a better idea of the true frame duration. Any error would feed straight + * into the correction term because of how it uses it to get the "nearest" frame. + */ + if (frameCount_ == 0) + frameDurationEstimated_ = frameDuration; + else { + double diff = (wallClockFrameTimestamp - lastWallClockFrameTimestamp_) / (1 + droppedFrames); + int N = std::min(frameCount_, 99U); + frameDurationEstimated_ = frameCount_ == 1 ? diff : (N * frameDurationEstimated_ + diff) / (N + 1); + } + + /* Calculate frames remaining, and therefore "time left until ready". */ + int framesRemaining = readyFrame_ - frameCount_; + uint64_t wallClockReadyTime = wallClockFrameTimestamp + (int64_t)framesRemaining * frameDurationEstimated_; + + if (serverFrameCountPeriod_ >= syncPeriod_) { + serverFrameCountPeriod_ = 0; + + payload.frameDuration = frameDurationEstimated_ + .5; /* round to nearest */ + payload.wallClockFrameTimestamp = wallClockFrameTimestamp; + payload.wallClockReadyTime = wallClockReadyTime; + + LOG(RPiSync, Debug) << "Send packet (frameNumber " << frameCount_ << "):"; + LOG(RPiSync, Debug) << " frameDuration " << payload.frameDuration; + LOG(RPiSync, Debug) << " wallClockFrameTimestamp " << wallClockFrameTimestamp + << " (" << wallClockFrameTimestamp - lastWallClockFrameTimestamp_ << ")"; + LOG(RPiSync, Debug) << " wallClockReadyTime " << wallClockReadyTime; + + if (sendto(socket_, &payload, sizeof(payload), 0, (const sockaddr *)&addr_, sizeof(addr_)) < 0) + LOG(RPiSync, Error) << "Send error! " << strerror(errno); + } + + timerValue_ = static_cast(wallClockReadyTime - wallClockFrameTimestamp); + if (!syncReady_ && wallClockFrameTimestamp + frameDurationEstimated_ / 2 > wallClockReadyTime) { + syncReady_ = true; + LOG(RPiSync, Info) << "*** Sync achieved! Difference " << timerValue_ << "us"; + } + + serverFrameCountPeriod_ += 1; + + } else if (mode_ == Mode::Client) { + uint64_t serverFrameTimestamp = 0; + + bool packetReceived = false; + while (true) { + socklen_t addrlen = sizeof(addr_); + int ret = recvfrom(socket_, &payload, sizeof(payload), 0, (struct sockaddr *)&addr_, &addrlen); + + if (ret < 0) + break; + packetReceived = (ret > 0); + clientSeenPacket_ = true; + + frameDurationEstimated_ = payload.frameDuration; + serverFrameTimestamp = payload.wallClockFrameTimestamp; + serverReadyTime_ = payload.wallClockReadyTime; + } + + if (packetReceived) { + uint64_t clientFrameTimestamp = wallClockFrameTimestamp; + int64_t clientServerDelta = clientFrameTimestamp - serverFrameTimestamp; + /* "A few frames ago" may have better matched the server's frame. Calculate when it was. */ + int framePeriodErrors = (clientServerDelta + frameDurationEstimated_ / 2) / frameDurationEstimated_; + int64_t clientFrameTimestampNearest = clientFrameTimestamp - framePeriodErrors * frameDurationEstimated_; + /* We must shorten a single client frame by this amount if it exceeds the minimum: */ + int32_t correction = clientFrameTimestampNearest - serverFrameTimestamp; + if (std::abs(correction) < minAdjustment_) + correction = 0; + + LOG(RPiSync, Debug) << "Received packet (frameNumber " << frameCount_ << "):"; + LOG(RPiSync, Debug) << " serverFrameTimestamp " << serverFrameTimestamp; + LOG(RPiSync, Debug) << " serverReadyTime " << serverReadyTime_; + LOG(RPiSync, Debug) << " clientFrameTimestamp " << clientFrameTimestamp; + LOG(RPiSync, Debug) << " clientFrameTimestampNearest " << clientFrameTimestampNearest + << " (" << framePeriodErrors << ")"; + LOG(RPiSync, Debug) << " correction " << correction; + + status.frameDurationOffset = correction * 1us; + } + + timerValue_ = static_cast(serverReadyTime_ - wallClockFrameTimestamp); + timerKnown = clientSeenPacket_; /* client must receive a packet before the timer value is correct */ + if (clientSeenPacket_ && !syncReady_ && wallClockFrameTimestamp + frameDurationEstimated_ / 2 > serverReadyTime_) { + syncReady_ = true; + LOG(RPiSync, Info) << "*** Sync achieved! Difference " << timerValue_ << "us"; + } + } + + lastWallClockFrameTimestamp_ = wallClockFrameTimestamp; + + status.ready = syncReady_; + status.timerValue = timerValue_; + status.timerKnown = timerKnown; + imageMetadata->set("sync.status", status); + frameCount_++; +} + +void Sync::reset() +{ + /* This resets the state so that the synchronisation procedure will start over. */ + syncReady_ = false; + frameCount_ = 0; + timerValue_ = 0; + serverFrameCountPeriod_ = 0; + serverReadyTime_ = 0; + clientSeenPacket_ = false; +} + +void Sync::setMode(Mode mode) +{ + mode_ = mode; + + /* Another "sync session" can be started by turning it off and on again. */ + if (mode == Mode::Off) + reset(); +} + +void Sync::setFrameDuration(libcamera::utils::Duration frameDuration) +{ + frameDuration_ = frameDuration; +}; + +void Sync::setReadyFrame(unsigned int frame) +{ + readyFrame_ = frame; +}; + +/* Register algorithm with the system. */ +static Algorithm *create(Controller *controller) +{ + return (Algorithm *)new Sync(controller); +} +static RegisterAlgorithm reg(NAME, &create); diff --git a/src/ipa/rpi/controller/rpi/sync.h b/src/ipa/rpi/controller/rpi/sync.h new file mode 100644 index 00000000..d3c79b7a --- /dev/null +++ b/src/ipa/rpi/controller/rpi/sync.h @@ -0,0 +1,68 @@ +/* SPDX-License-Identifier: BSD-2-Clause */ +/* + * Copyright (C) 2024, Raspberry Pi Ltd + * + * sync.h - sync algorithm + */ +#pragma once + +#include + +#include "../sync_algorithm.h" + +namespace RPiController { + +struct SyncPayload { + /* Frame duration in microseconds. */ + uint32_t frameDuration; + /* Server system (kernel) frame timestamp. */ + uint64_t systemFrameTimestamp; + /* Server wall clock version of the frame timestamp. */ + uint64_t wallClockFrameTimestamp; + /* Server system (kernel) sync time (the time at which frames are marked ready). */ + uint64_t systemReadyTime; + /* Server wall clock version of the sync time. */ + uint64_t wallClockReadyTime; +}; + +class Sync : public SyncAlgorithm +{ +public: + Sync(Controller *controller); + ~Sync(); + char const *name() const override; + int read(const libcamera::YamlObject ¶ms) override; + void setMode(Mode mode) override; + void initialiseSocket(); + void switchMode(CameraMode const &cameraMode, Metadata *metadata) override; + void process(StatisticsPtr &stats, Metadata *imageMetadata) override; + void setFrameDuration(libcamera::utils::Duration frameDuration) override; + void setReadyFrame(unsigned int frame) override; + +private: + void reset(); /* reset internal state and start over */ + + Mode mode_; /* server or client */ + std::string group_; /* IP group address for sync messages */ + uint16_t port_; /* port number for messages */ + uint32_t syncPeriod_; /* send a sync message every this many frames */ + uint32_t readyFrame_; /* tell the application we're ready after this many frames */ + uint32_t minAdjustment_; /* don't adjust the client frame length by less than this */ + + struct sockaddr_in addr_; + int socket_ = -1; + libcamera::utils::Duration frameDuration_; + unsigned int frameCount_; + bool syncReady_; + int64_t timerValue_ = 0; /* time until "ready time" */ + + double frameDurationEstimated_ = 0; /* estimate the true frame duration of the sensor */ + uint64_t lastWallClockFrameTimestamp_; /* wall clock timestamp of previous frame */ + + uint32_t serverFrameCountPeriod_ = 0; /* send the next packet when this reaches syncPeriod_ */ + + bool clientSeenPacket_ = false; /* whether the client has received a packet yet */ + uint64_t serverReadyTime_ = 0; /* the client's latest value for when the server will be "ready" */ +}; + +} /* namespace RPiController */ From patchwork Wed Jan 22 14:53:49 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Plowman X-Patchwork-Id: 22688 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by patchwork.libcamera.org (Postfix) with ESMTPS id F27A4BD808 for ; Thu, 30 Jan 2025 11:53:13 +0000 (UTC) Received: from pendragon.ideasonboard.com (cpc89244-aztw30-2-0-cust6594.18-1.cable.virginm.net [86.31.185.195]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 7FE0A886 for ; Thu, 30 Jan 2025 12:52:05 +0100 (CET) Authentication-Results: perceval.ideasonboard.com; dkim=permerror header.d=raspberrypi.com header.i=@raspberrypi.com header.a=rsa-sha1 header.s=google header.b=BrmCdtFG; dkim-atps=neutral Delivered-To: kbingham@ideasonboard.com Received: from perceval.ideasonboard.com by perceval.ideasonboard.com with LMTP id oLeDJFYGkWcsRzQA4E0KoQ (envelope-from ) for ; Wed, 22 Jan 2025 15:53:10 +0100 Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by perceval.ideasonboard.com (Postfix) with ESMTPS id 6253CF91; Wed, 22 Jan 2025 15:53:10 +0100 (CET) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id B9D2B68567; Wed, 22 Jan 2025 15:54:11 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="BrmCdtFG"; dkim-atps=neutral Received: from mail-wm1-x333.google.com (mail-wm1-x333.google.com [IPv6:2a00:1450:4864:20::333]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 058BD68562 for ; Wed, 22 Jan 2025 15:53:58 +0100 (CET) Received: by mail-wm1-x333.google.com with SMTP id 5b1f17b1804b1-436a03197b2so49277385e9.2 for ; Wed, 22 Jan 2025 06:53:58 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; t=1737557637; x=1738162437; darn=lists.libcamera.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=/Dy6Hx3kbtTcgF/PyAd3hSdgM8hdg0Y6+QQND9a6teU=; b=BrmCdtFGMXYAZ+QsxR43ORvgkf4aVE5zXnIgpnIMQaK2aLZYZnGYZRo+tUAPbkKnr0 S1SqYB8YVDg5s4+hR6H9hjd70lpfvu6yN7uZFVB3a+M8eG+NuKPdPKQEKJAsCxj3kUes q05Hq9CwVP6hkAuLITX/m7rCHz0RQBObEN4GRXsp0a65Xx1xl4XgRLjzl5Hp3Ctfspoe q1WGoiIv5fsYAdSgGUc4LrKrf7KTRgHzP0HfiWgSITWLCmFzeRz9eHDwMfE7jxAJEYNZ 0pcOFKPTRem9lavKW4WrBNkqTB5NoUNqPoL4f0uOEqjxhx53tFnJxb1wXMV6iu9TtIgd vxTw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1737557637; x=1738162437; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=/Dy6Hx3kbtTcgF/PyAd3hSdgM8hdg0Y6+QQND9a6teU=; b=LgKiWpPcA9Q5lQMjbKEXBy29aKnT4bkinRw0niBuPQ+ZMwlb6D12x2LzCWLaA2snza mvk2ZWTBTaTXvM31CeQjPXZqLn+MTXcJ+RU1cOGEKp+8xcqnpJENykE2WXyJD0fUjrx0 v+gHESwY3Ef2Dn+M3lJjpJ1JodxTTk3ubvowBLo4EEm9BsP/APzAOojLiJ1bayy4D6A+ 9j9GOWOMi8EO1AudZgp497/egDcX+J7Y1KJ5+zvdysTsFjCh7vmL7uj+dInrj3c04MO2 rvQI5SR+42iGTjHjkYgj1fOJPRbuUN2akEaxm656UxczZLYPVlBvWqtZBwiOnxYr0hk4 7ybg== X-Gm-Message-State: AOJu0Yyb+aOurogTpxJVD6YvMkSl2JjmyrzbUaECwn1Qc+mCSxo9717E SqugMBKbPyeJfzojEoO/SQ2uMxEj7NNsh2Omqz99CV/7wg/GuxglQwOvrczerILcdStogiFVlgx S X-Gm-Gg: ASbGncvP9q9Z2etR5YTgvVBtdsOkckmsJ4M3RZcXPyyL49kentZpnJIy+OOOG5FbhmK byq77k0GKTO7neUKK9pOo4tIeJ6U3syM4gxvC0SDw0gC/NIxh46RArcY5qg+ak+Red/EwA6VoCI Q+hbGf2dvhC/DycE97N7ThveNZ4JNpOjvUK29G7J+u4+hz4RB5XjEysMwxw0whHuCILLqqnbNzo pCGEo5yB7/bQpQ+cg69Ac+HVHdPfpSDHPfiUQzPllTNlSjn3EirugbmtaX0z7KrUhZaDSbajxov wZd3VS5KPDeavksNfVxUK8haFQ== X-Google-Smtp-Source: =?utf-8?q?AGHT+IFMTFFLPc4Zh4f83boSBmFMZ/5Z7kUbW1ZwU?= =?utf-8?q?vBSJrYihTGLvNi5r4e+tqVRd2iw5kNg/w4lug=3D=3D?= X-Received: by 2002:a05:600c:3d97:b0:436:f960:3427 with SMTP id 5b1f17b1804b1-4389142745amr187008275e9.22.1737557637305; Wed, 22 Jan 2025 06:53:57 -0800 (PST) Received: from raspberrypi.pitowers.org ([2a00:1098:3142:1f:ffc9:aff6:7f7f:893b]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-438b66dc08bsm11551395e9.37.2025.01.22.06.53.56 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 22 Jan 2025 06:53:57 -0800 (PST) From: David Plowman To: libcamera-devel@lists.libcamera.org Cc: David Plowman , Naushir Patuck Subject: [PATCH v4 7/7] ipa: rpi: vc4: Update all tuning files for sync algorithm Date: Wed, 22 Jan 2025 14:53:49 +0000 Message-Id: <20250122145349.7220-8-david.plowman@raspberrypi.com> X-Mailer: git-send-email 2.39.5 In-Reply-To: <20250122145349.7220-1-david.plowman@raspberrypi.com> References: <20250122145349.7220-1-david.plowman@raspberrypi.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" X-TUID: WjNE82OR16Mx Resent-From: Kieran Bingham Resent-To: parsemail@patchwork.libcamera.org Default to all the standard parameter values. Signed-off-by: David Plowman Reviewed-by: Naushir Patuck --- src/ipa/rpi/vc4/data/imx219.json | 7 ++++++- src/ipa/rpi/vc4/data/imx219_noir.json | 7 ++++++- src/ipa/rpi/vc4/data/imx283.json | 7 ++++++- src/ipa/rpi/vc4/data/imx290.json | 7 ++++++- src/ipa/rpi/vc4/data/imx296.json | 7 ++++++- src/ipa/rpi/vc4/data/imx296_mono.json | 7 ++++++- src/ipa/rpi/vc4/data/imx378.json | 7 ++++++- src/ipa/rpi/vc4/data/imx477.json | 7 ++++++- src/ipa/rpi/vc4/data/imx477_noir.json | 7 ++++++- src/ipa/rpi/vc4/data/imx477_scientific.json | 7 ++++++- src/ipa/rpi/vc4/data/imx519.json | 7 ++++++- src/ipa/rpi/vc4/data/imx708.json | 7 ++++++- src/ipa/rpi/vc4/data/imx708_noir.json | 7 ++++++- src/ipa/rpi/vc4/data/imx708_wide.json | 7 ++++++- src/ipa/rpi/vc4/data/imx708_wide_noir.json | 7 ++++++- src/ipa/rpi/vc4/data/ov5647.json | 7 ++++++- src/ipa/rpi/vc4/data/ov5647_noir.json | 7 ++++++- src/ipa/rpi/vc4/data/se327m12.json | 7 ++++++- 18 files changed, 108 insertions(+), 18 deletions(-) diff --git a/src/ipa/rpi/vc4/data/imx219.json b/src/ipa/rpi/vc4/data/imx219.json index a020b12f..21f82dd2 100644 --- a/src/ipa/rpi/vc4/data/imx219.json +++ b/src/ipa/rpi/vc4/data/imx219.json @@ -690,6 +690,11 @@ } } } - } + }, + { + "rpi.sync": + { + } + } ] } diff --git a/src/ipa/rpi/vc4/data/imx219_noir.json b/src/ipa/rpi/vc4/data/imx219_noir.json index d8bc9639..a51a5387 100644 --- a/src/ipa/rpi/vc4/data/imx219_noir.json +++ b/src/ipa/rpi/vc4/data/imx219_noir.json @@ -624,6 +624,11 @@ } } } - } + }, + { + "rpi.sync": + { + } + } ] } diff --git a/src/ipa/rpi/vc4/data/imx283.json b/src/ipa/rpi/vc4/data/imx283.json index bfacecc8..d98f3f08 100644 --- a/src/ipa/rpi/vc4/data/imx283.json +++ b/src/ipa/rpi/vc4/data/imx283.json @@ -308,6 +308,11 @@ }, { "rpi.sharpen": { } - } + }, + { + "rpi.sync": + { + } + } ] } diff --git a/src/ipa/rpi/vc4/data/imx290.json b/src/ipa/rpi/vc4/data/imx290.json index 8f41bf51..b2757e29 100644 --- a/src/ipa/rpi/vc4/data/imx290.json +++ b/src/ipa/rpi/vc4/data/imx290.json @@ -209,6 +209,11 @@ } ] } - } + }, + { + "rpi.sync": + { + } + } ] } \ No newline at end of file diff --git a/src/ipa/rpi/vc4/data/imx296.json b/src/ipa/rpi/vc4/data/imx296.json index 8f24ce5b..222cb372 100644 --- a/src/ipa/rpi/vc4/data/imx296.json +++ b/src/ipa/rpi/vc4/data/imx296.json @@ -438,6 +438,11 @@ "strength": 1.0, "limit": 0.18 } - } + }, + { + "rpi.sync": + { + } + } ] } \ No newline at end of file diff --git a/src/ipa/rpi/vc4/data/imx296_mono.json b/src/ipa/rpi/vc4/data/imx296_mono.json index fe331569..bbf78b3b 100644 --- a/src/ipa/rpi/vc4/data/imx296_mono.json +++ b/src/ipa/rpi/vc4/data/imx296_mono.json @@ -235,6 +235,11 @@ "strength": 1.0, "limit": 0.18 } - } + }, + { + "rpi.sync": + { + } + } ] } \ No newline at end of file diff --git a/src/ipa/rpi/vc4/data/imx378.json b/src/ipa/rpi/vc4/data/imx378.json index 363b47e1..e4276732 100644 --- a/src/ipa/rpi/vc4/data/imx378.json +++ b/src/ipa/rpi/vc4/data/imx378.json @@ -422,6 +422,11 @@ }, { "rpi.sharpen": { } - } + }, + { + "rpi.sync": + { + } + } ] } \ No newline at end of file diff --git a/src/ipa/rpi/vc4/data/imx477.json b/src/ipa/rpi/vc4/data/imx477.json index fa25ee86..ece91db9 100644 --- a/src/ipa/rpi/vc4/data/imx477.json +++ b/src/ipa/rpi/vc4/data/imx477.json @@ -695,6 +695,11 @@ } } } - } + }, + { + "rpi.sync": + { + } + } ] } \ No newline at end of file diff --git a/src/ipa/rpi/vc4/data/imx477_noir.json b/src/ipa/rpi/vc4/data/imx477_noir.json index 472f33fe..13426ee3 100644 --- a/src/ipa/rpi/vc4/data/imx477_noir.json +++ b/src/ipa/rpi/vc4/data/imx477_noir.json @@ -651,6 +651,11 @@ } } } - } + }, + { + "rpi.sync": + { + } + } ] } diff --git a/src/ipa/rpi/vc4/data/imx477_scientific.json b/src/ipa/rpi/vc4/data/imx477_scientific.json index 9dc32eb1..de028986 100644 --- a/src/ipa/rpi/vc4/data/imx477_scientific.json +++ b/src/ipa/rpi/vc4/data/imx477_scientific.json @@ -483,6 +483,11 @@ }, { "rpi.sharpen": { } - } + }, + { + "rpi.sync": + { + } + } ] } \ No newline at end of file diff --git a/src/ipa/rpi/vc4/data/imx519.json b/src/ipa/rpi/vc4/data/imx519.json index ce194256..d7e33f9f 100644 --- a/src/ipa/rpi/vc4/data/imx519.json +++ b/src/ipa/rpi/vc4/data/imx519.json @@ -422,6 +422,11 @@ }, { "rpi.sharpen": { } - } + }, + { + "rpi.sync": + { + } + } ] } \ No newline at end of file diff --git a/src/ipa/rpi/vc4/data/imx708.json b/src/ipa/rpi/vc4/data/imx708.json index 4de6f079..35c60509 100644 --- a/src/ipa/rpi/vc4/data/imx708.json +++ b/src/ipa/rpi/vc4/data/imx708.json @@ -666,6 +666,11 @@ } } } - } + }, + { + "rpi.sync": + { + } + } ] } \ No newline at end of file diff --git a/src/ipa/rpi/vc4/data/imx708_noir.json b/src/ipa/rpi/vc4/data/imx708_noir.json index 7b7ee874..495a2a9e 100644 --- a/src/ipa/rpi/vc4/data/imx708_noir.json +++ b/src/ipa/rpi/vc4/data/imx708_noir.json @@ -765,6 +765,11 @@ } } } - } + }, + { + "rpi.sync": + { + } + } ] } \ No newline at end of file diff --git a/src/ipa/rpi/vc4/data/imx708_wide.json b/src/ipa/rpi/vc4/data/imx708_wide.json index 6f45aafc..077f740e 100644 --- a/src/ipa/rpi/vc4/data/imx708_wide.json +++ b/src/ipa/rpi/vc4/data/imx708_wide.json @@ -677,6 +677,11 @@ } } } - } + }, + { + "rpi.sync": + { + } + } ] } \ No newline at end of file diff --git a/src/ipa/rpi/vc4/data/imx708_wide_noir.json b/src/ipa/rpi/vc4/data/imx708_wide_noir.json index b9a5227e..3760925d 100644 --- a/src/ipa/rpi/vc4/data/imx708_wide_noir.json +++ b/src/ipa/rpi/vc4/data/imx708_wide_noir.json @@ -668,6 +668,11 @@ } } } - } + }, + { + "rpi.sync": + { + } + } ] } \ No newline at end of file diff --git a/src/ipa/rpi/vc4/data/ov5647.json b/src/ipa/rpi/vc4/data/ov5647.json index 40c6059c..d8241004 100644 --- a/src/ipa/rpi/vc4/data/ov5647.json +++ b/src/ipa/rpi/vc4/data/ov5647.json @@ -691,6 +691,11 @@ } } } - } + }, + { + "rpi.sync": + { + } + } ] } diff --git a/src/ipa/rpi/vc4/data/ov5647_noir.json b/src/ipa/rpi/vc4/data/ov5647_noir.json index 488b7119..accceb8b 100644 --- a/src/ipa/rpi/vc4/data/ov5647_noir.json +++ b/src/ipa/rpi/vc4/data/ov5647_noir.json @@ -407,6 +407,11 @@ }, { "rpi.sharpen": { } - } + }, + { + "rpi.sync": + { + } + } ] } \ No newline at end of file diff --git a/src/ipa/rpi/vc4/data/se327m12.json b/src/ipa/rpi/vc4/data/se327m12.json index 948169db..48512b50 100644 --- a/src/ipa/rpi/vc4/data/se327m12.json +++ b/src/ipa/rpi/vc4/data/se327m12.json @@ -427,6 +427,11 @@ "strength": 0.5, "limit": 0.5 } - } + }, + { + "rpi.sync": + { + } + } ] } \ No newline at end of file