From patchwork Thu May 7 12:20:34 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Plowman X-Patchwork-Id: 26672 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id B9592C32F6 for ; Thu, 7 May 2026 12:55:07 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 8CFEE63020; Thu, 7 May 2026 14:55:05 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="bWzQLUsp"; dkim-atps=neutral Received: from mail-wr1-x430.google.com (mail-wr1-x430.google.com [IPv6:2a00:1450:4864:20::430]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id E2B3D62010 for ; Thu, 7 May 2026 14:55:02 +0200 (CEST) Received: by mail-wr1-x430.google.com with SMTP id ffacd0b85a97d-44e1860558fso600771f8f.0 for ; Thu, 07 May 2026 05:55:02 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; t=1778158502; x=1778763302; darn=lists.libcamera.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=B/6c4x6aPmSj4hSBbdqWEJNfL9gt9CXnVpTwIUpnx7A=; b=bWzQLUsp53Wgdgey1p+mfNd46z3nO4Md6r3bNmAkU0QC219xdsoWEKCO6d6m/KuuXv 20TpWL4BMyBNnOa2eBPX1K50pZ6a+yR4egBdaNFHaGRlCXrU41ePcRjtZEI2GpZikcrv ipWO+vJCRB2v7rF/z8IMIcME2QmSlPeRqJxfQGeb6jJGwJsy1HrUmiwGKbTgp1vi4HIV x8EOd6LV/iXxJYy414VC4Nea+9ReY6e0zhZ5gNTnJ8zwH9Z7KWYUOk3qMIBc8OLSWRL9 WSF+paLQQsimt20jMmvbjR1VyQ8mG5WLq42151uSDLhQJp4i1KRQnB4r6mn0q0ngvhX+ HSVw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1778158502; x=1778763302; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=B/6c4x6aPmSj4hSBbdqWEJNfL9gt9CXnVpTwIUpnx7A=; b=k2qoByikWj6UVi2Y2VyIRF93jiBPIpz31/cB4D8wl73rlRcKx9XSXTfABa4aNH0yPy MPZPvxsfj3GlYKZ+N88BuIVchk4GahLoJhvH9IJPNgsUcYgyJZEaznjsqLC22aLk5TGW PolXiSf5q2rZr2khLxIDS50tH+/+zlERIo1G3YrGaVXicSvDnBE5Nh4y6X27wM42cvz9 FImljWqHuljZ1Ym5yWAoDWDt8n+zewL4bFSQZAFJMcfHsNKKaW9SK6sqO6No9ra1nZIR JtOVFQJpMtJvl8+A0AgK9Q8Lz/ac69sT1niPVTKaAV099OAqKqa0YVgmcpfSNtZT7r0Z ++aw== X-Gm-Message-State: AOJu0Yy4PQTL3kPCnZWW7G18EnagUuyVfB3AN5CTG1ObaOl2SxmI9UWM g6CNkeskYLJcDKQuWUJIPujdM0K/ccVsmhs2VSrt3a59SZRhsJUrZWI2clTzCYczsXiExLb6097 Zyc0v X-Gm-Gg: AeBDievDiG9L2gUJbLQWH9cASCFbzVx/xcZf2UFSSJyO0XV5BeVEB8Bfn5lT47unO/6 qYPJ2TzgC5bsoNXGlAoIQN36GB3oKjI21rImag4e+yCy4DJpQ/aUYcLvumYM7vFd862VGe3Ce56 0qTic2HoAU+VewlC8SR9NVa7oKTXSFo1qol8rtiptbVSXXupgY8f9ZryqoaKOA4k5O2YZxRvuk1 UQXsbctollDyJOeQ0DXcRBI7E9KTofXHqDT+eOLqZZbsmKEEmxErF+j+CmZorQ4XDwAvwbeBzvP m1cjmZWqLBQ78XiJNVpfbHPZjdF9mI7yRJrPG9/VWv1ZKvhe0agwICgt/bg+kToqA3Zppqo/pAJ ygSpQ00zfMOjEoTLnLltnxcI1re6UFrtOV07iAT5/KfPGpFHj+XLqG4yE9qIVebAsllNjReeEn0 As4WL3cYveVkjxqts+LaPOyI1loRk2lq7R70zDSD3T0E8KXNdyylPmzCrUaQZXdAsoj0D9eVQDQ fh/kGrvO0ziMwsxJeGCD5AMcsnQQymvoxW6P59q1iJHFemB3qqd X-Received: by 2002:a05:6000:2901:b0:43d:613:4036 with SMTP id ffacd0b85a97d-4515da9535bmr12535111f8f.37.1778158501943; Thu, 07 May 2026 05:55:01 -0700 (PDT) Received: from davidp-pi5.pitowers.org ([2a00:1098:3142:1f:88ea:c658:5b20:5e46]) by smtp.gmail.com with ESMTPSA id ffacd0b85a97d-450524833e1sm20396324f8f.2.2026.05.07.05.55.01 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 07 May 2026 05:55:01 -0700 (PDT) From: David Plowman To: libcamera-devel@lists.libcamera.org Cc: David Plowman , Naushir Patuck , Jacopo Mondi Subject: [PATCH v4 1/3] pipeline: rpi: Simplify delayed controls Date: Thu, 7 May 2026 13:20:34 +0100 Message-ID: <20260507125458.12140-2-david.plowman@raspberrypi.com> X-Mailer: git-send-email 2.47.3 In-Reply-To: <20260507125458.12140-1-david.plowman@raspberrypi.com> References: <20260507125458.12140-1-david.plowman@raspberrypi.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" The queueCount_ value is redundant and can be removed. The pipeline handler calls the push and applyControls functions in a an interleaved manner, so queueCount_ was merely tracking writeCount_, and is therefore unnecessary. Signed-off-by: David Plowman Reviewed-by: Naushir Patuck Reviewed-by: Jacopo Mondi --- .../pipeline/rpi/common/delayed_controls.cpp | 62 +++++++++++-------- .../pipeline/rpi/common/delayed_controls.h | 1 - 2 files changed, 36 insertions(+), 27 deletions(-) diff --git a/src/libcamera/pipeline/rpi/common/delayed_controls.cpp b/src/libcamera/pipeline/rpi/common/delayed_controls.cpp index 19c71946..b647174f 100644 --- a/src/libcamera/pipeline/rpi/common/delayed_controls.cpp +++ b/src/libcamera/pipeline/rpi/common/delayed_controls.cpp @@ -32,14 +32,15 @@ namespace RPi { * * Some sensor controls take effect with a delay as the sensor needs time to * adjust, for example exposure and analog gain. This is a helper class to deal - * with such controls and the intended users are pipeline handlers. + * with such controls. * - * The idea is to extend the concept of the buffer depth of a pipeline the - * application needs to maintain to also cover controls. Just as with buffer - * depth if the application keeps the number of requests queued above the - * control depth the controls are guaranteed to take effect for the correct - * request. The control depth is determined by the control with the greatest - * delay. + * The idea is to maintain a queue of controls that have been submitted. + * Whenever a new frame starts, we can "peak back" into this queue to send + * controls at the correct number of frames in advance to account for each + * control's delay. + * + * The overall delay for controls becomes the maximum delay of any of the + * controls. */ /** @@ -119,7 +120,6 @@ DelayedControls::DelayedControls(V4L2Device *device, */ void DelayedControls::reset(unsigned int cookie) { - queueCount_ = 1; writeCount_ = 0; cookies_[0] = cookie; @@ -146,18 +146,25 @@ void DelayedControls::reset(unsigned int cookie) * \brief Push a set of controls on the queue * \param[in] controls List of controls to add to the device queue * - * Push a set of controls to the control queue. This increases the control queue - * depth by one. + * Push a set of controls to the control queue. The next call to applyControls + * will advance the slot in the queue where the next call to push will write + * controls. * * \returns true if \a controls are accepted, or false otherwise */ bool DelayedControls::push(const ControlList &controls, const unsigned int cookie) { /* Copy state from previous frame. */ - for (auto &ctrl : values_) { - Info &info = ctrl.second[queueCount_]; - info = values_[ctrl.first][queueCount_ - 1]; - info.updated = false; + if (writeCount_ > 0) { + for (auto &ctrl : values_) { + Info &info = ctrl.second[writeCount_]; + info = values_[ctrl.first][writeCount_ - 1]; + info.updated = false; + } + } else { + /* Although it works, we don't expect this. */ + LOG(RPiDelayedControls, Warning) + << "push called before applyControls"; } /* Update with new controls. */ @@ -175,18 +182,17 @@ bool DelayedControls::push(const ControlList &controls, const unsigned int cooki if (controlParams_.find(id) == controlParams_.end()) return false; - Info &info = values_[id][queueCount_]; + Info &info = values_[id][writeCount_]; info = Info(control.second); LOG(RPiDelayedControls, Debug) << "Queuing " << id->name() << " to " << info.toString() - << " at index " << queueCount_; + << " at index " << writeCount_; } - cookies_[queueCount_] = cookie; - queueCount_++; + cookies_[writeCount_] = cookie; return true; } @@ -200,9 +206,9 @@ bool DelayedControls::push(const ControlList &controls, const unsigned int cooki * the callers responsibility to not read too old sequence numbers that have been * pushed out of the history. * - * Historic values are evicted by pushing new values onto the queue using - * push(). The max history from the current sequence number that yields valid - * values are thus 16 minus number of controls pushed. + * Historic values are evicted by new frames arriving and applyControls + * advancing the head of the queue in the ring buffer. The valid history in + * the queue consists of 16 entries from this head of the queue. * * \return The controls at \a sequence number */ @@ -234,6 +240,10 @@ std::pair DelayedControls::get(uint32_t sequence) * number. Any user of these helpers is responsible to inform the helper about * the start of any frame. This can be connected with ease to the start of a * exposure (SOE) V4L2 event. + * + * Controls will be sent to the device, each staggered by the appropriate + * number of frames for that control, so that they are applied at the same + * time. */ void DelayedControls::applyControls(uint32_t sequence) { @@ -277,12 +287,12 @@ void DelayedControls::applyControls(uint32_t sequence) } } - writeCount_ = sequence + 1; - - while (writeCount_ > queueCount_) { + while (writeCount_ < sequence + 1) { + writeCount_++; LOG(RPiDelayedControls, Debug) - << "Queue is empty, auto queue no-op."; - push({}, cookies_[queueCount_ - 1]); + << "Pushing noop with for index " << writeCount_ + << " with cookie " << cookies_[writeCount_ - 1]; + push({}, cookies_[writeCount_ - 1]); } device_->setControls(&out); diff --git a/src/libcamera/pipeline/rpi/common/delayed_controls.h b/src/libcamera/pipeline/rpi/common/delayed_controls.h index 487b0057..48ad5a0f 100644 --- a/src/libcamera/pipeline/rpi/common/delayed_controls.h +++ b/src/libcamera/pipeline/rpi/common/delayed_controls.h @@ -76,7 +76,6 @@ private: std::unordered_map controlParams_; unsigned int maxDelay_; - uint32_t queueCount_; uint32_t writeCount_; std::unordered_map> values_; RingBuffer cookies_; From patchwork Thu May 7 12:20:35 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Plowman X-Patchwork-Id: 26673 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 068FAC32F7 for ; Thu, 7 May 2026 12:55:09 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 3DC436302A; Thu, 7 May 2026 14:55:07 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="uAJJ707M"; dkim-atps=neutral Received: from mail-wm1-x333.google.com (mail-wm1-x333.google.com [IPv6:2a00:1450:4864:20::333]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 7F2BB62010 for ; Thu, 7 May 2026 14:55:03 +0200 (CEST) Received: by mail-wm1-x333.google.com with SMTP id 5b1f17b1804b1-4891e86fabeso10150655e9.1 for ; Thu, 07 May 2026 05:55:03 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; t=1778158503; x=1778763303; darn=lists.libcamera.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=pgXLv83AtQy9mcAhCAUWdtflQBi69OvwGZ4unH4a9kc=; b=uAJJ707MHSZvsmnRWAnvGGmH6oHEgj6v5J2R8JwR8cCR3eHqdv62Jt7OQDQ8YBKng4 wpMMyrtRPf4VsVxLNkM/uH8xXA7ilxQk3Q6G2OorV8Cd1DoAGVlkVfBxLTOa7rtCwJtC fAOLhl2M0ZsvHJko6bsdz/It6xT8Tvn0iWSTlLZYJ+FduuoaoqlJhhLHu0U/u92pDVlW V39jjhNTf2JvCIQH37ndlbjvu2elKZ0jXi6d8TKVEDSLXc3uVgUH3tuQI8Xr0HXhdk7I PdGsN60HzeMkqQVfOpq/8JxRPSaNYKE+3KqAaVj83K4KYKbn9M+lWr+hzu0ko1vqwy/a Uuzg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1778158503; x=1778763303; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=pgXLv83AtQy9mcAhCAUWdtflQBi69OvwGZ4unH4a9kc=; b=gfmKZiTCP+llwwBxubKUky3RJ1XliGAG4MZeG5BaWbhwufb6tgX0A0ghOVwSkYIhjl X8sr69X73vlT6L1bMwD9GzNHsQqLUS7jWtWZ+2VBpQqIrwkJEsYSMLg/ZEKWHvRalql2 ry4/1ShiIrTrru6oEPep3TKvNAiS80KgKskgVG3nZuoKWsDtjIqH5AzwU6Ss4J2zrO3l UJ0E7g1pixqs8iUkpcE6Qr2EqXLnEZWHKrZhv6t2jCglco2/WuUOHlroP8tlCpSn3i7G ZxOu4B0IWRdzgSu0maEm4cm5FokIjUm4g+J0jlb4RgfWU6Zm0FbNi559gHfQgQWoCkmn cZeg== X-Gm-Message-State: AOJu0YxSkWxO6h8GUp9OZHE8fyHYGznGCO74iUGknpOpdWcSAQtKmeMh V/CPBx+g/tfVfYmGnGfCl2hitY2EX/8x68wOieesLco4HmehI384lGoz5lc8oDXJhSZAKlcT+ZP dfepI X-Gm-Gg: AeBDietQy8kMjHb/xDpH5WxxLRLe9aWnG2p4rq9kMWVy9RqFLvvb4OU5AAQvIVnvXFP 5UQcoRBrb2z6glO2szp2KXj2471z9IPIaszAMwAayb3cl5X5zVKUJXtBN3Awcob0dbw0nJpewu0 YWYK8pfI4AqzEwXH4TSKVk7nZQQ3CLnBjz3KQJ5Ra0l+at6KSR86V1HTX2825Mp9vaiYu5TFRXm ozCSfNk6koYhdA+YNNKTwIa6kSjKA3kkjMiGwD3FjrDyHwV95jAi2vdZwqB5LpE7Tt310Nk25q0 P1vbfRsw7/eqzDTr0N7QdtiN1pg4YVh7eaLr4oLzIG43P+djQ0pBuDcqcBAPn09D31l3JbGBhjY t/OnY/mamrPmgoxn32wFjuJNPhQOqYjFhqVMZjw/C+3tOIwl/bjTvItG7oF0ZsORvETddoMETPe K972JZ2AxcfaznRGoEvBLE7x6dFL8hTZyJYnFLozphD4lAvdxYy+QHcksvWdp1RfR5w/ivHYuC5 qN7oyATwqX+KYxwxene0ihoQ5AnvX01KcDd+2KmsjX6MTlDfnuH X-Received: by 2002:a05:600c:4f51:b0:48d:1a94:56c with SMTP id 5b1f17b1804b1-48e51f3054cmr139309425e9.18.1778158502475; Thu, 07 May 2026 05:55:02 -0700 (PDT) Received: from davidp-pi5.pitowers.org ([2a00:1098:3142:1f:88ea:c658:5b20:5e46]) by smtp.gmail.com with ESMTPSA id ffacd0b85a97d-450524833e1sm20396324f8f.2.2026.05.07.05.55.02 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 07 May 2026 05:55:02 -0700 (PDT) From: David Plowman To: libcamera-devel@lists.libcamera.org Cc: David Plowman , Naushir Patuck , Jacopo Mondi Subject: [PATCH v4 2/3] controls: rpi: Add ControlListSequence control Date: Thu, 7 May 2026 13:20:35 +0100 Message-ID: <20260507125458.12140-3-david.plowman@raspberrypi.com> X-Mailer: git-send-email 2.47.3 In-Reply-To: <20260507125458.12140-1-david.plowman@raspberrypi.com> References: <20260507125458.12140-1-david.plowman@raspberrypi.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" The ControlListSequence identifies the sequence number of the request whose controls have been applied to the images in this request for the first time. Signed-off-by: David Plowman Reviewed-by: Naushir Patuck Reviewed-by: Jacopo Mondi --- src/libcamera/control_ids_rpi.yaml | 9 +++++++++ 1 file changed, 9 insertions(+) diff --git a/src/libcamera/control_ids_rpi.yaml b/src/libcamera/control_ids_rpi.yaml index a8615112..0b7da5b2 100644 --- a/src/libcamera/control_ids_rpi.yaml +++ b/src/libcamera/control_ids_rpi.yaml @@ -183,4 +183,13 @@ controls: \sa SyncMode \sa SyncReady \sa SyncTimer + + - ControlListSequence: + type: int64_t + direction: out + description: | + This is the sequence number of the request whose control list has + just been applied. Controls normally take several frames to apply, + so the number here will refer to a request submitted a number of + frames earlier. ... From patchwork Thu May 7 12:20:36 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Plowman X-Patchwork-Id: 26674 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 1CD34C32F8 for ; Thu, 7 May 2026 12:55:10 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id D26656301E; Thu, 7 May 2026 14:55:08 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="QAQeXgLB"; dkim-atps=neutral Received: from mail-wm1-x32e.google.com (mail-wm1-x32e.google.com [IPv6:2a00:1450:4864:20::32e]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id EEC8662010 for ; Thu, 7 May 2026 14:55:03 +0200 (CEST) Received: by mail-wm1-x32e.google.com with SMTP id 5b1f17b1804b1-48a7fe4f40bso9326305e9.0 for ; Thu, 07 May 2026 05:55:03 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; t=1778158503; x=1778763303; darn=lists.libcamera.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=7qXhgolOuJk656ES/U26gIkNoakVkmPr3O4bl7zAoi4=; b=QAQeXgLBt1B4CGCOVEmQsoj67ezSqHZfhMC1kLQkLMDhbCUjDFLcyUo96QkdDBKFov TrXJ0spUDaG+zupzMpmFrR6Zyn5e5ikUKe7qFHo913C3jOEB8yQTIvenMuZaik7+vcg4 i2JHbf8gIx2iGZA4vEfA1LqtbNirPCd0GDaGTdfzVPsJLdH5jDmxK1tpfP80SxHfyHKZ zZMan8Qo+Fz2ShPGJxqWvmbhIAisA2X5Cwu526+4YiV8lSb6fgyUpXcC3EWrXxtQsfdf nWfkXRvvFO/lv/9QazjJuH6Cs8jnS1YjnN7ejumF1q3gNHXkws39wzpSDH40XS+ZBDmx bNDg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1778158503; x=1778763303; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=7qXhgolOuJk656ES/U26gIkNoakVkmPr3O4bl7zAoi4=; b=SHWtDXEcO9cfJyncXCk83B+ah0KKquU9S6M3uDyS9Vql8RiJVy+oC2JIVJycTDFEic HeVuLeC902AlQL69f+lkFveXBwsQAODzCvIUpdWOlNb8jMkjjZfAZN/KqSc3TIZcE/Fp VA0osta1yAwTD008knE2KInDm1zx56ClLP1ksTGmU1sAkHNMSrH2cD1aXDK+RrbbZp4w Pnj5xEcIpi1Bo6pnr1gbNoSsSvwTmb2DrGmYQXHrsEBEGRIDigmL0QNr5PHsN14l/kVH rnRWiA+aA+mKHOvwM5PsbcSxSS7licLmwb57Hj6I66qHKFsI300VtO8g3kEePvzVsUru bXwQ== X-Gm-Message-State: AOJu0Yygt9ygEGTYWlHPDyhBmGf4hnth1I0rJSBdEy2CEPp4QuJfKfoJ qQB4EMSgmlaeriTIDt56bNxXzMtrHcStLuP2VR5YZ1NqnHMhwdsDKt9am+/KfhCnrixkTxa/V1R Qjbls X-Gm-Gg: AeBDiesHv2gOMEzYb+nTI34Fb0Auk2a0F64hVvNR2S7d+GWNrqymUYCptQCrql75nkc Ez3w9YgpcJfkez0Jgl64VOVfI2CAyy+TRHBR/q7tkFEnTupFxp41ZBG6tXZt5ck6yvJ69ApHZXf F3Wm2Q+VT70pGillyoDxS/ZCjlpho4Q2gNbjOzANICczq0B2V5x6lkxmY6y192QhO5HmvRcMcyF Ji5+X8EqpeZtPAEHt/KEzYVHvz3Y9/9PsByKEmS/nWoSLIKEC8oRNZFgJ9b/3Xcslp9o7KHf5f/ qmZPciyFU1wkU6ZV7lS1fXOK7mDNS+FLgMtrfdwdS35zrQdVPBtuw1bnmKIUX7K0XX4rcHtC7vr lHXTgGrsFTDaF1jQHr1FKVJfPM4DOivPDp7CBSnwFS0q1/L88vDRSOKMUTH8kd5Guu1v5Fxx/EG 353SGvYylgx93yyOuARQXe+QrARHmjUWevjrNk6p/I7/YfznnpSFc6WFQRnYCHHYIYxj4eRCcGS T1GRorKqatgUWNhtz+B39SxxAH0HOiEkt/1zwrPXtBB+RapJENZxENGu2TGRq0= X-Received: by 2002:a05:600c:a305:b0:48a:5970:2005 with SMTP id 5b1f17b1804b1-48e51e08362mr87094505e9.2.1778158503007; Thu, 07 May 2026 05:55:03 -0700 (PDT) Received: from davidp-pi5.pitowers.org ([2a00:1098:3142:1f:88ea:c658:5b20:5e46]) by smtp.gmail.com with ESMTPSA id ffacd0b85a97d-450524833e1sm20396324f8f.2.2026.05.07.05.55.02 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 07 May 2026 05:55:02 -0700 (PDT) From: David Plowman To: libcamera-devel@lists.libcamera.org Cc: David Plowman , Naushir Patuck Subject: [PATCH v4 3/3] pipeline: rpi: Make control lists in requests properly atomic Date: Thu, 7 May 2026 13:20:36 +0100 Message-ID: <20260507125458.12140-4-david.plowman@raspberrypi.com> X-Mailer: git-send-email 2.47.3 In-Reply-To: <20260507125458.12140-1-david.plowman@raspberrypi.com> References: <20260507125458.12140-1-david.plowman@raspberrypi.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" When a request is about to be processed, this commit separates "delayed controls" (camera-related ones that take "a few frames" to apply) from "immediate controls" (ISP-related controls) that will happen instantly. It does so in a copy of the request's control list, so as to avoid disturbing the request's original version. The immediate controls are held back until the delayed controls that they were submitted with, have happened. This means all the controls submitted in a request happen atomically. We therefore already have the sequence number of the request whose controls have just been applied, so we additionally attach this to the current request as "ControlId" metadata. Signed-off-by: David Plowman Reviewed-by: Naushir Patuck --- src/ipa/rpi/common/ipa_base.cpp | 24 ++++++--- .../pipeline/rpi/common/pipeline_base.cpp | 49 +++++++++++++++++++ .../pipeline/rpi/common/pipeline_base.h | 9 ++++ src/libcamera/pipeline/rpi/pisp/pisp.cpp | 9 ++-- src/libcamera/pipeline/rpi/vc4/vc4.cpp | 9 ++-- 5 files changed, 86 insertions(+), 14 deletions(-) diff --git a/src/ipa/rpi/common/ipa_base.cpp b/src/ipa/rpi/common/ipa_base.cpp index faa77719..dacafa57 100644 --- a/src/ipa/rpi/common/ipa_base.cpp +++ b/src/ipa/rpi/common/ipa_base.cpp @@ -431,6 +431,15 @@ void IpaBase::prepareIsp(const PrepareParams ¶ms) rpiMetadata.clear(); fillDeviceStatus(params.sensorControls, ipaContext); + /* + * When there are controls, it's important that we don't skip running the + * IPAs, as that can mess with synchronisation. Crucially though, we need + * to know whether there were controls when this comes back as the + * _delayed_ metadata, hence why we flag this in the metadata itself. + */ + if (!params.requestControls.empty()) + rpiMetadata.set("ipa.request_controls", true); + if (params.buffers.embedded) { /* * Pipeline handler has supplied us with an embedded data buffer, @@ -451,7 +460,7 @@ void IpaBase::prepareIsp(const PrepareParams ¶ms) */ AgcStatus agcStatus; bool hdrChange = false; - RPiController::Metadata &delayedMetadata = rpiMetadata_[params.delayContext]; + RPiController::Metadata &delayedMetadata = rpiMetadata_[params.delayContext % rpiMetadata_.size()]; if (!delayedMetadata.get("agc.status", agcStatus)) { rpiMetadata.set("agc.delayed_status", agcStatus); hdrChange = agcStatus.hdr.mode != hdrStatus_.mode; @@ -464,9 +473,13 @@ void IpaBase::prepareIsp(const PrepareParams ¶ms) */ helper_->prepare(embeddedBuffer, rpiMetadata); + bool delayedRequestControls = false; + delayedMetadata.get("ipa.request_controls", delayedRequestControls); + /* Allow a 10% margin on the comparison below. */ Duration delta = (frameTimestamp - lastRunTimestamp_) * 1.0ns; - if (lastRunTimestamp_ && frameCount_ > invalidCount_ && + if (!delayedRequestControls && params.requestControls.empty() && + lastRunTimestamp_ && frameCount_ > invalidCount_ && delta < controllerMinFrameDuration_ * 0.9 && !hdrChange) { /* * Ensure we merge the previous frame's metadata with the current @@ -535,7 +548,7 @@ void IpaBase::processStats(const ProcessParams ¶ms) ControlList ctrls(sensorCtrls_); applyAGC(&agcStatus, ctrls); rpiMetadata.set("agc.status", agcStatus); - setDelayedControls.emit(ctrls, ipaContext); + setDelayedControls.emit(ctrls, params.ipaContext); setCameraTimeoutValue(); } @@ -951,8 +964,6 @@ void IpaBase::applyControls(const ControlList &controls) /* The control provides units of microseconds. */ agc->setFixedExposureTime(0, ctrl.second.get() * 1.0us); - - libcameraMetadata_.set(controls::ExposureTime, ctrl.second.get()); break; } @@ -976,9 +987,6 @@ void IpaBase::applyControls(const ControlList &controls) break; agc->setFixedGain(0, ctrl.second.get()); - - libcameraMetadata_.set(controls::AnalogueGain, - ctrl.second.get()); break; } diff --git a/src/libcamera/pipeline/rpi/common/pipeline_base.cpp b/src/libcamera/pipeline/rpi/common/pipeline_base.cpp index f6ef8a3b..0c82f549 100644 --- a/src/libcamera/pipeline/rpi/common/pipeline_base.cpp +++ b/src/libcamera/pipeline/rpi/common/pipeline_base.cpp @@ -1528,4 +1528,53 @@ void CameraData::fillRequestMetadata(const ControlList &bufferControls, Request } } +static bool isControlDelayed(unsigned int id) +{ + return id == controls::ExposureTime || + id == controls::AnalogueGain || + id == controls::FrameDurationLimits || + id == controls::AeEnable || + id == controls::ExposureTimeMode || + id == controls::AnalogueGainMode; +} + +void CameraData::handleControlLists(uint32_t delayContext, ControlList &requestControls) +{ + /* + * The delayContext is the sequence number after it's gone through the various + * pipeline delays, so that's what gets reported as the "ControlListSequence" + * in the metadata, being the sequence number of the request whose ControlList + * has just been applied. + */ + Request *request = requestQueue_.front(); + request->_d()->metadata().set(controls::rpi::ControlListSequence, delayContext); + + /* + * Controls that take effect immediately (typically ISP controls) have to be + * delayed so as to synchronise with those controls that do get delayed. So we + * must remove them from the current request, and push them onto a queue so + * that they can be used later. + * + * Note that we are passed a copy of the request's controls, so that we can + * avoid disturbing what was in the original request. + */ + ControlList controls = std::move(requestControls); + requestControls.clear(); + immediateControls_.push({ request->sequence(), {} }); + for (const auto &ctrl : controls) { + if (isControlDelayed(ctrl.first)) + requestControls.set(ctrl.first, ctrl.second); + else + immediateControls_.back().controls.set(ctrl.first, ctrl.second); + } + + /* "Immediate" controls that have become due are now merged back into this request. */ + while (!immediateControls_.empty() && + immediateControls_.front().controlListId <= delayContext) { + requestControls.merge(immediateControls_.front().controls, + ControlList::MergePolicy::OverwriteExisting); + immediateControls_.pop(); + } +} + } /* namespace libcamera */ diff --git a/src/libcamera/pipeline/rpi/common/pipeline_base.h b/src/libcamera/pipeline/rpi/common/pipeline_base.h index 7bfac33e..134262b6 100644 --- a/src/libcamera/pipeline/rpi/common/pipeline_base.h +++ b/src/libcamera/pipeline/rpi/common/pipeline_base.h @@ -180,10 +180,19 @@ public: ClockRecovery wallClockRecovery_; + struct ImmediateControlsEntry { + uint64_t controlListId; + ControlList controls; + }; + std::queue immediateControls_; + protected: void fillRequestMetadata(const ControlList &bufferControls, Request *request); + void handleControlLists(uint32_t delayContext, + ControlList &requestControls); + virtual void tryRunPipeline() = 0; private: diff --git a/src/libcamera/pipeline/rpi/pisp/pisp.cpp b/src/libcamera/pipeline/rpi/pisp/pisp.cpp index cc7e32c3..690d2511 100644 --- a/src/libcamera/pipeline/rpi/pisp/pisp.cpp +++ b/src/libcamera/pipeline/rpi/pisp/pisp.cpp @@ -2322,9 +2322,6 @@ void PiSPCameraData::tryRunPipeline() fillRequestMetadata(job.sensorControls, request); - /* Set our state to say the pipeline is active. */ - state_ = State::Busy; - unsigned int bayerId = cfe_[Cfe::Output0].getBufferId(job.buffers[&cfe_[Cfe::Output0]]); unsigned int statsId = cfe_[Cfe::Stats].getBufferId(job.buffers[&cfe_[Cfe::Stats]]); ASSERT(bayerId && statsId); @@ -2343,6 +2340,12 @@ void PiSPCameraData::tryRunPipeline() params.sensorControls = std::move(job.sensorControls); params.requestControls = request->controls(); + /* This sorts out synchronisation with ControlLists in earlier requests. */ + handleControlLists(job.delayContext, params.requestControls); + + /* Set our state to say the pipeline is active. */ + state_ = State::Busy; + if (sensorMetadata_) { unsigned int embeddedId = cfe_[Cfe::Embedded].getBufferId(job.buffers[&cfe_[Cfe::Embedded]]); diff --git a/src/libcamera/pipeline/rpi/vc4/vc4.cpp b/src/libcamera/pipeline/rpi/vc4/vc4.cpp index f99cfdbc..28e7ad46 100644 --- a/src/libcamera/pipeline/rpi/vc4/vc4.cpp +++ b/src/libcamera/pipeline/rpi/vc4/vc4.cpp @@ -939,9 +939,6 @@ void Vc4CameraData::tryRunPipeline() fillRequestMetadata(bayerFrame.controls, request); - /* Set our state to say the pipeline is active. */ - state_ = State::Busy; - unsigned int bayer = unicam_[Unicam::Image].getBufferId(bayerFrame.buffer); LOG(RPI, Debug) << "Signalling prepareIsp:" @@ -955,6 +952,12 @@ void Vc4CameraData::tryRunPipeline() params.delayContext = bayerFrame.delayContext; params.buffers.embedded = 0; + /* This sorts out synchronisation with ControlLists in earlier requests. */ + handleControlLists(bayerFrame.delayContext, params.requestControls); + + /* Set our state to say the pipeline is active. */ + state_ = State::Busy; + if (embeddedBuffer) { unsigned int embeddedId = unicam_[Unicam::Embedded].getBufferId(embeddedBuffer);