From patchwork Tue Apr 28 13:26:37 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Plowman X-Patchwork-Id: 26585 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 574E9C32F6 for ; Tue, 28 Apr 2026 13:40:03 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id A64B762FD2; Tue, 28 Apr 2026 15:40:00 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="eC+2Mlev"; dkim-atps=neutral Received: from mail-wm1-x332.google.com (mail-wm1-x332.google.com [IPv6:2a00:1450:4864:20::332]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 5795262FA9 for ; Tue, 28 Apr 2026 15:39:58 +0200 (CEST) Received: by mail-wm1-x332.google.com with SMTP id 5b1f17b1804b1-48334ee0aeaso119535585e9.1 for ; Tue, 28 Apr 2026 06:39:58 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; t=1777383598; x=1777988398; darn=lists.libcamera.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=DfyoUEdbNejVMNt5pagfVzjLrj4e3pi5RkF1g99Nv5w=; b=eC+2MlevifynfxlVaLOsoQCSxrqd70S9Eoq75Jm7vMZXXvktFDO5UI/cpAfJ1vmzh0 8wLWZ+wiFaz1+W1yf4DJuaYJmV9BjixT74+xrRfK4TnZ0nS+wmZ4b4yx8I56Lgr2YNRE PsZQMcS9DaAdC7CiAECpnzsbBdQHx7Frjqqd5tS/PWJjs5JYJwVWPG4l0eN5mHScbNjV UPOWOLaLF4hwTXBCk0kPAJ88Xr5a0fDqCfGQcUFIQ8WYxQl68y3rdryeny9mtLQRKn4+ fQTgOjGSrgTw3v93NohU3VDdd/GtQZ85tgtq3VMLzHhCpyCvX3Vr22/jWixANM+l+3vG 5gVg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1777383598; x=1777988398; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=DfyoUEdbNejVMNt5pagfVzjLrj4e3pi5RkF1g99Nv5w=; b=j6Ivh7PTTrTyajqyf9b5/moi3ZUwz/+F96Jy5jrKTzCx5CpR4kGUaP15gRGrR4FvLt IWUGUJqMk1yf9Z4CylyeEEAhFDwpSbPygFkQ4LZ8CKeBgdksVmK1wBYHahqNdW7YB2SO TjGdXAjF2RaBK5LtEdmfybp8YBks3RlO+a0kQEoXEY4OdqB4rzZytqGByB49kQn7cBkG LTtFCh/nf9w+jU2qawbh4OQvj+pPgfGURXLJvkupmPrjKVrFamkl41Nn1VedHLokiyOu e7RG71sqBCAVOSacFKER7JJYWbgEbYHpiAkUBWhTQ2IeU8mTZZH6q2WVAsRSDEhg/K3t SmFQ== X-Gm-Message-State: AOJu0YyF569gbkhA6HBWra9BEIBK/o7X0IrRl2tkEOs7Bny4Uv8+nUQf nnxP/d0/Vm9ZWvJrursn4vCH+lmkhTpt34xw6/+kUWWniWm4WUjpF8zThZCWJOrg+BS8OQ/g5Lh Catgz X-Gm-Gg: AeBDiesLo8Y5ZF0mtbVsWUgu+YhQrGN1Tn//Jw5091pZOOy6sj5M6OmZDuEMx4YCS72 f21aRy7KlksBu/6RE0GQiDvPi3La1lREe4aX6ZFgj5WTr/qFwG649Nl1l1mFhsfbyhhFQHtTSpP 7XqdkakQJXeyohTvM/z/WPCi7fE87NLxC0abMxfTgBbdlMdo2QgBM77BibvwJ0Xt1gp1/dLVCHf wWe4IGfU1vhYMGICxzoTd2k8r/e8q1vK/jqVLT3IfSfiAHDMOM5qSiKGOo9Nwhivar6VZaF/onM cgHsLCHkBtya+VjU+053sivGhzWlm/I8ny0o2ExWPX67V6jpz1ExD1zjuCE6v9KLZc1KTDmJW3m 8nISnrpDD07GGq/3kCKFo/fbRU85TFqTvhoCYQNibuWta7BiQGfENyBoMiQc+SpBuoCJ81yEQvK S4hqsQs4h4iuPutb5caUIqz6yr+Zp7xeCcoHr3czMlkwrMeFi4na6yo/iYnmb2DXDqjhHc4fgc2 tOoJK2MvR8pd6Xx4B85Bt58TA+Ldct/dkt38kozxmvNuBm3y0SI X-Received: by 2002:a05:600c:1f92:b0:48a:5821:5ffc with SMTP id 5b1f17b1804b1-48a77ae0329mr49672395e9.2.1777383597497; Tue, 28 Apr 2026 06:39:57 -0700 (PDT) Received: from davidp-pi5.pitowers.org ([2a00:1098:3142:1f:88ea:c658:5b20:5e46]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-48a77c24ca0sm51374525e9.14.2026.04.28.06.39.57 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 28 Apr 2026 06:39:57 -0700 (PDT) From: David Plowman To: libcamera-devel@lists.libcamera.org Cc: David Plowman Subject: [PATCH v3 1/3] pipeline: rpi: Simplify delayed controls Date: Tue, 28 Apr 2026 14:26:37 +0100 Message-ID: <20260428133952.6582-2-david.plowman@raspberrypi.com> X-Mailer: git-send-email 2.47.3 In-Reply-To: <20260428133952.6582-1-david.plowman@raspberrypi.com> References: <20260428133952.6582-1-david.plowman@raspberrypi.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" The queueCount_ value is redundant and can be removed. The pipeline handler calls the push and applyControls functions in a an interleaved manner, so queueCount_ was merely tracking writeCount_, and is therefore unnecessary. Signed-off-by: David Plowman --- .../pipeline/rpi/common/delayed_controls.cpp | 62 +++++++++++-------- .../pipeline/rpi/common/delayed_controls.h | 1 - 2 files changed, 36 insertions(+), 27 deletions(-) diff --git a/src/libcamera/pipeline/rpi/common/delayed_controls.cpp b/src/libcamera/pipeline/rpi/common/delayed_controls.cpp index 19c71946..b647174f 100644 --- a/src/libcamera/pipeline/rpi/common/delayed_controls.cpp +++ b/src/libcamera/pipeline/rpi/common/delayed_controls.cpp @@ -32,14 +32,15 @@ namespace RPi { * * Some sensor controls take effect with a delay as the sensor needs time to * adjust, for example exposure and analog gain. This is a helper class to deal - * with such controls and the intended users are pipeline handlers. + * with such controls. * - * The idea is to extend the concept of the buffer depth of a pipeline the - * application needs to maintain to also cover controls. Just as with buffer - * depth if the application keeps the number of requests queued above the - * control depth the controls are guaranteed to take effect for the correct - * request. The control depth is determined by the control with the greatest - * delay. + * The idea is to maintain a queue of controls that have been submitted. + * Whenever a new frame starts, we can "peak back" into this queue to send + * controls at the correct number of frames in advance to account for each + * control's delay. + * + * The overall delay for controls becomes the maximum delay of any of the + * controls. */ /** @@ -119,7 +120,6 @@ DelayedControls::DelayedControls(V4L2Device *device, */ void DelayedControls::reset(unsigned int cookie) { - queueCount_ = 1; writeCount_ = 0; cookies_[0] = cookie; @@ -146,18 +146,25 @@ void DelayedControls::reset(unsigned int cookie) * \brief Push a set of controls on the queue * \param[in] controls List of controls to add to the device queue * - * Push a set of controls to the control queue. This increases the control queue - * depth by one. + * Push a set of controls to the control queue. The next call to applyControls + * will advance the slot in the queue where the next call to push will write + * controls. * * \returns true if \a controls are accepted, or false otherwise */ bool DelayedControls::push(const ControlList &controls, const unsigned int cookie) { /* Copy state from previous frame. */ - for (auto &ctrl : values_) { - Info &info = ctrl.second[queueCount_]; - info = values_[ctrl.first][queueCount_ - 1]; - info.updated = false; + if (writeCount_ > 0) { + for (auto &ctrl : values_) { + Info &info = ctrl.second[writeCount_]; + info = values_[ctrl.first][writeCount_ - 1]; + info.updated = false; + } + } else { + /* Although it works, we don't expect this. */ + LOG(RPiDelayedControls, Warning) + << "push called before applyControls"; } /* Update with new controls. */ @@ -175,18 +182,17 @@ bool DelayedControls::push(const ControlList &controls, const unsigned int cooki if (controlParams_.find(id) == controlParams_.end()) return false; - Info &info = values_[id][queueCount_]; + Info &info = values_[id][writeCount_]; info = Info(control.second); LOG(RPiDelayedControls, Debug) << "Queuing " << id->name() << " to " << info.toString() - << " at index " << queueCount_; + << " at index " << writeCount_; } - cookies_[queueCount_] = cookie; - queueCount_++; + cookies_[writeCount_] = cookie; return true; } @@ -200,9 +206,9 @@ bool DelayedControls::push(const ControlList &controls, const unsigned int cooki * the callers responsibility to not read too old sequence numbers that have been * pushed out of the history. * - * Historic values are evicted by pushing new values onto the queue using - * push(). The max history from the current sequence number that yields valid - * values are thus 16 minus number of controls pushed. + * Historic values are evicted by new frames arriving and applyControls + * advancing the head of the queue in the ring buffer. The valid history in + * the queue consists of 16 entries from this head of the queue. * * \return The controls at \a sequence number */ @@ -234,6 +240,10 @@ std::pair DelayedControls::get(uint32_t sequence) * number. Any user of these helpers is responsible to inform the helper about * the start of any frame. This can be connected with ease to the start of a * exposure (SOE) V4L2 event. + * + * Controls will be sent to the device, each staggered by the appropriate + * number of frames for that control, so that they are applied at the same + * time. */ void DelayedControls::applyControls(uint32_t sequence) { @@ -277,12 +287,12 @@ void DelayedControls::applyControls(uint32_t sequence) } } - writeCount_ = sequence + 1; - - while (writeCount_ > queueCount_) { + while (writeCount_ < sequence + 1) { + writeCount_++; LOG(RPiDelayedControls, Debug) - << "Queue is empty, auto queue no-op."; - push({}, cookies_[queueCount_ - 1]); + << "Pushing noop with for index " << writeCount_ + << " with cookie " << cookies_[writeCount_ - 1]; + push({}, cookies_[writeCount_ - 1]); } device_->setControls(&out); diff --git a/src/libcamera/pipeline/rpi/common/delayed_controls.h b/src/libcamera/pipeline/rpi/common/delayed_controls.h index 487b0057..48ad5a0f 100644 --- a/src/libcamera/pipeline/rpi/common/delayed_controls.h +++ b/src/libcamera/pipeline/rpi/common/delayed_controls.h @@ -76,7 +76,6 @@ private: std::unordered_map controlParams_; unsigned int maxDelay_; - uint32_t queueCount_; uint32_t writeCount_; std::unordered_map> values_; RingBuffer cookies_; From patchwork Tue Apr 28 13:26:38 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Plowman X-Patchwork-Id: 26586 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 3CC60BE173 for ; Tue, 28 Apr 2026 13:40:05 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 2E3CB62FDF; Tue, 28 Apr 2026 15:40:02 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="LwpKAOC9"; dkim-atps=neutral Received: from mail-wm1-x333.google.com (mail-wm1-x333.google.com [IPv6:2a00:1450:4864:20::333]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id B128F62FA9 for ; Tue, 28 Apr 2026 15:39:58 +0200 (CEST) Received: by mail-wm1-x333.google.com with SMTP id 5b1f17b1804b1-488d2079582so135150905e9.2 for ; Tue, 28 Apr 2026 06:39:58 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; t=1777383598; x=1777988398; darn=lists.libcamera.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=ULQqqLF7O9qdYvQtiTM8U6/kpEm2D3bxw7VXrv1kcKQ=; b=LwpKAOC9P6HEoAVPK4VpZP9IFoGcbVxqMXvU2DsFglJS+re04T8JUoq9S5x3d6TlP4 AGMGMiyx5kyrmnhU24UT9A3+q4+XJjt/8qWKhALGiRsTUh+Pt23ZDqIi7V6q40bRA/yB t5CDXYQ93F518yvqlSF+PXepyjKnMG9qvYXPlDo2F4O/N40N8RRgMXIwTykhjwNpqLgo Q40LRNYjcRdBFOfJo+bAWWOpzfVdlmVkjxXeweXXJJCuSxV9kuqjM9H30MD/zV/NaVrq utH3YynhGfiDco6rY6w6SJGuCv0JS+o3kUJAKIXRGJRNnKp+t1B9duGYOYMHB1RqiKTT q5Bg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1777383598; x=1777988398; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=ULQqqLF7O9qdYvQtiTM8U6/kpEm2D3bxw7VXrv1kcKQ=; b=Jtjs7+xGPBAh2YWluf4m8ZXE0zHAUd1jyOXjU9oUq4Jmpia5SM6HrFG7J2YDJ4WaZf 79QDp+CyyLoYIP6pHyquJGJbtL36XNqBdm8zTM1Ak3L1AJmctO9m100TXmdCleLzMp8n F2AHzooJHR8xn5snMGqGdoat+YPWwuVRIW1uo0N/VhM3JXvrLRsAMnb1bIropxoKaC+o xXj+kYpmXXG2fikSM3Ddmxlw8DPVNyB9Z2KubZCyhJzV98H5t4iFvOKin+3KLS6BecCD T5Gbbcn/qe4flK38Xh/CXsggwkZq5P5vHD4EUrotBAriF7WVOU5WKGonJUVDn1oGPZhg 65jw== X-Gm-Message-State: AOJu0YyFN4DKIkXGOuLliC32ieTK7F1aoMyouR/7OzNM87ozjAfZeOOu xC8fXeMO5xIwXy0sGH+S6qidnWsmOHXywYwLI6s0y/n/psujsC3VcIF3H4OTtxsbB1BQR97t5W0 HfqdZ X-Gm-Gg: AeBDiesYVOEnZIA9izQOznov4teuiKVMI3wUoPEfes1frWk5yEUJKQ6WjPwvr1D4VHN 1mNnIjMUZH3pnj/nZYnIE2DduUO1qT1TZ7cTcdo/TBKEqNgU7mE/s+T1T543E4lJ8TVgJ1CC/8N Aa+ryE1LmTWhNr3iTkrdsdYq5Y+QWX+7pM1pn5xmR6hLoLrRq+3FkZvrh7hhr3LxvC0gu5wfJMJ /GksUiuGF6JK0mZHQJwvtTiB1ocvaj5xDrjzS/KPs3oKQv3eIZyWP7E1LMp99GE2ru91cdwweNi s38NVHw61vlSVDFOQz+apuFelUQKG0HTyKxgdcxx9tPTgs1+oX6McUeYr2qbT0CV6PSj/igUiES OR+RNyb62Ra/e1VMNTwPjb3puZx788LnXasSrks5DryBTeKg14ALVHVje3hmExavILVrptYhxDX EajSZIPBxvWj1aaysevJgyZJOilwkFB+dr136UMaBh7vR4UuF539xeGChkwQpRKg8g6hq26E4rR FlYgsbW2VckyDaKAm7HNPc+cx6VuC8lCJpXN9KbrQJ3m3ngv1qA X-Received: by 2002:a05:600c:4fc1:b0:486:fba7:b150 with SMTP id 5b1f17b1804b1-48a77b1b2d9mr52273905e9.15.1777383597902; Tue, 28 Apr 2026 06:39:57 -0700 (PDT) Received: from davidp-pi5.pitowers.org ([2a00:1098:3142:1f:88ea:c658:5b20:5e46]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-48a77c24ca0sm51374525e9.14.2026.04.28.06.39.57 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 28 Apr 2026 06:39:57 -0700 (PDT) From: David Plowman To: libcamera-devel@lists.libcamera.org Cc: David Plowman Subject: [PATCH v3 2/3] controls: rpi: Add ControlListSequence control Date: Tue, 28 Apr 2026 14:26:38 +0100 Message-ID: <20260428133952.6582-3-david.plowman@raspberrypi.com> X-Mailer: git-send-email 2.47.3 In-Reply-To: <20260428133952.6582-1-david.plowman@raspberrypi.com> References: <20260428133952.6582-1-david.plowman@raspberrypi.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" The ControlListSequence identifies the sequence number of the request whose controls have been applied to the images in this request for the first time. Signed-off-by: David Plowman --- src/libcamera/control_ids_rpi.yaml | 9 +++++++++ 1 file changed, 9 insertions(+) diff --git a/src/libcamera/control_ids_rpi.yaml b/src/libcamera/control_ids_rpi.yaml index a8615112..0b7da5b2 100644 --- a/src/libcamera/control_ids_rpi.yaml +++ b/src/libcamera/control_ids_rpi.yaml @@ -183,4 +183,13 @@ controls: \sa SyncMode \sa SyncReady \sa SyncTimer + + - ControlListSequence: + type: int64_t + direction: out + description: | + This is the sequence number of the request whose control list has + just been applied. Controls normally take several frames to apply, + so the number here will refer to a request submitted a number of + frames earlier. ... From patchwork Tue Apr 28 13:26:39 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Plowman X-Patchwork-Id: 26587 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 1F432C32F7 for ; Tue, 28 Apr 2026 13:40:06 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id CE6AE62FE2; Tue, 28 Apr 2026 15:40:03 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="WYxaUeRp"; dkim-atps=neutral Received: from mail-wm1-x335.google.com (mail-wm1-x335.google.com [IPv6:2a00:1450:4864:20::335]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 3F4FF62FD2 for ; Tue, 28 Apr 2026 15:39:59 +0200 (CEST) Received: by mail-wm1-x335.google.com with SMTP id 5b1f17b1804b1-4891c00e7aeso87679225e9.2 for ; Tue, 28 Apr 2026 06:39:59 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; t=1777383599; x=1777988399; darn=lists.libcamera.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=aX68pVIfwS/iqGU8xne8xIfWg+LTMMw5YrnovQVx2yY=; b=WYxaUeRpP3QnFdHivheoEPFknhNeocSGoPS+H/BG0Em71D9v1GDzd1vk6t2F2OdrAu JGEnKZpHIHquO/LZcJ9yhGzPW99NKlUAvuxXrw1ubyxWjS+Lm7lG3+m4ktR4kNiZzlte 7kAnY8nnX6SoYtD8F5qD4AE1bTfe3mBbDtQW7aW3pglyHWo6MjwFPGHy/YBO8epqlGkp ph4LAHSf6Vjg/HTvKTlX5PtwgAJTAxtIjjN6AgdAo2/r3b8hv8FpIXQThNSh583XheXU fDaX9anyFzrTKtdCxbtIPZYmkn8U3mH+4OBMMxsJ7LRiqP0K8M28rW6E+9D6Dr9mUEJ+ bT/Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1777383599; x=1777988399; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=aX68pVIfwS/iqGU8xne8xIfWg+LTMMw5YrnovQVx2yY=; b=rqnHvhlq7d6bIhbkBht6Xxdg96YbiYvmJER0Ds1ApaIEv/Gr7sna7wGeNOjHmP/9DQ bL9zqtjkJyLWiAVCSyPe4SLRnRNdlDv1YzGizjqhd+CoeBQooYUe+hUw9SeyS6V4h6l6 KRkBawVzqOBUPDY1AuoSw7ieFtyJLBWqI6CRid8T/Saevqqj1gGO9wE5RtBU7MGV0pWG jdHmtk3itFFoPksEoZIHJ+Ais/EEjqa/pIz6DR3gLorpNi0fEYVzN/MIb6sw4bHanqhS AAbPuJhteMnZC30rH/i9JRCyE7W8DkjXR+ePFkRX1PdtoAgFhFK5Poi56ikbIf+lZUJl sUyA== X-Gm-Message-State: AOJu0YzEP50Nt7S6eEQ4KAcTznEI00k1QKi5dgJUOEfuy/jnQvui6GGs Ym0X5EVa7MFGJz4G9TB+ypWd6pRCz4jpaBtw+xl8O/PST1lEqwXxyKAaIMNmIASni7TysnwPBBH bOihw X-Gm-Gg: AeBDieul1LLvF63S2ODhh5PfbanT9/4eXKNpGbXvz1LGgoMOVKrcjo73X4HNb9BbeE/ xvqEUKMPtmvWI+ujjxvaziIsrilWV+ktv5ca7YX6nxJzKBbgOK9IJnHDVTWOPwKLmj2mOcLIhVV BpEbUgDxoobRDER8K1vVQ2vYAwfbf3dzu85ebeKqkrIvGMQgv7rtAQh8cOuLq8l2wVMWIsfALma f6EX0oDRh6x0ui1qdUQMPfyKfzf6aQuekcUU+7o8x4QFImZ1mSsXqsG2UNut85Vod6wxHVkrY0R WxeOuKcpRSHKlH/WiFLNXaASLP+66CxMSh/97nxSUl+cRUFG65LyypWW3pb19lPoO4YylOdGlNR HDXUqqSGVRcFJ19JMl92Dm6FOeA9m2WB7yFqNQyJNiY9No75qgwbaaMQaF+3hJew7e1JbngsdAC I8IxqQA7K+YvSk9s3P35JFZv8UTIb0oERMb4bTACMeDneaYSwr8v7WUHy/mCmOqVp/r/NvVAbOn QwfqXeW3FFr2jfSv94LkRmbMEUBjCwCThHUMzn3fdnlKmIbGCuE X-Received: by 2002:a05:600c:3b0f:b0:487:20ee:bef6 with SMTP id 5b1f17b1804b1-48a77afec2amr51180015e9.11.1777383598474; Tue, 28 Apr 2026 06:39:58 -0700 (PDT) Received: from davidp-pi5.pitowers.org ([2a00:1098:3142:1f:88ea:c658:5b20:5e46]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-48a77c24ca0sm51374525e9.14.2026.04.28.06.39.58 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 28 Apr 2026 06:39:58 -0700 (PDT) From: David Plowman To: libcamera-devel@lists.libcamera.org Cc: David Plowman Subject: [PATCH v3 3/3] pipeline: rpi: Make control lists in requests properly atomic Date: Tue, 28 Apr 2026 14:26:39 +0100 Message-ID: <20260428133952.6582-4-david.plowman@raspberrypi.com> X-Mailer: git-send-email 2.47.3 In-Reply-To: <20260428133952.6582-1-david.plowman@raspberrypi.com> References: <20260428133952.6582-1-david.plowman@raspberrypi.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" When a request is about to be processed, this commit separates "delayed controls" (camera-related ones that take "a few frames" to apply) from "immediate controls" (ISP-related controls) that will happen instantly. The immediate controls are held back until the delayed controls that they were submitted with, have happened. This means all the controls submitted in a request happen atomically. We therefore already have the sequence number of the request whose controls have just been applied, so we additionally attach this to the current request as "ControlId" metadata. Signed-off-by: David Plowman --- src/ipa/rpi/common/ipa_base.cpp | 24 ++++++---- .../pipeline/rpi/common/pipeline_base.cpp | 46 +++++++++++++++++++ .../pipeline/rpi/common/pipeline_base.h | 8 ++++ src/libcamera/pipeline/rpi/pisp/pisp.cpp | 3 ++ src/libcamera/pipeline/rpi/vc4/vc4.cpp | 3 ++ 5 files changed, 76 insertions(+), 8 deletions(-) diff --git a/src/ipa/rpi/common/ipa_base.cpp b/src/ipa/rpi/common/ipa_base.cpp index faa77719..dacafa57 100644 --- a/src/ipa/rpi/common/ipa_base.cpp +++ b/src/ipa/rpi/common/ipa_base.cpp @@ -431,6 +431,15 @@ void IpaBase::prepareIsp(const PrepareParams ¶ms) rpiMetadata.clear(); fillDeviceStatus(params.sensorControls, ipaContext); + /* + * When there are controls, it's important that we don't skip running the + * IPAs, as that can mess with synchronisation. Crucially though, we need + * to know whether there were controls when this comes back as the + * _delayed_ metadata, hence why we flag this in the metadata itself. + */ + if (!params.requestControls.empty()) + rpiMetadata.set("ipa.request_controls", true); + if (params.buffers.embedded) { /* * Pipeline handler has supplied us with an embedded data buffer, @@ -451,7 +460,7 @@ void IpaBase::prepareIsp(const PrepareParams ¶ms) */ AgcStatus agcStatus; bool hdrChange = false; - RPiController::Metadata &delayedMetadata = rpiMetadata_[params.delayContext]; + RPiController::Metadata &delayedMetadata = rpiMetadata_[params.delayContext % rpiMetadata_.size()]; if (!delayedMetadata.get("agc.status", agcStatus)) { rpiMetadata.set("agc.delayed_status", agcStatus); hdrChange = agcStatus.hdr.mode != hdrStatus_.mode; @@ -464,9 +473,13 @@ void IpaBase::prepareIsp(const PrepareParams ¶ms) */ helper_->prepare(embeddedBuffer, rpiMetadata); + bool delayedRequestControls = false; + delayedMetadata.get("ipa.request_controls", delayedRequestControls); + /* Allow a 10% margin on the comparison below. */ Duration delta = (frameTimestamp - lastRunTimestamp_) * 1.0ns; - if (lastRunTimestamp_ && frameCount_ > invalidCount_ && + if (!delayedRequestControls && params.requestControls.empty() && + lastRunTimestamp_ && frameCount_ > invalidCount_ && delta < controllerMinFrameDuration_ * 0.9 && !hdrChange) { /* * Ensure we merge the previous frame's metadata with the current @@ -535,7 +548,7 @@ void IpaBase::processStats(const ProcessParams ¶ms) ControlList ctrls(sensorCtrls_); applyAGC(&agcStatus, ctrls); rpiMetadata.set("agc.status", agcStatus); - setDelayedControls.emit(ctrls, ipaContext); + setDelayedControls.emit(ctrls, params.ipaContext); setCameraTimeoutValue(); } @@ -951,8 +964,6 @@ void IpaBase::applyControls(const ControlList &controls) /* The control provides units of microseconds. */ agc->setFixedExposureTime(0, ctrl.second.get() * 1.0us); - - libcameraMetadata_.set(controls::ExposureTime, ctrl.second.get()); break; } @@ -976,9 +987,6 @@ void IpaBase::applyControls(const ControlList &controls) break; agc->setFixedGain(0, ctrl.second.get()); - - libcameraMetadata_.set(controls::AnalogueGain, - ctrl.second.get()); break; } diff --git a/src/libcamera/pipeline/rpi/common/pipeline_base.cpp b/src/libcamera/pipeline/rpi/common/pipeline_base.cpp index f6ef8a3b..28f44753 100644 --- a/src/libcamera/pipeline/rpi/common/pipeline_base.cpp +++ b/src/libcamera/pipeline/rpi/common/pipeline_base.cpp @@ -1528,4 +1528,50 @@ void CameraData::fillRequestMetadata(const ControlList &bufferControls, Request } } +static bool isControlDelayed(unsigned int id) +{ + return id == controls::ExposureTime || + id == controls::AnalogueGain || + id == controls::FrameDurationLimits || + id == controls::AeEnable || + id == controls::ExposureTimeMode || + id == controls::AnalogueGainMode; +} + +void CameraData::handleControlLists(uint32_t delayContext) +{ + /* + * The delayContext is the sequence number after it's gone through the various + * pipeline delays, so that's what gets reported as the "ControlListSequence" + * in the metadata, being the sequence number of the request whose ControlList + * has just been applied. + */ + Request *request = requestQueue_.front(); + request->_d()->metadata().set(controls::rpi::ControlListSequence, delayContext); + + /* + * Controls that take effect immediately (typically ISP controls) have to be + * delayed so as to synchronise with those controls that do get delayed. So we + * must remove them from the current request, and push them onto a queue so + * that they can be used later. + */ + ControlList controls = std::move(request->controls()); + request->controls().clear(); + immediateControls_.push({ request->sequence(), {} }); + for (const auto &ctrl : controls) { + if (isControlDelayed(ctrl.first)) + request->controls().set(ctrl.first, ctrl.second); + else + immediateControls_.back().controls.set(ctrl.first, ctrl.second); + } + + /* "Immediate" controls that have become due are now merged back into this request. */ + while (!immediateControls_.empty() && + immediateControls_.front().controlListId <= delayContext) { + request->controls().merge(immediateControls_.front().controls, + ControlList::MergePolicy::OverwriteExisting); + immediateControls_.pop(); + } +} + } /* namespace libcamera */ diff --git a/src/libcamera/pipeline/rpi/common/pipeline_base.h b/src/libcamera/pipeline/rpi/common/pipeline_base.h index 7bfac33e..d3f044a4 100644 --- a/src/libcamera/pipeline/rpi/common/pipeline_base.h +++ b/src/libcamera/pipeline/rpi/common/pipeline_base.h @@ -180,10 +180,18 @@ public: ClockRecovery wallClockRecovery_; + struct ImmediateControlsEntry { + uint64_t controlListId; + ControlList controls; + }; + std::queue immediateControls_; + protected: void fillRequestMetadata(const ControlList &bufferControls, Request *request); + void handleControlLists(uint32_t delayContext); + virtual void tryRunPipeline() = 0; private: diff --git a/src/libcamera/pipeline/rpi/pisp/pisp.cpp b/src/libcamera/pipeline/rpi/pisp/pisp.cpp index c7799f2b..70bbac5a 100644 --- a/src/libcamera/pipeline/rpi/pisp/pisp.cpp +++ b/src/libcamera/pipeline/rpi/pisp/pisp.cpp @@ -2322,6 +2322,9 @@ void PiSPCameraData::tryRunPipeline() fillRequestMetadata(job.sensorControls, request); + /* This sorts out synchronisation with ControlLists in earlier requests. */ + handleControlLists(job.delayContext); + /* Set our state to say the pipeline is active. */ state_ = State::Busy; diff --git a/src/libcamera/pipeline/rpi/vc4/vc4.cpp b/src/libcamera/pipeline/rpi/vc4/vc4.cpp index f99cfdbc..a1b44af1 100644 --- a/src/libcamera/pipeline/rpi/vc4/vc4.cpp +++ b/src/libcamera/pipeline/rpi/vc4/vc4.cpp @@ -939,6 +939,9 @@ void Vc4CameraData::tryRunPipeline() fillRequestMetadata(bayerFrame.controls, request); + /* This sorts out synchronisation with ControlLists in earlier requests. */ + handleControlLists(bayerFrame.delayContext); + /* Set our state to say the pipeline is active. */ state_ = State::Busy;