From patchwork Fri May 8 12:57:41 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Plowman X-Patchwork-Id: 26691 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id D5796BDCBD for ; Fri, 8 May 2026 13:08:11 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 1D26F6301A; Fri, 8 May 2026 15:08:10 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="J/06+Mj+"; dkim-atps=neutral Received: from mail-wm1-x334.google.com (mail-wm1-x334.google.com [IPv6:2a00:1450:4864:20::334]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id B67DD62FE1 for ; Fri, 8 May 2026 15:08:07 +0200 (CEST) Received: by mail-wm1-x334.google.com with SMTP id 5b1f17b1804b1-48a7fe4f40bso21547685e9.0 for ; Fri, 08 May 2026 06:08:07 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; t=1778245687; x=1778850487; darn=lists.libcamera.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=B/6c4x6aPmSj4hSBbdqWEJNfL9gt9CXnVpTwIUpnx7A=; b=J/06+Mj+lT+FyhgOldADVQqllQW+b3fxVGBvhhfRtMOxm91OOUILYWcMNmG7Y/2vDG NAHizb8RmlXY0k8GALU2ZWiskh2n5/CxAYqxQlY6sPr5n7WDjxfwYhSBuaLi33ih48Zi CHmgV8rJnhWuKRW41Jb0kUdP3/gFbIoKOFTydHj5HrWM7j6T3QM2SPK5JzYJUdTT3km7 TqH/icAW+348EiIu2kaZD3UiXcjYtxQDLA+IAYkSzG72hQ8Up62107jqxxrLcbR+/0pJ QyXqReCD1mrACbDgCalOwBAWVPCS4HRX39UVeBCTHkJOY2BZv5xnCnqWd2I2jS1s1r02 vW7Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1778245687; x=1778850487; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=B/6c4x6aPmSj4hSBbdqWEJNfL9gt9CXnVpTwIUpnx7A=; b=QNoBzCAXToqvVCrxoKQwlACRhpRNv17s2elMXfokRtBj03kCvJ+CSGE5FYuc3hCeKO jDVtbm9H0FIladqGICrFsh9nor13cAz4oRc3tDaYIMx3EP5qSrBZCc2Wlk1iy1jKt+k4 c49KbVcKNZ16QnWaEuz/uU+JQtA9uREzxTNZjxL98thNxfR0ocPSdkByFV0ZLS3M77QM XWq5Z8NKxwdPFK+xLGYrzkOKBbee0CSAYHlL+yEhQl62KdFXwgp79lieAIUbecrnyzeG X6ITJ/Eorvt04GE3Yt/0p8LsX4ZeVFJi6eb3lCRfX3PoKtsTTlFCJgsqT1ziVczBI++7 jklw== X-Gm-Message-State: AOJu0YxVR9XKLkbXlrgYK+hT+NJUOIIzTmxF26o6re672lIQVPsAqkJa tQUHZU5LFyiMwA2JBlU0F+My2muUYQNqLNgZZTk6faRWPX2N0S0mIxLpyPTRjikVRJwnThMmKjk lCafR X-Gm-Gg: AeBDiets+FT0lpg2L3U8ukt4bkyNEuWip+9bYdvlRH1YLAOuG0yAPcat8IwaeLfNDTL k99WdDUJnh9eIb1XVxNCSw/Xbup86XVR3wY20KnFgji4zr+GtnlIcMs306SRjY+6llFjy6xFIdq rxUsaoS0m04nrU6u2T8nqDQA0f2px0X+9BO1//P1ayQ52gsHsd9tKG2JKUoYFXN7LnfXJI8/Dxz 21kEJ3qM8LT/lK4yoIJ9ATuNyVzwoW1BEJE816iUmXJd8b0lEM7AvqJYKmBJHw54h9p+52EKQsf tDPMCk3I0gD4MNnWnXNr2cgkeB+K4IcUeHYTl+SX7NYZGlTRCVk1F0+wUrUH7nCmzQdFBYEC/1O UoOmyURzY1M+bo+l1Mm8QyKaMc3IJ2ZBZ0ZUsFAs4WXQeqMGzeH9DXhZcAGZ8jYTAVHFHzzqG28 Z70NqzQLUiU0anNg901b9CFhI3L1nFleEJ9XSiSZKBvfDuxthI9PPLi3CD0hiV1IZOdwMLjEEnr jkpsClSQg9diroKA9oIT661m2HTgbHo/vRjAw== X-Received: by 2002:a05:600d:d:b0:489:1ff5:edda with SMTP id 5b1f17b1804b1-48e51e08240mr169742525e9.6.1778245686903; Fri, 08 May 2026 06:08:06 -0700 (PDT) Received: from localhost.localdomain ([2a06:61c0:f337:0:9c1f:b517:931a:3b19]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-48e68f5d9b5sm34044905e9.15.2026.05.08.06.08.06 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 08 May 2026 06:08:06 -0700 (PDT) From: David Plowman To: libcamera-devel@lists.libcamera.org Cc: David Plowman , Naushir Patuck , Jacopo Mondi Subject: [PATCH v5 1/3] pipeline: rpi: Simplify delayed controls Date: Fri, 8 May 2026 13:57:41 +0100 Message-ID: <20260508130804.8238-2-david.plowman@raspberrypi.com> X-Mailer: git-send-email 2.47.3 In-Reply-To: <20260508130804.8238-1-david.plowman@raspberrypi.com> References: <20260508130804.8238-1-david.plowman@raspberrypi.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" The queueCount_ value is redundant and can be removed. The pipeline handler calls the push and applyControls functions in a an interleaved manner, so queueCount_ was merely tracking writeCount_, and is therefore unnecessary. Signed-off-by: David Plowman Reviewed-by: Naushir Patuck Reviewed-by: Jacopo Mondi --- .../pipeline/rpi/common/delayed_controls.cpp | 62 +++++++++++-------- .../pipeline/rpi/common/delayed_controls.h | 1 - 2 files changed, 36 insertions(+), 27 deletions(-) diff --git a/src/libcamera/pipeline/rpi/common/delayed_controls.cpp b/src/libcamera/pipeline/rpi/common/delayed_controls.cpp index 19c71946..b647174f 100644 --- a/src/libcamera/pipeline/rpi/common/delayed_controls.cpp +++ b/src/libcamera/pipeline/rpi/common/delayed_controls.cpp @@ -32,14 +32,15 @@ namespace RPi { * * Some sensor controls take effect with a delay as the sensor needs time to * adjust, for example exposure and analog gain. This is a helper class to deal - * with such controls and the intended users are pipeline handlers. + * with such controls. * - * The idea is to extend the concept of the buffer depth of a pipeline the - * application needs to maintain to also cover controls. Just as with buffer - * depth if the application keeps the number of requests queued above the - * control depth the controls are guaranteed to take effect for the correct - * request. The control depth is determined by the control with the greatest - * delay. + * The idea is to maintain a queue of controls that have been submitted. + * Whenever a new frame starts, we can "peak back" into this queue to send + * controls at the correct number of frames in advance to account for each + * control's delay. + * + * The overall delay for controls becomes the maximum delay of any of the + * controls. */ /** @@ -119,7 +120,6 @@ DelayedControls::DelayedControls(V4L2Device *device, */ void DelayedControls::reset(unsigned int cookie) { - queueCount_ = 1; writeCount_ = 0; cookies_[0] = cookie; @@ -146,18 +146,25 @@ void DelayedControls::reset(unsigned int cookie) * \brief Push a set of controls on the queue * \param[in] controls List of controls to add to the device queue * - * Push a set of controls to the control queue. This increases the control queue - * depth by one. + * Push a set of controls to the control queue. The next call to applyControls + * will advance the slot in the queue where the next call to push will write + * controls. * * \returns true if \a controls are accepted, or false otherwise */ bool DelayedControls::push(const ControlList &controls, const unsigned int cookie) { /* Copy state from previous frame. */ - for (auto &ctrl : values_) { - Info &info = ctrl.second[queueCount_]; - info = values_[ctrl.first][queueCount_ - 1]; - info.updated = false; + if (writeCount_ > 0) { + for (auto &ctrl : values_) { + Info &info = ctrl.second[writeCount_]; + info = values_[ctrl.first][writeCount_ - 1]; + info.updated = false; + } + } else { + /* Although it works, we don't expect this. */ + LOG(RPiDelayedControls, Warning) + << "push called before applyControls"; } /* Update with new controls. */ @@ -175,18 +182,17 @@ bool DelayedControls::push(const ControlList &controls, const unsigned int cooki if (controlParams_.find(id) == controlParams_.end()) return false; - Info &info = values_[id][queueCount_]; + Info &info = values_[id][writeCount_]; info = Info(control.second); LOG(RPiDelayedControls, Debug) << "Queuing " << id->name() << " to " << info.toString() - << " at index " << queueCount_; + << " at index " << writeCount_; } - cookies_[queueCount_] = cookie; - queueCount_++; + cookies_[writeCount_] = cookie; return true; } @@ -200,9 +206,9 @@ bool DelayedControls::push(const ControlList &controls, const unsigned int cooki * the callers responsibility to not read too old sequence numbers that have been * pushed out of the history. * - * Historic values are evicted by pushing new values onto the queue using - * push(). The max history from the current sequence number that yields valid - * values are thus 16 minus number of controls pushed. + * Historic values are evicted by new frames arriving and applyControls + * advancing the head of the queue in the ring buffer. The valid history in + * the queue consists of 16 entries from this head of the queue. * * \return The controls at \a sequence number */ @@ -234,6 +240,10 @@ std::pair DelayedControls::get(uint32_t sequence) * number. Any user of these helpers is responsible to inform the helper about * the start of any frame. This can be connected with ease to the start of a * exposure (SOE) V4L2 event. + * + * Controls will be sent to the device, each staggered by the appropriate + * number of frames for that control, so that they are applied at the same + * time. */ void DelayedControls::applyControls(uint32_t sequence) { @@ -277,12 +287,12 @@ void DelayedControls::applyControls(uint32_t sequence) } } - writeCount_ = sequence + 1; - - while (writeCount_ > queueCount_) { + while (writeCount_ < sequence + 1) { + writeCount_++; LOG(RPiDelayedControls, Debug) - << "Queue is empty, auto queue no-op."; - push({}, cookies_[queueCount_ - 1]); + << "Pushing noop with for index " << writeCount_ + << " with cookie " << cookies_[writeCount_ - 1]; + push({}, cookies_[writeCount_ - 1]); } device_->setControls(&out); diff --git a/src/libcamera/pipeline/rpi/common/delayed_controls.h b/src/libcamera/pipeline/rpi/common/delayed_controls.h index 487b0057..48ad5a0f 100644 --- a/src/libcamera/pipeline/rpi/common/delayed_controls.h +++ b/src/libcamera/pipeline/rpi/common/delayed_controls.h @@ -76,7 +76,6 @@ private: std::unordered_map controlParams_; unsigned int maxDelay_; - uint32_t queueCount_; uint32_t writeCount_; std::unordered_map> values_; RingBuffer cookies_; From patchwork Fri May 8 12:57:42 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Plowman X-Patchwork-Id: 26692 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 1B514BDCBD for ; Fri, 8 May 2026 13:08:13 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 947CB63025; Fri, 8 May 2026 15:08:10 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="k6PGXuw7"; dkim-atps=neutral Received: from mail-wm1-x333.google.com (mail-wm1-x333.google.com [IPv6:2a00:1450:4864:20::333]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 2C91362FD3 for ; Fri, 8 May 2026 15:08:08 +0200 (CEST) Received: by mail-wm1-x333.google.com with SMTP id 5b1f17b1804b1-48a563e4ef7so18483765e9.0 for ; Fri, 08 May 2026 06:08:08 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; t=1778245687; x=1778850487; darn=lists.libcamera.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=pgXLv83AtQy9mcAhCAUWdtflQBi69OvwGZ4unH4a9kc=; b=k6PGXuw7Q6oJ7L3LQB83+4yxyT1/Ek7oyl0N0F7pB87WnWmBvhxtbB9A4t2uwlVp0O QHDFqbNzJS7ZQ/6Jpeyugvq9VLCojb8GVLJYOmajloVnpxda2S2uz1IAIRrmmjvEqgp+ b0Iiwy1MV9WJ32SSug2nCe5ibHXHcmBexmuXq9ZlfG40AnbYhFU0GBJcFovJcxZ7Wwcp JoSmoEn/h25VnvitVaZyheIvYvp3OmAtJW4d+0+rGg6JoEoaXIHlZEB/nxaWKjOkGF1r CdEehBvivdZkpUoMupuk5/57a99APAtB2YJRq/8pX67gGHwBCe39ehuPq5oGiKMbQ9KO E1Aw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1778245687; x=1778850487; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=pgXLv83AtQy9mcAhCAUWdtflQBi69OvwGZ4unH4a9kc=; b=ZaEobsc0hRkKuXusK91T1TaBs5g3iV5pxO2UDYB3t/4NmSpvP1pyHtJGsJhhGN4RUi w9xqY30Z9OlkpbNKkV0EEH5GlEeUrR4QG0a/FaTEN9syr/zr3t26kYmtvmpxApLIjVmS /q1Y5PdukmkpUdt+WH4FFBDjJocjh7UlwsM6IOMPwWfsIlcL8GBYKFn0J6BjfhX0VNNr qnWWu7odV2DEBqb4m8QuPPTUVc7I2KvVhFkYJRumN4rmF1DYYdKcqkQzboRQ1UPPURlU DhRnEjzpiANIjczr1L+kJdH6Zdmk8g1wVoLWf0/07TLWan5JNLC4JvPhgHrEB5s2ckqI u9aA== X-Gm-Message-State: AOJu0Yxu0lwlUhuMvb6S1NQcvvRHouGbu+TadMojSFPIKAIdma/BRtNX gMOP3v4FFf+20GeE0S18ZnLxRD2Rb6drdwdfFHjC409UYGIjDOE9tU93vpMeoUxFKUU3PAfO47u IRa0c X-Gm-Gg: AeBDietElbDQlu5NQQJsu0Qnc8RuGZBlVAxMeDC3W4geYOYS6NAN/Mqs1caQav0k4GC j7/DBwNxtp34rHqM9tScArN0G5HFpXpkItBqNJrjJYAd1W4aWMQKscX2jQIaY6jCVR2CIe4JPDV VzJWmihdBfJ1XMKtYGSmUD+1vyl1nHVwJZNRu+96lUPTdwOKbppzYiEP7YpCZkoDZGeglqho2F+ xNBW57EsjaWSpkkjJ1hTfJqDLjN3bE/XJTerH568we7ZXQ5o7PfY0pC3bx8CgxLuJx1W5S3khVM 6hWmMeCVP4VcXZRq8Q83QBhYrdIgsxHwXDzP6MCWsTjda/dbbNpRQCgQQ8lAU2z7Bgvmw3Vj9tn ovwe+ckXcZzjDPvxWYQJLZeI2ZH93WiW2DO52NU61olnRWdhFFw4vb4T8scMSGNrROB/ZKWOU8H UTJUGRvIUdjQxR1YsTeYgg/hU7RhCyfox64qTfiMbU/oH8bNq8rGhZIDkVP593BVmrCv30FWV3/ bjK47zr+KHw3upFBSBPdlZII7MSZRPK5mWnZQ== X-Received: by 2002:a05:600c:4a13:b0:48d:35e:849d with SMTP id 5b1f17b1804b1-48e51e097d4mr122547485e9.6.1778245687403; Fri, 08 May 2026 06:08:07 -0700 (PDT) Received: from localhost.localdomain ([2a06:61c0:f337:0:9c1f:b517:931a:3b19]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-48e68f5d9b5sm34044905e9.15.2026.05.08.06.08.07 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 08 May 2026 06:08:07 -0700 (PDT) From: David Plowman To: libcamera-devel@lists.libcamera.org Cc: David Plowman , Naushir Patuck , Jacopo Mondi Subject: [PATCH v5 2/3] controls: rpi: Add ControlListSequence control Date: Fri, 8 May 2026 13:57:42 +0100 Message-ID: <20260508130804.8238-3-david.plowman@raspberrypi.com> X-Mailer: git-send-email 2.47.3 In-Reply-To: <20260508130804.8238-1-david.plowman@raspberrypi.com> References: <20260508130804.8238-1-david.plowman@raspberrypi.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" The ControlListSequence identifies the sequence number of the request whose controls have been applied to the images in this request for the first time. Signed-off-by: David Plowman Reviewed-by: Naushir Patuck Reviewed-by: Jacopo Mondi --- src/libcamera/control_ids_rpi.yaml | 9 +++++++++ 1 file changed, 9 insertions(+) diff --git a/src/libcamera/control_ids_rpi.yaml b/src/libcamera/control_ids_rpi.yaml index a8615112..0b7da5b2 100644 --- a/src/libcamera/control_ids_rpi.yaml +++ b/src/libcamera/control_ids_rpi.yaml @@ -183,4 +183,13 @@ controls: \sa SyncMode \sa SyncReady \sa SyncTimer + + - ControlListSequence: + type: int64_t + direction: out + description: | + This is the sequence number of the request whose control list has + just been applied. Controls normally take several frames to apply, + so the number here will refer to a request submitted a number of + frames earlier. ... From patchwork Fri May 8 12:57:43 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Plowman X-Patchwork-Id: 26693 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id AAFCFC32F6 for ; Fri, 8 May 2026 13:08:13 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 2F22363021; Fri, 8 May 2026 15:08:12 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="QjgoVAIx"; dkim-atps=neutral Received: from mail-wm1-x336.google.com (mail-wm1-x336.google.com [IPv6:2a00:1450:4864:20::336]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id CD62E62FE1 for ; Fri, 8 May 2026 15:08:08 +0200 (CEST) Received: by mail-wm1-x336.google.com with SMTP id 5b1f17b1804b1-4893940bb5eso12252505e9.3 for ; Fri, 08 May 2026 06:08:08 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; t=1778245688; x=1778850488; darn=lists.libcamera.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=p0mMKpb1i8B3EQhJz0nd+cLEbb2uFro+HYoXPQ9QXgw=; b=QjgoVAIxa9Pmgypr5d4vOeelXdQwvwntBu4NrBAZBKJ3HQrT/pWwb2CPf9f8Zx38Xn GHlZrtpIP4tXKZxb8li62uJsRHrtQ4P+p3hplAt4Uy3FYRNTEvNIFErTJgKOMtPbLbO+ sIkxHeus0n8PySaYk9Tlzx0VnAH6YfPHxP7KQI+B/P43UFsDatYFoln68VcSQ7g2dRc4 wItpcSi+g36/zkkdBA6YVHQcfHUh8AXv215/XUgpOrwsiXQS1ABT9czRDGnB6mP37Mma T9wmiQUgUtR1eH0uyYsv1rrUQJaWHs1dBahd87erbJ6RCeWl4qvspI4ibNXtjU11fjCi 7Atg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1778245688; x=1778850488; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=p0mMKpb1i8B3EQhJz0nd+cLEbb2uFro+HYoXPQ9QXgw=; b=DJrx4jE6QPYo9+WTIjXYrSAyp0pgBJNAflIen72G/Ndf8ABFWL/hv13NouapwbvXYe 1zW+EpQt0Ysb81gM6Sd/D3nqcapfsrRCNnnlB51SBlCoJBB+iHy9OK5f9fGTRraJWykB ShtRJAdj1byARu2fQrwhOC7+GV8d2fmkp/4ITGTM/i9jyh4NIw7FOdGjFTCpmYNKVNJ+ GbKvqWTBI4tcjaDrqnrQT3i/kYyPfavxWs4xZNaCFCplkNjtm0tvKqF57mVlepbRAZKB LqGg8fic6JMt+rvT4/7n7m3d6j3o0Qs5pdO1id3/L1FRUIYYFDpMlXTexctw9wtSLklR 40Ig== X-Gm-Message-State: AOJu0YwP+6V2vy5Irp8StQfwabvTmnqB+L+WsBKlHqzm+dr6nyCQGLeO KqtmkrYEXxg3I2gjY1MCujL49AWY9vm07PIbO45dogUpkUQfqBgaX8B4zTwAF52DrPtcaU/bjQ2 vT8An X-Gm-Gg: AeBDieuXG9KtKxzYExQ4PWQlIp9kmVvlkU+S4Wc/oLkbxgNAkqu1sD8sVkO1x7Yi6vw LIMK9ncyYmjwnCqHnvYmyOLLt6jIi13291UeddFLxMskwdNYJA7z3kVFaEXrHy+dTCEfBM/4CAs ob5FIyB6lNhSiPoIRXDOrhPRnwP6JV6HTA7SR6+FlvMHCFPddIpEPz7RSCmMxdMHiEkYveopKkj a/KBNPKtsMWYeaKPNZt26O9Yl/ZnWoNEjme9UiICM9tertukG4/3IV7VdN0R86XIJJBM12PFTtQ ss8ATFbWsLFiyP8DGCvDVn+XXC60atQBQlozbZAzvqx6ivbTR3R4Xq/ZM3L/U09D80HHjJuAC8H JscViiRTRg0QzIsd2zB2wQdGZpBL78aySFPJECEk0zq7gDvtr7eRI3YV0OrXBsaZOuzVlaXEXav R5CY7aXYbgXQ8VU/fdUdSYF129JXamXEquYoQzza1SeXvs66KJ0GY++OwmfzBQ9Lrjb5yEDt4IM GI1vR6hqu1YxRkeN50i+C/1S5dzPf7aoszACQ== X-Received: by 2002:a05:600c:198b:b0:48a:592c:e63d with SMTP id 5b1f17b1804b1-48e676a5245mr41755705e9.14.1778245687886; Fri, 08 May 2026 06:08:07 -0700 (PDT) Received: from localhost.localdomain ([2a06:61c0:f337:0:9c1f:b517:931a:3b19]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-48e68f5d9b5sm34044905e9.15.2026.05.08.06.08.07 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 08 May 2026 06:08:07 -0700 (PDT) From: David Plowman To: libcamera-devel@lists.libcamera.org Cc: David Plowman , Naushir Patuck Subject: [PATCH v5 3/3] pipeline: rpi: Make control lists in requests properly atomic Date: Fri, 8 May 2026 13:57:43 +0100 Message-ID: <20260508130804.8238-4-david.plowman@raspberrypi.com> X-Mailer: git-send-email 2.47.3 In-Reply-To: <20260508130804.8238-1-david.plowman@raspberrypi.com> References: <20260508130804.8238-1-david.plowman@raspberrypi.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" When a request is about to be processed, this commit separates "delayed controls" (camera-related ones that take "a few frames" to apply) from "immediate controls" (ISP-related controls) that will happen instantly. The immediate controls are held back until the delayed controls that they were submitted with, have happened. We merge back together the immediate controls that must happen now, along with the delayed controls that must be forwarded early, the result being that control lists submitted with a request happen atomically. We avoid overwriting the original control list submitted with the request. We therefore already have the sequence number of the request whose controls have just been applied, so we additionally attach this to the current request as "ControlId" metadata. Signed-off-by: David Plowman Reviewed-by: Naushir Patuck --- src/ipa/rpi/common/ipa_base.cpp | 24 ++++++--- .../pipeline/rpi/common/pipeline_base.cpp | 49 +++++++++++++++++++ .../pipeline/rpi/common/pipeline_base.h | 9 ++++ src/libcamera/pipeline/rpi/pisp/pisp.cpp | 11 +++-- src/libcamera/pipeline/rpi/vc4/vc4.cpp | 11 +++-- 5 files changed, 88 insertions(+), 16 deletions(-) diff --git a/src/ipa/rpi/common/ipa_base.cpp b/src/ipa/rpi/common/ipa_base.cpp index faa77719..dacafa57 100644 --- a/src/ipa/rpi/common/ipa_base.cpp +++ b/src/ipa/rpi/common/ipa_base.cpp @@ -431,6 +431,15 @@ void IpaBase::prepareIsp(const PrepareParams ¶ms) rpiMetadata.clear(); fillDeviceStatus(params.sensorControls, ipaContext); + /* + * When there are controls, it's important that we don't skip running the + * IPAs, as that can mess with synchronisation. Crucially though, we need + * to know whether there were controls when this comes back as the + * _delayed_ metadata, hence why we flag this in the metadata itself. + */ + if (!params.requestControls.empty()) + rpiMetadata.set("ipa.request_controls", true); + if (params.buffers.embedded) { /* * Pipeline handler has supplied us with an embedded data buffer, @@ -451,7 +460,7 @@ void IpaBase::prepareIsp(const PrepareParams ¶ms) */ AgcStatus agcStatus; bool hdrChange = false; - RPiController::Metadata &delayedMetadata = rpiMetadata_[params.delayContext]; + RPiController::Metadata &delayedMetadata = rpiMetadata_[params.delayContext % rpiMetadata_.size()]; if (!delayedMetadata.get("agc.status", agcStatus)) { rpiMetadata.set("agc.delayed_status", agcStatus); hdrChange = agcStatus.hdr.mode != hdrStatus_.mode; @@ -464,9 +473,13 @@ void IpaBase::prepareIsp(const PrepareParams ¶ms) */ helper_->prepare(embeddedBuffer, rpiMetadata); + bool delayedRequestControls = false; + delayedMetadata.get("ipa.request_controls", delayedRequestControls); + /* Allow a 10% margin on the comparison below. */ Duration delta = (frameTimestamp - lastRunTimestamp_) * 1.0ns; - if (lastRunTimestamp_ && frameCount_ > invalidCount_ && + if (!delayedRequestControls && params.requestControls.empty() && + lastRunTimestamp_ && frameCount_ > invalidCount_ && delta < controllerMinFrameDuration_ * 0.9 && !hdrChange) { /* * Ensure we merge the previous frame's metadata with the current @@ -535,7 +548,7 @@ void IpaBase::processStats(const ProcessParams ¶ms) ControlList ctrls(sensorCtrls_); applyAGC(&agcStatus, ctrls); rpiMetadata.set("agc.status", agcStatus); - setDelayedControls.emit(ctrls, ipaContext); + setDelayedControls.emit(ctrls, params.ipaContext); setCameraTimeoutValue(); } @@ -951,8 +964,6 @@ void IpaBase::applyControls(const ControlList &controls) /* The control provides units of microseconds. */ agc->setFixedExposureTime(0, ctrl.second.get() * 1.0us); - - libcameraMetadata_.set(controls::ExposureTime, ctrl.second.get()); break; } @@ -976,9 +987,6 @@ void IpaBase::applyControls(const ControlList &controls) break; agc->setFixedGain(0, ctrl.second.get()); - - libcameraMetadata_.set(controls::AnalogueGain, - ctrl.second.get()); break; } diff --git a/src/libcamera/pipeline/rpi/common/pipeline_base.cpp b/src/libcamera/pipeline/rpi/common/pipeline_base.cpp index f6ef8a3b..ace38997 100644 --- a/src/libcamera/pipeline/rpi/common/pipeline_base.cpp +++ b/src/libcamera/pipeline/rpi/common/pipeline_base.cpp @@ -1528,4 +1528,53 @@ void CameraData::fillRequestMetadata(const ControlList &bufferControls, Request } } +static bool isControlDelayed(unsigned int id) +{ + return id == controls::ExposureTime || + id == controls::AnalogueGain || + id == controls::FrameDurationLimits || + id == controls::AeEnable || + id == controls::ExposureTimeMode || + id == controls::AnalogueGainMode; +} + +void CameraData::handleControlLists(uint32_t delayContext, ControlList ¶mControls) +{ + /* + * The delayContext is the sequence number after it's gone through the various + * pipeline delays, so that's what gets reported as the "ControlListSequence" + * in the metadata, being the sequence number of the request whose ControlList + * has just been applied. + */ + Request *request = requestQueue_.front(); + request->_d()->metadata().set(controls::rpi::ControlListSequence, delayContext); + + /* + * Controls that take effect immediately (typically ISP controls) have to be + * delayed so as to synchronise with those controls that do get delayed. So we + * must remove them from the current request, and push them onto a queue so + * that they can be used later. + * + * Note that we are given a separate control list (paramControls) so that + * we can pass back the controls that really need to happen now, without + * disturbing the controls that were submitted with the request. + */ + ASSERT(paramControls.empty()); + immediateControls_.push({ request->sequence(), {} }); + for (const auto &ctrl : request->controls()) { + if (isControlDelayed(ctrl.first)) + paramControls.set(ctrl.first, ctrl.second); + else + immediateControls_.back().controls.set(ctrl.first, ctrl.second); + } + + /* "Immediate" controls that have become due are now merged back into this request. */ + while (!immediateControls_.empty() && + immediateControls_.front().controlListId <= delayContext) { + paramControls.merge(immediateControls_.front().controls, + ControlList::MergePolicy::OverwriteExisting); + immediateControls_.pop(); + } +} + } /* namespace libcamera */ diff --git a/src/libcamera/pipeline/rpi/common/pipeline_base.h b/src/libcamera/pipeline/rpi/common/pipeline_base.h index 7bfac33e..758155ee 100644 --- a/src/libcamera/pipeline/rpi/common/pipeline_base.h +++ b/src/libcamera/pipeline/rpi/common/pipeline_base.h @@ -180,10 +180,19 @@ public: ClockRecovery wallClockRecovery_; + struct ImmediateControlsEntry { + uint64_t controlListId; + ControlList controls; + }; + std::queue immediateControls_; + protected: void fillRequestMetadata(const ControlList &bufferControls, Request *request); + void handleControlLists(uint32_t delayContext, + ControlList ¶mControls); + virtual void tryRunPipeline() = 0; private: diff --git a/src/libcamera/pipeline/rpi/pisp/pisp.cpp b/src/libcamera/pipeline/rpi/pisp/pisp.cpp index cc7e32c3..b744c901 100644 --- a/src/libcamera/pipeline/rpi/pisp/pisp.cpp +++ b/src/libcamera/pipeline/rpi/pisp/pisp.cpp @@ -2322,9 +2322,6 @@ void PiSPCameraData::tryRunPipeline() fillRequestMetadata(job.sensorControls, request); - /* Set our state to say the pipeline is active. */ - state_ = State::Busy; - unsigned int bayerId = cfe_[Cfe::Output0].getBufferId(job.buffers[&cfe_[Cfe::Output0]]); unsigned int statsId = cfe_[Cfe::Stats].getBufferId(job.buffers[&cfe_[Cfe::Stats]]); ASSERT(bayerId && statsId); @@ -2341,7 +2338,13 @@ void PiSPCameraData::tryRunPipeline() params.ipaContext = requestQueue_.front()->sequence(); params.delayContext = job.delayContext; params.sensorControls = std::move(job.sensorControls); - params.requestControls = request->controls(); + /* params.requestControls is set by handleControlLists. */ + + /* This sorts out synchronisation with ControlLists in earlier requests. */ + handleControlLists(job.delayContext, params.requestControls); + + /* Set our state to say the pipeline is active. */ + state_ = State::Busy; if (sensorMetadata_) { unsigned int embeddedId = diff --git a/src/libcamera/pipeline/rpi/vc4/vc4.cpp b/src/libcamera/pipeline/rpi/vc4/vc4.cpp index f99cfdbc..3e9a4905 100644 --- a/src/libcamera/pipeline/rpi/vc4/vc4.cpp +++ b/src/libcamera/pipeline/rpi/vc4/vc4.cpp @@ -939,9 +939,6 @@ void Vc4CameraData::tryRunPipeline() fillRequestMetadata(bayerFrame.controls, request); - /* Set our state to say the pipeline is active. */ - state_ = State::Busy; - unsigned int bayer = unicam_[Unicam::Image].getBufferId(bayerFrame.buffer); LOG(RPI, Debug) << "Signalling prepareIsp:" @@ -950,10 +947,16 @@ void Vc4CameraData::tryRunPipeline() ipa::RPi::PrepareParams params; params.buffers.bayer = RPi::MaskBayerData | bayer; params.sensorControls = std::move(bayerFrame.controls); - params.requestControls = request->controls(); params.ipaContext = request->sequence(); params.delayContext = bayerFrame.delayContext; params.buffers.embedded = 0; + /* params.requestControls is set by handleControlLists. */ + + /* This sorts out synchronisation with ControlLists in earlier requests. */ + handleControlLists(bayerFrame.delayContext, params.requestControls); + + /* Set our state to say the pipeline is active. */ + state_ = State::Busy; if (embeddedBuffer) { unsigned int embeddedId = unicam_[Unicam::Embedded].getBufferId(embeddedBuffer);