{"id":26359,"url":"https://patchwork.libcamera.org/api/1.1/patches/26359/?format=json","web_url":"https://patchwork.libcamera.org/patch/26359/","project":{"id":1,"url":"https://patchwork.libcamera.org/api/1.1/projects/1/?format=json","name":"libcamera","link_name":"libcamera","list_id":"libcamera_core","list_email":"libcamera-devel@lists.libcamera.org","web_url":"","scm_url":"","webscm_url":""},"msgid":"<20260325151416.2114564-18-stefan.klug@ideasonboard.com>","date":"2026-03-25T15:13:49","name":"[v2,17/32] libcamera: delayed_controls: Change semantics of sequence numbers","commit_ref":null,"pull_url":null,"state":"new","archived":false,"hash":"ad920dded2e609fccd55e49a7d39cc109e0bf3d3","submitter":{"id":184,"url":"https://patchwork.libcamera.org/api/1.1/people/184/?format=json","name":"Stefan Klug","email":"stefan.klug@ideasonboard.com"},"delegate":null,"mbox":"https://patchwork.libcamera.org/patch/26359/mbox/","series":[{"id":5849,"url":"https://patchwork.libcamera.org/api/1.1/series/5849/?format=json","web_url":"https://patchwork.libcamera.org/project/libcamera/list/?series=5849","date":"2026-03-25T15:13:32","name":"rkisp1: pipeline rework for PFC","version":2,"mbox":"https://patchwork.libcamera.org/series/5849/mbox/"}],"comments":"https://patchwork.libcamera.org/api/patches/26359/comments/","check":"pending","checks":"https://patchwork.libcamera.org/api/patches/26359/checks/","tags":{},"headers":{"Return-Path":"","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id 61131C32F6\n\tfor ;\n\tWed, 25 Mar 2026 15:15:30 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id 10F0A62C1A;\n\tWed, 25 Mar 2026 16:15:30 +0100 (CET)","from perceval.ideasonboard.com (perceval.ideasonboard.com\n\t[IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id 1C76A62BC0\n\tfor ;\n\tWed, 25 Mar 2026 16:15:29 +0100 (CET)","from ideasonboard.com (unknown\n\t[IPv6:2a00:6020:448c:6c00:b16a:5ed9:4ada:a95a])\n\tby perceval.ideasonboard.com (Postfix) with UTF8SMTPSA id 1F8801B98; \n\tWed, 25 Mar 2026 16:14:11 +0100 (CET)"],"Authentication-Results":"lancelot.ideasonboard.com; dkim=pass (1024-bit key;\n\tunprotected) header.d=ideasonboard.com header.i=@ideasonboard.com\n\theader.b=\"BpXjWCLU\"; dkim-atps=neutral","DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com;\n\ts=mail; t=1774451651;\n\tbh=K0nTSvVIRBMzsU2O5sVVts7K7edW3VA1pbvMF5UEXU8=;\n\th=From:To:Cc:Subject:Date:In-Reply-To:References:From;\n\tb=BpXjWCLUOG4K9B64zpECQkY+EGLH6XaJotMk1LwSmiUtRhDthrNjreoon5m9SB1Kf\n\t1DGayPRYuOj9JJSSAlqHzaO55EigqU92lim5SYa1oVe8NLNGE7OkUmbwLRWIoEH0iv\n\tzpk/yUEeiDyFxVWnj5v1IyTDKDuNl2hEvnlGxwIo=","From":"Stefan Klug ","To":"libcamera-devel@lists.libcamera.org","Cc":"Stefan Klug ","Subject":"[PATCH v2 17/32] libcamera: delayed_controls: Change semantics of\n\tsequence numbers","Date":"Wed, 25 Mar 2026 16:13:49 +0100","Message-ID":"<20260325151416.2114564-18-stefan.klug@ideasonboard.com>","X-Mailer":"git-send-email 2.51.0","In-Reply-To":"<20260325151416.2114564-1-stefan.klug@ideasonboard.com>","References":"<20260325151416.2114564-1-stefan.klug@ideasonboard.com>","MIME-Version":"1.0","Content-Transfer-Encoding":"8bit","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" "},"content":"In the context of per frame controls I believe it is easier to think\nabout what needs to be done for a specific sequence number.\n\nSo (assuming a max sensor delay of 2) the actions would be:\n\n- delayedControls.push(n) stores the controls that shall be active on\n frame n\n- delayedControls.get(n) returns these controls again\n- delayedControls.apply(n) applies the slowest control for frame n and\n does a look back on other controls. So when a frameStart for frame n\noccurs, it is time to call delayedControls.apply(n + maxDelay)\n\nChanging these semantics on delayed controls doesn't require much code\nchange and has the added benefit that we don't run into clamping for\nget() on frames < maxDelay.\n\nToDo: This breaks the delayed controls test at the moment. The tests\nneed to be fixed. This is not straight forward as the tests need a\ndeeper overhaul.\n\nSigned-off-by: Stefan Klug \n\n---\n\nChanges in v2:\n- Applied the semantics change to other pipelines as well.\n---\n src/libcamera/delayed_controls.cpp | 15 ++++++++-------\n src/libcamera/pipeline/ipu3/ipu3.cpp | 2 +-\n src/libcamera/pipeline/mali-c55/mali-c55.cpp | 6 ++++--\n src/libcamera/pipeline/rkisp1/rkisp1.cpp | 3 ++-\n test/meson.build | 2 +-\n 5 files changed, 16 insertions(+), 12 deletions(-)","diff":"diff --git a/src/libcamera/delayed_controls.cpp b/src/libcamera/delayed_controls.cpp\nindex 71071a0c670d..4efe3b39c3f6 100644\n--- a/src/libcamera/delayed_controls.cpp\n+++ b/src/libcamera/delayed_controls.cpp\n@@ -240,7 +240,7 @@ bool DelayedControls::push(uint32_t sequence, const ControlList &controls)\n */\n ControlList DelayedControls::get(uint32_t sequence)\n {\n-\tunsigned int index = std::max(0, sequence - maxDelay_);\n+\tunsigned int index = sequence;\n \n \tControlList out(device_->controls());\n \tfor (const auto &ctrl : values_) {\n@@ -266,13 +266,14 @@ ControlList DelayedControls::get(uint32_t sequence)\n */\n \n /**\n- * \\brief Inform DelayedControls of the start of a new frame\n- * \\param[in] sequence Sequence number of the frame that started\n+ * \\brief Apply controls for a frame\n+ * \\param[in] sequence Sequence number of the frame to apply\n *\n- * Inform the state machine that a new frame has started and of its sequence\n- * number. Any user of these helpers is responsible to inform the helper about\n- * the start of any frame. This can be connected with ease to the start of a\n- * exposure (SOE) V4L2 event.\n+ * Apply controls for the frame \\a sequence. This applies the controls with the\n+ * largest delay. For controls with a smaller delay it does a look back and\n+ * applies the controls for the previous sequence. So usually this function is\n+ * called in a start of exposure event as applyControls(startedSequence +\n+ * maxDelay)\n */\n void DelayedControls::applyControls(uint32_t sequence)\n {\ndiff --git a/src/libcamera/pipeline/ipu3/ipu3.cpp b/src/libcamera/pipeline/ipu3/ipu3.cpp\nindex bac6f1c2ef40..4e7e8cd49e13 100644\n--- a/src/libcamera/pipeline/ipu3/ipu3.cpp\n+++ b/src/libcamera/pipeline/ipu3/ipu3.cpp\n@@ -1385,7 +1385,7 @@ void IPU3CameraData::statBufferReady(FrameBuffer *buffer)\n */\n void IPU3CameraData::frameStart(uint32_t sequence)\n {\n-\tdelayedCtrls_->applyControls(sequence);\n+\tdelayedCtrls_->applyControls(sequence + delayedCtrls_->maxDelay());\n \n \tif (processingRequests_.empty())\n \t\treturn;\ndiff --git a/src/libcamera/pipeline/mali-c55/mali-c55.cpp b/src/libcamera/pipeline/mali-c55/mali-c55.cpp\nindex c209b0b070b1..2a8bbca098bb 100644\n--- a/src/libcamera/pipeline/mali-c55/mali-c55.cpp\n+++ b/src/libcamera/pipeline/mali-c55/mali-c55.cpp\n@@ -1615,8 +1615,10 @@ bool PipelineHandlerMaliC55::registerSensorCamera(MediaLink *ispLink)\n \t\tdata->delayedCtrls_ =\n \t\t\tstd::make_unique(data->sensor_->device(),\n \t\t\t\t\t\t\t params);\n-\t\tisp_->frameStart.connect(data->delayedCtrls_.get(),\n-\t\t\t\t\t &DelayedControls::applyControls);\n+\t\tisp_->frameStart.connect(data->delayedCtrls_.get(), [&](uint32_t seq) {\n+\t\t\tuint32_t lookahead = data->delayedCtrls_->maxDelay();\n+\t\t\tdata->delayedCtrls_->applyControls(seq + lookahead);\n+\t\t});\n \n \t\t/* \\todo Init properties. */\n \ndiff --git a/src/libcamera/pipeline/rkisp1/rkisp1.cpp b/src/libcamera/pipeline/rkisp1/rkisp1.cpp\nindex 89e8ab0999f4..b70c551fc775 100644\n--- a/src/libcamera/pipeline/rkisp1/rkisp1.cpp\n+++ b/src/libcamera/pipeline/rkisp1/rkisp1.cpp\n@@ -1523,7 +1523,8 @@ void PipelineHandlerRkISP1::frameStart(uint32_t sequence)\n \t\treturn;\n \n \tRkISP1CameraData *data = cameraData(activeCamera_);\n-\tdata->delayedCtrls_->applyControls(sequence);\n+\tuint32_t sequenceToApply = sequence + data->delayedCtrls_->maxDelay();\n+\tdata->delayedCtrls_->applyControls(sequenceToApply);\n }\n \n bool PipelineHandlerRkISP1::match(DeviceEnumerator *enumerator)\ndiff --git a/test/meson.build b/test/meson.build\nindex 52f04364e4fc..80fb543f2201 100644\n--- a/test/meson.build\n+++ b/test/meson.build\n@@ -53,7 +53,7 @@ internal_tests = [\n {'name': 'bayer-format', 'sources': ['bayer-format.cpp']},\n {'name': 'byte-stream-buffer', 'sources': ['byte-stream-buffer.cpp']},\n {'name': 'camera-sensor', 'sources': ['camera-sensor.cpp']},\n- {'name': 'delayed_controls', 'sources': ['delayed_controls.cpp']},\n+ {'name': 'delayed_controls', 'sources': ['delayed_controls.cpp'], 'should_fail': true},\n {'name': 'event', 'sources': ['event.cpp']},\n {'name': 'event-dispatcher', 'sources': ['event-dispatcher.cpp']},\n {'name': 'event-thread', 'sources': ['event-thread.cpp']},\n","prefixes":["v2","17/32"]}