From patchwork Mon Sep 14 14:02:39 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefan Klug X-Patchwork-Id: 28270 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id EC8A7C3226 for ; Mon, 14 Sep 2026 14:04:37 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 84D2C686FC; Mon, 14 Sep 2026 16:04:37 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="HTSwPGDC"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 3EFFF686D4 for ; Mon, 14 Sep 2026 16:04:36 +0200 (CEST) Received: from ideasonboard.com (unknown [IPv6:2a00:6020:448c:6c00:a279:75fa:1f6c:7f40]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 3BB77929; Mon, 14 Sep 2026 16:02:56 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1789394576; bh=RgJeZuCuLUZXqt4RcR5NXpcplWtH6zhGbt4PDH8M3o4=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=HTSwPGDC90mjOd7IKMHkeJD5QDAiVIN25jt0WRG1ktNzZ81d90JkQTaLnGyMm8Qbz 4jivW+eySmKJNSgEBtzFH0UB22usgms1OAsOCdsdjwmQrQKNDmcwoGDq9b/dQTUSsB g3qkyT2svdCmtJ+UO+L9Y9hmJKSFitfy9CX+CMwk= From: Stefan Klug To: libcamera-devel@lists.libcamera.org Cc: Stefan Klug Subject: [PATCH v3 26/41] libcamera: internal: Add SequenceSyncHelper class Date: Mon, 14 Sep 2026 16:02:39 +0200 Message-ID: <20260914140309.3354666-27-stefan.klug@ideasonboard.com> X-Mailer: git-send-email 2.53.0 In-Reply-To: <20260914140309.3354666-1-stefan.klug@ideasonboard.com> References: <20260914140309.3354666-1-stefan.klug@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" On a V4L2 buffer the assigned sequence is not known until the buffer is dequeued. But for per frame controls we have to prepare other data like sensor controls and ISP params in advance. So we try to anticipate the sequence number a given buffer will be. In a perfect world this works well as long as the initial sequence is assigned correctly. But it breaks as soon as things like running out of buffers or incomplete images happen. To make things even more complicated, in most cases more than one buffer is queued to the kernel at a time. So as soon as a sequence number doesn't match the expected one after dequeuing, most likely all the already queued buffers will be dequeued with the same error. It is not sufficient to simply add the correction after dequeuing because the error on all queued frames would accumulate and the whole system starts to oscillate. To work around that add a SequenceSyncHelper class that tracks the expected error and allows to easily query the necessary correction when queuing new buffers. Signed-off-by: Stefan Klug --- Changes in v3: - Moved implementation into cpp - Renamed some functions - Added class documentation Changes in v2: - Moved files to man src dir, to be able to reuse it in other pipelines - Added cancel() function. --- include/libcamera/internal/meson.build | 1 + .../libcamera/internal/sequence_sync_helper.h | 31 +++++ src/libcamera/meson.build | 1 + src/libcamera/sequence_sync_helper.cpp | 113 ++++++++++++++++++ 4 files changed, 146 insertions(+) create mode 100644 include/libcamera/internal/sequence_sync_helper.h create mode 100644 src/libcamera/sequence_sync_helper.cpp diff --git a/include/libcamera/internal/meson.build b/include/libcamera/internal/meson.build index fd375134a5c4..73635e31bad9 100644 --- a/include/libcamera/internal/meson.build +++ b/include/libcamera/internal/meson.build @@ -40,6 +40,7 @@ libcamera_internal_headers = files([ 'process.h', 'pub_key.h', 'request.h', + 'sequence_sync_helper.h', 'shared_mem_object.h', 'source_paths.h', 'sysfs.h', diff --git a/include/libcamera/internal/sequence_sync_helper.h b/include/libcamera/internal/sequence_sync_helper.h new file mode 100644 index 000000000000..d8dcfe2ae81d --- /dev/null +++ b/include/libcamera/internal/sequence_sync_helper.h @@ -0,0 +1,31 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2025, Ideas on Board + * + * Sequence sync helper + */ + +#pragma once + +#include + +#include + +namespace libcamera { + +class SequenceSyncHelper +{ +public: + int receivedFrame(size_t expectedSequence, size_t actualSequence); + void cancelFrame(); + int correction(); + void pushCorrection(int correction); + void reset(); + +private: + std::queue corrections_; + int correctionToApply_ = 0; + int expectedOffset_ = 0; +}; + +} /* namespace libcamera */ diff --git a/src/libcamera/meson.build b/src/libcamera/meson.build index 17c1b2cb3479..038d2dbf12cf 100644 --- a/src/libcamera/meson.build +++ b/src/libcamera/meson.build @@ -49,6 +49,7 @@ libcamera_internal_sources = files([ 'pipeline_handler.cpp', 'process.cpp', 'pub_key.cpp', + 'sequence_sync_helper.cpp', 'shared_mem_object.cpp', 'source_paths.cpp', 'sysfs.cpp', diff --git a/src/libcamera/sequence_sync_helper.cpp b/src/libcamera/sequence_sync_helper.cpp new file mode 100644 index 000000000000..7c4e2b423666 --- /dev/null +++ b/src/libcamera/sequence_sync_helper.cpp @@ -0,0 +1,113 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2026, Ideas on Board. + * + * Helper to synchronize buffer sequences + */ + +#include "libcamera/internal/sequence_sync_helper.h" + +#include + +namespace libcamera { + +LOG_DEFINE_CATEGORY(SequenceSyncHelper) + +/** + * \file sequence_sync_helper.h + * \class SequenceSyncHelper + * \brief Helper to synchronize buffer sequences + * + * On a V4L2 buffers the sequence is not known until the buffer was dequeued. To + * pre plan regulation it is however necessary to know the sequence of a buffer + * before queueing the buffer (or multiple buffers). By the time an offset is + * detected in most cases more than one buffers are already queued in and all of + * them carry the same error. Simple adding the perceived difference on dequeue + * therefore doesn't help and leads to oscillations. This class tracks the + * sequence corrections over time and helps in keeping the sequence numbers in + * sync. + */ + +/** + * \brief Tell the sync helper that a frame was received + * \param expectedSequence The sequence that was expected for that frame + * \param actualSequence The actual sequence of the frame + * + * This function needs to be called when a frame was dequeued. The sync helper + * calculates necessary corrections and keeps track of corrections already + * applied. + */ +int SequenceSyncHelper::receivedFrame(size_t expectedSequence, + size_t actualSequence) +{ + ASSERT(!corrections_.empty()); + int diff = actualSequence - expectedSequence; + int corr = corrections_.front(); + corrections_.pop(); + expectedOffset_ -= corr; + int necessaryCorrection = diff - expectedOffset_; + correctionToApply_ += necessaryCorrection; + + LOG(SequenceSyncHelper, Debug) + << "Sync frame " + << "expected: " << expectedSequence + << " actual: " << actualSequence + << " correction: " << corr + << " expectedOffset: " << expectedOffset_ + << " correctionToApply " << correctionToApply_; + + expectedOffset_ += necessaryCorrection; + return necessaryCorrection; +} + +/** + * \brief Tell the sync helper that a frame was cancelled + * + * This function needs to be called when a frame was cancelled. + */ +void SequenceSyncHelper::cancelFrame() +{ + int corr = corrections_.front(); + corrections_.pop(); + expectedOffset_ -= corr; +} + +/** + * \brief Get the necessary correction + * + * Get the correction that must be applied to the sequence numbers to + * synchronize. + * + * \return The correction to apply + */ +int SequenceSyncHelper::correction() +{ + return correctionToApply_; +} + +/** + * \brief Tell the sync helper that a correction was pushed + * + * This must be called for every frame that gets pushed into the queue. If + * no correction was applied, it must be called with a correction of 0. + */ +void SequenceSyncHelper::pushCorrection(int correction) +{ + corrections_.push(correction); + correctionToApply_ -= correction; + LOG(SequenceSyncHelper, Debug) + << "Push correction " << correction + << " correctionToApply " << correctionToApply_; +} + +/** + * \brief Reset the sync helper + */ +void SequenceSyncHelper::reset() +{ + corrections_ = {}; + correctionToApply_ = 0; + expectedOffset_ = 0; +} + +} /* namespace libcamera */