From patchwork Fri Apr 29 19:34:32 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kieran Bingham X-Patchwork-Id: 15751 X-Patchwork-Delegate: kieran.bingham@ideasonboard.com Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 74EDAC0F2A for ; Fri, 29 Apr 2022 19:34:26 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 7C78365647; Fri, 29 Apr 2022 21:34:25 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1651260865; bh=XpMx92d9ohZG71rvFQnype5rF2q57xwiF6S5Lz/tXvk=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To: From; b=Jp5wTw3/+c6da9A+P0jpJu2uxYVCUrn4LiwZhI7vVNXD/V36rmJY+6NBSQEz2BSKB hjBbsmOha1ujY1jMB2BOX8KSrRo4jECOUyfNdoYZbmxK1Mk6fQIjtF4qLaKxLbodaR i0RdrlOb8WH5SeQg/aB5cpf58ecdavcZncDXq1LdYf+R0V5b6RmqgxEi77VnH1ZXN6 ErDMI3OILZKhDx9xOG7godeu03NmQyhraG09ayyVdAzadpybnHlwkYB0ADTTG1E7Ik iy4Q6+LJii520TjOQr0oD7zpQAfHWMBI7PadK+0E4uKZh1PiuJQltatFJWh0HytHnQ XNyOzAzNvydbw== Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 0D8F2604A5 for ; Fri, 29 Apr 2022 21:34:23 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="P8b9AvCJ"; dkim-atps=neutral Received: from Q.ksquared.org.uk.beta.tailscale.net (unknown [178.237.134.231]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 859AA475; Fri, 29 Apr 2022 21:34:22 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1651260862; bh=XpMx92d9ohZG71rvFQnype5rF2q57xwiF6S5Lz/tXvk=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=P8b9AvCJcPqw3l+hx0mFe5B5YvHZhpRKirNY90RD/cZEJqq2jJQ+qDNIHYWrpjKhz Br0PMLrqBt/aHl4yq85GpXy/KyzbcjP0oUiPbd50r3EYu19iBEx4nU/Ap2nO75P7l8 dac4RDUELRlbsWgSwhGdXy4PdCCOG4cbJNfYfe1M= To: libcamera devel Date: Fri, 29 Apr 2022 21:34:32 +0200 Message-Id: <20220429193434.167990-2-kieran.bingham@ideasonboard.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220429193434.167990-1-kieran.bingham@ideasonboard.com> References: <20220429193434.167990-1-kieran.bingham@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH 1/3] libcamera: Add sequence value observer X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Kieran Bingham via libcamera-devel From: Kieran Bingham Reply-To: Kieran Bingham Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Components will often need to track and store a sequence number. Sequence numbers should be monotonically incrementing, but may require features such as validating that an external source matches this one, or ensuring that there are no gaps. Support the operations needed on sequence numbers in a dedicated class to reduce code duplication across components. Signed-off-by: Kieran Bingham Reviewed-by: Umang Jain --- include/libcamera/meson.build | 1 + include/libcamera/sequence.h | 20 +++++++++++ src/libcamera/meson.build | 1 + src/libcamera/sequence.cpp | 66 +++++++++++++++++++++++++++++++++++ 4 files changed, 88 insertions(+) create mode 100644 include/libcamera/sequence.h create mode 100644 src/libcamera/sequence.cpp diff --git a/include/libcamera/meson.build b/include/libcamera/meson.build index 408b7acf152c..2911d4bf4ec7 100644 --- a/include/libcamera/meson.build +++ b/include/libcamera/meson.build @@ -14,6 +14,7 @@ libcamera_public_headers = files([ 'logging.h', 'pixel_format.h', 'request.h', + 'sequence.h', 'stream.h', 'transform.h', ]) diff --git a/include/libcamera/sequence.h b/include/libcamera/sequence.h new file mode 100644 index 000000000000..ad6e99726a4e --- /dev/null +++ b/include/libcamera/sequence.h @@ -0,0 +1,20 @@ +#pragma once + +#include + +#include + +namespace libcamera { + +class Sequence +{ +public: + __nodiscard int update(unsigned int seq); + void reset() { sequence_.reset(); } + +private: + std::optional sequence_; +}; + +}; // namespace libcamera + diff --git a/src/libcamera/meson.build b/src/libcamera/meson.build index 26912ca17819..ffdffeb4bfdc 100644 --- a/src/libcamera/meson.build +++ b/src/libcamera/meson.build @@ -38,6 +38,7 @@ libcamera_sources = files([ 'process.cpp', 'pub_key.cpp', 'request.cpp', + 'sequence.cpp', 'source_paths.cpp', 'stream.cpp', 'sysfs.cpp', diff --git a/src/libcamera/sequence.cpp b/src/libcamera/sequence.cpp new file mode 100644 index 000000000000..374b4f04d0a3 --- /dev/null +++ b/src/libcamera/sequence.cpp @@ -0,0 +1,66 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2022, Ideas on Board Oy. + * + * sequence.cpp Sequence Number Observer + */ + +#include + +#include + +/** + * \file sequence.h + * \brief Sequence number observer + */ + +namespace libcamera { + +/** + * \class Sequence + * \brief Sequence number tracking which expects monotonically incrementing + * numbers + * + * The Sequence number observer is initialised with the first value it is given. + * It will return a difference of the expected update value, against the newly + * provided value - allowing the consumer to identify if a break in a sequence + * has occured. + */ + +/** + * \brief Update the sequence observer with the latest value + * \param seq The latest value for the sequence + * + * This function will update the state of the Sequence observer and identify any + * non-monotonic increment or change that may occur and return the difference + * from the expected update value. + * + * The sequence is initialised to the first value passed into \a update. + * + * \return The number of drops in the sequence that were detected + */ +__nodiscard int Sequence::update(unsigned int seq) +{ + if (!sequence_) + sequence_ = seq - 1; + + /* + * Any update expects a single integer difference from + * the previous value. + */ + int diff = seq - sequence_.value() - 1; + + sequence_ = seq; + + return diff; +}; + +/** + * \fn Sequence::reset + * \brief Resets the sequence observer + * + * Re-initialises the sequence observer so that any known break in the monotonic + * sequence is not reported. + */ + +} /* namespace libcamera */ From patchwork Fri Apr 29 19:34:33 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kieran Bingham X-Patchwork-Id: 15752 X-Patchwork-Delegate: kieran.bingham@ideasonboard.com Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 4495DC326C for ; Fri, 29 Apr 2022 19:34:27 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id EB42B6564C; Fri, 29 Apr 2022 21:34:26 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1651260866; bh=MEXKjD8dZehuJXB/0YoKuGf/nuPVIYzzNduzLWMkliI=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To: From; b=YbJSEBBT2jjULFv1xmZzanTeiFyjc2f8YezpBXXYp2EwaZ09HDQO2T4FZGk1qkxWj PJXUjxbG09yeFbRmIjbpa9rdFrXUYoKzmJK/7SFs7cKTrBYFWShOpjJuz+kTxgtM/0 D37lhF9/7UtLJj0duLd7O3JwCE2NV7rXzdcyeHauIXjjOItcRMijNSfZLLlRdGRLT/ 1Wn8tLAwDBSRgC6fZAeX7qXGB3WvPm/E/uEC+D/aIaNRTZtZyIEAeGc9Ga0tntWNQ+ +K0NnkZLOolHaufoBx6K/7qlTHC+Rgcgl/ZAb3tYjJuNjvTgjbV/0U2eYP8b1YjxHu jhNJDYkiKMwzA== Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 91C0765640 for ; Fri, 29 Apr 2022 21:34:23 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="IVpb/14s"; dkim-atps=neutral Received: from Q.ksquared.org.uk.beta.tailscale.net (unknown [178.237.134.231]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 15A1745F; Fri, 29 Apr 2022 21:34:23 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1651260863; bh=MEXKjD8dZehuJXB/0YoKuGf/nuPVIYzzNduzLWMkliI=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=IVpb/14sbSgzNZXuEvNNaomf/AUYgBQlYIO23Qa3twMkvvr4iBd6GmwkH+ygdDaHF pj18wL940jpF9UNzt1kkUFCHXPmEsd55ZfT12KmCPUxubcmUoP2//U6Vywqzip1asj +gO+czxn0oQVmNncDLdtw5WuPV73Sv9pokF0FJH0= To: libcamera devel Date: Fri, 29 Apr 2022 21:34:33 +0200 Message-Id: <20220429193434.167990-3-kieran.bingham@ideasonboard.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220429193434.167990-1-kieran.bingham@ideasonboard.com> References: <20220429193434.167990-1-kieran.bingham@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH 2/3] test: Add Sequence observer tests X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Kieran Bingham via libcamera-devel From: Kieran Bingham Reply-To: Kieran Bingham Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Validate the Sequence object for a set of possible permutations. Signed-off-by: Kieran Bingham Reviewed-by: Umang Jain --- test/meson.build | 1 + test/sequence.cpp | 80 +++++++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 81 insertions(+) create mode 100644 test/sequence.cpp diff --git a/test/meson.build b/test/meson.build index fd4c5ca07c15..7c53fe5d0d76 100644 --- a/test/meson.build +++ b/test/meson.build @@ -47,6 +47,7 @@ internal_tests = [ ['object-delete', 'object-delete.cpp'], ['object-invoke', 'object-invoke.cpp'], ['pixel-format', 'pixel-format.cpp'], + ['sequence', 'sequence.cpp'], ['shared-fd', 'shared-fd.cpp'], ['signal-threads', 'signal-threads.cpp'], ['threads', 'threads.cpp'], diff --git a/test/sequence.cpp b/test/sequence.cpp new file mode 100644 index 000000000000..4afb9998527a --- /dev/null +++ b/test/sequence.cpp @@ -0,0 +1,80 @@ +/* SPDX-License-Identifier: GPL-2.0-or-later */ +/* + * Copyright (C) 2022, Ideas on Board Oy. + * + * sequence.cpp - Sequence observer tests + */ + +#include + +#include + +#include "test.h" + +using namespace std; +using namespace libcamera; + +class SequenceTest : public Test +{ +private: + int Fail(std::string m) + { + cout << m << endl; + return TestFail; + } + +protected: + int run() + { + Sequence seq; + int diff; + + /* Validate non-zero initialization */ + diff = seq.update(10); + if (diff) + return Fail("Initialisation test failed"); + + diff = seq.update(11); + if (diff) + return Fail("Sequential sequence failure"); + + /* Validate 1 drop */ + diff = seq.update(13); + if (diff != 1) + return Fail("Sequence gap not detected"); + + /* Validate 10 drops - currently expect sequence 14 */ + diff = seq.update(24); + if (diff != 10) + return Fail("Large sequence gap not detected"); + + /* Validate reset */ + seq.reset(); + diff = seq.update(50); + if (diff) + return Fail("Reset failed"); + + /* Validate reverse sequence detected */ + diff = seq.update(49); + if (diff == 0) + return Fail("Reverse sequence detection error"); + + /* Validate integer wrap around (Shouldn't ever happen but...) */ + seq.reset(); + diff = seq.update(-2); + if (diff) + return Fail("Integer wrap test reset failed"); + + diff = seq.update(-1); + if (diff) + return Fail("Negative sequence failed"); + + diff = seq.update(0); + if (diff) + return Fail("Integer wrap test failed"); + + return TestPass; + } +}; + +TEST_REGISTER(SequenceTest) From patchwork Fri Apr 29 19:34:34 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kieran Bingham X-Patchwork-Id: 15753 X-Patchwork-Delegate: kieran.bingham@ideasonboard.com Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 031C9C0F2A for ; Fri, 29 Apr 2022 19:34:27 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 95F5465649; Fri, 29 Apr 2022 21:34:27 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1651260867; bh=oRF/c0525Cus9kCvVz/f8Vi+n7Gq6xMrXq8wK4Dt6hw=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To: From; b=ws/Vw9TEz8hg8f5WC8jSns5WqRTWYzEeNe01QMaZ/D8fIhwPhk1pC6Xkl3qU5nmX6 EQrCdXga9VQtxOvwOORdS8s88pNZRVa3loHfATolebm/IxZDo5mOubjFroZjcr3BYY pBTNVg57xEbSF1sR2U6pWzN5J8QgwuHW7NPFPVxl7zNzrvzdpKsG/l73N47YSFX1B1 NnSnBk0O0ApFR+rQstQA+pRArvoIS0b2j8k7s+NB7k2MRjMkwG/YwZ+JTnnnfeC5J7 cf1/8pLQk69rUPZuucAUAbLMBDlLVu1Y+takMknj0qlk37K4C2TgxfPmEL3Q7Rwv+S QwuzQBmTIO0EQ== Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 1B52D65640 for ; Fri, 29 Apr 2022 21:34:24 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="IqKNT10U"; dkim-atps=neutral Received: from Q.ksquared.org.uk.beta.tailscale.net (unknown [178.237.134.231]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id A0C84475; Fri, 29 Apr 2022 21:34:23 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1651260863; bh=oRF/c0525Cus9kCvVz/f8Vi+n7Gq6xMrXq8wK4Dt6hw=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=IqKNT10UJm6iwr69UlSK1zzTEmrjOt98v7ixajLFv0i/p1DG5aEPA3Kz51rsiKlRR OWaAfvOdvZirPswTFZ2UT3+GeCYIJxYICI3QWkAJ7tE12VY3XJ2Ei+609jdfJroPTY iT9Pra2+iKNXFK/UfgRmj06zd9RUwQmCj/WmJymY= To: libcamera devel Date: Fri, 29 Apr 2022 21:34:34 +0200 Message-Id: <20220429193434.167990-4-kieran.bingham@ideasonboard.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220429193434.167990-1-kieran.bingham@ideasonboard.com> References: <20220429193434.167990-1-kieran.bingham@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH 3/3] libcamera: v4l2: Detect frame drops on V4L2VideoDevice X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Kieran Bingham via libcamera-devel From: Kieran Bingham Reply-To: Kieran Bingham Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Pipelines with ISPs will report the output sequence number of the V4L2VideoDevice of the ISP as the sequence number of the request buffer completion. These have been seen to provide their own monotonic sequence number, which means that any frame drop from the CSI2 receiver is silently ignored and can easily go unnoticed. (except for frame rate changes). Add an internal tracking of the sequence number on V4L2VideoDevice instances to identify if there is ever a break in the stream. Signed-off-by: Kieran Bingham Reviewed-by: Umang Jain --- include/libcamera/internal/v4l2_videodevice.h | 2 ++ src/libcamera/v4l2_videodevice.cpp | 10 ++++++++++ 2 files changed, 12 insertions(+) diff --git a/include/libcamera/internal/v4l2_videodevice.h b/include/libcamera/internal/v4l2_videodevice.h index 9c9493cc16ed..ab4c4c56436d 100644 --- a/include/libcamera/internal/v4l2_videodevice.h +++ b/include/libcamera/internal/v4l2_videodevice.h @@ -28,6 +28,7 @@ #include #include #include +#include #include "libcamera/internal/formats.h" #include "libcamera/internal/v4l2_device.h" @@ -273,6 +274,7 @@ private: EventNotifier *fdBufferNotifier_; State state_; + Sequence monotonicObserver_; Timer watchdog_; utils::Duration watchdogDuration_; diff --git a/src/libcamera/v4l2_videodevice.cpp b/src/libcamera/v4l2_videodevice.cpp index 05d3273ebb58..4cef5d40e0c6 100644 --- a/src/libcamera/v4l2_videodevice.cpp +++ b/src/libcamera/v4l2_videodevice.cpp @@ -1749,6 +1749,14 @@ FrameBuffer *V4L2VideoDevice::dequeueBuffer() if (V4L2_TYPE_IS_OUTPUT(buf.type)) return buffer; + /* + * Observe the buffer sequence number on capture devices to check for + * frame drops. + */ + int drops = monotonicObserver_.update(buf.sequence); + if (drops) + LOG(V4L2, Warning) << "Dropped " << drops << " frames"; + unsigned int numV4l2Planes = multiPlanar ? buf.length : 1; FrameMetadata &metadata = buffer->metadata_; @@ -1832,6 +1840,8 @@ int V4L2VideoDevice::streamOn() return ret; } + monotonicObserver_.reset(); + state_ = State::Streaming; if (watchdogDuration_.count()) watchdog_.start(std::chrono::duration_cast(watchdogDuration_));