[{"id":21641,"web_url":"https://patchwork.libcamera.org/comment/21641/","msgid":"","date":"2021-12-07T13:58:17","subject":"Re: [libcamera-devel] [PATCH 6/8] qcam: Use Sensor sequence numbers\n\tand detect frame drop","submitter":{"id":86,"url":"https://patchwork.libcamera.org/api/people/86/","name":"Umang Jain","email":"umang.jain@ideasonboard.com"},"content":"Hi Kieran\n\nOn 12/7/21 5:09 AM, Kieran Bingham wrote:\n> The stream buffer sequence numbers might produce sequential\n> monotonic sequence numbers from an ISP producing a frame for every\n> input.\n>\n> This however, doesn't capture pipeline stalls that cause us to miss or\n> drop frames from the sensor.\n>\n> Use the SensorSequence metadata to report sequence information, and\n> report with qInfo if a frame drop is detected.\n>\n> Signed-off-by: Kieran Bingham \n\n\nReviewed-by: Umang Jain \n\n> ---\n> src/qcam/main_window.cpp | 22 ++++++++++++++++++++++\n> src/qcam/main_window.h | 1 +\n> 2 files changed, 23 insertions(+)\n>\n> diff --git a/src/qcam/main_window.cpp b/src/qcam/main_window.cpp\n> index dd0e51f55b70..8d1d5b275aed 100644\n> --- a/src/qcam/main_window.cpp\n> +++ b/src/qcam/main_window.cpp\n> @@ -12,6 +12,7 @@\n> #include \n> \n> #include \n> +#include \n> #include \n> \n> #include \n> @@ -510,6 +511,7 @@ int MainWindow::startCapture()\n> \tpreviousFrames_ = 0;\n> \tframesCaptured_ = 0;\n> \tlastBufferTime_ = 0;\n> +\tsequence_ = 0;\n> \n> \tret = camera_->start();\n> \tif (ret) {\n> @@ -719,6 +721,26 @@ void MainWindow::processCapture()\n> \t\trequest = doneQueue_.dequeue();\n> \t}\n> \n> +\t/* Parse the request metadata for useful information */\n> +\tfor (const auto &ctrl : request->metadata()) {\n> +\t\tconst int id = ctrl.first;\n> +\t\tconst ControlValue &value = ctrl.second;\n> +\n> +\t\tif (id == controls::SensorSequence) {\n> +\t\t\t/* Handle basic frame drop detection and reporting. */\n> +\t\t\tint64_t sequence = value.get();\n> +\t\t\tif (sequence_ == 0)\n> +\t\t\t\tsequence_ = sequence - 1;\n> +\t\t\tunsigned int drops = sequence - sequence_ - 1;\n> +\t\t\tif (drops)\n> +\t\t\t\tqInfo() << \"sequence: [\" << sequence << \"] *\"\n> +\t\t\t\t\t<< drops << \" frame drops detected *\";\n> +\t\t\tsequence_ = sequence;\n> +\t\t}\n> +\n> +\t\t/* \\todo Handle all/other metadata types here. */\n> +\t}\n> +\n> \t/* Process buffers. */\n> \tif (request->buffers().count(vfStream_))\n> \t\tprocessViewfinder(request->buffers().at(vfStream_));\n> diff --git a/src/qcam/main_window.h b/src/qcam/main_window.h\n> index 3fbe872c0b5b..97568d6b619d 100644\n> --- a/src/qcam/main_window.h\n> +++ b/src/qcam/main_window.h\n> @@ -120,6 +120,7 @@ private:\n> \tQMutex mutex_; /* Protects freeBuffers_, doneQueue_, and freeQueue_ */\n> \n> \tuint64_t lastBufferTime_;\n> +\tuint64_t sequence_;\n> \tQElapsedTimer frameRateInterval_;\n> \tuint32_t previousFrames_;\n> \tuint32_t framesCaptured_;","headers":{"Return-Path":"","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id 2A067BDB13\n\tfor ;\n\tTue, 7 Dec 2021 13:58:26 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id D33886086A;\n\tTue, 7 Dec 2021 14:58:25 +0100 (CET)","from perceval.ideasonboard.com (perceval.ideasonboard.com\n\t[IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id C41AF60592\n\tfor ;\n\tTue, 7 Dec 2021 14:58:24 +0100 (CET)","from [IPv6:2401:4900:1f3e:16f1:4e1b:f869:54d4:8c0c] (unknown\n\t[IPv6:2401:4900:1f3e:16f1:4e1b:f869:54d4:8c0c])\n\tby perceval.ideasonboard.com (Postfix) with ESMTPSA id 4071B556;\n\tTue, 7 Dec 2021 14:58:22 +0100 (CET)"],"Authentication-Results":"lancelot.ideasonboard.com; dkim=pass (1024-bit key;\n\tunprotected) header.d=ideasonboard.com header.i=@ideasonboard.com\n\theader.b=\"voPGRaBl\"; dkim-atps=neutral","DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com;\n\ts=mail; t=1638885504;\n\tbh=j0LiPgxXeRVJFy3DO9D5N3FMKyE+y4Cwr5W2cWvkGW4=;\n\th=Subject:To:References:From:Date:In-Reply-To:From;\n\tb=voPGRaBlZ+MXyoq18fl9hXVLP/88OlOmnB+3YAOT/ZcT7gxQfgd9s6ZpZ3zIW23ad\n\tSUCQAZb+OyXUl5loAZKyT/H+wZbbORSLj5s4+/mfCEh/vGgqIZkhCtlagaWaiBSs3s\n\ttU5wni9t5q4Aw6uZdXUBSi1bXUDx9B8lCXryajFU=","To":"Kieran Bingham ,\n\tlibcamera devel ","References":"<20211206233948.1351206-1-kieran.bingham@ideasonboard.com>\n\t<20211206233948.1351206-7-kieran.bingham@ideasonboard.com>","From":"Umang Jain ","Message-ID":"","Date":"Tue, 7 Dec 2021 19:28:17 +0530","User-Agent":"Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101\n\tThunderbird/78.10.2","MIME-Version":"1.0","In-Reply-To":"<20211206233948.1351206-7-kieran.bingham@ideasonboard.com>","Content-Type":"text/plain; charset=utf-8; format=flowed","Content-Transfer-Encoding":"7bit","Content-Language":"en-US","Subject":"Re: [libcamera-devel] [PATCH 6/8] qcam: Use Sensor sequence numbers\n\tand detect frame drop","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" "}}]