{"id":15058,"url":"https://patchwork.libcamera.org/api/patches/15058/?format=json","web_url":"https://patchwork.libcamera.org/patch/15058/","project":{"id":1,"url":"https://patchwork.libcamera.org/api/projects/1/?format=json","name":"libcamera","link_name":"libcamera","list_id":"libcamera_core","list_email":"libcamera-devel@lists.libcamera.org","web_url":"","scm_url":"","webscm_url":""},"msgid":"<20211206233948.1351206-3-kieran.bingham@ideasonboard.com>","date":"2021-12-06T23:39:42","name":"[libcamera-devel,2/8] libcamera: pipeline: Set the Sensor sequence number for all pipelines","commit_ref":null,"pull_url":null,"state":"new","archived":false,"hash":"75e73522770905111929d10405f843435172adc1","submitter":{"id":4,"url":"https://patchwork.libcamera.org/api/people/4/?format=json","name":"Kieran Bingham","email":"kieran.bingham@ideasonboard.com"},"delegate":{"id":11,"url":"https://patchwork.libcamera.org/api/users/11/?format=json","username":"kbingham","first_name":"Kieran","last_name":"Bingham","email":"kieran.bingham@ideasonboard.com"},"mbox":"https://patchwork.libcamera.org/patch/15058/mbox/","series":[{"id":2822,"url":"https://patchwork.libcamera.org/api/series/2822/?format=json","web_url":"https://patchwork.libcamera.org/project/libcamera/list/?series=2822","date":"2021-12-06T23:39:40","name":"Request metadata: SensorSequence","version":1,"mbox":"https://patchwork.libcamera.org/series/2822/mbox/"}],"comments":"https://patchwork.libcamera.org/api/patches/15058/comments/","check":"pending","checks":"https://patchwork.libcamera.org/api/patches/15058/checks/","tags":{},"headers":{"Return-Path":"","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id 47AFAC324B\n\tfor ;\n\tMon, 6 Dec 2021 23:40:00 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id 950F86088B;\n\tTue, 7 Dec 2021 00:39:58 +0100 (CET)","from perceval.ideasonboard.com (perceval.ideasonboard.com\n\t[213.167.242.64])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id CA5456086C\n\tfor ;\n\tTue, 7 Dec 2021 00:39:54 +0100 (CET)","from Monstersaurus.local\n\t(cpc89244-aztw30-2-0-cust3082.18-1.cable.virginm.net [86.31.172.11])\n\tby perceval.ideasonboard.com (Postfix) with ESMTPSA id 694E1A1A;\n\tTue, 7 Dec 2021 00:39:54 +0100 (CET)"],"Authentication-Results":"lancelot.ideasonboard.com;\n\tdkim=fail reason=\"signature verification failed\" (1024-bit key;\n\tunprotected) header.d=ideasonboard.com header.i=@ideasonboard.com\n\theader.b=\"CiuwlI68\"; dkim-atps=neutral","DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com;\n\ts=mail; t=1638833994;\n\tbh=9sS+DqXITQIA53Wy/oaQIbdNzQPKykliP5txDOgCS84=;\n\th=From:To:Cc:Subject:Date:In-Reply-To:References:From;\n\tb=CiuwlI683/yX8a2P93VWVvf55Ph8piX2vKO5+HyvlgjHfLTCW4KNBTOsJA7+wsFIG\n\tHtxiQveoV1oKBRduTJaNPChcRWDW1kZFt9Q4HZic7xtlJDAjOp4Sh/D3xWatu/kVJy\n\tXH1oYY+pSFXYXqyACwXTU0Dh1G9QLn5M4Wn2essI=","From":"Kieran Bingham ","To":"libcamera devel ","Date":"Mon, 6 Dec 2021 23:39:42 +0000","Message-Id":"<20211206233948.1351206-3-kieran.bingham@ideasonboard.com>","X-Mailer":"git-send-email 2.30.2","In-Reply-To":"<20211206233948.1351206-1-kieran.bingham@ideasonboard.com>","References":"<20211206233948.1351206-1-kieran.bingham@ideasonboard.com>","MIME-Version":"1.0","Content-Transfer-Encoding":"8bit","Subject":"[libcamera-devel] [PATCH 2/8] libcamera: pipeline: Set the Sensor\n\tsequence number for all pipelines","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" "},"content":"The SensorSequence metadata provides the sequence number associated with\nthe image capture directly.\n\nWhile we already report sequence numbers in the metadata of the buffers,\nthis can lead to mis-leading parsing of those sequence numbers which may\nbe purely monotonic sequences from an ISP, and not represent the true\nsequence counts from a capture device.\n\nUse the newly added SensorSequence metadata control and populate\ncompleted requests with the sequence number from the appropriate stream.\nFor ISP based devices, such as the IPU3, RPi, and RkISP1, this comes\nfrom the CSI2 receiver capture device, while for the Simple pipeline and\nthe UVC Video pipeline, this comes from the single video capture device.\n\nSigned-off-by: Kieran Bingham \n---\n src/libcamera/pipeline/ipu3/ipu3.cpp | 4 +++-\n src/libcamera/pipeline/raspberrypi/raspberrypi.cpp | 3 +++\n src/libcamera/pipeline/rkisp1/rkisp1.cpp | 4 +++-\n src/libcamera/pipeline/simple/simple.cpp | 12 ++++++++----\n src/libcamera/pipeline/uvcvideo/uvcvideo.cpp | 2 ++\n 5 files changed, 19 insertions(+), 6 deletions(-)","diff":"diff --git a/src/libcamera/pipeline/ipu3/ipu3.cpp b/src/libcamera/pipeline/ipu3/ipu3.cpp\nindex 313220624aff..a7f35840afbb 100644\n--- a/src/libcamera/pipeline/ipu3/ipu3.cpp\n+++ b/src/libcamera/pipeline/ipu3/ipu3.cpp\n@@ -1372,13 +1372,15 @@ void IPU3CameraData::cio2BufferReady(FrameBuffer *buffer)\n \t}\n \n \t/*\n-\t * Record the sensor's timestamp in the request metadata.\n+\t * Record the sensor's timestamp and sequence in the request metadata.\n \t *\n \t * \\todo The sensor timestamp should be better estimated by connecting\n \t * to the V4L2Device::frameStart signal.\n \t */\n \trequest->metadata().set(controls::SensorTimestamp,\n \t\t\t\tbuffer->metadata().timestamp);\n+\trequest->metadata().set(controls::SensorSequence,\n+\t\t\t\tbuffer->metadata().sequence);\n \n \tinfo->effectiveSensorControls = delayedCtrls_->get(buffer->metadata().sequence);\n \ndiff --git a/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp b/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp\nindex 321b72adbbf7..f458587a8920 100644\n--- a/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp\n+++ b/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp\n@@ -1549,6 +1549,7 @@ void RPiCameraData::unicamBufferDequeue(FrameBuffer *buffer)\n \t\t * as it does not receive the FrameBuffer object.\n \t\t */\n \t\tctrl.set(controls::SensorTimestamp, buffer->metadata().timestamp);\n+\t\tctrl.set(controls::SensorSequence, buffer->metadata().sequence);\n \t\tbayerQueue_.push({ buffer, std::move(ctrl) });\n \t} else {\n \t\tembeddedQueue_.push(buffer);\n@@ -1786,6 +1787,8 @@ void RPiCameraData::fillRequestMetadata(const ControlList &bufferControls,\n {\n \trequest->metadata().set(controls::SensorTimestamp,\n \t\t\t\tbufferControls.get(controls::SensorTimestamp));\n+\trequest->metadata().set(controls::SensorSequence,\n+\t\t\t\tbufferControls.get(controls::SensorSequence));\n \n \trequest->metadata().set(controls::ScalerCrop, scalerCrop_);\n }\ndiff --git a/src/libcamera/pipeline/rkisp1/rkisp1.cpp b/src/libcamera/pipeline/rkisp1/rkisp1.cpp\nindex 36ef6a02ae90..27592e473c14 100644\n--- a/src/libcamera/pipeline/rkisp1/rkisp1.cpp\n+++ b/src/libcamera/pipeline/rkisp1/rkisp1.cpp\n@@ -1077,13 +1077,15 @@ void PipelineHandlerRkISP1::bufferReady(FrameBuffer *buffer)\n \tRequest *request = buffer->request();\n \n \t/*\n-\t * Record the sensor's timestamp in the request metadata.\n+\t * Record the sensor's timestamp and sequence in the request metadata.\n \t *\n \t * \\todo The sensor timestamp should be better estimated by connecting\n \t * to the V4L2Device::frameStart signal.\n \t */\n \trequest->metadata().set(controls::SensorTimestamp,\n \t\t\t\tbuffer->metadata().timestamp);\n+\trequest->metadata().set(controls::SensorSequence,\n+\t\t\t\tbuffer->metadata().sequence);\n \n \tcompleteBuffer(request, buffer);\n \ttryCompleteRequest(request);\ndiff --git a/src/libcamera/pipeline/simple/simple.cpp b/src/libcamera/pipeline/simple/simple.cpp\nindex 701fb4be0b71..4c5d9c0b9919 100644\n--- a/src/libcamera/pipeline/simple/simple.cpp\n+++ b/src/libcamera/pipeline/simple/simple.cpp\n@@ -667,9 +667,10 @@ void SimpleCameraData::bufferReady(FrameBuffer *buffer)\n \t}\n \n \t/*\n-\t * Record the sensor's timestamp in the request metadata. The request\n-\t * needs to be obtained from the user-facing buffer, as internal\n-\t * buffers are free-wheeling and have no request associated with them.\n+\t * Record the sensor's timestamp and sequence in the request metadata.\n+\t * The request needs to be obtained from the user-facing buffer, as\n+\t * internal buffers are free-wheeling and have no request associated\n+\t * with them.\n \t *\n \t * \\todo The sensor timestamp should be better estimated by connecting\n \t * to the V4L2Device::frameStart signal if the platform provides it.\n@@ -686,9 +687,12 @@ void SimpleCameraData::bufferReady(FrameBuffer *buffer)\n \t\t}\n \t}\n \n-\tif (request)\n+\tif (request) {\n \t\trequest->metadata().set(controls::SensorTimestamp,\n \t\t\t\t\tbuffer->metadata().timestamp);\n+\t\trequest->metadata().set(controls::SensorSequence,\n+\t\t\t\t\tbuffer->metadata().sequence);\n+\t}\n \n \t/*\n \t * Queue the captured and the request buffer to the converter if format\ndiff --git a/src/libcamera/pipeline/uvcvideo/uvcvideo.cpp b/src/libcamera/pipeline/uvcvideo/uvcvideo.cpp\nindex 264f5370cf32..f5fab3b69e6d 100644\n--- a/src/libcamera/pipeline/uvcvideo/uvcvideo.cpp\n+++ b/src/libcamera/pipeline/uvcvideo/uvcvideo.cpp\n@@ -666,6 +666,8 @@ void UVCCameraData::bufferReady(FrameBuffer *buffer)\n \t/* \\todo Use the UVC metadata to calculate a more precise timestamp */\n \trequest->metadata().set(controls::SensorTimestamp,\n \t\t\t\tbuffer->metadata().timestamp);\n+\trequest->metadata().set(controls::SensorSequence,\n+\t\t\t\tbuffer->metadata().sequence);\n \n \tpipe()->completeBuffer(request, buffer);\n \tpipe()->completeRequest(request);\n","prefixes":["libcamera-devel","2/8"]}