From patchwork Mon Dec 6 23:39:42 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kieran Bingham X-Patchwork-Id: 15058 X-Patchwork-Delegate: kieran.bingham@ideasonboard.com Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 47AFAC324B for ; Mon, 6 Dec 2021 23:40:00 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 950F86088B; Tue, 7 Dec 2021 00:39:58 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="CiuwlI68"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id CA5456086C for ; Tue, 7 Dec 2021 00:39:54 +0100 (CET) Received: from Monstersaurus.local (cpc89244-aztw30-2-0-cust3082.18-1.cable.virginm.net [86.31.172.11]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 694E1A1A; Tue, 7 Dec 2021 00:39:54 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1638833994; bh=9sS+DqXITQIA53Wy/oaQIbdNzQPKykliP5txDOgCS84=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=CiuwlI683/yX8a2P93VWVvf55Ph8piX2vKO5+HyvlgjHfLTCW4KNBTOsJA7+wsFIG HtxiQveoV1oKBRduTJaNPChcRWDW1kZFt9Q4HZic7xtlJDAjOp4Sh/D3xWatu/kVJy XH1oYY+pSFXYXqyACwXTU0Dh1G9QLn5M4Wn2essI= From: Kieran Bingham To: libcamera devel Date: Mon, 6 Dec 2021 23:39:42 +0000 Message-Id: <20211206233948.1351206-3-kieran.bingham@ideasonboard.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20211206233948.1351206-1-kieran.bingham@ideasonboard.com> References: <20211206233948.1351206-1-kieran.bingham@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH 2/8] libcamera: pipeline: Set the Sensor sequence number for all pipelines X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" The SensorSequence metadata provides the sequence number associated with the image capture directly. While we already report sequence numbers in the metadata of the buffers, this can lead to mis-leading parsing of those sequence numbers which may be purely monotonic sequences from an ISP, and not represent the true sequence counts from a capture device. Use the newly added SensorSequence metadata control and populate completed requests with the sequence number from the appropriate stream. For ISP based devices, such as the IPU3, RPi, and RkISP1, this comes from the CSI2 receiver capture device, while for the Simple pipeline and the UVC Video pipeline, this comes from the single video capture device. Signed-off-by: Kieran Bingham Reviewed-by: Umang Jain Reviewed-by: Naushir Patuck --- src/libcamera/pipeline/ipu3/ipu3.cpp | 4 +++- src/libcamera/pipeline/raspberrypi/raspberrypi.cpp | 3 +++ src/libcamera/pipeline/rkisp1/rkisp1.cpp | 4 +++- src/libcamera/pipeline/simple/simple.cpp | 12 ++++++++---- src/libcamera/pipeline/uvcvideo/uvcvideo.cpp | 2 ++ 5 files changed, 19 insertions(+), 6 deletions(-) diff --git a/src/libcamera/pipeline/ipu3/ipu3.cpp b/src/libcamera/pipeline/ipu3/ipu3.cpp index 313220624aff..a7f35840afbb 100644 --- a/src/libcamera/pipeline/ipu3/ipu3.cpp +++ b/src/libcamera/pipeline/ipu3/ipu3.cpp @@ -1372,13 +1372,15 @@ void IPU3CameraData::cio2BufferReady(FrameBuffer *buffer) } /* - * Record the sensor's timestamp in the request metadata. + * Record the sensor's timestamp and sequence in the request metadata. * * \todo The sensor timestamp should be better estimated by connecting * to the V4L2Device::frameStart signal. */ request->metadata().set(controls::SensorTimestamp, buffer->metadata().timestamp); + request->metadata().set(controls::SensorSequence, + buffer->metadata().sequence); info->effectiveSensorControls = delayedCtrls_->get(buffer->metadata().sequence); diff --git a/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp b/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp index 321b72adbbf7..f458587a8920 100644 --- a/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp +++ b/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp @@ -1549,6 +1549,7 @@ void RPiCameraData::unicamBufferDequeue(FrameBuffer *buffer) * as it does not receive the FrameBuffer object. */ ctrl.set(controls::SensorTimestamp, buffer->metadata().timestamp); + ctrl.set(controls::SensorSequence, buffer->metadata().sequence); bayerQueue_.push({ buffer, std::move(ctrl) }); } else { embeddedQueue_.push(buffer); @@ -1786,6 +1787,8 @@ void RPiCameraData::fillRequestMetadata(const ControlList &bufferControls, { request->metadata().set(controls::SensorTimestamp, bufferControls.get(controls::SensorTimestamp)); + request->metadata().set(controls::SensorSequence, + bufferControls.get(controls::SensorSequence)); request->metadata().set(controls::ScalerCrop, scalerCrop_); } diff --git a/src/libcamera/pipeline/rkisp1/rkisp1.cpp b/src/libcamera/pipeline/rkisp1/rkisp1.cpp index 36ef6a02ae90..27592e473c14 100644 --- a/src/libcamera/pipeline/rkisp1/rkisp1.cpp +++ b/src/libcamera/pipeline/rkisp1/rkisp1.cpp @@ -1077,13 +1077,15 @@ void PipelineHandlerRkISP1::bufferReady(FrameBuffer *buffer) Request *request = buffer->request(); /* - * Record the sensor's timestamp in the request metadata. + * Record the sensor's timestamp and sequence in the request metadata. * * \todo The sensor timestamp should be better estimated by connecting * to the V4L2Device::frameStart signal. */ request->metadata().set(controls::SensorTimestamp, buffer->metadata().timestamp); + request->metadata().set(controls::SensorSequence, + buffer->metadata().sequence); completeBuffer(request, buffer); tryCompleteRequest(request); diff --git a/src/libcamera/pipeline/simple/simple.cpp b/src/libcamera/pipeline/simple/simple.cpp index 701fb4be0b71..4c5d9c0b9919 100644 --- a/src/libcamera/pipeline/simple/simple.cpp +++ b/src/libcamera/pipeline/simple/simple.cpp @@ -667,9 +667,10 @@ void SimpleCameraData::bufferReady(FrameBuffer *buffer) } /* - * Record the sensor's timestamp in the request metadata. The request - * needs to be obtained from the user-facing buffer, as internal - * buffers are free-wheeling and have no request associated with them. + * Record the sensor's timestamp and sequence in the request metadata. + * The request needs to be obtained from the user-facing buffer, as + * internal buffers are free-wheeling and have no request associated + * with them. * * \todo The sensor timestamp should be better estimated by connecting * to the V4L2Device::frameStart signal if the platform provides it. @@ -686,9 +687,12 @@ void SimpleCameraData::bufferReady(FrameBuffer *buffer) } } - if (request) + if (request) { request->metadata().set(controls::SensorTimestamp, buffer->metadata().timestamp); + request->metadata().set(controls::SensorSequence, + buffer->metadata().sequence); + } /* * Queue the captured and the request buffer to the converter if format diff --git a/src/libcamera/pipeline/uvcvideo/uvcvideo.cpp b/src/libcamera/pipeline/uvcvideo/uvcvideo.cpp index 264f5370cf32..f5fab3b69e6d 100644 --- a/src/libcamera/pipeline/uvcvideo/uvcvideo.cpp +++ b/src/libcamera/pipeline/uvcvideo/uvcvideo.cpp @@ -666,6 +666,8 @@ void UVCCameraData::bufferReady(FrameBuffer *buffer) /* \todo Use the UVC metadata to calculate a more precise timestamp */ request->metadata().set(controls::SensorTimestamp, buffer->metadata().timestamp); + request->metadata().set(controls::SensorSequence, + buffer->metadata().sequence); pipe()->completeBuffer(request, buffer); pipe()->completeRequest(request);