From patchwork Wed Aug 5 15:18:18 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bruce Allen X-Patchwork-Id: 27621 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id A8E19C3301 for ; Wed, 5 Aug 2026 15:18:34 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id BFB286811B; Wed, 5 Aug 2026 17:18:32 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (2048-bit key; unprotected) header.d=googlemail.com header.i=@googlemail.com header.b="XMr7yEpt"; dkim-atps=neutral Received: from mail-wm1-x32e.google.com (mail-wm1-x32e.google.com [IPv6:2a00:1450:4864:20::32e]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 298EC6805D for ; Wed, 5 Aug 2026 17:18:31 +0200 (CEST) Received: by mail-wm1-x32e.google.com with SMTP id 5b1f17b1804b1-4954afac04bso12034925e9.0 for ; Wed, 05 Aug 2026 08:18:31 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=googlemail.com; s=20251104; t=1785943111; x=1786547911; darn=lists.libcamera.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to:content-type; bh=Y1KpSTXWaighwBWCPCAYgtY24ptn8RtzAIqsmYDuivw=; b=XMr7yEptwsrisM6zV5gLy62XFZMlX4vhFwMcwfc+ZCtOe+CXq+tot8wXIbCw8IuK6z 4kFiItYEOSttL3Jou1zB8Auc29jEqoiQ6mbHi+DlWzD0PDRW6kzKJ/+fU/XI1lAcUT2Y jrlRxGwkuh1yXMlGCyZAxoTILc15FoE6fkvVW9NUnGB2FvxZr9iRr9a5IotC0Ro3dDYa rYRgrLza8vE/O1NiLKEltUvCYFVuRbST/0WNf+B21oWmoqVxeEXKstodXGJfzKkZ1L2x WwwA/XjlPS3+A8d5zwKYlGf25OEoSOQ+YJNPXuvLZC83n5TgFrW7fqporMM+BB5nCE+w ZMqA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1785943111; x=1786547911; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to:content-type; bh=Y1KpSTXWaighwBWCPCAYgtY24ptn8RtzAIqsmYDuivw=; b=rPWADPtDyVnSF8imRiWOka/4okt6+A9ZfA9DGCZ/58q7gIiclqK79VaXyNcpd1Wa6W G/kPRFmNKDtWEyr3/k+Y8JrcLwIXwaWMCIzwUxXogjxcBT6hwbftg11x69AZjOhbpQl0 4SMKt+NfN7w3jYzx9kEnxoIl86QZ6O81mWyBHXfL19ZaDR46YsBRQaLYgvGlF74JTP8k aq4w6cXHFTo/K6ecuvv43NQeKX239CFH0HKff6OGnJarWkILMiQrRiJ49fMvFgvW+Aku zmvCQd2HkRv9ne/QDOIuTg4koo3czwOJN/qdF1Tv2RoB78KInrTjGl0Xsa5nYA5Caje8 JUfA== X-Gm-Message-State: AOJu0YwPHWMvYl3fG4QB0r6IM6ratntGH3az+X8HWWNGqvHEvi5sEbxU XjbYXdFXJxxmlw3WOg0sKY3IRvPqrppgRw7QeS9k78gvdOB82uZNdWzBCD3YSQ== X-Gm-Gg: AR+sD13U2iZL5l8fdWZvF+HnCDi/ct1rTFlwfGCMn3okhTr9smC8ZL8xvEnjFXYQhMF Jz6SbJ0mgdvFMPLXEOa3if+PXZnuVxiQfHZrRwnBxUSFzJOnZovZGDGPDyjYrAoyMrneq822SkB z3tbxlI2yyxCkwUE53RaAZgt2rOcTEwCbRkuyy6xwpBC/v49/SladRW5Yl+bKgMpQ9ohhMsYM1b q3kwCMc5RuQD52zVXwhWpobhauPNzOzuun/0AiCn6Oi2VOC1x8tS38DNmVxVMnVdt8DjClH3Es6 7iL5A7cGYc+glbK5yhuUQ77117wXiCcjTQcqepveEH3+fN3Ahe0cvtUvSN0vCnwmz7JPXd2VGhS dozWRE1FkaKjbtVCDf0P3a/Rqtrpufmo3b0MF2fLApszGtRSZ5eVrWilGYxxcxIXWOclQxY7lkq HC4/SlRYWS28kkW1G6oTmiVRN6YiMweZ6NWOKmljYne2y/zvVwHbQvyBM4AcesActFeeoAaWxKq vbQpCZs/E9kKka+0w== X-Received: by 2002:a05:600c:154a:b0:493:eed4:7210 with SMTP id 5b1f17b1804b1-4994e7bb08bmr79497605e9.11.1785943110611; Wed, 05 Aug 2026 08:18:30 -0700 (PDT) Received: from localhost.localdomain ([194.94.98.191]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-4994e52d819sm47193625e9.1.2026.08.05.08.18.29 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 05 Aug 2026 08:18:30 -0700 (PDT) From: Bruce Allen To: libcamera-devel@lists.libcamera.org Cc: barnabas.pocze@ideasonboard.com Subject: [PATCH v3 1/2] libcamera: Add SensorSequence metadata control Date: Wed, 5 Aug 2026 17:18:18 +0200 Message-ID: <20260805151819.16382-2-ballen4705@googlemail.com> X-Mailer: git-send-email 2.55.0 In-Reply-To: <20260805151819.16382-1-ballen4705@googlemail.com> References: <20260703224817.15650-1-ballen4705@googlemail.com> <20260805151819.16382-1-ballen4705@googlemail.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Add controls::SensorSequence (int64_t, out) to report the monotonic sequence number from the sensor-facing capture device in request metadata, and populate it in pipeline handlers that already report SensorTimestamp. Signed-off-by: Bruce Allen --- src/libcamera/control_ids_core.yaml | 29 +++++++++++++++++++ src/libcamera/pipeline/imx8-isi/imx8-isi.cpp | 3 ++ src/libcamera/pipeline/ipu3/ipu3.cpp | 2 ++ src/libcamera/pipeline/mali-c55/mali-c55.cpp | 2 ++ src/libcamera/pipeline/rkisp1/rkisp1.cpp | 2 ++ .../pipeline/rpi/common/pipeline_base.cpp | 2 ++ src/libcamera/pipeline/rpi/pisp/pisp.cpp | 2 ++ src/libcamera/pipeline/rpi/vc4/vc4.cpp | 2 ++ src/libcamera/pipeline/simple/simple.cpp | 5 +++- src/libcamera/pipeline/uvcvideo/uvcvideo.cpp | 2 ++ src/libcamera/pipeline/vimc/vimc.cpp | 2 ++ 11 files changed, 52 insertions(+), 1 deletion(-) diff --git a/src/libcamera/control_ids_core.yaml b/src/libcamera/control_ids_core.yaml index 89991d0..bebfb49 100644 --- a/src/libcamera/control_ids_core.yaml +++ b/src/libcamera/control_ids_core.yaml @@ -792,6 +792,35 @@ controls: \todo Define how the sensor timestamp has to be used in the reprocessing use case. + - SensorSequence: + type: int64_t + direction: out + description: | + The monotonic sequence number from the sensor capture device. + + Pipeline handlers populate this metadata control with the sequence + number reported by the sensor-facing capture device (for example the + CSI-2 receiver or V4L2 video node) for the frame associated with the + request. + + While per-buffer FrameMetadata::sequence identifies frames in each + completed stream, SensorSequence reflects the sensor-side capture path. + If the value increases by more than one between consecutive completed + requests, frames were not captured before the earlier request finished. + This complements sequence numbers derived from request delivery and is + intended for diagnosing frame continuity. + + After Camera::stop() followed by Camera::start(), SensorSequence restarts + from 0 for the new streaming session. The first returned frame has + SensorSequence == 0 when the first capture buffer of the session is + completed to that request. Applications must not assume continuity of + SensorSequence values across camera session stop/start boundaries. + + On successful frames where SensorTimestamp is populated, + SensorSequence will also be populated. + + The SensorSequence control can only be returned in metadata. + - AfMode: type: int32_t direction: inout diff --git a/src/libcamera/pipeline/imx8-isi/imx8-isi.cpp b/src/libcamera/pipeline/imx8-isi/imx8-isi.cpp index c3d2695..d2175b9 100644 --- a/src/libcamera/pipeline/imx8-isi/imx8-isi.cpp +++ b/src/libcamera/pipeline/imx8-isi/imx8-isi.cpp @@ -1130,6 +1130,9 @@ void PipelineHandlerISI::bufferReady(FrameBuffer *buffer) if (!metadata.contains(controls::SensorTimestamp.id())) metadata.set(controls::SensorTimestamp, buffer->metadata().timestamp); + if (!metadata.contains(controls::SensorSequence.id())) + metadata.set(controls::SensorSequence, + static_cast(buffer->metadata().sequence)); if (completeBuffer(request, buffer)) completeRequest(request); diff --git a/src/libcamera/pipeline/ipu3/ipu3.cpp b/src/libcamera/pipeline/ipu3/ipu3.cpp index 0f3e169..6494259 100644 --- a/src/libcamera/pipeline/ipu3/ipu3.cpp +++ b/src/libcamera/pipeline/ipu3/ipu3.cpp @@ -1317,6 +1317,8 @@ void IPU3CameraData::cio2BufferReady(FrameBuffer *buffer) */ request->_d()->metadata().set(controls::SensorTimestamp, buffer->metadata().timestamp); + request->_d()->metadata().set(controls::SensorSequence, + static_cast(buffer->metadata().sequence)); info->effectiveSensorControls = delayedCtrls_->get(buffer->metadata().sequence); diff --git a/src/libcamera/pipeline/mali-c55/mali-c55.cpp b/src/libcamera/pipeline/mali-c55/mali-c55.cpp index 599ff88..119b249 100644 --- a/src/libcamera/pipeline/mali-c55/mali-c55.cpp +++ b/src/libcamera/pipeline/mali-c55/mali-c55.cpp @@ -1704,6 +1704,8 @@ void PipelineHandlerMaliC55::cruBufferReady(FrameBuffer *buffer) Request *request = info->request; request->_d()->metadata().set(controls::SensorTimestamp, buffer->metadata().timestamp); + request->_d()->metadata().set(controls::SensorSequence, + static_cast(buffer->metadata().sequence)); MaliC55CameraData *data = cameraData(request->_d()->camera()); data->ipa_->fillParams(request->sequence(), info->paramBuffer->cookie()); diff --git a/src/libcamera/pipeline/rkisp1/rkisp1.cpp b/src/libcamera/pipeline/rkisp1/rkisp1.cpp index 4f9c0aa..0c73f5a 100644 --- a/src/libcamera/pipeline/rkisp1/rkisp1.cpp +++ b/src/libcamera/pipeline/rkisp1/rkisp1.cpp @@ -1647,6 +1647,8 @@ void PipelineHandlerRkISP1::imageBufferReady(FrameBuffer *buffer) */ request->_d()->metadata().set(controls::SensorTimestamp, metadata.timestamp); + request->_d()->metadata().set(controls::SensorSequence, + static_cast(metadata.sequence)); if (isRaw_) { const ControlList &ctrls = diff --git a/src/libcamera/pipeline/rpi/common/pipeline_base.cpp b/src/libcamera/pipeline/rpi/common/pipeline_base.cpp index 5a5acf6..dcf8e6a 100644 --- a/src/libcamera/pipeline/rpi/common/pipeline_base.cpp +++ b/src/libcamera/pipeline/rpi/common/pipeline_base.cpp @@ -1514,6 +1514,8 @@ void CameraData::fillRequestMetadata(const ControlList &bufferControls, Request { if (auto x = bufferControls.get(controls::SensorTimestamp)) request->_d()->metadata().set(controls::SensorTimestamp, *x); + if (auto x = bufferControls.get(controls::SensorSequence)) + request->_d()->metadata().set(controls::SensorSequence, *x); if (auto x = bufferControls.get(controls::FrameWallClock)) request->_d()->metadata().set(controls::FrameWallClock, *x); diff --git a/src/libcamera/pipeline/rpi/pisp/pisp.cpp b/src/libcamera/pipeline/rpi/pisp/pisp.cpp index 406d683..5cf428c 100644 --- a/src/libcamera/pipeline/rpi/pisp/pisp.cpp +++ b/src/libcamera/pipeline/rpi/pisp/pisp.cpp @@ -1783,6 +1783,8 @@ void PiSPCameraData::cfeBufferDequeue(FrameBuffer *buffer) ctrl.set(controls::SensorTimestamp, sensorTimestamp); ctrl.set(controls::FrameWallClock, wallClockTimestamp); + ctrl.set(controls::SensorSequence, + static_cast(buffer->metadata().sequence)); job.sensorControls = std::move(ctrl); job.delayContext = delayContext; } else if (stream == &cfe_[Cfe::Config]) { diff --git a/src/libcamera/pipeline/rpi/vc4/vc4.cpp b/src/libcamera/pipeline/rpi/vc4/vc4.cpp index 3e9a490..1e7b26a 100644 --- a/src/libcamera/pipeline/rpi/vc4/vc4.cpp +++ b/src/libcamera/pipeline/rpi/vc4/vc4.cpp @@ -787,6 +787,8 @@ void Vc4CameraData::unicamBufferDequeue(FrameBuffer *buffer) ctrl.set(controls::SensorTimestamp, sensorTimestamp); ctrl.set(controls::FrameWallClock, wallClockTimestamp); + ctrl.set(controls::SensorSequence, + static_cast(buffer->metadata().sequence)); bayerQueue_.push({ buffer, std::move(ctrl), delayContext }); } else { embeddedQueue_.push(buffer); diff --git a/src/libcamera/pipeline/simple/simple.cpp b/src/libcamera/pipeline/simple/simple.cpp index e26f438..ce34bb2 100644 --- a/src/libcamera/pipeline/simple/simple.cpp +++ b/src/libcamera/pipeline/simple/simple.cpp @@ -939,9 +939,12 @@ void SimpleCameraData::imageBufferReady(FrameBuffer *buffer) } } - if (request) + if (request) { request->_d()->metadata().set(controls::SensorTimestamp, buffer->metadata().timestamp); + request->_d()->metadata().set(controls::SensorSequence, + static_cast(buffer->metadata().sequence)); + } /* * Queue the captured and the request buffer to the converter or Software diff --git a/src/libcamera/pipeline/uvcvideo/uvcvideo.cpp b/src/libcamera/pipeline/uvcvideo/uvcvideo.cpp index 3435a76..e0b7dad 100644 --- a/src/libcamera/pipeline/uvcvideo/uvcvideo.cpp +++ b/src/libcamera/pipeline/uvcvideo/uvcvideo.cpp @@ -897,6 +897,8 @@ void UVCCameraData::imageBufferReady(FrameBuffer *buffer) /* \todo Use the UVC metadata to calculate a more precise timestamp */ request->_d()->metadata().set(controls::SensorTimestamp, buffer->metadata().timestamp); + request->_d()->metadata().set(controls::SensorSequence, + static_cast(buffer->metadata().sequence)); pipe()->completeBuffer(request, buffer); pipe()->completeRequest(request); diff --git a/src/libcamera/pipeline/vimc/vimc.cpp b/src/libcamera/pipeline/vimc/vimc.cpp index 01d8fb2..ff809a1 100644 --- a/src/libcamera/pipeline/vimc/vimc.cpp +++ b/src/libcamera/pipeline/vimc/vimc.cpp @@ -617,6 +617,8 @@ void VimcCameraData::imageBufferReady(FrameBuffer *buffer) /* Record the sensor's timestamp in the request metadata. */ request->_d()->metadata().set(controls::SensorTimestamp, buffer->metadata().timestamp); + request->_d()->metadata().set(controls::SensorSequence, + static_cast(buffer->metadata().sequence)); pipe->completeBuffer(request, buffer); pipe->completeRequest(request);