From patchwork Fri Jun 26 00:37:11 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bruce Allen X-Patchwork-Id: 27038 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id AB9EAC3264 for ; Fri, 26 Jun 2026 00:37:19 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 73062658D1; Fri, 26 Jun 2026 02:37:17 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (2048-bit key; unprotected) header.d=googlemail.com header.i=@googlemail.com header.b="k+3J6B7L"; dkim-atps=neutral Received: from mail-wr1-x434.google.com (mail-wr1-x434.google.com [IPv6:2a00:1450:4864:20::434]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 2B704658C5 for ; Fri, 26 Jun 2026 02:37:16 +0200 (CEST) Received: by mail-wr1-x434.google.com with SMTP id ffacd0b85a97d-46cacc39191so101764f8f.0 for ; Thu, 25 Jun 2026 17:37:16 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=googlemail.com; s=20251104; t=1782434236; x=1783039036; darn=lists.libcamera.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:from:to:cc:subject:date:message-id :reply-to; bh=rWAwHULZ1wiKIxC9iBzivs8WOUmYC1Q6/xiTdP58SIg=; b=k+3J6B7LEe1vizSSa4FQJ60pH47v3FTBKbN5HDQHaR84eGW3Fa3r1rqQvdNZx2hgTT 6s3lFxz3S1pOBmX2MpOw6JkfR6joRWyB/sJ2q4uO5glhWA+Rv7PSpE6VqwZND9sG8+91 S6fYQVaOT5CwH0N0QHFZVADpo8h4Amgf29J27twPkDMmKDxdAFxr99tok9Qc7E9R5YF1 xkpgFlrGZq2WAzOEKEZ8GasyH9cfnsSo5V7FiPp+g5oy6F+lRlBExLzipJ8snvNvX29b n5lrfCoOGjN/JUSq8dhxFmQ1jxNs+fo4aPbl6pQObolURKqmPIJuM71q+pO5YNxmFrWf pbAA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1782434236; x=1783039036; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:x-gm-gg:x-gm-message-state:from:to :cc:subject:date:message-id:reply-to; bh=rWAwHULZ1wiKIxC9iBzivs8WOUmYC1Q6/xiTdP58SIg=; b=A2B5XoP2EezF9Lvq2xIz0ETUflZ91SHBMbJ/Kqgg+TdYbsnBkt4ftqiPsxlGGtfl49 ZTxsibrI+Vc7wI8L+qxvbZTXb9qwzlJlJcJeAyJ7Qum1E6h23J373OBL77C4dPGQgDv5 prats9UrPq65xK5Uqs2IGmkMuR/yzVOC6axsA3m4OfixzgNwM2fbBTZuN3zVQZMOI4eR mMyhdDjS7KXYl1dU5t32VlT79BiSu83hyPXmoYCiZAS8TpXqYVcIfrMkn5WKuVF7bwUn FPWZWKes2D5sGjoHzHl14D4D9ftOqSsxF1SAhau5HG/EzGqP0STW6a6o450f9vSmvXD4 y2Og== X-Gm-Message-State: AOJu0Yzy3Kx8pYFDZ1ADN/d/i06zK7OoTT2R+cR+LBeJfFe1v3VdU9rk oUCvXa8QadFjWY2TqfcNjRjVjSf/jFHfCBCw5fjRz6H+QVhazbkil+kIQOIcng== X-Gm-Gg: AfdE7ckenEj57VCXB+elzA3eSW8bznN5m+k0OcfP3Y42TKDN2KgNZ54q5xPOyEmUtQ3 8Cjy89srrveQPEiBy2ZdGjSMw4cJMbz2s2/kiL+tcQ7dq0GMlS1cswZ4cQvQc8zpmhR9hTGuPjS jGJKD4p7t2TUh7tQtZszRNTMu67B21MUA6H/UaU4iENOXpFR3tv3JlenoR6z0Q0guY6dw3EDAPt V/iSWoQzWHGiANugV8LUyVbqs6fAIK68UqNO7ctIis/8U6MX6qWwyK52OlItrXVEK6B5v7Naksq +zsaEXHlxNEVUVOzqc75i7uqkTae016/7G784ZG7mgj85VRC1BroniDMtaLBrNg+BN8BwbaRvi0 CDKcJxBG0RDS8AI/v4dRtdtKXRde1+/Dx3YNQl4vR8sHhruCyzDKrHX/KZF4A6ChyYqiRhbeAuP t/vVzPDbUZvJWpxNjLp6Nm+GXB9Iw7V9XcRHAdMhFTmdVk4ujIGhClG18N5dmi+Stpv6x5kNPcB nqiTy3DseSHNB25 X-Received: by 2002:a05:6000:2904:b0:462:caf6:8d06 with SMTP id ffacd0b85a97d-46a7e9f3ea0mr25572542f8f.3.1782434235674; Thu, 25 Jun 2026 17:37:15 -0700 (PDT) Received: from localhost.localdomain (p5ddcc513.dip0.t-ipconnect.de. [93.220.197.19]) by smtp.gmail.com with ESMTPSA id ffacd0b85a97d-46f0db007b3sm762055f8f.2.2026.06.25.17.37.14 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 25 Jun 2026 17:37:15 -0700 (PDT) From: Bruce Allen To: libcamera-devel@lists.libcamera.org Subject: [PATCH v1 1/2] rpi: Add SensorSequence vendor control Date: Fri, 26 Jun 2026 02:37:11 +0200 Message-ID: <20260626003712.26624-2-ballen4705@googlemail.com> X-Mailer: git-send-email 2.54.0 In-Reply-To: <20260626003712.26624-1-ballen4705@googlemail.com> References: <20260626003712.26624-1-ballen4705@googlemail.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Expose the Unicam frontend FrameBuffer sequence as controls::rpi::SensorSequence (int64_t, out) in request metadata on the VC4 pipeline. Signed-off-by: Bruce Allen --- src/libcamera/control_ids_rpi.yaml | 11 +++++++++++ src/libcamera/pipeline/rpi/common/pipeline_base.cpp | 2 ++ src/libcamera/pipeline/rpi/vc4/vc4.cpp | 8 +++++--- 3 files changed, 18 insertions(+), 3 deletions(-) diff --git a/src/libcamera/control_ids_rpi.yaml b/src/libcamera/control_ids_rpi.yaml index 0b7da5b..5348060 100644 --- a/src/libcamera/control_ids_rpi.yaml +++ b/src/libcamera/control_ids_rpi.yaml @@ -192,4 +192,15 @@ controls: just been applied. Controls normally take several frames to apply, so the number here will refer to a request submitted a number of frames earlier. + + - SensorSequence: + type: int64_t + direction: out + description: | + Monotonic frontend/Unicam sequence number for the sensor frame. + + Any break in this sequence number indicates that frames were not + captured into the frontend stream before this request completed. This + value is Raspberry Pi pipeline metadata and is intended for diagnostics + of sensor-side frame continuity. ... diff --git a/src/libcamera/pipeline/rpi/common/pipeline_base.cpp b/src/libcamera/pipeline/rpi/common/pipeline_base.cpp index 5a5acf6..22327bd 100644 --- a/src/libcamera/pipeline/rpi/common/pipeline_base.cpp +++ b/src/libcamera/pipeline/rpi/common/pipeline_base.cpp @@ -1516,6 +1516,8 @@ void CameraData::fillRequestMetadata(const ControlList &bufferControls, Request request->_d()->metadata().set(controls::SensorTimestamp, *x); if (auto x = bufferControls.get(controls::FrameWallClock)) request->_d()->metadata().set(controls::FrameWallClock, *x); + if (auto x = bufferControls.get(controls::rpi::SensorSequence)) + request->_d()->metadata().set(controls::rpi::SensorSequence, *x); if (cropParams_.size()) { std::vector crops; diff --git a/src/libcamera/pipeline/rpi/vc4/vc4.cpp b/src/libcamera/pipeline/rpi/vc4/vc4.cpp index 3e9a490..0f437b2 100644 --- a/src/libcamera/pipeline/rpi/vc4/vc4.cpp +++ b/src/libcamera/pipeline/rpi/vc4/vc4.cpp @@ -777,16 +777,18 @@ void Vc4CameraData::unicamBufferDequeue(FrameBuffer *buffer) */ auto [ctrl, delayContext] = delayedCtrls_->get(buffer->metadata().sequence); /* - * Add the frame timestamp to the ControlList for the IPA to use - * as it does not receive the FrameBuffer object. Also derive a - * corresponding wallclock value. + * Add the frame timestamp and sequence number to the ControlList + * for the IPA to use as it does not receive the FrameBuffer object. + * Also derive a corresponding wallclock value. */ wallClockRecovery_.addSample(); uint64_t sensorTimestamp = buffer->metadata().timestamp; uint64_t wallClockTimestamp = wallClockRecovery_.getOutput(sensorTimestamp); + unsigned int sensorSequence = buffer->metadata().sequence; ctrl.set(controls::SensorTimestamp, sensorTimestamp); ctrl.set(controls::FrameWallClock, wallClockTimestamp); + ctrl.set(controls::rpi::SensorSequence, sensorSequence); bayerQueue_.push({ buffer, std::move(ctrl), delayContext }); } else { embeddedQueue_.push(buffer); From patchwork Fri Jun 26 00:37:12 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bruce Allen X-Patchwork-Id: 27039 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 66E69C3301 for ; Fri, 26 Jun 2026 00:37:21 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id F2812658D9; Fri, 26 Jun 2026 02:37:20 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (2048-bit key; unprotected) header.d=googlemail.com header.i=@googlemail.com header.b="X1I7OVhf"; dkim-atps=neutral Received: from mail-wm1-x330.google.com (mail-wm1-x330.google.com [IPv6:2a00:1450:4864:20::330]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id D4E50658D4 for ; Fri, 26 Jun 2026 02:37:17 +0200 (CEST) Received: by mail-wm1-x330.google.com with SMTP id 5b1f17b1804b1-491609cdd8fso2620675e9.2 for ; Thu, 25 Jun 2026 17:37:17 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=googlemail.com; s=20251104; t=1782434237; x=1783039037; darn=lists.libcamera.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:from:to:cc:subject:date:message-id :reply-to; bh=VpZcHw2NBHq0BkAD+UBdRuJ5E7VdAGmrSf5WbR61kBg=; b=X1I7OVhfelFlTrydGrvkimqG8rVGyULtCLN7fmmxKwDL4CX+L3DFdyeYnjYvS1t/tm E2LslzR7P4WMOS/nMjmxaglEP8ycCxr4XyNEnlGlfHJ+EeQvLz5btS+gC6A8TTIg8/SV WBCaG+So2hZA+jQMsbW/xFVz0NHOtYDizEELwNXthXlg5udVp7k/9IAeAKzJov2KwNK3 ntU+xj79Y6UwHiraXpcWnTohOPBx9Ut6taHO1F+PpIcCB1U4tQANAZh7bu68Jb+16xTT Bbeh25gx1Z3VMwHFatmbO+EkYG02SEQnKg5xWSsZRXezkXpJpM35OHwoi9hUehaJXnlt c/mg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1782434237; x=1783039037; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:x-gm-gg:x-gm-message-state:from:to :cc:subject:date:message-id:reply-to; bh=VpZcHw2NBHq0BkAD+UBdRuJ5E7VdAGmrSf5WbR61kBg=; b=kk7GlV7g7glNbAQgi1JT1uOLj2WfT+5mIRXJSdXz4872DK09L0cuPkNsiLvevXl+UF C34QFiQpwuV6as37ACgCIFJp1tws1A8M0mzB4bwwxm2wOqQbzOqrCXy2Ao/27Apo/2Fx OUUk7uhtsn0csIoaClF5gjgjY1omDZcPCFOPpzsQXstJgdE35sd2adEZxCCxsIhOldR9 OlvX13vL4L8cudLugG5mlTqAE6xJmovI4sutUC0JgWaXZKw12hmK27XK9UKV4/bMOCmk sriWQ4MjUuBpQVpMVN5egSnzrLacFKGa7KNw8kkQsSHMlUNlEu/ls9DZNeYqfXAJTu+P FkWQ== X-Gm-Message-State: AOJu0YyDf/OHOOGvHLNKHAqFFZt3/BocprJjA6B7Rmk67oh/3oTQL8TV CMFqyIl+G6uD6YmYp+4Q3SBnyG/QequK4OyJUxIAC7EENNLIbRXZtMqrTxhXLw== X-Gm-Gg: AfdE7clwUcnIEGMhVaGVlnrkukl70wUUtkziNtBUy/ENUg57IUkwPavOHgZ+AFao51+ Z+t4bK4THaL/wU82WlSyyfIHmqHF54Z8vKhGPy588bacUzD7qne0VmFhNCrAuCnyRLPfx2d7YdK 5/SxAfQWX7V+GI8YcbMxjGYASy7ACLwh5Rg9h/nqKnQ5Ha8pAqvnu5AqnS9T4+vpMTxUe8GwLNV vrlcSyhX7KJs5RXBMW3tfaYlkpGJugRVYZMs1ZfmbMFjo3Mik8O7FK+NgJ15x8G1XYTMXCG0299 j3BGMR2UfWnYrWLwN9NdydlGjYviAFhGmAxFSL7fRmVStIaOrfsUzTbmP29upy5N8f9b2/6E1zu hnbTWt0l7GuhUOq//e4zeBBGePP27+nT236IoyQgdL6XR/FMMn9ynzgdviJCmx964WNF4vJoETM LEtlOe1fiavmYSc+WuiAShGhSBqlaTrFRJUhagD8ajQlezZlg6S0xK7M6i65vEAR0ssQU8rjgg4 wBJ6A== X-Received: by 2002:a05:600d:844f:10b0:490:ae52:499c with SMTP id 5b1f17b1804b1-49266899ff0mr53044655e9.21.1782434237283; Thu, 25 Jun 2026 17:37:17 -0700 (PDT) Received: from localhost.localdomain (p5ddcc513.dip0.t-ipconnect.de. [93.220.197.19]) by smtp.gmail.com with ESMTPSA id ffacd0b85a97d-46f0db007b3sm762055f8f.2.2026.06.25.17.37.15 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 25 Jun 2026 17:37:16 -0700 (PDT) From: Bruce Allen To: libcamera-devel@lists.libcamera.org Subject: [PATCH v1 2/2] docs: Document SensorSequence request metadata Date: Fri, 26 Jun 2026 02:37:12 +0200 Message-ID: <20260626003712.26624-3-ballen4705@googlemail.com> X-Mailer: git-send-email 2.54.0 In-Reply-To: <20260626003712.26624-1-ballen4705@googlemail.com> References: <20260626003712.26624-1-ballen4705@googlemail.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Describe how applications read SensorSequence from Request::metadata() and how it differs from per-buffer FrameMetadata::sequence. Signed-off-by: Bruce Allen --- .../guides/application-developer.rst | 28 +++++++++++++++++++ 1 file changed, 28 insertions(+) diff --git a/Documentation/guides/application-developer.rst b/Documentation/guides/application-developer.rst index a9620ca..996aa9b 100644 --- a/Documentation/guides/application-developer.rst +++ b/Documentation/guides/application-developer.rst @@ -469,6 +469,34 @@ and the bytes used by planes. seq: 000040 bytesused: 1843200 seq: 000042 bytesused: 1843200 +Request metadata +~~~~~~~~~~~~~~~~ + +Completed requests also expose metadata as a :doxy-pub:`ControlList` through +:doxy-pub:`Request::metadata()`. Pipeline handlers populate this list with +controls defined as ``direction: out`` in the control definition YAML files. + +Common entries include :doxy-pub:`controls::SensorTimestamp` and +:doxy-pub:`controls::FrameWallClock`. On Raspberry Pi VC4 pipelines, the vendor +control ``controls::rpi::SensorSequence`` reports the Unicam frontend +``FrameBuffer::metadata().sequence`` for the sensor frame. + +While per-buffer :doxy-pub:`FrameMetadata::sequence` identifies frames in each +completed stream, ``SensorSequence`` reflects the sensor/frontend capture path. +If the value increases by more than one between consecutive completed requests, +frames were not captured into the frontend stream before the earlier request +finished. This complements sequence numbers derived from request delivery and +is intended for diagnosing sensor-side frame continuity. + +.. code:: cpp + + const ControlList &metadata = request->metadata(); + if (auto seq = metadata.get(controls::rpi::SensorSequence)) + std::cout << " sensor sequence: " << *seq << std::endl; + +See also ``controls::rpi::ControlListSequence``, which reports when a request's +control list was applied rather than frontend capture timing. + A completed buffer contains of course image data which can be accessed through the per-plane dma-buf file descriptor transported by the ``FrameBuffer`` instance. An example of how to write image data to disk is available in the