| Message ID | 20260703224817.15650-1-ballen4705@googlemail.com |
|---|---|
| Headers | show |
| Series |
|
| Related | show |
To help motivate one of the core developers to merge the patch, I’m attaching a plot showing sensor frame drops from an IMX477 camera connected to a Raspberry Pi Compute Module 4. Over a 21-hour period, the system recorded 17 dropped frames while capturing 2028 × 1520, 12-bit images at 73 FPS. The CSI bus was running at 750 MHz, which is reliable in this setup because the cable is only 50 mm long. Most events involved a single dropped frame, although a few occurred close together and two involved consecutive frame drops. For my application, a drop rate of roughly three frames per million is negligible. However, achieving that rate required some development effort, and having this instrumentation available was valuable for measuring the impact of those changes. For what it’s worth, I have not been able to correlate the frame drops with any other activity on the system. Cheers, Bruce On 04.07.26 00:48, Bruce Allen wrote: > Hi, > > This v2 series follows up on my v1 submission that exposed the Unicam > frontend sequence as an RPi vendor control [1], and Kieran's feedback that > this should be a standard metadata control across all platforms [2]. > > The series revives the approach from Kieran's 2021 RFC "Request metadata: > SensorSequence" [3], updated for the current control_ids_core.yaml layout > and additional pipelines. Compared with that original 8-patch series, this > submission is limited to the control definition, pipeline plumbing, and > application documentation (no cam/qcam/gstreamer changes). > > Changes since v1: > - Define controls::SensorSequence in control_ids_core.yaml (int64_t, out) > instead of controls::rpi::SensorSequence in control_ids_rpi.yaml > - Populate the control in every pipeline handler that already reports > SensorTimestamp (simple, ipu3, rkisp1, rpi/vc4, rpi/pisp, mali-c55, > imx8-isi, uvcvideo, vimc) > - Document SensorSequence in the application developer guide > > Motivation (unchanged from v1): applications diagnosing frame loss need a > counter tied to sensor-side capture, not just delivered-frame or request- > queue sequence. Gaps in SensorSequence indicate frames that did not reach the > capture device stream before the previous request completed. Python bindings > follow from existing pycamera codegen. > > Tested: patch applies cleanly to master @ 709ad59; builds with > -Dpipelines=rpi/vc4 -Dipas=rpi/vc4 -Dpycamera=enabled. Verified on > Raspberry Pi CM4 that monotonic SensorSequence detects dropped frames. > > [1] https://lists.libcamera.org/pipermail/libcamera-devel/2026-June/059669.html > [2] https://lists.libcamera.org/pipermail/libcamera-devel/2026-June/059681.html > [3] https://patchwork.libcamera.org/project/libcamera/list/?series=2822 > > Cheers, > Bruce Allen
Hi 2026. 07. 04. 0:48 keltezéssel, Bruce Allen írta: > Hi, > > This v2 series follows up on my v1 submission that exposed the Unicam > frontend sequence as an RPi vendor control [1], and Kieran's feedback that > this should be a standard metadata control across all platforms [2]. > > The series revives the approach from Kieran's 2021 RFC "Request metadata: > SensorSequence" [3], updated for the current control_ids_core.yaml layout > and additional pipelines. Compared with that original 8-patch series, this > submission is limited to the control definition, pipeline plumbing, and > application documentation (no cam/qcam/gstreamer changes). > > Changes since v1: > - Define controls::SensorSequence in control_ids_core.yaml (int64_t, out) > instead of controls::rpi::SensorSequence in control_ids_rpi.yaml > - Populate the control in every pipeline handler that already reports > SensorTimestamp (simple, ipu3, rkisp1, rpi/vc4, rpi/pisp, mali-c55, > imx8-isi, uvcvideo, vimc) > - Document SensorSequence in the application developer guide > > Motivation (unchanged from v1): applications diagnosing frame loss need a > counter tied to sensor-side capture, not just delivered-frame or request- > queue sequence. Gaps in SensorSequence indicate frames that did not reach the > capture device stream before the previous request completed. Python bindings > follow from existing pycamera codegen. I think this seems like a reasonable idea, after all, there is already `SensorTimestamp`, and arguably it's not very convenient to get that info from `FrameBuffer::Metadata::sequence` (if at all possible with the given configuration). > > Tested: patch applies cleanly to master @ 709ad59; builds with > -Dpipelines=rpi/vc4 -Dipas=rpi/vc4 -Dpycamera=enabled. Verified on > Raspberry Pi CM4 that monotonic SensorSequence detects dropped frames. > > [1] https://lists.libcamera.org/pipermail/libcamera-devel/2026-June/059669.html > [2] https://lists.libcamera.org/pipermail/libcamera-devel/2026-June/059681.html > [3] https://patchwork.libcamera.org/project/libcamera/list/?series=2822 There was an interesting comment in [3]: https://patchwork.libcamera.org/patch/15057/#21643 about the value of this control after a stop/start sequence. I think this should be very clearly documented. Another thing that should be clearly laid out in my opinion is whether the application can expect this metadata item on every (successfully) completed request or not. And finally, there was yet again a comment in [3]: https://patchwork.libcamera.org/cover/15056/#21618 regarding whether requests should be failed in case of frame drops, instead. That definitely seems like a much larger work as this behaviour is not implemented anywhere in libcamera (as far as I am aware). And while the two are somewhat orthogonal, if the main purpose of this control is to detect frame drops, then the two are kind of at odds with each other, so some kind of decision needs to be made here. Regards, Barnabás Pőcze > > Cheers, > Bruce Allen