[{"id":39845,"web_url":"https://patchwork.libcamera.org/comment/39845/","msgid":"<49aacfaf-5a70-410c-ba08-1a1185010228@googlemail.com>","date":"2026-07-24T15:01:45","subject":"Re: [PATCH v2 0/2] libcamera: Add SensorSequence request metadata\n\tcontrol","submitter":{"id":321,"url":"https://patchwork.libcamera.org/api/people/321/","name":"Bruce Allen","email":"ballen4705@googlemail.com"},"content":"To help motivate one of the core developers to merge the patch, I’m \nattaching a plot showing sensor frame drops from an IMX477 camera \nconnected to a Raspberry Pi Compute Module 4.\n\nOver a 21-hour period, the system recorded 17 dropped frames while \ncapturing 2028 × 1520, 12-bit images at 73 FPS. The CSI bus was running \nat 750 MHz, which is reliable in this setup because the cable is only 50 \nmm long.\n\nMost events involved a single dropped frame, although a few occurred \nclose together and two involved consecutive frame drops.\n\nFor my application, a drop rate of roughly three frames per million is \nnegligible. However, achieving that rate required some development \neffort, and having this instrumentation available was valuable for \nmeasuring the impact of those changes.\n\nFor what it’s worth, I have not been able to correlate the frame drops \nwith any other activity on the system.\n\nCheers,\n\tBruce\n\n\n\nOn 04.07.26 00:48, Bruce Allen wrote:\n> Hi,\n> \n> This v2 series follows up on my v1 submission that exposed the Unicam\n> frontend sequence as an RPi vendor control [1], and Kieran's feedback that\n> this should be a standard metadata control across all platforms [2].\n> \n> The series revives the approach from Kieran's 2021 RFC \"Request metadata:\n> SensorSequence\" [3], updated for the current control_ids_core.yaml layout\n> and additional pipelines. Compared with that original 8-patch series, this\n> submission is limited to the control definition, pipeline plumbing, and\n> application documentation (no cam/qcam/gstreamer changes).\n> \n> Changes since v1:\n> - Define controls::SensorSequence in control_ids_core.yaml (int64_t, out)\n> instead of controls::rpi::SensorSequence in control_ids_rpi.yaml\n> - Populate the control in every pipeline handler that already reports\n> SensorTimestamp (simple, ipu3, rkisp1, rpi/vc4, rpi/pisp, mali-c55,\n> imx8-isi, uvcvideo, vimc)\n> - Document SensorSequence in the application developer guide\n> \n> Motivation (unchanged from v1): applications diagnosing frame loss need a\n> counter tied to sensor-side capture, not just delivered-frame or request-\n> queue sequence. Gaps in SensorSequence indicate frames that did not reach the\n> capture device stream before the previous request completed. Python bindings\n> follow from existing pycamera codegen.\n> \n> Tested: patch applies cleanly to master @ 709ad59; builds with\n> -Dpipelines=rpi/vc4 -Dipas=rpi/vc4 -Dpycamera=enabled. Verified on\n> Raspberry Pi CM4 that monotonic SensorSequence detects dropped frames.\n> \n> [1] https://lists.libcamera.org/pipermail/libcamera-devel/2026-June/059669.html\n> [2] https://lists.libcamera.org/pipermail/libcamera-devel/2026-June/059681.html\n> [3] https://patchwork.libcamera.org/project/libcamera/list/?series=2822\n> \n> Cheers,\n> Bruce Allen","headers":{"Return-Path":"","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id 208B3BDE4C\n\tfor ;\n\tFri, 24 Jul 2026 15:33:50 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id 0814767F34;\n\tFri, 24 Jul 2026 17:33:49 +0200 (CEST)","from mail-wr1-x432.google.com (mail-wr1-x432.google.com\n\t[IPv6:2a00:1450:4864:20::432])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id 3C58C67F22\n\tfor ;\n\tFri, 24 Jul 2026 17:01:48 +0200 (CEST)","by mail-wr1-x432.google.com with SMTP id\n\tffacd0b85a97d-47f707e5d75so425927f8f.1\n\tfor ;\n\tFri, 24 Jul 2026 08:01:48 -0700 (PDT)","from [192.168.123.201] (p5ddcc513.dip0.t-ipconnect.de.\n\t[93.220.197.19]) by smtp.gmail.com with ESMTPSA id\n\tffacd0b85a97d-47f85c531basm25170149f8f.19.2026.07.24.08.01.45\n\t(version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);\n\tFri, 24 Jul 2026 08:01:46 -0700 (PDT)"],"Authentication-Results":"lancelot.ideasonboard.com; dkim=pass (2048-bit key;\n\tunprotected) header.d=googlemail.com header.i=@googlemail.com\n\theader.b=\"W3fYoeTq\"; dkim-atps=neutral","DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed;\n\td=googlemail.com; s=20251104; t=1784905308; x=1785510108;\n\tdarn=lists.libcamera.org; \n\th=in-reply-to:from:cc:content-language:references:to:subject\n\t:user-agent:mime-version:date:message-id:content-type:from:to:cc\n\t:subject:date:message-id:reply-to:content-type;\n\tbh=cUE8q6rE/AAEDHMX+51VchLymmWQGBYmjwuuASwQe40=;\n\tb=W3fYoeTqafokrXDvDF8l3+XpTvFBcAU2pB+Su+vB5YldIG4OISt6l0t32UW0Bl73Hk\n\tcvMGVpwgKbNXyEQjRX3306Oy24vg/HGqfy1Xs38Sz2NYg/goKX4IzEjiloL6S1gN8kKl\n\t/8mKmZAnIi2ov2D0eirO9jGc3S2XGlCWijafufjHszyGuxrKuAmkmtvfGlI2NahqrwB0\n\tKWQ8ZZxqU4UWqho17uj9Y2kLuDCpb/iqyjWywdEmWru298X+LTOXjA9cEUV14xYybKs+\n\tRlt7LaNPQ/79XOKSLfcOACe6hXQLBHvtJaznIimfCdufnkxH8SDmq4b1+1FpP+JcT2H2\n\tlvPg==","X-Google-DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed;\n\td=1e100.net; s=20251104; t=1784905308; x=1785510108;\n\th=in-reply-to:from:cc:content-language:references:to:subject\n\t:user-agent:mime-version:date:message-id:content-type:x-gm-gg\n\t:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to\n\t:content-type;\n\tbh=cUE8q6rE/AAEDHMX+51VchLymmWQGBYmjwuuASwQe40=;\n\tb=qYA0UBzoqfcmAwQ/2sy23LGtA0ic1GQO2gOlUa3rnaBkdzrossyP6gR2Nh+b6Blkc9\n\tBgI6IltzLizRk6ORUQwoleplZMXv7X/PskOhjP7tO17ZVtIeS8i7eO2VQ3E2kmq5Mi6E\n\tNiyf/YLW1O+x3P8zjZ8IwoL4EIxytpF34uPunQy9gBO13v1XJ7zbyi4rARRuMwP6DbRw\n\tRYAIgl3aLhvkLSs7Oroih2tnMmNzeFFb29i4z7BqNf+5n12UsZxuty2o8Ozw2MgNXCYR\n\t5ZOufxU2yXXQgFY1TU9p4TbGxfisnjKucQy8eWuRvfC1UfTCrG5pAxT5yL5d0OFbX030\n\tB4Uw==","X-Gm-Message-State":"AOJu0YyQx9vPRUJ0jcYqDuWnZLj4kXHCxfJuPvuy6lmlz0jXbYrfqi5M\n\tzKBUlBu1rCKC0Jscyx5iACFR2Y9BP3gPbdkbqx8/9yS0/QlB4r7vffuX0Cg7Z+lp","X-Gm-Gg":"AR+sD10/ZcfoOiMpWkA2SeTo3wBfi4Sp91FfmVBfIZNvq+mVa15P7p5d18BceHGyTlM\n\tfJOcVxV9Lb5z1mibXhONmhsIKWDKOojE6bTeYWtD9U6eaKFFHllgwjcZkC/0CYGmzwmiVfDKp1h\n\thxpqwTOFqSscZaYY3JSWFYXuGYDoRYhDbRrkLCzQ+Jtl2GxDo4YASkOCBBou0hspuUCerrcNVjP\n\tduugh8qMW3q8tCvC84znIeLyIDjYG6JbHRNe8BL77ql75L/fc3gpAXoO7lLLuA7g+QmlF9VVgTv\n\tH5BoJR7ss8lOUYTXk3riFOovgySgI0bCWHdxPVQ8XmUq8rV8QGVnzyHJZCwz2Of5Hi3QsDaoiOB\n\tt+gk3eirAl3TGSQZkdwOUqMIMJEojFbh3FL4HsQcR0juhX+LbJmPFhQOGVPjIlNxvKyK3aogELj\n\tmkmppRzXAEEqJ+3qEW1UCkq574vnmq8I9K2RkNJq6vgpGRlJ3bRaDgtfh59yZT8w==","X-Received":"by 2002:a05:6000:2410:b0:45e:64b3:af44 with SMTP id\n\tffacd0b85a97d-47f8dca37b5mr9931197f8f.36.1784905307449; \n\tFri, 24 Jul 2026 08:01:47 -0700 (PDT)","Content-Type":"multipart/mixed;\n\tboundary=\"------------u8t0i7hr3E4JIHcMmHiW9aeq\"","Message-ID":"<49aacfaf-5a70-410c-ba08-1a1185010228@googlemail.com>","Date":"Fri, 24 Jul 2026 17:01:45 +0200","MIME-Version":"1.0","User-Agent":"Mozilla Thunderbird","Subject":"Re: [PATCH v2 0/2] libcamera: Add SensorSequence request metadata\n\tcontrol","To":"libcamera-devel@lists.libcamera.org","References":"<20260626003712.26624-1-ballen4705@googlemail.com>\n\t<20260703224817.15650-1-ballen4705@googlemail.com>","Content-Language":"en-US","Cc":"Kieran Bingham ","From":"Bruce Allen ","In-Reply-To":"<20260703224817.15650-1-ballen4705@googlemail.com>","X-Mailman-Approved-At":"Fri, 24 Jul 2026 17:33:47 +0200","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" "}},{"id":39902,"web_url":"https://patchwork.libcamera.org/comment/39902/","msgid":"<33cd4273-3013-4938-87bb-3c955b2f283b@ideasonboard.com>","date":"2026-07-29T09:21:54","subject":"Re: [PATCH v2 0/2] libcamera: Add SensorSequence request metadata\n\tcontrol","submitter":{"id":216,"url":"https://patchwork.libcamera.org/api/people/216/","name":"Barnabás Pőcze","email":"barnabas.pocze@ideasonboard.com"},"content":"Hi\n\n2026. 07. 04. 0:48 keltezéssel, Bruce Allen írta:\n> Hi,\n> \n> This v2 series follows up on my v1 submission that exposed the Unicam\n> frontend sequence as an RPi vendor control [1], and Kieran's feedback that\n> this should be a standard metadata control across all platforms [2].\n> \n> The series revives the approach from Kieran's 2021 RFC \"Request metadata:\n> SensorSequence\" [3], updated for the current control_ids_core.yaml layout\n> and additional pipelines. Compared with that original 8-patch series, this\n> submission is limited to the control definition, pipeline plumbing, and\n> application documentation (no cam/qcam/gstreamer changes).\n> \n> Changes since v1:\n> - Define controls::SensorSequence in control_ids_core.yaml (int64_t, out)\n> instead of controls::rpi::SensorSequence in control_ids_rpi.yaml\n> - Populate the control in every pipeline handler that already reports\n> SensorTimestamp (simple, ipu3, rkisp1, rpi/vc4, rpi/pisp, mali-c55,\n> imx8-isi, uvcvideo, vimc)\n> - Document SensorSequence in the application developer guide\n> \n> Motivation (unchanged from v1): applications diagnosing frame loss need a\n> counter tied to sensor-side capture, not just delivered-frame or request-\n> queue sequence. Gaps in SensorSequence indicate frames that did not reach the\n> capture device stream before the previous request completed. Python bindings\n> follow from existing pycamera codegen.\n\nI think this seems like a reasonable idea, after all, there is already `SensorTimestamp`,\nand arguably it's not very convenient to get that info from `FrameBuffer::Metadata::sequence`\n(if at all possible with the given configuration).\n\n\n> \n> Tested: patch applies cleanly to master @ 709ad59; builds with\n> -Dpipelines=rpi/vc4 -Dipas=rpi/vc4 -Dpycamera=enabled. Verified on\n> Raspberry Pi CM4 that monotonic SensorSequence detects dropped frames.\n> \n> [1] https://lists.libcamera.org/pipermail/libcamera-devel/2026-June/059669.html\n> [2] https://lists.libcamera.org/pipermail/libcamera-devel/2026-June/059681.html\n> [3] https://patchwork.libcamera.org/project/libcamera/list/?series=2822\n\nThere was an interesting comment in [3]: https://patchwork.libcamera.org/patch/15057/#21643\nabout the value of this control after a stop/start sequence. I think this should\nbe very clearly documented.\n\nAnother thing that should be clearly laid out in my opinion is whether the application\ncan expect this metadata item on every (successfully) completed request or not.\n\nAnd finally, there was yet again a comment in [3]: https://patchwork.libcamera.org/cover/15056/#21618\nregarding whether requests should be failed in case of frame drops, instead. That\ndefinitely seems like a much larger work as this behaviour is not implemented anywhere\nin libcamera (as far as I am aware). And while the two are somewhat orthogonal,\nif the main purpose of this control is to detect frame drops, then the two are\nkind of at odds with each other, so some kind of decision needs to be made here.\n\n\nRegards,\nBarnabás Pőcze\n\n> \n> Cheers,\n> Bruce Allen","headers":{"Return-Path":"","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id CC68EC328C\n\tfor ;\n\tWed, 29 Jul 2026 09:21:58 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id 771C067FAC;\n\tWed, 29 Jul 2026 11:21:58 +0200 (CEST)","from perceval.ideasonboard.com (perceval.ideasonboard.com\n\t[IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id C7D6F65F9F\n\tfor ;\n\tWed, 29 Jul 2026 11:21:56 +0200 (CEST)","from [192.168.33.16] (185.182.215.156.nat.pool.zt.hu\n\t[185.182.215.156])\n\tby perceval.ideasonboard.com (Postfix) with ESMTPSA id B175463C;\n\tWed, 29 Jul 2026 11:20:51 +0200 (CEST)"],"Authentication-Results":"lancelot.ideasonboard.com; dkim=pass (1024-bit key;\n\tunprotected) header.d=ideasonboard.com header.i=@ideasonboard.com\n\theader.b=\"PPvtga63\"; dkim-atps=neutral","DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com;\n\ts=mail; t=1785316851;\n\tbh=YYYM13joXNcuDnR2jEW+PgpdDEbKLo4tPe2CrRlFEj8=;\n\th=Date:Subject:To:References:From:In-Reply-To:From;\n\tb=PPvtga63R9JVvvgLhe+WAaEAgNkLhlfL4xrn/KOQLpEETg2plRgXkrV1ZXmRwobUy\n\tyPbiJAytCG7X0HGO82SfQ9PdTnLkuxkRl49wS3brEnC+IjdEc/fPAjgFNKeM8v0Bb2\n\t2nCoban/R2XmX7YH/ZOFFo6NffgjFCrYlyDpTpJw=","Message-ID":"<33cd4273-3013-4938-87bb-3c955b2f283b@ideasonboard.com>","Date":"Wed, 29 Jul 2026 11:21:54 +0200","MIME-Version":"1.0","User-Agent":"Mozilla Thunderbird","Subject":"Re: [PATCH v2 0/2] libcamera: Add SensorSequence request metadata\n\tcontrol","To":"Bruce Allen ,\n\tlibcamera-devel@lists.libcamera.org","References":"<20260626003712.26624-1-ballen4705@googlemail.com>\n\t<20260703224817.15650-1-ballen4705@googlemail.com>","From":"=?utf-8?q?Barnab=C3=A1s_P=C5=91cze?= ","Content-Language":"en-US, hu-HU","In-Reply-To":"<20260703224817.15650-1-ballen4705@googlemail.com>","Content-Type":"text/plain; charset=UTF-8; format=flowed","Content-Transfer-Encoding":"8bit","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" "}}]