[{"id":10877,"web_url":"https://patchwork.libcamera.org/comment/10877/","msgid":"<20200626023507.GY5865@pendragon.ideasonboard.com>","date":"2020-06-26T02:35:07","subject":"Re: [libcamera-devel] [PATCH v3 5/8] libcamera: properties: Add\n\tFrameDurationLimits","submitter":{"id":2,"url":"https://patchwork.libcamera.org/api/people/2/","name":"Laurent Pinchart","email":"laurent.pinchart@ideasonboard.com"},"content":"Hi Jacopo,\n\nThank you for the patch.\n\nOn Fri, Jun 05, 2020 at 04:09:59PM +0200, Jacopo Mondi wrote:\n> Add a camera property to express the minimum and maximum frame durations.\n> \n> Signed-off-by: Jacopo Mondi \n> ---\n> \n> Cc Naush and Dave as this could potentially conflict with their on-going FPS\n> handling series. Sending it out for discussion.\n> \n> ---\n> src/libcamera/property_ids.yaml | 47 +++++++++++++++++++++++++++++++++\n> 1 file changed, 47 insertions(+)\n> \n> diff --git a/src/libcamera/property_ids.yaml b/src/libcamera/property_ids.yaml\n> index ce627fa042ba..d703ab31eaac 100644\n> --- a/src/libcamera/property_ids.yaml\n> +++ b/src/libcamera/property_ids.yaml\n> @@ -386,4 +386,51 @@ controls:\n> | |\n> | |\n> +--------------------+\n> +\n> + - FrameDurationLimits:\n> + type: int32_t\n> + size: [2]\n> + description: |\n> + The camera supported frame durations interval.\n> +\n> + This property reports the camera minimum and maximum frame durations (in\n> + this order) expressed in nanoseconds to report the limits of the\n> + achievable frame rate.\n> +\n> + Camera devices should here report durations calculated by inspecting the\n> + camera sensor supported frame rate interval, and adding to it any\n> + known additional delay caused by the image acquisition process that\n> + incurs in the time between a frame is captured and delivered to\n> + applications.\n\nThose are two unrelated properties. The processing delay doesn't affect\nthe frame duration. Reporting the processing delay may make sense, but\nit should then be another property.\n\n> + The here reported durations represent the time interval that occurs\n> + between the delivery of two consecutive frames for the fastest and\n> + slower streams, without considering additional delays introduced by the\n> + sharing of system resources when multiple streams are captured at the\n> + same time.\n> +\n> + This implies that the minimum reported frame duration, which corresponds\n> + to the highest possible camera frame rate, is calculated without taking\n> + into consideration how multiple streams capture requests sent to\n> + the camera as part of the same capture session might influence the frame\n> + delivery rate by, in example, introducing delays due to the requirement\n> + of time-sharing components part of the image acquisition pipeline. In\n> + example, request containing two scaled-down streams might require\n> + time-sharing the single scaler available in the system, introducing an\n> + additional delay that prevents the camera to deliver frames at the\n> + here reported rate.\n\nThis also mixes the delays and frame rates, it's quite unclear to me :-S\n\n> +\n> + In the same way, the maximum frame duration, which corresponds to the\n> + lowest possible frame rate, does not take into consideration additional\n> + processing delays introduced by image encoding and processing that\n> + happens sporadically, in example, to produce still images in JPEG\n> + format.\n> +\n> + As a consequence, application should inspect this property to know the\n> + camera capability limits, but should not assume the here reported values\n> + are achievable under all circumstances.\n> +\n> + # \\todo Expand to better describe how stalling streams (ie JPEG) impacts\n> + # the frame rate calculation.\n\nThis ends up in the generated .cpp file, and generates a doxygen\nwarning. You can either drop the # (in which case the todo will end up\nin doxygen), or reduce the indentation by two spaces.\n\n> +\n> ...","headers":{"Return-Path":"","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id 4D097C0109\n\tfor ;\n\tFri, 26 Jun 2020 02:35:13 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id C76DE609C6;\n\tFri, 26 Jun 2020 04:35:12 +0200 (CEST)","from perceval.ideasonboard.com (perceval.ideasonboard.com\n\t[213.167.242.64])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id 3870D603BA\n\tfor ;\n\tFri, 26 Jun 2020 04:35:11 +0200 (CEST)","from pendragon.ideasonboard.com (81-175-216-236.bb.dnainternet.fi\n\t[81.175.216.236])\n\tby perceval.ideasonboard.com (Postfix) with ESMTPSA id 3C6F472E;\n\tFri, 26 Jun 2020 04:35:10 +0200 (CEST)"],"Authentication-Results":"lancelot.ideasonboard.com;\n\tdkim=fail reason=\"signature verification failed\" (1024-bit key;\n\tunprotected) header.d=ideasonboard.com header.i=@ideasonboard.com\n\theader.b=\"m2Iq0QnP\"; dkim-atps=neutral","DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com;\n\ts=mail; t=1593138910;\n\tbh=OsMFh3ZlqV3A+Q5UPaGVfI9AEuvCJsBc0Whge+fX5S8=;\n\th=Date:From:To:Cc:Subject:References:In-Reply-To:From;\n\tb=m2Iq0QnP8JF2IxC2JibeFXHD2+fUcyAceZFfish11o738QWeCdZUf4bRvWW4rt3ME\n\tVLnV7YB9WreeJEMSWG0XMcbX24p6d15mQkaKrQ9+JjEvi6xDP16zVbSf3J4nfOQPRl\n\tMkGgXQOb7MgM3vs5Y2S9XBGRD/aiENJWeGp/a9pA=","Date":"Fri, 26 Jun 2020 05:35:07 +0300","From":"Laurent Pinchart ","To":"Jacopo Mondi ","Message-ID":"<20200626023507.GY5865@pendragon.ideasonboard.com>","References":"<20200605141002.49119-1-jacopo@jmondi.org>\n\t<20200605141002.49119-6-jacopo@jmondi.org>","MIME-Version":"1.0","Content-Disposition":"inline","In-Reply-To":"<20200605141002.49119-6-jacopo@jmondi.org>","Subject":"Re: [libcamera-devel] [PATCH v3 5/8] libcamera: properties: Add\n\tFrameDurationLimits","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Cc":"libcamera-devel@lists.libcamera.org","Content-Type":"text/plain; charset=\"us-ascii\"","Content-Transfer-Encoding":"7bit","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" "}},{"id":10892,"web_url":"https://patchwork.libcamera.org/comment/10892/","msgid":"<20200626155932.sjuxioi4y2qxdbdr@uno.localdomain>","date":"2020-06-26T15:59:32","subject":"Re: [libcamera-devel] [PATCH v3 5/8] libcamera: properties: Add\n\tFrameDurationLimits","submitter":{"id":3,"url":"https://patchwork.libcamera.org/api/people/3/","name":"Jacopo Mondi","email":"jacopo@jmondi.org"},"content":"Hi Laurent,\n\nOn Fri, Jun 26, 2020 at 05:35:07AM +0300, Laurent Pinchart wrote:\n> Hi Jacopo,\n>\n> Thank you for the patch.\n>\n> On Fri, Jun 05, 2020 at 04:09:59PM +0200, Jacopo Mondi wrote:\n> > Add a camera property to express the minimum and maximum frame durations.\n> >\n> > Signed-off-by: Jacopo Mondi \n> > ---\n> >\n> > Cc Naush and Dave as this could potentially conflict with their on-going FPS\n> > handling series. Sending it out for discussion.\n> >\n> > ---\n> > src/libcamera/property_ids.yaml | 47 +++++++++++++++++++++++++++++++++\n> > 1 file changed, 47 insertions(+)\n> >\n> > diff --git a/src/libcamera/property_ids.yaml b/src/libcamera/property_ids.yaml\n> > index ce627fa042ba..d703ab31eaac 100644\n> > --- a/src/libcamera/property_ids.yaml\n> > +++ b/src/libcamera/property_ids.yaml\n> > @@ -386,4 +386,51 @@ controls:\n> > | |\n> > | |\n> > +--------------------+\n> > +\n> > + - FrameDurationLimits:\n> > + type: int32_t\n> > + size: [2]\n> > + description: |\n> > + The camera supported frame durations interval.\n> > +\n> > + This property reports the camera minimum and maximum frame durations (in\n> > + this order) expressed in nanoseconds to report the limits of the\n> > + achievable frame rate.\n> > +\n> > + Camera devices should here report durations calculated by inspecting the\n> > + camera sensor supported frame rate interval, and adding to it any\n> > + known additional delay caused by the image acquisition process that\n> > + incurs in the time between a frame is captured and delivered to\n> > + applications.\n>\n> Those are two unrelated properties. The processing delay doesn't affect\n> the frame duration. Reporting the processing delay may make sense, but\n\nDoesn't it ? From an application perspective the frame rate is\nactually defined by both the sensor frame duration and the ISP\nprocessing delays, if known. I understand one is a sensor property and\nthe other one is platform/use-case specific, but from an application,\nwould it be better to have the two properties split and then having to\nconsider both ?\n\n> it should then be another property.\n>\n> > + The here reported durations represent the time interval that occurs\n> > + between the delivery of two consecutive frames for the fastest and\n> > + slower streams, without considering additional delays introduced by the\n> > + sharing of system resources when multiple streams are captured at the\n> > + same time.\n> > +\n> > + This implies that the minimum reported frame duration, which corresponds\n> > + to the highest possible camera frame rate, is calculated without taking\n> > + into consideration how multiple streams capture requests sent to\n> > + the camera as part of the same capture session might influence the frame\n> > + delivery rate by, in example, introducing delays due to the requirement\n> > + of time-sharing components part of the image acquisition pipeline. In\n> > + example, request containing two scaled-down streams might require\n> > + time-sharing the single scaler available in the system, introducing an\n> > + additional delay that prevents the camera to deliver frames at the\n> > + here reported rate.\n>\n> This also mixes the delays and frame rates, it's quite unclear to me :-S\n>\n> > +\n> > + In the same way, the maximum frame duration, which corresponds to the\n> > + lowest possible frame rate, does not take into consideration additional\n> > + processing delays introduced by image encoding and processing that\n> > + happens sporadically, in example, to produce still images in JPEG\n> > + format.\n> > +\n> > + As a consequence, application should inspect this property to know the\n> > + camera capability limits, but should not assume the here reported values\n> > + are achievable under all circumstances.\n> > +\n> > + # \\todo Expand to better describe how stalling streams (ie JPEG) impacts\n> > + # the frame rate calculation.\n>\n> This ends up in the generated .cpp file, and generates a doxygen\n> warning. You can either drop the # (in which case the todo will end up\n> in doxygen), or reduce the indentation by two spaces.\n>\n> > +\n> > ...\n>\n> --\n> Regards,\n>\n> Laurent Pinchart","headers":{"Return-Path":"","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id E3BB9C0109\n\tfor ;\n\tFri, 26 Jun 2020 15:56:04 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id 85971609C9;\n\tFri, 26 Jun 2020 17:56:04 +0200 (CEST)","from relay8-d.mail.gandi.net (relay8-d.mail.gandi.net\n\t[217.70.183.201])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id 503D8603BB\n\tfor ;\n\tFri, 26 Jun 2020 17:56:03 +0200 (CEST)","from uno.localdomain (2-224-242-101.ip172.fastwebnet.it\n\t[2.224.242.101]) (Authenticated sender: jacopo@jmondi.org)\n\tby relay8-d.mail.gandi.net (Postfix) with ESMTPSA id B167C1BF20B;\n\tFri, 26 Jun 2020 15:56:02 +0000 (UTC)"],"X-Originating-IP":"2.224.242.101","Date":"Fri, 26 Jun 2020 17:59:32 +0200","From":"Jacopo Mondi ","To":"Laurent Pinchart ","Message-ID":"<20200626155932.sjuxioi4y2qxdbdr@uno.localdomain>","References":"<20200605141002.49119-1-jacopo@jmondi.org>\n\t<20200605141002.49119-6-jacopo@jmondi.org>\n\t<20200626023507.GY5865@pendragon.ideasonboard.com>","MIME-Version":"1.0","Content-Disposition":"inline","In-Reply-To":"<20200626023507.GY5865@pendragon.ideasonboard.com>","Subject":"Re: [libcamera-devel] [PATCH v3 5/8] libcamera: properties: Add\n\tFrameDurationLimits","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Cc":"libcamera-devel@lists.libcamera.org","Content-Type":"text/plain; charset=\"us-ascii\"","Content-Transfer-Encoding":"7bit","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" "}}]