[{"id":14203,"web_url":"https://patchwork.libcamera.org/comment/14203/","msgid":"<20201210174415.cf2hli25kay5lb2l@uno.localdomain>","date":"2020-12-10T17:44:15","subject":"Re: [libcamera-devel] [PATCH v6 1/3] libcamera: controls: Add frame\n\tduration control","submitter":{"id":3,"url":"https://patchwork.libcamera.org/api/people/3/","name":"Jacopo Mondi","email":"jacopo@jmondi.org"},"content":"Hi Naush,\n thanks for the update\n\nOn Thu, Dec 10, 2020 at 04:33:35PM +0000, Naushir Patuck wrote:\n> Add a float array control (controls::FrameDurations) to specify the\n> minimum and maximum (in that order) frame duration to be used by the\n> camera sensor.\n>\n> Signed-off-by: Naushir Patuck \n> Reviewed-by: David Plowman \n> Tested-by: David Plowman \n> ---\n> src/libcamera/control_ids.yaml | 40 ++++++++++++++++++++++++++++++++++\n> 1 file changed, 40 insertions(+)\n>\n> diff --git a/src/libcamera/control_ids.yaml b/src/libcamera/control_ids.yaml\n> index 6d6f0fee..5ee31865 100644\n> --- a/src/libcamera/control_ids.yaml\n> +++ b/src/libcamera/control_ids.yaml\n> @@ -306,6 +306,46 @@ controls:\n> maximum valid value is given by the properties::ScalerCropMaximum\n> property, and the two can be used to implement digital zoom.\n>\n> + - FrameDurations:\n> + type: int64_t\n> + description: |\n> + The minimum and maximum (in that order) frame duration,\n> + expressed in micro-seconds.\n> +\n> + When provided by applications, the control specifies the sensor frame\n> + duration interval the pipeline has to use. This could also limit the\n> + largest exposure times the sensor can use. For example, if a maximum\n\nIs 'times' intentional ? Or did you mean 'time' ? Honest question..\n\n> + frame duration of 33ms is requested (corresponding to 30 frames per\n> + second), the sensor will not be able to raise the exposure time above\n> + 33ms. A fixed frame duration is achieved by setting the minimum and\n> + maximum values to be the same. Setting both values to 0 resets the\n> + frame duration to the IPA defaults.\n\nI would drop the last statement as we should better think where\ndefaults will be specified (a property or implicitly in the IPA). I'll\nrather have a \\todo.\n\n> +\n> + The maximum frame duration provides the absolute limit to the shutter\n> + speed available to the AE algorithm. This limit also overrides any\n> + limits set by the exposure mode (AeExposureMode). Similarly, a manual\n> + ExposureTime value provided by the application may also be clipped by\n> + this limit.\n\nI would re-phrase this as:\n\n The maximum frame duration provides the absolute limit to the shutter\n speed computed by the AE algorithm and it overrides any exposure mode\n setting specified with controls::AeExposureMode. Similarly, when a\n manual exposure time is set through controls::ExposureTime, it also\n gets clipped in the limits set by this control.\n\nto make it more 'imperative'. It might sound all very \"must/shall\", but\nI fear documenting things as \"may/might\" as it means one application\nmight behave differently when run on different platforms. I would\nprefer to start strict, and in case the need arise, re-think how to\nmake these precedence relationships controllable ?\n\n> +\n> + \\sa AeExposureMode\n> + \\sa ExposureTime\n> +\n> + \\todo Refer to the frame duration limits property to describe how\n> + application-provided values gets clipped.\n\nIf you agree the statement above about reset should be dropped, I\nwould \"gets clipped and reset\".\n> +\n> + When reported by pipelines, the control expresses the duration of the\n> + sensor frame used to produce streams part of the completed Request.\n> + The minimum and maximum values shall then be the same, as the sensor\n> + frame duration is a fixed parameter. The sensor frame duration is one\n> + of the parameter that defines the capture frame rate but it does not\n> + alone provide enough information to fully calculate it as it does not\n> + account for pipeline processing delays.\n> +\n> + \\todo Define how to calculate the capture frame rate by\n> + defining controls to report additional delays introduced by\n> + the capture pipeline or post-processing stages (ie JPEG\n> + conversion, frame scaling).\n> +\n\nThanks, minor apart this looks good\nReviewed-by: Jacopo Mondi \n\nThanks\n j\n\n> # ----------------------------------------------------------------------------\n> # Draft controls section\n>\n> --\n> 2.25.1\n>\n> _______________________________________________\n> libcamera-devel mailing list\n> libcamera-devel@lists.libcamera.org\n> https://lists.libcamera.org/listinfo/libcamera-devel","headers":{"Return-Path":"","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id 6E472BD80A\n\tfor ;\n\tThu, 10 Dec 2020 17:44:08 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id 0735067F6E;\n\tThu, 10 Dec 2020 18:44:08 +0100 (CET)","from relay1-d.mail.gandi.net (relay1-d.mail.gandi.net\n\t[217.70.183.193])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id 5AF5A67E4D\n\tfor ;\n\tThu, 10 Dec 2020 18:44:06 +0100 (CET)","from uno.localdomain (93-34-118-233.ip49.fastwebnet.it\n\t[93.34.118.233]) (Authenticated sender: jacopo@jmondi.org)\n\tby relay1-d.mail.gandi.net (Postfix) with ESMTPSA id D41E8240005;\n\tThu, 10 Dec 2020 17:44:05 +0000 (UTC)"],"X-Originating-IP":"93.34.118.233","Date":"Thu, 10 Dec 2020 18:44:15 +0100","From":"Jacopo Mondi ","To":"Naushir Patuck ","Message-ID":"<20201210174415.cf2hli25kay5lb2l@uno.localdomain>","References":"<20201210163337.212857-1-naush@raspberrypi.com>","MIME-Version":"1.0","Content-Disposition":"inline","In-Reply-To":"<20201210163337.212857-1-naush@raspberrypi.com>","Subject":"Re: [libcamera-devel] [PATCH v6 1/3] libcamera: controls: Add frame\n\tduration control","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Cc":"libcamera-devel@lists.libcamera.org","Content-Type":"text/plain; charset=\"us-ascii\"","Content-Transfer-Encoding":"7bit","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" "}},{"id":14217,"web_url":"https://patchwork.libcamera.org/comment/14217/","msgid":"","date":"2020-12-11T09:48:11","subject":"Re: [libcamera-devel] [PATCH v6 1/3] libcamera: controls: Add frame\n\tduration control","submitter":{"id":34,"url":"https://patchwork.libcamera.org/api/people/34/","name":"Naushir Patuck","email":"naush@raspberrypi.com"},"content":"Hi Jacopo,\n\nOn Thu, 10 Dec 2020 at 17:44, Jacopo Mondi wrote:\n\n> Hi Naush,\n> thanks for the update\n>\n> On Thu, Dec 10, 2020 at 04:33:35PM +0000, Naushir Patuck wrote:\n> > Add a float array control (controls::FrameDurations) to specify the\n> > minimum and maximum (in that order) frame duration to be used by the\n> > camera sensor.\n> >\n> > Signed-off-by: Naushir Patuck \n> > Reviewed-by: David Plowman \n> > Tested-by: David Plowman \n> > ---\n> > src/libcamera/control_ids.yaml | 40 ++++++++++++++++++++++++++++++++++\n> > 1 file changed, 40 insertions(+)\n> >\n> > diff --git a/src/libcamera/control_ids.yaml\n> b/src/libcamera/control_ids.yaml\n> > index 6d6f0fee..5ee31865 100644\n> > --- a/src/libcamera/control_ids.yaml\n> > +++ b/src/libcamera/control_ids.yaml\n> > @@ -306,6 +306,46 @@ controls:\n> > maximum valid value is given by the\n> properties::ScalerCropMaximum\n> > property, and the two can be used to implement digital zoom.\n> >\n> > + - FrameDurations:\n> > + type: int64_t\n> > + description: |\n> > + The minimum and maximum (in that order) frame duration,\n> > + expressed in micro-seconds.\n> > +\n> > + When provided by applications, the control specifies the\n> sensor frame\n> > + duration interval the pipeline has to use. This could also\n> limit the\n> > + largest exposure times the sensor can use. For example, if a\n> maximum\n>\n> Is 'times' intentional ? Or did you mean 'time' ? Honest question..\n>\n\nYou are correct, it should be 'time' :)\n\n\n>\n> > + frame duration of 33ms is requested (corresponding to 30\n> frames per\n> > + second), the sensor will not be able to raise the exposure\n> time above\n> > + 33ms. A fixed frame duration is achieved by setting the\n> minimum and\n> > + maximum values to be the same. Setting both values to 0\n> resets the\n> > + frame duration to the IPA defaults.\n>\n> I would drop the last statement as we should better think where\n> defaults will be specified (a property or implicitly in the IPA). I'll\n> rather have a \\todo.\n>\n\nGood point. I will add a todo for this. FYI, for Raspberry Pi, if you set\n0, it does return back to the IPA specified default. If we decide this is\nnot appropriate, it is an easy modification.\n\n\n>\n> > +\n> > + The maximum frame duration provides the absolute limit to the\n> shutter\n> > + speed available to the AE algorithm. This limit also\n> overrides any\n> > + limits set by the exposure mode (AeExposureMode). Similarly,\n> a manual\n> > + ExposureTime value provided by the application may also be\n> clipped by\n> > + this limit.\n>\n> I would re-phrase this as:\n>\n> The maximum frame duration provides the absolute limit to the\n> shutter\n> speed computed by the AE algorithm and it overrides any exposure\n> mode\n> setting specified with controls::AeExposureMode. Similarly, when\n> a\n> manual exposure time is set through controls::ExposureTime, it\n> also\n> gets clipped in the limits set by this control.\n>\n> to make it more 'imperative'. It might sound all very \"must/shall\", but\n> I fear documenting things as \"may/might\" as it means one application\n> might behave differently when run on different platforms. I would\n> prefer to start strict, and in case the need arise, re-think how to\n> make these precedence relationships controllable ?\n>\n\nSure, I will reword that to your suggested text.\n\n\n>\n> > +\n> > + \\sa AeExposureMode\n> > + \\sa ExposureTime\n> > +\n> > + \\todo Refer to the frame duration limits property to describe\n> how\n> > + application-provided values gets clipped.\n>\n> If you agree the statement above about reset should be dropped, I\n> would \"gets clipped and reset\".\n>\n\nAgreed. Will post a new version with the updates shortly.\n\n> +\n> > + When reported by pipelines, the control expresses the\n> duration of the\n> > + sensor frame used to produce streams part of the completed\n> Request.\n> > + The minimum and maximum values shall then be the same, as the\n> sensor\n> > + frame duration is a fixed parameter. The sensor frame\n> duration is one\n> > + of the parameter that defines the capture frame rate but it\n> does not\n> > + alone provide enough information to fully calculate it as it\n> does not\n> > + account for pipeline processing delays.\n> > +\n> > + \\todo Define how to calculate the capture frame rate by\n> > + defining controls to report additional delays introduced by\n> > + the capture pipeline or post-processing stages (ie JPEG\n> > + conversion, frame scaling).\n> > +\n>\n> Thanks, minor apart this looks good\n> Reviewed-by: Jacopo Mondi \n>\n> Thanks\n> j\n>\n> > #\n> ----------------------------------------------------------------------------\n> > # Draft controls section\n> >\n> > --\n> > 2.25.1\n> >\n> > _______________________________________________\n> > libcamera-devel mailing list\n> > libcamera-devel@lists.libcamera.org\n> > https://lists.libcamera.org/listinfo/libcamera-devel\n>","headers":{"Return-Path":"","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id 0CE04BD808\n\tfor ;\n\tFri, 11 Dec 2020 09:48:30 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id 96B55635C6;\n\tFri, 11 Dec 2020 10:48:29 +0100 (CET)","from mail-lj1-x22c.google.com (mail-lj1-x22c.google.com\n\t[IPv6:2a00:1450:4864:20::22c])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id 700B9600FF\n\tfor ;\n\tFri, 11 Dec 2020 10:48:28 +0100 (CET)","by mail-lj1-x22c.google.com with SMTP id m13so10150995ljo.11\n\tfor ;\n\tFri, 11 Dec 2020 01:48:28 -0800 (PST)"],"Authentication-Results":"lancelot.ideasonboard.com;\n\tdkim=fail reason=\"signature verification failed\" (2048-bit key;\n\tunprotected) header.d=raspberrypi.com header.i=@raspberrypi.com\n\theader.b=\"LMqkK+En\"; dkim-atps=neutral","DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed;\n\td=raspberrypi.com; s=google;\n\th=mime-version:references:in-reply-to:from:date:message-id:subject:to\n\t:cc; bh=XAa/RvMbWTfLlAFBnkVUm7fCkfP6Hb1cPTkiRBzfeNc=;\n\tb=LMqkK+EnTu2GCe/UWzi9Xp4rJ7lD0wi5kpkqd0D7+3pHs7QoU1skLdy3emfTpF1I4n\n\tw6CnzkDqOOyZc3wAVh2/htHXkKogXHWTNlR7vyI4woi7HEWA1tCDAXegOSkES/unDFNS\n\tl6+StZRHcwdcaR3MgaQS/2PqjxJK+bv3CNFOO6pvcQtA8rQ8gZGJYB9uyummgQC7lGli\n\tgc0/WvllVm9VuXp2BAU4KzsfdQcRhHO8C70Plvx855GndXJUxqeu+dyiQL2lzkqBzWK/\n\t/uvpE95aFmN1XQHxHw8GX4pndbTU3TRvXaAYAe2K42TeNJCAKJaDC7CHurZ4t/Ws3y3Y\n\tzYow==","X-Google-DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed;\n\td=1e100.net; s=20161025;\n\th=x-gm-message-state:mime-version:references:in-reply-to:from:date\n\t:message-id:subject:to:cc;\n\tbh=XAa/RvMbWTfLlAFBnkVUm7fCkfP6Hb1cPTkiRBzfeNc=;\n\tb=kdhqXArT9Bxn4t3DUsRdGriWcTOWignT3T8jL/JQSfmJNbHiktI018+Dsz+fhtnH8N\n\tz9s1acX6altWZOICJHA2vjlyNKC26y5Hjinh6/HJdmQ/Q3q6OUpVXqH8df+ibNUQW6EX\n\tSMq7IAhyRUDMDhIn7RfVUdjDBh6ziqTPk8Z46iV62xjY8z0pQ2tFpUvJ1vzpaAfuGj/l\n\t9X3exRKPIDtd/NkbaFWURBDO1aRBdiG8GtIjrxsNjGVcoHVnqrexKjanNByE4Ie6rigM\n\tj9ve/d9euQxFZCBYHSwJD8cgZ9Gk819kOgXH6bl2+0Z784Mik37jq/LQKoPcY1yvG8pv\n\ta66Q==","X-Gm-Message-State":"AOAM533RyQ5wDxI50c/WdKsZ3VrBcYca5j/XrhaUgWFtekfVTURaq+zs\n\tk1nWpIEExr5uaB6wScFRKqRdIZGnKCoQlilns11uUmvnQU8=","X-Google-Smtp-Source":"ABdhPJy6igbFEQlfNkUVR2k6zF2Fz/PWorSH2mmhO46ns6P3K7aooNW5+djyMBWKtROax+6O3AUU/dysvY3NBdISL8Q=","X-Received":"by 2002:a2e:810c:: with SMTP id d12mr483417ljg.400.1607680107655;\n\tFri, 11 Dec 2020 01:48:27 -0800 (PST)","MIME-Version":"1.0","References":"<20201210163337.212857-1-naush@raspberrypi.com>\n\t<20201210174415.cf2hli25kay5lb2l@uno.localdomain>","In-Reply-To":"<20201210174415.cf2hli25kay5lb2l@uno.localdomain>","From":"Naushir Patuck ","Date":"Fri, 11 Dec 2020 09:48:11 +0000","Message-ID":"","To":"Jacopo Mondi ","Subject":"Re: [libcamera-devel] [PATCH v6 1/3] libcamera: controls: Add frame\n\tduration control","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Cc":"libcamera devel ","Content-Type":"multipart/mixed;\n\tboundary=\"===============5046720174611543241==\"","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" "}}]