[{"id":14271,"web_url":"https://patchwork.libcamera.org/comment/14271/","msgid":"<13758e44-f364-1acd-d4ef-f21754f2e220@ideasonboard.com>","date":"2020-12-18T12:44:13","subject":"Re: [libcamera-devel] [PATCH v10 1/3] libcamera: controls: Add\n\tframe duration control","submitter":{"id":4,"url":"https://patchwork.libcamera.org/api/people/4/","name":"Kieran Bingham","email":"kieran.bingham@ideasonboard.com"},"content":"Hi Naush,\n\nOn 18/12/2020 10:06, Naushir Patuck wrote:\n> Add an int64_t array control (controls::FrameDurations) to specify the\n> minimum and maximum (in that order) frame duration to be used by the\n> camera sensor.\n> \n> Signed-off-by: Naushir Patuck \n> Reviewed-by: David Plowman \n> Tested-by: David Plowman \n> Reviewed-by: Jacopo Mondi \n> ---\n> src/libcamera/control_ids.yaml | 41 ++++++++++++++++++++++++++++++++++\n> 1 file changed, 41 insertions(+)\n> \n> diff --git a/src/libcamera/control_ids.yaml b/src/libcamera/control_ids.yaml\n> index 6d6f0fee..a58bff18 100644\n> --- a/src/libcamera/control_ids.yaml\n> +++ b/src/libcamera/control_ids.yaml\n> @@ -306,6 +306,47 @@ controls:\n> maximum valid value is given by the properties::ScalerCropMaximum\n> property, and the two can be used to implement digital zoom.\n> \n> + - FrameDurations:\n> + type: int64_t\n> + description: |\n> + The minimum and maximum (in that order) frame duration,\n> + expressed in micro-seconds.\n> +\n> + When provided by applications, the control specifies the sensor frame\n> + duration interval the pipeline has to use. This could also limit the\n> + largest exposure time the sensor can use. For example, if a maximum\n> + frame duration of 33ms is requested (corresponding to 30 frames per\n> + second), the sensor will not be able to raise the exposure time above\n> + 33ms. A fixed frame duration is achieved by setting the minimum and\n> + maximum values to be the same.\n> +\n> + The maximum frame duration provides the absolute limit to the shutter\n> + speed computed by the AE algorithm and it overrides any exposure mode\n> + setting specified with controls::AeExposureMode. Similarly, when a\n> + manual exposure time is set through controls::ExposureTime, it also\n> + gets clipped to the limits set by this control.\n> +\n> + \\sa AeExposureMode\n> + \\sa ExposureTime\n> +\n> + \\todo Refer to the frame duration limits property to describe how\n> + application-provided values gets clipped and reset.\n> +\n> + When reported by pipelines, the control expresses the minimum and\n> + maximum frame durations used after being clipped to what the current\n> + sensor mode supports, and what is achievable based on the exposure\n> + mode setting specified with controls::AeExposureMode or manual\n> + exposure time set through controls::ExposureTime. The sensor frame\n> + duration is one of the parameter that defines the capture frame rate\n\ns/parameter/parameters/ \n\n> + but it does not alone provide enough information to fully calculate it\n> + as it does not account for pipeline processing delays.\n> +\n> + \\todo Define how to calculate the capture frame rate by\n> + defining controls to report additional delays introduced by\n> + the capture pipeline or post-processing stages (ie JPEG\n> + conversion, frame scaling).\n\nPresumably, the frame rate becomes a function of\n max(FrameDuration, OtherSteps)\n\n\nReviewed-by: Kieran Bingham \n\n\n> + size: [2]\n> +\n> # ----------------------------------------------------------------------------\n> # Draft controls section\n> \n>","headers":{"Return-Path":"","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id DB4E1BD80A\n\tfor ;\n\tFri, 18 Dec 2020 12:44:17 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id 59FE661595;\n\tFri, 18 Dec 2020 13:44:17 +0100 (CET)","from perceval.ideasonboard.com (perceval.ideasonboard.com\n\t[IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id 231866052C\n\tfor ;\n\tFri, 18 Dec 2020 13:44:16 +0100 (CET)","from [192.168.0.20]\n\t(cpc89244-aztw30-2-0-cust3082.18-1.cable.virginm.net [86.31.172.11])\n\tby perceval.ideasonboard.com (Postfix) with ESMTPSA id 8754E2CF;\n\tFri, 18 Dec 2020 13:44:15 +0100 (CET)"],"Authentication-Results":"lancelot.ideasonboard.com;\n\tdkim=fail reason=\"signature verification failed\" (1024-bit key;\n\tunprotected) header.d=ideasonboard.com header.i=@ideasonboard.com\n\theader.b=\"u4hK2+q0\"; dkim-atps=neutral","DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com;\n\ts=mail; t=1608295455;\n\tbh=II1t3adIVuIAnEOBI0nXDQMbBMqXjT7rvQ9lctqnL4o=;\n\th=Reply-To:Subject:To:References:From:Date:In-Reply-To:From;\n\tb=u4hK2+q0u4riNnk9ZMpRgzoqu9MptgJG3xqUfvzgO/Co4i06Jf3dn+r1J8PlgrC0o\n\tg7lFsusJYZCn00WMcJicRPyTbPDIA1UXjgBBt38I1o1lDa4Cg0BWdOonJXvJ6xHwhb\n\t8n/SEBx/PTrXV5n0RxOPHGiKs7Xwk1QVirz+NYso=","To":"Naushir Patuck ,\n\tlibcamera-devel@lists.libcamera.org","References":"<20201218100626.205134-1-naush@raspberrypi.com>\n\t<20201218100626.205134-2-naush@raspberrypi.com>","From":"Kieran Bingham ","Autocrypt":"addr=kieran.bingham@ideasonboard.com; keydata=\n\tmQINBFYE/WYBEACs1PwjMD9rgCu1hlIiUA1AXR4rv2v+BCLUq//vrX5S5bjzxKAryRf0uHat\n\tV/zwz6hiDrZuHUACDB7X8OaQcwhLaVlq6byfoBr25+hbZG7G3+5EUl9cQ7dQEdvNj6V6y/SC\n\trRanWfelwQThCHckbobWiQJfK9n7rYNcPMq9B8e9F020LFH7Kj6YmO95ewJGgLm+idg1Kb3C\n\tpotzWkXc1xmPzcQ1fvQMOfMwdS+4SNw4rY9f07Xb2K99rjMwZVDgESKIzhsDB5GY465sCsiQ\n\tcSAZRxqE49RTBq2+EQsbrQpIc8XiffAB8qexh5/QPzCmR4kJgCGeHIXBtgRj+nIkCJPZvZtf\n\tKr2EAbc6tgg6DkAEHJb+1okosV09+0+TXywYvtEop/WUOWQ+zo+Y/OBd+8Ptgt1pDRyOBzL8\n\tRXa8ZqRf0Mwg75D+dKntZeJHzPRJyrlfQokngAAs4PaFt6UfS+ypMAF37T6CeDArQC41V3ko\n\tlPn1yMsVD0p+6i3DPvA/GPIksDC4owjnzVX9kM8Zc5Cx+XoAN0w5Eqo4t6qEVbuettxx55gq\n\t8K8FieAjgjMSxngo/HST8TpFeqI5nVeq0/lqtBRQKumuIqDg+Bkr4L1V/PSB6XgQcOdhtd36\n\tOe9X9dXB8YSNt7VjOcO7BTmFn/Z8r92mSAfHXpb07YJWJosQOQARAQABtDBLaWVyYW4gQmlu\n\tZ2hhbSA8a2llcmFuLmJpbmdoYW1AaWRlYXNvbmJvYXJkLmNvbT6JAlcEEwEKAEECGwMFCwkI\n\tBwIGFQgJCgsCBBYCAwECHgECF4ACGQEWIQSQLdeYP70o/eNy1HqhHkZyEKRh/QUCXWTtygUJ\n\tCyJXZAAKCRChHkZyEKRh/f8dEACTDsbLN2nioNZMwyLuQRUAFcXNolDX48xcUXsWS2QjxaPm\n\tVsJx8Uy8aYkS85mdPBh0C83OovQR/OVbr8AxhGvYqBs3nQvbWuTl/+4od7DfK2VZOoKBAu5S\n\tQK2FYuUcikDqYcFWJ8DQnubxfE8dvzojHEkXw0sA4igINHDDFX3HJGZtLio+WpEFQtCbfTAG\n\tYZslasz1YZRbwEdSsmO3/kqy5eMnczlm8a21A3fKUo3g8oAZEFM+f4DUNzqIltg31OAB/kZS\n\tenKZQ/SWC8PmLg/ZXBrReYakxXtkP6w3FwMlzOlhGxqhIRNiAJfXJBaRhuUWzPOpEDE9q5YJ\n\tBmqQL2WJm1VSNNVxbXJHpaWMH1sA2R00vmvRrPXGwyIO0IPYeUYQa3gsy6k+En/aMQJd27dp\n\taScf9am9PFICPY5T4ppneeJLif2lyLojo0mcHOV+uyrds9XkLpp14GfTkeKPdPMrLLTsHRfH\n\tfA4I4OBpRrEPiGIZB/0im98MkGY/Mu6qxeZmYLCcgD6qz4idOvfgVOrNh+aA8HzIVR+RMW8H\n\tQGBN9f0E3kfwxuhl3omo6V7lDw8XOdmuWZNC9zPq1UfryVHANYbLGz9KJ4Aw6M+OgBC2JpkD\n\thXMdHUkC+d20dwXrwHTlrJi1YNp6rBc+xald3wsUPOZ5z8moTHUX/uPA/qhGsbkCDQRWBP1m\n\tARAAzijkb+Sau4hAncr1JjOY+KyFEdUNxRy+hqTJdJfaYihxyaj0Ee0P0zEi35CbE6lgU0Uz\n\ttih9fiUbSV3wfsWqg1Ut3/5rTKu7kLFp15kF7eqvV4uezXRD3Qu4yjv/rMmEJbbD4cTvGCYI\n\td6MDC417f7vK3hCbCVIZSp3GXxyC1LU+UQr3fFcOyCwmP9vDUR9JV0BSqHHxRDdpUXE26Dk6\n\tmhf0V1YkspE5St814ETXpEus2urZE5yJIUROlWPIL+hm3NEWfAP06vsQUyLvr/GtbOT79vXl\n\tEn1aulcYyu20dRRxhkQ6iILaURcxIAVJJKPi8dsoMnS8pB0QW12AHWuirPF0g6DiuUfPmrA5\n\tPKe56IGlpkjc8cO51lIxHkWTpCMWigRdPDexKX+Sb+W9QWK/0JjIc4t3KBaiG8O4yRX8ml2R\n\t+rxfAVKM6V769P/hWoRGdgUMgYHFpHGSgEt80OKK5HeUPy2cngDUXzwrqiM5Sz6Od0qw5pCk\n\tNlXqI0W/who0iSVM+8+RmyY0OEkxEcci7rRLsGnM15B5PjLJjh1f2ULYkv8s4SnDwMZ/kE04\n\t/UqCMK/KnX8pwXEMCjz0h6qWNpGwJ0/tYIgQJZh6bqkvBrDogAvuhf60Sogw+mH8b+PBlx1L\n\toeTK396wc+4c3BfiC6pNtUS5GpsPMMjYMk7kVvEAEQEAAYkCPAQYAQoAJgIbDBYhBJAt15g/\n\tvSj943LUeqEeRnIQpGH9BQJdizzIBQkLSKZiAAoJEKEeRnIQpGH9eYgQAJpjaWNgqNOnMTmD\n\tMJggbwjIotypzIXfhHNCeTkG7+qCDlSaBPclcPGYrTwCt0YWPU2TgGgJrVhYT20ierN8LUvj\n\t6qOPTd+Uk7NFzL65qkh80ZKNBFddx1AabQpSVQKbdcLb8OFs85kuSvFdgqZwgxA1vl4TFhNz\n\tPZ79NAmXLackAx3sOVFhk4WQaKRshCB7cSl+RIng5S/ThOBlwNlcKG7j7W2MC06BlTbdEkUp\n\tECzuuRBv8wX4OQl+hbWbB/VKIx5HKlLu1eypen/5lNVzSqMMIYkkZcjV2SWQyUGxSwq0O/sx\n\tS0A8/atCHUXOboUsn54qdxrVDaK+6jIAuo8JiRWctP16KjzUM7MO0/+4zllM8EY57rXrj48j\n\tsbEYX0YQnzaj+jO6kJtoZsIaYR7rMMq9aUAjyiaEZpmP1qF/2sYenDx0Fg2BSlLvLvXM0vU8\n\tpQk3kgDu7kb/7PRYrZvBsr21EIQoIjXbZxDz/o7z95frkP71EaICttZ6k9q5oxxA5WC6sTXc\n\tMW8zs8avFNuA9VpXt0YupJd2ijtZy2mpZNG02fFVXhIn4G807G7+9mhuC4XG5rKlBBUXTvPU\n\tAfYnB4JBDLmLzBFavQfvonSfbitgXwCG3vS+9HEwAjU30Bar1PEOmIbiAoMzuKeRm2LVpmq4\n\tWZw01QYHU/GUV/zHJSFk","Organization":"Ideas on Board","Message-ID":"<13758e44-f364-1acd-d4ef-f21754f2e220@ideasonboard.com>","Date":"Fri, 18 Dec 2020 12:44:13 +0000","User-Agent":"Mozilla/5.0 (X11; Linux x86_64; rv:68.0) Gecko/20100101\n\tThunderbird/68.10.0","MIME-Version":"1.0","In-Reply-To":"<20201218100626.205134-2-naush@raspberrypi.com>","Content-Language":"en-GB","Subject":"Re: [libcamera-devel] [PATCH v10 1/3] libcamera: controls: Add\n\tframe duration control","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Reply-To":"kieran.bingham@ideasonboard.com","Content-Type":"text/plain; charset=\"us-ascii\"","Content-Transfer-Encoding":"7bit","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" "}},{"id":14274,"web_url":"https://patchwork.libcamera.org/comment/14274/","msgid":"","date":"2020-12-18T13:12:24","subject":"Re: [libcamera-devel] [PATCH v10 1/3] libcamera: controls: Add\n\tframe duration control","submitter":{"id":34,"url":"https://patchwork.libcamera.org/api/people/34/","name":"Naushir Patuck","email":"naush@raspberrypi.com"},"content":"Hi Kieran,\n\nThank you for the review.\n\nOn Fri, 18 Dec 2020 at 12:44, Kieran Bingham <\nkieran.bingham@ideasonboard.com> wrote:\n\n> Hi Naush,\n>\n> On 18/12/2020 10:06, Naushir Patuck wrote:\n> > Add an int64_t array control (controls::FrameDurations) to specify the\n> > minimum and maximum (in that order) frame duration to be used by the\n> > camera sensor.\n> >\n> > Signed-off-by: Naushir Patuck \n> > Reviewed-by: David Plowman \n> > Tested-by: David Plowman \n> > Reviewed-by: Jacopo Mondi \n> > ---\n> > src/libcamera/control_ids.yaml | 41 ++++++++++++++++++++++++++++++++++\n> > 1 file changed, 41 insertions(+)\n> >\n> > diff --git a/src/libcamera/control_ids.yaml\n> b/src/libcamera/control_ids.yaml\n> > index 6d6f0fee..a58bff18 100644\n> > --- a/src/libcamera/control_ids.yaml\n> > +++ b/src/libcamera/control_ids.yaml\n> > @@ -306,6 +306,47 @@ controls:\n> > maximum valid value is given by the\n> properties::ScalerCropMaximum\n> > property, and the two can be used to implement digital zoom.\n> >\n> > + - FrameDurations:\n> > + type: int64_t\n> > + description: |\n> > + The minimum and maximum (in that order) frame duration,\n> > + expressed in micro-seconds.\n> > +\n> > + When provided by applications, the control specifies the\n> sensor frame\n> > + duration interval the pipeline has to use. This could also\n> limit the\n> > + largest exposure time the sensor can use. For example, if a\n> maximum\n> > + frame duration of 33ms is requested (corresponding to 30\n> frames per\n> > + second), the sensor will not be able to raise the exposure\n> time above\n> > + 33ms. A fixed frame duration is achieved by setting the\n> minimum and\n> > + maximum values to be the same.\n> > +\n> > + The maximum frame duration provides the absolute limit to the\n> shutter\n> > + speed computed by the AE algorithm and it overrides any\n> exposure mode\n> > + setting specified with controls::AeExposureMode. Similarly,\n> when a\n> > + manual exposure time is set through controls::ExposureTime,\n> it also\n> > + gets clipped to the limits set by this control.\n> > +\n> > + \\sa AeExposureMode\n> > + \\sa ExposureTime\n> > +\n> > + \\todo Refer to the frame duration limits property to describe\n> how\n> > + application-provided values gets clipped and reset.\n> > +\n> > + When reported by pipelines, the control expresses the minimum\n> and\n> > + maximum frame durations used after being clipped to what the\n> current\n> > + sensor mode supports, and what is achievable based on the\n> exposure\n> > + mode setting specified with controls::AeExposureMode or manual\n> > + exposure time set through controls::ExposureTime. The sensor\n> frame\n> > + duration is one of the parameter that defines the capture\n> frame rate\n>\n> s/parameter/parameters/ \n>\n> > + but it does not alone provide enough information to fully\n> calculate it\n> > + as it does not account for pipeline processing delays.\n> > +\n> > + \\todo Define how to calculate the capture frame rate by\n> > + defining controls to report additional delays introduced by\n> > + the capture pipeline or post-processing stages (ie JPEG\n> > + conversion, frame scaling).\n>\n> Presumably, the frame rate becomes a function of\n> max(FrameDuration, OtherSteps)\n>\n\nThat's right. The tricky bit is fully defining OtherSteps :-)\n\nRegards,\nNaush\n\n\n\n>\n>\n> Reviewed-by: Kieran Bingham \n>\n>\n> > + size: [2]\n> > +\n> > #\n> ----------------------------------------------------------------------------\n> > # Draft controls section\n> >\n> >\n>\n> --\n> Regards\n> --\n> Kieran\n>","headers":{"Return-Path":"","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id 5111DC0F1A\n\tfor ;\n\tFri, 18 Dec 2020 13:12:44 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id D924361593;\n\tFri, 18 Dec 2020 14:12:43 +0100 (CET)","from mail-lf1-x134.google.com (mail-lf1-x134.google.com\n\t[IPv6:2a00:1450:4864:20::134])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id D84F76052C\n\tfor ;\n\tFri, 18 Dec 2020 14:12:41 +0100 (CET)","by mail-lf1-x134.google.com with SMTP id y19so5192654lfa.13\n\tfor ;\n\tFri, 18 Dec 2020 05:12:41 -0800 (PST)"],"Authentication-Results":"lancelot.ideasonboard.com;\n\tdkim=fail reason=\"signature verification failed\" (2048-bit key;\n\tunprotected) header.d=raspberrypi.com header.i=@raspberrypi.com\n\theader.b=\"SzbTfkrg\"; dkim-atps=neutral","DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed;\n\td=raspberrypi.com; s=google;\n\th=mime-version:references:in-reply-to:from:date:message-id:subject:to\n\t:cc; bh=VzAoZq1NCQ3F+euzffT94/znc5NXv9aZX1p4KeTos1E=;\n\tb=SzbTfkrgNBiCwxRPMABS9ISKIk5Ghk/iIKcjUWky4xkDo6DeBtNBf9RXT0VAgnStUj\n\tcL6yY1ZpwBHbglhGFd0kkEUioS4uzG2PAZzL1kltU+rgiRnZA750xNgPwA/J0SWLU3OH\n\taDj+Vr26+UEFnYznkRrHOHt++k2PXIqWndzL5eajILTS9Cti1ugQlV66IZUzjBbxwh/b\n\tY+1lMtgzNwy64dMocU6VtTUDI4oGlvYrrRHN5xUgMCXflf+VeLXYghcK1LRjy5Zn+Wbx\n\tfb7jj49HAt0cmnPPCWFBlBo+3HtxQqJRPWlYoQKWNqPvt0mqH+fbjxdZ7Fa+pp7gko93\n\tMbTA==","X-Google-DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed;\n\td=1e100.net; s=20161025;\n\th=x-gm-message-state:mime-version:references:in-reply-to:from:date\n\t:message-id:subject:to:cc;\n\tbh=VzAoZq1NCQ3F+euzffT94/znc5NXv9aZX1p4KeTos1E=;\n\tb=moYkJEiWp8mR2rD6PdkIiCKbZn88bEdZmMmiUsdx1Tch+ku/wRp0bXEKv8zk948SkS\n\tO7cmlzZweV6eJNK5jJWghTwA8sMn6uK7NMdYqgpcxeirbCCFfpxc0PZekxOWz82vu5Ud\n\tsltm/M9FfiYLt1kd22hpxGVjO8COudpasWFA4hVzRCautwglvAdDFaUrR5tQAdO5hJfp\n\tJNFGK/TTA3ELk6/OZd2iCwXwRRPy2QwxgQKaNXUEQjFoW21lFPPsn8NJzmCrnG/e0ND3\n\tEaxfYFcNTEEVw2bwFCBWcnJIcA21NawWZxfuU/Jzeuuv2vQfgxz3RZuc/cyBBf0Knj2Y\n\tNcjg==","X-Gm-Message-State":"AOAM531oTsGFpw+pjW9RwyfinstzdN2eOb2XWUtPAV0eifDzeW0F3IrB\n\t47gouIFVc0MPjIj9qco1F+6kN9xpm/ZCT07AJ3+P9Q==","X-Google-Smtp-Source":"ABdhPJy/eWuMEwXOJ4/37UpvyOLMRcKQQLW2LP7BieSuyRQxTypIpkpXqrMSKQcF7MuOa+GbB3KZdNlXvpTOFXGgRdc=","X-Received":"by 2002:a05:651c:503:: with SMTP id\n\to3mr1742027ljp.253.1608297160889; \n\tFri, 18 Dec 2020 05:12:40 -0800 (PST)","MIME-Version":"1.0","References":"<20201218100626.205134-1-naush@raspberrypi.com>\n\t<20201218100626.205134-2-naush@raspberrypi.com>\n\t<13758e44-f364-1acd-d4ef-f21754f2e220@ideasonboard.com>","In-Reply-To":"<13758e44-f364-1acd-d4ef-f21754f2e220@ideasonboard.com>","From":"Naushir Patuck ","Date":"Fri, 18 Dec 2020 13:12:24 +0000","Message-ID":"","To":"Kieran Bingham ","Subject":"Re: [libcamera-devel] [PATCH v10 1/3] libcamera: controls: Add\n\tframe duration control","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Cc":"libcamera devel ","Content-Type":"multipart/mixed;\n\tboundary=\"===============1087745593793156093==\"","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" "}},{"id":14290,"web_url":"https://patchwork.libcamera.org/comment/14290/","msgid":"","date":"2020-12-20T05:38:29","subject":"Re: [libcamera-devel] [PATCH v10 1/3] libcamera: controls: Add\n\tframe duration control","submitter":{"id":2,"url":"https://patchwork.libcamera.org/api/people/2/","name":"Laurent Pinchart","email":"laurent.pinchart@ideasonboard.com"},"content":"Hi Naush,\n\nThank you for the patch.\n\nOn Fri, Dec 18, 2020 at 10:06:25AM +0000, Naushir Patuck wrote:\n> Add an int64_t array control (controls::FrameDurations) to specify the\n> minimum and maximum (in that order) frame duration to be used by the\n> camera sensor.\n> \n> Signed-off-by: Naushir Patuck \n> Reviewed-by: David Plowman \n> Tested-by: David Plowman \n> Reviewed-by: Jacopo Mondi \n> ---\n> src/libcamera/control_ids.yaml | 41 ++++++++++++++++++++++++++++++++++\n> 1 file changed, 41 insertions(+)\n> \n> diff --git a/src/libcamera/control_ids.yaml b/src/libcamera/control_ids.yaml\n> index 6d6f0fee..a58bff18 100644\n> --- a/src/libcamera/control_ids.yaml\n> +++ b/src/libcamera/control_ids.yaml\n> @@ -306,6 +306,47 @@ controls:\n> maximum valid value is given by the properties::ScalerCropMaximum\n> property, and the two can be used to implement digital zoom.\n> \n> + - FrameDurations:\n> + type: int64_t\n> + description: |\n> + The minimum and maximum (in that order) frame duration,\n> + expressed in micro-seconds.\n> +\n> + When provided by applications, the control specifies the sensor frame\n> + duration interval the pipeline has to use. This could also limit the\n\ns/could also limit/limits/ ?\n\n> + largest exposure time the sensor can use. For example, if a maximum\n> + frame duration of 33ms is requested (corresponding to 30 frames per\n> + second), the sensor will not be able to raise the exposure time above\n> + 33ms. A fixed frame duration is achieved by setting the minimum and\n> + maximum values to be the same.\n> +\n> + The maximum frame duration provides the absolute limit to the shutter\n> + speed computed by the AE algorithm and it overrides any exposure mode\n\nI think we should talk about exposure time instead of shutter speed, but\nwe'll need to go through the whole documentation and code base to ensure\nconsistency, so this can be done later.\n\n> + setting specified with controls::AeExposureMode. Similarly, when a\n> + manual exposure time is set through controls::ExposureTime, it also\n> + gets clipped to the limits set by this control.\n> +\n> + \\sa AeExposureMode\n> + \\sa ExposureTime\n> +\n> + \\todo Refer to the frame duration limits property to describe how\n> + application-provided values gets clipped and reset.\n> +\n> + When reported by pipelines, the control expresses the minimum and\n\nIs this about Request::metadata(), or Camera::properties() ? I assume\nthe former as it's defined in control_ids.yaml.\n\n> + maximum frame durations used after being clipped to what the current\n> + sensor mode supports, and what is achievable based on the exposure\n> + mode setting specified with controls::AeExposureMode or manual\n> + exposure time set through controls::ExposureTime. The sensor frame\n\nI have a hard time parsing this, and a hard time seeing how it should be\nused by applications :-S Do you need this for your use cases ?\n\n> + duration is one of the parameter that defines the capture frame rate\n> + but it does not alone provide enough information to fully calculate it\n> + as it does not account for pipeline processing delays.\n\nDelays in the pipeline don't affect the frame duration, do they ? We\ncould drop frames when processing streams in software (for instance for\nJPEG compression), but that's out of scope as libcamera doesn't support\nthat.\n\n> +\n> + \\todo Define how to calculate the capture frame rate by\n> + defining controls to report additional delays introduced by\n> + the capture pipeline or post-processing stages (ie JPEG\n> + conversion, frame scaling).\n> + size: [2]\n> +\n> # ----------------------------------------------------------------------------\n> # Draft controls section\n>","headers":{"Return-Path":"","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id A9CB5C0F1B\n\tfor ;\n\tSun, 20 Dec 2020 05:38:39 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id C37B561595;\n\tSun, 20 Dec 2020 06:38:38 +0100 (CET)","from perceval.ideasonboard.com (perceval.ideasonboard.com\n\t[IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id 4D17260526\n\tfor ;\n\tSun, 20 Dec 2020 06:38:37 +0100 (CET)","from pendragon.ideasonboard.com (62-78-145-57.bb.dnainternet.fi\n\t[62.78.145.57])\n\tby perceval.ideasonboard.com (Postfix) with ESMTPSA id ADD4F593;\n\tSun, 20 Dec 2020 06:38:36 +0100 (CET)"],"Authentication-Results":"lancelot.ideasonboard.com;\n\tdkim=fail reason=\"signature verification failed\" (1024-bit key;\n\tunprotected) header.d=ideasonboard.com header.i=@ideasonboard.com\n\theader.b=\"R+cs/hHQ\"; dkim-atps=neutral","DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com;\n\ts=mail; t=1608442716;\n\tbh=A25aV2bBqIpAm6FL4P5cihs3RTMiq2X9LW69CF37djU=;\n\th=Date:From:To:Cc:Subject:References:In-Reply-To:From;\n\tb=R+cs/hHQMpKerWwb+zDYSn3V8/FwL3Uz+QdvKdDgXoSBNUe4p8aq8QHX+FAQsxXbb\n\tsRdpyjhhYflA/aMxHBq5BXmA0cV6tBnGcsZbaLcGWn31y3Dk80TXlqhxojPjdwOiAh\n\tgSX1PCG80Xp3RIoLhwnB9ZcyFpMQN7soSGZcZfNM=","Date":"Sun, 20 Dec 2020 07:38:29 +0200","From":"Laurent Pinchart ","To":"Naushir Patuck ","Message-ID":"","References":"<20201218100626.205134-1-naush@raspberrypi.com>\n\t<20201218100626.205134-2-naush@raspberrypi.com>","MIME-Version":"1.0","Content-Disposition":"inline","In-Reply-To":"<20201218100626.205134-2-naush@raspberrypi.com>","Subject":"Re: [libcamera-devel] [PATCH v10 1/3] libcamera: controls: Add\n\tframe duration control","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Cc":"libcamera-devel@lists.libcamera.org","Content-Type":"text/plain; charset=\"us-ascii\"","Content-Transfer-Encoding":"7bit","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" "}},{"id":14294,"web_url":"https://patchwork.libcamera.org/comment/14294/","msgid":"","date":"2020-12-21T11:11:59","subject":"Re: [libcamera-devel] [PATCH v10 1/3] libcamera: controls: Add\n\tframe duration control","submitter":{"id":34,"url":"https://patchwork.libcamera.org/api/people/34/","name":"Naushir Patuck","email":"naush@raspberrypi.com"},"content":"Hi Laurent,\n\nThank you for the review feedback.\n\nOn Sun, 20 Dec 2020 at 05:38, Laurent Pinchart <\nlaurent.pinchart@ideasonboard.com> wrote:\n\n> Hi Naush,\n>\n> Thank you for the patch.\n>\n> On Fri, Dec 18, 2020 at 10:06:25AM +0000, Naushir Patuck wrote:\n> > Add an int64_t array control (controls::FrameDurations) to specify the\n> > minimum and maximum (in that order) frame duration to be used by the\n> > camera sensor.\n> >\n> > Signed-off-by: Naushir Patuck \n> > Reviewed-by: David Plowman \n> > Tested-by: David Plowman \n> > Reviewed-by: Jacopo Mondi \n> > ---\n> > src/libcamera/control_ids.yaml | 41 ++++++++++++++++++++++++++++++++++\n> > 1 file changed, 41 insertions(+)\n> >\n> > diff --git a/src/libcamera/control_ids.yaml\n> b/src/libcamera/control_ids.yaml\n> > index 6d6f0fee..a58bff18 100644\n> > --- a/src/libcamera/control_ids.yaml\n> > +++ b/src/libcamera/control_ids.yaml\n> > @@ -306,6 +306,47 @@ controls:\n> > maximum valid value is given by the\n> properties::ScalerCropMaximum\n> > property, and the two can be used to implement digital zoom.\n> >\n> > + - FrameDurations:\n> > + type: int64_t\n> > + description: |\n> > + The minimum and maximum (in that order) frame duration,\n> > + expressed in micro-seconds.\n> > +\n> > + When provided by applications, the control specifies the\n> sensor frame\n> > + duration interval the pipeline has to use. This could also\n> limit the\n>\n> s/could also limit/limits/ ?\n>\n\nAck\n\n\n>\n> > + largest exposure time the sensor can use. For example, if a\n> maximum\n> > + frame duration of 33ms is requested (corresponding to 30\n> frames per\n> > + second), the sensor will not be able to raise the exposure\n> time above\n> > + 33ms. A fixed frame duration is achieved by setting the\n> minimum and\n> > + maximum values to be the same.\n> > +\n> > + The maximum frame duration provides the absolute limit to the\n> shutter\n> > + speed computed by the AE algorithm and it overrides any\n> exposure mode\n>\n> I think we should talk about exposure time instead of shutter speed, but\n> we'll need to go through the whole documentation and code base to ensure\n> consistency, so this can be done later.\n>\n> > + setting specified with controls::AeExposureMode. Similarly,\n> when a\n> > + manual exposure time is set through controls::ExposureTime,\n> it also\n> > + gets clipped to the limits set by this control.\n> > +\n> > + \\sa AeExposureMode\n> > + \\sa ExposureTime\n> > +\n> > + \\todo Refer to the frame duration limits property to describe\n> how\n> > + application-provided values gets clipped and reset.\n> > +\n> > + When reported by pipelines, the control expresses the minimum\n> and\n>\n> Is this about Request::metadata(), or Camera::properties() ? I assume\n> the former as it's defined in control_ids.yaml.\n>\n\nThis is through Request::metadata(). Should I perhaps reword to make this\nmore explicit?\n\n\"When reported by pipelines through metadata, the control expresses....\" ?\n\n\n>\n> > + maximum frame durations used after being clipped to what the\n> current\n> > + sensor mode supports, and what is achievable based on the\n> exposure\n> > + mode setting specified with controls::AeExposureMode or manual\n> > + exposure time set through controls::ExposureTime. The sensor\n> frame\n>\n> I have a hard time parsing this, and a hard time seeing how it should be\n> used by applications :-S Do you need this for your use cases ?\n>\n\nYes, this was also a point of confusion between Jacopo and me :-)\n\nWhat I was trying to convey is that the values returned through the\nmetadata is the actual limits that will be used (after being constrained by\nthe exposure mode and/or shutter speed), and may not be the same as what\nthe application requested. I thought this might be useful for the\napplication to know the actual limits used. If you think this may not be\nneeded, I could remove this control from the metadata?\n\n\n>\n> > + duration is one of the parameter that defines the capture\n> frame rate\n> > + but it does not alone provide enough information to fully\n> calculate it\n> > + as it does not account for pipeline processing delays.\n>\n> Delays in the pipeline don't affect the frame duration, do they ? We\n> could drop frames when processing streams in software (for instance for\n> JPEG compression), but that's out of scope as libcamera doesn't support\n> that.\n>\n\nIf you have, e.g. a slow software denoise running in the IPA whose\nprocessing time limits your frame throughput, it could be limiting the\noverall pipeline framerate (increasing frame duration). Do you think\nperhaps this should not be mentioned here to avoid complicating things for\nnow?\n\n\n>\n> > +\n> > + \\todo Define how to calculate the capture frame rate by\n> > + defining controls to report additional delays introduced by\n> > + the capture pipeline or post-processing stages (ie JPEG\n> > + conversion, frame scaling).\n> > + size: [2]\n> > +\n> > #\n> ----------------------------------------------------------------------------\n> > # Draft controls section\n> >\n>\n> --\n> Regards,\n>\n> Laurent Pinchart\n>","headers":{"Return-Path":"","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id C6ECBC0F1A\n\tfor ;\n\tMon, 21 Dec 2020 11:12:19 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id EEC4161595;\n\tMon, 21 Dec 2020 12:12:18 +0100 (CET)","from mail-lf1-x12f.google.com (mail-lf1-x12f.google.com\n\t[IPv6:2a00:1450:4864:20::12f])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id 0111461592\n\tfor ;\n\tMon, 21 Dec 2020 12:12:16 +0100 (CET)","by mail-lf1-x12f.google.com with SMTP id o19so22787435lfo.1\n\tfor ;\n\tMon, 21 Dec 2020 03:12:16 -0800 (PST)"],"Authentication-Results":"lancelot.ideasonboard.com;\n\tdkim=fail reason=\"signature verification failed\" (2048-bit key;\n\tunprotected) header.d=raspberrypi.com header.i=@raspberrypi.com\n\theader.b=\"N4vYbcIX\"; dkim-atps=neutral","DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed;\n\td=raspberrypi.com; s=google;\n\th=mime-version:references:in-reply-to:from:date:message-id:subject:to\n\t:cc; bh=L+oc7HeFfv7fOCgTiM2jANrd6cjl5nWD2hbKHz+uO/8=;\n\tb=N4vYbcIXb6kQ1kDv434Boxm9qtmCbUz+S+kuR1rHaSP/DYomMhRWEX/WmKgduc8cVU\n\tbftrkS2zclBFzKKnkQD19xhmgB95bolPvUfUh1+6NuZbbaJ/rnxMHSIm3BFdHG6AIrI2\n\t/shw2RanQfjtgnXO/056xZqNGNAasUxH1kqut154FYIy/5UWCpZ2qViKxOSE+L2YRRfw\n\tTOU25saua9l7+lKck7SeuK9jr88RrcLEinqwtASiAYqG3iWYkN4kY98ZlELa8hh6xv83\n\tw9qNJvJ51DNY5t52mxaYbu8vp7s0cK/9FVewzXLwXMTQXvPNc5qn7yDE1L7g0Oo1EGIG\n\tASyg==","X-Google-DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed;\n\td=1e100.net; s=20161025;\n\th=x-gm-message-state:mime-version:references:in-reply-to:from:date\n\t:message-id:subject:to:cc;\n\tbh=L+oc7HeFfv7fOCgTiM2jANrd6cjl5nWD2hbKHz+uO/8=;\n\tb=tRZe37YvibeUy+kgzSkcuQoZ2UV+rv7rE01R8hbaTxe5ibb3acxdaCLGerzcZmOBMG\n\tWL89S2AOh8UWrq+m2Wxf6Dh4Yj9nsWeYXP5hyBIFxLUQFy8LiDAjbxYIHJjfwmqWuAa6\n\tx8Ql3zI2N/6WrcAkrP+/n+fxuaA+vP6Xz3IHKBTLOvoMQwlp5qCZnQ0POWO3Hr6h9Ixz\n\tJxOEVc/YiLco1nVQGd3+LQG7cZy7WaTHW40Ea7YKoYDoKccx++PKaKV0c46IHT+kM+gR\n\tUanq5L7NzEWQuzIiYHnc1hp3XgbzMjt624LhoaZEQEy0ZFz03KVjqsNcG5SLas8MI0A3\n\teSvQ==","X-Gm-Message-State":"AOAM531uhU9xhcQ25I8QP5HRZ7N4NU6ZHPVomGjgl2QwM7MDwycnjOBO\n\tEhkDkIKbkr7TXHEDUVPa9s2VEvFp4gN0Uw1qhvhNWJwAQ5Q=","X-Google-Smtp-Source":"ABdhPJyQs5QF4npKuHuwXTLhM0cxDyNdRJVJ7k28y13Jx03OHoFP0s+1tdBw/DBTjUKeDVrJ6SNL05WkDSv3lnWLP8E=","X-Received":"by 2002:a19:7f90:: with SMTP id\n\ta138mr6523067lfd.617.1608549136005; \n\tMon, 21 Dec 2020 03:12:16 -0800 (PST)","MIME-Version":"1.0","References":"<20201218100626.205134-1-naush@raspberrypi.com>\n\t<20201218100626.205134-2-naush@raspberrypi.com>\n\t","In-Reply-To":"","From":"Naushir Patuck ","Date":"Mon, 21 Dec 2020 11:11:59 +0000","Message-ID":"","To":"Laurent Pinchart ","Subject":"Re: [libcamera-devel] [PATCH v10 1/3] libcamera: controls: Add\n\tframe duration control","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Cc":"libcamera devel ","Content-Type":"multipart/mixed;\n\tboundary=\"===============0068216470270217406==\"","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" "}},{"id":14297,"web_url":"https://patchwork.libcamera.org/comment/14297/","msgid":"","date":"2020-12-21T11:35:12","subject":"Re: [libcamera-devel] [PATCH v10 1/3] libcamera: controls: Add\n\tframe duration control","submitter":{"id":25,"url":"https://patchwork.libcamera.org/api/people/25/","name":"Andrey Konovalov","email":"andrey.konovalov@linaro.org"},"content":"Hi Naushir,\n\nOn 21.12.2020 14:11, Naushir Patuck wrote:\n> Hi Laurent,\n> \n> Thank you for the review feedback.\n> \n> On Sun, 20 Dec 2020 at 05:38, Laurent Pinchart > wrote:\n> \n> Hi Naush,\n> \n> Thank you for the patch.\n> \n> On Fri, Dec 18, 2020 at 10:06:25AM +0000, Naushir Patuck wrote:\n> > Add an int64_t array control (controls::FrameDurations) to specify the\n> > minimum and maximum (in that order) frame duration to be used by the\n> > camera sensor.\n> >\n> > Signed-off-by: Naushir Patuck >\n> > Reviewed-by: David Plowman >\n> > Tested-by: David Plowman >\n> > Reviewed-by: Jacopo Mondi >\n> > ---\n> >  src/libcamera/control_ids.yaml | 41 ++++++++++++++++++++++++++++++++++\n> >  1 file changed, 41 insertions(+)\n> >\n> > diff --git a/src/libcamera/control_ids.yaml b/src/libcamera/control_ids.yaml\n> > index 6d6f0fee..a58bff18 100644\n> > --- a/src/libcamera/control_ids.yaml\n> > +++ b/src/libcamera/control_ids.yaml\n> > @@ -306,6 +306,47 @@ controls:\n> >          maximum valid value is given by the properties::ScalerCropMaximum\n> >          property, and the two can be used to implement digital zoom.\n> >\n> > +  - FrameDurations:\n> > +      type: int64_t\n> > +      description: |\n> > +          The minimum and maximum (in that order) frame duration,\n> > +          expressed in micro-seconds.\n> > +\n> > +          When provided by applications, the control specifies the sensor frame\n> > +          duration interval the pipeline has to use. This could also limit the\n> \n> s/could also limit/limits/ ?\n> \n> \n> Ack\n> \n> \n> > +          largest exposure time the sensor can use. For example, if a maximum\n> > +          frame duration of 33ms is requested (corresponding to 30 frames per\n> > +          second), the sensor will not be able to raise the exposure time above\n> > +          33ms. A fixed frame duration is achieved by setting the minimum and\n> > +          maximum values to be the same.\n> > +\n> > +          The maximum frame duration provides the absolute limit to the shutter\n> > +          speed computed by the AE algorithm and it overrides any exposure mode\n> \n> I think we should talk about exposure time instead of shutter speed, but\n> we'll need to go through the whole documentation and code base to ensure\n> consistency, so this can be done later.\n> \n> > +          setting specified with controls::AeExposureMode. Similarly, when a\n> > +          manual exposure time is set through controls::ExposureTime, it also\n> > +          gets clipped to the limits set by this control.\n> > +\n> > +          \\sa AeExposureMode\n> > +          \\sa ExposureTime\n> > +\n> > +          \\todo Refer to the frame duration limits property to describe how\n> > +          application-provided values gets clipped and reset.\n> > +\n> > +          When reported by pipelines, the control expresses the minimum and\n> \n> Is this about Request::metadata(), or Camera::properties() ? I assume\n> the former as it's defined in control_ids.yaml.\n> \n> \n> This is through Request::metadata().  Should I perhaps reword to make this more explicit?\n> \n> \"When reported by pipelines through metadata, the control expresses....\" ?\n> \n> \n> > +          maximum frame durations used after being clipped to what the current\n> > +          sensor mode supports, and what is achievable based on the exposure\n> > +          mode setting specified with controls::AeExposureMode or manual\n> > +          exposure time set through controls::ExposureTime. The sensor frame\n> \n> I have a hard time parsing this, and a hard time seeing how it should be\n> used by applications :-S Do you need this for your use cases ?\n> \n> \n> Yes, this was also a point of confusion between Jacopo and me :-)\n> \n> What I was trying to convey is that the values returned through the metadata is the actual limits that will be used (after being \n> constrained by the exposure mode and/or shutter speed), and may not be the same as what the application requested.  I thought \n> this might be useful for the application to know the actual limits used.  If you think this may not be needed, I could remove \n> this control from the metadata?\n> \n> \n> > +          duration is one of the parameter that defines the capture frame rate\n> > +          but it does not alone provide enough information to fully calculate it\n> > +          as it does not account for pipeline processing delays.\n> \n> Delays in the pipeline don't affect the frame duration, do they ? We\n> could drop frames when processing streams in software (for instance for\n> JPEG compression), but that's out of scope as libcamera doesn't support\n> that.\n> \n> \n> If you have, e.g. a slow software denoise running in the IPA whose processing time limits your frame throughput, it could be \n> limiting the overall pipeline framerate (increasing frame duration).  Do you think perhaps this should not be mentioned here to \n> avoid complicating things for now?\n\nThe \"capture frame rate\" (vs sensor frame rate or sensor frame duration) is just fine.\n\"overall pipeline frame rate\" is also clear.\nBut \"pipeline frame duration\" (as the control is \"frame duration\", not \"frame interval\" or \"frame rate\") does\nsound confusing to me.\n\nThanks,\nAndrey\n\n> > +\n> > +          \\todo Define how to calculate the capture frame rate by\n> > +          defining controls to report additional delays introduced by\n> > +          the capture pipeline or post-processing stages (ie JPEG\n> > +          conversion, frame scaling).\n> > +      size: [2]\n> > +\n> >    # ----------------------------------------------------------------------------\n> >    # Draft controls section\n> >\n> \n> -- \n> Regards,\n> \n> Laurent Pinchart\n> \n> \n> _______________________________________________\n> libcamera-devel mailing list\n> libcamera-devel@lists.libcamera.org\n> https://lists.libcamera.org/listinfo/libcamera-devel\n>","headers":{"Return-Path":"","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id AD96AC0F1A\n\tfor ;\n\tMon, 21 Dec 2020 11:35:17 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id 3C8E06159A;\n\tMon, 21 Dec 2020 12:35:17 +0100 (CET)","from mail-lf1-x136.google.com (mail-lf1-x136.google.com\n\t[IPv6:2a00:1450:4864:20::136])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id EAA4261595\n\tfor ;\n\tMon, 21 Dec 2020 12:35:15 +0100 (CET)","by mail-lf1-x136.google.com with SMTP id h205so22857703lfd.5\n\tfor ;\n\tMon, 21 Dec 2020 03:35:15 -0800 (PST)","from [192.168.118.216] ([85.249.43.69])\n\tby smtp.gmail.com with ESMTPSA id\n\tj21sm1998411lfe.83.2020.12.21.03.35.14\n\t(version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);\n\tMon, 21 Dec 2020 03:35:14 -0800 (PST)"],"Authentication-Results":"lancelot.ideasonboard.com;\n\tdkim=fail reason=\"signature verification failed\" (2048-bit key;\n\tunprotected) header.d=linaro.org header.i=@linaro.org\n\theader.b=\"YFD4hHnF\"; dkim-atps=neutral","DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google;\n\th=subject:to:cc:references:from:message-id:date:user-agent\n\t:mime-version:in-reply-to:content-language:content-transfer-encoding; \n\tbh=cCF25fVvims1oIMOFKEI8XZCahSxT1+7yGK8+/I+2HM=;\n\tb=YFD4hHnFVf8TegwVgoa8eGHEb5wdN8mUGKjhCZoqvHDvOffOoJubC3Il+RwaEAsIOV\n\tNqSc8xwN4yG8XelxBIKB3E0Pp7nTj8JeXxJLCh66CXqWkWrPCrN/HJTSfJ0CZ6OgT0vA\n\tC7tLJtRLcs3GNOExKbaZbQt6bAVjG5hfrNbeAlxVtFCf8/zsx2Tst+dXhpVSylWBA1Ye\n\tIu5GljuvZw6E/bF0rf1qt4HFDFrTnd3ENS7+VBytlwZGufhKdsH5EUFfCAGTIn0TzTkg\n\totq7u1QKDUM3VSihMkWTWclFiJMSmXt6CFjh8RYGeqQ+3uyALd7q9KrEdWk5I+YBmvoD\n\tFyew==","X-Google-DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed;\n\td=1e100.net; s=20161025;\n\th=x-gm-message-state:subject:to:cc:references:from:message-id:date\n\t:user-agent:mime-version:in-reply-to:content-language\n\t:content-transfer-encoding;\n\tbh=cCF25fVvims1oIMOFKEI8XZCahSxT1+7yGK8+/I+2HM=;\n\tb=bMxUyyzPvbyN0EN/LJVbYX4HjCuGobnul/ErWsO4r0U2vBT8ZmRcH/8WsVm/Pi7N+G\n\tiD3f0CLLxDsWhJ+5G7rj8ZZmt0DUgs5wsLKgPjiRUqjQHnQ5qTgAjAaD6sudFJwLd0P3\n\tuV6/9E+PvLvOqIENjs9Pd39pgfIws8fB9034KcQyoYPDh7Mz5Ry43EPsKF/9XL6myv+e\n\tkSSgucMf4bbcsNdRooDHh0NZ7BlcHv3yYnnwCR2u2K4TNoCZmDDzz01O0m7ijieBp479\n\tChwcmesIf+qK63mXUT85ZxDGir1RneVNYGzOB1n9TOXk4fQoz0E2IDn4xg1Y8C0V9pKK\n\tt/yg==","X-Gm-Message-State":"AOAM530TeGlKVvsTTVZd1HOYGaANi7aE3HgKR1+iQwHz1jcbgbfwgxRO\n\tMuP1/aZgjhO6zBnXpMJR43/sGg==","X-Google-Smtp-Source":"ABdhPJz+pQ/82BU0sEkNU74MIx58Q8uLKaiy4ZANdbWKHimOZXFf2maDn/5oDskehdsOJLoJryDuYw==","X-Received":"by 2002:a2e:89d9:: with SMTP id\n\tc25mr7738350ljk.410.1608550515297; \n\tMon, 21 Dec 2020 03:35:15 -0800 (PST)","To":"Naushir Patuck ","References":"<20201218100626.205134-1-naush@raspberrypi.com>\n\t<20201218100626.205134-2-naush@raspberrypi.com>\n\t\n\t","From":"Andrey Konovalov ","Message-ID":"","Date":"Mon, 21 Dec 2020 14:35:12 +0300","User-Agent":"Mozilla/5.0 (X11; Linux x86_64; rv:68.0) Gecko/20100101\n\tThunderbird/68.10.0","MIME-Version":"1.0","In-Reply-To":"","Content-Language":"en-US","Subject":"Re: [libcamera-devel] [PATCH v10 1/3] libcamera: controls: Add\n\tframe duration control","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Cc":"libcamera devel ","Content-Transfer-Encoding":"base64","Content-Type":"text/plain; charset=\"utf-8\"; Format=\"flowed\"","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" "}},{"id":14447,"web_url":"https://patchwork.libcamera.org/comment/14447/","msgid":"","date":"2021-01-05T12:49:09","subject":"Re: [libcamera-devel] [PATCH v10 1/3] libcamera: controls: Add\n\tframe duration control","submitter":{"id":2,"url":"https://patchwork.libcamera.org/api/people/2/","name":"Laurent Pinchart","email":"laurent.pinchart@ideasonboard.com"},"content":"Hi Naush,\n\nOn Mon, Dec 21, 2020 at 11:11:59AM +0000, Naushir Patuck wrote:\n> On Sun, 20 Dec 2020 at 05:38, Laurent Pinchart wrote:\n> > On Fri, Dec 18, 2020 at 10:06:25AM +0000, Naushir Patuck wrote:\n> > > Add an int64_t array control (controls::FrameDurations) to specify the\n> > > minimum and maximum (in that order) frame duration to be used by the\n> > > camera sensor.\n> > >\n> > > Signed-off-by: Naushir Patuck \n> > > Reviewed-by: David Plowman \n> > > Tested-by: David Plowman \n> > > Reviewed-by: Jacopo Mondi \n> > > ---\n> > > src/libcamera/control_ids.yaml | 41 ++++++++++++++++++++++++++++++++++\n> > > 1 file changed, 41 insertions(+)\n> > >\n> > > diff --git a/src/libcamera/control_ids.yaml b/src/libcamera/control_ids.yaml\n> > > index 6d6f0fee..a58bff18 100644\n> > > --- a/src/libcamera/control_ids.yaml\n> > > +++ b/src/libcamera/control_ids.yaml\n> > > @@ -306,6 +306,47 @@ controls:\n> > > maximum valid value is given by the properties::ScalerCropMaximum\n> > > property, and the two can be used to implement digital zoom.\n> > >\n> > > + - FrameDurations:\n> > > + type: int64_t\n> > > + description: |\n> > > + The minimum and maximum (in that order) frame duration,\n> > > + expressed in micro-seconds.\n> > > +\n> > > + When provided by applications, the control specifies the sensor frame\n> > > + duration interval the pipeline has to use. This could also limit the\n> >\n> > s/could also limit/limits/ ?\n> \n> Ack\n> \n> > > + largest exposure time the sensor can use. For example, if a maximum\n> > > + frame duration of 33ms is requested (corresponding to 30 frames per\n> > > + second), the sensor will not be able to raise the exposure time above\n> > > + 33ms. A fixed frame duration is achieved by setting the minimum and\n> > > + maximum values to be the same.\n> > > +\n> > > + The maximum frame duration provides the absolute limit to the shutter\n> > > + speed computed by the AE algorithm and it overrides any exposure mode\n> >\n> > I think we should talk about exposure time instead of shutter speed, but\n> > we'll need to go through the whole documentation and code base to ensure\n> > consistency, so this can be done later.\n> >\n> > > + setting specified with controls::AeExposureMode. Similarly, when a\n> > > + manual exposure time is set through controls::ExposureTime, it also\n> > > + gets clipped to the limits set by this control.\n> > > +\n> > > + \\sa AeExposureMode\n> > > + \\sa ExposureTime\n> > > +\n> > > + \\todo Refer to the frame duration limits property to describe how\n> > > + application-provided values gets clipped and reset.\n> > > +\n> > > + When reported by pipelines, the control expresses the minimum and\n> >\n> > Is this about Request::metadata(), or Camera::properties() ? I assume\n> > the former as it's defined in control_ids.yaml.\n> \n> This is through Request::metadata(). Should I perhaps reword to make this\n> more explicit?\n> \n> \"When reported by pipelines through metadata, the control expresses....\" ?\n\nYou could just say \"When reported in metadata, ...\".\n\n> > > + maximum frame durations used after being clipped to what the current\n> > > + sensor mode supports, and what is achievable based on the exposure\n> > > + mode setting specified with controls::AeExposureMode or manual\n> > > + exposure time set through controls::ExposureTime. The sensor frame\n> >\n> > I have a hard time parsing this, and a hard time seeing how it should be\n> > used by applications :-S Do you need this for your use cases ?\n> \n> Yes, this was also a point of confusion between Jacopo and me :-)\n> \n> What I was trying to convey is that the values returned through the\n> metadata is the actual limits that will be used (after being constrained by\n> the exposure mode and/or shutter speed), and may not be the same as what\n> the application requested. I thought this might be useful for the\n> application to know the actual limits used. If you think this may not be\n> needed, I could remove this control from the metadata?\n\nNow I understand. Sorry for being slow :-)\n\nOne thing we should report in metadata for sure (but not necessarily as\npart of this series) is the actual frame duration. I've so far thought\nit could be done with this control, with the minimum and maximum values\nset to the actual frame duration. That would however preclude from using\nthe control to report clamped values. We could introduce a new\nFrameDuration control used in metadata only for this purpose, but it\ncould be a bit confusing when setting a particular frame duration to set\nit through FrameDurations and read it back through FrameDuration.\n\nI see pros and cons in both cases, even if I don't see at the moment use\ncases for reporting the constrained limits. If you think it would be\nuseful, I'm fine keeping this usage, if possible with a slightly\nclarified documentation. Maybe this paragraph could be reworked to first\nexplain that the current configuration of the camera sensor and of the\nalgorithms restrict possible frame durations, followed by the sentence\nstating that, when used in metadata, the control reports the limits used\nby the camera after those constraints are taken into account ?\n\n> > > + duration is one of the parameter that defines the capture frame rate\n> > > + but it does not alone provide enough information to fully calculate it\n> > > + as it does not account for pipeline processing delays.\n> >\n> > Delays in the pipeline don't affect the frame duration, do they ? We\n> > could drop frames when processing streams in software (for instance for\n> > JPEG compression), but that's out of scope as libcamera doesn't support\n> > that.\n> \n> If you have, e.g. a slow software denoise running in the IPA whose\n> processing time limits your frame throughput, it could be limiting the\n> overall pipeline framerate (increasing frame duration). Do you think\n> perhaps this should not be mentioned here to avoid complicating things for\n> now?\n\nWhile you're right, I'd drop it for now. We'll need to document how\ncertain processing blocks may not have any bandwidth to process all\nframes at the sensor frame rate, but before we do so, I think partial\ndocumentation would be confusing. The todo comment should be enough for\nnow.\n\n> > > +\n> > > + \\todo Define how to calculate the capture frame rate by\n> > > + defining controls to report additional delays introduced by\n> > > + the capture pipeline or post-processing stages (ie JPEG\n> > > + conversion, frame scaling).\n> > > + size: [2]\n> > > +\n> > > # ----------------------------------------------------------------------------\n> > > # Draft controls section\n> > >","headers":{"Return-Path":"","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id CFE94C0F1A\n\tfor ;\n\tTue, 5 Jan 2021 12:49:24 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id 4FBDE621C2;\n\tTue, 5 Jan 2021 13:49:24 +0100 (CET)","from perceval.ideasonboard.com (perceval.ideasonboard.com\n\t[213.167.242.64])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id 37EDF62012\n\tfor ;\n\tTue, 5 Jan 2021 13:49:22 +0100 (CET)","from pendragon.ideasonboard.com (62-78-145-57.bb.dnainternet.fi\n\t[62.78.145.57])\n\tby perceval.ideasonboard.com (Postfix) with ESMTPSA id A5D203D7;\n\tTue, 5 Jan 2021 13:49:21 +0100 (CET)"],"Authentication-Results":"lancelot.ideasonboard.com;\n\tdkim=fail reason=\"signature verification failed\" (1024-bit key;\n\tunprotected) header.d=ideasonboard.com header.i=@ideasonboard.com\n\theader.b=\"gnoSyoIu\"; dkim-atps=neutral","DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com;\n\ts=mail; t=1609850961;\n\tbh=r1p/2TH1hliVFbFM8KSDRWgvhHifP2GF4C0s1j0h/6w=;\n\th=Date:From:To:Cc:Subject:References:In-Reply-To:From;\n\tb=gnoSyoIuL9FPZoLduUjf0KWIkPjKahmREA5FamRuNP7gLD75mMIgGCKhvZ8hu0hHI\n\t6vk3vduoYGWkiKBpcbDgzdhM3rs/ObDvQqto6Y5vPM/ITwJqaQ/lPeXg01uJhr0CIZ\n\tkCziTp2NlHR7Y9jsjixM/R8CIdL/QwzxK0QlZon0=","Date":"Tue, 5 Jan 2021 14:49:09 +0200","From":"Laurent Pinchart ","To":"Naushir Patuck ","Message-ID":"","References":"<20201218100626.205134-1-naush@raspberrypi.com>\n\t<20201218100626.205134-2-naush@raspberrypi.com>\n\t\n\t","MIME-Version":"1.0","Content-Disposition":"inline","In-Reply-To":"","Subject":"Re: [libcamera-devel] [PATCH v10 1/3] libcamera: controls: Add\n\tframe duration control","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Cc":"libcamera devel ","Content-Type":"text/plain; charset=\"us-ascii\"","Content-Transfer-Encoding":"7bit","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" "}}]