[{"id":15727,"web_url":"https://patchwork.libcamera.org/comment/15727/","msgid":"","date":"2021-03-16T19:28:01","subject":"Re: [libcamera-devel] [RFC PATCH 3/6] libcamera: v4l2_subdevice:\n\tAdd methods to get/set frame interval","submitter":{"id":2,"url":"https://patchwork.libcamera.org/api/people/2/","name":"Laurent Pinchart","email":"laurent.pinchart@ideasonboard.com"},"content":"Hi Marian,\n\nThank you for the patch.\n\nOn Tue, Mar 16, 2021 at 04:52:08PM +0100, Marian Cichy wrote:\n> Add API-methods to get and set the frame interval on V4L2-subdevices.\n> \n> Signed-off-by: Marian Cichy \n> ---\n> include/libcamera/internal/v4l2_subdevice.h | 4 ++\n> src/libcamera/v4l2_subdevice.cpp | 43 +++++++++++++++++++++\n> 2 files changed, 47 insertions(+)\n> \n> diff --git a/include/libcamera/internal/v4l2_subdevice.h b/include/libcamera/internal/v4l2_subdevice.h\n> index d2b9ca55..310746d9 100644\n> --- a/include/libcamera/internal/v4l2_subdevice.h\n> +++ b/include/libcamera/internal/v4l2_subdevice.h\n> @@ -12,6 +12,7 @@\n> #include \n> \n> #include \n> +#include \n> #include \n> \n> #include \"libcamera/internal/formats.h\"\n> @@ -60,6 +61,9 @@ public:\n> \tint setFormat(unsigned int pad, V4L2SubdeviceFormat *format,\n> \t\t Whence whence = ActiveFormat);\n> \n> +\tint getFrameInterval(unsigned int pad, Fraction *frameInterval);\n> +\tint setFrameInterval(unsigned int pad, Fraction *frameInterval);\n> +\n> \tstatic std::unique_ptr\n> \tfromEntityName(const MediaDevice *media, const std::string &entity);\n> \n> diff --git a/src/libcamera/v4l2_subdevice.cpp b/src/libcamera/v4l2_subdevice.cpp\n> index 721ff5a9..eac2d7d3 100644\n> --- a/src/libcamera/v4l2_subdevice.cpp\n> +++ b/src/libcamera/v4l2_subdevice.cpp\n> @@ -17,6 +17,7 @@\n> #include \n> #include \n> \n> +#include \n> #include \n> \n> #include \"libcamera/internal/log.h\"\n> @@ -440,6 +441,48 @@ int V4L2Subdevice::setFormat(unsigned int pad, V4L2SubdeviceFormat *format,\n> \treturn 0;\n> }\n> \n> +int V4L2Subdevice::getFrameInterval(unsigned int pad, Fraction *frameInterval)\n> +{\n> +\tstruct v4l2_subdev_frame_interval subdevFrameInterval = {};\n> +\tsubdevFrameInterval.pad = pad;\n> +\n> +\tint ret = ioctl(VIDIOC_SUBDEV_G_FRAME_INTERVAL, &subdevFrameInterval);\n\nSensor drivers are supposed to control the frame rate through horizontal\nand vertical blanking. Only in very specific cases do we want to use\nthese ioctls, when the sensor exposes a frame interval at the register\nlevel (this will only be applicable to sensors containing an ISP). What\nsensor are you using ?\n\n> +\tif (ret) {\n> +\t\tLOG(V4L2, Debug)\n> +\t\t\t<< \"Unable to get frame interval on pad \" << pad\n> +\t\t\t<< \": \" << strerror(-ret);\n> +\t\treturn ret;\n> +\t}\n> +\n> +\tframeInterval->numerator = subdevFrameInterval.interval.numerator;\n> +\tframeInterval->denominator = subdevFrameInterval.interval.denominator;\n> +\n> +\treturn 0;\n> +}\n> +\n> +int V4L2Subdevice::setFrameInterval(unsigned int pad, Fraction *frameInterval)\n> +{\n> +\tstruct v4l2_subdev_frame_interval subdevFrameInterval = {};\n> +\tstruct v4l2_fract fract = { frameInterval->numerator,\n> +\t\t\t\t frameInterval->denominator };\n> +\n> +\tsubdevFrameInterval.pad = pad;\n> +\tsubdevFrameInterval.interval = fract;\n> +\n> +\tint ret = ioctl(VIDIOC_SUBDEV_S_FRAME_INTERVAL, &subdevFrameInterval);\n> +\tif (ret) {\n> +\t\tLOG(V4L2, Debug)\n> +\t\t\t<< \"Unable to set frame interval on pad \" << pad\n> +\t\t\t<< \": \" << strerror(-ret);\n> +\t\treturn ret;\n> +\t}\n> +\n> +\tframeInterval->numerator = subdevFrameInterval.interval.numerator;\n> +\tframeInterval->denominator = subdevFrameInterval.interval.denominator;\n> +\n> +\treturn 0;\n> +}\n> +\n> /**\n> * \\brief Create a new video subdevice instance from \\a entity in media device\n> * \\a media","headers":{"Return-Path":"","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id 0FE36C32E1\n\tfor ;\n\tTue, 16 Mar 2021 19:28:40 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id 5EF9B68D4E;\n\tTue, 16 Mar 2021 20:28:39 +0100 (CET)","from perceval.ideasonboard.com (perceval.ideasonboard.com\n\t[IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id 66EB968D49\n\tfor ;\n\tTue, 16 Mar 2021 20:28:38 +0100 (CET)","from pendragon.ideasonboard.com (62-78-145-57.bb.dnainternet.fi\n\t[62.78.145.57])\n\tby perceval.ideasonboard.com (Postfix) with ESMTPSA id C9340D8B;\n\tTue, 16 Mar 2021 20:28:37 +0100 (CET)"],"Authentication-Results":"lancelot.ideasonboard.com;\n\tdkim=fail reason=\"signature verification failed\" (1024-bit key;\n\tunprotected) header.d=ideasonboard.com header.i=@ideasonboard.com\n\theader.b=\"OZobDECv\"; dkim-atps=neutral","DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com;\n\ts=mail; t=1615922918;\n\tbh=n9w1LIYrWYsPt+yDf5nrf0Ufk/01GmZl9fG/DLNE6QU=;\n\th=Date:From:To:Cc:Subject:References:In-Reply-To:From;\n\tb=OZobDECvB87lTq2Wa04O0mVi7qJoIz0ljgJVC5o8H1y4ymuP9cMnxKbvQ6LiGupWn\n\to64ylBQlnpq+4W1AqzLGYiktNecgUbEZtXr2nLtcguHVN6hS07/UaK7ZwehG3FrM5i\n\t6ZwG+gukg7ryU/+Vt+u1qmr6bUTbgsEeE+cHB/6U=","Date":"Tue, 16 Mar 2021 21:28:01 +0200","From":"Laurent Pinchart ","To":"Marian Cichy ","Message-ID":"","References":"<20210316155211.6679-1-m.cichy@pengutronix.de>\n\t<20210316155211.6679-4-m.cichy@pengutronix.de>","MIME-Version":"1.0","Content-Disposition":"inline","In-Reply-To":"<20210316155211.6679-4-m.cichy@pengutronix.de>","Subject":"Re: [libcamera-devel] [RFC PATCH 3/6] libcamera: v4l2_subdevice:\n\tAdd methods to get/set frame interval","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Cc":"libcamera-devel@lists.libcamera.org, graphics@pengutronix.de","Content-Type":"text/plain; charset=\"us-ascii\"","Content-Transfer-Encoding":"7bit","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" "}},{"id":15736,"web_url":"https://patchwork.libcamera.org/comment/15736/","msgid":"<753330db-5b5d-fbd1-115e-9d8c6705e55c@pengutronix.de>","date":"2021-03-17T09:55:25","subject":"Re: [libcamera-devel] [RFC PATCH 3/6] libcamera: v4l2_subdevice:\n\tAdd methods to get/set frame interval","submitter":{"id":81,"url":"https://patchwork.libcamera.org/api/people/81/","name":"Marian Cichy","email":"mci@pengutronix.de"},"content":"On 3/16/21 8:28 PM, Laurent Pinchart wrote:\n> Hi Marian,\n>\n> Thank you for the patch.\n>\n> On Tue, Mar 16, 2021 at 04:52:08PM +0100, Marian Cichy wrote:\n>> Add API-methods to get and set the frame interval on V4L2-subdevices.\n>>\n>> Signed-off-by: Marian Cichy \n>> ---\n>> include/libcamera/internal/v4l2_subdevice.h | 4 ++\n>> src/libcamera/v4l2_subdevice.cpp | 43 +++++++++++++++++++++\n>> 2 files changed, 47 insertions(+)\n>>\n>> diff --git a/include/libcamera/internal/v4l2_subdevice.h b/include/libcamera/internal/v4l2_subdevice.h\n>> index d2b9ca55..310746d9 100644\n>> --- a/include/libcamera/internal/v4l2_subdevice.h\n>> +++ b/include/libcamera/internal/v4l2_subdevice.h\n>> @@ -12,6 +12,7 @@\n>> #include \n>> \n>> #include \n>> +#include \n>> #include \n>> \n>> #include \"libcamera/internal/formats.h\"\n>> @@ -60,6 +61,9 @@ public:\n>> \tint setFormat(unsigned int pad, V4L2SubdeviceFormat *format,\n>> \t\t Whence whence = ActiveFormat);\n>> \n>> +\tint getFrameInterval(unsigned int pad, Fraction *frameInterval);\n>> +\tint setFrameInterval(unsigned int pad, Fraction *frameInterval);\n>> +\n>> \tstatic std::unique_ptr\n>> \tfromEntityName(const MediaDevice *media, const std::string &entity);\n>> \n>> diff --git a/src/libcamera/v4l2_subdevice.cpp b/src/libcamera/v4l2_subdevice.cpp\n>> index 721ff5a9..eac2d7d3 100644\n>> --- a/src/libcamera/v4l2_subdevice.cpp\n>> +++ b/src/libcamera/v4l2_subdevice.cpp\n>> @@ -17,6 +17,7 @@\n>> #include \n>> #include \n>> \n>> +#include \n>> #include \n>> \n>> #include \"libcamera/internal/log.h\"\n>> @@ -440,6 +441,48 @@ int V4L2Subdevice::setFormat(unsigned int pad, V4L2SubdeviceFormat *format,\n>> \treturn 0;\n>> }\n>> \n>> +int V4L2Subdevice::getFrameInterval(unsigned int pad, Fraction *frameInterval)\n>> +{\n>> +\tstruct v4l2_subdev_frame_interval subdevFrameInterval = {};\n>> +\tsubdevFrameInterval.pad = pad;\n>> +\n>> +\tint ret = ioctl(VIDIOC_SUBDEV_G_FRAME_INTERVAL, &subdevFrameInterval);\n> Sensor drivers are supposed to control the frame rate through horizontal\n> and vertical blanking. Only in very specific cases do we want to use\n> these ioctls, when the sensor exposes a frame interval at the register\n> level (this will only be applicable to sensors containing an ISP). What\n> sensor are you using ?\n\nI am using the ar0237 sensor, it's driver is not upstream yet. We \nimplement hblank/vblank only as read-only controls for now.\n\nHowever, using the i.MX-6Q, it is also possible to configure \nframe-dropping on the CSI-subdev. Afaik, this can only be configured \nwith the frame_interval ioctl since the frame interval has to be set on \nsource and sink pad.\n\nThis is also why we propagate the frame interval from the sensor through \nthe media pipeline. If for some reason, the CSI is configured to drop \nframes, we query the frame interval at the sensor, then try to set it on \nall following subdevices to effectively deactivate frame dropping.\n\nOf course, it is also thinkable that we want to configure the CSI to \ndrop frames, for example to manage bandwidth.\n\n>\n>> +\tif (ret) {\n>> +\t\tLOG(V4L2, Debug)\n>> +\t\t\t<< \"Unable to get frame interval on pad \" << pad\n>> +\t\t\t<< \": \" << strerror(-ret);\n>> +\t\treturn ret;\n>> +\t}\n>> +\n>> +\tframeInterval->numerator = subdevFrameInterval.interval.numerator;\n>> +\tframeInterval->denominator = subdevFrameInterval.interval.denominator;\n>> +\n>> +\treturn 0;\n>> +}\n>> +\n>> +int V4L2Subdevice::setFrameInterval(unsigned int pad, Fraction *frameInterval)\n>> +{\n>> +\tstruct v4l2_subdev_frame_interval subdevFrameInterval = {};\n>> +\tstruct v4l2_fract fract = { frameInterval->numerator,\n>> +\t\t\t\t frameInterval->denominator };\n>> +\n>> +\tsubdevFrameInterval.pad = pad;\n>> +\tsubdevFrameInterval.interval = fract;\n>> +\n>> +\tint ret = ioctl(VIDIOC_SUBDEV_S_FRAME_INTERVAL, &subdevFrameInterval);\n>> +\tif (ret) {\n>> +\t\tLOG(V4L2, Debug)\n>> +\t\t\t<< \"Unable to set frame interval on pad \" << pad\n>> +\t\t\t<< \": \" << strerror(-ret);\n>> +\t\treturn ret;\n>> +\t}\n>> +\n>> +\tframeInterval->numerator = subdevFrameInterval.interval.numerator;\n>> +\tframeInterval->denominator = subdevFrameInterval.interval.denominator;\n>> +\n>> +\treturn 0;\n>> +}\n>> +\n>> /**\n>> * \\brief Create a new video subdevice instance from \\a entity in media device\n>> * \\a media","headers":{"Return-Path":"","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id 9127AC32E1\n\tfor ;\n\tWed, 17 Mar 2021 09:55:29 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id 062F368D62;\n\tWed, 17 Mar 2021 10:55:29 +0100 (CET)","from metis.ext.pengutronix.de (metis.ext.pengutronix.de\n\t[IPv6:2001:67c:670:201:290:27ff:fe1d:cc33])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id 4770B6084D\n\tfor ;\n\tWed, 17 Mar 2021 10:55:27 +0100 (CET)","from erdnuss.hi.pengutronix.de\n\t([2001:67c:670:100:2e4d:54ff:fe9d:849c])\n\tby metis.ext.pengutronix.de with esmtp (Exim 4.92)\n\t(envelope-from )\n\tid 1lMSti-000262-JF; Wed, 17 Mar 2021 10:55:26 +0100"],"To":"Laurent Pinchart ","References":"<20210316155211.6679-1-m.cichy@pengutronix.de>\n\t<20210316155211.6679-4-m.cichy@pengutronix.de>\n\t","From":"Marian Cichy ","Message-ID":"<753330db-5b5d-fbd1-115e-9d8c6705e55c@pengutronix.de>","Date":"Wed, 17 Mar 2021 10:55:25 +0100","User-Agent":"Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101\n\tThunderbird/78.8.0","MIME-Version":"1.0","In-Reply-To":"","Content-Language":"en-US","X-SA-Exim-Connect-IP":"2001:67c:670:100:2e4d:54ff:fe9d:849c","X-SA-Exim-Mail-From":"mci@pengutronix.de","X-SA-Exim-Scanned":"No (on metis.ext.pengutronix.de);\n\tSAEximRunCond expanded to false","X-PTX-Original-Recipient":"libcamera-devel@lists.libcamera.org","Subject":"Re: [libcamera-devel] [RFC PATCH 3/6] libcamera: v4l2_subdevice:\n\tAdd methods to get/set frame interval","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Cc":"libcamera-devel@lists.libcamera.org, graphics@pengutronix.de","Content-Transfer-Encoding":"7bit","Content-Type":"text/plain; charset=\"us-ascii\"; Format=\"flowed\"","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" "}},{"id":15737,"web_url":"https://patchwork.libcamera.org/comment/15737/","msgid":"","date":"2021-03-17T10:58:28","subject":"Re: [libcamera-devel] [RFC PATCH 3/6] libcamera: v4l2_subdevice:\n\tAdd methods to get/set frame interval","submitter":{"id":2,"url":"https://patchwork.libcamera.org/api/people/2/","name":"Laurent Pinchart","email":"laurent.pinchart@ideasonboard.com"},"content":"Hi Marian,\n\nOn Wed, Mar 17, 2021 at 10:55:25AM +0100, Marian Cichy wrote:\n> On 3/16/21 8:28 PM, Laurent Pinchart wrote:\n> > On Tue, Mar 16, 2021 at 04:52:08PM +0100, Marian Cichy wrote:\n> >> Add API-methods to get and set the frame interval on V4L2-subdevices.\n> >>\n> >> Signed-off-by: Marian Cichy \n> >> ---\n> >> include/libcamera/internal/v4l2_subdevice.h | 4 ++\n> >> src/libcamera/v4l2_subdevice.cpp | 43 +++++++++++++++++++++\n> >> 2 files changed, 47 insertions(+)\n> >>\n> >> diff --git a/include/libcamera/internal/v4l2_subdevice.h b/include/libcamera/internal/v4l2_subdevice.h\n> >> index d2b9ca55..310746d9 100644\n> >> --- a/include/libcamera/internal/v4l2_subdevice.h\n> >> +++ b/include/libcamera/internal/v4l2_subdevice.h\n> >> @@ -12,6 +12,7 @@\n> >> #include \n> >> \n> >> #include \n> >> +#include \n> >> #include \n> >> \n> >> #include \"libcamera/internal/formats.h\"\n> >> @@ -60,6 +61,9 @@ public:\n> >> \tint setFormat(unsigned int pad, V4L2SubdeviceFormat *format,\n> >> \t\t Whence whence = ActiveFormat);\n> >> \n> >> +\tint getFrameInterval(unsigned int pad, Fraction *frameInterval);\n> >> +\tint setFrameInterval(unsigned int pad, Fraction *frameInterval);\n> >> +\n> >> \tstatic std::unique_ptr\n> >> \tfromEntityName(const MediaDevice *media, const std::string &entity);\n> >> \n> >> diff --git a/src/libcamera/v4l2_subdevice.cpp b/src/libcamera/v4l2_subdevice.cpp\n> >> index 721ff5a9..eac2d7d3 100644\n> >> --- a/src/libcamera/v4l2_subdevice.cpp\n> >> +++ b/src/libcamera/v4l2_subdevice.cpp\n> >> @@ -17,6 +17,7 @@\n> >> #include \n> >> #include \n> >> \n> >> +#include \n> >> #include \n> >> \n> >> #include \"libcamera/internal/log.h\"\n> >> @@ -440,6 +441,48 @@ int V4L2Subdevice::setFormat(unsigned int pad, V4L2SubdeviceFormat *format,\n> >> \treturn 0;\n> >> }\n> >> \n> >> +int V4L2Subdevice::getFrameInterval(unsigned int pad, Fraction *frameInterval)\n> >> +{\n> >> +\tstruct v4l2_subdev_frame_interval subdevFrameInterval = {};\n> >> +\tsubdevFrameInterval.pad = pad;\n> >> +\n> >> +\tint ret = ioctl(VIDIOC_SUBDEV_G_FRAME_INTERVAL, &subdevFrameInterval);\n> >\n> > Sensor drivers are supposed to control the frame rate through horizontal\n> > and vertical blanking. Only in very specific cases do we want to use\n> > these ioctls, when the sensor exposes a frame interval at the register\n> > level (this will only be applicable to sensors containing an ISP). What\n> > sensor are you using ?\n> \n> I am using the ar0237 sensor, it's driver is not upstream yet. We \n> implement hblank/vblank only as read-only controls for now.\n\nLooks like there's some work to do on the sensor side then ;-)\n\n> However, using the i.MX-6Q, it is also possible to configure \n> frame-dropping on the CSI-subdev. Afaik, this can only be configured \n> with the frame_interval ioctl since the frame interval has to be set on \n> source and sink pad.\n\nDo you have a use case for this, compared to configuring the sensor with\na longer vertical blanking ?\n\n> This is also why we propagate the frame interval from the sensor through \n> the media pipeline. If for some reason, the CSI is configured to drop \n> frames, we query the frame interval at the sensor, then try to set it on \n> all following subdevices to effectively deactivate frame dropping.\n> \n> Of course, it is also thinkable that we want to configure the CSI to \n> drop frames, for example to manage bandwidth.\n> \n> >> +\tif (ret) {\n> >> +\t\tLOG(V4L2, Debug)\n> >> +\t\t\t<< \"Unable to get frame interval on pad \" << pad\n> >> +\t\t\t<< \": \" << strerror(-ret);\n> >> +\t\treturn ret;\n> >> +\t}\n> >> +\n> >> +\tframeInterval->numerator = subdevFrameInterval.interval.numerator;\n> >> +\tframeInterval->denominator = subdevFrameInterval.interval.denominator;\n> >> +\n> >> +\treturn 0;\n> >> +}\n> >> +\n> >> +int V4L2Subdevice::setFrameInterval(unsigned int pad, Fraction *frameInterval)\n> >> +{\n> >> +\tstruct v4l2_subdev_frame_interval subdevFrameInterval = {};\n> >> +\tstruct v4l2_fract fract = { frameInterval->numerator,\n> >> +\t\t\t\t frameInterval->denominator };\n> >> +\n> >> +\tsubdevFrameInterval.pad = pad;\n> >> +\tsubdevFrameInterval.interval = fract;\n> >> +\n> >> +\tint ret = ioctl(VIDIOC_SUBDEV_S_FRAME_INTERVAL, &subdevFrameInterval);\n> >> +\tif (ret) {\n> >> +\t\tLOG(V4L2, Debug)\n> >> +\t\t\t<< \"Unable to set frame interval on pad \" << pad\n> >> +\t\t\t<< \": \" << strerror(-ret);\n> >> +\t\treturn ret;\n> >> +\t}\n> >> +\n> >> +\tframeInterval->numerator = subdevFrameInterval.interval.numerator;\n> >> +\tframeInterval->denominator = subdevFrameInterval.interval.denominator;\n> >> +\n> >> +\treturn 0;\n> >> +}\n> >> +\n> >> /**\n> >> * \\brief Create a new video subdevice instance from \\a entity in media device\n> >> * \\a media","headers":{"Return-Path":"","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id 47C07BD80C\n\tfor ;\n\tWed, 17 Mar 2021 10:59:06 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id 9A9FC68D61;\n\tWed, 17 Mar 2021 11:59:05 +0100 (CET)","from perceval.ideasonboard.com (perceval.ideasonboard.com\n\t[213.167.242.64])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id 4E008602DB\n\tfor ;\n\tWed, 17 Mar 2021 11:59:04 +0100 (CET)","from pendragon.ideasonboard.com (62-78-145-57.bb.dnainternet.fi\n\t[62.78.145.57])\n\tby perceval.ideasonboard.com (Postfix) with ESMTPSA id CDADF596;\n\tWed, 17 Mar 2021 11:59:03 +0100 (CET)"],"Authentication-Results":"lancelot.ideasonboard.com;\n\tdkim=fail reason=\"signature verification failed\" (1024-bit key;\n\tunprotected) header.d=ideasonboard.com header.i=@ideasonboard.com\n\theader.b=\"rLbWNRar\"; dkim-atps=neutral","DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com;\n\ts=mail; t=1615978744;\n\tbh=ueWeUvwcF6sMQv5o5iNn7wxToTWblhmToeSbkAo5skk=;\n\th=Date:From:To:Cc:Subject:References:In-Reply-To:From;\n\tb=rLbWNRaraH48cWDkajr3THhmndWSq10YAlojoGY0uGfGmUaoh6/XT5cKQZcJ5YQV7\n\tJuQyUkJxQ61UeONyx+Gt3RwsR8tKc5MhMX93VtbS5cYV1/2Znqy/XNd834ahllMWpV\n\tXIoptthHZLMXZb5HbUPUJXAEvuSswur8KyLG+GMU=","Date":"Wed, 17 Mar 2021 12:58:28 +0200","From":"Laurent Pinchart ","To":"Marian Cichy ","Message-ID":"","References":"<20210316155211.6679-1-m.cichy@pengutronix.de>\n\t<20210316155211.6679-4-m.cichy@pengutronix.de>\n\t\n\t<753330db-5b5d-fbd1-115e-9d8c6705e55c@pengutronix.de>","MIME-Version":"1.0","Content-Disposition":"inline","In-Reply-To":"<753330db-5b5d-fbd1-115e-9d8c6705e55c@pengutronix.de>","Subject":"Re: [libcamera-devel] [RFC PATCH 3/6] libcamera: v4l2_subdevice:\n\tAdd methods to get/set frame interval","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Cc":"libcamera-devel@lists.libcamera.org, graphics@pengutronix.de","Content-Type":"text/plain; charset=\"us-ascii\"","Content-Transfer-Encoding":"7bit","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" "}},{"id":15742,"web_url":"https://patchwork.libcamera.org/comment/15742/","msgid":"<3f79d29a-3f6d-6a99-a646-d33bb49806d7@pengutronix.de>","date":"2021-03-17T13:17:44","subject":"Re: [libcamera-devel] [RFC PATCH 3/6] libcamera: v4l2_subdevice:\n\tAdd methods to get/set frame interval","submitter":{"id":81,"url":"https://patchwork.libcamera.org/api/people/81/","name":"Marian Cichy","email":"mci@pengutronix.de"},"content":"On 3/17/21 11:58 AM, Laurent Pinchart wrote:\n> Hi Marian,\n>\n> On Wed, Mar 17, 2021 at 10:55:25AM +0100, Marian Cichy wrote:\n>> On 3/16/21 8:28 PM, Laurent Pinchart wrote:\n>>> On Tue, Mar 16, 2021 at 04:52:08PM +0100, Marian Cichy wrote:\n>>>> Add API-methods to get and set the frame interval on V4L2-subdevices.\n>>>>\n>>>> Signed-off-by: Marian Cichy \n>>>> ---\n>>>> include/libcamera/internal/v4l2_subdevice.h | 4 ++\n>>>> src/libcamera/v4l2_subdevice.cpp | 43 +++++++++++++++++++++\n>>>> 2 files changed, 47 insertions(+)\n>>>>\n>>>> diff --git a/include/libcamera/internal/v4l2_subdevice.h b/include/libcamera/internal/v4l2_subdevice.h\n>>>> index d2b9ca55..310746d9 100644\n>>>> --- a/include/libcamera/internal/v4l2_subdevice.h\n>>>> +++ b/include/libcamera/internal/v4l2_subdevice.h\n>>>> @@ -12,6 +12,7 @@\n>>>> #include \n>>>> \n>>>> #include \n>>>> +#include \n>>>> #include \n>>>> \n>>>> #include \"libcamera/internal/formats.h\"\n>>>> @@ -60,6 +61,9 @@ public:\n>>>> \tint setFormat(unsigned int pad, V4L2SubdeviceFormat *format,\n>>>> \t\t Whence whence = ActiveFormat);\n>>>> \n>>>> +\tint getFrameInterval(unsigned int pad, Fraction *frameInterval);\n>>>> +\tint setFrameInterval(unsigned int pad, Fraction *frameInterval);\n>>>> +\n>>>> \tstatic std::unique_ptr\n>>>> \tfromEntityName(const MediaDevice *media, const std::string &entity);\n>>>> \n>>>> diff --git a/src/libcamera/v4l2_subdevice.cpp b/src/libcamera/v4l2_subdevice.cpp\n>>>> index 721ff5a9..eac2d7d3 100644\n>>>> --- a/src/libcamera/v4l2_subdevice.cpp\n>>>> +++ b/src/libcamera/v4l2_subdevice.cpp\n>>>> @@ -17,6 +17,7 @@\n>>>> #include \n>>>> #include \n>>>> \n>>>> +#include \n>>>> #include \n>>>> \n>>>> #include \"libcamera/internal/log.h\"\n>>>> @@ -440,6 +441,48 @@ int V4L2Subdevice::setFormat(unsigned int pad, V4L2SubdeviceFormat *format,\n>>>> \treturn 0;\n>>>> }\n>>>> \n>>>> +int V4L2Subdevice::getFrameInterval(unsigned int pad, Fraction *frameInterval)\n>>>> +{\n>>>> +\tstruct v4l2_subdev_frame_interval subdevFrameInterval = {};\n>>>> +\tsubdevFrameInterval.pad = pad;\n>>>> +\n>>>> +\tint ret = ioctl(VIDIOC_SUBDEV_G_FRAME_INTERVAL, &subdevFrameInterval);\n>>> Sensor drivers are supposed to control the frame rate through horizontal\n>>> and vertical blanking. Only in very specific cases do we want to use\n>>> these ioctls, when the sensor exposes a frame interval at the register\n>>> level (this will only be applicable to sensors containing an ISP). What\n>>> sensor are you using ?\n>> I am using the ar0237 sensor, it's driver is not upstream yet. We\n>> implement hblank/vblank only as read-only controls for now.\n> Looks like there's some work to do on the sensor side then ;-)\n>\n>> However, using the i.MX-6Q, it is also possible to configure\n>> frame-dropping on the CSI-subdev. Afaik, this can only be configured\n>> with the frame_interval ioctl since the frame interval has to be set on\n>> source and sink pad.\n> Do you have a use case for this, compared to configuring the sensor with\n> a longer vertical blanking ?\n\nNot yet, but is it always possible to make my vblank arbitrarily long? \nFor example I have a 60 fps camera but I want to reduce it down to 10 \nfps. Maybe some devices won't allow big enough vblank values.\n\nAnother possibility could be that you don't have a sensor but an HDMI \ninput as your source, where you can't control the frame rate (but maybe \nthis isn't even in the scope of libcamera).\n\n>\n>> This is also why we propagate the frame interval from the sensor through\n>> the media pipeline. If for some reason, the CSI is configured to drop\n>> frames, we query the frame interval at the sensor, then try to set it on\n>> all following subdevices to effectively deactivate frame dropping.\n>>\n>> Of course, it is also thinkable that we want to configure the CSI to\n>> drop frames, for example to manage bandwidth.\n>>\n>>>> +\tif (ret) {\n>>>> +\t\tLOG(V4L2, Debug)\n>>>> +\t\t\t<< \"Unable to get frame interval on pad \" << pad\n>>>> +\t\t\t<< \": \" << strerror(-ret);\n>>>> +\t\treturn ret;\n>>>> +\t}\n>>>> +\n>>>> +\tframeInterval->numerator = subdevFrameInterval.interval.numerator;\n>>>> +\tframeInterval->denominator = subdevFrameInterval.interval.denominator;\n>>>> +\n>>>> +\treturn 0;\n>>>> +}\n>>>> +\n>>>> +int V4L2Subdevice::setFrameInterval(unsigned int pad, Fraction *frameInterval)\n>>>> +{\n>>>> +\tstruct v4l2_subdev_frame_interval subdevFrameInterval = {};\n>>>> +\tstruct v4l2_fract fract = { frameInterval->numerator,\n>>>> +\t\t\t\t frameInterval->denominator };\n>>>> +\n>>>> +\tsubdevFrameInterval.pad = pad;\n>>>> +\tsubdevFrameInterval.interval = fract;\n>>>> +\n>>>> +\tint ret = ioctl(VIDIOC_SUBDEV_S_FRAME_INTERVAL, &subdevFrameInterval);\n>>>> +\tif (ret) {\n>>>> +\t\tLOG(V4L2, Debug)\n>>>> +\t\t\t<< \"Unable to set frame interval on pad \" << pad\n>>>> +\t\t\t<< \": \" << strerror(-ret);\n>>>> +\t\treturn ret;\n>>>> +\t}\n>>>> +\n>>>> +\tframeInterval->numerator = subdevFrameInterval.interval.numerator;\n>>>> +\tframeInterval->denominator = subdevFrameInterval.interval.denominator;\n>>>> +\n>>>> +\treturn 0;\n>>>> +}\n>>>> +\n>>>> /**\n>>>> * \\brief Create a new video subdevice instance from \\a entity in media device\n>>>> * \\a media","headers":{"Return-Path":"","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id 371E3BD80C\n\tfor ;\n\tWed, 17 Mar 2021 13:17:47 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id 8A41268D60;\n\tWed, 17 Mar 2021 14:17:46 +0100 (CET)","from metis.ext.pengutronix.de (metis.ext.pengutronix.de\n\t[IPv6:2001:67c:670:201:290:27ff:fe1d:cc33])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id 3CB9B605B1\n\tfor ;\n\tWed, 17 Mar 2021 14:17:45 +0100 (CET)","from erdnuss.hi.pengutronix.de\n\t([2001:67c:670:100:2e4d:54ff:fe9d:849c])\n\tby metis.ext.pengutronix.de with esmtp (Exim 4.92)\n\t(envelope-from )\n\tid 1lMW3U-0006nZ-JW; Wed, 17 Mar 2021 14:17:44 +0100"],"To":"Laurent Pinchart ","References":"<20210316155211.6679-1-m.cichy@pengutronix.de>\n\t<20210316155211.6679-4-m.cichy@pengutronix.de>\n\t\n\t<753330db-5b5d-fbd1-115e-9d8c6705e55c@pengutronix.de>\n\t","From":"Marian Cichy ","Message-ID":"<3f79d29a-3f6d-6a99-a646-d33bb49806d7@pengutronix.de>","Date":"Wed, 17 Mar 2021 14:17:44 +0100","User-Agent":"Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101\n\tThunderbird/78.8.0","MIME-Version":"1.0","In-Reply-To":"","Content-Language":"en-US","X-SA-Exim-Connect-IP":"2001:67c:670:100:2e4d:54ff:fe9d:849c","X-SA-Exim-Mail-From":"mci@pengutronix.de","X-SA-Exim-Scanned":"No (on metis.ext.pengutronix.de);\n\tSAEximRunCond expanded to false","X-PTX-Original-Recipient":"libcamera-devel@lists.libcamera.org","Subject":"Re: [libcamera-devel] [RFC PATCH 3/6] libcamera: v4l2_subdevice:\n\tAdd methods to get/set frame interval","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Cc":"libcamera-devel@lists.libcamera.org, graphics@pengutronix.de","Content-Transfer-Encoding":"7bit","Content-Type":"text/plain; charset=\"us-ascii\"; Format=\"flowed\"","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" "}},{"id":15745,"web_url":"https://patchwork.libcamera.org/comment/15745/","msgid":"","date":"2021-03-17T14:06:40","subject":"Re: [libcamera-devel] [RFC PATCH 3/6] libcamera: v4l2_subdevice:\n\tAdd methods to get/set frame interval","submitter":{"id":2,"url":"https://patchwork.libcamera.org/api/people/2/","name":"Laurent Pinchart","email":"laurent.pinchart@ideasonboard.com"},"content":"Hi Marian,\n\nOn Wed, Mar 17, 2021 at 02:17:44PM +0100, Marian Cichy wrote:\n> On 3/17/21 11:58 AM, Laurent Pinchart wrote:\n> > On Wed, Mar 17, 2021 at 10:55:25AM +0100, Marian Cichy wrote:\n> >> On 3/16/21 8:28 PM, Laurent Pinchart wrote:\n> >>> On Tue, Mar 16, 2021 at 04:52:08PM +0100, Marian Cichy wrote:\n> >>>> Add API-methods to get and set the frame interval on V4L2-subdevices.\n> >>>>\n> >>>> Signed-off-by: Marian Cichy \n> >>>> ---\n> >>>> include/libcamera/internal/v4l2_subdevice.h | 4 ++\n> >>>> src/libcamera/v4l2_subdevice.cpp | 43 +++++++++++++++++++++\n> >>>> 2 files changed, 47 insertions(+)\n> >>>>\n> >>>> diff --git a/include/libcamera/internal/v4l2_subdevice.h b/include/libcamera/internal/v4l2_subdevice.h\n> >>>> index d2b9ca55..310746d9 100644\n> >>>> --- a/include/libcamera/internal/v4l2_subdevice.h\n> >>>> +++ b/include/libcamera/internal/v4l2_subdevice.h\n> >>>> @@ -12,6 +12,7 @@\n> >>>> #include \n> >>>> \n> >>>> #include \n> >>>> +#include \n> >>>> #include \n> >>>> \n> >>>> #include \"libcamera/internal/formats.h\"\n> >>>> @@ -60,6 +61,9 @@ public:\n> >>>> \tint setFormat(unsigned int pad, V4L2SubdeviceFormat *format,\n> >>>> \t\t Whence whence = ActiveFormat);\n> >>>> \n> >>>> +\tint getFrameInterval(unsigned int pad, Fraction *frameInterval);\n> >>>> +\tint setFrameInterval(unsigned int pad, Fraction *frameInterval);\n> >>>> +\n> >>>> \tstatic std::unique_ptr\n> >>>> \tfromEntityName(const MediaDevice *media, const std::string &entity);\n> >>>> \n> >>>> diff --git a/src/libcamera/v4l2_subdevice.cpp b/src/libcamera/v4l2_subdevice.cpp\n> >>>> index 721ff5a9..eac2d7d3 100644\n> >>>> --- a/src/libcamera/v4l2_subdevice.cpp\n> >>>> +++ b/src/libcamera/v4l2_subdevice.cpp\n> >>>> @@ -17,6 +17,7 @@\n> >>>> #include \n> >>>> #include \n> >>>> \n> >>>> +#include \n> >>>> #include \n> >>>> \n> >>>> #include \"libcamera/internal/log.h\"\n> >>>> @@ -440,6 +441,48 @@ int V4L2Subdevice::setFormat(unsigned int pad, V4L2SubdeviceFormat *format,\n> >>>> \treturn 0;\n> >>>> }\n> >>>> \n> >>>> +int V4L2Subdevice::getFrameInterval(unsigned int pad, Fraction *frameInterval)\n> >>>> +{\n> >>>> +\tstruct v4l2_subdev_frame_interval subdevFrameInterval = {};\n> >>>> +\tsubdevFrameInterval.pad = pad;\n> >>>> +\n> >>>> +\tint ret = ioctl(VIDIOC_SUBDEV_G_FRAME_INTERVAL, &subdevFrameInterval);\n> >>>\n> >>> Sensor drivers are supposed to control the frame rate through horizontal\n> >>> and vertical blanking. Only in very specific cases do we want to use\n> >>> these ioctls, when the sensor exposes a frame interval at the register\n> >>> level (this will only be applicable to sensors containing an ISP). What\n> >>> sensor are you using ?\n> >>\n> >> I am using the ar0237 sensor, it's driver is not upstream yet. We\n> >> implement hblank/vblank only as read-only controls for now.\n> >\n> > Looks like there's some work to do on the sensor side then ;-)\n> >\n> >> However, using the i.MX-6Q, it is also possible to configure\n> >> frame-dropping on the CSI-subdev. Afaik, this can only be configured\n> >> with the frame_interval ioctl since the frame interval has to be set on\n> >> source and sink pad.\n> >\n> > Do you have a use case for this, compared to configuring the sensor with\n> > a longer vertical blanking ?\n> \n> Not yet, but is it always possible to make my vblank arbitrarily long? \n> For example I have a 60 fps camera but I want to reduce it down to 10 \n> fps. Maybe some devices won't allow big enough vblank values.\n\nIf you want to go for very low frame rates, such as one frame per\nminute, then sensors may not be able to achieve that. What's the\ntheoretical higher frame duration limit of the CSI ?\n\n> Another possibility could be that you don't have a sensor but an HDMI \n> input as your source, where you can't control the frame rate (but maybe \n> this isn't even in the scope of libcamera).\n\nIndeed out of scope for libcamera, at least for now :-)\n\n> >> This is also why we propagate the frame interval from the sensor through\n> >> the media pipeline. If for some reason, the CSI is configured to drop\n> >> frames, we query the frame interval at the sensor, then try to set it on\n> >> all following subdevices to effectively deactivate frame dropping.\n> >>\n> >> Of course, it is also thinkable that we want to configure the CSI to\n> >> drop frames, for example to manage bandwidth.\n> >>\n> >>>> +\tif (ret) {\n> >>>> +\t\tLOG(V4L2, Debug)\n> >>>> +\t\t\t<< \"Unable to get frame interval on pad \" << pad\n> >>>> +\t\t\t<< \": \" << strerror(-ret);\n> >>>> +\t\treturn ret;\n> >>>> +\t}\n> >>>> +\n> >>>> +\tframeInterval->numerator = subdevFrameInterval.interval.numerator;\n> >>>> +\tframeInterval->denominator = subdevFrameInterval.interval.denominator;\n> >>>> +\n> >>>> +\treturn 0;\n> >>>> +}\n> >>>> +\n> >>>> +int V4L2Subdevice::setFrameInterval(unsigned int pad, Fraction *frameInterval)\n> >>>> +{\n> >>>> +\tstruct v4l2_subdev_frame_interval subdevFrameInterval = {};\n> >>>> +\tstruct v4l2_fract fract = { frameInterval->numerator,\n> >>>> +\t\t\t\t frameInterval->denominator };\n> >>>> +\n> >>>> +\tsubdevFrameInterval.pad = pad;\n> >>>> +\tsubdevFrameInterval.interval = fract;\n> >>>> +\n> >>>> +\tint ret = ioctl(VIDIOC_SUBDEV_S_FRAME_INTERVAL, &subdevFrameInterval);\n> >>>> +\tif (ret) {\n> >>>> +\t\tLOG(V4L2, Debug)\n> >>>> +\t\t\t<< \"Unable to set frame interval on pad \" << pad\n> >>>> +\t\t\t<< \": \" << strerror(-ret);\n> >>>> +\t\treturn ret;\n> >>>> +\t}\n> >>>> +\n> >>>> +\tframeInterval->numerator = subdevFrameInterval.interval.numerator;\n> >>>> +\tframeInterval->denominator = subdevFrameInterval.interval.denominator;\n> >>>> +\n> >>>> +\treturn 0;\n> >>>> +}\n> >>>> +\n> >>>> /**\n> >>>> * \\brief Create a new video subdevice instance from \\a entity in media device\n> >>>> * \\a media\n>","headers":{"Return-Path":"","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id CC454C32E1\n\tfor ;\n\tWed, 17 Mar 2021 14:07:18 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id 3BD9A68D62;\n\tWed, 17 Mar 2021 15:07:18 +0100 (CET)","from perceval.ideasonboard.com (perceval.ideasonboard.com\n\t[IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id 7B0EA605B1\n\tfor ;\n\tWed, 17 Mar 2021 15:07:16 +0100 (CET)","from pendragon.ideasonboard.com (62-78-145-57.bb.dnainternet.fi\n\t[62.78.145.57])\n\tby perceval.ideasonboard.com (Postfix) with ESMTPSA id E281845D;\n\tWed, 17 Mar 2021 15:07:15 +0100 (CET)"],"Authentication-Results":"lancelot.ideasonboard.com;\n\tdkim=fail reason=\"signature verification failed\" (1024-bit key;\n\tunprotected) header.d=ideasonboard.com header.i=@ideasonboard.com\n\theader.b=\"CdIoEdVf\"; dkim-atps=neutral","DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com;\n\ts=mail; t=1615990036;\n\tbh=Ne/A7lForN0FHwyqph7NxquA24PO530jAnxyCsdovJE=;\n\th=Date:From:To:Cc:Subject:References:In-Reply-To:From;\n\tb=CdIoEdVfpeB6vPh5nlACJYT0eIxGiJUlLVRoCzE/DkAOwBz/L2wBh5ktQbaNMxnEJ\n\tsGG+XAIl7INITYkNpmCUlfchq+a6A4BzrI4M3rVU9lJzmfs4eo2STVVsQvyEPnvEmj\n\tUpTbezTCT8K3eB3ZCCYnrYJrRmsUV1575sPjsmeE=","Date":"Wed, 17 Mar 2021 16:06:40 +0200","From":"Laurent Pinchart ","To":"Marian Cichy ","Message-ID":"","References":"<20210316155211.6679-1-m.cichy@pengutronix.de>\n\t<20210316155211.6679-4-m.cichy@pengutronix.de>\n\t\n\t<753330db-5b5d-fbd1-115e-9d8c6705e55c@pengutronix.de>\n\t\n\t<3f79d29a-3f6d-6a99-a646-d33bb49806d7@pengutronix.de>","MIME-Version":"1.0","Content-Disposition":"inline","In-Reply-To":"<3f79d29a-3f6d-6a99-a646-d33bb49806d7@pengutronix.de>","Subject":"Re: [libcamera-devel] [RFC PATCH 3/6] libcamera: v4l2_subdevice:\n\tAdd methods to get/set frame interval","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Cc":"libcamera-devel@lists.libcamera.org, graphics@pengutronix.de","Content-Type":"text/plain; charset=\"us-ascii\"","Content-Transfer-Encoding":"7bit","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" "}}]