[{"id":37259,"web_url":"https://patchwork.libcamera.org/comment/37259/","msgid":"","date":"2025-12-10T15:39:34","subject":"Re: [RFC PATCH v1 1/2] libcamera: pipeline: uvcvideo: Report\n\t`FrameDuration`","submitter":{"id":143,"url":"https://patchwork.libcamera.org/api/people/143/","name":"Jacopo Mondi","email":"jacopo.mondi@ideasonboard.com"},"content":"Hi Barnabás\n\nOn Wed, Dec 10, 2025 at 02:37:03PM +0100, Barnabás Pőcze wrote:\n> If available, query the time-per-frame parameter from the device after\n> starting, and report it in the metadata.\n>\n> The reported frame duration remains constant during a particular streaming\n> session, and will most likely not accurately reflect the actual frame\n> duration for any given frame, depending on the manual exposure time, or if\n> `V4L2_CID_EXPOSURE_AUTO_PRIORITY` or similar is in effect. But at least it\n> shows the \"intended\" frame duration.\n\nCould the value be refetched during streaming ?\n\n>\n> Signed-off-by: Barnabás Pőcze \n> ---\n> include/libcamera/internal/v4l2_videodevice.h | 2 +\n> src/libcamera/pipeline/uvcvideo/uvcvideo.cpp | 13 +++++\n> src/libcamera/v4l2_videodevice.cpp | 55 +++++++++++++++++++\n\nI would split the change in two, one for vdev and one for uvc\n\n> 3 files changed, 70 insertions(+)\n>\n> diff --git a/include/libcamera/internal/v4l2_videodevice.h b/include/libcamera/internal/v4l2_videodevice.h\n> index 57db0036db..82d98184ed 100644\n> --- a/include/libcamera/internal/v4l2_videodevice.h\n> +++ b/include/libcamera/internal/v4l2_videodevice.h\n> @@ -208,6 +208,8 @@ public:\n> \tint setFormat(V4L2DeviceFormat *format);\n> \tFormats formats(uint32_t code = 0);\n>\n> +\tint getFrameInterval(std::chrono::microseconds *interval);\n> +\n> \tint getSelection(unsigned int target, Rectangle *rect);\n> \tint setSelection(unsigned int target, Rectangle *rect);\n>\n> diff --git a/src/libcamera/pipeline/uvcvideo/uvcvideo.cpp b/src/libcamera/pipeline/uvcvideo/uvcvideo.cpp\n> index cb8cc82dff..3f98e8ece0 100644\n> --- a/src/libcamera/pipeline/uvcvideo/uvcvideo.cpp\n> +++ b/src/libcamera/pipeline/uvcvideo/uvcvideo.cpp\n> @@ -60,6 +60,7 @@ public:\n>\n> \tstd::optional autoExposureMode_;\n> \tstd::optional manualExposureMode_;\n> +\tstd::optional timePerFrame_;\n>\n> private:\n> \tbool generateId();\n> @@ -295,6 +296,8 @@ int PipelineHandlerUVC::start(Camera *camera, const ControlList *controls)\n> \tUVCCameraData *data = cameraData(camera);\n> \tunsigned int count = data->stream_.configuration().bufferCount;\n>\n> +\tdata->timePerFrame_.reset();\n> +\n> \tint ret = data->video_->importBuffers(count);\n> \tif (ret < 0)\n> \t\treturn ret;\n> @@ -309,6 +312,13 @@ int PipelineHandlerUVC::start(Camera *camera, const ControlList *controls)\n> \tif (ret < 0)\n> \t\tgoto err_release_buffers;\n>\n> +\tif (!data->timePerFrame_) {\n\nHaven't you just reset it here above ?\n\n> +\t\tstd::chrono::microseconds interval;\n> +\t\tret = data->video_->getFrameInterval(&interval);\n> +\t\tif (ret == 0)\n> +\t\t\tdata->timePerFrame_ = interval;\n> +\t}\n> +\n> \treturn 0;\n>\n> err_release_buffers:\n> @@ -898,6 +908,9 @@ void UVCCameraData::imageBufferReady(FrameBuffer *buffer)\n> \trequest->metadata().set(controls::SensorTimestamp,\n> \t\t\t\tbuffer->metadata().timestamp);\n>\n> +\tif (timePerFrame_)\n> +\t\trequest->metadata().set(controls::FrameDuration, timePerFrame_->count());\n> +\n> \tpipe()->completeBuffer(request, buffer);\n> \tpipe()->completeRequest(request);\n> }\n> diff --git a/src/libcamera/v4l2_videodevice.cpp b/src/libcamera/v4l2_videodevice.cpp\n> index 25b61d049a..3836dabef3 100644\n> --- a/src/libcamera/v4l2_videodevice.cpp\n> +++ b/src/libcamera/v4l2_videodevice.cpp\n> @@ -1147,6 +1147,61 @@ V4L2VideoDevice::Formats V4L2VideoDevice::formats(uint32_t code)\n> \treturn formats;\n> }\n>\n> +namespace {\n> +\n> +std::chrono::microseconds\n> +v4l2FractionToMs(const v4l2_fract &f)\n> +{\n> +\tauto seconds = std::chrono::duration(f.numerator) / f.denominator;\n> +\treturn std::chrono::duration_cast(seconds);\n> +}\n> +\n> +}\n> +\n> +/**\n> + * \\brief Retrieve the frame interval set on the V4L2 video device\n\nI think it would be nice to specify the unit in the function\ndocumentation (microseconds)\n\n> + * \\param[out] interval The frame interval applied on the device\n> + *\n> + * Retrieve the current time-per-frame parameter from the device.\n> + *\n> + * \\return 0 on success or a negative error code otherwise\n> + */\n> +int V4L2VideoDevice::getFrameInterval(std::chrono::microseconds *interval)\n> +{\n> +\tconst v4l2_fract *frameInterval = nullptr;\n> +\tv4l2_streamparm sparm = {};\n> +\tuint32_t caps = 0;\n> +\n> +\tsparm.type = bufferType_;\n> +\n> +\tint ret = ioctl(VIDIOC_G_PARM, &sparm);\n\nNow, this really is not an area I'm expert, so better check with\nKieran and Laurent on UVC, but to me G_PARM has always been a no-no.\nProbably because I always had to deal with complex devices where the\nduration wasn't set/retrieved from the video device but rather from\nother entities on the media graph. Maybe it is still legit for UVC and\nother videodev-centric platforms ?\n\n> +\tif (ret)\n> +\t\treturn ret;\n\nFor most if not all devices supported by libcamera, this will return\n-ENOTTY. In other words, I do expect uvc to be the only valid user of\nthis function. Do you think it's a concern ?\n\n\n> +\n> +\tswitch (sparm.type) {\n> +\tcase V4L2_BUF_TYPE_VIDEO_CAPTURE:\n> +\tcase V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE:\n> +\t\tframeInterval = &sparm.parm.capture.timeperframe;\n> +\t\tcaps = sparm.parm.capture.capability;\n> +\t\tbreak;\n> +\tcase V4L2_BUF_TYPE_VIDEO_OUTPUT:\n> +\tcase V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE:\n> +\t\tframeInterval = &sparm.parm.output.timeperframe;\n> +\t\tcaps = sparm.parm.output.capability;\n> +\t\tbreak;\n> +\t}\n> +\n> +\tif (!frameInterval)\n> +\t\treturn -EINVAL;\n> +\n> +\tif (!(caps & V4L2_CAP_TIMEPERFRAME))\n> +\t\treturn -ENOTSUP;\n\nShould we check this before even trying to ioctl() or is this only to\nsignal to userspace that time-per-frame can be changed ?\n\nI read in Documentation/userspace-api/media/v4l/vidioc-g-parm.rst\n\n * - ``V4L2_CAP_TIMEPERFRAME``\n - 0x1000\n - The frame period can be modified by setting the ``timeperframe``\n\tfield.\n\n> +\n> +\t*interval = v4l2FractionToMs(*frameInterval);\n> +\n> +\treturn 0;\n> +}\n> +\n> std::vector V4L2VideoDevice::enumPixelformats(uint32_t code)\n> {\n> \tstd::vector formats;\n> --\n> 2.52.0\n>","headers":{"Return-Path":"","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id 6E80DBD1F1\n\tfor ;\n\tWed, 10 Dec 2025 15:39:40 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id 9D82761491;\n\tWed, 10 Dec 2025 16:39:39 +0100 (CET)","from perceval.ideasonboard.com (perceval.ideasonboard.com\n\t[IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id A113761480\n\tfor ;\n\tWed, 10 Dec 2025 16:39:38 +0100 (CET)","from ideasonboard.com (mob-5-90-55-146.net.vodafone.it\n\t[5.90.55.146])\n\tby perceval.ideasonboard.com (Postfix) with ESMTPSA id D3E37667;\n\tWed, 10 Dec 2025 16:39:36 +0100 (CET)"],"Authentication-Results":"lancelot.ideasonboard.com; dkim=pass (1024-bit key;\n\tunprotected) header.d=ideasonboard.com header.i=@ideasonboard.com\n\theader.b=\"H5qCoROP\"; dkim-atps=neutral","DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com;\n\ts=mail; t=1765381176;\n\tbh=7gzp/xVEPELU5E1zqQubKfLOw9o+23F6I0XQl9nmBcY=;\n\th=Date:From:To:Cc:Subject:References:In-Reply-To:From;\n\tb=H5qCoROP+emWlhvCLlIBpgoJpwk9Xit0tr9m9UP2GpHYuLgQdI+nMc0JzNHB2iM33\n\tvlYJuHMQNi/nfNbobTu5IV8F2DQqjjmpQQ5uGp4TnCbGxMJBO68dXVI1hiZju31A4l\n\te2iQe739a46n9kbunfpm244jYgu4ZO3Nk2Mh6Ptc=","Date":"Wed, 10 Dec 2025 16:39:34 +0100","From":"Jacopo Mondi ","To":"=?utf-8?q?Barnab=C3=A1s_P=C5=91cze?= ","Cc":"libcamera-devel@lists.libcamera.org","Subject":"Re: [RFC PATCH v1 1/2] libcamera: pipeline: uvcvideo: Report\n\t`FrameDuration`","Message-ID":"","References":"<20251210133704.2711629-1-barnabas.pocze@ideasonboard.com>\n\t<20251210133704.2711629-2-barnabas.pocze@ideasonboard.com>","MIME-Version":"1.0","Content-Type":"text/plain; charset=utf-8","Content-Disposition":"inline","Content-Transfer-Encoding":"8bit","In-Reply-To":"<20251210133704.2711629-2-barnabas.pocze@ideasonboard.com>","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" "}},{"id":37268,"web_url":"https://patchwork.libcamera.org/comment/37268/","msgid":"","date":"2025-12-10T16:55:53","subject":"Re: [RFC PATCH v1 1/2] libcamera: pipeline: uvcvideo: Report\n\t`FrameDuration`","submitter":{"id":216,"url":"https://patchwork.libcamera.org/api/people/216/","name":"Barnabás Pőcze","email":"barnabas.pocze@ideasonboard.com"},"content":"Hi\n\n2025. 12. 10. 16:39 keltezéssel, Jacopo Mondi írta:\n> Hi Barnabás\n> \n> On Wed, Dec 10, 2025 at 02:37:03PM +0100, Barnabás Pőcze wrote:\n>> If available, query the time-per-frame parameter from the device after\n>> starting, and report it in the metadata.\n>>\n>> The reported frame duration remains constant during a particular streaming\n>> session, and will most likely not accurately reflect the actual frame\n>> duration for any given frame, depending on the manual exposure time, or if\n>> `V4L2_CID_EXPOSURE_AUTO_PRIORITY` or similar is in effect. But at least it\n>> shows the \"intended\" frame duration.\n> \n> Could the value be refetched during streaming ?\n\nAs far as I am aware this parameter is fixed for a given streaming session, so\nit would return the value that was set.\n\n\n> \n>>\n>> Signed-off-by: Barnabás Pőcze \n>> ---\n>> include/libcamera/internal/v4l2_videodevice.h | 2 +\n>> src/libcamera/pipeline/uvcvideo/uvcvideo.cpp | 13 +++++\n>> src/libcamera/v4l2_videodevice.cpp | 55 +++++++++++++++++++\n> \n> I would split the change in two, one for vdev and one for uvc\n\nI will try.\n\n\n> \n>> 3 files changed, 70 insertions(+)\n>>\n>> diff --git a/include/libcamera/internal/v4l2_videodevice.h b/include/libcamera/internal/v4l2_videodevice.h\n>> index 57db0036db..82d98184ed 100644\n>> --- a/include/libcamera/internal/v4l2_videodevice.h\n>> +++ b/include/libcamera/internal/v4l2_videodevice.h\n>> @@ -208,6 +208,8 @@ public:\n>> \tint setFormat(V4L2DeviceFormat *format);\n>> \tFormats formats(uint32_t code = 0);\n>>\n>> +\tint getFrameInterval(std::chrono::microseconds *interval);\n>> +\n>> \tint getSelection(unsigned int target, Rectangle *rect);\n>> \tint setSelection(unsigned int target, Rectangle *rect);\n>>\n>> diff --git a/src/libcamera/pipeline/uvcvideo/uvcvideo.cpp b/src/libcamera/pipeline/uvcvideo/uvcvideo.cpp\n>> index cb8cc82dff..3f98e8ece0 100644\n>> --- a/src/libcamera/pipeline/uvcvideo/uvcvideo.cpp\n>> +++ b/src/libcamera/pipeline/uvcvideo/uvcvideo.cpp\n>> @@ -60,6 +60,7 @@ public:\n>>\n>> \tstd::optional autoExposureMode_;\n>> \tstd::optional manualExposureMode_;\n>> +\tstd::optional timePerFrame_;\n>>\n>> private:\n>> \tbool generateId();\n>> @@ -295,6 +296,8 @@ int PipelineHandlerUVC::start(Camera *camera, const ControlList *controls)\n>> \tUVCCameraData *data = cameraData(camera);\n>> \tunsigned int count = data->stream_.configuration().bufferCount;\n>>\n>> +\tdata->timePerFrame_.reset();\n>> +\n>> \tint ret = data->video_->importBuffers(count);\n>> \tif (ret < 0)\n>> \t\treturn ret;\n>> @@ -309,6 +312,13 @@ int PipelineHandlerUVC::start(Camera *camera, const ControlList *controls)\n>> \tif (ret < 0)\n>> \t\tgoto err_release_buffers;\n>>\n>> +\tif (!data->timePerFrame_) {\n> \n> Haven't you just reset it here above ?\n\nThat's true, this only makes sense with the next patch.\n\n\n> \n>> +\t\tstd::chrono::microseconds interval;\n>> +\t\tret = data->video_->getFrameInterval(&interval);\n>> +\t\tif (ret == 0)\n>> +\t\t\tdata->timePerFrame_ = interval;\n>> +\t}\n>> +\n>> \treturn 0;\n>>\n>> err_release_buffers:\n>> @@ -898,6 +908,9 @@ void UVCCameraData::imageBufferReady(FrameBuffer *buffer)\n>> \trequest->metadata().set(controls::SensorTimestamp,\n>> \t\t\t\tbuffer->metadata().timestamp);\n>>\n>> +\tif (timePerFrame_)\n>> +\t\trequest->metadata().set(controls::FrameDuration, timePerFrame_->count());\n>> +\n>> \tpipe()->completeBuffer(request, buffer);\n>> \tpipe()->completeRequest(request);\n>> }\n>> diff --git a/src/libcamera/v4l2_videodevice.cpp b/src/libcamera/v4l2_videodevice.cpp\n>> index 25b61d049a..3836dabef3 100644\n>> --- a/src/libcamera/v4l2_videodevice.cpp\n>> +++ b/src/libcamera/v4l2_videodevice.cpp\n>> @@ -1147,6 +1147,61 @@ V4L2VideoDevice::Formats V4L2VideoDevice::formats(uint32_t code)\n>> \treturn formats;\n>> }\n>>\n>> +namespace {\n>> +\n>> +std::chrono::microseconds\n>> +v4l2FractionToMs(const v4l2_fract &f)\n>> +{\n>> +\tauto seconds = std::chrono::duration(f.numerator) / f.denominator;\n>> +\treturn std::chrono::duration_cast(seconds);\n>> +}\n>> +\n>> +}\n>> +\n>> +/**\n>> + * \\brief Retrieve the frame interval set on the V4L2 video device\n> \n> I think it would be nice to specify the unit in the function\n> documentation (microseconds)\n\nOk.\n\n> \n>> + * \\param[out] interval The frame interval applied on the device\n>> + *\n>> + * Retrieve the current time-per-frame parameter from the device.\n>> + *\n>> + * \\return 0 on success or a negative error code otherwise\n>> + */\n>> +int V4L2VideoDevice::getFrameInterval(std::chrono::microseconds *interval)\n>> +{\n>> +\tconst v4l2_fract *frameInterval = nullptr;\n>> +\tv4l2_streamparm sparm = {};\n>> +\tuint32_t caps = 0;\n>> +\n>> +\tsparm.type = bufferType_;\n>> +\n>> +\tint ret = ioctl(VIDIOC_G_PARM, &sparm);\n> \n> Now, this really is not an area I'm expert, so better check with\n> Kieran and Laurent on UVC, but to me G_PARM has always been a no-no.\n> Probably because I always had to deal with complex devices where the\n> duration wasn't set/retrieved from the video device but rather from\n> other entities on the media graph. Maybe it is still legit for UVC and\n> other videodev-centric platforms ?\n\nAs far as I'm aware this is the way for UVC cameras.\n\n\n> \n>> +\tif (ret)\n>> +\t\treturn ret;\n> \n> For most if not all devices supported by libcamera, this will return\n> -ENOTTY. In other words, I do expect uvc to be the only valid user of\n> this function. Do you think it's a concern ?\n\nI'm not too concerned, an argument could be made that this VIDIOC_{G,S}_PARM\nis a function video devices, so exposing it in the V4L2VideoDevice class does\nnot seem too bad to me.\n\n\n> \n> \n>> +\n>> +\tswitch (sparm.type) {\n>> +\tcase V4L2_BUF_TYPE_VIDEO_CAPTURE:\n>> +\tcase V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE:\n>> +\t\tframeInterval = &sparm.parm.capture.timeperframe;\n>> +\t\tcaps = sparm.parm.capture.capability;\n>> +\t\tbreak;\n>> +\tcase V4L2_BUF_TYPE_VIDEO_OUTPUT:\n>> +\tcase V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE:\n>> +\t\tframeInterval = &sparm.parm.output.timeperframe;\n>> +\t\tcaps = sparm.parm.output.capability;\n>> +\t\tbreak;\n>> +\t}\n>> +\n>> +\tif (!frameInterval)\n>> +\t\treturn -EINVAL;\n>> +\n>> +\tif (!(caps & V4L2_CAP_TIMEPERFRAME))\n>> +\t\treturn -ENOTSUP;\n> \n> Should we check this before even trying to ioctl() or is this only to\n> signal to userspace that time-per-frame can be changed ?\n\nI considered checking this capability in `V4L2VideoDevice::open()` and saving\nit into a member variable. That could be better I suppose, not sure.\n\n\n> \n> I read in Documentation/userspace-api/media/v4l/vidioc-g-parm.rst\n> \n> * - ``V4L2_CAP_TIMEPERFRAME``\n> - 0x1000\n> - The frame period can be modified by setting the ``timeperframe``\n> \tfield.\n> \n>> +\n>> +\t*interval = v4l2FractionToMs(*frameInterval);\n>> +\n>> +\treturn 0;\n>> +}\n>> +\n>> std::vector V4L2VideoDevice::enumPixelformats(uint32_t code)\n>> {\n>> \tstd::vector formats;\n>> --\n>> 2.52.0\n>>","headers":{"Return-Path":"","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id F2898C3257\n\tfor ;\n\tWed, 10 Dec 2025 16:55:55 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id AA695614BA;\n\tWed, 10 Dec 2025 17:55:55 +0100 (CET)","from perceval.ideasonboard.com (perceval.ideasonboard.com\n\t[IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id 095EA6149D\n\tfor ;\n\tWed, 10 Dec 2025 17:55:54 +0100 (CET)","from [192.168.33.37] (185.221.143.114.nat.pool.zt.hu\n\t[185.221.143.114])\n\tby perceval.ideasonboard.com (Postfix) with ESMTPSA id 9252410C4;\n\tWed, 10 Dec 2025 17:55:52 +0100 (CET)"],"Authentication-Results":"lancelot.ideasonboard.com; dkim=pass (1024-bit key;\n\tunprotected) header.d=ideasonboard.com header.i=@ideasonboard.com\n\theader.b=\"KwpxnqC2\"; dkim-atps=neutral","DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com;\n\ts=mail; t=1765385752;\n\tbh=UcMnHz0pMmCUTsJvIQ70duOgfrC627oboyU0LNRxx4U=;\n\th=Date:Subject:To:Cc:References:From:In-Reply-To:From;\n\tb=KwpxnqC2PRiP86+nl1b7uJzq0nwhWZRuarZinH+e/fXISsHmrY+pIJJT7Wfdj9BSL\n\tCtOUcsK2CyC9hohx6//S9+UwGOkU5gdsabNiQQ1W9FBrBcDS6mmda1kRC1bm/D/4Ci\n\t7One1a3IdfG/CtwqxjS3tnj7RtR1wSlGYn8QHexE=","Message-ID":"","Date":"Wed, 10 Dec 2025 17:55:53 +0100","MIME-Version":"1.0","User-Agent":"Mozilla Thunderbird","Subject":"Re: [RFC PATCH v1 1/2] libcamera: pipeline: uvcvideo: Report\n\t`FrameDuration`","To":"Jacopo Mondi ","Cc":"libcamera-devel@lists.libcamera.org","References":"<20251210133704.2711629-1-barnabas.pocze@ideasonboard.com>\n\t<20251210133704.2711629-2-barnabas.pocze@ideasonboard.com>\n\t","From":"=?utf-8?q?Barnab=C3=A1s_P=C5=91cze?= ","Content-Language":"en-US, hu-HU","In-Reply-To":"","Content-Type":"text/plain; charset=UTF-8; format=flowed","Content-Transfer-Encoding":"8bit","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" "}}]