[{"id":23821,"web_url":"https://patchwork.libcamera.org/comment/23821/","msgid":"","date":"2022-07-11T22:15:36","subject":"Re: [libcamera-devel] [RFC PATCH] utils: ipu3-process: Stream\n\tmultiple frames using v4l2-ctl","submitter":{"id":2,"url":"https://patchwork.libcamera.org/api/people/2/","name":"Laurent Pinchart","email":"laurent.pinchart@ideasonboard.com"},"content":"Hi Umang,\n\nThank you for the patch.\n\nOn Mon, Jul 11, 2022 at 10:14:13PM +0530, Umang Jain via libcamera-devel wrote:\n> Stream IMGU using v4l2-ctl as it provides more granular controls\n> on the video nodes. The goal here is to stream a single input\n> file containing multiple frames in 10-bit IPU3 packed bayer format.\n> Such a input file can be created using the ipu3-pack utility.\n\ns/a input/an input/\n\n> This patch enables the IMGU's parameter node as well in\n> configure_pipeline() without which, the input node won't stream\n\ns/ without which,/, without which/\n\n> successive input frames.\n> \n> The viewfinder and output nodes will still stream using yavta, so\n> no changes required there.\n\nWould it be difficult to use v4l2-ctl everywhere ? It would be nice to\navoid depending on multiple tools.\n\n> It's is quite tricky to stream input and parameters nodes synchronously\n> with v4l2-ctl commands for each. The patch overcomes this short-coming\n> by streaming both the nodes endlessly until both of them are killed.\n> A sleep of 1 second is introduced to make sure $frame_count buffers\n> are captured on the output and vf nodes.\n\nWhat's the issue, can't we get v4l2-ctl to stream a given number of\nframes ?\n\n> Another short-coming of this patch is when specific parameter-buffer(s)\n> (from a file) are needed to streamed on the parameters node, this might\n> require some adjusting. For now, the parameters node streams with\n> un-assigned buffers. It depends on the decision how this script is\n> expanded further down the line.\n\nAny proposal ? :-)\n\n> Signed-off-by: Umang Jain \n> ---\n> utils/ipu3/ipu3-process.sh | 29 ++++++++++++++++++++++++-----\n> 1 file changed, 24 insertions(+), 5 deletions(-)\n> \n> diff --git a/utils/ipu3/ipu3-process.sh b/utils/ipu3/ipu3-process.sh\n> index bb4abbe8..48466d70 100755\n> --- a/utils/ipu3/ipu3-process.sh\n> +++ b/utils/ipu3/ipu3-process.sh\n> @@ -37,11 +37,13 @@ configure_pipeline() {\n> \tlocal enable_3a=1\n> \tlocal enable_out=1\n> \tlocal enable_vf=1\n> +\tlocal enable_param=1\n> \tlocal mode=0\n> \n> \t# Configure the links\n> \t$mediactl -r\n> \t$mediactl -l \"\\\"$imgu_entity input\\\":0 -> \\\"$imgu_entity\\\":0[1]\"\n> +\t$mediactl -l \"\\\"$imgu_entity parameters\\\":0 -> \\\"$imgu_entity\\\":1[$enable_param]\"\n> \t$mediactl -l \"\\\"$imgu_entity\\\":2 -> \\\"$imgu_entity output\\\":0[$enable_out]\"\n> \t$mediactl -l \"\\\"$imgu_entity\\\":3 -> \\\"$imgu_entity viewfinder\\\":0[$enable_vf]\"\n> \t$mediactl -l \"\\\"$imgu_entity\\\":4 -> \\\"$imgu_entity 3a stat\\\":0[$enable_3a]\"\n> @@ -76,9 +78,26 @@ process_frames() {\n> \t$yavta $($mediactl -e \"$imgu_entity 3a stat\") &\n> \tsleep 0.5\n> \n> -\t# Feed the IMGU input.\n> -\t$yavta -f $IMGU_IN_PIXELFORMAT -s $in_size \"-F$in_file\" \\\n> -\t\t$($mediactl -e \"$imgu_entity input\")\n> +\techo \"Feeding IMGU $in_size\"\n> +\t# Start stream parameters node first\n> +\tv4l2-ctl -d$($mediactl -e \"$imgu_entity parameters\") --stream-out-mmap &\n> +\tlocal param_streaming=$!\n> +\n> +\t# Start streaming on input node\n> +\tlocal width=$(echo $in_size | awk -F 'x' '{print $1}')\n> +\tlocal height=$(echo $in_size | awk -F 'x' '{print $2}')\n> +\tv4l2-ctl -d $($mediactl -e \"$imgu_entity input\") \\\n> + --set-fmt-video-out=width=$width,height=$height,pixelformat=$IMGU_IN_PIXELFORMAT \\\n> + --stream-out-mmap --stream-from=$in_file --stream-loop &\n> +\tlocal input_streaming=$!\n> +\n> +\t# Sleep for 1 second and then kill the IMGU streaming. By then $frame_count output and vf\n> +\t# buffers should have been captured.\n> +\tsleep 1\n> +\tkill $input_streaming\n> +\tkill $param_streaming\n> +\n> +\techo \"IMGU FED\"\n> }\n> \n> # Convert captured files to ppm\n> @@ -98,7 +117,7 @@ convert_files() {\n> }\n> \n> run_test() {\n> -\tIMGU_IN_PIXELFORMAT=IPU3_SGRBG10\n> +\tIMGU_IN_PIXELFORMAT=ip3G\n> \tIMGU_OUT_PIXELFORMAT=NV12\n> \tIMGU_VF_PIXELFORMAT=NV12\n> \n> @@ -193,6 +212,6 @@ mediactl=\"media-ctl -d $mdev\"\n> echo \"Using device $mdev\"\n> \n> output_dir=\"/tmp\"\n> -frame_count=5\n> +frame_count=10\n> nbufs=7\n> run_test","headers":{"Return-Path":"","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id 360A8BE173\n\tfor ;\n\tMon, 11 Jul 2022 22:16:06 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id 90B3363312;\n\tTue, 12 Jul 2022 00:16:05 +0200 (CEST)","from perceval.ideasonboard.com (perceval.ideasonboard.com\n\t[213.167.242.64])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id F3C336042E\n\tfor ;\n\tTue, 12 Jul 2022 00:16:03 +0200 (CEST)","from pendragon.ideasonboard.com (62-78-145-57.bb.dnainternet.fi\n\t[62.78.145.57])\n\tby perceval.ideasonboard.com (Postfix) with ESMTPSA id 6B4AD2E4;\n\tTue, 12 Jul 2022 00:16:03 +0200 (CEST)"],"DKIM-Signature":["v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org;\n\ts=mail; t=1657577765;\n\tbh=nFTiVn2HihDkBs0TyI+2t0BnJqpqeg+klEaLilYDYVk=;\n\th=Date:To:References:In-Reply-To:Subject:List-Id:List-Unsubscribe:\n\tList-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To:Cc:\n\tFrom;\n\tb=OkLtQ+zwjc3xwoOf83FHxuKsRzPWSBUt/0yXdZw+p2UXzFlx5/D5aEQy5K3C3TmsM\n\tnSgCO9fPPkpX3oGp5CrQXvvUA/61KTIc6k7Ev39jDsPTgivxIMc5Q0a83DHDzDXxJ2\n\t5alN1cmpv0EK5+SofW15MjOfOPbLcKsXjVXk6vFiox/qTN/cOGluLklppTpHN49KG1\n\t6QMhuhnSmc2JYZ/jQHO6woW8EuX5TiN1hxmdosvPAcWNbIy3Z1Wti89CluTJtaIpMF\n\tduTD6a1Y28zQavbl0QGY1RWaENxktUdcNeXrt0OfozWSrDbNMCZs2SqENpyyI5gtG9\n\tT+uFqpmChVMYg==","v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com;\n\ts=mail; t=1657577763;\n\tbh=nFTiVn2HihDkBs0TyI+2t0BnJqpqeg+klEaLilYDYVk=;\n\th=Date:From:To:Cc:Subject:References:In-Reply-To:From;\n\tb=Ry3rXog9crnM0ELfw+lxbE70VFJzLUOcDp/lae6eGfobGlunWs6Gva1i4ObeLg+vk\n\t3Ar244IG+mtoT/LADptClJ0D9Fi9jQ/z/0sCA14PJ6EC4yEy/+EqYXxcVprSNkntPI\n\t5Rmg4NjXz2quGVb/jIehMorcddzTx3nNH4lgJApA="],"Authentication-Results":"lancelot.ideasonboard.com; dkim=pass (1024-bit key; \n\tunprotected) header.d=ideasonboard.com\n\theader.i=@ideasonboard.com\n\theader.b=\"Ry3rXog9\"; dkim-atps=neutral","Date":"Tue, 12 Jul 2022 01:15:36 +0300","To":"Umang Jain ","Message-ID":"","References":"<20220711164413.76229-1-umang.jain@ideasonboard.com>","MIME-Version":"1.0","Content-Type":"text/plain; charset=utf-8","Content-Disposition":"inline","In-Reply-To":"<20220711164413.76229-1-umang.jain@ideasonboard.com>","Subject":"Re: [libcamera-devel] [RFC PATCH] utils: ipu3-process: Stream\n\tmultiple frames using v4l2-ctl","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","From":"Laurent Pinchart via libcamera-devel\n\t","Reply-To":"Laurent Pinchart ","Cc":"libcamera-devel@lists.libcamera.org","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" "}},{"id":23828,"web_url":"https://patchwork.libcamera.org/comment/23828/","msgid":"","date":"2022-07-12T07:51:24","subject":"Re: [libcamera-devel] [RFC PATCH] utils: ipu3-process: Stream\n\tmultiple frames using v4l2-ctl","submitter":{"id":86,"url":"https://patchwork.libcamera.org/api/people/86/","name":"Umang Jain","email":"umang.jain@ideasonboard.com"},"content":"Hi Laurent,\n\nOn 7/12/22 03:45, Laurent Pinchart wrote:\n> Hi Umang,\n>\n> Thank you for the patch.\n>\n> On Mon, Jul 11, 2022 at 10:14:13PM +0530, Umang Jain via libcamera-devel wrote:\n>> Stream IMGU using v4l2-ctl as it provides more granular controls\n>> on the video nodes. The goal here is to stream a single input\n>> file containing multiple frames in 10-bit IPU3 packed bayer format.\n>> Such a input file can be created using the ipu3-pack utility.\n> s/a input/an input/\n>\n>> This patch enables the IMGU's parameter node as well in\n>> configure_pipeline() without which, the input node won't stream\n> s/ without which,/, without which/\n>\n>> successive input frames.\n>>\n>> The viewfinder and output nodes will still stream using yavta, so\n>> no changes required there.\n> Would it be difficult to use v4l2-ctl everywhere ? It would be nice to\n> avoid depending on multiple tools.\n\n\nOk, if you prefer not to use multiple tools, I can live with that. yavta \nseems to be more concise and clear to me so I preferred keeping those parts.\n\n>\n>> It's is quite tricky to stream input and parameters nodes synchronously\n>> with v4l2-ctl commands for each. The patch overcomes this short-coming\n>> by streaming both the nodes endlessly until both of them are killed.\n>> A sleep of 1 second is introduced to make sure $frame_count buffers\n>> are captured on the output and vf nodes.\n> What's the issue, can't we get v4l2-ctl to stream a given number of\n> frames ?\n\n\nSo the commands in the script runs sequentially so if you stream nodes \none after the other for a given set of frame_count - only (frame_count - \nX) frames get captured on output / vf nodes. So for frame_count=20, I \ngot somewhere between first 7-15 outputs only(this number various as \nwell). We had a brief discussion on this on IRC - seems whichever node \ncalls \"stream-off\" will make the entire script halt/waiting.\n\nIn my observation, the parameters node can stream independently. But the \ninput node, waits for parameter node to start streaming.\n\nI tried multiple combinations to capture frame_count deterministically \nbut didn't get much success, except the approach I adopted in this patch.\n\nI should have provided exact commands here with issues in each so I need \nto re-construct the scripts first and post the excerpts here. I'll play \naround with v4l2-ctl (since you prefer not having yavta+v4l2-ctl \ncombination) and see if I can replicate the goal as achieved here.\n\n>\n>> Another short-coming of this patch is when specific parameter-buffer(s)\n>> (from a file) are needed to streamed on the parameters node, this might\n>> require some adjusting. For now, the parameters node streams with\n>> un-assigned buffers. It depends on the decision how this script is\n>> expanded further down the line.\n> Any proposal ? :-)\n\n\nNot yet. I need to design a test case to develop it. Would you like to \nsuggest any?\n\n>\n>> Signed-off-by: Umang Jain \n>> ---\n>> utils/ipu3/ipu3-process.sh | 29 ++++++++++++++++++++++++-----\n>> 1 file changed, 24 insertions(+), 5 deletions(-)\n>>\n>> diff --git a/utils/ipu3/ipu3-process.sh b/utils/ipu3/ipu3-process.sh\n>> index bb4abbe8..48466d70 100755\n>> --- a/utils/ipu3/ipu3-process.sh\n>> +++ b/utils/ipu3/ipu3-process.sh\n>> @@ -37,11 +37,13 @@ configure_pipeline() {\n>> \tlocal enable_3a=1\n>> \tlocal enable_out=1\n>> \tlocal enable_vf=1\n>> +\tlocal enable_param=1\n>> \tlocal mode=0\n>> \n>> \t# Configure the links\n>> \t$mediactl -r\n>> \t$mediactl -l \"\\\"$imgu_entity input\\\":0 -> \\\"$imgu_entity\\\":0[1]\"\n>> +\t$mediactl -l \"\\\"$imgu_entity parameters\\\":0 -> \\\"$imgu_entity\\\":1[$enable_param]\"\n>> \t$mediactl -l \"\\\"$imgu_entity\\\":2 -> \\\"$imgu_entity output\\\":0[$enable_out]\"\n>> \t$mediactl -l \"\\\"$imgu_entity\\\":3 -> \\\"$imgu_entity viewfinder\\\":0[$enable_vf]\"\n>> \t$mediactl -l \"\\\"$imgu_entity\\\":4 -> \\\"$imgu_entity 3a stat\\\":0[$enable_3a]\"\n>> @@ -76,9 +78,26 @@ process_frames() {\n>> \t$yavta $($mediactl -e \"$imgu_entity 3a stat\") &\n>> \tsleep 0.5\n>> \n>> -\t# Feed the IMGU input.\n>> -\t$yavta -f $IMGU_IN_PIXELFORMAT -s $in_size \"-F$in_file\" \\\n>> -\t\t$($mediactl -e \"$imgu_entity input\")\n>> +\techo \"Feeding IMGU $in_size\"\n>> +\t# Start stream parameters node first\n>> +\tv4l2-ctl -d$($mediactl -e \"$imgu_entity parameters\") --stream-out-mmap &\n>> +\tlocal param_streaming=$!\n>> +\n>> +\t# Start streaming on input node\n>> +\tlocal width=$(echo $in_size | awk -F 'x' '{print $1}')\n>> +\tlocal height=$(echo $in_size | awk -F 'x' '{print $2}')\n>> +\tv4l2-ctl -d $($mediactl -e \"$imgu_entity input\") \\\n>> + --set-fmt-video-out=width=$width,height=$height,pixelformat=$IMGU_IN_PIXELFORMAT \\\n>> + --stream-out-mmap --stream-from=$in_file --stream-loop &\n>> +\tlocal input_streaming=$!\n>> +\n>> +\t# Sleep for 1 second and then kill the IMGU streaming. By then $frame_count output and vf\n>> +\t# buffers should have been captured.\n>> +\tsleep 1\n>> +\tkill $input_streaming\n>> +\tkill $param_streaming\n>> +\n>> +\techo \"IMGU FED\"\n>> }\n>> \n>> # Convert captured files to ppm\n>> @@ -98,7 +117,7 @@ convert_files() {\n>> }\n>> \n>> run_test() {\n>> -\tIMGU_IN_PIXELFORMAT=IPU3_SGRBG10\n>> +\tIMGU_IN_PIXELFORMAT=ip3G\n>> \tIMGU_OUT_PIXELFORMAT=NV12\n>> \tIMGU_VF_PIXELFORMAT=NV12\n>> \n>> @@ -193,6 +212,6 @@ mediactl=\"media-ctl -d $mdev\"\n>> echo \"Using device $mdev\"\n>> \n>> output_dir=\"/tmp\"\n>> -frame_count=5\n>> +frame_count=10\n>> nbufs=7\n>> run_test","headers":{"Return-Path":"","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id EB2B2BE173\n\tfor ;\n\tTue, 12 Jul 2022 07:51:31 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id 6609663312;\n\tTue, 12 Jul 2022 09:51:31 +0200 (CEST)","from perceval.ideasonboard.com (perceval.ideasonboard.com\n\t[213.167.242.64])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id 01B826330E\n\tfor ;\n\tTue, 12 Jul 2022 09:51:30 +0200 (CEST)","from [IPV6:2401:4900:1f3f:dcbd:bc88:bb0e:d3d8:8610] (unknown\n\t[IPv6:2401:4900:1f3f:dcbd:bc88:bb0e:d3d8:8610])\n\tby perceval.ideasonboard.com (Postfix) with ESMTPSA id 0C6FD30B;\n\tTue, 12 Jul 2022 09:51:28 +0200 (CEST)"],"DKIM-Signature":["v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org;\n\ts=mail; t=1657612291;\n\tbh=TcdMruTf5Xn5gaz7pr3oYbHiHRwxX9RA0a8pk2fnXPo=;\n\th=Date:To:References:In-Reply-To:Subject:List-Id:List-Unsubscribe:\n\tList-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To:Cc:\n\tFrom;\n\tb=FpIuH2yuc3D81cMoQobED9HrcNZe4XOCd57L1M8sH0PkYw45Tbr/kXvC+L8a7f5ky\n\tp9ox95YHbKCv1lZIyLm9L/PGOKQ3w0qhr8XuboIlBGgt6u972fOkeHCkvqfWrBIJx5\n\t4z/DYii87nSyWgS9xr4ry6tWV5i7PpI16Wb0kDiiQ0+z6riN6i9DZrZJd6erN3j1iv\n\t2bTbNFwoW/1mj0jmw5+8XN/5FiX7hYjXCrrlvtzyIuq4iU0x99AVpMH3QJpGN3BWxR\n\tZ9U0Bfw/t5NILpAq3663k7ytvE1Txwa7O9WmKc+exN5t6swVPbQscrK0AGm1C8ekRK\n\thrPH51TWyPUcw==","v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com;\n\ts=mail; t=1657612289;\n\tbh=TcdMruTf5Xn5gaz7pr3oYbHiHRwxX9RA0a8pk2fnXPo=;\n\th=Date:Subject:To:Cc:References:From:In-Reply-To:From;\n\tb=dlC9sgruO15OlDwz6CAXsHRlTQ6rkSjmkeHfPH5s3a68IZCMiHCSFKVwSG9b4Ty40\n\tmkPDzL5EalyIx55SR5Jif8q930fI+INEd7OANsIUTR6bk3zK5MHLq3sFXZPZfUu/Et\n\tR7iVFMLvLzWHzdUAIa1NWhvLR/fqP0Dd2oiUM4e0="],"Authentication-Results":"lancelot.ideasonboard.com; dkim=pass (1024-bit key; \n\tunprotected) header.d=ideasonboard.com\n\theader.i=@ideasonboard.com\n\theader.b=\"dlC9sgru\"; dkim-atps=neutral","Message-ID":"","Date":"Tue, 12 Jul 2022 13:21:24 +0530","MIME-Version":"1.0","User-Agent":"Mozilla/5.0 (X11; Linux x86_64; rv:91.0) Gecko/20100101\n\tThunderbird/91.4.1","Content-Language":"en-US","To":"Laurent Pinchart ","References":"<20220711164413.76229-1-umang.jain@ideasonboard.com>\n\t","In-Reply-To":"","Content-Type":"text/plain; charset=UTF-8; format=flowed","Content-Transfer-Encoding":"7bit","Subject":"Re: [libcamera-devel] [RFC PATCH] utils: ipu3-process: Stream\n\tmultiple frames using v4l2-ctl","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","From":"Umang Jain via libcamera-devel ","Reply-To":"Umang Jain ","Cc":"libcamera-devel@lists.libcamera.org","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" "}}]