From patchwork Tue Jul 19 07:58:17 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Umang Jain X-Patchwork-Id: 16681 X-Patchwork-Delegate: umang.jain@ideasonboard.com Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 554DEBD1F1 for ; Tue, 19 Jul 2022 07:58:31 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 1606863316; Tue, 19 Jul 2022 09:58:31 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1658217511; bh=jUULhcV8lDdhRTK48rcn6x/3aLIjh+YoUlTmj2AV7DI=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To: From; b=OLnCa38I9h+S2hoi0dwWgvM9OJot3+1EtEb6waTOOhLGsKaExHfhZq7LcDeai9DlJ +4V/7JmXiD8haAzQJZ9sG6w2/z4ZlPAd4vRjPQcGJsg+vnXrbBX/Vcznsrq2IIIptu 62DGDoeZs2nm00Os4wu2M1YjpSJqfbhUwTHhHpAl6ZR6ZbQ+gBby/iKgpxt8gloGqF 0BIzuTeSYf2sLZWxM+vh27jcsryOc9DwjYGp9lAsVKHwV6PSsNXcxR5A/4gyZV9YGX L++gUwsyVGD7mw5xRbOx9PZjjazpyGzpF1zZWw0Pgc6PqO2tyhdzimB88iNszXsfzb vxTlNaR1DEugw== Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 2B3D46330E for ; Tue, 19 Jul 2022 09:58:28 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="DAZ91eAw"; dkim-atps=neutral Received: from perceval.ideasonboard.com (unknown [IPv6:2401:4900:1f3f:73ce:9a9d:bd79:3fd0:39f1]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 3A6F36EE; Tue, 19 Jul 2022 09:58:27 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1658217507; bh=jUULhcV8lDdhRTK48rcn6x/3aLIjh+YoUlTmj2AV7DI=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=DAZ91eAwaVcrWvpDPjFUasIGCkjGDfjX4X8YfXXw5sLmd2ak0HSwbt2TDSdM5PGMO 8TJu2YCXPUGGNpHLjZypvRpx13534cyus2O1b+sHJS/BiK4c5JH2b+dkU9IXyfLbTi twkumqkntp8Kfr7mZy8IhvnWF8Bo7n47uRhGBH9Q= To: libcamera-devel@lists.libcamera.org Date: Tue, 19 Jul 2022 13:28:17 +0530 Message-Id: <20220719075818.895728-2-umang.jain@ideasonboard.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20220719075818.895728-1-umang.jain@ideasonboard.com> References: <20220719075818.895728-1-umang.jain@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v1 1/2] utils: ipu3-process: Stream multiple frames using v4l2-ctl X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Umang Jain via libcamera-devel From: Umang Jain Reply-To: Umang Jain Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Stream IMGU using v4l2-ctl as it provides more granular controls on the video nodes. The goal here is to stream a single input file containing multiple frames in 10-bit IPU3 packed bayer format. Such an input file can be created using the ipu3-pack utility. This patch enables the IMGU's parameter node as well in configure_pipeline() without which the input node won't stream successive input frames. yavta commands have been replaced with equivalent v4l2-ctl commands. Also the v4l2-ctl outputs a single file(--stream-to=) as opposed to yavta's one frame per file, so the conversion via raw2pnm needed adjusting. raw2pnm will now read from a single input file (produced as a result of v4l2-ctl ... --stream-to) and use the bytes offset to output separate frames. Lastly, $nbufs have reduced to 4 and $frame_count has been increased to 10. Signed-off-by: Umang Jain --- utils/ipu3/ipu3-process.sh | 54 +++++++++++++++++++++++++++----------- 1 file changed, 38 insertions(+), 16 deletions(-) diff --git a/utils/ipu3/ipu3-process.sh b/utils/ipu3/ipu3-process.sh index bb4abbe8..828a72f8 100755 --- a/utils/ipu3/ipu3-process.sh +++ b/utils/ipu3/ipu3-process.sh @@ -11,7 +11,7 @@ # # - media-ctl (from v4l-utils git://linuxtv.org/v4l-utils.git) # - raw2pnm (from nvt https://github.com/intel/nvt.git) -# - yavta (from git://git.ideasonboard.org/yavta.git) +# - v4l2-ctl (from http://git.linuxtv.org/v4l-utils.git) imgu_entity="ipu3-imgu 0" @@ -37,17 +37,19 @@ configure_pipeline() { local enable_3a=1 local enable_out=1 local enable_vf=1 + local enable_param=1 local mode=0 # Configure the links $mediactl -r $mediactl -l "\"$imgu_entity input\":0 -> \"$imgu_entity\":0[1]" + $mediactl -l "\"$imgu_entity parameters\":0 -> \"$imgu_entity\":1[$enable_param]" $mediactl -l "\"$imgu_entity\":2 -> \"$imgu_entity output\":0[$enable_out]" $mediactl -l "\"$imgu_entity\":3 -> \"$imgu_entity viewfinder\":0[$enable_vf]" $mediactl -l "\"$imgu_entity\":4 -> \"$imgu_entity 3a stat\":0[$enable_3a]" # Select processing mode (0 for video, 1 for still image) - yavta --no-query -w "0x009819c1 $mode" $($mediactl -e "$imgu_entity") + v4l2-ctl -d $($mediactl -e "$imgu_entity") -c 0x009819c1=$mode # Set formats. The media bus code doesn't matter as it is ignored by the # driver. We should use the FIXED format, but media-ctl doesn't support @@ -62,23 +64,41 @@ configure_pipeline() { process_frames() { configure_pipeline - local yavta="yavta -n $nbufs -c$frame_count" + local stream_setting=" --stream-mmap $nbufs --stream-count=$frame_count" + local out_width=$(echo $out_size | awk -F 'x' '{print $1}') + local out_height=$(echo $out_size | awk -F 'x' '{print $2}') + local vf_width=$(echo $vf_size | awk -F 'x' '{print $1}') + local vf_height=$(echo $vf_size | awk -F 'x' '{print $2}') + local in_width=$(echo $in_size | awk -F 'x' '{print $1}') + local in_height=$(echo $in_size | awk -F 'x' '{print $2}') # Save the main and viewfinder outputs to disk, capture and drop 3A - # statistics. Sleep 500ms between each execution of yavta to keep the + # statistics. Sleep 500ms between each execution of v4l2-ctl to keep the # stdout messages readable. - $yavta -f $IMGU_OUT_PIXELFORMAT -s $out_size "-F$output_dir/frame-out-#.bin" \ - $($mediactl -e "$imgu_entity output") & + v4l2-ctl -d$($mediactl -e "$imgu_entity output") \ + --set-fmt-video=pixelformat=$IMGU_OUT_PIXELFORMAT,width=$out_width,height=$out_height \ + $stream_setting --stream-to=$output_dir/frames-out.bin & sleep 0.5 - $yavta -f $IMGU_VF_PIXELFORMAT -s $vf_size "-F$output_dir/frame-vf-#.bin" \ - $($mediactl -e "$imgu_entity viewfinder") & + v4l2-ctl -d$($mediactl -e "$imgu_entity viewfinder") \ + --set-fmt-video=pixelformat=$IMGU_OUT_PIXELFORMAT,width=$vf_width,height=$vf_height \ + $stream_setting --stream-to=$output_dir/frames-vf.bin & sleep 0.5 - $yavta $($mediactl -e "$imgu_entity 3a stat") & + v4l2-ctl -d $($mediactl -e "$imgu_entity 3a stat") $stream_setting & sleep 0.5 # Feed the IMGU input. - $yavta -f $IMGU_IN_PIXELFORMAT -s $in_size "-F$in_file" \ - $($mediactl -e "$imgu_entity input") + + # To stream out $frame_count on output and vf nodes, input and parameters nodes need to atleast + # stream ($frame_count + $nbufs) frames. + local in_stream_count=$(expr $nbufs + $frame_count) + v4l2-ctl -d $($mediactl -e "$imgu_entity input") \ + --set-fmt-video-out=width=$in_width\,height=$in_height\,pixelformat=$IMGU_IN_PIXELFORMAT \ + --stream-out-mmap $nbufs --stream-from=$in_file --stream-count=$in_stream_count \ + --stream-loop --stream-poll & + + # The input node would not stream until parameters start streaming. + v4l2-ctl -d$($mediactl -e "$imgu_entity parameters") --stream-out-mmap 1 \ + --stream-count=$in_stream_count --stream-poll } # Convert captured files to ppm @@ -92,13 +112,15 @@ convert_files() { local height=$(echo $size | awk -F 'x' '{print $2}') local padded_width=$(expr $(expr $width + 63) / 64 \* 64) - raw2pnm -x$padded_width -y$height -f$format \ - $output_dir/frame-$type-$index.bin \ + local bytes_offset=$(expr $index \* $(expr $padded_width \* $height \* 12) / 8) + echo $bytes_offset + raw2pnm -x$padded_width -y$height -f$format -b $bytes_offset \ + $output_dir/frames-$type.bin \ $output_dir/frame-$type-$index.ppm } run_test() { - IMGU_IN_PIXELFORMAT=IPU3_SGRBG10 + IMGU_IN_PIXELFORMAT=ip3G IMGU_OUT_PIXELFORMAT=NV12 IMGU_VF_PIXELFORMAT=NV12 @@ -193,6 +215,6 @@ mediactl="media-ctl -d $mdev" echo "Using device $mdev" output_dir="/tmp" -frame_count=5 -nbufs=7 +frame_count=10 +nbufs=4 run_test From patchwork Tue Jul 19 07:58:18 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Umang Jain X-Patchwork-Id: 16682 X-Patchwork-Delegate: umang.jain@ideasonboard.com Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id ED769BD1F1 for ; Tue, 19 Jul 2022 07:58:32 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id A50BF63318; Tue, 19 Jul 2022 09:58:32 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1658217512; bh=ZhNavlpmdRSddSQo8QA2pmbNcVOml11Ur7hNgZEMLEM=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To: From; b=udZ8Yv+pQ2cUAEo2irXi9fNfGMZVEyHXrjc8e6Encv74kaqYsAB1nB8CZ4eNkGLDB 4+CV0BCvtaxBPxpXpTFcpZt/OGzuAcV3IwuTXBnOzEEj8GCvm/L/O+H/ymDitfSmt/ cCRuAD10QlkFXEjWwopfbOeAKOvVCo/316f75E9OcQCBQNj24+0NMpgDO93SQtbiT5 0qNK1DgGHsIgIyVS5PKLl90wfAPdyNDhTrtanIJwCUxCDdnccQKFg50dx9Cfi4fYT8 MnRoASZbKh7kXNnfWdCgk8EYTRKfSg6/iMIzOXJy9hMv5qj/Ol3fUks81bX40bJnik zD0kMt1nw+uwg== Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 3889A6330E for ; Tue, 19 Jul 2022 09:58:30 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="Cy5AMvPa"; dkim-atps=neutral Received: from perceval.ideasonboard.com (unknown [IPv6:2401:4900:1f3f:73ce:9a9d:bd79:3fd0:39f1]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 3249F6EE; Tue, 19 Jul 2022 09:58:29 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1658217510; bh=ZhNavlpmdRSddSQo8QA2pmbNcVOml11Ur7hNgZEMLEM=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=Cy5AMvPacqjB1xI8jFdaatu2Qh2tzldWHr51OysoeFT40p8KV30Luu6TXnVJyIgny jYG7ZCgGTKN6xZf8/w4/19AX4sCqg83lDbLR7zrkzNR3t+1d5iv7oCHvaHVTT1hM5S mXQ8PnO5/WG1PW93+PKGas6ajanlnVK1Us0ploek= To: libcamera-devel@lists.libcamera.org Date: Tue, 19 Jul 2022 13:28:18 +0530 Message-Id: <20220719075818.895728-3-umang.jain@ideasonboard.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20220719075818.895728-1-umang.jain@ideasonboard.com> References: <20220719075818.895728-1-umang.jain@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v1 2/2] utils: ipu3: Helper to generate IMGU input streaming file X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Umang Jain via libcamera-devel From: Umang Jain Reply-To: Umang Jain Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Usage: ipu3-gen-imgu-input.sh --count --size The input file is generated in 10-bit bayer which can be directly streamed to IMGU. Each frame (starting from 0) has a black background with a visible "Frame " label in the center. Signed-off-by: Umang Jain --- utils/ipu3/ipu3-gen-imgu-input.sh | 78 +++++++++++++++++++++++++++++++ 1 file changed, 78 insertions(+) create mode 100755 utils/ipu3/ipu3-gen-imgu-input.sh diff --git a/utils/ipu3/ipu3-gen-imgu-input.sh b/utils/ipu3/ipu3-gen-imgu-input.sh new file mode 100755 index 00000000..ea0d385d --- /dev/null +++ b/utils/ipu3/ipu3-gen-imgu-input.sh @@ -0,0 +1,78 @@ +#!/bin/bash +# SPDX-License-Identifier: GPL-2.0-or-later +# Copyright (C) 2022, Google Inc. +# +# Author: Umang Jain +# +# ipu3-gen-imgu-input.sh - Generate input file for IMGU streaming +# +# The scripts makes use of the following tools, which are expected to be +# found in $PATH: +# +# - raw2pnm (from nvt https://github.com/intel/nvt.git) +# - pnm2raw (from nvt https://github.com/intel/nvt.git) +# - ipu-pack (from https://git.libcamera.org/libcamera/libcamera.git/) +# - convert (from https://github.com/ImageMagick/ImageMagick/tree/main/utilities) + +#default frame count +frame_count=10 + +validate_size() { + local size=$1 + local width=$(echo $size | awk -F 'x' '{print $1}') + local height=$(echo $size | awk -F 'x' '{print $2}') + + [[ "x${size}" == "x${width}x${height}" ]] +} + +# Print usage message +usage() { + echo "Usage: $(basename $1) --count --size " + echo "Supported options:" + echo "--count count Generate input file containing frames, default=10" + echo "--size size Frame size ('width' x 'height')" + echo "" +} + +# Parse command line arguments +while (( "$#" )) ; do + case $1 in + --count) + frame_count=$2 + shift 2 + ;; + --size) + in_size=$2 + if ! validate_size $in_size ; then + echo "Invalid size '$in_size'" + usage $0 + exit 1 + fi + shift 2 + ;; + -*) + echo "Unsupported option $1" >&2 + usage $0 + exit 1 + ;; + *) + break + ;; + esac +done + +if [[ -z ${in_size+x} ]] ; then + usage $0 + exit 1 +fi + +echo "Generating Input file frame-$in_size.raw" +for i in `seq 0 $(expr $frame_count - 1)` +do + convert -background "#000000" -size $in_size -fill "#ffffff" -pointsize 120 -gravity center label:"Frame $i" frame-$i.ppm + pnm2raw frame-$i.ppm frame-$i.raw + ipu3-pack frame-$i.raw - >> frame-$in_size.raw + + rm frame-$i.ppm + rm frame-$i.raw +done