[{"id":11340,"web_url":"https://patchwork.libcamera.org/comment/11340/","msgid":"<20200710154604.GD5964@pendragon.ideasonboard.com>","date":"2020-07-10T15:46:04","subject":"Re: [libcamera-devel] [PATCH 1/2] tests: v4l2_compat: Prettify\n\tfailure output","submitter":{"id":2,"url":"https://patchwork.libcamera.org/api/people/2/","name":"Laurent Pinchart","email":"laurent.pinchart@ideasonboard.com"},"content":"Hi Paul,\n\nThank you for the patch.\n\nOn Fri, Jul 10, 2020 at 11:15:07PM +0900, Paul Elder wrote:\n> When a test is being run on a video device, there is output like so:\n> \n> Testing /dev/video0 with uvcvideo driver...\n> \n> and when it succeeds, \"success\" is appended:\n> \n> Testing /dev/video0 with uvcvideo driver... success\n> \n> On failure, however, the output of v4l2-compliance is printed before\n> \"failure\" is printed, resulting in the first line of the v4l2-compliance\n> output to be printed on the same line as the message:\n> \n> Testing /dev/video2 with uvcvideo driver... v4l2-compliance SHA: not available, 64 bits\n> \n> failure\n> \n> Move printing of \"failure\" to the print_output_arr(), where the failed\n> test output is printed, to prettify the output upon failure.\n> \n> Signed-off-by: Paul Elder \n> ---\n> test/v4l2_compat/v4l2_compat_test.py | 2 +-\n> 1 file changed, 1 insertion(+), 1 deletion(-)\n> \n> diff --git a/test/v4l2_compat/v4l2_compat_test.py b/test/v4l2_compat/v4l2_compat_test.py\n> index 99b3d5a..8a8e59e 100755\n> --- a/test/v4l2_compat/v4l2_compat_test.py\n> +++ b/test/v4l2_compat/v4l2_compat_test.py\n> @@ -55,6 +55,7 @@ def extract_result(result):\n> \n> \n> def print_output_arr(output_arr):\n> + print('failed')\n> print('\\n'.join(output_arr))\n> \n> \n> @@ -135,7 +136,6 @@ def main(argv):\n> ret = test_v4l2_compliance(v4l2_compliance, v4l2_compat, device, driver)\n> if ret == TestFail:\n> failed.append(device)\n> - print('failed')\n> else:\n> print('success')\n\nI'm not a big fan of the resulting asymmetry. Would the following make\nsense (not fully tested) ?\n\ndiff --git a/test/v4l2_compat/v4l2_compat_test.py b/test/v4l2_compat/v4l2_compat_test.py\nindex 99b3d5a3e414..8aca22203e46 100755\n--- a/test/v4l2_compat/v4l2_compat_test.py\n+++ b/test/v4l2_compat/v4l2_compat_test.py\n@@ -54,32 +54,25 @@ def extract_result(result):\n return ret\n\n\n-def print_output_arr(output_arr):\n- print('\\n'.join(output_arr))\n-\n-\n def test_v4l2_compliance(v4l2_compliance, v4l2_compat, device, base_driver):\n ret, output = run_with_stdout(v4l2_compliance, '-s', '-d', device, env={'LD_PRELOAD': v4l2_compat})\n if ret < 0:\n- print_output_arr(output)\n- print(f'Test for {device} terminated due to signal {signal.Signals(-ret).name}')\n- return TestFail\n+ output.append(f'Test for {device} terminated due to signal {signal.Signals(-ret).name}')\n+ return TestFail, output\n\n result = extract_result(output[-2])\n if result['failed'] == 0:\n- return TestPass\n+ return TestPass, None\n\n # vimc will fail s_fmt because it only supports framesizes that are\n # multiples of 3\n if base_driver == 'vimc' and result['failed'] == 1:\n failures = grep('fail', output)\n if re.search('S_FMT cannot handle an invalid format', failures[0]) is None:\n- print_output_arr(output)\n- return TestFail\n- return TestPass\n+ return TestFail, output\n+ return TestPass, None\n\n- print_output_arr(output)\n- return TestFail\n+ return TestFail, output\n\n\n def main(argv):\n@@ -132,10 +125,11 @@ def main(argv):\n continue\n\n print(f'Testing {device} with {driver} driver... ', end='')\n- ret = test_v4l2_compliance(v4l2_compliance, v4l2_compat, device, driver)\n+ ret, msg = test_v4l2_compliance(v4l2_compliance, v4l2_compat, device, driver)\n if ret == TestFail:\n failed.append(device)\n print('failed')\n+ print('\\n'.join(msg))\n else:\n print('success')\n drivers_tested[driver] = True\n\n\n> drivers_tested[driver] = True","headers":{"Return-Path":"","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id F2C6EBD790\n\tfor ;\n\tFri, 10 Jul 2020 15:46:12 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id 90ACA6049A;\n\tFri, 10 Jul 2020 17:46:12 +0200 (CEST)","from perceval.ideasonboard.com (perceval.ideasonboard.com\n\t[IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id 309B96048F\n\tfor ;\n\tFri, 10 Jul 2020 17:46:11 +0200 (CEST)","from pendragon.ideasonboard.com (81-175-216-236.bb.dnainternet.fi\n\t[81.175.216.236])\n\tby perceval.ideasonboard.com (Postfix) with ESMTPSA id 9E35B2C0;\n\tFri, 10 Jul 2020 17:46:10 +0200 (CEST)"],"Authentication-Results":"lancelot.ideasonboard.com;\n\tdkim=fail reason=\"signature verification failed\" (1024-bit key;\n\tunprotected) header.d=ideasonboard.com header.i=@ideasonboard.com\n\theader.b=\"c/CfAo1Y\"; dkim-atps=neutral","DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com;\n\ts=mail; t=1594395970;\n\tbh=nKB95F/4a6QACRHdytxtGKegsQacoRSP4jWZXv/tmOM=;\n\th=Date:From:To:Cc:Subject:References:In-Reply-To:From;\n\tb=c/CfAo1YZq3p7+n+3FpN5oVgPqDNrvQcebCPBgQ1QgCNgm9fBWYWbq69YWTmiXJvT\n\tdHmYuV5mvtR0ycCg9L0CVNRM9hVY7gnYFePyO41bFkWWj4sGgnNSSvowOgdSZ1EMKK\n\tdOh57LD5GCa6rZVAr7gLwJ/HwYQKfmh6pxoxljEk=","Date":"Fri, 10 Jul 2020 18:46:04 +0300","From":"Laurent Pinchart ","To":"Paul Elder ","Message-ID":"<20200710154604.GD5964@pendragon.ideasonboard.com>","References":"<20200710141508.468834-1-paul.elder@ideasonboard.com>","MIME-Version":"1.0","Content-Disposition":"inline","In-Reply-To":"<20200710141508.468834-1-paul.elder@ideasonboard.com>","Subject":"Re: [libcamera-devel] [PATCH 1/2] tests: v4l2_compat: Prettify\n\tfailure output","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Cc":"libcamera-devel@lists.libcamera.org","Content-Type":"text/plain; charset=\"us-ascii\"","Content-Transfer-Encoding":"7bit","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" "}}]