[{"id":13572,"web_url":"https://patchwork.libcamera.org/comment/13572/","msgid":"<20201103003052.GG30506@pendragon.ideasonboard.com>","date":"2020-11-03T00:30:52","subject":"Re: [libcamera-devel] [PATCH v4 4/5] utils: tracepoints: Add simple\n\tstatistics script","submitter":{"id":2,"url":"https://patchwork.libcamera.org/api/people/2/","name":"Laurent Pinchart","email":"laurent.pinchart@ideasonboard.com"},"content":"Hi Paul,\n\nThank you for the patch.\n\nOn Fri, Oct 30, 2020 at 05:57:55PM +0900, Paul Elder wrote:\n> Add a script that scans a trace for IPA call tracepoints, and returns\n> statistics on the time taken for IPA calls.\n> \n> Signed-off-by: Paul Elder \n> \n> ---\n> Changes in v4:\n> - rename script to analyze-ipa-trace.py\n> - align output stats table\n> \n> Changes in v3:\n> - check for the new tracepoint names, ipa_call_begin and ipa_call_end\n> - fix babeltrace2 parsing in the case that the event doesn't have a\n> pipeline_name field\n> - change script description\n> - add argparse description\n> - add example for trace_path argument\n> - change double quotes to single quotes\n> \n> New in v2\n> ---\n> utils/tracepoints/analyze-ipa-trace.py | 77 ++++++++++++++++++++++++++\n> 1 file changed, 77 insertions(+)\n> create mode 100755 utils/tracepoints/analyze-ipa-trace.py\n> \n> diff --git a/utils/tracepoints/analyze-ipa-trace.py b/utils/tracepoints/analyze-ipa-trace.py\n> new file mode 100755\n> index 00000000..50fbbf42\n> --- /dev/null\n> +++ b/utils/tracepoints/analyze-ipa-trace.py\n> @@ -0,0 +1,77 @@\n> +#!/usr/bin/env python3\n> +# SPDX-License-Identifier: GPL-2.0-or-later\n> +# Copyright (C) 2020, Google Inc.\n> +#\n> +# Author: Paul Elder \n> +#\n> +# analyze-ipa-trace.py - Example of how to extract information from libcamera lttng traces\n> +\n> +import argparse\n> +import bt2\n> +import statistics as stats\n> +import sys\n> +\n> +# pipeline -> {function -> stack(timestamps)}\n> +timestamps = {}\n> +\n> +# pipeline:function -> samples[]\n> +samples = {}\n> +\n> +def main(argv):\n> + parser = argparse.ArgumentParser(\n> + description='A simple analysis script to get statistics on time taken for IPA calls')\n> + parser.add_argument('-p', '--pipeline', type=str,\n> + help='Name of pipeline to filter for')\n> + parser.add_argument('trace_path', type=str,\n> + help='Path to lttng trace (eg. ~/lttng-traces/demo-20201029-184003)')\n> + args = parser.parse_args(argv[1:])\n> +\n> + traces = bt2.TraceCollectionMessageIterator(args.trace_path)\n> + for msg in traces:\n> + if type(msg) is not bt2._EventMessageConst or \\\n> + 'pipeline_name' not in msg.event.payload_field or \\\n> + (args.pipeline is not None and \\\n> + msg.event.payload_field['pipeline_name'] != args.pipeline):\n> + continue\n> +\n> + pipeline = msg.event.payload_field['pipeline_name']\n> + event = msg.event.name\n> + func = msg.event.payload_field['function_name']\n> + timestamp_ns = msg.default_clock_snapshot.ns_from_origin\n> +\n> + if event == 'libcamera:ipa_call_begin':\n> + if pipeline not in timestamps:\n> + timestamps[pipeline] = {}\n> + if func not in timestamps[pipeline]:\n> + timestamps[pipeline][func] = []\n> + timestamps[pipeline][func].append(timestamp_ns)\n> +\n> + if event == 'libcamera:ipa_call_end':\n> + ts = timestamps[pipeline][func].pop()\n> + key = f'{pipeline}:{func}'\n> + if key not in samples:\n> + samples[key] = []\n> + samples[key].append(timestamp_ns - ts)\n> +\n> + # Compute stats\n> + rows = []\n> + rows.append(['pipeline:function', 'min', 'max', 'mean', 'stddev'])\n> + for k, v in samples.items():\n> + mean = int(stats.mean(v))\n> + stddev = int(stats.stdev(v))\n> + minv = min(v)\n> + maxv = max(v)\n> + rows.append([k, str(minv), str(maxv), str(mean), str(stddev)])\n> +\n> + # Get maximum string width for every column\n> + widths = []\n> + for i in range(len(rows[0])):\n> + widths.append(max([len(row[i]) for row in rows]))\n\nYou can write this\n\n widths.append(max(len(row[i]) for row in rows))\n\nand python will create a generator for the expression passed to max(),\ninstead of creating an actual list.\n\nAnd I've now tried to measure the efficiency, and the generator is\nslower -_-'. Let's ignore this.\n\n> +\n> + # Print stats table\n> + for row in rows:\n> + fmt = [row[i].rjust(widths[i]) for i in range(1, 5)]\n> + print('{} {} {} {} {}'.format(row[0].ljust(widths[0]), *fmt))\n\nYou've made all the code here independent of the number of columns,\nexcept for the format string :-) No big deal though.\n\nReviewed-by: Laurent Pinchart \n\n> +\n> +if __name__ == '__main__':\n> + sys.exit(main(sys.argv))","headers":{"Return-Path":"","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id CDFB7BDB1E\n\tfor ;\n\tTue, 3 Nov 2020 00:31:42 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id 5A43A62B93;\n\tTue, 3 Nov 2020 01:31:42 +0100 (CET)","from perceval.ideasonboard.com (perceval.ideasonboard.com\n\t[IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id D463C62B90\n\tfor ;\n\tTue, 3 Nov 2020 01:31:40 +0100 (CET)","from pendragon.ideasonboard.com (62-78-145-57.bb.dnainternet.fi\n\t[62.78.145.57])\n\tby perceval.ideasonboard.com (Postfix) with ESMTPSA id 424E0332;\n\tTue, 3 Nov 2020 01:31:40 +0100 (CET)"],"Authentication-Results":"lancelot.ideasonboard.com;\n\tdkim=fail reason=\"signature verification failed\" (1024-bit key;\n\tunprotected) header.d=ideasonboard.com header.i=@ideasonboard.com\n\theader.b=\"LM2hY8dD\"; dkim-atps=neutral","DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com;\n\ts=mail; t=1604363500;\n\tbh=7iUQagttFkLW660hC2qmLi5AV0ciVVEIg7sqV27ngHY=;\n\th=Date:From:To:Cc:Subject:References:In-Reply-To:From;\n\tb=LM2hY8dD8sXKyuKnSxE46SmZKdJrorvg2x83WWdakbypeOiWqGf0ozZOgHBgHZHS2\n\t69gRSrjSsnc0JoQguxzWsuGm0dWEvNKHeRN8wfYc7ZJbcBgffOh5Wh8RP0viphx0ax\n\ts0qlyxELRPQedT9qMaIVa8+xUTQUCO2c1OuhNj5o=","Date":"Tue, 3 Nov 2020 02:30:52 +0200","From":"Laurent Pinchart ","To":"Paul Elder ","Message-ID":"<20201103003052.GG30506@pendragon.ideasonboard.com>","References":"<20201030085756.79329-1-paul.elder@ideasonboard.com>\n\t<20201030085756.79329-5-paul.elder@ideasonboard.com>","MIME-Version":"1.0","Content-Disposition":"inline","In-Reply-To":"<20201030085756.79329-5-paul.elder@ideasonboard.com>","Subject":"Re: [libcamera-devel] [PATCH v4 4/5] utils: tracepoints: Add simple\n\tstatistics script","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Cc":"libcamera-devel@lists.libcamera.org","Content-Type":"text/plain; charset=\"us-ascii\"","Content-Transfer-Encoding":"7bit","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" "}}]