[{"id":13562,"web_url":"https://patchwork.libcamera.org/comment/13562/","msgid":"<20201030011949.GP15024@pendragon.ideasonboard.com>","date":"2020-10-30T01:19:49","subject":"Re: [libcamera-devel] [PATCH v3 6/6] Documentation: tracing: Add\n\ttracing guide","submitter":{"id":2,"url":"https://patchwork.libcamera.org/api/people/2/","name":"Laurent Pinchart","email":"laurent.pinchart@ideasonboard.com"},"content":"Hi Paul,\n\nThank you for the patch.\n\nOn Thu, Oct 29, 2020 at 07:16:29PM +0900, Paul Elder wrote:\n> Add guide for tracepoints, including how to define and use them.\n> \n> Signed-off-by: Paul Elder \n> \n> ---\n> Changes in v3:\n> - fix compilation error\n> - reword english\n> - add stuff about the new IPA call tracepoint macros\n> \n> New in v2\n> ---\n> Documentation/guides/tracing.rst | 140 +++++++++++++++++++++++++++++++\n> Documentation/index.rst | 1 +\n> Documentation/meson.build | 1 +\n> 3 files changed, 142 insertions(+)\n> create mode 100644 Documentation/guides/tracing.rst\n> \n> diff --git a/Documentation/guides/tracing.rst b/Documentation/guides/tracing.rst\n> new file mode 100644\n> index 00000000..8c59e7ed\n> --- /dev/null\n> +++ b/Documentation/guides/tracing.rst\n> @@ -0,0 +1,140 @@\n> +.. SPDX-License-Identifier: CC-BY-SA-4.0\n> +\n> +Tracing Guide\n> +=============\n> +\n> +Guide to tracing in libcamera.\n> +\n> +Profiling vs Tracing\n> +--------------------\n> +\n> +Tracing is recording timestamps at specific locations. libcamera provides a\n> +tracing facility. This guide shows how to use this tracing facility.\n> +\n> +Tracing should not be confused with profiling, which samples execution\n> +at periodic points in time. This can be done with other tools such as\n> +callgrind, perf, gprof, etc., without modification to the application,\n> +and is out of scope for this guide.\n> +\n> +Compiling\n> +---------\n> +\n> +To compile libcamera with tracing support, it must be enabled through the\n> +meson ``tracing`` option. It depends on the lttng-ust library (available in the\n> +``liblttng-ust-dev`` package for Debian-based distributions).\n> +By default the tracing option in meson is set to ``auto``, so if\n> +liblttng is detected, it will be enabled by default. Conversely, if the option\n> +is set to disabled, then libcamera will be compiled without tracing support.\n> +\n> +Defining tracepoints\n> +--------------------\n> +\n> +The first of two steps to using tracepoints is to define the tracepoints.\n> +\n\nHow about adding here\n\n\"libcamera already contains a set of tracepoints. To define additional\ntracepoints, create ...\"\n\n?\n\n> +Create a file ``include/libcamera/internal/tracepoints/{file}.tp``, where\n> +``file`` is a reasonable name related to the category of tracepoints that\n> +you wish to define. For example, a tracepoints file for the Request object\n\ns/a tracepoints/the tracepoints/\n\n> +would be called ``request.tp``. An entry for this file must be added in\n\ns/would be/is/\n\nas it already exists.\n\n> +``include/libcamera/internal/tracepoints/meson.build``.\n> +\n> +In this tracepoints file, define your tracepoints `as mandated by lttng\n> +`_. The header boilerplate must *not* be\n> +included (as it will conflict with the rest of our infrastructure), and\n> +only the tracepoint definitions (with the ``TRACEPOINT_*`` macros) should be\n> +included.\n> +\n> +All tracepoint providers shall be ``libcamera``. According to lttng, the\n> +tracepoint provider should be per-project; this is the rationale for this\n> +decision. To group tracepoint events, we recommend using\n> +``{class_name}_{tracepoint_name}``, for example, ``request_construct`` for a\n> +tracepoint for the constructor of the Request class.\n> +\n> +Tracepoint arguments may take C++ objects pointers, in which case the usual\n> +C++ namespacing rules apply. The header that contains the necessary class\n> +definitions must be included at the top of the tracepoint provider file.\n> +\n> +Note: the final parameter in ``TP_ARGS`` *must not* have a trailing comma, and\n> +the parameters to ``TP_FIELDS`` are *space-separated*. Not following these will\n> +cause compilation errors.\n> +\n> +Using tracepoints (in libcamera)\n> +--------------------------------\n> +\n> +To use tracepoints in libcamera, first the header needs to be included:\n> +\n> +``#include \"libcamera/internal/tracepoints.h\"``\n> +\n> +Then to use the tracepoint:\n> +\n> +``LIBCAMERA_TRACEPOINT({tracepoint_event}, args...)``\n> +\n> +This macro must be used, as opposed to lttng's macros directly, because\n> +lttng is an optional dependency of libcamera, so the code must compile and run\n> +even when lttng is not present or when tracing is disabled.\n> +\n> +The tracepoint provider name, as declared in the tracepoint definition, is not\n> +included in the parameters of the tracepoint.\n> +\n> +There are also two special tracepoints available for tracing IPA calls:\n> +\n> +``LIBCAMERA_TRACEPOINT_IPA_BEGIN({pipeline_name}, {ipa_function})``\n> +\n> +``LIBCAMERA_TRACEPOINT_IPA_END({pipeline_name}, {ipa_function})``\n> +\n> +These shall be placed where an IPA function is called from the pipeline handler,\n> +and when the pipeline handler receives the corresponding response from the IPA,\n> +respecively. These are the tracepoints that our sample analysis script\n\ns/respecively/respectively/\n\n> +(see \"Analyzing a trace\") scans for when computing statistics on IPA call time.\n> +\n> +Using tracepoints (from an application)\n> +---------------------------------------\n> +\n> +As applications are not part of libcamera, but rather users of libcamera,\n> +applications should seek their own tracing mechanisms. For ease of tracing\n> +the application alongside tracing libcamera, it is recommended to also\n> +`use lttng `_.\n> +\n> +Using tracepoints (from closed-source IPA)\n> +------------------------------------------\n> +\n> +Similar to applications, closed-source IPAs can simply use lttng on their own,\n> +or any other tracing mechanism if desired.\n> +\n> +Collecting a trace\n> +------------------\n> +\n> +A trace can be collected fairly simply from lttng:\n> +\n> +.. code-block:: bash\n> +\n> + lttng create $SESSION_NAME\n> + lttng enable-event -u libcamera:\\*\n> + lttng start\n> + # run libcamera application\n> + lttng stop\n> + lttng view\n> + lttng destroy $SESSION_NAME\n> +\n> +See the `lttng documentation `_ for further details.\n> +\n> +Analyzing a trace\n> +-----------------\n> +\n> +As mentioned above, while an lttng tracing session exists and the trace is not\n> +running, the trace output can be viewd as text by ``lttng view``.\n\ns/viewd/viewed/\n\n> +\n> +The trace log can also be viewed as text using babeltrace2. See the\n> +`lttng trace analysis documentation `_\n> +for further details.\n> +\n> +babeltrace2 also has a C API and python bindings that can be used to process\n> +traces. See the\n> +`lttng python bindings documentation `_\n\nCan't you break lines in the middle of the link name to make the reflow\na bit nicer ? It's one once above so I assume it should work.\n\n> +and the\n> +`lttng C API documentation `_\n> +for more details.\n> +\n> +As an example, there is a script ``utils/tracepoints/analyze.py`` that gathers\n\nDon't forget to update the script name here if you rename it as proposed\nin the review of patch 5/6.\n\nReviewed-by: Laurent Pinchart \n\n> +statistics for the time taken for an IPA function call, by measuring the time\n> +difference between pairs of events ``libcamera:ipa_call_start`` and\n> +``libcamera:ipa_call_finish``.\n> diff --git a/Documentation/index.rst b/Documentation/index.rst\n> index 173c326f..8bc8922e 100644\n> --- a/Documentation/index.rst\n> +++ b/Documentation/index.rst\n> @@ -17,3 +17,4 @@\n> Application Writer's Guide \n> Pipeline Handler Writer's Guide \n> IPA Writer's guide \n> + Tracing guide \n> diff --git a/Documentation/meson.build b/Documentation/meson.build\n> index f2300dac..17f3b9d7 100644\n> --- a/Documentation/meson.build\n> +++ b/Documentation/meson.build\n> @@ -55,6 +55,7 @@ if sphinx.found()\n> 'guides/ipa.rst',\n> 'guides/application-developer.rst',\n> 'guides/pipeline-handler.rst',\n> + 'guides/tracing.rst',\n> ]\n> \n> release = 'release=v' + libcamera_git_version","headers":{"Return-Path":"","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id CB4D6BDB9B\n\tfor ;\n\tFri, 30 Oct 2020 01:20:40 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id 5F530628F5;\n\tFri, 30 Oct 2020 02:20:40 +0100 (CET)","from perceval.ideasonboard.com (perceval.ideasonboard.com\n\t[IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id 1F7CA60349\n\tfor ;\n\tFri, 30 Oct 2020 02:20:39 +0100 (CET)","from pendragon.ideasonboard.com (62-78-145-57.bb.dnainternet.fi\n\t[62.78.145.57])\n\tby perceval.ideasonboard.com (Postfix) with ESMTPSA id 885E69B7;\n\tFri, 30 Oct 2020 02:20:38 +0100 (CET)"],"Authentication-Results":"lancelot.ideasonboard.com;\n\tdkim=fail reason=\"signature verification failed\" (1024-bit key;\n\tunprotected) header.d=ideasonboard.com header.i=@ideasonboard.com\n\theader.b=\"gI1AnlC9\"; dkim-atps=neutral","DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com;\n\ts=mail; t=1604020838;\n\tbh=7rsQHYFes4tVv+9os1O4QL0naXSrVPRaNrxxVfcGM40=;\n\th=Date:From:To:Cc:Subject:References:In-Reply-To:From;\n\tb=gI1AnlC9QvSjEJDlFkN5/IR7WMCzo2mFRcj7LwVIiJyrloz7109cRgYxakWqOB3w8\n\tar2GDXHq3ERH33aY6pgL0rytGgN42lYdqO8tF8rSBFYarZJ/NWmQU3jD/1B4niESkF\n\t4IdVcOVS/lYp0wwa7AO22/vWaXqdwr6mc6jet5+8=","Date":"Fri, 30 Oct 2020 03:19:49 +0200","From":"Laurent Pinchart ","To":"Paul Elder ","Message-ID":"<20201030011949.GP15024@pendragon.ideasonboard.com>","References":"<20201029101629.61798-1-paul.elder@ideasonboard.com>\n\t<20201029101629.61798-7-paul.elder@ideasonboard.com>","MIME-Version":"1.0","Content-Disposition":"inline","In-Reply-To":"<20201029101629.61798-7-paul.elder@ideasonboard.com>","Subject":"Re: [libcamera-devel] [PATCH v3 6/6] Documentation: tracing: Add\n\ttracing guide","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Cc":"libcamera-devel@lists.libcamera.org","Content-Type":"text/plain; charset=\"us-ascii\"","Content-Transfer-Encoding":"7bit","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" "}}]