From patchwork Fri Oct 30 08:57:52 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Paul Elder X-Patchwork-Id: 10306 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 17C03BDB1E for ; Fri, 30 Oct 2020 08:58:08 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id D9DC062983; Fri, 30 Oct 2020 09:58:07 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="u55kHE0h"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 9F9E5628A5 for ; Fri, 30 Oct 2020 09:58:06 +0100 (CET) Received: from pyrite.rasen.tech (unknown [IPv6:2400:4051:61:600:2c71:1b79:d06d:5032]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 866849B7; Fri, 30 Oct 2020 09:58:04 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1604048286; bh=F4kQeRRzyycG1YWcGjo52mW1fxnD5O2thVzGFsuAGSk=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=u55kHE0hRr0RcC7PSkTCRZenXdsFV8Vz5qlfMC9HfHy3vEJy/ON12Frr+r24kaU7H RwzZgZ6TboLVKe9pVa1uiV/o+WrFj2079VgZUAH4VzVMSXh6E9q7nQAJxBcCBw9/Pe JGtW19vFTdNfPfwqDRF0reeahqw/k4RcZ1C3mT0k= From: Paul Elder To: libcamera-devel@lists.libcamera.org Date: Fri, 30 Oct 2020 17:57:52 +0900 Message-Id: <20201030085756.79329-2-paul.elder@ideasonboard.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20201030085756.79329-1-paul.elder@ideasonboard.com> References: <20201030085756.79329-1-paul.elder@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v4 1/5] libcamera: tracing: Implement tracing infrastructure X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Implement tracing infrastructure in libcamera. It takes .tp files, as required by lttng, and generates a tracepoint header and C file, as lttng requires. meson is updated accordingly to get it to compile with the rest of libcamera. Update the documentation accordingly. Signed-off-by: Paul Elder Reviewed-by: Laurent Pinchart Reviewed-by: Kieran Bingham --- Changes in v4: - simplify no-tracing tracepoint macros - make gen-tp-header.py open *all* files as utf-8 - rebase on master instead of ipc Changes in v3: - rename the tracepoints header generateor in meson - add protection against accidental unused variables when tracing is disabled - make gen-tp-header.py read files as utf-8 - add macros LIBCAMERA_TRACEPOINT_IPA_{BEGIN,END} Changes in v2: - move header.tmpl to tracepoints.h.in - rename meson option 'tracepoints' to 'tracing' - make sure C++ objects work in tracepoint arguments - rename generate_header.py to gen-tp-header.py --- Documentation/Doxyfile.in | 4 +- README.rst | 3 + include/libcamera/internal/meson.build | 9 +++ include/libcamera/internal/tracepoints.h.in | 61 +++++++++++++++++++ .../internal/tracepoints/meson.build | 4 ++ meson.build | 3 + meson_options.txt | 5 ++ src/libcamera/meson.build | 7 +++ src/libcamera/tracepoints.cpp | 10 +++ utils/meson.build | 1 + utils/tracepoints/gen-tp-header.py | 38 ++++++++++++ utils/tracepoints/meson.build | 5 ++ 12 files changed, 149 insertions(+), 1 deletion(-) create mode 100644 include/libcamera/internal/tracepoints.h.in create mode 100644 include/libcamera/internal/tracepoints/meson.build create mode 100644 src/libcamera/tracepoints.cpp create mode 100644 utils/tracepoints/gen-tp-header.py create mode 100644 utils/tracepoints/meson.build diff --git a/Documentation/Doxyfile.in b/Documentation/Doxyfile.in index fa8d744b..87959e23 100644 --- a/Documentation/Doxyfile.in +++ b/Documentation/Doxyfile.in @@ -840,7 +840,9 @@ EXCLUDE = @TOP_SRCDIR@/include/libcamera/span.h \ @TOP_SRCDIR@/src/libcamera/device_enumerator_sysfs.cpp \ @TOP_SRCDIR@/src/libcamera/device_enumerator_udev.cpp \ @TOP_SRCDIR@/src/libcamera/pipeline/ \ - @TOP_SRCDIR@/src/libcamera/proxy/ + @TOP_SRCDIR@/src/libcamera/proxy/ \ + @TOP_SRCDIR@/src/libcamera/tracepoints.cpp \ + @TOP_BUILDDIR@/include/libcamera/internal/tracepoints.h # The EXCLUDE_SYMLINKS tag can be used to select whether or not files or # directories that are symbolic links (a Unix file system feature) are excluded diff --git a/README.rst b/README.rst index 74babaf7..f8519b44 100644 --- a/README.rst +++ b/README.rst @@ -81,6 +81,9 @@ for gstreamer: [optional] for qcam: [optional] qtbase5-dev libqt5core5a libqt5gui5 libqt5widgets5 qttools5-dev-tools libtiff-dev +for tracing with lttng: [optional] + lttng-ust-dev python3-jinja2 + Using GStreamer plugin ~~~~~~~~~~~~~~~~~~~~~~ diff --git a/include/libcamera/internal/meson.build b/include/libcamera/internal/meson.build index 15010338..6500fe2a 100644 --- a/include/libcamera/internal/meson.build +++ b/include/libcamera/internal/meson.build @@ -1,5 +1,14 @@ # SPDX-License-Identifier: CC0-1.0 +subdir('tracepoints') + +libcamera_tracepoint_header = custom_target( + 'tp_header', + input: [ 'tracepoints.h.in', tracepoint_files ], + output: 'tracepoints.h', + command: [ gen_tracepoints_header, '@OUTPUT@', '@INPUT@' ], +) + libcamera_internal_headers = files([ 'byte_stream_buffer.h', 'camera_controls.h', diff --git a/include/libcamera/internal/tracepoints.h.in b/include/libcamera/internal/tracepoints.h.in new file mode 100644 index 00000000..d0fc1365 --- /dev/null +++ b/include/libcamera/internal/tracepoints.h.in @@ -0,0 +1,61 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) {{year}}, Google Inc. + * + * tracepoints.h - Tracepoints with lttng + * + * This file is auto-generated. Do not edit. + */ +#ifndef __LIBCAMERA_INTERNAL_TRACEPOINTS_H__ +#define __LIBCAMERA_INTERNAL_TRACEPOINTS_H__ + +#if HAVE_TRACING +#define LIBCAMERA_TRACEPOINT(...) tracepoint(libcamera, __VA_ARGS__) + +#define LIBCAMERA_TRACEPOINT_IPA_BEGIN(pipe, func) \ +tracepoint(libcamera, ipa_call_begin, #pipe, #func) + +#define LIBCAMERA_TRACEPOINT_IPA_END(pipe, func) \ +tracepoint(libcamera, ipa_call_end, #pipe, #func) + +#else + +namespace { + +template +inline void unused([[maybe_unused]] Args&& ...args) +{ +} + +} /* namespace */ + +#define LIBCAMERA_TRACEPOINT(category, ...) unused(__VA_ARGS__) + +#define LIBCAMERA_TRACEPOINT_IPA_BEGIN(pipe, func) +#define LIBCAMERA_TRACEPOINT_IPA_END(pipe, func) + +#endif /* HAVE_TRACING */ + +#endif /* __LIBCAMERA_INTERNAL_TRACEPOINTS_H__ */ + + +#if HAVE_TRACING + +#undef TRACEPOINT_PROVIDER +#define TRACEPOINT_PROVIDER libcamera + +#undef TRACEPOINT_INCLUDE +#define TRACEPOINT_INCLUDE "{{path}}" + +#if !defined(INCLUDE_LIBCAMERA_INTERNAL_TRACEPOINTS_TP_H) || defined(TRACEPOINT_HEADER_MULTI_READ) +#define INCLUDE_LIBCAMERA_INTERNAL_TRACEPOINTS_TP_H + +#include + +{{source}} + +#endif /* INCLUDE_LIBCAMERA_INTERNAL_TRACEPOINTS_TP_H */ + +#include + +#endif /* HAVE_TRACING */ diff --git a/include/libcamera/internal/tracepoints/meson.build b/include/libcamera/internal/tracepoints/meson.build new file mode 100644 index 00000000..2dd6733e --- /dev/null +++ b/include/libcamera/internal/tracepoints/meson.build @@ -0,0 +1,4 @@ +# SPDX-License-Identifier: CC0-1.0 + +tracepoint_files = files([ +]) diff --git a/meson.build b/meson.build index e6f6c84a..55cf36e1 100644 --- a/meson.build +++ b/meson.build @@ -108,6 +108,9 @@ libcamera_includes = include_directories('include') # Sub-directories fill py_modules with their dependencies. py_modules = [] +# Libraries used by multiple components +liblttng = cc.find_library('lttng-ust', required : get_option('tracing')) + # Utilities are parsed first to provide support for other components. subdir('utils') diff --git a/meson_options.txt b/meson_options.txt index 7f7b3e59..53f2675e 100644 --- a/meson_options.txt +++ b/meson_options.txt @@ -28,6 +28,11 @@ option('test', type : 'boolean', description: 'Compile and include the tests') +option('tracing', + type : 'feature', + value : 'auto', + description: 'Enable tracing (based on lttng)') + option('v4l2', type : 'boolean', value : false, diff --git a/src/libcamera/meson.build b/src/libcamera/meson.build index 07711b5f..99b27e66 100644 --- a/src/libcamera/meson.build +++ b/src/libcamera/meson.build @@ -55,6 +55,7 @@ libcamera_sources = files([ ]) libcamera_sources += libcamera_public_headers +libcamera_sources += libcamera_tracepoint_header includes = [ libcamera_includes, @@ -72,6 +73,11 @@ if libgnutls.found() config_h.set('HAVE_GNUTLS', 1) endif +if liblttng.found() + config_h.set('HAVE_TRACING', 1) + libcamera_sources += files(['tracepoints.cpp']) +endif + if libudev.found() config_h.set('HAVE_LIBUDEV', 1) libcamera_sources += files([ @@ -114,6 +120,7 @@ libcamera_deps = [ libatomic, libdl, libgnutls, + liblttng, libudev, dependency('threads'), ] diff --git a/src/libcamera/tracepoints.cpp b/src/libcamera/tracepoints.cpp new file mode 100644 index 00000000..0173b75a --- /dev/null +++ b/src/libcamera/tracepoints.cpp @@ -0,0 +1,10 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2020, Google Inc. + * + * tracepoints.cpp - Tracepoints with lttng + */ +#define TRACEPOINT_CREATE_PROBES +#define TRACEPOINT_DEFINE + +#include "libcamera/internal/tracepoints.h" diff --git a/utils/meson.build b/utils/meson.build index 3d83b975..7f2b9b4d 100644 --- a/utils/meson.build +++ b/utils/meson.build @@ -1,6 +1,7 @@ # SPDX-License-Identifier: CC0-1.0 subdir('ipu3') +subdir('tracepoints') ## Code generation py_modules += ['yaml'] diff --git a/utils/tracepoints/gen-tp-header.py b/utils/tracepoints/gen-tp-header.py new file mode 100644 index 00000000..bbd472d9 --- /dev/null +++ b/utils/tracepoints/gen-tp-header.py @@ -0,0 +1,38 @@ +#!/usr/bin/env python3 +# SPDX-License-Identifier: GPL-2.0-or-later +# Copyright (C) 2020, Google Inc. +# +# Author: Paul Elder +# +# gen-tp-header.py - Generate header file to contain lttng tracepoints + +import datetime +import jinja2 +import os +import sys + +def main(argv): + if len(argv) < 3: + print(f'Usage: {argv[0]} output template tp_files...') + return 1 + + output = argv[1] + template = argv[2] + + year = datetime.datetime.now().year + path = output.replace('include/', '', 1) + + source = '' + for fname in argv[3:]: + source += open(fname, 'r', encoding='utf-8').read() + '\n\n' + + template = jinja2.Template(open(template, 'r', encoding='utf-8').read()) + string = template.render(year=year, path=path, source=source) + + f = open(output, 'w', encoding='utf-8').write(string) + + return 0 + + +if __name__ == '__main__': + sys.exit(main(sys.argv)) diff --git a/utils/tracepoints/meson.build b/utils/tracepoints/meson.build new file mode 100644 index 00000000..807230fc --- /dev/null +++ b/utils/tracepoints/meson.build @@ -0,0 +1,5 @@ +# SPDX-License-Identifier: CC0-1.0 + +py_modules += ['jinja2'] + +gen_tracepoints_header = find_program('./gen-tp-header.py') From patchwork Fri Oct 30 08:57:53 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Paul Elder X-Patchwork-Id: 10307 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 8312DBDB1E for ; Fri, 30 Oct 2020 08:58:10 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 50DDC6298D; Fri, 30 Oct 2020 09:58:10 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="ffM73Z8t"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id C0B9662034 for ; Fri, 30 Oct 2020 09:58:08 +0100 (CET) Received: from pyrite.rasen.tech (unknown [IPv6:2400:4051:61:600:2c71:1b79:d06d:5032]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 06742A31; Fri, 30 Oct 2020 09:58:06 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1604048288; bh=/rMPkHHCCNX+FAze8b7AmntRFqBfDXEjYgdMhjZs2Jw=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=ffM73Z8tAVfGnnJxx3q8dg0GnCbhrkcWgSJMeW2iqM5iXRZFxeDLRlvH0HkrIcDJR 8+qg7WU/fZPGXmn5uoVeygdFYQNzfplVCWIIDUOMto2gOUT5MeLCWE6GiUUHhDDdfj 5PLUUUMTUYTM1T6283TxkaAuJNwSFbudCNGNTgPU= From: Paul Elder To: libcamera-devel@lists.libcamera.org Date: Fri, 30 Oct 2020 17:57:53 +0900 Message-Id: <20201030085756.79329-3-paul.elder@ideasonboard.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20201030085756.79329-1-paul.elder@ideasonboard.com> References: <20201030085756.79329-1-paul.elder@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v4 2/5] libcamera: request: Add tracepoints X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Add and use tracepoints in Request. Requests are core to libcamera operation, thus detecting delays in their processing is important, and serves as a good usage example of tracepoints. Signed-off-by: Paul Elder Reviewed-by: Laurent Pinchart --- Changes in v4: - move enum tp definitions to "header" tp files - add them to meson accordingly - remove cancelled field from request_complete_buffer tracepoint Changes in v3: - expand the changelog - add enum tracepoint values so that request status and buffer status are strings instead of ints in the trace - remove cancelled from all request tracepoints, except complete_buffer - expand the complete_buffer tracepoint to include information on the buffer - display the address of the request in all tracepoints - remove tracepoints for add_buffer and find_buffer Changes in v2: - remove tracepoints from uvcvideo - remove comment in changelog that this is only used for demonstration - use Request pointers instead of feeding the fields manually to the tracepoint --- .../internal/tracepoints/buffer_enums.tp | 9 +++ .../internal/tracepoints/meson.build | 5 ++ .../libcamera/internal/tracepoints/request.tp | 68 +++++++++++++++++++ .../internal/tracepoints/request_enums.tp | 9 +++ src/libcamera/request.cpp | 11 +++ 5 files changed, 102 insertions(+) create mode 100644 include/libcamera/internal/tracepoints/buffer_enums.tp create mode 100644 include/libcamera/internal/tracepoints/request.tp create mode 100644 include/libcamera/internal/tracepoints/request_enums.tp diff --git a/include/libcamera/internal/tracepoints/buffer_enums.tp b/include/libcamera/internal/tracepoints/buffer_enums.tp new file mode 100644 index 00000000..fdbea69a --- /dev/null +++ b/include/libcamera/internal/tracepoints/buffer_enums.tp @@ -0,0 +1,9 @@ +TRACEPOINT_ENUM( + libcamera, + buffer_status, + TP_ENUM_VALUES( + ctf_enum_value("FrameSuccess", 0) + ctf_enum_value("FrameError", 1) + ctf_enum_value("FrameCancelled", 2) + ) +) diff --git a/include/libcamera/internal/tracepoints/meson.build b/include/libcamera/internal/tracepoints/meson.build index 2dd6733e..fee0758f 100644 --- a/include/libcamera/internal/tracepoints/meson.build +++ b/include/libcamera/internal/tracepoints/meson.build @@ -1,4 +1,9 @@ # SPDX-License-Identifier: CC0-1.0 +# enum files must go first tracepoint_files = files([ + 'buffer_enums.tp', + 'request_enums.tp', + + 'request.tp', ]) diff --git a/include/libcamera/internal/tracepoints/request.tp b/include/libcamera/internal/tracepoints/request.tp new file mode 100644 index 00000000..dd28a714 --- /dev/null +++ b/include/libcamera/internal/tracepoints/request.tp @@ -0,0 +1,68 @@ +#include +#include + +TRACEPOINT_EVENT_CLASS( + libcamera, + request, + TP_ARGS( + libcamera::Request *, req + ), + TP_FIELDS( + ctf_integer_hex(uintptr_t, request, reinterpret_cast(req)) + ctf_integer(uint64_t, cookie, req->cookie()) + ctf_enum(libcamera, request_status, uint32_t, status, req->status()) + ) +) + +TRACEPOINT_EVENT_INSTANCE( + libcamera, + request, + request_construct, + TP_ARGS( + libcamera::Request *, req + ) +) + +TRACEPOINT_EVENT_INSTANCE( + libcamera, + request, + request_destroy, + TP_ARGS( + libcamera::Request *, req + ) +) + +TRACEPOINT_EVENT_INSTANCE( + libcamera, + request, + request_reuse, + TP_ARGS( + libcamera::Request *, req + ) +) + +TRACEPOINT_EVENT_INSTANCE( + libcamera, + request, + request_complete, + TP_ARGS( + libcamera::Request *, req + ) +) + + +TRACEPOINT_EVENT( + libcamera, + request_complete_buffer, + TP_ARGS( + libcamera::Request *, req, + libcamera::FrameBuffer *, buf + ), + TP_FIELDS( + ctf_integer_hex(uintptr_t, request, reinterpret_cast(req)) + ctf_integer(uint64_t, cookie, req->cookie()) + ctf_integer(int, status, req->status()) + ctf_integer_hex(uintptr_t, buffer, reinterpret_cast(buf)) + ctf_enum(libcamera, buffer_status, uint32_t, buf_status, buf->metadata().status) + ) +) diff --git a/include/libcamera/internal/tracepoints/request_enums.tp b/include/libcamera/internal/tracepoints/request_enums.tp new file mode 100644 index 00000000..371f6544 --- /dev/null +++ b/include/libcamera/internal/tracepoints/request_enums.tp @@ -0,0 +1,9 @@ +TRACEPOINT_ENUM( + libcamera, + request_status, + TP_ENUM_VALUES( + ctf_enum_value("RequestPending", 0) + ctf_enum_value("RequestComplete", 1) + ctf_enum_value("RequestCancelled", 2) + ) +) diff --git a/src/libcamera/request.cpp b/src/libcamera/request.cpp index ae8b1660..a68684ef 100644 --- a/src/libcamera/request.cpp +++ b/src/libcamera/request.cpp @@ -16,6 +16,7 @@ #include "libcamera/internal/camera_controls.h" #include "libcamera/internal/log.h" +#include "libcamera/internal/tracepoints.h" /** * \file request.h @@ -85,10 +86,14 @@ Request::Request(Camera *camera, uint64_t cookie) * \todo: Add a validator for metadata controls. */ metadata_ = new ControlList(controls::controls); + + LIBCAMERA_TRACEPOINT(request_construct, this); } Request::~Request() { + LIBCAMERA_TRACEPOINT(request_destroy, this); + delete metadata_; delete controls_; delete validator_; @@ -106,6 +111,8 @@ Request::~Request() */ void Request::reuse(ReuseFlag flags) { + LIBCAMERA_TRACEPOINT(request_reuse, this); + pending_.clear(); if (flags & ReuseBuffers) { for (auto pair : bufferMap_) { @@ -259,6 +266,8 @@ void Request::complete() LOG(Request, Debug) << "Request has completed - cookie: " << cookie_ << (cancelled_ ? " [Cancelled]" : ""); + + LIBCAMERA_TRACEPOINT(request_complete, this); } /** @@ -276,6 +285,8 @@ void Request::complete() */ bool Request::completeBuffer(FrameBuffer *buffer) { + LIBCAMERA_TRACEPOINT(request_complete_buffer, this, buffer); + int ret = pending_.erase(buffer); ASSERT(ret == 1); From patchwork Fri Oct 30 08:57:54 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Paul Elder X-Patchwork-Id: 10308 X-Patchwork-Delegate: paul.elder@ideasonboard.com Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 4E640BDB1E for ; Fri, 30 Oct 2020 08:58:13 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 0116D62995; Fri, 30 Oct 2020 09:58:12 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="aDmuHJPf"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 0543F62034 for ; Fri, 30 Oct 2020 09:58:11 +0100 (CET) Received: from pyrite.rasen.tech (unknown [IPv6:2400:4051:61:600:2c71:1b79:d06d:5032]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 3E1F99B7; Fri, 30 Oct 2020 09:58:08 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1604048290; bh=vpp8Kl6o++fsPCHmkBXiH5ssIzgHm6Jg2FTrQyAcqro=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=aDmuHJPfXWu7b2eDJIVB42LvfL7QnwCiYNPm8Lvg0/ZHcszM2N5iYweeL4Fjyr0S6 IsxK5r3+2IQW5jjfx6lP+sItbww2mmtobzeHqovtrIwiFmCKfv2BodV8cFE45/iNQW l6dyKESQopqRkYQwXjz0L5hn56XbqGUTsN+7FmT8= From: Paul Elder To: libcamera-devel@lists.libcamera.org Date: Fri, 30 Oct 2020 17:57:54 +0900 Message-Id: <20201030085756.79329-4-paul.elder@ideasonboard.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20201030085756.79329-1-paul.elder@ideasonboard.com> References: <20201030085756.79329-1-paul.elder@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v4 3/5] tracepoints: Add pipeline tracepoints for tracing IPA calls X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Add a pair of tracepoints to a general pipeline tracepoints file, libcamera:ipa_call_start and libcamera:ipa_call_finish, to trace IPA calls. This allows us to obtain the time taken for the IPA call. Signed-off-by: Paul Elder Reviewed-by: Laurent Pinchart --- No change in v4 Changes in v3: - rename the ipa tracepoints to ipa_call_begin and ipa_call_end - make string args const, so that strings can be passed directly New in v2 --- .../internal/tracepoints/meson.build | 1 + .../internal/tracepoints/pipeline.tp | 25 +++++++++++++++++++ 2 files changed, 26 insertions(+) create mode 100644 include/libcamera/internal/tracepoints/pipeline.tp diff --git a/include/libcamera/internal/tracepoints/meson.build b/include/libcamera/internal/tracepoints/meson.build index fee0758f..e2a5470a 100644 --- a/include/libcamera/internal/tracepoints/meson.build +++ b/include/libcamera/internal/tracepoints/meson.build @@ -5,5 +5,6 @@ tracepoint_files = files([ 'buffer_enums.tp', 'request_enums.tp', + 'pipeline.tp', 'request.tp', ]) diff --git a/include/libcamera/internal/tracepoints/pipeline.tp b/include/libcamera/internal/tracepoints/pipeline.tp new file mode 100644 index 00000000..c0a94635 --- /dev/null +++ b/include/libcamera/internal/tracepoints/pipeline.tp @@ -0,0 +1,25 @@ +TRACEPOINT_EVENT( + libcamera, + ipa_call_begin, + TP_ARGS( + const char *, pipe, + const char *, func + ), + TP_FIELDS( + ctf_string(pipeline_name, pipe) + ctf_string(function_name, func) + ) +) + +TRACEPOINT_EVENT( + libcamera, + ipa_call_end, + TP_ARGS( + const char *, pipe, + const char *, func + ), + TP_FIELDS( + ctf_string(pipeline_name, pipe) + ctf_string(function_name, func) + ) +) From patchwork Fri Oct 30 08:57:55 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Paul Elder X-Patchwork-Id: 10309 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id AC1D9BDB1E for ; Fri, 30 Oct 2020 08:58:14 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 7379C62993; Fri, 30 Oct 2020 09:58:14 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="uPCmpeiW"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id C92D462989 for ; Fri, 30 Oct 2020 09:58:12 +0100 (CET) Received: from pyrite.rasen.tech (unknown [IPv6:2400:4051:61:600:2c71:1b79:d06d:5032]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 62DD69B7; Fri, 30 Oct 2020 09:58:11 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1604048292; bh=NggESQQjOGLezqZG4Eaf/+OnHf4qqL3sgBns2AH90LI=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=uPCmpeiWetgZYG4I6W68kZ1KA6HvHv77CKjzEH2mSscZMA4sNqmaxWooaip/ORoed H7hLDEa2/n7odRXHp8jwLTTlEjvwrPr8tLbd9blKRRKfmzTgku+mr8uXHo/EpQIcjO 2rL8CvxamzuTVSKb1nFnOGfFt/Fi63C27g1JESJU= From: Paul Elder To: libcamera-devel@lists.libcamera.org Date: Fri, 30 Oct 2020 17:57:55 +0900 Message-Id: <20201030085756.79329-5-paul.elder@ideasonboard.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20201030085756.79329-1-paul.elder@ideasonboard.com> References: <20201030085756.79329-1-paul.elder@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v4 4/5] utils: tracepoints: Add simple statistics script X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Add a script that scans a trace for IPA call tracepoints, and returns statistics on the time taken for IPA calls. Signed-off-by: Paul Elder Reviewed-by: Laurent Pinchart --- Changes in v4: - rename script to analyze-ipa-trace.py - align output stats table Changes in v3: - check for the new tracepoint names, ipa_call_begin and ipa_call_end - fix babeltrace2 parsing in the case that the event doesn't have a pipeline_name field - change script description - add argparse description - add example for trace_path argument - change double quotes to single quotes New in v2 --- utils/tracepoints/analyze-ipa-trace.py | 77 ++++++++++++++++++++++++++ 1 file changed, 77 insertions(+) create mode 100755 utils/tracepoints/analyze-ipa-trace.py diff --git a/utils/tracepoints/analyze-ipa-trace.py b/utils/tracepoints/analyze-ipa-trace.py new file mode 100755 index 00000000..50fbbf42 --- /dev/null +++ b/utils/tracepoints/analyze-ipa-trace.py @@ -0,0 +1,77 @@ +#!/usr/bin/env python3 +# SPDX-License-Identifier: GPL-2.0-or-later +# Copyright (C) 2020, Google Inc. +# +# Author: Paul Elder +# +# analyze-ipa-trace.py - Example of how to extract information from libcamera lttng traces + +import argparse +import bt2 +import statistics as stats +import sys + +# pipeline -> {function -> stack(timestamps)} +timestamps = {} + +# pipeline:function -> samples[] +samples = {} + +def main(argv): + parser = argparse.ArgumentParser( + description='A simple analysis script to get statistics on time taken for IPA calls') + parser.add_argument('-p', '--pipeline', type=str, + help='Name of pipeline to filter for') + parser.add_argument('trace_path', type=str, + help='Path to lttng trace (eg. ~/lttng-traces/demo-20201029-184003)') + args = parser.parse_args(argv[1:]) + + traces = bt2.TraceCollectionMessageIterator(args.trace_path) + for msg in traces: + if type(msg) is not bt2._EventMessageConst or \ + 'pipeline_name' not in msg.event.payload_field or \ + (args.pipeline is not None and \ + msg.event.payload_field['pipeline_name'] != args.pipeline): + continue + + pipeline = msg.event.payload_field['pipeline_name'] + event = msg.event.name + func = msg.event.payload_field['function_name'] + timestamp_ns = msg.default_clock_snapshot.ns_from_origin + + if event == 'libcamera:ipa_call_begin': + if pipeline not in timestamps: + timestamps[pipeline] = {} + if func not in timestamps[pipeline]: + timestamps[pipeline][func] = [] + timestamps[pipeline][func].append(timestamp_ns) + + if event == 'libcamera:ipa_call_end': + ts = timestamps[pipeline][func].pop() + key = f'{pipeline}:{func}' + if key not in samples: + samples[key] = [] + samples[key].append(timestamp_ns - ts) + + # Compute stats + rows = [] + rows.append(['pipeline:function', 'min', 'max', 'mean', 'stddev']) + for k, v in samples.items(): + mean = int(stats.mean(v)) + stddev = int(stats.stdev(v)) + minv = min(v) + maxv = max(v) + rows.append([k, str(minv), str(maxv), str(mean), str(stddev)]) + + # Get maximum string width for every column + widths = [] + for i in range(len(rows[0])): + widths.append(max([len(row[i]) for row in rows])) + + # Print stats table + for row in rows: + fmt = [row[i].rjust(widths[i]) for i in range(1, 5)] + print('{} {} {} {} {}'.format(row[0].ljust(widths[0]), *fmt)) + +if __name__ == '__main__': + sys.exit(main(sys.argv)) From patchwork Fri Oct 30 08:57:56 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Paul Elder X-Patchwork-Id: 10310 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 26C09BDB1E for ; Fri, 30 Oct 2020 08:58:17 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id E886162979; Fri, 30 Oct 2020 09:58:16 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="RJX1PfR0"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 084CF628A5 for ; Fri, 30 Oct 2020 09:58:15 +0100 (CET) Received: from pyrite.rasen.tech (unknown [IPv6:2400:4051:61:600:2c71:1b79:d06d:5032]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 3C68F9B7; Fri, 30 Oct 2020 09:58:12 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1604048294; bh=2kOoaciY+HnAFQ6qs4Pfm49M1NlG3j95wVFQF3CtmhQ=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=RJX1PfR0uE9bw0dPIhzvmXeGrov/ym2dRlhZy9Bdxf6/+1pG6kb9HKQmau1RK8ATt mKN2bJZZnv/Td2/VtnlXmxXuRqqn9I596l6+2HS24S8w0QyEOB2IaS0QvHCwNpKQ04 EADI+5pu0Y2np+YbUOChdEOlD6JYkPB6zO10i2DA= From: Paul Elder To: libcamera-devel@lists.libcamera.org Date: Fri, 30 Oct 2020 17:57:56 +0900 Message-Id: <20201030085756.79329-6-paul.elder@ideasonboard.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20201030085756.79329-1-paul.elder@ideasonboard.com> References: <20201030085756.79329-1-paul.elder@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v4 5/5] Documentation: tracing: Add tracing guide X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Add guide for tracepoints, including how to define and use them. Signed-off-by: Paul Elder Reviewed-by: Laurent Pinchart --- Changes in v4: - fix spelling and some indentation - rebase on master instead of ipc Changes in v3: - fix compilation error - reword english - add stuff about the new IPA call tracepoint macros New in v2 --- Documentation/guides/tracing.rst | 141 +++++++++++++++++++++++++++++++ Documentation/index.rst | 1 + Documentation/meson.build | 1 + 3 files changed, 143 insertions(+) create mode 100644 Documentation/guides/tracing.rst diff --git a/Documentation/guides/tracing.rst b/Documentation/guides/tracing.rst new file mode 100644 index 00000000..9c688b93 --- /dev/null +++ b/Documentation/guides/tracing.rst @@ -0,0 +1,141 @@ +.. SPDX-License-Identifier: CC-BY-SA-4.0 + +Tracing Guide +============= + +Guide to tracing in libcamera. + +Profiling vs Tracing +-------------------- + +Tracing is recording timestamps at specific locations. libcamera provides a +tracing facility. This guide shows how to use this tracing facility. + +Tracing should not be confused with profiling, which samples execution +at periodic points in time. This can be done with other tools such as +callgrind, perf, gprof, etc., without modification to the application, +and is out of scope for this guide. + +Compiling +--------- + +To compile libcamera with tracing support, it must be enabled through the +meson ``tracing`` option. It depends on the lttng-ust library (available in the +``liblttng-ust-dev`` package for Debian-based distributions). +By default the tracing option in meson is set to ``auto``, so if +liblttng is detected, it will be enabled by default. Conversely, if the option +is set to disabled, then libcamera will be compiled without tracing support. + +Defining tracepoints +-------------------- + +libcamera already contains a set of tracepoints. To define additional +tracepoints, create a file +``include/libcamera/internal/tracepoints/{file}.tp``, where ``file`` is a +reasonable name related to the category of tracepoints that you wish to +define. For example, the tracepoints file for the Request object is called +``request.tp``. An entry for this file must be added in +``include/libcamera/internal/tracepoints/meson.build``. + +In this tracepoints file, define your tracepoints `as mandated by lttng +`_. The header boilerplate must *not* be +included (as it will conflict with the rest of our infrastructure), and +only the tracepoint definitions (with the ``TRACEPOINT_*`` macros) should be +included. + +All tracepoint providers shall be ``libcamera``. According to lttng, the +tracepoint provider should be per-project; this is the rationale for this +decision. To group tracepoint events, we recommend using +``{class_name}_{tracepoint_name}``, for example, ``request_construct`` for a +tracepoint for the constructor of the Request class. + +Tracepoint arguments may take C++ objects pointers, in which case the usual +C++ namespacing rules apply. The header that contains the necessary class +definitions must be included at the top of the tracepoint provider file. + +Note: the final parameter in ``TP_ARGS`` *must not* have a trailing comma, and +the parameters to ``TP_FIELDS`` are *space-separated*. Not following these will +cause compilation errors. + +Using tracepoints (in libcamera) +-------------------------------- + +To use tracepoints in libcamera, first the header needs to be included: + +``#include "libcamera/internal/tracepoints.h"`` + +Then to use the tracepoint: + +``LIBCAMERA_TRACEPOINT({tracepoint_event}, args...)`` + +This macro must be used, as opposed to lttng's macros directly, because +lttng is an optional dependency of libcamera, so the code must compile and run +even when lttng is not present or when tracing is disabled. + +The tracepoint provider name, as declared in the tracepoint definition, is not +included in the parameters of the tracepoint. + +There are also two special tracepoints available for tracing IPA calls: + +``LIBCAMERA_TRACEPOINT_IPA_BEGIN({pipeline_name}, {ipa_function})`` + +``LIBCAMERA_TRACEPOINT_IPA_END({pipeline_name}, {ipa_function})`` + +These shall be placed where an IPA function is called from the pipeline handler, +and when the pipeline handler receives the corresponding response from the IPA, +respectively. These are the tracepoints that our sample analysis script +(see "Analyzing a trace") scans for when computing statistics on IPA call time. + +Using tracepoints (from an application) +--------------------------------------- + +As applications are not part of libcamera, but rather users of libcamera, +applications should seek their own tracing mechanisms. For ease of tracing +the application alongside tracing libcamera, it is recommended to also +`use lttng `_. + +Using tracepoints (from closed-source IPA) +------------------------------------------ + +Similar to applications, closed-source IPAs can simply use lttng on their own, +or any other tracing mechanism if desired. + +Collecting a trace +------------------ + +A trace can be collected fairly simply from lttng: + +.. code-block:: bash + + lttng create $SESSION_NAME + lttng enable-event -u libcamera:\* + lttng start + # run libcamera application + lttng stop + lttng view + lttng destroy $SESSION_NAME + +See the `lttng documentation `_ for further details. + +Analyzing a trace +----------------- + +As mentioned above, while an lttng tracing session exists and the trace is not +running, the trace output can be viewed as text by ``lttng view``. + +The trace log can also be viewed as text using babeltrace2. See the +`lttng trace analysis documentation +`_ +for further details. + +babeltrace2 also has a C API and python bindings that can be used to process +traces. See the +`lttng python bindings documentation `_ +and the +`lttng C API documentation `_ +for more details. + +As an example, there is a script ``utils/tracepoints/analyze-ipa-trace.py`` +that gathers statistics for the time taken for an IPA function call, by +measuring the time difference between pairs of events +``libcamera:ipa_call_start`` and ``libcamera:ipa_call_finish``. diff --git a/Documentation/index.rst b/Documentation/index.rst index a30688ac..ff697d4f 100644 --- a/Documentation/index.rst +++ b/Documentation/index.rst @@ -16,3 +16,4 @@ Developer Guide Application Writer's Guide Pipeline Handler Writer's Guide + Tracing guide diff --git a/Documentation/meson.build b/Documentation/meson.build index d3d64f71..26a12fcd 100644 --- a/Documentation/meson.build +++ b/Documentation/meson.build @@ -56,6 +56,7 @@ if sphinx.found() 'guides/introduction.rst', 'guides/application-developer.rst', 'guides/pipeline-handler.rst', + 'guides/tracing.rst', ] release = 'release=v' + libcamera_git_version