From patchwork Wed Oct 28 10:31:46 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Paul Elder X-Patchwork-Id: 10287 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id BA411BDB1E for ; Wed, 28 Oct 2020 10:32:04 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 1858F625AC; Wed, 28 Oct 2020 11:32:04 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="olHRXR35"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 66FAA62067 for ; Wed, 28 Oct 2020 11:32:02 +0100 (CET) Received: from pyrite.rasen.tech (unknown [IPv6:2400:4051:61:600:2c71:1b79:d06d:5032]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id C654F99A; Wed, 28 Oct 2020 11:32:00 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1603881122; bh=4cR7B4knACVHzkBsai8Fl8tHfbMhf7WFIiH5DYvxy+E=; h=From:To:Cc:Subject:Date:From; b=olHRXR35TQGFBYiDOYZqGaJQOrAgDfGrM8PyMcPr79jnQE3UrhAOJl1/k2j+zJTso kPYPs/Ma/VeUPKF8tGXKN897pp4QwPPpmR4iFt5LLOc4nIRhZVs8a2/tx++E9lVv5k 7cek2uEgBx7qsWZi/nTOPgqTxeJd+w66RAUUJkxM= From: Paul Elder To: libcamera-devel@lists.libcamera.org Date: Wed, 28 Oct 2020 19:31:46 +0900 Message-Id: <20201028103151.34575-1-paul.elder@ideasonboard.com> X-Mailer: git-send-email 2.27.0 MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v2 1/6] libcamera: tracing: Implement tracing infrastructure X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Implement tracing infrastructure in libcamera. It takes .tp files, as required by lttng, and generates a tracepoint header and C file, as lttng requires. meson is updated accordingly to get it to compile with the rest of libcamera. Update the documentation accordingly. Signed-off-by: Paul Elder Reviewed-by: Laurent Pinchart --- Changes in v2: - move header.tmpl to tracepoints.h.in - rename meson option 'tracepoints' to 'tracing' - make sure C++ objects work in tracepoint arguments - rename generate_header.py to gen-tp-header.py --- Documentation/Doxyfile.in | 2 + README.rst | 3 ++ include/libcamera/internal/meson.build | 9 +++++ include/libcamera/internal/tracepoints.h.in | 40 +++++++++++++++++++ .../internal/tracepoints/meson.build | 4 ++ meson.build | 3 ++ meson_options.txt | 5 +++ src/libcamera/meson.build | 7 ++++ src/libcamera/tracepoints.cpp | 10 +++++ utils/meson.build | 1 + utils/tracepoints/gen-tp-header.py | 38 ++++++++++++++++++ utils/tracepoints/meson.build | 5 +++ 12 files changed, 127 insertions(+) create mode 100644 include/libcamera/internal/tracepoints.h.in create mode 100644 include/libcamera/internal/tracepoints/meson.build create mode 100644 src/libcamera/tracepoints.cpp create mode 100644 utils/tracepoints/gen-tp-header.py create mode 100644 utils/tracepoints/meson.build diff --git a/Documentation/Doxyfile.in b/Documentation/Doxyfile.in index 5ccd0d05..9a7a3fb4 100644 --- a/Documentation/Doxyfile.in +++ b/Documentation/Doxyfile.in @@ -842,6 +842,8 @@ EXCLUDE = @TOP_SRCDIR@/include/libcamera/span.h \ @TOP_SRCDIR@/src/libcamera/ipa_ipc_unixsocket.cpp \ @TOP_SRCDIR@/src/libcamera/pipeline/ \ @TOP_SRCDIR@/src/libcamera/proxy/ \ + @TOP_SRCDIR@/src/libcamera/tracepoints.cpp \ + @TOP_BUILDDIR@/include/libcamera/internal/tracepoints.h \ @TOP_BUILDDIR@/include/libcamera/ipa/ \ @TOP_BUILDDIR@/src/libcamera/proxy/ diff --git a/README.rst b/README.rst index fc8f4948..5c4b6989 100644 --- a/README.rst +++ b/README.rst @@ -81,6 +81,9 @@ for gstreamer: [optional] for qcam: [optional] qtbase5-dev libqt5core5a libqt5gui5 libqt5widgets5 qttools5-dev-tools libtiff-dev +for tracing with lttng: [optional] + lttng-ust-dev python3-jinja2 + Using GStreamer plugin ~~~~~~~~~~~~~~~~~~~~~~ diff --git a/include/libcamera/internal/meson.build b/include/libcamera/internal/meson.build index 8dd744fd..b925b8e8 100644 --- a/include/libcamera/internal/meson.build +++ b/include/libcamera/internal/meson.build @@ -1,5 +1,14 @@ # SPDX-License-Identifier: CC0-1.0 +subdir('tracepoints') + +libcamera_tracepoint_header = custom_target( + 'tp_header', + input: [ 'tracepoints.h.in', tracepoint_files ], + output: 'tracepoints.h', + command: [ tracepoints_header_generator, '@OUTPUT@', '@INPUT@' ], +) + libcamera_internal_headers = files([ 'byte_stream_buffer.h', 'camera_controls.h', diff --git a/include/libcamera/internal/tracepoints.h.in b/include/libcamera/internal/tracepoints.h.in new file mode 100644 index 00000000..9b6c3907 --- /dev/null +++ b/include/libcamera/internal/tracepoints.h.in @@ -0,0 +1,40 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) {{year}}, Google Inc. + * + * tracepoints.h - Tracepoints with lttng + * + * This file is auto-generated. Do not edit. + */ +#ifndef __LIBCAMERA_INTERNAL_TRACEPOINTS_H__ +#define __LIBCAMERA_INTERNAL_TRACEPOINTS_H__ + +#if HAVE_TRACING +#define LIBCAMERA_TRACEPOINT(...) tracepoint(libcamera, __VA_ARGS__) +#else +#define LIBCAMERA_TRACEPOINT(...) do {} while (0) +#endif + +#endif /* __LIBCAMERA_INTERNAL_TRACEPOINTS_H__ */ + + +#if HAVE_TRACING + +#undef TRACEPOINT_PROVIDER +#define TRACEPOINT_PROVIDER libcamera + +#undef TRACEPOINT_INCLUDE +#define TRACEPOINT_INCLUDE "{{path}}" + +#if !defined(INCLUDE_LIBCAMERA_INTERNAL_TRACEPOINTS_TP_H) || defined(TRACEPOINT_HEADER_MULTI_READ) +#define INCLUDE_LIBCAMERA_INTERNAL_TRACEPOINTS_TP_H + +#include + +{{source}} + +#endif /* INCLUDE_LIBCAMERA_INTERNAL_TRACEPOINTS_TP_H */ + +#include + +#endif /* HAVE_TRACING */ diff --git a/include/libcamera/internal/tracepoints/meson.build b/include/libcamera/internal/tracepoints/meson.build new file mode 100644 index 00000000..2dd6733e --- /dev/null +++ b/include/libcamera/internal/tracepoints/meson.build @@ -0,0 +1,4 @@ +# SPDX-License-Identifier: CC0-1.0 + +tracepoint_files = files([ +]) diff --git a/meson.build b/meson.build index e6f6c84a..55cf36e1 100644 --- a/meson.build +++ b/meson.build @@ -108,6 +108,9 @@ libcamera_includes = include_directories('include') # Sub-directories fill py_modules with their dependencies. py_modules = [] +# Libraries used by multiple components +liblttng = cc.find_library('lttng-ust', required : get_option('tracing')) + # Utilities are parsed first to provide support for other components. subdir('utils') diff --git a/meson_options.txt b/meson_options.txt index 7f7b3e59..53f2675e 100644 --- a/meson_options.txt +++ b/meson_options.txt @@ -28,6 +28,11 @@ option('test', type : 'boolean', description: 'Compile and include the tests') +option('tracing', + type : 'feature', + value : 'auto', + description: 'Enable tracing (based on lttng)') + option('v4l2', type : 'boolean', value : false, diff --git a/src/libcamera/meson.build b/src/libcamera/meson.build index 24f7ea3d..51925bd6 100644 --- a/src/libcamera/meson.build +++ b/src/libcamera/meson.build @@ -58,6 +58,7 @@ libcamera_sources = files([ libcamera_sources += libcamera_public_headers libcamera_sources += libcamera_generated_ipa_headers +libcamera_sources += libcamera_tracepoint_header includes = [ libcamera_includes, @@ -75,6 +76,11 @@ if libgnutls.found() config_h.set('HAVE_GNUTLS', 1) endif +if liblttng.found() + config_h.set('HAVE_TRACING', 1) + libcamera_sources += files(['tracepoints.cpp']) +endif + if libudev.found() config_h.set('HAVE_LIBUDEV', 1) libcamera_sources += files([ @@ -117,6 +123,7 @@ libcamera_deps = [ libatomic, libdl, libgnutls, + liblttng, libudev, dependency('threads'), ] diff --git a/src/libcamera/tracepoints.cpp b/src/libcamera/tracepoints.cpp new file mode 100644 index 00000000..0173b75a --- /dev/null +++ b/src/libcamera/tracepoints.cpp @@ -0,0 +1,10 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2020, Google Inc. + * + * tracepoints.cpp - Tracepoints with lttng + */ +#define TRACEPOINT_CREATE_PROBES +#define TRACEPOINT_DEFINE + +#include "libcamera/internal/tracepoints.h" diff --git a/utils/meson.build b/utils/meson.build index 383d5285..8e28ada7 100644 --- a/utils/meson.build +++ b/utils/meson.build @@ -2,6 +2,7 @@ subdir('ipc') subdir('ipu3') +subdir('tracepoints') ## Code generation py_modules += ['yaml'] diff --git a/utils/tracepoints/gen-tp-header.py b/utils/tracepoints/gen-tp-header.py new file mode 100644 index 00000000..7e7d76b2 --- /dev/null +++ b/utils/tracepoints/gen-tp-header.py @@ -0,0 +1,38 @@ +#!/usr/bin/env python3 +# SPDX-License-Identifier: GPL-2.0-or-later +# Copyright (C) 2020, Google Inc. +# +# Author: Paul Elder +# +# gen-tp-header.py - Generate header file to contain lttng tracepoints + +import datetime +import jinja2 +import os +import sys + +def main(argv): + if len(argv) < 3: + print(f'Usage: {argv[0]} output template tp_files...') + return 1 + + output = argv[1] + template = argv[2] + + year = datetime.datetime.now().year + path = output.replace('include/', '', 1) + + source = '' + for fname in argv[3:]: + source += open(fname, 'r').read() + '\n\n' + + template = jinja2.Template(open(template, 'r').read()) + string = template.render(year=year, path=path, source=source) + + f = open(output, 'w').write(string) + + return 0 + + +if __name__ == '__main__': + sys.exit(main(sys.argv)) diff --git a/utils/tracepoints/meson.build b/utils/tracepoints/meson.build new file mode 100644 index 00000000..46946f36 --- /dev/null +++ b/utils/tracepoints/meson.build @@ -0,0 +1,5 @@ +# SPDX-License-Identifier: CC0-1.0 + +py_modules += ['jinja2'] + +tracepoints_header_generator = find_program('./gen-tp-header.py') From patchwork Wed Oct 28 10:31:47 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Paul Elder X-Patchwork-Id: 10288 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 1A112BDB1E for ; Wed, 28 Oct 2020 10:32:06 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id C405D625FE; Wed, 28 Oct 2020 11:32:05 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="BZW0CaZ2"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 2B130625EF for ; Wed, 28 Oct 2020 11:32:04 +0100 (CET) Received: from pyrite.rasen.tech (unknown [IPv6:2400:4051:61:600:2c71:1b79:d06d:5032]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 9FC3199A; Wed, 28 Oct 2020 11:32:02 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1603881123; bh=4r43Df56Lzm3dEGcbjXd7izkQmLPHXCdRVDpiAWMSDQ=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=BZW0CaZ2RU6elEUUpBbHpSxryaZQ+2y73Lzk5/lIRhp3pcAe69Q6pqcsLJxpQckje qNn9MJMJgoGEIQWqi/c5erbtjV4GXgzymqNMvJ9dGKKuGK5NpgV8uLo42eqimUisMH 7YeQEHUfyrKoAjf7dpnyLfmiiqABThVMRFoipW9U= From: Paul Elder To: libcamera-devel@lists.libcamera.org Date: Wed, 28 Oct 2020 19:31:47 +0900 Message-Id: <20201028103151.34575-2-paul.elder@ideasonboard.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20201028103151.34575-1-paul.elder@ideasonboard.com> References: <20201028103151.34575-1-paul.elder@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v2 2/6] libcamera: request: Add tracepoints X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Add and use tracepoints in Request. Signed-off-by: Paul Elder Reviewed-by: Laurent Pinchart --- Changes in v2: - remove tracepoints from uvcvideo - remove comment in changelog that this is only used for demonstration - use Request pointers instead of feeding the fields manually to the tracepoint --- .../internal/tracepoints/meson.build | 1 + .../libcamera/internal/tracepoints/request.tp | 85 +++++++++++++++++++ src/libcamera/request.cpp | 15 ++++ 3 files changed, 101 insertions(+) create mode 100644 include/libcamera/internal/tracepoints/request.tp diff --git a/include/libcamera/internal/tracepoints/meson.build b/include/libcamera/internal/tracepoints/meson.build index 2dd6733e..8410c205 100644 --- a/include/libcamera/internal/tracepoints/meson.build +++ b/include/libcamera/internal/tracepoints/meson.build @@ -1,4 +1,5 @@ # SPDX-License-Identifier: CC0-1.0 tracepoint_files = files([ + 'request.tp', ]) diff --git a/include/libcamera/internal/tracepoints/request.tp b/include/libcamera/internal/tracepoints/request.tp new file mode 100644 index 00000000..481a3670 --- /dev/null +++ b/include/libcamera/internal/tracepoints/request.tp @@ -0,0 +1,85 @@ +#include + +TRACEPOINT_EVENT_CLASS( + libcamera, + request, + TP_ARGS( + libcamera::Request *, req, + int, cancelled + ), + TP_FIELDS( + ctf_integer(uint64_t, cookie, req->cookie()) + ctf_integer(int, status, req->status()) + ctf_integer(int, cancelled, cancelled) + ) +) + +TRACEPOINT_EVENT_INSTANCE( + libcamera, + request, + request_construct, + TP_ARGS( + libcamera::Request *, req, + int, cancelled + ) +) + +TRACEPOINT_EVENT_INSTANCE( + libcamera, + request, + request_deconstruct, + TP_ARGS( + libcamera::Request *, req, + int, cancelled + ) +) + +TRACEPOINT_EVENT_INSTANCE( + libcamera, + request, + request_reuse, + TP_ARGS( + libcamera::Request *, req, + int, cancelled + ) +) + +TRACEPOINT_EVENT_INSTANCE( + libcamera, + request, + request_add_buffer, + TP_ARGS( + libcamera::Request *, req, + int, cancelled + ) +) + +TRACEPOINT_EVENT_INSTANCE( + libcamera, + request, + request_find_buffer, + TP_ARGS( + const libcamera::Request *, req, + int, cancelled + ) +) + +TRACEPOINT_EVENT_INSTANCE( + libcamera, + request, + request_complete, + TP_ARGS( + libcamera::Request *, req, + int, cancelled + ) +) + +TRACEPOINT_EVENT_INSTANCE( + libcamera, + request, + request_complete_buffer, + TP_ARGS( + libcamera::Request *, req, + int, cancelled + ) +) diff --git a/src/libcamera/request.cpp b/src/libcamera/request.cpp index ae8b1660..bec2cab5 100644 --- a/src/libcamera/request.cpp +++ b/src/libcamera/request.cpp @@ -16,6 +16,7 @@ #include "libcamera/internal/camera_controls.h" #include "libcamera/internal/log.h" +#include "libcamera/internal/tracepoints.h" /** * \file request.h @@ -85,10 +86,14 @@ Request::Request(Camera *camera, uint64_t cookie) * \todo: Add a validator for metadata controls. */ metadata_ = new ControlList(controls::controls); + + LIBCAMERA_TRACEPOINT(request_construct, this, cancelled_); } Request::~Request() { + LIBCAMERA_TRACEPOINT(request_deconstruct, this, cancelled_); + delete metadata_; delete controls_; delete validator_; @@ -106,6 +111,8 @@ Request::~Request() */ void Request::reuse(ReuseFlag flags) { + LIBCAMERA_TRACEPOINT(request_reuse, this, cancelled_); + pending_.clear(); if (flags & ReuseBuffers) { for (auto pair : bufferMap_) { @@ -167,6 +174,8 @@ void Request::reuse(ReuseFlag flags) */ int Request::addBuffer(const Stream *stream, FrameBuffer *buffer) { + LIBCAMERA_TRACEPOINT(request_add_buffer, this, cancelled_); + if (!stream) { LOG(Request, Error) << "Invalid stream reference"; return -EINVAL; @@ -202,6 +211,8 @@ int Request::addBuffer(const Stream *stream, FrameBuffer *buffer) */ FrameBuffer *Request::findBuffer(const Stream *stream) const { + LIBCAMERA_TRACEPOINT(request_find_buffer, this, cancelled_); + const auto it = bufferMap_.find(stream); if (it == bufferMap_.end()) return nullptr; @@ -253,6 +264,8 @@ FrameBuffer *Request::findBuffer(const Stream *stream) const */ void Request::complete() { + LIBCAMERA_TRACEPOINT(request_complete, this, cancelled_); + ASSERT(!hasPendingBuffers()); status_ = cancelled_ ? RequestCancelled : RequestComplete; @@ -276,6 +289,8 @@ void Request::complete() */ bool Request::completeBuffer(FrameBuffer *buffer) { + LIBCAMERA_TRACEPOINT(request_complete_buffer, this, cancelled_); + int ret = pending_.erase(buffer); ASSERT(ret == 1); From patchwork Wed Oct 28 10:31:48 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Paul Elder X-Patchwork-Id: 10289 X-Patchwork-Delegate: paul.elder@ideasonboard.com Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 90AE7BDB1E for ; Wed, 28 Oct 2020 10:32:08 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 5F47862600; Wed, 28 Oct 2020 11:32:08 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="dOTOL4sq"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id E5A6C62600 for ; Wed, 28 Oct 2020 11:32:05 +0100 (CET) Received: from pyrite.rasen.tech (unknown [IPv6:2400:4051:61:600:2c71:1b79:d06d:5032]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 79A8499A; Wed, 28 Oct 2020 11:32:04 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1603881125; bh=21ZqDIEgZYtF6D/fdJWOofjwXBRx1F5hdjr8KUnaEe8=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=dOTOL4sqIOBVvhy+gIGEFtnw+Ny80hHPgM7lQFqR0oejZ9OqQqKALq5jbZmE5G9He R/ds/5aaA0W1n4exieSGthXvCNFwKigKkx5yk73G2iTJThrdU6WSu1ycIMX7+5wri+ tTscx1+/Ag+vmqt6J/xpOPFBXV5Eh9wgFXNISjfI= From: Paul Elder To: libcamera-devel@lists.libcamera.org Date: Wed, 28 Oct 2020 19:31:48 +0900 Message-Id: <20201028103151.34575-3-paul.elder@ideasonboard.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20201028103151.34575-1-paul.elder@ideasonboard.com> References: <20201028103151.34575-1-paul.elder@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v2 3/6] tracepoints: Add pipeline tracepoints for tracing IPA calls X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Add a pair of tracepoints to a general pipeline tracepoints file, libcamera:ipa_call_start and libcamera:ipa_call_finish, to trace IPA calls. This allows us to obtain the time taken for the IPA call. Signed-off-by: Paul Elder Reviewed-by: Laurent Pinchart --- New in v2 --- .../internal/tracepoints/meson.build | 1 + .../internal/tracepoints/pipeline.tp | 25 +++++++++++++++++++ 2 files changed, 26 insertions(+) create mode 100644 include/libcamera/internal/tracepoints/pipeline.tp diff --git a/include/libcamera/internal/tracepoints/meson.build b/include/libcamera/internal/tracepoints/meson.build index 8410c205..a34135ce 100644 --- a/include/libcamera/internal/tracepoints/meson.build +++ b/include/libcamera/internal/tracepoints/meson.build @@ -1,5 +1,6 @@ # SPDX-License-Identifier: CC0-1.0 tracepoint_files = files([ + 'pipeline.tp', 'request.tp', ]) diff --git a/include/libcamera/internal/tracepoints/pipeline.tp b/include/libcamera/internal/tracepoints/pipeline.tp new file mode 100644 index 00000000..603f4a8c --- /dev/null +++ b/include/libcamera/internal/tracepoints/pipeline.tp @@ -0,0 +1,25 @@ +TRACEPOINT_EVENT( + libcamera, + ipa_call_start, + TP_ARGS( + char *, pipe, + char *, func + ), + TP_FIELDS( + ctf_string(pipeline_name, pipe) + ctf_string(function_name, func) + ) +) + +TRACEPOINT_EVENT( + libcamera, + ipa_call_finish, + TP_ARGS( + char *, pipe, + char *, func + ), + TP_FIELDS( + ctf_string(pipeline_name, pipe) + ctf_string(function_name, func) + ) +) From patchwork Wed Oct 28 10:31:49 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Paul Elder X-Patchwork-Id: 10290 X-Patchwork-Delegate: paul.elder@ideasonboard.com Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 06E98BDB1E for ; Wed, 28 Oct 2020 10:32:10 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id C5146625F5; Wed, 28 Oct 2020 11:32:09 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="vramt9DL"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id C3E8B62067 for ; Wed, 28 Oct 2020 11:32:07 +0100 (CET) Received: from pyrite.rasen.tech (unknown [IPv6:2400:4051:61:600:2c71:1b79:d06d:5032]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 52F60B95; Wed, 28 Oct 2020 11:32:06 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1603881127; bh=wwEgF6bKhjW005jowIXVR22ZTCtwzTyTBVkycKHI+eE=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=vramt9DLeKeJn93hG8aTywhA/fG9Kq3FeV7/mwdAAhi+m/c2cg1w/NHKseRhdcJCH cws+6fxjge5d66hI/ypq1iNhWzX3r2RkQkMdsLuWigugQWzPNZmeosE1Dj/sv6OInk 4qscig00d0E2Ybkf6nnVtFGvcQosZX09v8uhBzc4= From: Paul Elder To: libcamera-devel@lists.libcamera.org Date: Wed, 28 Oct 2020 19:31:49 +0900 Message-Id: <20201028103151.34575-4-paul.elder@ideasonboard.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20201028103151.34575-1-paul.elder@ideasonboard.com> References: <20201028103151.34575-1-paul.elder@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v2 4/6] pipeline: raspberrypi: Add IPA call tracepoints X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Emit tracepoints for IPA calls in the raspberrypi pipeline handler, to allow profiling of IPA calls. Signed-off-by: Paul Elder --- New in v2 I was getting compiler complaints when I passed direct strings to the tracepoint without casting... I wasn't getting it in tests... --- src/libcamera/pipeline/raspberrypi/raspberrypi.cpp | 9 +++++++++ 1 file changed, 9 insertions(+) diff --git a/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp b/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp index 0a5a7288..2de60d3d 100644 --- a/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp +++ b/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp @@ -31,6 +31,7 @@ #include "libcamera/internal/ipa_manager.h" #include "libcamera/internal/media_device.h" #include "libcamera/internal/pipeline_handler.h" +#include "libcamera/internal/tracepoints.h" #include "libcamera/internal/utils.h" #include "libcamera/internal/v4l2_controls.h" #include "libcamera/internal/v4l2_videodevice.h" @@ -1240,6 +1241,8 @@ int RPiCameraData::configureIPA(const CameraConfiguration *config) void RPiCameraData::statsMetadataComplete(uint32_t bufferId, const ControlList &controls) { + LIBCAMERA_TRACEPOINT(ipa_call_finish, (char *)"rpi", (char *)"signalStatReady"); + if (state_ == State::Stopped) handleState(); @@ -1273,6 +1276,8 @@ void RPiCameraData::statsMetadataComplete(uint32_t bufferId, const ControlList & void RPiCameraData::runIsp(uint32_t bufferId) { + LIBCAMERA_TRACEPOINT(ipa_call_finish, (char *)"rpi", (char *)"signalIspPrepare"); + if (state_ == State::Stopped) handleState(); @@ -1406,6 +1411,7 @@ void RPiCameraData::ispOutputDequeue(FrameBuffer *buffer) * application until after the IPA signals so. */ if (stream == &isp_[Isp::Stats]) { + LIBCAMERA_TRACEPOINT(ipa_call_start, (char *)"rpi", (char *)"signalStatReady"); ipa_->signalStatReady(RPi::BufferMask::STATS | static_cast(index)); } else { /* Any other ISP output can be handed back to the application now. */ @@ -1658,7 +1664,9 @@ void RPiCameraData::tryRunPipeline() * queue the ISP output buffer listed in the request to start the HW * pipeline. */ + LIBCAMERA_TRACEPOINT(ipa_call_start, (char *)"rpi", (char *)"signalQueueRequest"); ipa_->signalQueueRequest(request->controls()); + LIBCAMERA_TRACEPOINT(ipa_call_finish, (char *)"rpi", (char *)"signalQueueRequest"); /* Ready to use the buffers, pop them off the queue. */ bayerQueue_.pop(); @@ -1677,6 +1685,7 @@ void RPiCameraData::tryRunPipeline() ipa::rpi::IspPreparePayload ispPrepare; ispPrepare.embeddedbufferId_ = RPi::BufferMask::EMBEDDED_DATA | embeddedId; ispPrepare.bayerbufferId_ = RPi::BufferMask::BAYER_DATA | bayerId; + LIBCAMERA_TRACEPOINT(ipa_call_start, (char *)"rpi", (char *)"signalIspPrepare"); ipa_->signalIspPrepare(ispPrepare); } From patchwork Wed Oct 28 10:31:50 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Paul Elder X-Patchwork-Id: 10291 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id B1B5CBDB1E for ; Wed, 28 Oct 2020 10:32:12 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 7F66B625F9; Wed, 28 Oct 2020 11:32:12 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="ICa8DWQW"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id CDA33625F9 for ; Wed, 28 Oct 2020 11:32:09 +0100 (CET) Received: from pyrite.rasen.tech (unknown [IPv6:2400:4051:61:600:2c71:1b79:d06d:5032]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 45DA799A; Wed, 28 Oct 2020 11:32:07 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1603881129; bh=zFRmPxhWd2ZJKQXD7T3TbMRoyG3X0kE1R0Hff/julZY=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=ICa8DWQWXrGsPHDHc2H3auzDU0LxPk3CNg6mNWZoLqYMkDvwl8gouZJLNXaOP9YY/ ESquJdFI4ZnsliDPIfsO1GnOkKWHY1E7j5Typovn4hbrcEHk0j5/EenJxs9W1MTZEu jU9rzbzB8t1T8o3PdcY+vqzUn/gF0VmgwY6xYjXU= From: Paul Elder To: libcamera-devel@lists.libcamera.org Date: Wed, 28 Oct 2020 19:31:50 +0900 Message-Id: <20201028103151.34575-5-paul.elder@ideasonboard.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20201028103151.34575-1-paul.elder@ideasonboard.com> References: <20201028103151.34575-1-paul.elder@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v2 5/6] utils: tracepoints: Add simple statistics script X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Add a script that scans a trace for IPA call tracepoints, and returns statistics on the time taken for IPA calls. Signed-off-by: Paul Elder --- New in v2 --- utils/tracepoints/analyze.py | 64 ++++++++++++++++++++++++++++++++++++ 1 file changed, 64 insertions(+) create mode 100755 utils/tracepoints/analyze.py diff --git a/utils/tracepoints/analyze.py b/utils/tracepoints/analyze.py new file mode 100755 index 00000000..be56ca28 --- /dev/null +++ b/utils/tracepoints/analyze.py @@ -0,0 +1,64 @@ +#!/usr/bin/env python3 +# SPDX-License-Identifier: GPL-2.0-or-later +# Copyright (C) 2020, Google Inc. +# +# Author: Paul Elder +# +# analyze.py - Analysis tool for libcamera lttng output + +import argparse +import bt2 +import statistics as stats +import sys + +# pipeline -> {function -> stack(timestamps)} +timestamps = {} + +# pipeline:function -> samples[] +samples = {} + +def main(argv): + parser = argparse.ArgumentParser() + parser.add_argument("-p", "--pipeline", type=str, + help="Name of pipeline to filter for") + parser.add_argument('trace_path', type=str, + help='Path to lttng trace') + args = parser.parse_args(argv[1:]) + + traces = bt2.TraceCollectionMessageIterator(args.trace_path) + for msg in traces: + if type(msg) is not bt2._EventMessageConst or \ + (args.pipeline is not None and \ + msg.event.payload_field['pipeline_name'] != args.pipeline): + continue + + pipeline = msg.event.payload_field['pipeline_name'] + event = msg.event.name + func = msg.event.payload_field["function_name"] + timestamp_ns = msg.default_clock_snapshot.ns_from_origin + + if event == 'libcamera:ipa_call_start': + if pipeline not in timestamps: + timestamps[pipeline] = {} + if func not in timestamps[pipeline]: + timestamps[pipeline][func] = [] + timestamps[pipeline][func].append(timestamp_ns) + + if event == 'libcamera:ipa_call_finish': + ts = timestamps[pipeline][func].pop() + key = f'{pipeline}:{func}' + if key not in samples: + samples[key] = [] + samples[key].append(timestamp_ns - ts) + + print('pipeline:function\t:\tmin\tmax\tmean\tstddev') + for k, v in samples.items(): + mean = int(stats.mean(v)) + stddev = int(stats.stdev(v)) + minv = min(v) + maxv = max(v) + print(f'{k}\t:\t{minv}\t{maxv}\t{mean}\t{stddev}') + + +if __name__ == '__main__': + sys.exit(main(sys.argv)) From patchwork Wed Oct 28 10:31:51 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Paul Elder X-Patchwork-Id: 10292 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 23320BDB1E for ; Wed, 28 Oct 2020 10:32:14 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id E2162625EF; Wed, 28 Oct 2020 11:32:13 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="m49K5NSn"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id B743D622AA for ; Wed, 28 Oct 2020 11:32:11 +0100 (CET) Received: from pyrite.rasen.tech (unknown [IPv6:2400:4051:61:600:2c71:1b79:d06d:5032]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 38B0199A; Wed, 28 Oct 2020 11:32:09 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1603881131; bh=+838Y5Lx+BBccpV3d352tfg42yj5PqS21elx1n5dEK4=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=m49K5NSneDk3dtLn7s2l6wUFN4CK1ZbqUvjdVYgxXpqcRRFhX7JMc4vzpN9Bypuik pGJXqCulrFy9HN47C6Yi7k33QBGM+FvquFsVZ5EApTeHrDNsuv8UQnEXvHpaqK2r74 Pf9cDSXb57oXyDZpW+y30l+38bnU4XPv7013I+Qg= From: Paul Elder To: libcamera-devel@lists.libcamera.org Date: Wed, 28 Oct 2020 19:31:51 +0900 Message-Id: <20201028103151.34575-6-paul.elder@ideasonboard.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20201028103151.34575-1-paul.elder@ideasonboard.com> References: <20201028103151.34575-1-paul.elder@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v2 6/6] Documentation: tracing: Add tracing guide X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Add guide for tracepoints, including how to define and use them. Signed-off-by: Paul Elder --- New in v2 --- Documentation/guides/tracing.rst | 127 +++++++++++++++++++++++++++++++ Documentation/index.rst | 1 + Documentation/meson.build | 1 + 3 files changed, 129 insertions(+) create mode 100644 Documentation/guides/tracing.rst diff --git a/Documentation/guides/tracing.rst b/Documentation/guides/tracing.rst new file mode 100644 index 00000000..31d4ff6e --- /dev/null +++ b/Documentation/guides/tracing.rst @@ -0,0 +1,127 @@ +.. SPDX-License-Identifier: CC-BY-SA-4.0 + +Tracing Guide +============= + +Guide to tracing in libcamera. + +Profiling vs Tracing +-------------------- + +Profiling is sampling at periodic points in time. This can be done with other +tools such as callgrind, perf, gprof, etc., without modification to the +application. + +Tracing is recording timestamps at specific locations. libcamera provides a +tracing facility. This guide shows how to use this tracing facility. + +Compiling +--------- + +To compile libcamera with tracing support, it must be enabled in meson, with +``-Dtracepoints=enabled``. It depends on ``liblttng-ust-dev`` (for Debian-based +distros). By default this build configuration option is set to ``auto``, so if +liblttng is detected, it will be enabled by default. Conversely, if the option +is set to disabled, then libcamera will be compiled without tracing support. + +Defining tracepoints +-------------------- + +The first of two steps to using tracepoints is to define the tracepoints. + +Create a file ``include/libcamera/internal/tracepoints/{file}.tp``, where +``file`` is a reasonable name related to the category of tracepoints that +you wish to define. For example, a tracepoints file for the Request object +would be called ``request.tp``. An entry for this file must be added in +``include/libcamera/internal/tracepoints/meson.build``. + +In this tracepoints file, define your tracepoints `as mandated by lttng +`_. The header boilerplate is not necessary, +and you may simply begin with ``TRACEPOINT_EVENT`` (or similar). + +All tracepoint providers shall be ``libcamera``. According to lttng, the +tracepoint provider shold be per-project; this is the rationale for this +decision. To group tracepoint events, we recommend using +``{class_name}_{tracepoint_name}``, for example, ``request_construct`` for a +tracepoint for the constructor of a Request object. + +Tracepoint arguments may take C++ objects pointers, in which case the usual +C++ namespacing rules apply. The header that contains the necessary class +definitions should be ``#include`` ed as well. + +Note: the final parameter in ``TP_ARGS`` *must not* have a trailing comma, and +the parametsrs to ``TP_FIELDS`` are *space-separated*. Not following these will +cause compiler errors. + +Using tracepoints (in libcamera) +-------------------------------- + +To use tracepoints in libcamera, first the header needs to be included: + +``#include "libcamera/internal/tracepoints.h"`` + +Then to use the tracepoint: + +``LIBCAMERA_TRACEPOINT({tracepoint_event}, args...)`` + +This macro must be used, as opposed to lttng's macros directly, because +lttng is an optional dependency of libcamera, so the code must compile and run +even when lttng is not present. + +The tracepoint provider name, as declared in the tracepoint definition, is not +included in the parameters of the tracepoint. + +Using tracepoints (from an application) +--------------------------------------- + +As applications are not part of libcamera, but rather users of libcamera, +applications should seek their own tracing mechanisms. For ease of tracing +the application alongside tracing libcamera, it is recommended to also +`use lttng `_. + +Using tracepoints (from closed-source IPA) +------------------------------------------ + +Similar to applications, closed-source IPAs can simply use lttng on their own, +or any other tracing mechanism if desired. + +Collecting a trace +------------------ + +A trace can be collected fairly simply from lttng: + +.. code-block:: bash + + lttng create $SESSION_NAME + lttng enable-event -u libcamera:\* + lttng start + # run libcamera application + lttng stop + lttng view + lttng destroy $SESSION_NAME + +See the `lttng documentation `_ for further details. + +Analyzing a trace +----------------- + +As mentioned above, while an lttng tracing session exists and the trace is not +running, the trace output can be obtained by ``lttng view``. + +babeltrace2 can also be used to view the trace log as text output. See the +`lttng documentation `_ +for further details. + +babeltrace2 also has a C API and python bindings that can be used to process +traces. See the +`lttng python bindings documentation `_ +and the +`lttng C API documentation `_ +for more details. + +As an example, there is a script ``utils/tracepoints/analyze.py`` that +searches the trace for pairs of events ``libcamera:ipa_call_start`` and +``libcamera:ipa_call_finish``, and gathers statistics on the time interval +between these pairs of these events, grouped by function name. In other words, +this script gathers statistics for the time taken for an IPA function call +(with the tracepoints inserted in the appropriate places). diff --git a/Documentation/index.rst b/Documentation/index.rst index 173c326f..8bc8922e 100644 --- a/Documentation/index.rst +++ b/Documentation/index.rst @@ -17,3 +17,4 @@ Application Writer's Guide Pipeline Handler Writer's Guide IPA Writer's guide + Tracing guide diff --git a/Documentation/meson.build b/Documentation/meson.build index f2300dac..17f3b9d7 100644 --- a/Documentation/meson.build +++ b/Documentation/meson.build @@ -55,6 +55,7 @@ if sphinx.found() 'guides/ipa.rst', 'guides/application-developer.rst', 'guides/pipeline-handler.rst', + 'guides/tracing.rst', ] release = 'release=v' + libcamera_git_version