From patchwork Tue Apr 23 10:30:16 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Milan Zamazal X-Patchwork-Id: 19916 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 6F13ABE08B for ; Tue, 23 Apr 2024 10:31:21 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 2CA9B61B28; Tue, 23 Apr 2024 12:31:21 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.b="STU03Tsl"; dkim-atps=neutral Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 500FF61B28 for ; Tue, 23 Apr 2024 12:31:19 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1713868278; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=5Y0cjrNXJikFyaiXri9dtLKH9ma0VST8InLnd5YrirA=; b=STU03TslvZowg8xYx/hGQkkSCDQRRa03X6BeFw6PRmp9Zsf+vpmV5RlwhqnhU1WHV+Z5Jk sYDJLyVe1ux7deec1SvGZH+p6qsH9Eq++XoegRwzCnJxqX2ztM1KNLvIGr+sg7cZJP1zPd CzS5sMAbf9w91zMeP3N1UzwRw/Kkxa8= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-623-1dzDlzlCOJGgGjxeyRunJw-1; Tue, 23 Apr 2024 06:31:16 -0400 X-MC-Unique: 1dzDlzlCOJGgGjxeyRunJw-1 Received: from smtp.corp.redhat.com (int-mx07.intmail.prod.int.rdu2.redhat.com [10.11.54.7]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 30C1818065AE; Tue, 23 Apr 2024 10:31:16 +0000 (UTC) Received: from nuthatch.redhat.com (unknown [10.45.225.63]) by smtp.corp.redhat.com (Postfix) with ESMTP id E91F81C060D0; Tue, 23 Apr 2024 10:31:14 +0000 (UTC) From: Milan Zamazal To: libcamera-devel@lists.libcamera.org Cc: Milan Zamazal , Naushir Patuck Subject: [PATCH v2 01/13] config: Introduce global runtime configuration Date: Tue, 23 Apr 2024 12:30:16 +0200 Message-ID: <20240423103034.364150-2-mzamazal@redhat.com> In-Reply-To: <20240423103034.364150-1-mzamazal@redhat.com> References: <20240423103034.364150-1-mzamazal@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.4.1 on 10.11.54.7 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Currently, libcamera can be configured in runtime using several environment variables. With introducing more and more variables, this mechanism reaches its limits. It would be simpler and more flexible if it was possible to configure libcamera in a single file. For example, there have been a request for defining pipeline precedence in runtime. We want to compile in multiple pipelines, in order to have them accessible within single packages in distributions. And then being able to select among the pipelines manually as needed based on the particular hardware or operating system environment. Having the configuration file then allows easy switching between hardware, GPU or CPU IPAs. Another possible use case is tuning image output, especially with software ISP, to user liking. For example, some users may prefer higher contrast without the need to use the corresponding knobs, if present at all, in every application. The configuration file can also be used to enable or disable experimental features and avoid the need to track local patches changing configuration options hard-wired in the code when working on new features. This patch introduces basic support for configuration files. GlobalConfiguration class reads, stores and accesses the configuration. Its instance is stored as a private singleton accessed using a static method of the class. There is currently no reason to have more than one instance. libcamera configuration can be specified using a system-wide configuration file or a user configuration file. The user configuration file takes precedence if present. There is currently no way to merge multiple configuration files, the one found is used as the only configuration file. If no configuration file is present, nothing changes to the current libcamera behavior (except for some log messages related to configuration file lookup). The configuration file is a YAML file. We already have a mechanism for handling YAML configuration files in libcamera and the given infrastructure can be reused for the purpose. However, the configuration type is abstracted to make contingent future change of the underlying class easier while retaining (most of) the original API. The configuration is versioned. This has currently no particular meaning but is likely to have its purpose in future, especially once configuration validation is introduced. The configuration YAML file looks as follows: --- version: 1 configuration: WHATEVER CONFIGURATION NEEDED This patch introduces just the basic idea. Actually using the configuration in the corresponding places (everything what is currently configurable via environment variables should be configurable in the file configuration), refining the mechanism (e.g. configuration file validation to avoid mistakes caused by typos) and other improvements will be addressed in followup patches. Signed-off-by: Milan Zamazal --- .../libcamera/internal/global_configuration.h | 38 ++++ include/libcamera/internal/meson.build | 1 + src/libcamera/global_configuration.cpp | 163 ++++++++++++++++++ src/libcamera/meson.build | 1 + 4 files changed, 203 insertions(+) create mode 100644 include/libcamera/internal/global_configuration.h create mode 100644 src/libcamera/global_configuration.cpp diff --git a/include/libcamera/internal/global_configuration.h b/include/libcamera/internal/global_configuration.h new file mode 100644 index 00000000..2f38056a --- /dev/null +++ b/include/libcamera/internal/global_configuration.h @@ -0,0 +1,38 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2024 Red Hat, inc. + * + * global_configuration.h - Global configuration handling + */ + +#pragma once + +#include +#include + +#include "libcamera/internal/yaml_parser.h" + +namespace libcamera { + +class GlobalConfiguration +{ +public: + using Configuration = const YamlObject &; + + static unsigned int version(); + static Configuration configuration(); + +private: + static const std::vector globalConfigurationFiles; + + bool initialized_; + std::unique_ptr configuration_; + + GlobalConfiguration(); + bool loadFile(const std::filesystem::path &fileName); + void load(); + static const GlobalConfiguration &instance(); + static Configuration get(); +}; + +} /* namespace libcamera */ diff --git a/include/libcamera/internal/meson.build b/include/libcamera/internal/meson.build index 160fdc37..640a54aa 100644 --- a/include/libcamera/internal/meson.build +++ b/include/libcamera/internal/meson.build @@ -28,6 +28,7 @@ libcamera_internal_headers = files([ 'dma_heaps.h', 'formats.h', 'framebuffer.h', + 'global_configuration.h', 'ipa_manager.h', 'ipa_module.h', 'ipa_proxy.h', diff --git a/src/libcamera/global_configuration.cpp b/src/libcamera/global_configuration.cpp new file mode 100644 index 00000000..96088b36 --- /dev/null +++ b/src/libcamera/global_configuration.cpp @@ -0,0 +1,163 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2024 Red Hat, inc. + * + * global_configuration.cpp - Global configuration handling + */ + +#include "libcamera/internal/global_configuration.h" + +#include +#include +#include +#include + +#include +#include +#include + +#include "libcamera/internal/yaml_parser.h" + +namespace libcamera { + +LOG_DEFINE_CATEGORY(Configuration) + +/** + * \class GlobalConfiguration + * \brief Support for global libcamera configuration + * + * The configuration file is a YAML file and the configuration itself is stored + * under `configuration' top-level item. + * + * The configuration file is looked up in user's home directory first and if it + * is not found then in system-wide configuration directories. If multiple + * configuration files exist then only the first one found is used and no + * configuration merging is performed. + * + * The class is used as a private singleton and the configuration can be + * accessed usingGlobalConfiguration::configuration(). + */ + +/** + * \typedef GlobalConfiguration::Configuration + * \brief Type representing global libcamera configuration + * + * All code outside GlobalConfiguration must use this type declaration and not + * the underlying type. + */ + +GlobalConfiguration::GlobalConfiguration() + : initialized_(false), configuration_(std::make_unique()) +{ +} + +const std::vector + GlobalConfiguration::globalConfigurationFiles = { + std::filesystem::path(LIBCAMERA_SYSCONF_DIR) / "configuration.yaml", + std::filesystem::path("/etc/libcamera/configuration.yaml"), + }; + +void GlobalConfiguration::load() +{ + std::filesystem::path userConfigurationDirectory; + char *xdgConfigHome = utils::secure_getenv("XDG_CONFIG_HOME"); + if (xdgConfigHome) { + userConfigurationDirectory = xdgConfigHome; + } else { + const char *home = utils::secure_getenv("HOME"); + if (home) + userConfigurationDirectory = + std::filesystem::path(home) / ".config"; + } + + if (!userConfigurationDirectory.empty()) { + std::filesystem::path user_configuration_file = + userConfigurationDirectory / "libcamera" / "configuration.yaml"; + if (loadFile(user_configuration_file)) + return; + } + + for (auto path : globalConfigurationFiles) + if (loadFile(path)) + return; + + initialized_ = true; + LOG(Configuration, Debug) << "No configuration file found"; +} + +bool GlobalConfiguration::loadFile(const std::filesystem::path &fileName) +{ + File file(fileName); + if (!file.exists()) { + return false; + } + + if (!file.open(File::OpenModeFlag::ReadOnly)) { + LOG(Configuration, Warning) + << "Failed to open configuration file '" << fileName << "'"; + return true; + } + + auto root = YamlParser::parse(file); + if (!root) { + LOG(Configuration, Warning) << "Failed to parse configuration file " + << fileName; + return true; + } + configuration_ = std::move(root); + initialized_ = true; + LOG(Configuration, Info) << "Configuration file " << fileName << "loaded"; + + return true; +} + +const GlobalConfiguration &GlobalConfiguration::instance() +{ + static GlobalConfiguration configuration; + if (!configuration.initialized_) { + configuration.load(); + } + return configuration; +} + +GlobalConfiguration::Configuration GlobalConfiguration::get() +{ + return (*instance().configuration_); +} + +/** + * \brief Return configuration version + * + * The version is (optionally) declared in the configuration file in the + * top-level section `version', alongside `configuration'. This has currently no + * real use but may be needed in future if configuration incompatibilities + * occur. + * + * \return Configuration version as declared in the configuration file or 0 if + * no version is declared there + */ +unsigned int GlobalConfiguration::version() +{ + return get()["version"].get().value_or(0); +} + +/** + * \brief Return libcamera global configuration + * + * This returns the whole configuration stored in the top-level section + * `configuration' of the YAML configuration file. + * + * The requested part of the configuration can be accessed using \a YamlObject + * methods. + * + * \note \a YamlObject type itself shouldn't be used in type declarations to + * avoid trouble if we decide to change the underlying data objects in future. + * + * \return The whole configuration section + */ +GlobalConfiguration::Configuration GlobalConfiguration::configuration() +{ + return get()["configuration"]; +} + +} /* namespace libcamera */ diff --git a/src/libcamera/meson.build b/src/libcamera/meson.build index a3b12bc1..b81c75b8 100644 --- a/src/libcamera/meson.build +++ b/src/libcamera/meson.build @@ -21,6 +21,7 @@ libcamera_sources = files([ 'framebuffer.cpp', 'framebuffer_allocator.cpp', 'geometry.cpp', + 'global_configuration.cpp', 'ipa_controls.cpp', 'ipa_data_serializer.cpp', 'ipa_interface.cpp', From patchwork Tue Apr 23 10:30:17 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Milan Zamazal X-Patchwork-Id: 19918 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 9BF5AC328D for ; Tue, 23 Apr 2024 10:31:25 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 3893E63418; Tue, 23 Apr 2024 12:31:25 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.b="iAiWinFJ"; dkim-atps=neutral Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 4D2EF633ED for ; Tue, 23 Apr 2024 12:31:22 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1713868281; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=woetzTY+HtEPtYJuEWfHtgLTZIgN8sOVk3SAn00PD3Y=; b=iAiWinFJLFSnljzcEN4gjg2qwkRfIhJgvwXsz0OCESB9x9xae91BO46FGKNLWxrSqHbbxB EUvAYSsqLvBn8FcTqeYaZSqMaYy8VIpUl7Is43ujo2XrAAcjTIItpJCnQ81qiaEAoqUk8v 4E8CnnIAjXWxogv0ukdT8YD073mYDk8= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-183-HdVQbnICMvmrGACv-amdfw-1; Tue, 23 Apr 2024 06:31:17 -0400 X-MC-Unique: HdVQbnICMvmrGACv-amdfw-1 Received: from smtp.corp.redhat.com (int-mx07.intmail.prod.int.rdu2.redhat.com [10.11.54.7]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 824AC830E85; Tue, 23 Apr 2024 10:31:17 +0000 (UTC) Received: from nuthatch.redhat.com (unknown [10.45.225.63]) by smtp.corp.redhat.com (Postfix) with ESMTP id 8563D1C060D0; Tue, 23 Apr 2024 10:31:16 +0000 (UTC) From: Milan Zamazal To: libcamera-devel@lists.libcamera.org Cc: Milan Zamazal , Naushir Patuck Subject: [PATCH v2 02/13] config: Move global configuration to base Date: Tue, 23 Apr 2024 12:30:17 +0200 Message-ID: <20240423103034.364150-3-mzamazal@redhat.com> In-Reply-To: <20240423103034.364150-1-mzamazal@redhat.com> References: <20240423103034.364150-1-mzamazal@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.4.1 on 10.11.54.7 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Global configuration may be used by some components from base, most notably logging. This means the configuration must also be present in base. And YAML object and parsing too. Let's move that all there. Signed-off-by: Milan Zamazal --- src/libcamera/{ => base}/global_configuration.cpp | 0 src/libcamera/base/meson.build | 15 +++++++++++++++ src/libcamera/{ => base}/yaml_parser.cpp | 0 src/libcamera/meson.build | 15 --------------- 4 files changed, 15 insertions(+), 15 deletions(-) rename src/libcamera/{ => base}/global_configuration.cpp (100%) rename src/libcamera/{ => base}/yaml_parser.cpp (100%) diff --git a/src/libcamera/global_configuration.cpp b/src/libcamera/base/global_configuration.cpp similarity index 100% rename from src/libcamera/global_configuration.cpp rename to src/libcamera/base/global_configuration.cpp diff --git a/src/libcamera/base/meson.build b/src/libcamera/base/meson.build index 7a7fd7e4..b71d3e23 100644 --- a/src/libcamera/base/meson.build +++ b/src/libcamera/base/meson.build @@ -9,6 +9,7 @@ libcamera_base_sources = files([ 'event_notifier.cpp', 'file.cpp', 'flags.cpp', + 'global_configuration.cpp', 'log.cpp', 'message.cpp', 'mutex.cpp', @@ -20,10 +21,12 @@ libcamera_base_sources = files([ 'timer.cpp', 'unique_fd.cpp', 'utils.cpp', + 'yaml_parser.cpp', ]) libdw = dependency('libdw', required : false) libunwind = dependency('libunwind', required : false) +libyaml = dependency('yaml-0.1', required : false) if cc.has_header_symbol('execinfo.h', 'backtrace') config_h.set('HAVE_BACKTRACE', 1) @@ -37,11 +40,23 @@ if libunwind.found() config_h.set('HAVE_UNWIND', 1) endif +# Fallback to a subproject if libyaml isn't found, as it's not packaged in AOSP. +if not libyaml.found() + cmake = import('cmake') + + libyaml_vars = cmake.subproject_options() + libyaml_vars.add_cmake_defines({'CMAKE_POSITION_INDEPENDENT_CODE': 'ON'}) + libyaml_vars.append_compile_args('c', '-Wno-unused-value') + libyaml_wrap = cmake.subproject('libyaml', options : libyaml_vars) + libyaml = libyaml_wrap.dependency('yaml') +endif + libcamera_base_deps = [ libatomic, libdw, libthreads, libunwind, + libyaml, ] # Internal components must use the libcamera_base_private dependency to enable diff --git a/src/libcamera/yaml_parser.cpp b/src/libcamera/base/yaml_parser.cpp similarity index 100% rename from src/libcamera/yaml_parser.cpp rename to src/libcamera/base/yaml_parser.cpp diff --git a/src/libcamera/meson.build b/src/libcamera/meson.build index b81c75b8..fb1ce3fc 100644 --- a/src/libcamera/meson.build +++ b/src/libcamera/meson.build @@ -21,7 +21,6 @@ libcamera_sources = files([ 'framebuffer.cpp', 'framebuffer_allocator.cpp', 'geometry.cpp', - 'global_configuration.cpp', 'ipa_controls.cpp', 'ipa_data_serializer.cpp', 'ipa_interface.cpp', @@ -49,7 +48,6 @@ libcamera_sources = files([ 'v4l2_pixelformat.cpp', 'v4l2_subdevice.cpp', 'v4l2_videodevice.cpp', - 'yaml_parser.cpp', ]) libcamera_sources += libcamera_public_headers @@ -81,7 +79,6 @@ if not cc.has_function('dlopen') libdl = cc.find_library('dl') endif libudev = dependency('libudev', required : get_option('udev')) -libyaml = dependency('yaml-0.1', required : false) # Use one of gnutls or libcrypto (provided by OpenSSL), trying gnutls first. libcrypto = dependency('gnutls', required : false) @@ -117,17 +114,6 @@ if libudev.found() ]) endif -# Fallback to a subproject if libyaml isn't found, as it's not packaged in AOSP. -if not libyaml.found() - cmake = import('cmake') - - libyaml_vars = cmake.subproject_options() - libyaml_vars.add_cmake_defines({'CMAKE_POSITION_INDEPENDENT_CODE': 'ON'}) - libyaml_vars.append_compile_args('c', '-Wno-unused-value') - libyaml_wrap = cmake.subproject('libyaml', options : libyaml_vars) - libyaml = libyaml_wrap.dependency('yaml') -endif - control_sources = [] controls_mode_files = { @@ -183,7 +169,6 @@ libcamera_deps += [ libdl, liblttng, libudev, - libyaml, ] # We add '/' to the build_rpath as a 'safe' path to act as a boolean flag. From patchwork Tue Apr 23 10:30:18 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Milan Zamazal X-Patchwork-Id: 19917 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 3DAE4BE08B for ; Tue, 23 Apr 2024 10:31:24 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id D7A3C633ED; Tue, 23 Apr 2024 12:31:23 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.b="BTealilz"; dkim-atps=neutral Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id EFC5761B3D for ; Tue, 23 Apr 2024 12:31:21 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1713868281; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=fP4E6jc5dGCpe9/ZUllkgi5x75IviwFPZInRlJ1lYv4=; b=BTealilzBRDJJGzVb/9Xdd2/OSSScdQWAoyPkFZx6qxLC45YKRbr8cWgMRac+qz9Az+02N NOyST0AWoVAXHKmdAjM6pgbJTuv3E2HJbY/w06uRlQMCHQeX2XyGucX5LZEJUeK8/xm6rr mp+L8ygp7toc9l9VfvEFbaAjbo2Thuc= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-609-l24x3RVgM0ioKpSfBTFiSw-1; Tue, 23 Apr 2024 06:31:19 -0400 X-MC-Unique: l24x3RVgM0ioKpSfBTFiSw-1 Received: from smtp.corp.redhat.com (int-mx07.intmail.prod.int.rdu2.redhat.com [10.11.54.7]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id F01DE800CA2; Tue, 23 Apr 2024 10:31:18 +0000 (UTC) Received: from nuthatch.redhat.com (unknown [10.45.225.63]) by smtp.corp.redhat.com (Postfix) with ESMTP id 07DA71C0654B; Tue, 23 Apr 2024 10:31:17 +0000 (UTC) From: Milan Zamazal To: libcamera-devel@lists.libcamera.org Cc: Milan Zamazal , Naushir Patuck Subject: [PATCH v2 03/13] config: Look up logging levels in the configuration file Date: Tue, 23 Apr 2024 12:30:18 +0200 Message-ID: <20240423103034.364150-4-mzamazal@redhat.com> In-Reply-To: <20240423103034.364150-1-mzamazal@redhat.com> References: <20240423103034.364150-1-mzamazal@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.4.1 on 10.11.54.7 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" In addition to LIBCAMERA_LOG_LEVELS environment variable, expose the same configuration in the configuration file. Because logging asks for configuration and configuration loading uses logging, extra care is needed. We must prevent calling any log function before configuration is loaded. Otherwise the log function would ask for configuration, it would load the configuration files and would try to log the results, which would cause mutex deadlock due to calling logging inside logging. Not logging in configuration is not an option, at least without introducing complicated mechanism such as delayed logging, because we need to log at least errors when reading the configuration. We must also prevent logging in configuration loading before the configuration is loaded and available (or there is an error in loading the configuration). Otherwise logging would see an empty configuration and wouldn't be configured as asked. Preventing logging before configuration is loaded is ensured by asking for configuration in CameraManager constructor and in IPAManager constructor (which is mysteriously loaded from CameraManager constructor before its body is executed). This seems to be sufficient at least for software ISP. The configuration snippet for logging looks as follows: configuration: log: levels: LIBCAMERA_LOG_LEVELS Signed-off-by: Milan Zamazal --- src/libcamera/base/global_configuration.cpp | 18 +++++++++++++++++- src/libcamera/base/log.cpp | 12 ++++++++++-- src/libcamera/camera_manager.cpp | 9 +++++++++ src/libcamera/ipa_manager.cpp | 12 ++++++++++++ 4 files changed, 48 insertions(+), 3 deletions(-) diff --git a/src/libcamera/base/global_configuration.cpp b/src/libcamera/base/global_configuration.cpp index 96088b36..b45fd341 100644 --- a/src/libcamera/base/global_configuration.cpp +++ b/src/libcamera/base/global_configuration.cpp @@ -29,6 +29,13 @@ LOG_DEFINE_CATEGORY(Configuration) * The configuration file is a YAML file and the configuration itself is stored * under `configuration' top-level item. * + * Example configuration file content: + * \code{.yaml} + * configuration: + * log: + * levels: 'IPAManager:DEBUG' + * \endcode + * * The configuration file is looked up in user's home directory first and if it * is not found then in system-wide configuration directories. If multiple * configuration files exist then only the first one found is used and no @@ -57,6 +64,11 @@ const std::vector std::filesystem::path("/etc/libcamera/configuration.yaml"), }; +/* + * Care is needed here to not log anything before the configuration is + * loaded otherwise the logger would be initialized with empty configuration. + */ + void GlobalConfiguration::load() { std::filesystem::path userConfigurationDirectory; @@ -87,10 +99,15 @@ void GlobalConfiguration::load() bool GlobalConfiguration::loadFile(const std::filesystem::path &fileName) { + /* + * initialized_ must be set to true before any logging is called! + */ + File file(fileName); if (!file.exists()) { return false; } + initialized_ = true; if (!file.open(File::OpenModeFlag::ReadOnly)) { LOG(Configuration, Warning) @@ -105,7 +122,6 @@ bool GlobalConfiguration::loadFile(const std::filesystem::path &fileName) return true; } configuration_ = std::move(root); - initialized_ = true; LOG(Configuration, Info) << "Configuration file " << fileName << "loaded"; return true; diff --git a/src/libcamera/base/log.cpp b/src/libcamera/base/log.cpp index c8045ef7..0564306d 100644 --- a/src/libcamera/base/log.cpp +++ b/src/libcamera/base/log.cpp @@ -14,6 +14,7 @@ #include #include #include +#include #include #include #include @@ -25,6 +26,8 @@ #include #include +#include "libcamera/internal/global_configuration.h" + /** * \file base/log.h * \brief Logging infrastructure @@ -626,8 +629,13 @@ void Logger::parseLogFile() void Logger::parseLogLevels() { const char *debug = utils::secure_getenv("LIBCAMERA_LOG_LEVELS"); - if (!debug) - return; + if (!debug) { + const std::optional confDebug = + GlobalConfiguration::configuration()["log"]["levels"].get(); + if (!confDebug.has_value()) + return; + debug = confDebug.value().c_str(); + } for (const char *pair = debug; *debug != '\0'; pair = debug) { const char *comma = strchrnul(debug, ','); diff --git a/src/libcamera/camera_manager.cpp b/src/libcamera/camera_manager.cpp index 355f3ada..cff8e80c 100644 --- a/src/libcamera/camera_manager.cpp +++ b/src/libcamera/camera_manager.cpp @@ -15,6 +15,7 @@ #include "libcamera/internal/camera.h" #include "libcamera/internal/device_enumerator.h" +#include "libcamera/internal/global_configuration.h" #include "libcamera/internal/pipeline_handler.h" /** @@ -37,6 +38,14 @@ LOG_DEFINE_CATEGORY(Camera) CameraManager::Private::Private() : initialized_(false) { + /* + * This is needed to initialize global configuration + * before any logging is called. Actually, IPAManager + * constructor is called before this, so the configuration + * call is superfluous here, but the initialization belongs + * here. + */ + GlobalConfiguration::configuration(); } int CameraManager::Private::start() diff --git a/src/libcamera/ipa_manager.cpp b/src/libcamera/ipa_manager.cpp index 7a4515d9..5104b930 100644 --- a/src/libcamera/ipa_manager.cpp +++ b/src/libcamera/ipa_manager.cpp @@ -16,6 +16,7 @@ #include #include +#include "libcamera/internal/global_configuration.h" #include "libcamera/internal/ipa_module.h" #include "libcamera/internal/ipa_proxy.h" #include "libcamera/internal/pipeline_handler.h" @@ -105,6 +106,17 @@ IPAManager *IPAManager::self_ = nullptr; */ IPAManager::IPAManager() { + /* + * Make sure configuration is loaded early. + * Otherwise there would be problems with mutual + * configuration-logging dependencies. + * + * This is needed to be here in addition to CameraManager constructor + * because logging is called before CameraManager constructor body is + * executed. + */ + GlobalConfiguration::configuration(); + if (self_) LOG(IPAManager, Fatal) << "Multiple IPAManager objects are not allowed"; From patchwork Tue Apr 23 10:30:19 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Milan Zamazal X-Patchwork-Id: 19919 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 95DB3BE08B for ; Tue, 23 Apr 2024 10:31:28 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 29BFC63422; Tue, 23 Apr 2024 12:31:28 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.b="LUT9tygm"; dkim-atps=neutral Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 9BF4861B28 for ; Tue, 23 Apr 2024 12:31:23 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1713868282; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=DRkufJIFwtkg2bVqj8KPNQyV0U3dyBvm1wXIDWIUiqU=; b=LUT9tygmaRd3B6F1bTuQr98GCwmq51iNy2uy80gHCd2YqverlhH8Z9SL6LLrnf6lVuLtsa 81EjwHSXidB4cjFJyEjoLQjH7XKsScaTApTTs2pmEAhZeHP+gyIQIhDqFe7oT425fkcQWX 61R7rKMYcdIJn0VgpHN3JdatR4FrepA= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-329-fLhySZ0fNEmV-ntxfukGwA-1; Tue, 23 Apr 2024 06:31:20 -0400 X-MC-Unique: fLhySZ0fNEmV-ntxfukGwA-1 Received: from smtp.corp.redhat.com (int-mx07.intmail.prod.int.rdu2.redhat.com [10.11.54.7]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 62D5A80BB33; Tue, 23 Apr 2024 10:31:20 +0000 (UTC) Received: from nuthatch.redhat.com (unknown [10.45.225.63]) by smtp.corp.redhat.com (Postfix) with ESMTP id 574451C060D0; Tue, 23 Apr 2024 10:31:19 +0000 (UTC) From: Milan Zamazal To: libcamera-devel@lists.libcamera.org Cc: Milan Zamazal , Naushir Patuck Subject: [PATCH v2 04/13] config: Add configuration retrieval helpers Date: Tue, 23 Apr 2024 12:30:19 +0200 Message-ID: <20240423103034.364150-5-mzamazal@redhat.com> In-Reply-To: <20240423103034.364150-1-mzamazal@redhat.com> References: <20240423103034.364150-1-mzamazal@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.4.1 on 10.11.54.7 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" They make accessing simple configuration values simpler. Due to the restrictions of YamlObject class, ugly type casting dance is needed. Signed-off-by: Milan Zamazal --- .../libcamera/internal/global_configuration.h | 4 ++ src/libcamera/base/global_configuration.cpp | 50 ++++++++++++++++++- src/libcamera/base/log.cpp | 13 ++--- 3 files changed, 57 insertions(+), 10 deletions(-) diff --git a/include/libcamera/internal/global_configuration.h b/include/libcamera/internal/global_configuration.h index 2f38056a..6c49f8e7 100644 --- a/include/libcamera/internal/global_configuration.h +++ b/include/libcamera/internal/global_configuration.h @@ -8,6 +8,7 @@ #pragma once #include +#include #include #include "libcamera/internal/yaml_parser.h" @@ -21,6 +22,9 @@ public: static unsigned int version(); static Configuration configuration(); + static std::optional option(const char *const confPath); + static std::optional envOption(const char *const envVariable, + const char *const confPath); private: static const std::vector globalConfigurationFiles; diff --git a/src/libcamera/base/global_configuration.cpp b/src/libcamera/base/global_configuration.cpp index b45fd341..c27c91b9 100644 --- a/src/libcamera/base/global_configuration.cpp +++ b/src/libcamera/base/global_configuration.cpp @@ -10,6 +10,7 @@ #include #include #include +#include #include #include @@ -41,8 +42,10 @@ LOG_DEFINE_CATEGORY(Configuration) * configuration files exist then only the first one found is used and no * configuration merging is performed. * - * The class is used as a private singleton and the configuration can be - * accessed usingGlobalConfiguration::configuration(). + * The class is used as a private singleton accessed by the provided + * helpers. Namely GlobalConfiguration::option or GlobalConfiguration::envOption + * to access individual options or GlobalConfiguration::configuration() to + * access the whole configuration. */ /** @@ -141,6 +144,49 @@ GlobalConfiguration::Configuration GlobalConfiguration::get() return (*instance().configuration_); } +/** + * \brief Return value of the configuration option identified by \a confPath + * \param[in] confPath Sequence of the YAML section names (excluding + * `configuration') leading to the requested option separated by dots + * \return A value if an item corresponding to \a confPath exists in the + * configuration file, no value otherwise + */ +std::optional GlobalConfiguration::option( + const char *const confPath) +{ + YamlObject *c = &const_cast(configuration()); + for (auto part : utils::details::StringSplitter(confPath, ".")) + c = &const_cast((*c)[part]); + return c->get(); +} + +/** + * \brief Return value of the configuration option from a file or environment + * \param[in] envVariable Environment variable to get the value from + * \param[in] confPath The same as in GlobalConfiguration::option + * + * This helper looks first at the given environment variable and if it is + * defined then it returns its value (even if it is empty). Otherwise it looks + * for \a confPath the same way as in GlobalConfiguration::option. Only string + * values are supported. + * + * \note Support for using environment variables to configure libcamera behavior + * is provided here mostly for backward compatibility reasons. Introducing new + * configuration environment variables is discouraged. + * + * \return A value retrieved from the given environment option or configuration + * file or no value if not found + */ +std::optional GlobalConfiguration::envOption( + const char *const envVariable, + const char *const confPath) +{ + const char *envValue = utils::secure_getenv(envVariable); + if (envValue) + return std::optional{ std::string{ envValue } }; + return option(confPath); +} + /** * \brief Return configuration version * diff --git a/src/libcamera/base/log.cpp b/src/libcamera/base/log.cpp index 0564306d..7e58e4f9 100644 --- a/src/libcamera/base/log.cpp +++ b/src/libcamera/base/log.cpp @@ -628,14 +628,11 @@ void Logger::parseLogFile() */ void Logger::parseLogLevels() { - const char *debug = utils::secure_getenv("LIBCAMERA_LOG_LEVELS"); - if (!debug) { - const std::optional confDebug = - GlobalConfiguration::configuration()["log"]["levels"].get(); - if (!confDebug.has_value()) - return; - debug = confDebug.value().c_str(); - } + const std::optional confDebug = + GlobalConfiguration::envOption("LIBCAMERA_LOG_LEVELS", "log.levels"); + if (!confDebug.has_value()) + return; + const char *debug = confDebug.value().c_str(); for (const char *pair = debug; *debug != '\0'; pair = debug) { const char *comma = strchrnul(debug, ','); From patchwork Tue Apr 23 10:30:20 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Milan Zamazal X-Patchwork-Id: 19920 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id D43A8C328D for ; Tue, 23 Apr 2024 10:31:29 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 51C6163421; Tue, 23 Apr 2024 12:31:29 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.b="Oqno9TMu"; dkim-atps=neutral Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 7C55A63417 for ; Tue, 23 Apr 2024 12:31:24 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1713868283; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=8llLoDoqFBF0Nnuatsa/CTk9VKfNAki6ZYbOdQOpl+E=; b=Oqno9TMu6xfE7nturNJZvmp6GN/FEwHGNG5uxAFJuu0h8wXYHcbq33ojJYLsA6chllls0N FnlAr0RyDyYL3ypFd9WPG4gqbcVeE3Q1hnnzjgZaPPcMA/tzSL0LTdNbb3hsYHnV+mCNZh n/azWS4uymCgyl6ltiw/VA0lYNNGqmE= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-695-WA5jLzVCOZmus-j7lsAyhg-1; Tue, 23 Apr 2024 06:31:21 -0400 X-MC-Unique: WA5jLzVCOZmus-j7lsAyhg-1 Received: from smtp.corp.redhat.com (int-mx07.intmail.prod.int.rdu2.redhat.com [10.11.54.7]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 9E74A812C39; Tue, 23 Apr 2024 10:31:21 +0000 (UTC) Received: from nuthatch.redhat.com (unknown [10.45.225.63]) by smtp.corp.redhat.com (Postfix) with ESMTP id C14DD1C0666C; Tue, 23 Apr 2024 10:31:20 +0000 (UTC) From: Milan Zamazal To: libcamera-devel@lists.libcamera.org Cc: Milan Zamazal , Naushir Patuck Subject: [PATCH v2 05/13] config: Look up log file in configuration file Date: Tue, 23 Apr 2024 12:30:20 +0200 Message-ID: <20240423103034.364150-6-mzamazal@redhat.com> In-Reply-To: <20240423103034.364150-1-mzamazal@redhat.com> References: <20240423103034.364150-1-mzamazal@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.4.1 on 10.11.54.7 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" This requires change in LIBCAMERA_LOG_FILE interpretation so that we can disable the log file in isolated processes. If LIBCAMERA_LOG_FILE is defined but empty, it means there is no log file, even if defined in the configuration file. This should cause no compatibility trouble since opening a file with an empty name makes little sense. The configuration snippet: configuration: log: file: FILENAME Signed-off-by: Milan Zamazal --- src/libcamera/base/global_configuration.cpp | 1 + src/libcamera/base/log.cpp | 13 +++++++++---- src/libcamera/process.cpp | 11 ++++++++--- 3 files changed, 18 insertions(+), 7 deletions(-) diff --git a/src/libcamera/base/global_configuration.cpp b/src/libcamera/base/global_configuration.cpp index c27c91b9..f72ca352 100644 --- a/src/libcamera/base/global_configuration.cpp +++ b/src/libcamera/base/global_configuration.cpp @@ -34,6 +34,7 @@ LOG_DEFINE_CATEGORY(Configuration) * \code{.yaml} * configuration: * log: + * file: syslog * levels: 'IPAManager:DEBUG' * \endcode * diff --git a/src/libcamera/base/log.cpp b/src/libcamera/base/log.cpp index 7e58e4f9..ff6be340 100644 --- a/src/libcamera/base/log.cpp +++ b/src/libcamera/base/log.cpp @@ -607,16 +607,21 @@ Logger::Logger() */ void Logger::parseLogFile() { - const char *file = utils::secure_getenv("LIBCAMERA_LOG_FILE"); - if (!file) + std::optional file = + GlobalConfiguration::envOption("LIBCAMERA_LOG_FILE", "log.file"); + if (!file.has_value()) return; - if (!strcmp(file, "syslog")) { + auto fileValue = file.value(); + if (fileValue == "") + return; + + if (fileValue == "syslog") { logSetTarget(LoggingTargetSyslog); return; } - logSetFile(file, false); + logSetFile(fileValue.c_str(), false); } /** diff --git a/src/libcamera/process.cpp b/src/libcamera/process.cpp index 86a382fb..4bfce285 100644 --- a/src/libcamera/process.cpp +++ b/src/libcamera/process.cpp @@ -12,7 +12,9 @@ #include #include #include +#include #include +#include #include #include #include @@ -24,6 +26,8 @@ #include #include +#include "libcamera/internal/global_configuration.h" + /** * \file process.h * \brief Process object @@ -263,9 +267,10 @@ int Process::start(const std::string &path, closeAllFdsExcept(fds); - const char *file = utils::secure_getenv("LIBCAMERA_LOG_FILE"); - if (file && strcmp(file, "syslog")) - unsetenv("LIBCAMERA_LOG_FILE"); + std::optional file = + GlobalConfiguration::envOption("LIBCAMERA_LOG_FILE", "log.file"); + if (file.has_value() && file.value() != "syslog") + setenv("LIBCAMERA_LOG_FILE", "", 1); const char **argv = new const char *[args.size() + 2]; unsigned int len = args.size(); From patchwork Tue Apr 23 10:30:21 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Milan Zamazal X-Patchwork-Id: 19922 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id AEDE7BE08B for ; Tue, 23 Apr 2024 10:31:36 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id F24546342C; Tue, 23 Apr 2024 12:31:35 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.b="C+fS3Bna"; dkim-atps=neutral Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id C62D66340B for ; Tue, 23 Apr 2024 12:31:27 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1713868286; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=81tpj4sF5aHADBZjfTHkU93z/3TXnfys6hJ6xl/sRAk=; b=C+fS3BnagWxBt1SLsJVlT7BQxkAsISohPH6qW78jpfkazvI1SlY2ci2Wk8aqrZY7kMtDWP BDQ/wIvkem7VHPJ7P1PhBETBv4wtSpf1lbL089RoSTH9mLZ49MGXYElEMHxm9/cYXVTHVU rGAHDVKzYW3BxZLVqfKDQlTBfMGWUQw= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-437-jUfMoCmXMsmdUPO2y2l7uQ-1; Tue, 23 Apr 2024 06:31:23 -0400 X-MC-Unique: jUfMoCmXMsmdUPO2y2l7uQ-1 Received: from smtp.corp.redhat.com (int-mx07.intmail.prod.int.rdu2.redhat.com [10.11.54.7]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 279F51049C97; Tue, 23 Apr 2024 10:31:23 +0000 (UTC) Received: from nuthatch.redhat.com (unknown [10.45.225.63]) by smtp.corp.redhat.com (Postfix) with ESMTP id 0579B1C0654E; Tue, 23 Apr 2024 10:31:21 +0000 (UTC) From: Milan Zamazal To: libcamera-devel@lists.libcamera.org Cc: Milan Zamazal , Naushir Patuck Subject: [PATCH v2 06/13] config: Look up log color configuration in configuration file Date: Tue, 23 Apr 2024 12:30:21 +0200 Message-ID: <20240423103034.364150-7-mzamazal@redhat.com> In-Reply-To: <20240423103034.364150-1-mzamazal@redhat.com> References: <20240423103034.364150-1-mzamazal@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.4.1 on 10.11.54.7 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" The configuration snippet: configuration: log: color: BOOL In order to use the global configuration access helper for a boolean value, we must make a template from it. To make the template visible, we must implement it in the header file, otherwise undefined reference error would be reported when linking. We don't implement envOption helper for boolean here, because we would have to deal with a specific value interpretation in this particular case. Signed-off-by: Milan Zamazal --- .../libcamera/internal/global_configuration.h | 24 ++++++++++++++++++- src/libcamera/base/global_configuration.cpp | 17 ++++--------- src/libcamera/base/log.cpp | 2 ++ 3 files changed, 30 insertions(+), 13 deletions(-) diff --git a/include/libcamera/internal/global_configuration.h b/include/libcamera/internal/global_configuration.h index 6c49f8e7..a8a965dd 100644 --- a/include/libcamera/internal/global_configuration.h +++ b/include/libcamera/internal/global_configuration.h @@ -11,6 +11,8 @@ #include #include +#include + #include "libcamera/internal/yaml_parser.h" namespace libcamera { @@ -22,7 +24,27 @@ public: static unsigned int version(); static Configuration configuration(); - static std::optional option(const char *const confPath); + + template || + std::is_same_v || + std::is_same_v || + std::is_same_v || + std::is_same_v || + std::is_same_v || + std::is_same_v || + std::is_same_v || + std::is_same_v || + std::is_same_v> * = nullptr> + static std::optional option(const char *const confPath) + { + YamlObject *c = &const_cast(configuration()); + for (auto part : utils::details::StringSplitter(confPath, ".")) + c = &const_cast((*c)[part]); + return c->get(); + } + static std::optional envOption(const char *const envVariable, const char *const confPath); diff --git a/src/libcamera/base/global_configuration.cpp b/src/libcamera/base/global_configuration.cpp index f72ca352..03d0369b 100644 --- a/src/libcamera/base/global_configuration.cpp +++ b/src/libcamera/base/global_configuration.cpp @@ -15,7 +15,6 @@ #include #include -#include #include "libcamera/internal/yaml_parser.h" @@ -146,20 +145,14 @@ GlobalConfiguration::Configuration GlobalConfiguration::get() } /** + * \fn static std::optional GlobalConfiguration::option(const char *const confPath) * \brief Return value of the configuration option identified by \a confPath + * \tparam T The type of the retrieved configuration value * \param[in] confPath Sequence of the YAML section names (excluding * `configuration') leading to the requested option separated by dots - * \return A value if an item corresponding to \a confPath exists in the - * configuration file, no value otherwise + * \return A value of type \a T if an item corresponding to \a confPath exists + * in the configuration file and matches type \a T, no value otherwise */ -std::optional GlobalConfiguration::option( - const char *const confPath) -{ - YamlObject *c = &const_cast(configuration()); - for (auto part : utils::details::StringSplitter(confPath, ".")) - c = &const_cast((*c)[part]); - return c->get(); -} /** * \brief Return value of the configuration option from a file or environment @@ -185,7 +178,7 @@ std::optional GlobalConfiguration::envOption( const char *envValue = utils::secure_getenv(envVariable); if (envValue) return std::optional{ std::string{ envValue } }; - return option(confPath); + return option(confPath); } /** diff --git a/src/libcamera/base/log.cpp b/src/libcamera/base/log.cpp index ff6be340..d4421a36 100644 --- a/src/libcamera/base/log.cpp +++ b/src/libcamera/base/log.cpp @@ -590,6 +590,8 @@ void Logger::logSetLevel(const char *category, const char *level) Logger::Logger() { bool color = !utils::secure_getenv("LIBCAMERA_LOG_NO_COLOR"); + if (color) + color = GlobalConfiguration::option("log.color").value_or(true); logSetStream(&std::cerr, color); parseLogFile(); From patchwork Tue Apr 23 10:30:22 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Milan Zamazal X-Patchwork-Id: 19921 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id EB517BE08B for ; Tue, 23 Apr 2024 10:31:33 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 83BE163429; Tue, 23 Apr 2024 12:31:33 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.b="GVhu98LB"; dkim-atps=neutral Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 512F763408 for ; Tue, 23 Apr 2024 12:31:27 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1713868286; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=G9gT7NCrUojJesv1bHb6Y4rhUHkBAfyVBisjLO6L0Xg=; b=GVhu98LBKgvhG2GEEpzMZRpLOFk39vPXWfRn/49yGfn8ncFGZd47jUzK/ccrwpMd+n+VOx 1fGzjB8txGvHvKnmY5rGVDqpcbrza0dgilRzfZgpyEXTWrQWZ9Zwof4lyenDF5MCOVFn+B aNtleIjuwli7sYuolhamIfURM8NskmQ= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-121-o-O65HC5OyGpEbUHMCyROA-1; Tue, 23 Apr 2024 06:31:24 -0400 X-MC-Unique: o-O65HC5OyGpEbUHMCyROA-1 Received: from smtp.corp.redhat.com (int-mx07.intmail.prod.int.rdu2.redhat.com [10.11.54.7]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 78E8518065AA; Tue, 23 Apr 2024 10:31:24 +0000 (UTC) Received: from nuthatch.redhat.com (unknown [10.45.225.63]) by smtp.corp.redhat.com (Postfix) with ESMTP id 87FEE1C0666B; Tue, 23 Apr 2024 10:31:23 +0000 (UTC) From: Milan Zamazal To: libcamera-devel@lists.libcamera.org Cc: Milan Zamazal , Naushir Patuck Subject: [PATCH v2 07/13] config: Look up rpi paths in configuration file Date: Tue, 23 Apr 2024 12:30:22 +0200 Message-ID: <20240423103034.364150-8-mzamazal@redhat.com> In-Reply-To: <20240423103034.364150-1-mzamazal@redhat.com> References: <20240423103034.364150-1-mzamazal@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.4.1 on 10.11.54.7 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" It may be arguable whether pipeline specific configurations belong to the global configuration file. But: - Having a single configuration file is generally easier for the user. - The original configuration via environment variables can be already considered global. - These options point to other configuration files and it makes little sense to add another configuration file to the chain. The configuration snippet: configuration: pipeline: rpi: config_file: FILENAME tuning_file: FILENAME Signed-off-by: Milan Zamazal --- .../pipeline/rpi/common/pipeline_base.cpp | 16 ++++++++++++---- 1 file changed, 12 insertions(+), 4 deletions(-) diff --git a/src/libcamera/pipeline/rpi/common/pipeline_base.cpp b/src/libcamera/pipeline/rpi/common/pipeline_base.cpp index 7e420b3f..eb6e030f 100644 --- a/src/libcamera/pipeline/rpi/common/pipeline_base.cpp +++ b/src/libcamera/pipeline/rpi/common/pipeline_base.cpp @@ -20,6 +20,7 @@ #include #include "libcamera/internal/camera_lens.h" +#include "libcamera/internal/global_configuration.h" #include "libcamera/internal/ipa_manager.h" #include "libcamera/internal/v4l2_subdevice.h" @@ -1086,7 +1087,12 @@ int CameraData::loadPipelineConfiguration() /* Initial configuration of the platform, in case no config file is present */ platformPipelineConfigure({}); - char const *configFromEnv = utils::secure_getenv("LIBCAMERA_RPI_CONFIG_FILE"); + std::optional configFile = + GlobalConfiguration::envOption("LIBCAMERA_RPI_CONFIG_FILE", + "pipeline.rpi.config_file"); + if (!configFile.has_value()) + return 0; + char const *configFromEnv = configFile.value().c_str(); if (!configFromEnv || *configFromEnv == '\0') return 0; @@ -1145,14 +1151,16 @@ int CameraData::loadIPA(ipa::RPi::InitResult *result) * the environment variable overrides it. */ std::string configurationFile; - char const *configFromEnv = utils::secure_getenv("LIBCAMERA_RPI_TUNING_FILE"); - if (!configFromEnv || *configFromEnv == '\0') { + std::optional configFromEnv = + GlobalConfiguration::envOption("LIBCAMERA_RPI_TUNING_FILE", + "pipeline.rpi.tuning_file"); + if (!configFromEnv.has_value() || configFromEnv.value().empty()) { std::string model = sensor_->model(); if (isMonoSensor(sensor_)) model += "_mono"; configurationFile = ipa_->configurationFile(model + ".json"); } else { - configurationFile = std::string(configFromEnv); + configurationFile = configFromEnv.value(); } IPASettings settings(configurationFile, sensor_->model()); From patchwork Tue Apr 23 10:30:23 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Milan Zamazal X-Patchwork-Id: 19923 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 09710C328D for ; Tue, 23 Apr 2024 10:31:38 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 85F8E6342B; Tue, 23 Apr 2024 12:31:37 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.b="Pr2IdIok"; dkim-atps=neutral Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 628B063424 for ; Tue, 23 Apr 2024 12:31:30 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1713868289; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=t2TslSbFuSDyHZHcHr2C23GUF38/PcUUW1mKWTLdcAc=; b=Pr2IdIoka/hd5MfRTfoFQ2Fvc7Lc/CewykZydjxznS1A1WB4QdO7xWmfdJ+jnq1rztEbCH qMZo7Pvm6aCvDNuuxuNI4N0OyDPZ2IlNR2+2Jf09PVOlXx76TGPKjj11VxbVE7k3L9p+OT XUZ7+d1fdUgyI0ioLZkVV6qzHVj2yoM= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-30-wn2lHdJNPy2EnGCmika4QA-1; Tue, 23 Apr 2024 06:31:26 -0400 X-MC-Unique: wn2lHdJNPy2EnGCmika4QA-1 Received: from smtp.corp.redhat.com (int-mx07.intmail.prod.int.rdu2.redhat.com [10.11.54.7]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id E1EEF81F317; Tue, 23 Apr 2024 10:31:25 +0000 (UTC) Received: from nuthatch.redhat.com (unknown [10.45.225.63]) by smtp.corp.redhat.com (Postfix) with ESMTP id BFDBB1C0666A; Tue, 23 Apr 2024 10:31:24 +0000 (UTC) From: Milan Zamazal To: libcamera-devel@lists.libcamera.org Cc: Milan Zamazal , Naushir Patuck Subject: [PATCH v2 08/13] config: Look up IPA configurables in configuration file Date: Tue, 23 Apr 2024 12:30:23 +0200 Message-ID: <20240423103034.364150-9-mzamazal@redhat.com> In-Reply-To: <20240423103034.364150-1-mzamazal@redhat.com> References: <20240423103034.364150-1-mzamazal@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.4.1 on 10.11.54.7 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" The configuration snippet: configuration: ipa: config_paths: CONFIG:PATHS:... module_paths: MODULE:PATHS:... force_isolation: BOOL Signed-off-by: Milan Zamazal --- src/libcamera/ipa_manager.cpp | 11 ++++++++--- src/libcamera/ipa_proxy.cpp | 8 ++++++-- 2 files changed, 14 insertions(+), 5 deletions(-) diff --git a/src/libcamera/ipa_manager.cpp b/src/libcamera/ipa_manager.cpp index 5104b930..02a09724 100644 --- a/src/libcamera/ipa_manager.cpp +++ b/src/libcamera/ipa_manager.cpp @@ -129,8 +129,11 @@ IPAManager::IPAManager() unsigned int ipaCount = 0; /* User-specified paths take precedence. */ - const char *modulePaths = utils::secure_getenv("LIBCAMERA_IPA_MODULE_PATH"); - if (modulePaths) { + const auto confModulePaths = + GlobalConfiguration::envOption( + "LIBCAMERA_IPA_MODULE_PATH", "ipa.module_paths"); + if (confModulePaths.has_value()) { + const char *modulePaths = confModulePaths.value().c_str(); for (const auto &dir : utils::split(modulePaths, ":")) { if (dir.empty()) continue; @@ -308,7 +311,9 @@ bool IPAManager::isSignatureValid([[maybe_unused]] IPAModule *ipa) const { #if HAVE_IPA_PUBKEY char *force = utils::secure_getenv("LIBCAMERA_IPA_FORCE_ISOLATION"); - if (force && force[0] != '\0') { + if ((force && force[0] != '\0') || + (!force && GlobalConfiguration::option("ipa.force_isolation") + .value_or(false))) { LOG(IPAManager, Debug) << "Isolation of IPA module " << ipa->path() << " forced through environment variable"; diff --git a/src/libcamera/ipa_proxy.cpp b/src/libcamera/ipa_proxy.cpp index 3f2cc6b8..368c1452 100644 --- a/src/libcamera/ipa_proxy.cpp +++ b/src/libcamera/ipa_proxy.cpp @@ -15,6 +15,7 @@ #include #include +#include "libcamera/internal/global_configuration.h" #include "libcamera/internal/ipa_module.h" /** @@ -104,8 +105,11 @@ std::string IPAProxy::configurationFile(const std::string &name) const std::string ipaName = ipam_->info().name; /* Check the environment variable first. */ - const char *confPaths = utils::secure_getenv("LIBCAMERA_IPA_CONFIG_PATH"); - if (confPaths) { + auto confConfPaths = + GlobalConfiguration::envOption( + "LIBCAMERA_IPA_CONFIG_PATH", "ipa.config_paths"); + if (confConfPaths.has_value()) { + const char *confPaths = confConfPaths.value().c_str(); for (const auto &dir : utils::split(confPaths, ":")) { if (dir.empty()) continue; From patchwork Tue Apr 23 10:30:24 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Milan Zamazal X-Patchwork-Id: 19925 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id F1F37C328D for ; Tue, 23 Apr 2024 10:31:40 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 7F6FD63428; Tue, 23 Apr 2024 12:31:40 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.b="SJ7J9plF"; dkim-atps=neutral Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id C47CB63425 for ; Tue, 23 Apr 2024 12:31:31 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1713868290; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=H5rObzSBGpbgDLgQyfdrfryihEhci/ELnlu2c5EUREA=; b=SJ7J9plFYmNhxVL7hVm3+VP/5aGgD9zld6IXycASpDS3BLz8b96qYlg+O4NxVttmQHpbKB QI7ZrnrBTndXswkUap5ZEXAtLD2jaqUvQv+QtNkgSKBsAGobm6Pax74LK6T7xdUXQBg/lA 56x92VjycB2O85yrTKmO2MI2mMDl+0w= Received: from mimecast-mx02.redhat.com (mx-ext.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-237-xHcNnt-9PAirn5_afKcBjw-1; Tue, 23 Apr 2024 06:31:27 -0400 X-MC-Unique: xHcNnt-9PAirn5_afKcBjw-1 Received: from smtp.corp.redhat.com (int-mx07.intmail.prod.int.rdu2.redhat.com [10.11.54.7]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 1231D3820EA2; Tue, 23 Apr 2024 10:31:27 +0000 (UTC) Received: from nuthatch.redhat.com (unknown [10.45.225.63]) by smtp.corp.redhat.com (Postfix) with ESMTP id 4E6CC1C0654E; Tue, 23 Apr 2024 10:31:26 +0000 (UTC) From: Milan Zamazal To: libcamera-devel@lists.libcamera.org Cc: Milan Zamazal , Naushir Patuck Subject: [PATCH v2 09/13] config: Look up RkISP1 tuning file in configuration file Date: Tue, 23 Apr 2024 12:30:24 +0200 Message-ID: <20240423103034.364150-10-mzamazal@redhat.com> In-Reply-To: <20240423103034.364150-1-mzamazal@redhat.com> References: <20240423103034.364150-1-mzamazal@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.4.1 on 10.11.54.7 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" The configuration snippet: configuration: pipeline: rkisp1: tuning_file: FILE This environment variable has not been documented. Signed-off-by: Milan Zamazal --- src/libcamera/pipeline/rkisp1/rkisp1.cpp | 9 ++++++--- 1 file changed, 6 insertions(+), 3 deletions(-) diff --git a/src/libcamera/pipeline/rkisp1/rkisp1.cpp b/src/libcamera/pipeline/rkisp1/rkisp1.cpp index abb21968..5be25698 100644 --- a/src/libcamera/pipeline/rkisp1/rkisp1.cpp +++ b/src/libcamera/pipeline/rkisp1/rkisp1.cpp @@ -36,6 +36,7 @@ #include "libcamera/internal/delayed_controls.h" #include "libcamera/internal/device_enumerator.h" #include "libcamera/internal/framebuffer.h" +#include "libcamera/internal/global_configuration.h" #include "libcamera/internal/ipa_manager.h" #include "libcamera/internal/media_device.h" #include "libcamera/internal/pipeline_handler.h" @@ -349,8 +350,10 @@ int RkISP1CameraData::loadIPA(unsigned int hwRevision) * environment variable overrides it. */ std::string ipaTuningFile; - char const *configFromEnv = utils::secure_getenv("LIBCAMERA_RKISP1_TUNING_FILE"); - if (!configFromEnv || *configFromEnv == '\0') { + const auto confTuningFile = + GlobalConfiguration::envOption( + "LIBCAMERA_RKISP1_TUNING_FILE", "pipeline.rkisp1.tuning_file"); + if (!confTuningFile.has_value() || confTuningFile.value() == "") { ipaTuningFile = ipa_->configurationFile(sensor_->model() + ".yaml"); /* * If the tuning file isn't found, fall back to the @@ -359,7 +362,7 @@ int RkISP1CameraData::loadIPA(unsigned int hwRevision) if (ipaTuningFile.empty()) ipaTuningFile = ipa_->configurationFile("uncalibrated.yaml"); } else { - ipaTuningFile = std::string(configFromEnv); + ipaTuningFile = confTuningFile.value(); } IPACameraSensorInfo sensorInfo{}; From patchwork Tue Apr 23 10:30:25 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Milan Zamazal X-Patchwork-Id: 19924 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id EF873BE08B for ; Tue, 23 Apr 2024 10:31:39 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id C734763430; Tue, 23 Apr 2024 12:31:38 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.b="HzGYu+an"; dkim-atps=neutral Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 46A016340D for ; Tue, 23 Apr 2024 12:31:31 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1713868290; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Pf2rbdOA0GhbtzTvmoyr7iQ8XY9yPxAgWpxSVOLUbZM=; b=HzGYu+an/rNDZ9T0epjWSPkfjXGGFtbzVnOH+Ad/5gbHmD44shu3ZFbsjNqYgiYwFzUiwf /yGxH7JB4Et/FrnqwHzO3tMyjjkzq3O2Zw3B/CNuyUNMBELr541tkX8EMfzzt7c0MsaGPm pD6irc3ab4mznZ88ZYNNcA8m1pkHEzw= Received: from mimecast-mx02.redhat.com (mx-ext.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-190--Tc9jtCPM-CDQcwz81qj4Q-1; Tue, 23 Apr 2024 06:31:28 -0400 X-MC-Unique: -Tc9jtCPM-CDQcwz81qj4Q-1 Received: from smtp.corp.redhat.com (int-mx07.intmail.prod.int.rdu2.redhat.com [10.11.54.7]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 492FE3820EA7; Tue, 23 Apr 2024 10:31:28 +0000 (UTC) Received: from nuthatch.redhat.com (unknown [10.45.225.63]) by smtp.corp.redhat.com (Postfix) with ESMTP id 5F7C31C0666A; Tue, 23 Apr 2024 10:31:27 +0000 (UTC) From: Milan Zamazal To: libcamera-devel@lists.libcamera.org Cc: Milan Zamazal , Naushir Patuck Subject: [PATCH v2 10/13] config: Allow enabling software ISP in runtime Date: Tue, 23 Apr 2024 12:30:25 +0200 Message-ID: <20240423103034.364150-11-mzamazal@redhat.com> In-Reply-To: <20240423103034.364150-1-mzamazal@redhat.com> References: <20240423103034.364150-1-mzamazal@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.4.1 on 10.11.54.7 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" This patch allows enabling or disabling software ISP in runtime in addition to compile time. This can be useful for software ISP testing on various platforms as well as for overriding the defaults in case the defaults don't work well (e.g. hardware ISP may or may not work on i.MX8MP depending on the kernel and libcamera patches present in the given system). The configuration is specified as follows: configuration: pipelines: simple: supported_devices: - driver: DRIVER-NAME software_isp: BOOLEAN - ... For example: configuration: pipelines: simple: supported_devices: - driver: mxc-isi software_isp: true The overall configuration of enabling or disabling software ISP may get dropped in future but this patch is still useful in the meantime. Signed-off-by: Milan Zamazal --- src/libcamera/pipeline/simple/simple.cpp | 12 ++++++++++++ 1 file changed, 12 insertions(+) diff --git a/src/libcamera/pipeline/simple/simple.cpp b/src/libcamera/pipeline/simple/simple.cpp index 61a59926..1c37ee8c 100644 --- a/src/libcamera/pipeline/simple/simple.cpp +++ b/src/libcamera/pipeline/simple/simple.cpp @@ -32,6 +32,7 @@ #include "libcamera/internal/camera_sensor.h" #include "libcamera/internal/converter.h" #include "libcamera/internal/device_enumerator.h" +#include "libcamera/internal/global_configuration.h" #include "libcamera/internal/media_device.h" #include "libcamera/internal/pipeline_handler.h" #include "libcamera/internal/software_isp/software_isp.h" @@ -1545,6 +1546,17 @@ bool SimplePipelineHandler::match(DeviceEnumerator *enumerator) } swIspEnabled_ = info->swIspEnabled; + for (GlobalConfiguration::Configuration entry : + GlobalConfiguration::configuration() + ["pipelines"]["simple"]["supported_devices"] + .asList()) { + auto name = entry["driver"].get(); + if (name.has_value() and !name.value().compare(info->driver)) { + swIspEnabled_ = entry["software_isp"].get().value_or(swIspEnabled_); + LOG(SimplePipeline, Debug) << "Overriding software ISP to " << swIspEnabled_; + break; + } + } /* Locate the sensors. */ std::vector sensors = locateSensors(); From patchwork Tue Apr 23 10:30:26 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Milan Zamazal X-Patchwork-Id: 19926 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id C7D5BBE08B for ; Tue, 23 Apr 2024 10:31:42 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 565176342E; Tue, 23 Apr 2024 12:31:42 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.b="eyI7jX5L"; dkim-atps=neutral Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 2C122633ED for ; Tue, 23 Apr 2024 12:31:32 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1713868291; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=ZTbw4do6UUbclIccaLrZ+AbTwZ279YD/xIq/O9saBy4=; b=eyI7jX5LIrzkThC3pRlVKTE3Jday0iR3EmBmG176bmnyR9nTe10WTwQ/3DjpwLd1kLyviG vNQym6XirKFrR39APYdd4J7btoc09k/ByJB+oJWfAeHXhwbBX4KkfgdscVIpKAYXNxW6CM XnW7ClV54024TT3GCxxAI56hJq5fE2o= Received: from mimecast-mx02.redhat.com (mx-ext.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-33-LwsQnSgcPfilU6fITyMY8g-1; Tue, 23 Apr 2024 06:31:29 -0400 X-MC-Unique: LwsQnSgcPfilU6fITyMY8g-1 Received: from smtp.corp.redhat.com (int-mx07.intmail.prod.int.rdu2.redhat.com [10.11.54.7]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 96E023C0ED4B; Tue, 23 Apr 2024 10:31:29 +0000 (UTC) Received: from nuthatch.redhat.com (unknown [10.45.225.63]) by smtp.corp.redhat.com (Postfix) with ESMTP id A51DF1C060D0; Tue, 23 Apr 2024 10:31:28 +0000 (UTC) From: Milan Zamazal To: libcamera-devel@lists.libcamera.org Cc: Milan Zamazal , Naushir Patuck Subject: [PATCH v2 11/13] config: Add global configuration file documentation Date: Tue, 23 Apr 2024 12:30:26 +0200 Message-ID: <20240423103034.364150-12-mzamazal@redhat.com> In-Reply-To: <20240423103034.364150-1-mzamazal@redhat.com> References: <20240423103034.364150-1-mzamazal@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.4.1 on 10.11.54.7 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Extend (and rename) the documentation of environment variables with information about the configuration file. Signed-off-by: Milan Zamazal --- Documentation/index.rst | 2 +- Documentation/meson.build | 2 +- ...ariables.rst => runtime_configuration.rst} | 96 ++++++++++++++++--- 3 files changed, 87 insertions(+), 13 deletions(-) rename Documentation/{environment_variables.rst => runtime_configuration.rst} (65%) diff --git a/Documentation/index.rst b/Documentation/index.rst index 5442ae75..aa277b85 100644 --- a/Documentation/index.rst +++ b/Documentation/index.rst @@ -19,7 +19,7 @@ Pipeline Handler Writer's Guide IPA Writer's guide Tracing guide - Environment variables + Runtime configuration Sensor driver requirements Lens driver requirements Python Bindings diff --git a/Documentation/meson.build b/Documentation/meson.build index 3872e0a8..f8ca63eb 100644 --- a/Documentation/meson.build +++ b/Documentation/meson.build @@ -70,7 +70,6 @@ if sphinx.found() 'conf.py', 'contributing.rst', 'docs.rst', - 'environment_variables.rst', 'guides/application-developer.rst', 'guides/introduction.rst', 'guides/ipa.rst', @@ -79,6 +78,7 @@ if sphinx.found() 'index.rst', 'lens_driver_requirements.rst', 'python-bindings.rst', + 'runtime_configuration.rst', 'sensor_driver_requirements.rst', 'software-isp-benchmarking.rst', '../README.rst', diff --git a/Documentation/environment_variables.rst b/Documentation/runtime_configuration.rst similarity index 65% rename from Documentation/environment_variables.rst rename to Documentation/runtime_configuration.rst index a9b230bc..d3972363 100644 --- a/Documentation/environment_variables.rst +++ b/Documentation/runtime_configuration.rst @@ -1,43 +1,117 @@ .. SPDX-License-Identifier: CC-BY-SA-4.0 -Environment variables +Runtime configuration ===================== -The libcamera behaviour can be tuned through environment variables. This -document lists all the available variables and describes their usage. +The libcamera behaviour can be tuned through a configuration file or +environment variables. This document lists all the configuration options +and describes their usage. + +General rules +------------- + +The configuration file is looked up in the following locations, in this +order: + +- $XDG_CONFIG_HOME/libcamera/configuration.yaml +- LIBCAMERA_SYSCONF_DIR/configuration.yaml +- /etc/libcamera/configuration.yaml + +The first configuration file found wins, contingent configuration files +in other locations are ignored. + +Settings in environment variables take precedence over settings in +configuration files. This allows overriding behaviour temporarily +without the need to modify configuration files. + +Configuration options +--------------------- + +Here is an overview of the available configuration options, in the YAML +file structure: + +:: + + configuration: + ipa: + config_paths: # full paths to directories, separated by colons + force_isolation: # true/false + module_paths: # full paths to directories, separated by colons + log: + color: # true/false for color/no-color + file: # either `syslog` or a full path + levels: # see Log levels + pipelines: + rkisp1: + tuning_file: # full path + rpi: + config_file: # full path + tuning_file: # full path + simple: + supported_devices: + - driver: # driver name, e.g. `mxc-isi` + software_isp: # true/false + +Configuration file example +-------------------------- + +:: + + --- + version: 1 + configuration: + ipa: + config_paths: /home/user/.libcamera/share/ipa:/opt/libcamera/vendor/share/ipa + module_paths: /home/user/.libcamera/lib:/opt/libcamera/vendor/lib + force_isolation: true + log: + color: false + file: syslog + levels: 'IPAManager:DEBUG' + pipelines: + rkisp1: + tuning_file: /home/user/.libcamera/rkisp1.yaml + rpi: + config_file: /usr/local/share/libcamera/pipeline/rpi/vc4/minimal_mem.yaml + simple: + supported_devices: + - driver: mxc-isi + software_isp: true List of variables ----------------- -LIBCAMERA_LOG_FILE +The corresponding configuration file paths are listed in parentheses. + +LIBCAMERA_LOG_FILE (log.file) The custom destination for log output. Example value: ``/home/{user}/camera_log.log`` -LIBCAMERA_LOG_LEVELS +LIBCAMERA_LOG_LEVELS (log.levels) Configure the verbosity of log messages for different categories (`more `__). Example value: ``*:DEBUG`` -LIBCAMERA_LOG_NO_COLOR +LIBCAMERA_LOG_NO_COLOR (log.color) Disable coloring of log messages (`more `__). -LIBCAMERA_IPA_CONFIG_PATH +LIBCAMERA_IPA_CONFIG_PATH (ipa.config_paths) Define custom search locations for IPA configurations (`more `__). Example value: ``${HOME}/.libcamera/share/ipa:/opt/libcamera/vendor/share/ipa`` -LIBCAMERA_IPA_FORCE_ISOLATION +LIBCAMERA_IPA_FORCE_ISOLATION (ipa.force_isolation) When set to a non-empty string, force process isolation of all IPA modules. Example value: ``1`` -LIBCAMERA_IPA_MODULE_PATH +LIBCAMERA_IPA_MODULE_PATH (ipa.module_paths) Define custom search locations for IPA modules (`more `__). Example value: ``${HOME}/.libcamera/lib:/opt/libcamera/vendor/lib`` -LIBCAMERA_RPI_CONFIG_FILE +LIBCAMERA_RPI_CONFIG_FILE (pipelines.rpi.config_file) Define a custom configuration file to use in the Raspberry Pi pipeline handler. Example value: ``/usr/local/share/libcamera/pipeline/rpi/vc4/minimal_mem.yaml`` @@ -136,7 +210,7 @@ code. IPA configuration ~~~~~~~~~~~~~~~~~ -IPA modules use configuration files to store parameters. The format and +IPA modules use their own configuration files to store parameters. The format and contents of the configuration files is specific to the IPA module. They usually contain tuning parameters for the algorithms, in JSON format. From patchwork Tue Apr 23 10:30:27 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Milan Zamazal X-Patchwork-Id: 19927 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id B155AC32C9 for ; Tue, 23 Apr 2024 10:31:43 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 4BFD063431; Tue, 23 Apr 2024 12:31:43 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.b="U4GP2+7/"; dkim-atps=neutral Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id B99A76342A for ; Tue, 23 Apr 2024 12:31:33 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1713868292; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=1c3Jh08M6XN/ds6T11ZInVfNuDP2I19YuRR0gfY96iA=; b=U4GP2+7/0XGasrY2VqukYWEbmg8qOioadNRBHx605bYyGEalIuS/+MimICEhcbxLix802m GZh09w3BylaVU/vJR3+LVE8NXjwpaaKbmJYN9WXDp7Egi6J2bdK6t8ptNu+djRhlXruA1N I9nxgg4e9l7CHz03nm+DzW4GXsyhY/I= Received: from mimecast-mx02.redhat.com (mx-ext.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-304-Typha7BmNEGWzfx8Rgmlbw-1; Tue, 23 Apr 2024 06:31:31 -0400 X-MC-Unique: Typha7BmNEGWzfx8Rgmlbw-1 Received: from smtp.corp.redhat.com (int-mx07.intmail.prod.int.rdu2.redhat.com [10.11.54.7]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id B5D8229ABA14; Tue, 23 Apr 2024 10:31:30 +0000 (UTC) Received: from nuthatch.redhat.com (unknown [10.45.225.63]) by smtp.corp.redhat.com (Postfix) with ESMTP id DD0221C060D0; Tue, 23 Apr 2024 10:31:29 +0000 (UTC) From: Milan Zamazal To: libcamera-devel@lists.libcamera.org Cc: Milan Zamazal , Naushir Patuck Subject: [PATCH v2 12/13] libcamera: software_isp: Make input buffer copying configurable Date: Tue, 23 Apr 2024 12:30:27 +0200 Message-ID: <20240423103034.364150-13-mzamazal@redhat.com> In-Reply-To: <20240423103034.364150-1-mzamazal@redhat.com> References: <20240423103034.364150-1-mzamazal@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.4.1 on 10.11.54.7 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" On some platforms, working directly on the input buffer is very slow due to disabled caching. This is why we copy the input buffer into standard (cached) memory. This is an unnecessary overhead on platforms with cached buffers. Let's make input buffer copying configurable. The default is still copying, as it's overhead is much lower than contingent operations on non-cached memory. Ideally, we should improve this in future to set the default to non-copying if we can be sure under observable circumstances that we are working with cached buffers. Completes software ISP TODO #6. Signed-off-by: Milan Zamazal --- Documentation/runtime_configuration.rst | 2 ++ src/libcamera/software_isp/TODO | 11 ----------- src/libcamera/software_isp/debayer_cpu.cpp | 5 ++++- 3 files changed, 6 insertions(+), 12 deletions(-) diff --git a/Documentation/runtime_configuration.rst b/Documentation/runtime_configuration.rst index d3972363..ed6370bb 100644 --- a/Documentation/runtime_configuration.rst +++ b/Documentation/runtime_configuration.rst @@ -48,6 +48,7 @@ file structure: config_file: # full path tuning_file: # full path simple: + copy_input_buffer: # true/false supported_devices: - driver: # driver name, e.g. `mxc-isi` software_isp: # true/false @@ -74,6 +75,7 @@ Configuration file example rpi: config_file: /usr/local/share/libcamera/pipeline/rpi/vc4/minimal_mem.yaml simple: + copy_input_buffer: false supported_devices: - driver: mxc-isi software_isp: true diff --git a/src/libcamera/software_isp/TODO b/src/libcamera/software_isp/TODO index 4fcee39b..15396322 100644 --- a/src/libcamera/software_isp/TODO +++ b/src/libcamera/software_isp/TODO @@ -103,17 +103,6 @@ per-frame buffers like we do for hardware ISPs. --- -6. Input buffer copying configuration - -> DebayerCpu::DebayerCpu(std::unique_ptr stats) -> : stats_(std::move(stats)), gammaCorrection_(1.0) -> { -> enableInputMemcpy_ = true; - -Set this appropriately and/or make it configurable. - ---- - 7. Performance measurement configuration > void DebayerCpu::process(FrameBuffer *input, FrameBuffer *output, DebayerParams params) diff --git a/src/libcamera/software_isp/debayer_cpu.cpp b/src/libcamera/software_isp/debayer_cpu.cpp index 88d6578b..fa538927 100644 --- a/src/libcamera/software_isp/debayer_cpu.cpp +++ b/src/libcamera/software_isp/debayer_cpu.cpp @@ -19,6 +19,7 @@ #include "libcamera/internal/bayer_format.h" #include "libcamera/internal/framebuffer.h" +#include "libcamera/internal/global_configuration.h" #include "libcamera/internal/mapped_framebuffer.h" namespace libcamera { @@ -45,7 +46,9 @@ DebayerCpu::DebayerCpu(std::unique_ptr stats) * always set it to true as the safer choice but this should be changed in * future. */ - enableInputMemcpy_ = true; + enableInputMemcpy_ = GlobalConfiguration::option( + "pipelines.simple.copy_input_buffer") + .value_or(true); /* Initialize gamma to 1.0 curve */ for (unsigned int i = 0; i < kGammaLookupSize; i++) From patchwork Tue Apr 23 10:30:28 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Milan Zamazal X-Patchwork-Id: 19928 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id CC92AC328D for ; Tue, 23 Apr 2024 10:31:44 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 216B463432; Tue, 23 Apr 2024 12:31:44 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.b="EG9GmfUm"; dkim-atps=neutral Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id AA3FF6340B for ; Tue, 23 Apr 2024 12:31:35 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1713868294; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=AuYGz76flo40VEPf686NvBJ9TEmU4ZMqBZOdIV/Z8UE=; b=EG9GmfUmHSddNXIC99b/Tk4EukJCUW+9EpUfrya7O+HeIoEg6GIJRHlZ4rozMqQoSvxLOy WJ4+7H8cAqsBq9P3olWZ1C1b3+fGjYYcm4d2oR4NWxhXDbV7Pmat12cPa53ImigXPyTthB NqcEYQ2IeTSnnWfAJvFW6PdDI3kPm+Q= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-503-JktzjzXJNfiIPpi8r8rS8A-1; Tue, 23 Apr 2024 06:31:32 -0400 X-MC-Unique: JktzjzXJNfiIPpi8r8rS8A-1 Received: from smtp.corp.redhat.com (int-mx07.intmail.prod.int.rdu2.redhat.com [10.11.54.7]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 2A7FE811000; Tue, 23 Apr 2024 10:31:32 +0000 (UTC) Received: from nuthatch.redhat.com (unknown [10.45.225.63]) by smtp.corp.redhat.com (Postfix) with ESMTP id 20B0E1C060D0; Tue, 23 Apr 2024 10:31:30 +0000 (UTC) From: Milan Zamazal To: libcamera-devel@lists.libcamera.org Cc: Milan Zamazal , Naushir Patuck Subject: [PATCH v2 13/13] libcamera: software_isp: Make measurement configurable Date: Tue, 23 Apr 2024 12:30:28 +0200 Message-ID: <20240423103034.364150-14-mzamazal@redhat.com> In-Reply-To: <20240423103034.364150-1-mzamazal@redhat.com> References: <20240423103034.364150-1-mzamazal@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.4.1 on 10.11.54.7 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Software ISP performs performance measurement on certain part of initial frames. Let's make this range configurable. For this purpose, this patch introduces new configuration options pipelines.simple.measure.skip and pipelines.simple.measure.number. Setting the latter one to 0 disables the measurement. Instead of the last frame, the class member and its configuration specify the number of frames to measure. This is easier to use for users and doesn't require to adjust two configuration parameters when the number of the initially skipped frames is changed. The patch also changes the names of the class members to make them more accurate. Completes software ISP TODO #7. Signed-off-by: Milan Zamazal --- Documentation/runtime_configuration.rst | 6 ++++++ src/libcamera/software_isp/TODO | 25 ---------------------- src/libcamera/software_isp/debayer_cpu.cpp | 25 ++++++++++++++-------- src/libcamera/software_isp/debayer_cpu.h | 7 +++--- 4 files changed, 25 insertions(+), 38 deletions(-) diff --git a/Documentation/runtime_configuration.rst b/Documentation/runtime_configuration.rst index ed6370bb..582e388f 100644 --- a/Documentation/runtime_configuration.rst +++ b/Documentation/runtime_configuration.rst @@ -49,6 +49,9 @@ file structure: tuning_file: # full path simple: copy_input_buffer: # true/false + measure: + skip: # non-negative integer, frames to skip initially + number: # non-negative integer, frames to measure supported_devices: - driver: # driver name, e.g. `mxc-isi` software_isp: # true/false @@ -76,6 +79,9 @@ Configuration file example config_file: /usr/local/share/libcamera/pipeline/rpi/vc4/minimal_mem.yaml simple: copy_input_buffer: false + measure: + skip: 50 + number: 30 supported_devices: - driver: mxc-isi software_isp: true diff --git a/src/libcamera/software_isp/TODO b/src/libcamera/software_isp/TODO index 15396322..ca305a63 100644 --- a/src/libcamera/software_isp/TODO +++ b/src/libcamera/software_isp/TODO @@ -103,31 +103,6 @@ per-frame buffers like we do for hardware ISPs. --- -7. Performance measurement configuration - -> void DebayerCpu::process(FrameBuffer *input, FrameBuffer *output, DebayerParams params) -> /* Measure before emitting signals */ -> if (measuredFrames_ < DebayerCpu::kLastFrameToMeasure && -> ++measuredFrames_ > DebayerCpu::kFramesToSkip) { -> timespec frameEndTime = {}; -> clock_gettime(CLOCK_MONOTONIC_RAW, &frameEndTime); -> frameProcessTime_ += timeDiff(frameEndTime, frameStartTime); -> if (measuredFrames_ == DebayerCpu::kLastFrameToMeasure) { -> const unsigned int measuredFrames = DebayerCpu::kLastFrameToMeasure - -> DebayerCpu::kFramesToSkip; -> LOG(Debayer, Info) -> << "Processed " << measuredFrames -> << " frames in " << frameProcessTime_ / 1000 << "us, " -> << frameProcessTime_ / (1000 * measuredFrames) -> << " us/frame"; -> } -> } - -I wonder if there would be a way to control at runtime when/how to -perform those measurements. Maybe that's a bit overkill. - ---- - 8. DebayerCpu cleanups > >> class DebayerCpu : public Debayer, public Object diff --git a/src/libcamera/software_isp/debayer_cpu.cpp b/src/libcamera/software_isp/debayer_cpu.cpp index fa538927..59694ad6 100644 --- a/src/libcamera/software_isp/debayer_cpu.cpp +++ b/src/libcamera/software_isp/debayer_cpu.cpp @@ -50,6 +50,13 @@ DebayerCpu::DebayerCpu(std::unique_ptr stats) "pipelines.simple.copy_input_buffer") .value_or(true); + skipBeforeMeasure_ = GlobalConfiguration::option( + "pipelines.simple.measure.skip") + .value_or(skipBeforeMeasure_); + framesToMeasure_ = GlobalConfiguration::option( + "pipelines.simple.measure.number") + .value_or(framesToMeasure_); + /* Initialize gamma to 1.0 curve */ for (unsigned int i = 0; i < kGammaLookupSize; i++) gamma_[i] = i / (kGammaLookupSize / kRGBLookupSize); @@ -496,7 +503,7 @@ int DebayerCpu::configure(const StreamConfiguration &inputCfg, return -ENOMEM; } - measuredFrames_ = 0; + encounteredFrames_ = 0; frameProcessTime_ = 0; return 0; @@ -696,7 +703,10 @@ void DebayerCpu::process(FrameBuffer *input, FrameBuffer *output, DebayerParams { timespec frameStartTime; - if (measuredFrames_ < DebayerCpu::kLastFrameToMeasure) { + bool measure = framesToMeasure_ > 0 && + encounteredFrames_ < skipBeforeMeasure_ + framesToMeasure_ && + ++encounteredFrames_ > skipBeforeMeasure_; + if (measure) { frameStartTime = {}; clock_gettime(CLOCK_MONOTONIC_RAW, &frameStartTime); } @@ -757,18 +767,15 @@ void DebayerCpu::process(FrameBuffer *input, FrameBuffer *output, DebayerParams metadata.planes()[0].bytesused = out.planes()[0].size(); /* Measure before emitting signals */ - if (measuredFrames_ < DebayerCpu::kLastFrameToMeasure && - ++measuredFrames_ > DebayerCpu::kFramesToSkip) { + if (measure) { timespec frameEndTime = {}; clock_gettime(CLOCK_MONOTONIC_RAW, &frameEndTime); frameProcessTime_ += timeDiff(frameEndTime, frameStartTime); - if (measuredFrames_ == DebayerCpu::kLastFrameToMeasure) { - const unsigned int measuredFrames = DebayerCpu::kLastFrameToMeasure - - DebayerCpu::kFramesToSkip; + if (encounteredFrames_ == skipBeforeMeasure_ + framesToMeasure_) { LOG(Debayer, Info) - << "Processed " << measuredFrames + << "Processed " << framesToMeasure_ << " frames in " << frameProcessTime_ / 1000 << "us, " - << frameProcessTime_ / (1000 * measuredFrames) + << frameProcessTime_ / (1000 * framesToMeasure_) << " us/frame"; } } diff --git a/src/libcamera/software_isp/debayer_cpu.h b/src/libcamera/software_isp/debayer_cpu.h index 689c1075..1306ef6f 100644 --- a/src/libcamera/software_isp/debayer_cpu.h +++ b/src/libcamera/software_isp/debayer_cpu.h @@ -148,11 +148,10 @@ private: bool swapRedBlueGains_; float gammaCorrection_; unsigned int blackLevel_; - unsigned int measuredFrames_; + unsigned int encounteredFrames_; int64_t frameProcessTime_; - /* Skip 30 frames for things to stabilize then measure 30 frames */ - static constexpr unsigned int kFramesToSkip = 30; - static constexpr unsigned int kLastFrameToMeasure = 60; + unsigned int skipBeforeMeasure_ = 30; + unsigned int framesToMeasure_ = 30; }; } /* namespace libcamera */