From patchwork Thu Jun 27 14:51:38 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Milan Zamazal X-Patchwork-Id: 20438 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id D089DBD87C for ; Thu, 27 Jun 2024 14:52:41 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 7BB3362E18; Thu, 27 Jun 2024 16:52:41 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.b="DJevaSCs"; dkim-atps=neutral Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 7381F619E8 for ; Thu, 27 Jun 2024 16:52:37 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1719499956; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=ogyn1wezofj3OjjexdEMa6ZWU6QI+eWN/lo+G03+6DI=; b=DJevaSCslAJ5v/4kCVvnJ6doPr5wQTfHH87ZaTYXRzlWMC8ppgG9V+h1Z/wdhmamoGed1S Rv9uFAw48AP87rlKKQCTrUm5GNuoWzCNYZS0ZZrHGe571TZOswKL8dWAykQg2nJEOBX2cE N//5HQnoCPL/9dOtgVdMbEoyLPVze6A= Received: from mx-prod-mc-02.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-332-IXhMRv92P0G5-PEwbN4weA-1; Thu, 27 Jun 2024 10:52:31 -0400 X-MC-Unique: IXhMRv92P0G5-PEwbN4weA-1 Received: from mx-prod-int-02.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-02.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.15]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-02.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id F016C1944DF4; Thu, 27 Jun 2024 14:52:30 +0000 (UTC) Received: from nuthatch.redhat.com (unknown [10.45.225.47]) by mx-prod-int-02.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id 345D21955BE0; Thu, 27 Jun 2024 14:52:28 +0000 (UTC) From: Milan Zamazal To: libcamera-devel@lists.libcamera.org Cc: Milan Zamazal , Kieran Bingham Subject: [PATCH v3 01/14] config: Introduce global runtime configuration Date: Thu, 27 Jun 2024 16:51:38 +0200 Message-ID: <20240627145156.1094127-2-mzamazal@redhat.com> In-Reply-To: <20240627145156.1094127-1-mzamazal@redhat.com> References: <20240627145156.1094127-1-mzamazal@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.0 on 10.30.177.15 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Currently, libcamera can be configured in runtime using several environment variables. With introducing more and more variables, this mechanism reaches its limits. It would be simpler and more flexible if it was possible to configure libcamera in a single file. For example, there have been a request for defining pipeline precedence in runtime. We want to compile in multiple pipelines, in order to have them accessible within single packages in distributions. And then being able to select among the pipelines manually as needed based on the particular hardware or operating system environment. Having the configuration file then allows easy switching between hardware, GPU or CPU IPAs. Another possible use case is tuning image output, especially with software ISP, to user liking. For example, some users may prefer higher contrast without the need to use the corresponding knobs, if present at all, in every application. The configuration file can also be used to enable or disable experimental features and avoid the need to track local patches changing configuration options hard-wired in the code when working on new features. This patch introduces basic support for configuration files. GlobalConfiguration class reads, stores and accesses the configuration. Its instance is stored as a private singleton accessed using a static method of the class. There is currently no reason to have more than one instance. libcamera configuration can be specified using a system-wide configuration file or a user configuration file. The user configuration file takes precedence if present. There is currently no way to merge multiple configuration files, the one found is used as the only configuration file. If no configuration file is present, nothing changes to the current libcamera behavior (except for some log messages related to configuration file lookup). The configuration file is a YAML file. We already have a mechanism for handling YAML configuration files in libcamera and the given infrastructure can be reused for the purpose. However, the configuration type is abstracted to make contingent future change of the underlying class easier while retaining (most of) the original API. The configuration is versioned. This has currently no particular meaning but is likely to have its purpose in future, especially once configuration validation is introduced. The configuration YAML file looks as follows: --- version: 1 configuration: WHATEVER CONFIGURATION NEEDED This patch introduces just the basic idea. Actually using the configuration in the corresponding places (everything what is currently configurable via environment variables should be configurable in the file configuration), refining the mechanism (e.g. configuration file validation to avoid mistakes caused by typos) and other improvements will be addressed in followup patches. Signed-off-by: Milan Zamazal --- .../libcamera/internal/global_configuration.h | 38 ++++ include/libcamera/internal/meson.build | 1 + src/libcamera/global_configuration.cpp | 163 ++++++++++++++++++ src/libcamera/meson.build | 1 + 4 files changed, 203 insertions(+) create mode 100644 include/libcamera/internal/global_configuration.h create mode 100644 src/libcamera/global_configuration.cpp diff --git a/include/libcamera/internal/global_configuration.h b/include/libcamera/internal/global_configuration.h new file mode 100644 index 00000000..2f38056a --- /dev/null +++ b/include/libcamera/internal/global_configuration.h @@ -0,0 +1,38 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2024 Red Hat, inc. + * + * global_configuration.h - Global configuration handling + */ + +#pragma once + +#include +#include + +#include "libcamera/internal/yaml_parser.h" + +namespace libcamera { + +class GlobalConfiguration +{ +public: + using Configuration = const YamlObject &; + + static unsigned int version(); + static Configuration configuration(); + +private: + static const std::vector globalConfigurationFiles; + + bool initialized_; + std::unique_ptr configuration_; + + GlobalConfiguration(); + bool loadFile(const std::filesystem::path &fileName); + void load(); + static const GlobalConfiguration &instance(); + static Configuration get(); +}; + +} /* namespace libcamera */ diff --git a/include/libcamera/internal/meson.build b/include/libcamera/internal/meson.build index 9713ea1c..daab7150 100644 --- a/include/libcamera/internal/meson.build +++ b/include/libcamera/internal/meson.build @@ -28,6 +28,7 @@ libcamera_internal_headers = files([ 'dma_buf_allocator.h', 'formats.h', 'framebuffer.h', + 'global_configuration.h', 'ipa_manager.h', 'ipa_module.h', 'ipa_proxy.h', diff --git a/src/libcamera/global_configuration.cpp b/src/libcamera/global_configuration.cpp new file mode 100644 index 00000000..96088b36 --- /dev/null +++ b/src/libcamera/global_configuration.cpp @@ -0,0 +1,163 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2024 Red Hat, inc. + * + * global_configuration.cpp - Global configuration handling + */ + +#include "libcamera/internal/global_configuration.h" + +#include +#include +#include +#include + +#include +#include +#include + +#include "libcamera/internal/yaml_parser.h" + +namespace libcamera { + +LOG_DEFINE_CATEGORY(Configuration) + +/** + * \class GlobalConfiguration + * \brief Support for global libcamera configuration + * + * The configuration file is a YAML file and the configuration itself is stored + * under `configuration' top-level item. + * + * The configuration file is looked up in user's home directory first and if it + * is not found then in system-wide configuration directories. If multiple + * configuration files exist then only the first one found is used and no + * configuration merging is performed. + * + * The class is used as a private singleton and the configuration can be + * accessed usingGlobalConfiguration::configuration(). + */ + +/** + * \typedef GlobalConfiguration::Configuration + * \brief Type representing global libcamera configuration + * + * All code outside GlobalConfiguration must use this type declaration and not + * the underlying type. + */ + +GlobalConfiguration::GlobalConfiguration() + : initialized_(false), configuration_(std::make_unique()) +{ +} + +const std::vector + GlobalConfiguration::globalConfigurationFiles = { + std::filesystem::path(LIBCAMERA_SYSCONF_DIR) / "configuration.yaml", + std::filesystem::path("/etc/libcamera/configuration.yaml"), + }; + +void GlobalConfiguration::load() +{ + std::filesystem::path userConfigurationDirectory; + char *xdgConfigHome = utils::secure_getenv("XDG_CONFIG_HOME"); + if (xdgConfigHome) { + userConfigurationDirectory = xdgConfigHome; + } else { + const char *home = utils::secure_getenv("HOME"); + if (home) + userConfigurationDirectory = + std::filesystem::path(home) / ".config"; + } + + if (!userConfigurationDirectory.empty()) { + std::filesystem::path user_configuration_file = + userConfigurationDirectory / "libcamera" / "configuration.yaml"; + if (loadFile(user_configuration_file)) + return; + } + + for (auto path : globalConfigurationFiles) + if (loadFile(path)) + return; + + initialized_ = true; + LOG(Configuration, Debug) << "No configuration file found"; +} + +bool GlobalConfiguration::loadFile(const std::filesystem::path &fileName) +{ + File file(fileName); + if (!file.exists()) { + return false; + } + + if (!file.open(File::OpenModeFlag::ReadOnly)) { + LOG(Configuration, Warning) + << "Failed to open configuration file '" << fileName << "'"; + return true; + } + + auto root = YamlParser::parse(file); + if (!root) { + LOG(Configuration, Warning) << "Failed to parse configuration file " + << fileName; + return true; + } + configuration_ = std::move(root); + initialized_ = true; + LOG(Configuration, Info) << "Configuration file " << fileName << "loaded"; + + return true; +} + +const GlobalConfiguration &GlobalConfiguration::instance() +{ + static GlobalConfiguration configuration; + if (!configuration.initialized_) { + configuration.load(); + } + return configuration; +} + +GlobalConfiguration::Configuration GlobalConfiguration::get() +{ + return (*instance().configuration_); +} + +/** + * \brief Return configuration version + * + * The version is (optionally) declared in the configuration file in the + * top-level section `version', alongside `configuration'. This has currently no + * real use but may be needed in future if configuration incompatibilities + * occur. + * + * \return Configuration version as declared in the configuration file or 0 if + * no version is declared there + */ +unsigned int GlobalConfiguration::version() +{ + return get()["version"].get().value_or(0); +} + +/** + * \brief Return libcamera global configuration + * + * This returns the whole configuration stored in the top-level section + * `configuration' of the YAML configuration file. + * + * The requested part of the configuration can be accessed using \a YamlObject + * methods. + * + * \note \a YamlObject type itself shouldn't be used in type declarations to + * avoid trouble if we decide to change the underlying data objects in future. + * + * \return The whole configuration section + */ +GlobalConfiguration::Configuration GlobalConfiguration::configuration() +{ + return get()["configuration"]; +} + +} /* namespace libcamera */ diff --git a/src/libcamera/meson.build b/src/libcamera/meson.build index 89504cee..32396ff2 100644 --- a/src/libcamera/meson.build +++ b/src/libcamera/meson.build @@ -21,6 +21,7 @@ libcamera_sources = files([ 'framebuffer.cpp', 'framebuffer_allocator.cpp', 'geometry.cpp', + 'global_configuration.cpp', 'ipa_controls.cpp', 'ipa_data_serializer.cpp', 'ipa_interface.cpp', From patchwork Thu Jun 27 14:51:39 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Milan Zamazal X-Patchwork-Id: 20437 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id ABE81BD87C for ; Thu, 27 Jun 2024 14:52:38 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 75D2E62C9D; Thu, 27 Jun 2024 16:52:38 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.b="XGbLdGBh"; dkim-atps=neutral Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 440B5619E8 for ; Thu, 27 Jun 2024 16:52:36 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1719499955; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=BZG0gaLQt60ZzZpv5RpQ8pBioEWeEkUn+Iay+YBkB6A=; b=XGbLdGBhgawrBQ/yrtZIzFAYy1R95yA9yNRakdKyB40hoXtv4VweXxlADVQjIwWWSxGGzn IFT5Xpexmc7V+ng/qk9iwkTQoR5e5TrEkwaQNmJuUDxfrmCQ88CkctSnUcKmrNFtWKpx9k CmEu7EQmxDI3csAWemEzgLWh7gmprsk= Received: from mx-prod-mc-04.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-637-X7Il_5HbPKSfTMGKOwDSww-1; Thu, 27 Jun 2024 10:52:34 -0400 X-MC-Unique: X7Il_5HbPKSfTMGKOwDSww-1 Received: from mx-prod-int-02.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-02.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.15]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-04.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id E5DBB19560B1; Thu, 27 Jun 2024 14:52:32 +0000 (UTC) Received: from nuthatch.redhat.com (unknown [10.45.225.47]) by mx-prod-int-02.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id 7820C1955BE0; Thu, 27 Jun 2024 14:52:31 +0000 (UTC) From: Milan Zamazal To: libcamera-devel@lists.libcamera.org Cc: Milan Zamazal , Kieran Bingham Subject: [PATCH v3 02/14] config: Move global configuration to base Date: Thu, 27 Jun 2024 16:51:39 +0200 Message-ID: <20240627145156.1094127-3-mzamazal@redhat.com> In-Reply-To: <20240627145156.1094127-1-mzamazal@redhat.com> References: <20240627145156.1094127-1-mzamazal@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.0 on 10.30.177.15 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Global configuration may be used by some components from base, most notably logging. This means the configuration must also be present in base. And YAML object and parsing too. Let's move that all there. Signed-off-by: Milan Zamazal --- src/libcamera/{ => base}/global_configuration.cpp | 0 src/libcamera/base/meson.build | 15 +++++++++++++++ src/libcamera/{ => base}/yaml_parser.cpp | 0 src/libcamera/meson.build | 15 --------------- 4 files changed, 15 insertions(+), 15 deletions(-) rename src/libcamera/{ => base}/global_configuration.cpp (100%) rename src/libcamera/{ => base}/yaml_parser.cpp (100%) diff --git a/src/libcamera/global_configuration.cpp b/src/libcamera/base/global_configuration.cpp similarity index 100% rename from src/libcamera/global_configuration.cpp rename to src/libcamera/base/global_configuration.cpp diff --git a/src/libcamera/base/meson.build b/src/libcamera/base/meson.build index 7a7fd7e4..b71d3e23 100644 --- a/src/libcamera/base/meson.build +++ b/src/libcamera/base/meson.build @@ -9,6 +9,7 @@ libcamera_base_sources = files([ 'event_notifier.cpp', 'file.cpp', 'flags.cpp', + 'global_configuration.cpp', 'log.cpp', 'message.cpp', 'mutex.cpp', @@ -20,10 +21,12 @@ libcamera_base_sources = files([ 'timer.cpp', 'unique_fd.cpp', 'utils.cpp', + 'yaml_parser.cpp', ]) libdw = dependency('libdw', required : false) libunwind = dependency('libunwind', required : false) +libyaml = dependency('yaml-0.1', required : false) if cc.has_header_symbol('execinfo.h', 'backtrace') config_h.set('HAVE_BACKTRACE', 1) @@ -37,11 +40,23 @@ if libunwind.found() config_h.set('HAVE_UNWIND', 1) endif +# Fallback to a subproject if libyaml isn't found, as it's not packaged in AOSP. +if not libyaml.found() + cmake = import('cmake') + + libyaml_vars = cmake.subproject_options() + libyaml_vars.add_cmake_defines({'CMAKE_POSITION_INDEPENDENT_CODE': 'ON'}) + libyaml_vars.append_compile_args('c', '-Wno-unused-value') + libyaml_wrap = cmake.subproject('libyaml', options : libyaml_vars) + libyaml = libyaml_wrap.dependency('yaml') +endif + libcamera_base_deps = [ libatomic, libdw, libthreads, libunwind, + libyaml, ] # Internal components must use the libcamera_base_private dependency to enable diff --git a/src/libcamera/yaml_parser.cpp b/src/libcamera/base/yaml_parser.cpp similarity index 100% rename from src/libcamera/yaml_parser.cpp rename to src/libcamera/base/yaml_parser.cpp diff --git a/src/libcamera/meson.build b/src/libcamera/meson.build index 32396ff2..1bee980c 100644 --- a/src/libcamera/meson.build +++ b/src/libcamera/meson.build @@ -21,7 +21,6 @@ libcamera_sources = files([ 'framebuffer.cpp', 'framebuffer_allocator.cpp', 'geometry.cpp', - 'global_configuration.cpp', 'ipa_controls.cpp', 'ipa_data_serializer.cpp', 'ipa_interface.cpp', @@ -49,7 +48,6 @@ libcamera_sources = files([ 'v4l2_pixelformat.cpp', 'v4l2_subdevice.cpp', 'v4l2_videodevice.cpp', - 'yaml_parser.cpp', ]) libcamera_sources += libcamera_public_headers @@ -81,7 +79,6 @@ if not cc.has_function('dlopen') libdl = cc.find_library('dl') endif libudev = dependency('libudev', required : get_option('udev')) -libyaml = dependency('yaml-0.1', required : false) # Use one of gnutls or libcrypto (provided by OpenSSL), trying gnutls first. libcrypto = dependency('gnutls', required : false) @@ -117,17 +114,6 @@ if libudev.found() ]) endif -# Fallback to a subproject if libyaml isn't found, as it's not packaged in AOSP. -if not libyaml.found() - cmake = import('cmake') - - libyaml_vars = cmake.subproject_options() - libyaml_vars.add_cmake_defines({'CMAKE_POSITION_INDEPENDENT_CODE': 'ON'}) - libyaml_vars.append_compile_args('c', '-Wno-unused-value') - libyaml_wrap = cmake.subproject('libyaml', options : libyaml_vars) - libyaml = libyaml_wrap.dependency('yaml') -endif - control_sources = [] controls_mode_files = { @@ -183,7 +169,6 @@ libcamera_deps += [ libdl, liblttng, libudev, - libyaml, ] # We add '/' to the build_rpath as a 'safe' path to act as a boolean flag. From patchwork Thu Jun 27 14:51:40 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Milan Zamazal X-Patchwork-Id: 20439 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id A9352BD87C for ; Thu, 27 Jun 2024 14:52:43 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 5A1C162CA0; Thu, 27 Jun 2024 16:52:43 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.b="X7YoRciN"; dkim-atps=neutral Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 2B4F562C99 for ; Thu, 27 Jun 2024 16:52:40 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1719499959; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=DDm2Qq3P9BjbCkivRku1/Kyuq1Xfr7KsFtzI6ZJsgyY=; b=X7YoRciNc8Vqq1hHyLxZsN6/FQXNpyzpXvVMzhUqfP80qGfQDXdqGmavP+8LVRe4ky3vLO PD9vm5tyrXQ/zExYa5g1wcqyelk4RqKtCt5gty3hTcu6PujtEcOrPPAejhCr8Nl5QmTWer /NUBuff1Kpos768wk4Vv/5dfLgc6DMU= Received: from mx-prod-mc-01.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-517-vjVH3vN7M4G4wGO2jKT4TQ-1; Thu, 27 Jun 2024 10:52:36 -0400 X-MC-Unique: vjVH3vN7M4G4wGO2jKT4TQ-1 Received: from mx-prod-int-02.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-02.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.15]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-01.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 30D5C1944CC9; Thu, 27 Jun 2024 14:52:35 +0000 (UTC) Received: from nuthatch.redhat.com (unknown [10.45.225.47]) by mx-prod-int-02.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id 7DFA51955BD4; Thu, 27 Jun 2024 14:52:33 +0000 (UTC) From: Milan Zamazal To: libcamera-devel@lists.libcamera.org Cc: Milan Zamazal , Kieran Bingham Subject: [PATCH v3 03/14] config: Look up logging levels in the configuration file Date: Thu, 27 Jun 2024 16:51:40 +0200 Message-ID: <20240627145156.1094127-4-mzamazal@redhat.com> In-Reply-To: <20240627145156.1094127-1-mzamazal@redhat.com> References: <20240627145156.1094127-1-mzamazal@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.0 on 10.30.177.15 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" In addition to LIBCAMERA_LOG_LEVELS environment variable, expose the same configuration in the configuration file. Because logging asks for configuration and configuration loading uses logging, extra care is needed. We must prevent calling any log function before configuration is loaded. Otherwise the log function would ask for configuration, it would load the configuration files and would try to log the results, which would cause mutex deadlock due to calling logging inside logging. Not logging in configuration is not an option, at least without introducing complicated mechanism such as delayed logging, because we need to log at least errors when reading the configuration. We must also prevent logging in configuration loading before the configuration is loaded and available (or there is an error in loading the configuration). Otherwise logging would see an empty configuration and wouldn't be configured as asked. Preventing logging before configuration is loaded is ensured by asking for configuration in CameraManager constructor and in IPAManager constructor (which is mysteriously loaded from CameraManager constructor before its body is executed). This seems to be sufficient at least for software ISP. The configuration snippet for logging looks as follows: configuration: log: levels: LIBCAMERA_LOG_LEVELS Signed-off-by: Milan Zamazal --- src/libcamera/base/global_configuration.cpp | 18 +++++++++++++++++- src/libcamera/base/log.cpp | 12 ++++++++++-- src/libcamera/camera_manager.cpp | 9 +++++++++ src/libcamera/ipa_manager.cpp | 12 ++++++++++++ 4 files changed, 48 insertions(+), 3 deletions(-) diff --git a/src/libcamera/base/global_configuration.cpp b/src/libcamera/base/global_configuration.cpp index 96088b36..b45fd341 100644 --- a/src/libcamera/base/global_configuration.cpp +++ b/src/libcamera/base/global_configuration.cpp @@ -29,6 +29,13 @@ LOG_DEFINE_CATEGORY(Configuration) * The configuration file is a YAML file and the configuration itself is stored * under `configuration' top-level item. * + * Example configuration file content: + * \code{.yaml} + * configuration: + * log: + * levels: 'IPAManager:DEBUG' + * \endcode + * * The configuration file is looked up in user's home directory first and if it * is not found then in system-wide configuration directories. If multiple * configuration files exist then only the first one found is used and no @@ -57,6 +64,11 @@ const std::vector std::filesystem::path("/etc/libcamera/configuration.yaml"), }; +/* + * Care is needed here to not log anything before the configuration is + * loaded otherwise the logger would be initialized with empty configuration. + */ + void GlobalConfiguration::load() { std::filesystem::path userConfigurationDirectory; @@ -87,10 +99,15 @@ void GlobalConfiguration::load() bool GlobalConfiguration::loadFile(const std::filesystem::path &fileName) { + /* + * initialized_ must be set to true before any logging is called! + */ + File file(fileName); if (!file.exists()) { return false; } + initialized_ = true; if (!file.open(File::OpenModeFlag::ReadOnly)) { LOG(Configuration, Warning) @@ -105,7 +122,6 @@ bool GlobalConfiguration::loadFile(const std::filesystem::path &fileName) return true; } configuration_ = std::move(root); - initialized_ = true; LOG(Configuration, Info) << "Configuration file " << fileName << "loaded"; return true; diff --git a/src/libcamera/base/log.cpp b/src/libcamera/base/log.cpp index 3a656b8f..0f676710 100644 --- a/src/libcamera/base/log.cpp +++ b/src/libcamera/base/log.cpp @@ -14,6 +14,7 @@ #include #include #include +#include #include #include #include @@ -25,6 +26,8 @@ #include #include +#include "libcamera/internal/global_configuration.h" + /** * \file base/log.h * \brief Logging infrastructure @@ -626,8 +629,13 @@ void Logger::parseLogFile() void Logger::parseLogLevels() { const char *debug = utils::secure_getenv("LIBCAMERA_LOG_LEVELS"); - if (!debug) - return; + if (!debug) { + const std::optional confDebug = + GlobalConfiguration::configuration()["log"]["levels"].get(); + if (!confDebug.has_value()) + return; + debug = confDebug.value().c_str(); + } for (const char *pair = debug; *debug != '\0'; pair = debug) { const char *comma = strchrnul(debug, ','); diff --git a/src/libcamera/camera_manager.cpp b/src/libcamera/camera_manager.cpp index 95a9e326..7c5e5606 100644 --- a/src/libcamera/camera_manager.cpp +++ b/src/libcamera/camera_manager.cpp @@ -15,6 +15,7 @@ #include "libcamera/internal/camera.h" #include "libcamera/internal/device_enumerator.h" +#include "libcamera/internal/global_configuration.h" #include "libcamera/internal/pipeline_handler.h" /** @@ -37,6 +38,14 @@ LOG_DEFINE_CATEGORY(Camera) CameraManager::Private::Private() : initialized_(false) { + /* + * This is needed to initialize global configuration + * before any logging is called. Actually, IPAManager + * constructor is called before this, so the configuration + * call is superfluous here, but the initialization belongs + * here. + */ + GlobalConfiguration::configuration(); } int CameraManager::Private::start() diff --git a/src/libcamera/ipa_manager.cpp b/src/libcamera/ipa_manager.cpp index f4e0b633..a678d0c5 100644 --- a/src/libcamera/ipa_manager.cpp +++ b/src/libcamera/ipa_manager.cpp @@ -16,6 +16,7 @@ #include #include +#include "libcamera/internal/global_configuration.h" #include "libcamera/internal/ipa_module.h" #include "libcamera/internal/ipa_proxy.h" #include "libcamera/internal/pipeline_handler.h" @@ -105,6 +106,17 @@ IPAManager *IPAManager::self_ = nullptr; */ IPAManager::IPAManager() { + /* + * Make sure configuration is loaded early. + * Otherwise there would be problems with mutual + * configuration-logging dependencies. + * + * This is needed to be here in addition to CameraManager constructor + * because logging is called before CameraManager constructor body is + * executed. + */ + GlobalConfiguration::configuration(); + if (self_) LOG(IPAManager, Fatal) << "Multiple IPAManager objects are not allowed"; From patchwork Thu Jun 27 14:51:41 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Milan Zamazal X-Patchwork-Id: 20440 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id D6C56C3295 for ; Thu, 27 Jun 2024 14:52:44 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 535B162E1F; Thu, 27 Jun 2024 16:52:44 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.b="KBCnV7pF"; dkim-atps=neutral Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id EC16562C98 for ; Thu, 27 Jun 2024 16:52:40 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1719499959; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=9/heDoBNts0Q7imp9oVcOpbLJI9Ky3xSbSP6pK3cVWc=; b=KBCnV7pFj+jOypF8y0o4EQZPIjceSAwmM08BMmIL5Q8rIBMUl3uk1ZdNA0Vn00BAuj8t+x b2BoX1KpeEKRhonXP87qZ8vSLwbXDMbP4vf2m5hvxbzlySj79QtRsK1xsVKiUcRzGKXyTW jUvpVKpdyKmasEHM5vWN9npCZH+JBys= Received: from mx-prod-mc-04.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-454-aWMyk1MYNhWxMBKnvNU0sg-1; Thu, 27 Jun 2024 10:52:38 -0400 X-MC-Unique: aWMyk1MYNhWxMBKnvNU0sg-1 Received: from mx-prod-int-02.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-02.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.15]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-04.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 5452719560B2; Thu, 27 Jun 2024 14:52:37 +0000 (UTC) Received: from nuthatch.redhat.com (unknown [10.45.225.47]) by mx-prod-int-02.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id 99F771955BE0; Thu, 27 Jun 2024 14:52:35 +0000 (UTC) From: Milan Zamazal To: libcamera-devel@lists.libcamera.org Cc: Milan Zamazal , Kieran Bingham Subject: [PATCH v3 04/14] rconfig: Add configuration retrieval helpers Date: Thu, 27 Jun 2024 16:51:41 +0200 Message-ID: <20240627145156.1094127-5-mzamazal@redhat.com> In-Reply-To: <20240627145156.1094127-1-mzamazal@redhat.com> References: <20240627145156.1094127-1-mzamazal@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.0 on 10.30.177.15 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" They make accessing simple configuration values simpler. Due to the restrictions of YamlObject class, ugly type casting dance is needed. Also, GlobalConfiguration::option must check for the key presence before retrieving the value using operator[] because if the key is not present then YamlObject::empty may be returned as a regular value. Signed-off-by: Milan Zamazal --- .../libcamera/internal/global_configuration.h | 4 ++ src/libcamera/base/global_configuration.cpp | 53 ++++++++++++++++++- src/libcamera/base/log.cpp | 13 ++--- 3 files changed, 60 insertions(+), 10 deletions(-) diff --git a/include/libcamera/internal/global_configuration.h b/include/libcamera/internal/global_configuration.h index 2f38056a..7c39c969 100644 --- a/include/libcamera/internal/global_configuration.h +++ b/include/libcamera/internal/global_configuration.h @@ -8,6 +8,7 @@ #pragma once #include +#include #include #include "libcamera/internal/yaml_parser.h" @@ -21,6 +22,9 @@ public: static unsigned int version(); static Configuration configuration(); + static std::optional option(const std::string &confPath); + static std::optional envOption(const char *const envVariable, + const std::string &confPath); private: static const std::vector globalConfigurationFiles; diff --git a/src/libcamera/base/global_configuration.cpp b/src/libcamera/base/global_configuration.cpp index b45fd341..b202cef9 100644 --- a/src/libcamera/base/global_configuration.cpp +++ b/src/libcamera/base/global_configuration.cpp @@ -10,6 +10,7 @@ #include #include #include +#include #include #include @@ -41,8 +42,10 @@ LOG_DEFINE_CATEGORY(Configuration) * configuration files exist then only the first one found is used and no * configuration merging is performed. * - * The class is used as a private singleton and the configuration can be - * accessed usingGlobalConfiguration::configuration(). + * The class is used as a private singleton accessed by the provided + * helpers. Namely GlobalConfiguration::option or GlobalConfiguration::envOption + * to access individual options or GlobalConfiguration::configuration() to + * access the whole configuration. */ /** @@ -141,6 +144,52 @@ GlobalConfiguration::Configuration GlobalConfiguration::get() return (*instance().configuration_); } +/** + * \brief Return value of the configuration option identified by \a confPath + * \param[in] confPath Sequence of the YAML section names (excluding + * `configuration') leading to the requested option separated by dots + * \return A value if an item corresponding to \a confPath exists in the + * configuration file, no value otherwise + */ +std::optional GlobalConfiguration::option( + const std::string &confPath) +{ + YamlObject *c = &const_cast(configuration()); + for (auto part : utils::details::StringSplitter(confPath, ".")) + if (c->contains(part)) + c = &const_cast((*c)[part]); + else + return std::optional(); + return c->get(); +} + +/** + * \brief Return value of the configuration option from a file or environment + * \param[in] envVariable Environment variable to get the value from + * \param[in] confPath The same as in GlobalConfiguration::option + * + * This helper looks first at the given environment variable and if it is + * defined then it returns its value (even if it is empty). Otherwise it looks + * for \a confPath the same way as in GlobalConfiguration::option. Only string + * values are supported. + * + * \note Support for using environment variables to configure libcamera behavior + * is provided here mostly for backward compatibility reasons. Introducing new + * configuration environment variables is discouraged. + * + * \return A value retrieved from the given environment option or configuration + * file or no value if not found + */ +std::optional GlobalConfiguration::envOption( + const char *const envVariable, + const std::string &confPath) +{ + const char *envValue = utils::secure_getenv(envVariable); + if (envValue) + return std::optional{ std::string{ envValue } }; + return option(confPath); +} + /** * \brief Return configuration version * diff --git a/src/libcamera/base/log.cpp b/src/libcamera/base/log.cpp index 0f676710..891b4ae9 100644 --- a/src/libcamera/base/log.cpp +++ b/src/libcamera/base/log.cpp @@ -628,14 +628,11 @@ void Logger::parseLogFile() */ void Logger::parseLogLevels() { - const char *debug = utils::secure_getenv("LIBCAMERA_LOG_LEVELS"); - if (!debug) { - const std::optional confDebug = - GlobalConfiguration::configuration()["log"]["levels"].get(); - if (!confDebug.has_value()) - return; - debug = confDebug.value().c_str(); - } + const std::optional confDebug = + GlobalConfiguration::envOption("LIBCAMERA_LOG_LEVELS", "log.levels"); + if (!confDebug.has_value()) + return; + const char *debug = confDebug.value().c_str(); for (const char *pair = debug; *debug != '\0'; pair = debug) { const char *comma = strchrnul(debug, ','); From patchwork Thu Jun 27 14:51:42 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Milan Zamazal X-Patchwork-Id: 20441 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 379D7BD87C for ; Thu, 27 Jun 2024 14:52:48 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id D2A4E62E1F; Thu, 27 Jun 2024 16:52:47 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.b="ceF7fLqJ"; dkim-atps=neutral Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id B895162E21 for ; Thu, 27 Jun 2024 16:52:44 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1719499963; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=kyWATA/LSuBSpxzRySXe/jUT/sLTe3hlJ9TY5xXsoNY=; b=ceF7fLqJr94OFh3G54lYkEZz21w1TdF0zzXVTSqXZXSF7kk/zAIs1tgh4nk9vdtCa3XAcw K7xZD8kTPEzsPqHXi2IuYPmdLxcnhwHKjw5GqJYxziUvgNUTdd4aKCsBDloEYWz4p4KhxK 1yC63qnYdnpLhWFBtcupBNXJ61QRTio= Received: from mx-prod-mc-02.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-663-RoT9Pl_NNM-IqDrmA_amzw-1; Thu, 27 Jun 2024 10:52:40 -0400 X-MC-Unique: RoT9Pl_NNM-IqDrmA_amzw-1 Received: from mx-prod-int-02.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-02.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.15]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-02.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 7541D1944DFE; Thu, 27 Jun 2024 14:52:39 +0000 (UTC) Received: from nuthatch.redhat.com (unknown [10.45.225.47]) by mx-prod-int-02.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id D7AE319773E4; Thu, 27 Jun 2024 14:52:37 +0000 (UTC) From: Milan Zamazal To: libcamera-devel@lists.libcamera.org Cc: Milan Zamazal , Kieran Bingham Subject: [PATCH v3 05/14] config: Look up log file in configuration file Date: Thu, 27 Jun 2024 16:51:42 +0200 Message-ID: <20240627145156.1094127-6-mzamazal@redhat.com> In-Reply-To: <20240627145156.1094127-1-mzamazal@redhat.com> References: <20240627145156.1094127-1-mzamazal@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.0 on 10.30.177.15 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" This requires change in LIBCAMERA_LOG_FILE interpretation so that we can disable the log file in isolated processes. If LIBCAMERA_LOG_FILE is defined but empty, it means there is no log file, even if defined in the configuration file. This should cause no compatibility trouble since opening a file with an empty name makes little sense. The configuration snippet: configuration: log: file: FILENAME Signed-off-by: Milan Zamazal --- src/libcamera/base/global_configuration.cpp | 1 + src/libcamera/base/log.cpp | 13 +++++++++---- src/libcamera/process.cpp | 11 ++++++++--- 3 files changed, 18 insertions(+), 7 deletions(-) diff --git a/src/libcamera/base/global_configuration.cpp b/src/libcamera/base/global_configuration.cpp index b202cef9..b64bf28e 100644 --- a/src/libcamera/base/global_configuration.cpp +++ b/src/libcamera/base/global_configuration.cpp @@ -34,6 +34,7 @@ LOG_DEFINE_CATEGORY(Configuration) * \code{.yaml} * configuration: * log: + * file: syslog * levels: 'IPAManager:DEBUG' * \endcode * diff --git a/src/libcamera/base/log.cpp b/src/libcamera/base/log.cpp index 891b4ae9..07352f6c 100644 --- a/src/libcamera/base/log.cpp +++ b/src/libcamera/base/log.cpp @@ -607,16 +607,21 @@ Logger::Logger() */ void Logger::parseLogFile() { - const char *file = utils::secure_getenv("LIBCAMERA_LOG_FILE"); - if (!file) + std::optional file = + GlobalConfiguration::envOption("LIBCAMERA_LOG_FILE", "log.file"); + if (!file.has_value()) return; - if (!strcmp(file, "syslog")) { + auto fileValue = file.value(); + if (fileValue == "") + return; + + if (fileValue == "syslog") { logSetTarget(LoggingTargetSyslog); return; } - logSetFile(file, false); + logSetFile(fileValue.c_str(), false); } /** diff --git a/src/libcamera/process.cpp b/src/libcamera/process.cpp index 86d27b2d..42f54e85 100644 --- a/src/libcamera/process.cpp +++ b/src/libcamera/process.cpp @@ -12,7 +12,9 @@ #include #include #include +#include #include +#include #include #include #include @@ -24,6 +26,8 @@ #include #include +#include "libcamera/internal/global_configuration.h" + /** * \file process.h * \brief Process object @@ -263,9 +267,10 @@ int Process::start(const std::string &path, closeAllFdsExcept(fds); - const char *file = utils::secure_getenv("LIBCAMERA_LOG_FILE"); - if (file && strcmp(file, "syslog")) - unsetenv("LIBCAMERA_LOG_FILE"); + std::optional file = + GlobalConfiguration::envOption("LIBCAMERA_LOG_FILE", "log.file"); + if (file.has_value() && file.value() != "syslog") + setenv("LIBCAMERA_LOG_FILE", "", 1); const char **argv = new const char *[args.size() + 2]; unsigned int len = args.size(); From patchwork Thu Jun 27 14:51:43 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Milan Zamazal X-Patchwork-Id: 20442 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 21358BD87C for ; Thu, 27 Jun 2024 14:52:50 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id A244C62E25; Thu, 27 Jun 2024 16:52:49 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.b="dG2t93/+"; dkim-atps=neutral Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 3449662C9D for ; Thu, 27 Jun 2024 16:52:47 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1719499965; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=w5RW8Do9fs2CBR+4offTqLnYppKeO8Hn9gRIrnEQQKo=; b=dG2t93/++iVUEnMvWwMwofxKGRt+qBMY11iqW6fgzoPhN6w/eQHgLuSHbRCX24TY2qIp0i RResyvfWvtqOuPcfqRW5cGybS665jOOW2vRtE8o//d1jMn/qAY2KBTkQqMSR4Khp9gNE7K sP2ozrfP9Fz+mzSjhq6Aee+Cla8eYp4= Received: from mx-prod-mc-04.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-244-4tN5TxgPNgW0DLT1l8MBxA-1; Thu, 27 Jun 2024 10:52:43 -0400 X-MC-Unique: 4tN5TxgPNgW0DLT1l8MBxA-1 Received: from mx-prod-int-02.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-02.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.15]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-04.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id A91A71954233; Thu, 27 Jun 2024 14:52:41 +0000 (UTC) Received: from nuthatch.redhat.com (unknown [10.45.225.47]) by mx-prod-int-02.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id 07F4719773D9; Thu, 27 Jun 2024 14:52:39 +0000 (UTC) From: Milan Zamazal To: libcamera-devel@lists.libcamera.org Cc: Milan Zamazal , Kieran Bingham Subject: [PATCH v3 06/14] config: Look up log color configuration in configuration file Date: Thu, 27 Jun 2024 16:51:43 +0200 Message-ID: <20240627145156.1094127-7-mzamazal@redhat.com> In-Reply-To: <20240627145156.1094127-1-mzamazal@redhat.com> References: <20240627145156.1094127-1-mzamazal@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.0 on 10.30.177.15 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" The configuration snippet: configuration: log: color: BOOL In order to use the global configuration access helper for a boolean value, we must make a template from it. To make the template visible, we must implement it in the header file, otherwise undefined reference error would be reported when linking. We don't implement envOption helper for boolean here, because we would have to deal with a specific value interpretation in this particular case. Signed-off-by: Milan Zamazal --- .../libcamera/internal/global_configuration.h | 27 ++++++++++++++++++- src/libcamera/base/global_configuration.cpp | 20 ++++---------- src/libcamera/base/log.cpp | 2 ++ 3 files changed, 33 insertions(+), 16 deletions(-) diff --git a/include/libcamera/internal/global_configuration.h b/include/libcamera/internal/global_configuration.h index 7c39c969..42c60417 100644 --- a/include/libcamera/internal/global_configuration.h +++ b/include/libcamera/internal/global_configuration.h @@ -11,6 +11,8 @@ #include #include +#include + #include "libcamera/internal/yaml_parser.h" namespace libcamera { @@ -22,7 +24,30 @@ public: static unsigned int version(); static Configuration configuration(); - static std::optional option(const std::string &confPath); + + template || + std::is_same_v || + std::is_same_v || + std::is_same_v || + std::is_same_v || + std::is_same_v || + std::is_same_v || + std::is_same_v || + std::is_same_v || + std::is_same_v> * = nullptr> + static std::optional option(const std::string &confPath) + { + YamlObject *c = &const_cast(configuration()); + for (auto part : utils::details::StringSplitter(confPath, ".")) + if (c->contains(part)) + c = &const_cast((*c)[part]); + else + return std::optional(); + return c->get(); + } + static std::optional envOption(const char *const envVariable, const std::string &confPath); diff --git a/src/libcamera/base/global_configuration.cpp b/src/libcamera/base/global_configuration.cpp index b64bf28e..f18ca28b 100644 --- a/src/libcamera/base/global_configuration.cpp +++ b/src/libcamera/base/global_configuration.cpp @@ -15,7 +15,6 @@ #include #include -#include #include "libcamera/internal/yaml_parser.h" @@ -146,23 +145,14 @@ GlobalConfiguration::Configuration GlobalConfiguration::get() } /** + * \fn static std::optional GlobalConfiguration::option(const char *const confPath) * \brief Return value of the configuration option identified by \a confPath + * \tparam T The type of the retrieved configuration value * \param[in] confPath Sequence of the YAML section names (excluding * `configuration') leading to the requested option separated by dots - * \return A value if an item corresponding to \a confPath exists in the - * configuration file, no value otherwise + * \return A value of type \a T if an item corresponding to \a confPath exists + * in the configuration file and matches type \a T, no value otherwise */ -std::optional GlobalConfiguration::option( - const std::string &confPath) -{ - YamlObject *c = &const_cast(configuration()); - for (auto part : utils::details::StringSplitter(confPath, ".")) - if (c->contains(part)) - c = &const_cast((*c)[part]); - else - return std::optional(); - return c->get(); -} /** * \brief Return value of the configuration option from a file or environment @@ -188,7 +178,7 @@ std::optional GlobalConfiguration::envOption( const char *envValue = utils::secure_getenv(envVariable); if (envValue) return std::optional{ std::string{ envValue } }; - return option(confPath); + return option(confPath); } /** diff --git a/src/libcamera/base/log.cpp b/src/libcamera/base/log.cpp index 07352f6c..d522e895 100644 --- a/src/libcamera/base/log.cpp +++ b/src/libcamera/base/log.cpp @@ -590,6 +590,8 @@ void Logger::logSetLevel(const char *category, const char *level) Logger::Logger() { bool color = !utils::secure_getenv("LIBCAMERA_LOG_NO_COLOR"); + if (color) + color = GlobalConfiguration::option("log.color").value_or(true); logSetStream(&std::cerr, color); parseLogFile(); From patchwork Thu Jun 27 14:51:44 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Milan Zamazal X-Patchwork-Id: 20443 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 2793EBD87C for ; Thu, 27 Jun 2024 14:52:54 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id C585662E1F; Thu, 27 Jun 2024 16:52:53 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.b="eWyVWZDd"; dkim-atps=neutral Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id E8FFB62C9D for ; Thu, 27 Jun 2024 16:52:48 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1719499968; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=hOSremVm4GpJfv4AZwy+x7gXKxOk44f3HaYZKKfhVdg=; b=eWyVWZDdX97+Z/CQK8Ik35rLTSLIibRDxm1a8NeWIg76iTY+omWKdtFRlzJZoz069k5Fu/ 24HwC6AKJPW9XSKg/VWj1Zdtw/GPpKmnI4Ku2vv+GpkMQvUEJmF902mJIlpnhY41dp2HiW 7hAoyMoSJBZbSLrDjqiHBNe2RckZ2fI= Received: from mx-prod-mc-04.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-85-ZcCJQIMKNjq9KKHD2hIQdA-1; Thu, 27 Jun 2024 10:52:44 -0400 X-MC-Unique: ZcCJQIMKNjq9KKHD2hIQdA-1 Received: from mx-prod-int-02.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-02.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.15]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-04.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id AC7EB1955D6B; Thu, 27 Jun 2024 14:52:43 +0000 (UTC) Received: from nuthatch.redhat.com (unknown [10.45.225.47]) by mx-prod-int-02.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id EAC171955BD4; Thu, 27 Jun 2024 14:52:41 +0000 (UTC) From: Milan Zamazal To: libcamera-devel@lists.libcamera.org Cc: Milan Zamazal , Kieran Bingham Subject: [PATCH v3 07/14] config: Look up rpi paths in configuration file Date: Thu, 27 Jun 2024 16:51:44 +0200 Message-ID: <20240627145156.1094127-8-mzamazal@redhat.com> In-Reply-To: <20240627145156.1094127-1-mzamazal@redhat.com> References: <20240627145156.1094127-1-mzamazal@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.0 on 10.30.177.15 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" It may be arguable whether pipeline specific configurations belong to the global configuration file. But: - Having a single configuration file is generally easier for the user. - The original configuration via environment variables can be already considered global. - These options point to other configuration files and it makes little sense to add another configuration file to the chain. The configuration snippet: configuration: pipeline: rpi: config_file: FILENAME tuning_file: FILENAME Signed-off-by: Milan Zamazal --- .../pipeline/rpi/common/pipeline_base.cpp | 18 ++++++++++++++---- 1 file changed, 14 insertions(+), 4 deletions(-) diff --git a/src/libcamera/pipeline/rpi/common/pipeline_base.cpp b/src/libcamera/pipeline/rpi/common/pipeline_base.cpp index 3041fd1e..6c9a3972 100644 --- a/src/libcamera/pipeline/rpi/common/pipeline_base.cpp +++ b/src/libcamera/pipeline/rpi/common/pipeline_base.cpp @@ -20,6 +20,7 @@ #include #include "libcamera/internal/camera_lens.h" +#include "libcamera/internal/global_configuration.h" #include "libcamera/internal/ipa_manager.h" #include "libcamera/internal/v4l2_subdevice.h" @@ -1088,7 +1089,12 @@ int CameraData::loadPipelineConfiguration() /* Initial configuration of the platform, in case no config file is present */ platformPipelineConfigure({}); - char const *configFromEnv = utils::secure_getenv("LIBCAMERA_RPI_CONFIG_FILE"); + std::optional configFile = + GlobalConfiguration::envOption("LIBCAMERA_RPI_CONFIG_FILE", + "pipelines.rpi.config_file"); + if (!configFile.has_value()) + return 0; + char const *configFromEnv = configFile.value().c_str(); if (!configFromEnv || *configFromEnv == '\0') return 0; @@ -1147,14 +1153,18 @@ int CameraData::loadIPA(ipa::RPi::InitResult *result) * the environment variable overrides it. */ std::string configurationFile; - char const *configFromEnv = utils::secure_getenv("LIBCAMERA_RPI_TUNING_FILE"); - if (!configFromEnv || *configFromEnv == '\0') { + const std::string confPath = + std::string("pipelines.rpi.cameras.") + sensor_->id() + std::string(".tuning_file"); + std::optional configFromEnv = + GlobalConfiguration::envOption("LIBCAMERA_RPI_TUNING_FILE", + confPath.c_str()); + if (!configFromEnv.has_value() || configFromEnv.value().empty()) { std::string model = sensor_->model(); if (isMonoSensor(sensor_)) model += "_mono"; configurationFile = ipa_->configurationFile(model + ".json"); } else { - configurationFile = std::string(configFromEnv); + configurationFile = configFromEnv.value(); } IPASettings settings(configurationFile, sensor_->model()); From patchwork Thu Jun 27 14:51:45 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Milan Zamazal X-Patchwork-Id: 20444 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 57018BD87C for ; Thu, 27 Jun 2024 14:52:55 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id ED11362E27; Thu, 27 Jun 2024 16:52:54 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.b="A0kikr7z"; dkim-atps=neutral Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 7AAB462CA0 for ; Thu, 27 Jun 2024 16:52:49 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1719499968; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=K1O+u7HqUqjfQgy01pt53GygIh0soo/pa3BL9DiScPQ=; b=A0kikr7zyL3ieyZLYU9UGAt/yjkvQrPe32/3VrkkfTLeDnfcu1r6XKmTeAQ2Erk6JwFRvC IU7nAdxdYkb3cM7RBDhYqE4jptlEUGeHCbq/8dU/OM+/pEr7vQnpjtkHoQeUkfOz1sdC4P ckrjNSfW82EEkMK+vworWOf6pNpa+Pg= Received: from mx-prod-mc-02.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-251-BDTEtlpuNFytqjHmMNoJnA-1; Thu, 27 Jun 2024 10:52:47 -0400 X-MC-Unique: BDTEtlpuNFytqjHmMNoJnA-1 Received: from mx-prod-int-02.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-02.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.15]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-02.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 9AD621944DD3; Thu, 27 Jun 2024 14:52:45 +0000 (UTC) Received: from nuthatch.redhat.com (unknown [10.45.225.47]) by mx-prod-int-02.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id 1F70C1955BE0; Thu, 27 Jun 2024 14:52:43 +0000 (UTC) From: Milan Zamazal To: libcamera-devel@lists.libcamera.org Cc: Milan Zamazal , Kieran Bingham Subject: [PATCH v3 08/14] config: Look up IPA configurables in configuration file Date: Thu, 27 Jun 2024 16:51:45 +0200 Message-ID: <20240627145156.1094127-9-mzamazal@redhat.com> In-Reply-To: <20240627145156.1094127-1-mzamazal@redhat.com> References: <20240627145156.1094127-1-mzamazal@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.0 on 10.30.177.15 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" The configuration snippet: configuration: ipa: config_paths: CONFIG:PATHS:... module_paths: MODULE:PATHS:... force_isolation: BOOL Signed-off-by: Milan Zamazal --- src/libcamera/ipa_manager.cpp | 11 ++++++++--- src/libcamera/ipa_proxy.cpp | 8 ++++++-- 2 files changed, 14 insertions(+), 5 deletions(-) diff --git a/src/libcamera/ipa_manager.cpp b/src/libcamera/ipa_manager.cpp index a678d0c5..612d66d7 100644 --- a/src/libcamera/ipa_manager.cpp +++ b/src/libcamera/ipa_manager.cpp @@ -129,8 +129,11 @@ IPAManager::IPAManager() unsigned int ipaCount = 0; /* User-specified paths take precedence. */ - const char *modulePaths = utils::secure_getenv("LIBCAMERA_IPA_MODULE_PATH"); - if (modulePaths) { + const auto confModulePaths = + GlobalConfiguration::envOption( + "LIBCAMERA_IPA_MODULE_PATH", "ipa.module_paths"); + if (confModulePaths.has_value()) { + const char *modulePaths = confModulePaths.value().c_str(); for (const auto &dir : utils::split(modulePaths, ":")) { if (dir.empty()) continue; @@ -308,7 +311,9 @@ bool IPAManager::isSignatureValid([[maybe_unused]] IPAModule *ipa) const { #if HAVE_IPA_PUBKEY char *force = utils::secure_getenv("LIBCAMERA_IPA_FORCE_ISOLATION"); - if (force && force[0] != '\0') { + if ((force && force[0] != '\0') || + (!force && GlobalConfiguration::option("ipa.force_isolation") + .value_or(false))) { LOG(IPAManager, Debug) << "Isolation of IPA module " << ipa->path() << " forced through environment variable"; diff --git a/src/libcamera/ipa_proxy.cpp b/src/libcamera/ipa_proxy.cpp index 6c17c456..16d91580 100644 --- a/src/libcamera/ipa_proxy.cpp +++ b/src/libcamera/ipa_proxy.cpp @@ -15,6 +15,7 @@ #include #include +#include "libcamera/internal/global_configuration.h" #include "libcamera/internal/ipa_module.h" /** @@ -104,8 +105,11 @@ std::string IPAProxy::configurationFile(const std::string &name) const std::string ipaName = ipam_->info().name; /* Check the environment variable first. */ - const char *confPaths = utils::secure_getenv("LIBCAMERA_IPA_CONFIG_PATH"); - if (confPaths) { + auto confConfPaths = + GlobalConfiguration::envOption( + "LIBCAMERA_IPA_CONFIG_PATH", "ipa.config_paths"); + if (confConfPaths.has_value()) { + const char *confPaths = confConfPaths.value().c_str(); for (const auto &dir : utils::split(confPaths, ":")) { if (dir.empty()) continue; From patchwork Thu Jun 27 14:51:46 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Milan Zamazal X-Patchwork-Id: 20445 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 368ECBD87C for ; Thu, 27 Jun 2024 14:52:57 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id D594963334; Thu, 27 Jun 2024 16:52:56 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.b="LxG5uYCc"; dkim-atps=neutral Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 4E3ED62E18 for ; Thu, 27 Jun 2024 16:52:51 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1719499970; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=6DEQiEX5fOPklDgBQb3aOaVo6i+RgJCENFDxlNYV3Ik=; b=LxG5uYCcXsUFY9mKXb3PsIzjhPdlViBEQdZzVdGMdsjbv1vT1hKIdjIMKlY0USUfna/LVR dHGfMQx0y/KdliIE9mrVocEcwNAVLdeaoXJCOdtlKWeDZl2PqBl5bRz6D2ExVlWmLH4LUc jWt6CjNetdGdQ1xhbXyNDehDsG9MW0Q= Received: from mx-prod-mc-04.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-387-s4iVzdmHMkODJPN4nGR5BQ-1; Thu, 27 Jun 2024 10:52:48 -0400 X-MC-Unique: s4iVzdmHMkODJPN4nGR5BQ-1 Received: from mx-prod-int-02.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-02.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.15]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-04.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id ACBF71955F36; Thu, 27 Jun 2024 14:52:47 +0000 (UTC) Received: from nuthatch.redhat.com (unknown [10.45.225.47]) by mx-prod-int-02.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id 11F7C1955BD4; Thu, 27 Jun 2024 14:52:45 +0000 (UTC) From: Milan Zamazal To: libcamera-devel@lists.libcamera.org Cc: Milan Zamazal , Kieran Bingham Subject: [PATCH v3 09/14] config: Look up RkISP1 tuning file in configuration file Date: Thu, 27 Jun 2024 16:51:46 +0200 Message-ID: <20240627145156.1094127-10-mzamazal@redhat.com> In-Reply-To: <20240627145156.1094127-1-mzamazal@redhat.com> References: <20240627145156.1094127-1-mzamazal@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.0 on 10.30.177.15 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" The configuration snippet: configuration: pipeline: rkisp1: tuning_file: FILE This environment variable has not been documented. Signed-off-by: Milan Zamazal --- src/libcamera/pipeline/rkisp1/rkisp1.cpp | 11 ++++++++--- 1 file changed, 8 insertions(+), 3 deletions(-) diff --git a/src/libcamera/pipeline/rkisp1/rkisp1.cpp b/src/libcamera/pipeline/rkisp1/rkisp1.cpp index 4cbf105d..c244ac6e 100644 --- a/src/libcamera/pipeline/rkisp1/rkisp1.cpp +++ b/src/libcamera/pipeline/rkisp1/rkisp1.cpp @@ -36,6 +36,7 @@ #include "libcamera/internal/delayed_controls.h" #include "libcamera/internal/device_enumerator.h" #include "libcamera/internal/framebuffer.h" +#include "libcamera/internal/global_configuration.h" #include "libcamera/internal/ipa_manager.h" #include "libcamera/internal/media_device.h" #include "libcamera/internal/pipeline_handler.h" @@ -349,8 +350,12 @@ int RkISP1CameraData::loadIPA(unsigned int hwRevision) * environment variable overrides it. */ std::string ipaTuningFile; - char const *configFromEnv = utils::secure_getenv("LIBCAMERA_RKISP1_TUNING_FILE"); - if (!configFromEnv || *configFromEnv == '\0') { + const std::string confPath = + std::string("pipelines.rkisp1.cameras.") + sensor_->id() + std::string(".tuning_file"); + const auto confTuningFile = + GlobalConfiguration::envOption( + "LIBCAMERA_RKISP1_TUNING_FILE", confPath.c_str()); + if (!confTuningFile.has_value() || confTuningFile.value() == "") { ipaTuningFile = ipa_->configurationFile(sensor_->model() + ".yaml"); /* * If the tuning file isn't found, fall back to the @@ -359,7 +364,7 @@ int RkISP1CameraData::loadIPA(unsigned int hwRevision) if (ipaTuningFile.empty()) ipaTuningFile = ipa_->configurationFile("uncalibrated.yaml"); } else { - ipaTuningFile = std::string(configFromEnv); + ipaTuningFile = confTuningFile.value(); } IPACameraSensorInfo sensorInfo{}; From patchwork Thu Jun 27 14:51:47 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Milan Zamazal X-Patchwork-Id: 20446 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id E7435BD87C for ; Thu, 27 Jun 2024 14:52:58 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 9CFF762E25; Thu, 27 Jun 2024 16:52:58 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.b="Q4EbjneH"; dkim-atps=neutral Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 8D56462C9D for ; Thu, 27 Jun 2024 16:52:54 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1719499973; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=xZmaibSI/9GVkZJQbkCmtUFceJFGssWw7Ct7aYJoDmo=; b=Q4EbjneHely/nsRZSs6+RapaNl9vuHKTfyBL1DGqdNgCxomfQ/nSeApfjwWM4Fuop3aI7M OgWFCOlvt+6TO5Ce0IMID7QZeV32DIwYeaxnBC6YV4L9iOcXG5d61oareG1QeBHBWPAYnE XmBKtWZhQ4Iy1NoAz2Mk/75py7zdFkw= Received: from mx-prod-mc-05.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-495-0UdWqDqsPI2qnT87Umao9g-1; Thu, 27 Jun 2024 10:52:50 -0400 X-MC-Unique: 0UdWqDqsPI2qnT87Umao9g-1 Received: from mx-prod-int-02.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-02.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.15]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-05.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 7EE731955D8E; Thu, 27 Jun 2024 14:52:49 +0000 (UTC) Received: from nuthatch.redhat.com (unknown [10.45.225.47]) by mx-prod-int-02.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id 1DE101955BD4; Thu, 27 Jun 2024 14:52:47 +0000 (UTC) From: Milan Zamazal To: libcamera-devel@lists.libcamera.org Cc: Milan Zamazal , Kieran Bingham Subject: [PATCH v3 10/14] config: Look up pipelines match list in configuration file Date: Thu, 27 Jun 2024 16:51:47 +0200 Message-ID: <20240627145156.1094127-11-mzamazal@redhat.com> In-Reply-To: <20240627145156.1094127-1-mzamazal@redhat.com> References: <20240627145156.1094127-1-mzamazal@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.0 on 10.30.177.15 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Signed-off-by: Milan Zamazal --- src/libcamera/camera_manager.cpp | 9 +++++---- 1 file changed, 5 insertions(+), 4 deletions(-) diff --git a/src/libcamera/camera_manager.cpp b/src/libcamera/camera_manager.cpp index 7c5e5606..836a7bc9 100644 --- a/src/libcamera/camera_manager.cpp +++ b/src/libcamera/camera_manager.cpp @@ -113,14 +113,15 @@ void CameraManager::Private::createPipelineHandlers() * file and only fallback on environment variable or all handlers, if * there is no configuration file. */ - const char *pipesList = - utils::secure_getenv("LIBCAMERA_PIPELINES_MATCH_LIST"); - if (pipesList) { + std::optional pipesList = + GlobalConfiguration::envOption("LIBCAMERA_PIPELINES_MATCH_LIST", + "pipelines_match_list"); + if (pipesList.has_value()) { /* * When a list of preferred pipelines is defined, iterate * through the ordered list to match the enumerated devices. */ - for (const auto &pipeName : utils::split(pipesList, ",")) { + for (const auto &pipeName : utils::split(pipesList.value(), ",")) { const PipelineHandlerFactoryBase *factory; factory = PipelineHandlerFactoryBase::getFactoryByName(pipeName); if (!factory) From patchwork Thu Jun 27 14:51:48 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Milan Zamazal X-Patchwork-Id: 20447 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id DA593BD87C for ; Thu, 27 Jun 2024 14:52:59 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 8342D62E27; Thu, 27 Jun 2024 16:52:59 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.b="OEY+0QVN"; dkim-atps=neutral Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id E7C6C62E22 for ; Thu, 27 Jun 2024 16:52:54 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1719499974; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=A10O+4dSdhePKlfsHWOGeAYLfihJVpDyChBVlSs30mo=; b=OEY+0QVNqaen6udWfVkFeFdabmuEurFLVBrQLF0rbXQO9o0ZPAhfBaDbAt3gYgJcYxSLmk b5+fCjJfwK32V65SGakTSZxCTbrSNaWdYDMG8WMVQyMnIruUpnVZLkuH4YP8o2lzZHl152 wReAr4cUJ7/DuvmQtzjJJltBZ+BXS1I= Received: from mx-prod-mc-01.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-614-CtGebut6PiywRXQ-h-lbWA-1; Thu, 27 Jun 2024 10:52:52 -0400 X-MC-Unique: CtGebut6PiywRXQ-h-lbWA-1 Received: from mx-prod-int-02.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-02.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.15]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-01.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 70FB51944CF2; Thu, 27 Jun 2024 14:52:51 +0000 (UTC) Received: from nuthatch.redhat.com (unknown [10.45.225.47]) by mx-prod-int-02.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id 1082319773E1; Thu, 27 Jun 2024 14:52:49 +0000 (UTC) From: Milan Zamazal To: libcamera-devel@lists.libcamera.org Cc: Milan Zamazal , Kieran Bingham Subject: [PATCH v3 11/14] config: Allow enabling software ISP in runtime Date: Thu, 27 Jun 2024 16:51:48 +0200 Message-ID: <20240627145156.1094127-12-mzamazal@redhat.com> In-Reply-To: <20240627145156.1094127-1-mzamazal@redhat.com> References: <20240627145156.1094127-1-mzamazal@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.0 on 10.30.177.15 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" This patch allows enabling or disabling software ISP in runtime in addition to compile time. This can be useful for software ISP testing on various platforms as well as for overriding the defaults in case the defaults don't work well (e.g. hardware ISP may or may not work on i.MX8MP depending on the kernel and libcamera patches present in the given system). The configuration is specified as follows: configuration: pipelines: simple: supported_devices: - driver: DRIVER-NAME software_isp: BOOLEAN - ... For example: configuration: pipelines: simple: supported_devices: - driver: mxc-isi software_isp: true The overall configuration of enabling or disabling software ISP may get dropped in future but this patch is still useful in the meantime. Signed-off-by: Milan Zamazal --- src/libcamera/pipeline/simple/simple.cpp | 12 ++++++++++++ 1 file changed, 12 insertions(+) diff --git a/src/libcamera/pipeline/simple/simple.cpp b/src/libcamera/pipeline/simple/simple.cpp index eb36578e..bb208a9e 100644 --- a/src/libcamera/pipeline/simple/simple.cpp +++ b/src/libcamera/pipeline/simple/simple.cpp @@ -32,6 +32,7 @@ #include "libcamera/internal/camera_sensor.h" #include "libcamera/internal/converter.h" #include "libcamera/internal/device_enumerator.h" +#include "libcamera/internal/global_configuration.h" #include "libcamera/internal/media_device.h" #include "libcamera/internal/pipeline_handler.h" #include "libcamera/internal/software_isp/software_isp.h" @@ -1544,6 +1545,17 @@ bool SimplePipelineHandler::match(DeviceEnumerator *enumerator) } swIspEnabled_ = info->swIspEnabled; + for (GlobalConfiguration::Configuration entry : + GlobalConfiguration::configuration() + ["pipelines"]["simple"]["supported_devices"] + .asList()) { + auto name = entry["driver"].get(); + if (name.has_value() and !name.value().compare(info->driver)) { + swIspEnabled_ = entry["software_isp"].get().value_or(swIspEnabled_); + LOG(SimplePipeline, Debug) << "Overriding software ISP to " << swIspEnabled_; + break; + } + } /* Locate the sensors. */ std::vector sensors = locateSensors(); From patchwork Thu Jun 27 14:51:49 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Milan Zamazal X-Patchwork-Id: 20448 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 33EC1BD87C for ; Thu, 27 Jun 2024 14:53:02 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id DC5C762E23; Thu, 27 Jun 2024 16:53:01 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.b="Rtdmms5x"; dkim-atps=neutral Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id A271D6333B for ; Thu, 27 Jun 2024 16:52:58 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1719499977; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=G62kRVlQgqifU0DDNCifs45oB6rVJOg74N798n/x2t0=; b=Rtdmms5xVyxZUqZZPk+kQYvFdMpF30QpU45bO1V/0NY2dqQTuqKxz2GTrtWbx+7eDZNkL3 3DR5s6GWgoVzncSDlPwxdbSC//+ZTYI+snBImBGunJDVacn8shwEqwsRT3a7D0ZZOYRNlh lrYJnDwDaBnvwUy/NZB2RqSozIV9qpg= Received: from mx-prod-mc-05.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-626-4_uW83tiNLOyLRjm8pA7EA-1; Thu, 27 Jun 2024 10:52:54 -0400 X-MC-Unique: 4_uW83tiNLOyLRjm8pA7EA-1 Received: from mx-prod-int-02.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-02.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.15]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-05.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 981F71944D22; Thu, 27 Jun 2024 14:52:53 +0000 (UTC) Received: from nuthatch.redhat.com (unknown [10.45.225.47]) by mx-prod-int-02.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id 031A81955BD4; Thu, 27 Jun 2024 14:52:51 +0000 (UTC) From: Milan Zamazal To: libcamera-devel@lists.libcamera.org Cc: Milan Zamazal , Kieran Bingham Subject: [PATCH v3 12/14] config: Add global configuration file documentation Date: Thu, 27 Jun 2024 16:51:49 +0200 Message-ID: <20240627145156.1094127-13-mzamazal@redhat.com> In-Reply-To: <20240627145156.1094127-1-mzamazal@redhat.com> References: <20240627145156.1094127-1-mzamazal@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.0 on 10.30.177.15 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Extend (and rename) the documentation of environment variables with information about the configuration file. Signed-off-by: Milan Zamazal --- Documentation/index.rst | 2 +- Documentation/meson.build | 2 +- ...ariables.rst => runtime_configuration.rst} | 108 ++++++++++++++++-- 3 files changed, 98 insertions(+), 14 deletions(-) rename Documentation/{environment_variables.rst => runtime_configuration.rst} (63%) diff --git a/Documentation/index.rst b/Documentation/index.rst index 5442ae75..aa277b85 100644 --- a/Documentation/index.rst +++ b/Documentation/index.rst @@ -19,7 +19,7 @@ Pipeline Handler Writer's Guide IPA Writer's guide Tracing guide - Environment variables + Runtime configuration Sensor driver requirements Lens driver requirements Python Bindings diff --git a/Documentation/meson.build b/Documentation/meson.build index 30d39523..c07b56d8 100644 --- a/Documentation/meson.build +++ b/Documentation/meson.build @@ -71,7 +71,6 @@ if sphinx.found() 'conf.py', 'contributing.rst', 'docs.rst', - 'environment_variables.rst', 'guides/application-developer.rst', 'guides/introduction.rst', 'guides/ipa.rst', @@ -80,6 +79,7 @@ if sphinx.found() 'index.rst', 'lens_driver_requirements.rst', 'python-bindings.rst', + 'runtime_configuration.rst', 'sensor_driver_requirements.rst', 'software-isp-benchmarking.rst', '../README.rst', diff --git a/Documentation/environment_variables.rst b/Documentation/runtime_configuration.rst similarity index 63% rename from Documentation/environment_variables.rst rename to Documentation/runtime_configuration.rst index 4e9fbb27..70a8bd57 100644 --- a/Documentation/environment_variables.rst +++ b/Documentation/runtime_configuration.rst @@ -1,43 +1,127 @@ .. SPDX-License-Identifier: CC-BY-SA-4.0 -Environment variables +Runtime configuration ===================== -The libcamera behaviour can be tuned through environment variables. This -document lists all the available variables and describes their usage. +The libcamera behaviour can be tuned through a configuration file or +environment variables. This document lists all the configuration options +and describes their usage. + +General rules +------------- + +The configuration file is looked up in the following locations, in this +order: + +- $XDG_CONFIG_HOME/libcamera/configuration.yaml +- LIBCAMERA_SYSCONF_DIR/configuration.yaml +- /etc/libcamera/configuration.yaml + +The first configuration file found wins, contingent configuration files +in other locations are ignored. + +Settings in environment variables take precedence over settings in +configuration files. This allows overriding behaviour temporarily +without the need to modify configuration files. + +Configuration options +--------------------- + +Here is an overview of the available configuration options, in the YAML +file structure: + +:: + + configuration: + ipa: + config_paths: # full paths to directories, separated by colons + force_isolation: # true/false + module_paths: # full paths to directories, separated by colons + log: + color: # true/false for color/no-color + file: # either `syslog` or a full path + levels: # see Log levels + pipelines_match_list: # pipeline names, separated by commas + pipelines: + rkisp1: + cameras: + - CAMERA-ID: + tuning-file: # full path + rpi: + config_file: # full path + cameras: + - CAMERA-ID: + tuning-file: # full path + simple: + supported_devices: + - driver: # driver name, e.g. `mxc-isi` + software_isp: # true/false + +Configuration file example +-------------------------- + +:: + + --- + version: 1 + configuration: + ipa: + config_paths: /home/user/.libcamera/share/ipa:/opt/libcamera/vendor/share/ipa + module_paths: /home/user/.libcamera/lib:/opt/libcamera/vendor/lib + force_isolation: true + log: + color: false + file: syslog + levels: 'IPAManager:DEBUG' + pipelines_match_list: rkisp1,simple + pipelines: + rkisp1: + cameras: + - "/base/axi/pcie@120000/rp1/i2c@88000/imx283@1a": + tuning_file: /home/pi/imx283.json + - "/base/axi/pcie@120000/rp1/i2c@80000/imx335@1a": + tuning_file: /home/pi/imx335.json + rpi: + config_file: /usr/local/share/libcamera/pipeline/rpi/vc4/minimal_mem.yaml + simple: + supported_devices: + - driver: mxc-isi + software_isp: true List of variables ----------------- -LIBCAMERA_LOG_FILE +The corresponding configuration file paths are listed in parentheses. + +LIBCAMERA_LOG_FILE (log.file) The custom destination for log output. Example value: ``/home/{user}/camera_log.log`` -LIBCAMERA_LOG_LEVELS +LIBCAMERA_LOG_LEVELS (log.levels) Configure the verbosity of log messages for different categories (`more `__). Example value: ``*:DEBUG`` -LIBCAMERA_LOG_NO_COLOR +LIBCAMERA_LOG_NO_COLOR (log.color) Disable coloring of log messages (`more `__). -LIBCAMERA_IPA_CONFIG_PATH +LIBCAMERA_IPA_CONFIG_PATH (ipa.config_paths) Define custom search locations for IPA configurations (`more `__). Example value: ``${HOME}/.libcamera/share/ipa:/opt/libcamera/vendor/share/ipa`` -LIBCAMERA_IPA_FORCE_ISOLATION +LIBCAMERA_IPA_FORCE_ISOLATION (ipa.force_isolation) When set to a non-empty string, force process isolation of all IPA modules. Example value: ``1`` -LIBCAMERA_IPA_MODULE_PATH +LIBCAMERA_IPA_MODULE_PATH (ipa.module_paths) Define custom search locations for IPA modules (`more `__). Example value: ``${HOME}/.libcamera/lib:/opt/libcamera/vendor/lib`` -LIBCAMERA_PIPELINES_MATCH_LIST +LIBCAMERA_PIPELINES_MATCH_LIST (pipelines_match_list) Define an ordered list of pipeline names to be used to match the media devices in the system. The pipeline handler names used to populate the variable are the ones passed to the REGISTER_PIPELINE_HANDLER() macro in the @@ -45,7 +129,7 @@ LIBCAMERA_PIPELINES_MATCH_LIST Example value: ``rkisp1,simple`` -LIBCAMERA_RPI_CONFIG_FILE +LIBCAMERA_RPI_CONFIG_FILE (pipelines.rpi.config_file) Define a custom configuration file to use in the Raspberry Pi pipeline handler. Example value: ``/usr/local/share/libcamera/pipeline/rpi/vc4/minimal_mem.yaml`` @@ -144,7 +228,7 @@ code. IPA configuration ~~~~~~~~~~~~~~~~~ -IPA modules use configuration files to store parameters. The format and +IPA modules use their own configuration files to store parameters. The format and contents of the configuration files is specific to the IPA module. They usually contain tuning parameters for the algorithms, in JSON format. From patchwork Thu Jun 27 14:51:50 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Milan Zamazal X-Patchwork-Id: 20450 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 04DDBBD87C for ; Thu, 27 Jun 2024 14:53:07 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id A93A663339; Thu, 27 Jun 2024 16:53:06 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.b="N/SdRYsN"; dkim-atps=neutral Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 080AD6333F for ; Thu, 27 Jun 2024 16:53:02 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1719499982; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=WpZFU8RLUOfV/hhxdF55OazzMqWQ8bo5HLg0WRkB+NU=; b=N/SdRYsNAIgV5ljpWk949LjOO1ndC6o8LE26/Tmnk3s9SQhx2190bTd/r13FY5ALdEeY9r 50HnbU4dbFcw5LqhBYe7Gw21QGOt4TVigtdGlvyfAdl+yUNfTPf4N/mdTMfUfssuTROS8o KtFJPppwrKvuid6RovNLXz5aDDxukUE= Received: from mx-prod-mc-05.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-467-IE7sLosaMV-_ya8O4o4BfQ-1; Thu, 27 Jun 2024 10:52:56 -0400 X-MC-Unique: IE7sLosaMV-_ya8O4o4BfQ-1 Received: from mx-prod-int-02.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-02.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.15]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-05.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id BAD36195608A; Thu, 27 Jun 2024 14:52:55 +0000 (UTC) Received: from nuthatch.redhat.com (unknown [10.45.225.47]) by mx-prod-int-02.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id 27E3B1955BD4; Thu, 27 Jun 2024 14:52:53 +0000 (UTC) From: Milan Zamazal To: libcamera-devel@lists.libcamera.org Cc: Milan Zamazal , Kieran Bingham Subject: [PATCH v3 13/14] libcamera: software_isp: Make input buffer copying configurable Date: Thu, 27 Jun 2024 16:51:50 +0200 Message-ID: <20240627145156.1094127-14-mzamazal@redhat.com> In-Reply-To: <20240627145156.1094127-1-mzamazal@redhat.com> References: <20240627145156.1094127-1-mzamazal@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.0 on 10.30.177.15 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" On some platforms, working directly on the input buffer is very slow due to disabled caching. This is why we copy the input buffer into standard (cached) memory. This is an unnecessary overhead on platforms with cached buffers. Let's make input buffer copying configurable. The default is still copying, as it's overhead is much lower than contingent operations on non-cached memory. Ideally, we should improve this in future to set the default to non-copying if we can be sure under observable circumstances that we are working with cached buffers. Completes software ISP TODO #6. Signed-off-by: Milan Zamazal --- Documentation/runtime_configuration.rst | 2 ++ src/libcamera/software_isp/TODO | 11 ----------- src/libcamera/software_isp/debayer_cpu.cpp | 5 ++++- 3 files changed, 6 insertions(+), 12 deletions(-) diff --git a/Documentation/runtime_configuration.rst b/Documentation/runtime_configuration.rst index 70a8bd57..ebe17a55 100644 --- a/Documentation/runtime_configuration.rst +++ b/Documentation/runtime_configuration.rst @@ -53,6 +53,7 @@ file structure: - CAMERA-ID: tuning-file: # full path simple: + copy_input_buffer: # true/false supported_devices: - driver: # driver name, e.g. `mxc-isi` software_isp: # true/false @@ -84,6 +85,7 @@ Configuration file example rpi: config_file: /usr/local/share/libcamera/pipeline/rpi/vc4/minimal_mem.yaml simple: + copy_input_buffer: false supported_devices: - driver: mxc-isi software_isp: true diff --git a/src/libcamera/software_isp/TODO b/src/libcamera/software_isp/TODO index 6bdc5905..f91551dc 100644 --- a/src/libcamera/software_isp/TODO +++ b/src/libcamera/software_isp/TODO @@ -90,17 +90,6 @@ per-frame buffers like we do for hardware ISPs. --- -6. Input buffer copying configuration - -> DebayerCpu::DebayerCpu(std::unique_ptr stats) -> : stats_(std::move(stats)), gammaCorrection_(1.0) -> { -> enableInputMemcpy_ = true; - -Set this appropriately and/or make it configurable. - ---- - 7. Performance measurement configuration > void DebayerCpu::process(FrameBuffer *input, FrameBuffer *output, DebayerParams params) diff --git a/src/libcamera/software_isp/debayer_cpu.cpp b/src/libcamera/software_isp/debayer_cpu.cpp index f8d2677d..9173ab9b 100644 --- a/src/libcamera/software_isp/debayer_cpu.cpp +++ b/src/libcamera/software_isp/debayer_cpu.cpp @@ -18,6 +18,7 @@ #include "libcamera/internal/bayer_format.h" #include "libcamera/internal/framebuffer.h" +#include "libcamera/internal/global_configuration.h" #include "libcamera/internal/mapped_framebuffer.h" namespace libcamera { @@ -44,7 +45,9 @@ DebayerCpu::DebayerCpu(std::unique_ptr stats) * always set it to true as the safer choice but this should be changed in * future. */ - enableInputMemcpy_ = true; + enableInputMemcpy_ = GlobalConfiguration::option( + "pipelines.simple.copy_input_buffer") + .value_or(true); /* Initialize color lookup tables */ for (unsigned int i = 0; i < DebayerParams::kRGBLookupSize; i++) From patchwork Thu Jun 27 14:51:51 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Milan Zamazal X-Patchwork-Id: 20449 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 8F95DBD87C for ; Thu, 27 Jun 2024 14:53:05 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 2E7846334D; Thu, 27 Jun 2024 16:53:05 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.b="ClgxcKjK"; dkim-atps=neutral Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 72E5E63339 for ; Thu, 27 Jun 2024 16:53:01 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1719499980; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=/PCf5v3DKrAXn2DPaauXnOA7Myge+EwO4yuK6rgi3J0=; b=ClgxcKjKwbLHokNqygnVFEukSrBj4Ry8ebfYBpTXPnpk1pERMVLgElqanyW+h3IVL7ZEnJ 2vfRe9y16RJ7fu3iFPDQ0WFiOcWye47H5wvW7rT1V/VpfpIYGCyihRmkvDLwjGgIkelVX/ 38EIpd8gB4qPN3TrMHoG5k6JScQEAbk= Received: from mx-prod-mc-01.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-157-AoSBk2s_N9uyjqxvsjszFg-1; Thu, 27 Jun 2024 10:52:58 -0400 X-MC-Unique: AoSBk2s_N9uyjqxvsjszFg-1 Received: from mx-prod-int-02.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-02.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.15]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-01.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id E5A6D1944CFB; Thu, 27 Jun 2024 14:52:57 +0000 (UTC) Received: from nuthatch.redhat.com (unknown [10.45.225.47]) by mx-prod-int-02.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id 4D2A419773EB; Thu, 27 Jun 2024 14:52:56 +0000 (UTC) From: Milan Zamazal To: libcamera-devel@lists.libcamera.org Cc: Milan Zamazal , Kieran Bingham Subject: [PATCH v3 14/14] libcamera: software_isp: Make measurement configurable Date: Thu, 27 Jun 2024 16:51:51 +0200 Message-ID: <20240627145156.1094127-15-mzamazal@redhat.com> In-Reply-To: <20240627145156.1094127-1-mzamazal@redhat.com> References: <20240627145156.1094127-1-mzamazal@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.0 on 10.30.177.15 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Software ISP performs performance measurement on certain part of initial frames. Let's make this range configurable. For this purpose, this patch introduces new configuration options pipelines.simple.measure.skip and pipelines.simple.measure.number. Setting the latter one to 0 disables the measurement. Instead of the last frame, the class member and its configuration specify the number of frames to measure. This is easier to use for users and doesn't require to adjust two configuration parameters when the number of the initially skipped frames is changed. The patch also changes the names of the class members to make them more accurate. Completes software ISP TODO #7. Signed-off-by: Milan Zamazal --- Documentation/runtime_configuration.rst | 6 ++++++ src/libcamera/software_isp/TODO | 25 ---------------------- src/libcamera/software_isp/debayer_cpu.cpp | 25 ++++++++++++++-------- src/libcamera/software_isp/debayer_cpu.h | 7 +++--- 4 files changed, 25 insertions(+), 38 deletions(-) diff --git a/Documentation/runtime_configuration.rst b/Documentation/runtime_configuration.rst index ebe17a55..c1ec5ac2 100644 --- a/Documentation/runtime_configuration.rst +++ b/Documentation/runtime_configuration.rst @@ -54,6 +54,9 @@ file structure: tuning-file: # full path simple: copy_input_buffer: # true/false + measure: + skip: # non-negative integer, frames to skip initially + number: # non-negative integer, frames to measure supported_devices: - driver: # driver name, e.g. `mxc-isi` software_isp: # true/false @@ -86,6 +89,9 @@ Configuration file example config_file: /usr/local/share/libcamera/pipeline/rpi/vc4/minimal_mem.yaml simple: copy_input_buffer: false + measure: + skip: 50 + number: 30 supported_devices: - driver: mxc-isi software_isp: true diff --git a/src/libcamera/software_isp/TODO b/src/libcamera/software_isp/TODO index f91551dc..950102f1 100644 --- a/src/libcamera/software_isp/TODO +++ b/src/libcamera/software_isp/TODO @@ -90,31 +90,6 @@ per-frame buffers like we do for hardware ISPs. --- -7. Performance measurement configuration - -> void DebayerCpu::process(FrameBuffer *input, FrameBuffer *output, DebayerParams params) -> /* Measure before emitting signals */ -> if (measuredFrames_ < DebayerCpu::kLastFrameToMeasure && -> ++measuredFrames_ > DebayerCpu::kFramesToSkip) { -> timespec frameEndTime = {}; -> clock_gettime(CLOCK_MONOTONIC_RAW, &frameEndTime); -> frameProcessTime_ += timeDiff(frameEndTime, frameStartTime); -> if (measuredFrames_ == DebayerCpu::kLastFrameToMeasure) { -> const unsigned int measuredFrames = DebayerCpu::kLastFrameToMeasure - -> DebayerCpu::kFramesToSkip; -> LOG(Debayer, Info) -> << "Processed " << measuredFrames -> << " frames in " << frameProcessTime_ / 1000 << "us, " -> << frameProcessTime_ / (1000 * measuredFrames) -> << " us/frame"; -> } -> } - -I wonder if there would be a way to control at runtime when/how to -perform those measurements. Maybe that's a bit overkill. - ---- - 8. DebayerCpu cleanups > >> class DebayerCpu : public Debayer, public Object diff --git a/src/libcamera/software_isp/debayer_cpu.cpp b/src/libcamera/software_isp/debayer_cpu.cpp index 9173ab9b..55c94044 100644 --- a/src/libcamera/software_isp/debayer_cpu.cpp +++ b/src/libcamera/software_isp/debayer_cpu.cpp @@ -49,6 +49,13 @@ DebayerCpu::DebayerCpu(std::unique_ptr stats) "pipelines.simple.copy_input_buffer") .value_or(true); + skipBeforeMeasure_ = GlobalConfiguration::option( + "pipelines.simple.measure.skip") + .value_or(skipBeforeMeasure_); + framesToMeasure_ = GlobalConfiguration::option( + "pipelines.simple.measure.number") + .value_or(framesToMeasure_); + /* Initialize color lookup tables */ for (unsigned int i = 0; i < DebayerParams::kRGBLookupSize; i++) red_[i] = green_[i] = blue_[i] = i; @@ -538,7 +545,7 @@ int DebayerCpu::configure(const StreamConfiguration &inputCfg, return -ENOMEM; } - measuredFrames_ = 0; + encounteredFrames_ = 0; frameProcessTime_ = 0; return 0; @@ -738,7 +745,10 @@ void DebayerCpu::process(FrameBuffer *input, FrameBuffer *output, DebayerParams { timespec frameStartTime; - if (measuredFrames_ < DebayerCpu::kLastFrameToMeasure) { + bool measure = framesToMeasure_ > 0 && + encounteredFrames_ < skipBeforeMeasure_ + framesToMeasure_ && + ++encounteredFrames_ > skipBeforeMeasure_; + if (measure) { frameStartTime = {}; clock_gettime(CLOCK_MONOTONIC_RAW, &frameStartTime); } @@ -771,18 +781,15 @@ void DebayerCpu::process(FrameBuffer *input, FrameBuffer *output, DebayerParams metadata.planes()[0].bytesused = out.planes()[0].size(); /* Measure before emitting signals */ - if (measuredFrames_ < DebayerCpu::kLastFrameToMeasure && - ++measuredFrames_ > DebayerCpu::kFramesToSkip) { + if (measure) { timespec frameEndTime = {}; clock_gettime(CLOCK_MONOTONIC_RAW, &frameEndTime); frameProcessTime_ += timeDiff(frameEndTime, frameStartTime); - if (measuredFrames_ == DebayerCpu::kLastFrameToMeasure) { - const unsigned int measuredFrames = DebayerCpu::kLastFrameToMeasure - - DebayerCpu::kFramesToSkip; + if (encounteredFrames_ == skipBeforeMeasure_ + framesToMeasure_) { LOG(Debayer, Info) - << "Processed " << measuredFrames + << "Processed " << framesToMeasure_ << " frames in " << frameProcessTime_ / 1000 << "us, " - << frameProcessTime_ / (1000 * measuredFrames) + << frameProcessTime_ / (1000 * framesToMeasure_) << " us/frame"; } } diff --git a/src/libcamera/software_isp/debayer_cpu.h b/src/libcamera/software_isp/debayer_cpu.h index 1dac6435..de7a6ade 100644 --- a/src/libcamera/software_isp/debayer_cpu.h +++ b/src/libcamera/software_isp/debayer_cpu.h @@ -153,11 +153,10 @@ private: unsigned int xShift_; /* Offset of 0/1 applied to window_.x */ bool enableInputMemcpy_; bool swapRedBlueGains_; - unsigned int measuredFrames_; + unsigned int encounteredFrames_; int64_t frameProcessTime_; - /* Skip 30 frames for things to stabilize then measure 30 frames */ - static constexpr unsigned int kFramesToSkip = 30; - static constexpr unsigned int kLastFrameToMeasure = 60; + unsigned int skipBeforeMeasure_ = 30; + unsigned int framesToMeasure_ = 30; }; } /* namespace libcamera */