[{"id":31357,"web_url":"https://patchwork.libcamera.org/comment/31357/","msgid":"<172726546598.3726802.18254223104028052063@ping.linuxembedded.co.uk>","date":"2024-09-25T11:57:45","subject":"Re: [PATCH v4 00/15] Add global configuration file","submitter":{"id":4,"url":"https://patchwork.libcamera.org/api/people/4/","name":"Kieran Bingham","email":"kieran.bingham@ideasonboard.com"},"content":"Quoting Milan Zamazal (2024-09-25 10:58:32)\n> This patch series introduces global configuration file for libcamera, to\n> provide runtime configuration means other than environment variables.\n> Instead of, or in addition to, the growing list of configuration\n> environment variables, the whole configuration can be specified in a\n> single configuration file.  This is both simpler and more flexible.\n> \n> This is not a replacement for specific configuration files already\n> present in libcamera.\n> \n> The patches implement what is needed to introduce a configuration file\n> that can handle the current environment variables and software ISP\n> TODOs.  They demonstrate how to deal with the key points that must be\n> considered.  See commit messages for more details.\n> \n> The configuration file is a YAML file.  It is looked up in user’s home\n> directory or, if not present, in system-wide libcamera directories.\n> Environment variables, if set, still take precedence.\n> \n> This patch series is not exhaustive, there can be future enhancements,\n> most notably configuration file validation to avoid confusions caused by\n> typos etc.\n> \n> Not everything has been tested because some of the patches are related\n> to specific hardware.\n\nIt looks like this breaks on the unit tests:\n\n - https://gitlab.freedesktop.org/camera/libcamera/-/jobs/64126236\n\nAnd I re-ran incase it was spurious, but that looks like it's going to\nfail too:\n\n - https://gitlab.freedesktop.org/camera/libcamera/-/jobs/64131685\n\nCould you check/take a look please?\n\n--\nKieran\n\n\n> \n> Changes in v4:\n> - Rebased on current master.\n> - Configuration option for LIBCAMERA_IPA_PROXY_PATH added.\n> - Added a patch to include stdlib.h instead of cstdlib in yaml_parser.cpp.\n> \n> Changes in v3:\n> - Added a configuration item for the newly introduced\n>   LIBCAMERA_PIPELINES_MATCH_LIST variable.\n> - A minor indentation fix.\n> - Fixed `pipelines.' x `pipeline.' configuration item naming mismatch.\n> - Tuning files are looked up now by particular cameras attached rather than\n>   being specified for the whole pipeline.\n> - Helpers use std::string& instead of char* for confPath arguments.\n> - Protection against returning YamlObject::empty as a regular value (the\n>   problem has been probably exposed by addition of\n>   LIBCAMERA_PIPELINES_MATCH_LIST).\n> \n> Changes in v2:\n> - Rebased on master.\n> - Various cleanups, documentation improvements and minor fixes.\n> - Configuration option for LIBCAMERA_RPI_TUNING_FILE added (Naush).\n> - Two more patches for software ISP configuration added.\n> \n> Milan Zamazal (15):\n>   config: Introduce global runtime configuration\n>   yaml: Include stdlib.h instead of cstdlib\n>   config: Move global configuration to base\n>   config: Look up logging levels in the configuration file\n>   config: Add configuration retrieval helpers\n>   config: Look up log file in configuration file\n>   config: Look up log color configuration in configuration file\n>   config: Look up rpi paths in configuration file\n>   config: Look up IPA configurables in configuration file\n>   config: Look up RkISP1 tuning file in configuration file\n>   config: Look up pipelines match list in configuration file\n>   config: Allow enabling software ISP in runtime\n>   config: Add global configuration file documentation\n>   libcamera: software_isp: Make input buffer copying configurable\n>   libcamera: software_isp: Make measurement configurable\n> \n>  Documentation/documentation-contents.rst      |   2 +-\n>  Documentation/index.rst                       |   4 +-\n>  Documentation/meson.build                     |   2 +-\n>  ...ariables.rst => runtime_configuration.rst} | 119 ++++++++--\n>  .../libcamera/internal/global_configuration.h |  67 ++++++\n>  include/libcamera/internal/meson.build        |   1 +\n>  src/libcamera/base/global_configuration.cpp   | 219 ++++++++++++++++++\n>  src/libcamera/base/log.cpp                    |  24 +-\n>  src/libcamera/base/meson.build                |  15 ++\n>  src/libcamera/{ => base}/yaml_parser.cpp      |   2 +-\n>  src/libcamera/camera_manager.cpp              |  19 +-\n>  src/libcamera/ipa_manager.cpp                 |  23 +-\n>  src/libcamera/ipa_proxy.cpp                   |  16 +-\n>  src/libcamera/meson.build                     |  14 --\n>  src/libcamera/pipeline/rkisp1/rkisp1.cpp      |  20 +-\n>  .../pipeline/rpi/common/pipeline_base.cpp     |  18 +-\n>  src/libcamera/pipeline/simple/simple.cpp      |  12 +\n>  src/libcamera/process.cpp                     |  11 +-\n>  src/libcamera/software_isp/TODO               |  36 ---\n>  src/libcamera/software_isp/debayer_cpu.cpp    |  30 ++-\n>  src/libcamera/software_isp/debayer_cpu.h      |   7 +-\n>  21 files changed, 549 insertions(+), 112 deletions(-)\n>  rename Documentation/{environment_variables.rst => runtime_configuration.rst} (61%)\n>  create mode 100644 include/libcamera/internal/global_configuration.h\n>  create mode 100644 src/libcamera/base/global_configuration.cpp\n>  rename src/libcamera/{ => base}/yaml_parser.cpp (99%)\n> \n> -- \n> 2.44.1\n>","headers":{"Return-Path":"<libcamera-devel-bounces@lists.libcamera.org>","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id D2529C3257\n\tfor <parsemail@patchwork.libcamera.org>;\n\tWed, 25 Sep 2024 11:57:51 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id E0E356350E;\n\tWed, 25 Sep 2024 13:57:50 +0200 (CEST)","from perceval.ideasonboard.com (perceval.ideasonboard.com\n\t[IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id 51F0B634F4\n\tfor <libcamera-devel@lists.libcamera.org>;\n\tWed, 25 Sep 2024 13:57:48 +0200 (CEST)","from pendragon.ideasonboard.com\n\t(cpc89244-aztw30-2-0-cust6594.18-1.cable.virginm.net [86.31.185.195])\n\tby perceval.ideasonboard.com (Postfix) with ESMTPSA id CBCF7A30;\n\tWed, 25 Sep 2024 13:56:20 +0200 (CEST)"],"Authentication-Results":"lancelot.ideasonboard.com; dkim=pass (1024-bit key;\n\tunprotected) header.d=ideasonboard.com header.i=@ideasonboard.com\n\theader.b=\"beqpxjUt\"; dkim-atps=neutral","DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com;\n\ts=mail; t=1727265380;\n\tbh=eSyI8Hw2xBt/YtntBtGiQi6nGcbo47JmgLVCeRpxdwA=;\n\th=In-Reply-To:References:Subject:From:Cc:To:Date:From;\n\tb=beqpxjUtTUm/srbZpL9KYLlpGy3VioArfN++ad/7KKtkamkIidVmGBU+pC5SRuddc\n\tjxvyQnXbLGpnEYuKDtyxzCq4BuPW1A1WYGdN89JCvbOch/9KeGcmrRjnTnycGgnibe\n\tYSoqG3Ao0z+oydt3yiU18uPt/zzCrU+Jz+ktnYAI=","Content-Type":"text/plain; charset=\"utf-8\"","MIME-Version":"1.0","Content-Transfer-Encoding":"quoted-printable","In-Reply-To":"<20240925095850.348259-1-mzamazal@redhat.com>","References":"<20240925095850.348259-1-mzamazal@redhat.com>","Subject":"Re: [PATCH v4 00/15] Add global configuration file","From":"Kieran Bingham <kieran.bingham@ideasonboard.com>","Cc":"Milan Zamazal <mzamazal@redhat.com>,\n\tNaushir Patuck <naush@raspberrypi.com>","To":"Milan Zamazal <mzamazal@redhat.com>, libcamera-devel@lists.libcamera.org","Date":"Wed, 25 Sep 2024 12:57:45 +0100","Message-ID":"<172726546598.3726802.18254223104028052063@ping.linuxembedded.co.uk>","User-Agent":"alot/0.10","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"<libcamera-devel.lists.libcamera.org>","List-Unsubscribe":"<https://lists.libcamera.org/options/libcamera-devel>,\n\t<mailto:libcamera-devel-request@lists.libcamera.org?subject=unsubscribe>","List-Archive":"<https://lists.libcamera.org/pipermail/libcamera-devel/>","List-Post":"<mailto:libcamera-devel@lists.libcamera.org>","List-Help":"<mailto:libcamera-devel-request@lists.libcamera.org?subject=help>","List-Subscribe":"<https://lists.libcamera.org/listinfo/libcamera-devel>,\n\t<mailto:libcamera-devel-request@lists.libcamera.org?subject=subscribe>","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" <libcamera-devel-bounces@lists.libcamera.org>"}},{"id":31365,"web_url":"https://patchwork.libcamera.org/comment/31365/","msgid":"<875xqj1lbx.fsf@redhat.com>","date":"2024-09-25T21:21:06","subject":"Re: [PATCH v4 00/15] Add global configuration file","submitter":{"id":177,"url":"https://patchwork.libcamera.org/api/people/177/","name":"Milan Zamazal","email":"mzamazal@redhat.com"},"content":"Kieran Bingham <kieran.bingham@ideasonboard.com> writes:\n\n> Quoting Milan Zamazal (2024-09-25 10:58:32)\n>> This patch series introduces global configuration file for libcamera, to\n>> provide runtime configuration means other than environment variables.\n>\n>> Instead of, or in addition to, the growing list of configuration\n>> environment variables, the whole configuration can be specified in a\n>> single configuration file.  This is both simpler and more flexible.\n>> \n>> This is not a replacement for specific configuration files already\n>> present in libcamera.\n>> \n>> The patches implement what is needed to introduce a configuration file\n>> that can handle the current environment variables and software ISP\n>> TODOs.  They demonstrate how to deal with the key points that must be\n>> considered.  See commit messages for more details.\n>> \n>> The configuration file is a YAML file.  It is looked up in user’s home\n>> directory or, if not present, in system-wide libcamera directories.\n>> Environment variables, if set, still take precedence.\n>> \n>> This patch series is not exhaustive, there can be future enhancements,\n>> most notably configuration file validation to avoid confusions caused by\n>> typos etc.\n>> \n>> Not everything has been tested because some of the patches are related\n>> to specific hardware.\n>\n> It looks like this breaks on the unit tests:\n>\n>  - https://gitlab.freedesktop.org/camera/libcamera/-/jobs/64126236\n>\n> And I re-ran incase it was spurious, but that looks like it's going to\n> fail too:\n>\n>  - https://gitlab.freedesktop.org/camera/libcamera/-/jobs/64131685\n>\n> Could you check/take a look please?\n\nI can reproduce it locally.  It happens in the patch #4 \"Look up logging\nlevels in the configuration file\" where the configuration starts being\nused.  And apparently only if the already loaded configuration is\nactually queried.  The errors don't make sense to me, indicating a\npossible memory or thread problem.  I'll continue investigating it\nlater.\n\n> --\n> Kieran\n>\n>\n>> \n>> Changes in v4:\n>> - Rebased on current master.\n>> - Configuration option for LIBCAMERA_IPA_PROXY_PATH added.\n>> - Added a patch to include stdlib.h instead of cstdlib in yaml_parser.cpp.\n>> \n>> Changes in v3:\n>> - Added a configuration item for the newly introduced\n>>   LIBCAMERA_PIPELINES_MATCH_LIST variable.\n>> - A minor indentation fix.\n>> - Fixed `pipelines.' x `pipeline.' configuration item naming mismatch.\n>> - Tuning files are looked up now by particular cameras attached rather than\n>>   being specified for the whole pipeline.\n>> - Helpers use std::string& instead of char* for confPath arguments.\n>> - Protection against returning YamlObject::empty as a regular value (the\n>>   problem has been probably exposed by addition of\n>>   LIBCAMERA_PIPELINES_MATCH_LIST).\n>> \n>> Changes in v2:\n>> - Rebased on master.\n>> - Various cleanups, documentation improvements and minor fixes.\n>> - Configuration option for LIBCAMERA_RPI_TUNING_FILE added (Naush).\n>> - Two more patches for software ISP configuration added.\n>> \n>> Milan Zamazal (15):\n>>   config: Introduce global runtime configuration\n>>   yaml: Include stdlib.h instead of cstdlib\n>>   config: Move global configuration to base\n>>   config: Look up logging levels in the configuration file\n>>   config: Add configuration retrieval helpers\n>>   config: Look up log file in configuration file\n>>   config: Look up log color configuration in configuration file\n>>   config: Look up rpi paths in configuration file\n>>   config: Look up IPA configurables in configuration file\n>>   config: Look up RkISP1 tuning file in configuration file\n>>   config: Look up pipelines match list in configuration file\n>>   config: Allow enabling software ISP in runtime\n>>   config: Add global configuration file documentation\n>>   libcamera: software_isp: Make input buffer copying configurable\n>>   libcamera: software_isp: Make measurement configurable\n>> \n>>  Documentation/documentation-contents.rst      |   2 +-\n>>  Documentation/index.rst                       |   4 +-\n>>  Documentation/meson.build                     |   2 +-\n>>  ...ariables.rst => runtime_configuration.rst} | 119 ++++++++--\n>>  .../libcamera/internal/global_configuration.h |  67 ++++++\n>>  include/libcamera/internal/meson.build        |   1 +\n>>  src/libcamera/base/global_configuration.cpp   | 219 ++++++++++++++++++\n>>  src/libcamera/base/log.cpp                    |  24 +-\n>>  src/libcamera/base/meson.build                |  15 ++\n>>  src/libcamera/{ => base}/yaml_parser.cpp      |   2 +-\n>>  src/libcamera/camera_manager.cpp              |  19 +-\n>>  src/libcamera/ipa_manager.cpp                 |  23 +-\n>>  src/libcamera/ipa_proxy.cpp                   |  16 +-\n>>  src/libcamera/meson.build                     |  14 --\n>>  src/libcamera/pipeline/rkisp1/rkisp1.cpp      |  20 +-\n>>  .../pipeline/rpi/common/pipeline_base.cpp     |  18 +-\n>>  src/libcamera/pipeline/simple/simple.cpp      |  12 +\n>>  src/libcamera/process.cpp                     |  11 +-\n>>  src/libcamera/software_isp/TODO               |  36 ---\n>>  src/libcamera/software_isp/debayer_cpu.cpp    |  30 ++-\n>>  src/libcamera/software_isp/debayer_cpu.h      |   7 +-\n>>  21 files changed, 549 insertions(+), 112 deletions(-)\n>>  rename Documentation/{environment_variables.rst => runtime_configuration.rst} (61%)\n>>  create mode 100644 include/libcamera/internal/global_configuration.h\n>>  create mode 100644 src/libcamera/base/global_configuration.cpp\n>>  rename src/libcamera/{ => base}/yaml_parser.cpp (99%)\n>> \n>> -- \n>> 2.44.1\n>>","headers":{"Return-Path":"<libcamera-devel-bounces@lists.libcamera.org>","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id DEC3EC3257\n\tfor <parsemail@patchwork.libcamera.org>;\n\tWed, 25 Sep 2024 21:21:14 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id 1972F634F4;\n\tWed, 25 Sep 2024 23:21:14 +0200 (CEST)","from us-smtp-delivery-124.mimecast.com\n\t(us-smtp-delivery-124.mimecast.com [170.10.133.124])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id B3CF7634F4\n\tfor <libcamera-devel@lists.libcamera.org>;\n\tWed, 25 Sep 2024 23:21:12 +0200 (CEST)","from mail-wr1-f69.google.com (mail-wr1-f69.google.com\n\t[209.85.221.69]) by relay.mimecast.com with ESMTP with STARTTLS\n\t(version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id\n\tus-mta-500-EPx4bacWPQmSZsOAGmXETg-1; Wed, 25 Sep 2024 17:21:10 -0400","by mail-wr1-f69.google.com with SMTP id\n\tffacd0b85a97d-37cc9b5e533so95881f8f.3\n\tfor <libcamera-devel@lists.libcamera.org>;\n\tWed, 25 Sep 2024 14:21:10 -0700 (PDT)","from nuthatch (ip-77-48-47-2.net.vodafone.cz. [77.48.47.2])\n\tby smtp.gmail.com with ESMTPSA id\n\tffacd0b85a97d-37cbc2f9c27sm4879070f8f.70.2024.09.25.14.21.07\n\t(version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);\n\tWed, 25 Sep 2024 14:21:07 -0700 (PDT)"],"Authentication-Results":"lancelot.ideasonboard.com; dkim=pass (1024-bit key;\n\tunprotected) header.d=redhat.com header.i=@redhat.com\n\theader.b=\"PiTDK1Gi\"; dkim-atps=neutral","DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com;\n\ts=mimecast20190719; t=1727299271;\n\th=from:from:reply-to:subject:subject:date:date:message-id:message-id:\n\tto:to:cc:cc:mime-version:mime-version:content-type:content-type:\n\tcontent-transfer-encoding:content-transfer-encoding:\n\tin-reply-to:in-reply-to:references:references;\n\tbh=xKsfydW4NE9D0Ea3zeAYMw0dq/yvHQoOzS56cAJ01wA=;\n\tb=PiTDK1GiwVU19TaJ3Ca77ijwQ5gGsNpPqPs+NNjcXCQpDYyq8R6OeclntxHFFao9EeS5wj\n\t6ALZe4+FU7uQIWfgyx/BEL287eIH9KZZpxANmO9nrst3lbBLTOXwX8A70HBdfDWYUFu0el\n\t+kRF82Z2zrG50EQMcLXsqQAvTpyH4ss=","X-MC-Unique":"EPx4bacWPQmSZsOAGmXETg-1","X-Google-DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed;\n\td=1e100.net; s=20230601; t=1727299268; x=1727904068;\n\th=content-transfer-encoding:mime-version:user-agent:message-id:date\n\t:references:in-reply-to:subject:cc:to:from:x-gm-message-state:from\n\t:to:cc:subject:date:message-id:reply-to;\n\tbh=xKsfydW4NE9D0Ea3zeAYMw0dq/yvHQoOzS56cAJ01wA=;\n\tb=paSXDK1DUIeycUVro2VL3LmVVvALYdUJkDpRXaNQxGEtNSxXWu/SVQFnNo4zqrrBZv\n\tDu4xX2/VkZA0df5IAXKABLaQIbn1K14BPoD4Z1O2uKn3znZiiZDOcDH1kkPmgooAJCC4\n\toHXr6mD9AubWFeLJsh0A+ODgQIESVzVBirxVMbSUMnWpiLeHi9iWUYLdu1RhhOl6//X2\n\tgNMZ2yq3pBVTuXyViUSWnM8GmZmTqLXh0Qkbp9nBLZaREAvDFabY3xZVUOi3XyGxqo0l\n\tFyacrKxSsNFjwsY0+C72AXfhl9ortNolASnru2tr0ZWwvp8A466gy3+7ZaVypH9sjP4f\n\tC/aw==","X-Gm-Message-State":"AOJu0YzutJb/OohZz5daIW6c71g3+aXjafO12K8/UPO9768SVTxK7xCT\n\tSeNrEcJ388yfozq0yOSj7GFFmU0Sylosnj5r6Fe7aAZ9tDzXFbsYr5JuPByFPx2jN4vqbWgIWSN\n\tWRT91T45SVswJunhBrq2Yw9aTaPkgenrt4WwsBmxIOjjkELmOr/A9INiNezrWXx2ERbFZtJmaga\n\tTB6g4=","X-Received":["by 2002:adf:fc4b:0:b0:374:ca4f:bd70 with SMTP id\n\tffacd0b85a97d-37cc24b50b6mr2697459f8f.53.1727299268460; \n\tWed, 25 Sep 2024 14:21:08 -0700 (PDT)","by 2002:adf:fc4b:0:b0:374:ca4f:bd70 with SMTP id\n\tffacd0b85a97d-37cc24b50b6mr2697449f8f.53.1727299268032; \n\tWed, 25 Sep 2024 14:21:08 -0700 (PDT)"],"X-Google-Smtp-Source":"AGHT+IFCpjnmnPCyWNlWHqknTswgfVNARNEDsCRezg+UP4x0gJufNC0XgQUUqdS4OjnX3J3NilIwDw==","From":"Milan Zamazal <mzamazal@redhat.com>","To":"Kieran Bingham <kieran.bingham@ideasonboard.com>","Cc":"libcamera-devel@lists.libcamera.org,  Naushir Patuck\n\t<naush@raspberrypi.com>","Subject":"Re: [PATCH v4 00/15] Add global configuration file","In-Reply-To":"<172726546598.3726802.18254223104028052063@ping.linuxembedded.co.uk>\n\t(Kieran Bingham's message of \"Wed, 25 Sep 2024 12:57:45 +0100\")","References":"<20240925095850.348259-1-mzamazal@redhat.com>\n\t<172726546598.3726802.18254223104028052063@ping.linuxembedded.co.uk>","Date":"Wed, 25 Sep 2024 23:21:06 +0200","Message-ID":"<875xqj1lbx.fsf@redhat.com>","User-Agent":"Gnus/5.13 (Gnus v5.13)","MIME-Version":"1.0","X-Mimecast-Spam-Score":"0","X-Mimecast-Originator":"redhat.com","Content-Type":"text/plain; charset=utf-8","Content-Transfer-Encoding":"quoted-printable","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"<libcamera-devel.lists.libcamera.org>","List-Unsubscribe":"<https://lists.libcamera.org/options/libcamera-devel>,\n\t<mailto:libcamera-devel-request@lists.libcamera.org?subject=unsubscribe>","List-Archive":"<https://lists.libcamera.org/pipermail/libcamera-devel/>","List-Post":"<mailto:libcamera-devel@lists.libcamera.org>","List-Help":"<mailto:libcamera-devel-request@lists.libcamera.org?subject=help>","List-Subscribe":"<https://lists.libcamera.org/listinfo/libcamera-devel>,\n\t<mailto:libcamera-devel-request@lists.libcamera.org?subject=subscribe>","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" <libcamera-devel-bounces@lists.libcamera.org>"}},{"id":31494,"web_url":"https://patchwork.libcamera.org/comment/31494/","msgid":"<87wmisw1py.fsf@redhat.com>","date":"2024-10-01T10:34:49","subject":"Re: [PATCH v4 00/15] Add global configuration file","submitter":{"id":177,"url":"https://patchwork.libcamera.org/api/people/177/","name":"Milan Zamazal","email":"mzamazal@redhat.com"},"content":"Milan Zamazal <mzamazal@redhat.com> writes:\n\n> Kieran Bingham <kieran.bingham@ideasonboard.com> writes:\n>\n>> Quoting Milan Zamazal (2024-09-25 10:58:32)\n>>> This patch series introduces global configuration file for libcamera, to\n>>> provide runtime configuration means other than environment variables.\n>>\n>>> Instead of, or in addition to, the growing list of configuration\n>>> environment variables, the whole configuration can be specified in a\n>>> single configuration file.  This is both simpler and more flexible.\n>>> \n>>> This is not a replacement for specific configuration files already\n>>> present in libcamera.\n>>> \n>>> The patches implement what is needed to introduce a configuration file\n>>> that can handle the current environment variables and software ISP\n>>> TODOs.  They demonstrate how to deal with the key points that must be\n>>> considered.  See commit messages for more details.\n>>> \n>>> The configuration file is a YAML file.  It is looked up in user’s home\n>>> directory or, if not present, in system-wide libcamera directories.\n>>> Environment variables, if set, still take precedence.\n>>> \n>>> This patch series is not exhaustive, there can be future enhancements,\n>>> most notably configuration file validation to avoid confusions caused by\n>>> typos etc.\n>>> \n>>> Not everything has been tested because some of the patches are related\n>>> to specific hardware.\n>>\n>> It looks like this breaks on the unit tests:\n>>\n>>  - https://gitlab.freedesktop.org/camera/libcamera/-/jobs/64126236\n>>\n>> And I re-ran incase it was spurious, but that looks like it's going to\n>> fail too:\n>>\n>>  - https://gitlab.freedesktop.org/camera/libcamera/-/jobs/64131685\n>>\n>> Could you check/take a look please?\n>\n> I can reproduce it locally.  It happens in the patch #4 \"Look up logging\n> levels in the configuration file\" where the configuration starts being\n> used.  And apparently only if the already loaded configuration is\n> actually queried.  The errors don't make sense to me, indicating a\n> possible memory or thread problem.  I'll continue investigating it\n> later.\n\nI haven't identified the exact cause but I reworked storing the\nconfiguration and its initialization to be a bit more robust, which also\nfixed the test.\n\n>> --\n>> Kieran\n>>\n>>\n>>> \n>>> Changes in v4:\n>>> - Rebased on current master.\n>>> - Configuration option for LIBCAMERA_IPA_PROXY_PATH added.\n>>> - Added a patch to include stdlib.h instead of cstdlib in yaml_parser.cpp.\n>>> \n>>> Changes in v3:\n>>> - Added a configuration item for the newly introduced\n>>>   LIBCAMERA_PIPELINES_MATCH_LIST variable.\n>>> - A minor indentation fix.\n>>> - Fixed `pipelines.' x `pipeline.' configuration item naming mismatch.\n>>> - Tuning files are looked up now by particular cameras attached rather than\n>>>   being specified for the whole pipeline.\n>>> - Helpers use std::string& instead of char* for confPath arguments.\n>>> - Protection against returning YamlObject::empty as a regular value (the\n>>>   problem has been probably exposed by addition of\n>>>   LIBCAMERA_PIPELINES_MATCH_LIST).\n>>> \n>>> Changes in v2:\n>>> - Rebased on master.\n>>> - Various cleanups, documentation improvements and minor fixes.\n>>> - Configuration option for LIBCAMERA_RPI_TUNING_FILE added (Naush).\n>>> - Two more patches for software ISP configuration added.\n>>> \n>>> Milan Zamazal (15):\n>>>   config: Introduce global runtime configuration\n>>>   yaml: Include stdlib.h instead of cstdlib\n>>>   config: Move global configuration to base\n>>>   config: Look up logging levels in the configuration file\n>>>   config: Add configuration retrieval helpers\n>>>   config: Look up log file in configuration file\n>>>   config: Look up log color configuration in configuration file\n>>>   config: Look up rpi paths in configuration file\n>>>   config: Look up IPA configurables in configuration file\n>>>   config: Look up RkISP1 tuning file in configuration file\n>>>   config: Look up pipelines match list in configuration file\n>>>   config: Allow enabling software ISP in runtime\n>>>   config: Add global configuration file documentation\n>>>   libcamera: software_isp: Make input buffer copying configurable\n>>>   libcamera: software_isp: Make measurement configurable\n>>> \n>>>  Documentation/documentation-contents.rst      |   2 +-\n>>>  Documentation/index.rst                       |   4 +-\n>>>  Documentation/meson.build                     |   2 +-\n>>>  ...ariables.rst => runtime_configuration.rst} | 119 ++++++++--\n>>>  .../libcamera/internal/global_configuration.h |  67 ++++++\n>>>  include/libcamera/internal/meson.build        |   1 +\n>>>  src/libcamera/base/global_configuration.cpp   | 219 ++++++++++++++++++\n>>>  src/libcamera/base/log.cpp                    |  24 +-\n>>>  src/libcamera/base/meson.build                |  15 ++\n>>>  src/libcamera/{ => base}/yaml_parser.cpp      |   2 +-\n>>>  src/libcamera/camera_manager.cpp              |  19 +-\n>>>  src/libcamera/ipa_manager.cpp                 |  23 +-\n>>>  src/libcamera/ipa_proxy.cpp                   |  16 +-\n>>>  src/libcamera/meson.build                     |  14 --\n>>>  src/libcamera/pipeline/rkisp1/rkisp1.cpp      |  20 +-\n>>>  .../pipeline/rpi/common/pipeline_base.cpp     |  18 +-\n>>>  src/libcamera/pipeline/simple/simple.cpp      |  12 +\n>>>  src/libcamera/process.cpp                     |  11 +-\n>>>  src/libcamera/software_isp/TODO               |  36 ---\n>>>  src/libcamera/software_isp/debayer_cpu.cpp    |  30 ++-\n>>>  src/libcamera/software_isp/debayer_cpu.h      |   7 +-\n>>>  21 files changed, 549 insertions(+), 112 deletions(-)\n>>>  rename Documentation/{environment_variables.rst => runtime_configuration.rst} (61%)\n>>>  create mode 100644 include/libcamera/internal/global_configuration.h\n>>>  create mode 100644 src/libcamera/base/global_configuration.cpp\n>>>  rename src/libcamera/{ => base}/yaml_parser.cpp (99%)\n>>> \n>>> -- \n>>> 2.44.1\n>>>","headers":{"Return-Path":"<libcamera-devel-bounces@lists.libcamera.org>","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id E8887C3257\n\tfor <parsemail@patchwork.libcamera.org>;\n\tTue,  1 Oct 2024 10:34:57 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id B4720618D7;\n\tTue,  1 Oct 2024 12:34:56 +0200 (CEST)","from us-smtp-delivery-124.mimecast.com\n\t(us-smtp-delivery-124.mimecast.com [170.10.133.124])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id 1ABD960553\n\tfor <libcamera-devel@lists.libcamera.org>;\n\tTue,  1 Oct 2024 12:34:54 +0200 (CEST)","from mail-wm1-f72.google.com (mail-wm1-f72.google.com\n\t[209.85.128.72]) by relay.mimecast.com with ESMTP with STARTTLS\n\t(version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id\n\tus-mta-556-wOq7IDKHN_-tCrmoWgBRLQ-1; Tue, 01 Oct 2024 06:34:53 -0400","by mail-wm1-f72.google.com with SMTP id\n\t5b1f17b1804b1-42cb236ad4aso30385185e9.3\n\tfor <libcamera-devel@lists.libcamera.org>;\n\tTue, 01 Oct 2024 03:34:52 -0700 (PDT)","from nuthatch (nat-pool-brq-t.redhat.com. [213.175.37.10])\n\tby smtp.gmail.com with ESMTPSA id\n\t5b1f17b1804b1-42f73d75941sm9655455e9.1.2024.10.01.03.34.49\n\t(version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);\n\tTue, 01 Oct 2024 03:34:50 -0700 (PDT)"],"Authentication-Results":"lancelot.ideasonboard.com; dkim=pass (1024-bit key;\n\tunprotected) header.d=redhat.com header.i=@redhat.com\n\theader.b=\"CzqKG8Ms\"; dkim-atps=neutral","DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com;\n\ts=mimecast20190719; t=1727778893;\n\th=from:from:reply-to:subject:subject:date:date:message-id:message-id:\n\tto:to:cc:cc:mime-version:mime-version:content-type:content-type:\n\tcontent-transfer-encoding:content-transfer-encoding:\n\tin-reply-to:in-reply-to:references:references;\n\tbh=X4H6WO5b5vyqLi93nVvO7/azbWbKFhGznkRnyPvjG0o=;\n\tb=CzqKG8MsrBiWQxja5gV++D1k37yKE1IgGrrkBO1TYM85sBVqLBMTofpEyuKgKZuRgK63R4\n\tnh9QxqaSfEdOsTqoYHB371Yi3lL4+luce8kncYM8+kCwdzx09kChxt4RC5aLNDyBpopq1C\n\toMqzYHYLXYgLvMRpJuK4xOZGYuTwWic=","X-MC-Unique":"wOq7IDKHN_-tCrmoWgBRLQ-1","X-Google-DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed;\n\td=1e100.net; s=20230601; t=1727778891; x=1728383691;\n\th=content-transfer-encoding:mime-version:user-agent:message-id:date\n\t:references:in-reply-to:subject:cc:to:from:x-gm-message-state:from\n\t:to:cc:subject:date:message-id:reply-to;\n\tbh=3aZhNcpz9u9V0ZIGxf3dgz9mFP1NNotVry8ic7a0U7g=;\n\tb=NlkDgABdvZ4zD4oTs5bk9/w2q35USEfzMPHjKRGhgDr2pkIwcEEvzBi/NsvlLl8YrZ\n\tllNrVXtgDTU3bV2qkeq5jK810obBr/K63d4CA+BC06hZSL4u4HWtQkqBNRtruvf3puVj\n\tcrZKZDvvLNzsvhY57Qb/FcB/7u+DfT099Q25Ri2N5ClASnzyOK+c35rUhYNivhRGabVJ\n\t3IcP8Ad78jBsf5f2Yy20Q19BCdnLFl1T867PnQ94WA7t1eXjcU+qarnuwej/i+4O9C6B\n\t/rvdWQK60RuYVwniWa+lYJglNW1QcRmT53hboH0GVOBEUm6abjZR+oS17RuplrTacjO0\n\thcxg==","X-Gm-Message-State":"AOJu0Yxv0+fex0YUrX3D1CFlXQprKeGQURINGYoU1s6m8Nu5f/RL52Xq\n\tZkPQihCye3BA/1Hg1JHqxzNWlXk57d7KnaeNfDBy3IQXUNyffqMctkE8xPJHcnnagToWuzP9Ztn\n\tl/9deO5QyQes4Mx5d30KGLpRSCG3zc7swPlqRQDKDbrWQ5i79Gpcy2QtGfSkYLYHMbaFuM96f1f\n\tUxboY=","X-Received":["by 2002:a05:600c:3b17:b0:426:50e1:ea6e with SMTP id\n\t5b1f17b1804b1-42f5844c301mr117479695e9.18.1727778891462; \n\tTue, 01 Oct 2024 03:34:51 -0700 (PDT)","by 2002:a05:600c:3b17:b0:426:50e1:ea6e with SMTP id\n\t5b1f17b1804b1-42f5844c301mr117479545e9.18.1727778891028; \n\tTue, 01 Oct 2024 03:34:51 -0700 (PDT)"],"X-Google-Smtp-Source":"AGHT+IF+0p/NnLYe7appm/+Bcg7yYauwLhlTeulmbrRkTBJHVYlGoQeh7dtdkvQ6zUvSrhHNJVQE+g==","From":"Milan Zamazal <mzamazal@redhat.com>","To":"Kieran Bingham <kieran.bingham@ideasonboard.com>","Cc":"libcamera-devel@lists.libcamera.org,  Naushir Patuck\n\t<naush@raspberrypi.com>","Subject":"Re: [PATCH v4 00/15] Add global configuration file","In-Reply-To":"<875xqj1lbx.fsf@redhat.com> (Milan Zamazal's message of \"Wed, 25\n\tSep 2024 23:21:06 +0200\")","References":"<20240925095850.348259-1-mzamazal@redhat.com>\n\t<172726546598.3726802.18254223104028052063@ping.linuxembedded.co.uk>\n\t<875xqj1lbx.fsf@redhat.com>","Date":"Tue, 01 Oct 2024 12:34:49 +0200","Message-ID":"<87wmisw1py.fsf@redhat.com>","User-Agent":"Gnus/5.13 (Gnus v5.13)","MIME-Version":"1.0","X-Mimecast-Spam-Score":"0","X-Mimecast-Originator":"redhat.com","Content-Type":"text/plain; charset=utf-8","Content-Transfer-Encoding":"quoted-printable","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"<libcamera-devel.lists.libcamera.org>","List-Unsubscribe":"<https://lists.libcamera.org/options/libcamera-devel>,\n\t<mailto:libcamera-devel-request@lists.libcamera.org?subject=unsubscribe>","List-Archive":"<https://lists.libcamera.org/pipermail/libcamera-devel/>","List-Post":"<mailto:libcamera-devel@lists.libcamera.org>","List-Help":"<mailto:libcamera-devel-request@lists.libcamera.org?subject=help>","List-Subscribe":"<https://lists.libcamera.org/listinfo/libcamera-devel>,\n\t<mailto:libcamera-devel-request@lists.libcamera.org?subject=subscribe>","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" <libcamera-devel-bounces@lists.libcamera.org>"}}]