[{"id":33164,"web_url":"https://patchwork.libcamera.org/comment/33164/","msgid":"","date":"2025-01-24T18:19:47","subject":"Re: [PATCH] libcamera: pipeline: Move tuning file override handling\n\tto IPAProxy","submitter":{"id":184,"url":"https://patchwork.libcamera.org/api/people/184/","name":"Stefan Klug","email":"stefan.klug@ideasonboard.com"},"content":"Hi Laurent,\n\nThank you for the patch.\n\nOn Thu, Jan 23, 2025 at 04:38:18PM +0200, Laurent Pinchart wrote:\n> The rkisp1 and rpi pipeline handlers duplicate code to handle the\n> LIBCAMERA_RKISP1_TUNING_FILE and LIBCAMERA_RPI_TUNING_FILE environment\n> variables that override tuning file selection. Move the common code to\n> IPAProxy::configurationFile() to avoid the duplication, and make the\n> feature available to all pipeline handlers with the same behaviour.\n> \n> Signed-off-by: Laurent Pinchart \n> ---\n> Documentation/environment_variables.rst | 4 +--\n> src/libcamera/ipa_proxy.cpp | 27 +++++++++++++++----\n> src/libcamera/pipeline/rkisp1/rkisp1.cpp | 15 +++--------\n> .../pipeline/rpi/common/pipeline_base.cpp | 19 ++++---------\n> 4 files changed, 32 insertions(+), 33 deletions(-)\n> \n> diff --git a/Documentation/environment_variables.rst b/Documentation/environment_variables.rst\n> index 7da9883a8380..6f1235587a40 100644\n> --- a/Documentation/environment_variables.rst\n> +++ b/Documentation/environment_variables.rst\n> @@ -57,8 +57,8 @@ LIBCAMERA_RPI_CONFIG_FILE\n> \n> Example value: ``/usr/local/share/libcamera/pipeline/rpi/vc4/minimal_mem.yaml``\n> \n> -LIBCAMERA_RPI_TUNING_FILE\n> - Define a custom JSON tuning file to use in the Raspberry Pi.\n> +LIBCAMERA__TUNING_FILE\n> + Define a custom IPA tuning file to use with the pipeline handler `NAME`.\n> \n> Example value: ``/usr/local/share/libcamera/ipa/rpi/vc4/custom_sensor.json``\n\nNot directly related to this patch but anyways. This variable is quite\nuseful while developing, but fails if there are multiple sensors\nattached to the device. Should we add a LIBCAMERA__TUNING_DIR to\nadd the ability to run multiple sensors with custom tuning files?\n\n> \n> diff --git a/src/libcamera/ipa_proxy.cpp b/src/libcamera/ipa_proxy.cpp\n> index 85004737c171..25f772a41bf8 100644\n> --- a/src/libcamera/ipa_proxy.cpp\n> +++ b/src/libcamera/ipa_proxy.cpp\n> @@ -98,16 +98,33 @@ IPAProxy::~IPAProxy()\n> std::string IPAProxy::configurationFile(const std::string &name,\n> \t\t\t\t\tconst std::string &fallbackName) const\n> {\n> -\tstruct stat statbuf;\n> -\tint ret;\n> -\n> \t/*\n> \t * The IPA module name can be used as-is to build directory names as it\n> \t * has been validated when loading the module.\n> \t */\n> -\tstd::string ipaName = ipam_->info().name;\n> +\tconst std::string ipaName = ipam_->info().name;\n\nShould this be a string_view?\n\n> \n> -\t/* Check the environment variable first. */\n> +\t/*\n> +\t * Start with any user override through the module-specific environment\n> +\t * variable. Use the name of the IPA module up to the first '/' to\n> +\t * construct the variable name.\n> +\t */\n> +\tstd::string ipaEnvName = ipaName.substr(0, ipaName.find('/'));\n> +\tstd::transform(ipaEnvName.begin(), ipaEnvName.end(), ipaEnvName.begin(),\n> +\t\t [](unsigned char c) { return std::toupper(c); });\n\nSometimes c++ is really awful :-). Googling for this revealed\n\nfor (auto & c: ipaEnvName) c = std::toupper(c);\n\nas short alternative.\n\n> +\tipaEnvName = \"LIBCAMERA_\" + ipaEnvName + \"_TUNING_FILE\";\n> +\n> +\tchar const *configFromEnv = utils::secure_getenv(ipaEnvName.c_str());\n> +\tif (configFromEnv && *configFromEnv == '\\0')\n> +\t\treturn { configFromEnv };\n> +\n> +\tstruct stat statbuf;\n> +\tint ret;\n> +\n> +\t/*\n> +\t * Check the directory pointed to by the IPA config path environment\n> +\t * variable next.\n> +\t */\n> \tconst char *confPaths = utils::secure_getenv(\"LIBCAMERA_IPA_CONFIG_PATH\");\n> \tif (confPaths) {\n> \t\tfor (const auto &dir : utils::split(confPaths, \":\")) {\n> diff --git a/src/libcamera/pipeline/rkisp1/rkisp1.cpp b/src/libcamera/pipeline/rkisp1/rkisp1.cpp\n> index 35c793da9bba..1ac8d8ae7ed9 100644\n> --- a/src/libcamera/pipeline/rkisp1/rkisp1.cpp\n> +++ b/src/libcamera/pipeline/rkisp1/rkisp1.cpp\n> @@ -380,18 +380,9 @@ int RkISP1CameraData::loadIPA(unsigned int hwRevision)\n> \tipa_->paramsComputed.connect(this, &RkISP1CameraData::paramsComputed);\n> \tipa_->metadataReady.connect(this, &RkISP1CameraData::metadataReady);\n> \n> -\t/*\n> -\t * The API tuning file is made from the sensor name unless the\n> -\t * environment variable overrides it.\n> -\t */\n> -\tstd::string ipaTuningFile;\n> -\tchar const *configFromEnv = utils::secure_getenv(\"LIBCAMERA_RKISP1_TUNING_FILE\");\n> -\tif (!configFromEnv || *configFromEnv == '\\0') {\n> -\t\tipaTuningFile =\n> -\t\t\tipa_->configurationFile(sensor_->model() + \".yaml\", \"uncalibrated.yaml\");\n> -\t} else {\n> -\t\tipaTuningFile = std::string(configFromEnv);\n> -\t}\n> +\t/* The IPA tuning file is made from the sensor name. */\n> +\tstd::string ipaTuningFile =\n> +\t\tipa_->configurationFile(sensor_->model() + \".yaml\", \"uncalibrated.yaml\");\n\nOh I like this file getting shorter.\n\n> \n> \tIPACameraSensorInfo sensorInfo{};\n> \tint ret = sensor_->sensorInfo(&sensorInfo);\n> diff --git a/src/libcamera/pipeline/rpi/common/pipeline_base.cpp b/src/libcamera/pipeline/rpi/common/pipeline_base.cpp\n> index 4b147fdb379a..1f13e5230fae 100644\n> --- a/src/libcamera/pipeline/rpi/common/pipeline_base.cpp\n> +++ b/src/libcamera/pipeline/rpi/common/pipeline_base.cpp\n> @@ -1156,20 +1156,11 @@ int CameraData::loadIPA(ipa::RPi::InitResult *result)\n> \tif (!ipa_)\n> \t\treturn -ENOENT;\n> \n> -\t/*\n> -\t * The configuration (tuning file) is made from the sensor name unless\n> -\t * the environment variable overrides it.\n> -\t */\n> -\tstd::string configurationFile;\n> -\tchar const *configFromEnv = utils::secure_getenv(\"LIBCAMERA_RPI_TUNING_FILE\");\n> -\tif (!configFromEnv || *configFromEnv == '\\0') {\n> -\t\tstd::string model = sensor_->model();\n> -\t\tif (isMonoSensor(sensor_))\n> -\t\t\tmodel += \"_mono\";\n> -\t\tconfigurationFile = ipa_->configurationFile(model + \".json\");\n> -\t} else {\n> -\t\tconfigurationFile = std::string(configFromEnv);\n> -\t}\n> +\t/* The configuration (tuning file) is made from the sensor name. */\n> +\tstd::string model = sensor_->model();\n> +\tif (isMonoSensor(sensor_))\n> +\t\tmodel += \"_mono\";\n> +\tstd::string configurationFile = ipa_->configurationFile(model + \".json\");\n> \n> \tIPASettings settings(configurationFile, sensor_->model());\n> \tipa::RPi::InitParams params;\n\nThe other comments were nits, so\nReviewed-by: Stefan Klug \n\nCheers,\nStefan\n\n> \n> base-commit: fdc01dc3e0969423d65ee87f118164ec74373e5d\n> --\n> Regards,\n> \n> Laurent Pinchart\n>","headers":{"Return-Path":"","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id AE364C322E\n\tfor ;\n\tFri, 24 Jan 2025 18:19:53 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id D044D6855D;\n\tFri, 24 Jan 2025 19:19:52 +0100 (CET)","from perceval.ideasonboard.com (perceval.ideasonboard.com\n\t[IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id 4763A68556\n\tfor ;\n\tFri, 24 Jan 2025 19:19:51 +0100 (CET)","from ideasonboard.com (unknown\n\t[IPv6:2a01:599:702:3519:f8c4:a06b:f1d0:c7ac])\n\tby perceval.ideasonboard.com (Postfix) with ESMTPSA id 9C77721C;\n\tFri, 24 Jan 2025 19:18:46 +0100 (CET)"],"Authentication-Results":"lancelot.ideasonboard.com; dkim=pass (1024-bit key;\n\tunprotected) header.d=ideasonboard.com header.i=@ideasonboard.com\n\theader.b=\"A3ME2EAp\"; dkim-atps=neutral","DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com;\n\ts=mail; t=1737742726;\n\tbh=33RzA7XWG68pqNAo7/l0jNYZC3IlvZgdISXzheABSTk=;\n\th=Date:From:To:Cc:Subject:References:In-Reply-To:From;\n\tb=A3ME2EApMeopObghc78PW7uYoqEHKm6a8Bc7a0rOIQUdeN3Bv/a/fYmRwRvya8g+H\n\tkDL0cP86w373HSf8ThnbqK3L3a1wDdRiOyZk9nkn2uAUPYpIj+5G+RO3/04PWrca8u\n\trJMD2FW+1bH/R930ZJ74256a1XaKWJ5XCFXCnJsM=","Date":"Fri, 24 Jan 2025 19:19:47 +0100","From":"Stefan Klug ","To":"Laurent Pinchart ","Cc":"libcamera-devel@lists.libcamera.org","Subject":"Re: [PATCH] libcamera: pipeline: Move tuning file override handling\n\tto IPAProxy","Message-ID":"","References":"<20250123143818.29703-1-laurent.pinchart@ideasonboard.com>","MIME-Version":"1.0","Content-Type":"text/plain; charset=utf-8","Content-Disposition":"inline","In-Reply-To":"<20250123143818.29703-1-laurent.pinchart@ideasonboard.com>","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" "}},{"id":33168,"web_url":"https://patchwork.libcamera.org/comment/33168/","msgid":"<20250124203317.GA1805@pendragon.ideasonboard.com>","date":"2025-01-24T20:33:17","subject":"Re: [PATCH] libcamera: pipeline: Move tuning file override handling\n\tto IPAProxy","submitter":{"id":2,"url":"https://patchwork.libcamera.org/api/people/2/","name":"Laurent Pinchart","email":"laurent.pinchart@ideasonboard.com"},"content":"Hi Stefan,\n\nOn Fri, Jan 24, 2025 at 07:19:47PM +0100, Stefan Klug wrote:\n> On Thu, Jan 23, 2025 at 04:38:18PM +0200, Laurent Pinchart wrote:\n> > The rkisp1 and rpi pipeline handlers duplicate code to handle the\n> > LIBCAMERA_RKISP1_TUNING_FILE and LIBCAMERA_RPI_TUNING_FILE environment\n> > variables that override tuning file selection. Move the common code to\n> > IPAProxy::configurationFile() to avoid the duplication, and make the\n> > feature available to all pipeline handlers with the same behaviour.\n> > \n> > Signed-off-by: Laurent Pinchart \n> > ---\n> > Documentation/environment_variables.rst | 4 +--\n> > src/libcamera/ipa_proxy.cpp | 27 +++++++++++++++----\n> > src/libcamera/pipeline/rkisp1/rkisp1.cpp | 15 +++--------\n> > .../pipeline/rpi/common/pipeline_base.cpp | 19 ++++---------\n> > 4 files changed, 32 insertions(+), 33 deletions(-)\n> > \n> > diff --git a/Documentation/environment_variables.rst b/Documentation/environment_variables.rst\n> > index 7da9883a8380..6f1235587a40 100644\n> > --- a/Documentation/environment_variables.rst\n> > +++ b/Documentation/environment_variables.rst\n> > @@ -57,8 +57,8 @@ LIBCAMERA_RPI_CONFIG_FILE\n> > \n> > Example value: ``/usr/local/share/libcamera/pipeline/rpi/vc4/minimal_mem.yaml``\n> > \n> > -LIBCAMERA_RPI_TUNING_FILE\n> > - Define a custom JSON tuning file to use in the Raspberry Pi.\n> > +LIBCAMERA__TUNING_FILE\n> > + Define a custom IPA tuning file to use with the pipeline handler `NAME`.\n> > \n> > Example value: ``/usr/local/share/libcamera/ipa/rpi/vc4/custom_sensor.json``\n> \n> Not directly related to this patch but anyways. This variable is quite\n> useful while developing, but fails if there are multiple sensors\n> attached to the device. Should we add a LIBCAMERA__TUNING_DIR to\n> add the ability to run multiple sensors with custom tuning files?\n\nWe could, but I think I would prefer exploring a configuration file\ninstead of environment variables.\n\n> > diff --git a/src/libcamera/ipa_proxy.cpp b/src/libcamera/ipa_proxy.cpp\n> > index 85004737c171..25f772a41bf8 100644\n> > --- a/src/libcamera/ipa_proxy.cpp\n> > +++ b/src/libcamera/ipa_proxy.cpp\n> > @@ -98,16 +98,33 @@ IPAProxy::~IPAProxy()\n> > std::string IPAProxy::configurationFile(const std::string &name,\n> > \t\t\t\t\tconst std::string &fallbackName) const\n> > {\n> > -\tstruct stat statbuf;\n> > -\tint ret;\n> > -\n> > \t/*\n> > \t * The IPA module name can be used as-is to build directory names as it\n> > \t * has been validated when loading the module.\n> > \t */\n> > -\tstd::string ipaName = ipam_->info().name;\n> > +\tconst std::string ipaName = ipam_->info().name;\n> \n> Should this be a string_view?\n\nWe can't, because the operators that concatenate a std::string and a\nstd::string_view have only been added in C++20 :-( See\nhttps://en.cppreference.com/w/cpp/string/basic_string/operator%2B.\nWorkarounds would be possible by defining our own operators (I actually\nhave a patch that does just that), but that's out of scope for this\npatch.\n\n> > -\t/* Check the environment variable first. */\n> > +\t/*\n> > +\t * Start with any user override through the module-specific environment\n> > +\t * variable. Use the name of the IPA module up to the first '/' to\n> > +\t * construct the variable name.\n> > +\t */\n> > +\tstd::string ipaEnvName = ipaName.substr(0, ipaName.find('/'));\n> > +\tstd::transform(ipaEnvName.begin(), ipaEnvName.end(), ipaEnvName.begin(),\n> > +\t\t [](unsigned char c) { return std::toupper(c); });\n> \n> Sometimes c++ is really awful :-). Googling for this revealed\n> \n> for (auto & c: ipaEnvName) c = std::toupper(c);\n> \n> as short alternative.\n\nI kind of like std::transform(), but it may be a case of Stockholm\nsyndrome :-)\n\n> > +\tipaEnvName = \"LIBCAMERA_\" + ipaEnvName + \"_TUNING_FILE\";\n> > +\n> > +\tchar const *configFromEnv = utils::secure_getenv(ipaEnvName.c_str());\n> > +\tif (configFromEnv && *configFromEnv == '\\0')\n> > +\t\treturn { configFromEnv };\n> > +\n> > +\tstruct stat statbuf;\n> > +\tint ret;\n> > +\n> > +\t/*\n> > +\t * Check the directory pointed to by the IPA config path environment\n> > +\t * variable next.\n> > +\t */\n> > \tconst char *confPaths = utils::secure_getenv(\"LIBCAMERA_IPA_CONFIG_PATH\");\n> > \tif (confPaths) {\n> > \t\tfor (const auto &dir : utils::split(confPaths, \":\")) {\n> > diff --git a/src/libcamera/pipeline/rkisp1/rkisp1.cpp b/src/libcamera/pipeline/rkisp1/rkisp1.cpp\n> > index 35c793da9bba..1ac8d8ae7ed9 100644\n> > --- a/src/libcamera/pipeline/rkisp1/rkisp1.cpp\n> > +++ b/src/libcamera/pipeline/rkisp1/rkisp1.cpp\n> > @@ -380,18 +380,9 @@ int RkISP1CameraData::loadIPA(unsigned int hwRevision)\n> > \tipa_->paramsComputed.connect(this, &RkISP1CameraData::paramsComputed);\n> > \tipa_->metadataReady.connect(this, &RkISP1CameraData::metadataReady);\n> > \n> > -\t/*\n> > -\t * The API tuning file is made from the sensor name unless the\n> > -\t * environment variable overrides it.\n> > -\t */\n> > -\tstd::string ipaTuningFile;\n> > -\tchar const *configFromEnv = utils::secure_getenv(\"LIBCAMERA_RKISP1_TUNING_FILE\");\n> > -\tif (!configFromEnv || *configFromEnv == '\\0') {\n> > -\t\tipaTuningFile =\n> > -\t\t\tipa_->configurationFile(sensor_->model() + \".yaml\", \"uncalibrated.yaml\");\n> > -\t} else {\n> > -\t\tipaTuningFile = std::string(configFromEnv);\n> > -\t}\n> > +\t/* The IPA tuning file is made from the sensor name. */\n> > +\tstd::string ipaTuningFile =\n> > +\t\tipa_->configurationFile(sensor_->model() + \".yaml\", \"uncalibrated.yaml\");\n> \n> Oh I like this file getting shorter.\n> \n> > \n> > \tIPACameraSensorInfo sensorInfo{};\n> > \tint ret = sensor_->sensorInfo(&sensorInfo);\n> > diff --git a/src/libcamera/pipeline/rpi/common/pipeline_base.cpp b/src/libcamera/pipeline/rpi/common/pipeline_base.cpp\n> > index 4b147fdb379a..1f13e5230fae 100644\n> > --- a/src/libcamera/pipeline/rpi/common/pipeline_base.cpp\n> > +++ b/src/libcamera/pipeline/rpi/common/pipeline_base.cpp\n> > @@ -1156,20 +1156,11 @@ int CameraData::loadIPA(ipa::RPi::InitResult *result)\n> > \tif (!ipa_)\n> > \t\treturn -ENOENT;\n> > \n> > -\t/*\n> > -\t * The configuration (tuning file) is made from the sensor name unless\n> > -\t * the environment variable overrides it.\n> > -\t */\n> > -\tstd::string configurationFile;\n> > -\tchar const *configFromEnv = utils::secure_getenv(\"LIBCAMERA_RPI_TUNING_FILE\");\n> > -\tif (!configFromEnv || *configFromEnv == '\\0') {\n> > -\t\tstd::string model = sensor_->model();\n> > -\t\tif (isMonoSensor(sensor_))\n> > -\t\t\tmodel += \"_mono\";\n> > -\t\tconfigurationFile = ipa_->configurationFile(model + \".json\");\n> > -\t} else {\n> > -\t\tconfigurationFile = std::string(configFromEnv);\n> > -\t}\n> > +\t/* The configuration (tuning file) is made from the sensor name. */\n> > +\tstd::string model = sensor_->model();\n> > +\tif (isMonoSensor(sensor_))\n> > +\t\tmodel += \"_mono\";\n> > +\tstd::string configurationFile = ipa_->configurationFile(model + \".json\");\n> > \n> > \tIPASettings settings(configurationFile, sensor_->model());\n> > \tipa::RPi::InitParams params;\n> \n> The other comments were nits, so\n> Reviewed-by: Stefan Klug \n> \n> > base-commit: fdc01dc3e0969423d65ee87f118164ec74373e5d","headers":{"Return-Path":"","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id B5D28C31E9\n\tfor ;\n\tFri, 24 Jan 2025 20:33:30 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id D69CE6855D;\n\tFri, 24 Jan 2025 21:33:29 +0100 (CET)","from perceval.ideasonboard.com (perceval.ideasonboard.com\n\t[IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id 4349968556\n\tfor ;\n\tFri, 24 Jan 2025 21:33:28 +0100 (CET)","from pendragon.ideasonboard.com (81-175-209-231.bb.dnainternet.fi\n\t[81.175.209.231])\n\tby perceval.ideasonboard.com (Postfix) with ESMTPSA id 8B90521C;\n\tFri, 24 Jan 2025 21:32:23 +0100 (CET)"],"Authentication-Results":"lancelot.ideasonboard.com; dkim=pass (1024-bit key;\n\tunprotected) header.d=ideasonboard.com header.i=@ideasonboard.com\n\theader.b=\"QZiZ+Pdj\"; dkim-atps=neutral","DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com;\n\ts=mail; t=1737750743;\n\tbh=mfafXSbKpnDlT3DWPBvDRwq4fhkoymPcW+/IH961hsE=;\n\th=Date:From:To:Cc:Subject:References:In-Reply-To:From;\n\tb=QZiZ+PdjAty7n7bqf+5A1fwutLX5OvGP1aMAxpzdVPI2kppra1Y+yRQtAbtwFO5KJ\n\tu5gXVsHA9s06trq0ddRjlLiqWzRqwMWxDPpUfC/D3ZLMtG/iyoaD6n/Pi5wAKNJ8E3\n\tzjZIB4PdFrmMtVcbFVyOWv95pQSd+22nDm5palp0=","Date":"Fri, 24 Jan 2025 22:33:17 +0200","From":"Laurent Pinchart ","To":"Stefan Klug ","Cc":"libcamera-devel@lists.libcamera.org","Subject":"Re: [PATCH] libcamera: pipeline: Move tuning file override handling\n\tto IPAProxy","Message-ID":"<20250124203317.GA1805@pendragon.ideasonboard.com>","References":"<20250123143818.29703-1-laurent.pinchart@ideasonboard.com>\n\t","MIME-Version":"1.0","Content-Type":"text/plain; charset=utf-8","Content-Disposition":"inline","In-Reply-To":"","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" "}},{"id":33171,"web_url":"https://patchwork.libcamera.org/comment/33171/","msgid":"<173779233076.3973026.16250099092103054384@ping.linuxembedded.co.uk>","date":"2025-01-25T08:05:30","subject":"Re: [PATCH] libcamera: pipeline: Move tuning file override handling\n\tto IPAProxy","submitter":{"id":4,"url":"https://patchwork.libcamera.org/api/people/4/","name":"Kieran Bingham","email":"kieran.bingham@ideasonboard.com"},"content":"Quoting Laurent Pinchart (2025-01-24 20:33:17)\n> Hi Stefan,\n> \n> On Fri, Jan 24, 2025 at 07:19:47PM +0100, Stefan Klug wrote:\n> > On Thu, Jan 23, 2025 at 04:38:18PM +0200, Laurent Pinchart wrote:\n> > > The rkisp1 and rpi pipeline handlers duplicate code to handle the\n> > > LIBCAMERA_RKISP1_TUNING_FILE and LIBCAMERA_RPI_TUNING_FILE environment\n> > > variables that override tuning file selection. Move the common code to\n> > > IPAProxy::configurationFile() to avoid the duplication, and make the\n> > > feature available to all pipeline handlers with the same behaviour.\n> > > \n\nCentralising all this was going to be my review comment to Stefan's\npatch, \n\n> > > Signed-off-by: Laurent Pinchart \n> > > ---\n> > > Documentation/environment_variables.rst | 4 +--\n> > > src/libcamera/ipa_proxy.cpp | 27 +++++++++++++++----\n> > > src/libcamera/pipeline/rkisp1/rkisp1.cpp | 15 +++--------\n> > > .../pipeline/rpi/common/pipeline_base.cpp | 19 ++++---------\n> > > 4 files changed, 32 insertions(+), 33 deletions(-)\n> > > \n> > > diff --git a/Documentation/environment_variables.rst b/Documentation/environment_variables.rst\n> > > index 7da9883a8380..6f1235587a40 100644\n> > > --- a/Documentation/environment_variables.rst\n> > > +++ b/Documentation/environment_variables.rst\n> > > @@ -57,8 +57,8 @@ LIBCAMERA_RPI_CONFIG_FILE\n> > > \n> > > Example value: ``/usr/local/share/libcamera/pipeline/rpi/vc4/minimal_mem.yaml``\n> > > \n> > > -LIBCAMERA_RPI_TUNING_FILE\n> > > - Define a custom JSON tuning file to use in the Raspberry Pi.\n> > > +LIBCAMERA__TUNING_FILE\n> > > + Define a custom IPA tuning file to use with the pipeline handler `NAME`.\n> > > \n> > > Example value: ``/usr/local/share/libcamera/ipa/rpi/vc4/custom_sensor.json``\n> > \n> > Not directly related to this patch but anyways. This variable is quite\n> > useful while developing, but fails if there are multiple sensors\n> > attached to the device. Should we add a LIBCAMERA__TUNING_DIR to\n> > add the ability to run multiple sensors with custom tuning files?\n> \n> We could, but I think I would prefer exploring a configuration file\n> instead of environment variables.\n> \n> > > diff --git a/src/libcamera/ipa_proxy.cpp b/src/libcamera/ipa_proxy.cpp\n> > > index 85004737c171..25f772a41bf8 100644\n> > > --- a/src/libcamera/ipa_proxy.cpp\n> > > +++ b/src/libcamera/ipa_proxy.cpp\n> > > @@ -98,16 +98,33 @@ IPAProxy::~IPAProxy()\n> > > std::string IPAProxy::configurationFile(const std::string &name,\n> > > const std::string &fallbackName) const\n> > > {\n> > > - struct stat statbuf;\n> > > - int ret;\n> > > -\n> > > /*\n> > > * The IPA module name can be used as-is to build directory names as it\n> > > * has been validated when loading the module.\n> > > */\n> > > - std::string ipaName = ipam_->info().name;\n> > > + const std::string ipaName = ipam_->info().name;\n> > \n> > Should this be a string_view?\n> \n> We can't, because the operators that concatenate a std::string and a\n> std::string_view have only been added in C++20 :-( See\n> https://en.cppreference.com/w/cpp/string/basic_string/operator%2B.\n> Workarounds would be possible by defining our own operators (I actually\n> have a patch that does just that), but that's out of scope for this\n> patch.\n> \n> > > - /* Check the environment variable first. */\n> > > + /*\n> > > + * Start with any user override through the module-specific environment\n> > > + * variable. Use the name of the IPA module up to the first '/' to\n> > > + * construct the variable name.\n> > > + */\n> > > + std::string ipaEnvName = ipaName.substr(0, ipaName.find('/'));\n> > > + std::transform(ipaEnvName.begin(), ipaEnvName.end(), ipaEnvName.begin(),\n> > > + [](unsigned char c) { return std::toupper(c); });\n> > \n> > Sometimes c++ is really awful :-). Googling for this revealed\n> > \n> > for (auto & c: ipaEnvName) c = std::toupper(c);\n> > \n> > as short alternative.\n> \n> I kind of like std::transform(), but it may be a case of Stockholm\n> syndrome :-)\n\nI find almost every use of std::transform in libcamera hard to read :_(\n\nbut whatever works,\n\n\nReviewed-by: Kieran Bingham \n\n> \n> > > + ipaEnvName = \"LIBCAMERA_\" + ipaEnvName + \"_TUNING_FILE\";\n> > > +\n> > > + char const *configFromEnv = utils::secure_getenv(ipaEnvName.c_str());\n> > > + if (configFromEnv && *configFromEnv == '\\0')\n> > > + return { configFromEnv };\n> > > +\n> > > + struct stat statbuf;\n> > > + int ret;\n> > > +\n> > > + /*\n> > > + * Check the directory pointed to by the IPA config path environment\n> > > + * variable next.\n> > > + */\n> > > const char *confPaths = utils::secure_getenv(\"LIBCAMERA_IPA_CONFIG_PATH\");\n> > > if (confPaths) {\n> > > for (const auto &dir : utils::split(confPaths, \":\")) {\n> > > diff --git a/src/libcamera/pipeline/rkisp1/rkisp1.cpp b/src/libcamera/pipeline/rkisp1/rkisp1.cpp\n> > > index 35c793da9bba..1ac8d8ae7ed9 100644\n> > > --- a/src/libcamera/pipeline/rkisp1/rkisp1.cpp\n> > > +++ b/src/libcamera/pipeline/rkisp1/rkisp1.cpp\n> > > @@ -380,18 +380,9 @@ int RkISP1CameraData::loadIPA(unsigned int hwRevision)\n> > > ipa_->paramsComputed.connect(this, &RkISP1CameraData::paramsComputed);\n> > > ipa_->metadataReady.connect(this, &RkISP1CameraData::metadataReady);\n> > > \n> > > - /*\n> > > - * The API tuning file is made from the sensor name unless the\n> > > - * environment variable overrides it.\n> > > - */\n> > > - std::string ipaTuningFile;\n> > > - char const *configFromEnv = utils::secure_getenv(\"LIBCAMERA_RKISP1_TUNING_FILE\");\n> > > - if (!configFromEnv || *configFromEnv == '\\0') {\n> > > - ipaTuningFile =\n> > > - ipa_->configurationFile(sensor_->model() + \".yaml\", \"uncalibrated.yaml\");\n> > > - } else {\n> > > - ipaTuningFile = std::string(configFromEnv);\n> > > - }\n> > > + /* The IPA tuning file is made from the sensor name. */\n> > > + std::string ipaTuningFile =\n> > > + ipa_->configurationFile(sensor_->model() + \".yaml\", \"uncalibrated.yaml\");\n> > \n> > Oh I like this file getting shorter.\n> > \n> > > \n> > > IPACameraSensorInfo sensorInfo{};\n> > > int ret = sensor_->sensorInfo(&sensorInfo);\n> > > diff --git a/src/libcamera/pipeline/rpi/common/pipeline_base.cpp b/src/libcamera/pipeline/rpi/common/pipeline_base.cpp\n> > > index 4b147fdb379a..1f13e5230fae 100644\n> > > --- a/src/libcamera/pipeline/rpi/common/pipeline_base.cpp\n> > > +++ b/src/libcamera/pipeline/rpi/common/pipeline_base.cpp\n> > > @@ -1156,20 +1156,11 @@ int CameraData::loadIPA(ipa::RPi::InitResult *result)\n> > > if (!ipa_)\n> > > return -ENOENT;\n> > > \n> > > - /*\n> > > - * The configuration (tuning file) is made from the sensor name unless\n> > > - * the environment variable overrides it.\n> > > - */\n> > > - std::string configurationFile;\n> > > - char const *configFromEnv = utils::secure_getenv(\"LIBCAMERA_RPI_TUNING_FILE\");\n> > > - if (!configFromEnv || *configFromEnv == '\\0') {\n> > > - std::string model = sensor_->model();\n> > > - if (isMonoSensor(sensor_))\n> > > - model += \"_mono\";\n> > > - configurationFile = ipa_->configurationFile(model + \".json\");\n> > > - } else {\n> > > - configurationFile = std::string(configFromEnv);\n> > > - }\n> > > + /* The configuration (tuning file) is made from the sensor name. */\n> > > + std::string model = sensor_->model();\n> > > + if (isMonoSensor(sensor_))\n> > > + model += \"_mono\";\n> > > + std::string configurationFile = ipa_->configurationFile(model + \".json\");\n> > > \n> > > IPASettings settings(configurationFile, sensor_->model());\n> > > ipa::RPi::InitParams params;\n> > \n> > The other comments were nits, so\n> > Reviewed-by: Stefan Klug \n> > \n> > > base-commit: fdc01dc3e0969423d65ee87f118164ec74373e5d\n> \n> -- \n> Regards,\n> \n> Laurent Pinchart","headers":{"Return-Path":"","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id C65ADC31E9\n\tfor ;\n\tSat, 25 Jan 2025 08:05:36 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id A2AF56855D;\n\tSat, 25 Jan 2025 09:05:35 +0100 (CET)","from perceval.ideasonboard.com (perceval.ideasonboard.com\n\t[IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id DD31468549\n\tfor ;\n\tSat, 25 Jan 2025 09:05:33 +0100 (CET)","from pendragon.ideasonboard.com\n\t(cpc89244-aztw30-2-0-cust6594.18-1.cable.virginm.net [86.31.185.195])\n\tby perceval.ideasonboard.com (Postfix) with ESMTPSA id D51B61E3;\n\tSat, 25 Jan 2025 09:04:28 +0100 (CET)"],"Authentication-Results":"lancelot.ideasonboard.com; dkim=pass (1024-bit key;\n\tunprotected) header.d=ideasonboard.com header.i=@ideasonboard.com\n\theader.b=\"OJwVHXnh\"; dkim-atps=neutral","DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com;\n\ts=mail; t=1737792268;\n\tbh=6mG0pjw9siEcdUGQ1TV9mq/5cipqDlRU3m+OzSgyU58=;\n\th=In-Reply-To:References:Subject:From:Cc:To:Date:From;\n\tb=OJwVHXnh0C++7lFaje9XKicqZuufe6oGxTblnpC8zbYNpkgfzB1Stuwma3KiqaNTP\n\tC1bjbmK7xikFPpstpkaZPv5nOv3dBlsFD5fPKpTsvovMiGaouC1Svbw2wMIjF5GSku\n\tl0LcOQXInMawZHjOynzSmOHCUvQa/wSi4uWbBBmY=","Content-Type":"text/plain; charset=\"utf-8\"","MIME-Version":"1.0","Content-Transfer-Encoding":"quoted-printable","In-Reply-To":"<20250124203317.GA1805@pendragon.ideasonboard.com>","References":"<20250123143818.29703-1-laurent.pinchart@ideasonboard.com>\n\t\n\t<20250124203317.GA1805@pendragon.ideasonboard.com>","Subject":"Re: [PATCH] libcamera: pipeline: Move tuning file override handling\n\tto IPAProxy","From":"Kieran Bingham ","Cc":"libcamera-devel@lists.libcamera.org","To":"Laurent Pinchart ,\n\tStefan Klug ","Date":"Sat, 25 Jan 2025 08:05:30 +0000","Message-ID":"<173779233076.3973026.16250099092103054384@ping.linuxembedded.co.uk>","User-Agent":"alot/0.10","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" "}}]