[{"id":36724,"web_url":"https://patchwork.libcamera.org/comment/36724/","msgid":"<176242942679.2116251.2612016294684019248@neptunite.rasen.tech>","date":"2025-11-06T11:43:46","subject":"Re: [PATCH v2 26/35] pipeline: rkisp1: Enable the dewarper based on\n\tthe tuning file","submitter":{"id":17,"url":"https://patchwork.libcamera.org/api/people/17/","name":"Paul Elder","email":"paul.elder@ideasonboard.com"},"content":"Quoting Stefan Klug (2025-10-23 23:48:27)\n> To do actual lens dewarping, the dewarper will be configured based on\n> the tuning file. As a first step implement the basic loading of the\n> tuning file and enable/disable the dewarper for the given camera based\n> on the existence of the \"Dewarp\" algorithm in the tuning file.\n> \n> Todo: This is an backwards incompatible change in that the dewarper is\n> currently included in the chain unconditionally. Some users may want to\n> not use the dewarper, so it is sensible to make that configurable. If it\n> should be in the algorithms section or in a different one is open for\n> debate.\n\nIt indeed does not feel like an algorithm; it's part of the pipeline that\nimplements ScalerCrop... although I suppose in the future it will do actual\ndewarping and then it will become an algorithm. Thus I think it's fine to put\nin the algorithms section.\n\nAs for backwards compatibility, I would imagine that most users would want the\ndewarper on, but so do they other algorithms that need to be specified in the\ntuning file to be enabled. Therefore I think it's fine do be backwards\nincompatible. It's debatable whether or not it's an API change (I would think\nnot). If you're worried about users complaining we could add a warning for a\ncouple versions?\n\n> \n> Signed-off-by: Stefan Klug \n\nI'm not going to bikeshed variable names; they're good enough :)\n\nReviewed-by: Paul Elder \n\n> \n> ---\n> \n> Changes in v2:\n> - Drop unused params variable\n> - Moved patch a bit earlier so canUseDewarper is available for handling\n> the orientation\n> ---\n> src/ipa/libipa/module.h | 4 ++\n> src/libcamera/pipeline/rkisp1/rkisp1.cpp | 48 +++++++++++++++++++++++-\n> 2 files changed, 51 insertions(+), 1 deletion(-)\n> \n> diff --git a/src/ipa/libipa/module.h b/src/ipa/libipa/module.h\n> index 0fb51916fff6..84386f901534 100644\n> --- a/src/ipa/libipa/module.h\n> +++ b/src/ipa/libipa/module.h\n> @@ -74,6 +74,10 @@ private:\n> int createAlgorithm(Context &context, const YamlObject &data)\n> {\n> const auto &[name, algoData] = *data.asDict().begin();\n> +\n> + if (name == \"Dewarp\")\n> + return 0;\n> +\n> std::unique_ptr> algo = createAlgorithm(name);\n> if (!algo) {\n> LOG(IPAModuleAlgo, Error)\n> diff --git a/src/libcamera/pipeline/rkisp1/rkisp1.cpp b/src/libcamera/pipeline/rkisp1/rkisp1.cpp\n> index d7bb30f20668..943f26ece974 100644\n> --- a/src/libcamera/pipeline/rkisp1/rkisp1.cpp\n> +++ b/src/libcamera/pipeline/rkisp1/rkisp1.cpp\n> @@ -16,6 +16,7 @@\n> #include \n> #include \n> \n> +#include \n> #include \n> #include \n> \n> @@ -47,6 +48,7 @@\n> #include \"libcamera/internal/v4l2_request.h\"\n> #include \"libcamera/internal/v4l2_subdevice.h\"\n> #include \"libcamera/internal/v4l2_videodevice.h\"\n> +#include \"libcamera/internal/yaml_parser.h\"\n> \n> #include \"rkisp1_path.h\"\n> \n> @@ -123,6 +125,7 @@ public:\n> */\n> MediaPipeline pipe_;\n> \n> + bool canUseDewarper_;\n> bool usesDewarper_;\n> \n> private:\n> @@ -131,6 +134,7 @@ private:\n> const ControlList &sensorControls);\n> \n> void metadataReady(unsigned int frame, const ControlList &metadata);\n> + int loadTuningFile(const std::string &file);\n> };\n> \n> class RkISP1CameraConfiguration : public CameraConfiguration\n> @@ -416,6 +420,48 @@ int RkISP1CameraData::loadIPA(unsigned int hwRevision, uint32_t supportedBlocks)\n> return ret;\n> }\n> \n> + ret = loadTuningFile(ipaTuningFile);\n> + if (ret < 0) {\n> + LOG(RkISP1, Error) << \"Failed to load tuning file\";\n> + return ret;\n> + }\n> +\n> + return 0;\n> +}\n> +\n> +int RkISP1CameraData::loadTuningFile(const std::string &path)\n> +{\n> + if (!pipe()->dewarper_)\n> + /* Nothing to do without dewarper */\n> + return 0;\n> +\n> + LOG(RkISP1, Debug) << \"Load tuning file \" << path;\n> +\n> + File file(path);\n> + if (!file.open(File::OpenModeFlag::ReadOnly)) {\n> + int ret = file.error();\n> + LOG(RkISP1, Error)\n> + << \"Failed to open tuning file \"\n> + << path << \": \" << strerror(-ret);\n> + return ret;\n> + }\n> +\n> + std::unique_ptr data = YamlParser::parse(file);\n> + if (!data)\n> + return -EINVAL;\n> +\n> + if (!data->contains(\"algorithms\")) {\n> + LOG(RkISP1, Error)\n> + << \"Tuning file doesn't contain any algorithm\";\n> + return -EINVAL;\n> + }\n> +\n> + const auto &algos = (*data)[\"algorithms\"].asList();\n> + for (const auto &algo : algos) {\n> + if (algo.contains(\"Dewarp\"))\n> + canUseDewarper_ = true;\n> + }\n> +\n> return 0;\n> }\n> \n> @@ -573,7 +619,7 @@ CameraConfiguration::Status RkISP1CameraConfiguration::validate()\n> }\n> \n> bool useDewarper = false;\n> - if (pipe->dewarper_ && !isRaw)\n> + if (data_->canUseDewarper_ && !isRaw)\n> useDewarper = true;\n> \n> /*\n> -- \n> 2.48.1\n>","headers":{"Return-Path":"","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id 167E5C3241\n\tfor ;\n\tThu, 6 Nov 2025 11:43:57 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id 2BB6960A80;\n\tThu, 6 Nov 2025 12:43:56 +0100 (CET)","from perceval.ideasonboard.com (perceval.ideasonboard.com\n\t[IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id 8C95C606E6\n\tfor ;\n\tThu, 6 Nov 2025 12:43:54 +0100 (CET)","from neptunite.rasen.tech (unknown\n\t[IPv6:2404:7a81:160:2100:d4d0:27ea:7a74:8a9e])\n\tby perceval.ideasonboard.com (Postfix) with UTF8SMTPSA id 10B2C6A6;\n\tThu, 6 Nov 2025 12:41:58 +0100 (CET)"],"Authentication-Results":"lancelot.ideasonboard.com; dkim=pass (1024-bit key;\n\tunprotected) header.d=ideasonboard.com header.i=@ideasonboard.com\n\theader.b=\"VI8Ki4YU\"; dkim-atps=neutral","DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com;\n\ts=mail; t=1762429319;\n\tbh=NhsaqtYkIp8wdyCkWdCI7TBIVfOn+SxDyiXtA1ZR12M=;\n\th=In-Reply-To:References:Subject:From:Cc:To:Date:From;\n\tb=VI8Ki4YU9qegtTZt8k9h44Lz/b1dwogJGokHS/AGuGriWuDtfVIDOVZQaOqmhspau\n\tXE/F/V4R2D2npLMAASHxKf9WQTXMRTyZQTtQCn1uapwyttB+vx5gddfafdKPlX6jFG\n\tlkmQncn3JfLCLPELLZO9Sec7GfQRmvZncf/jTxwM=","Content-Type":"text/plain; charset=\"utf-8\"","MIME-Version":"1.0","Content-Transfer-Encoding":"quoted-printable","In-Reply-To":"<20251023144841.403689-27-stefan.klug@ideasonboard.com>","References":"<20251023144841.403689-1-stefan.klug@ideasonboard.com>\n\t<20251023144841.403689-27-stefan.klug@ideasonboard.com>","Subject":"Re: [PATCH v2 26/35] pipeline: rkisp1: Enable the dewarper based on\n\tthe tuning file","From":"Paul Elder ","Cc":"Stefan Klug ","To":"Stefan Klug ,\n\tlibcamera-devel@lists.libcamera.org","Date":"Thu, 06 Nov 2025 20:43:46 +0900","Message-ID":"<176242942679.2116251.2612016294684019248@neptunite.rasen.tech>","User-Agent":"alot/0.0.0","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" "}},{"id":37049,"web_url":"https://patchwork.libcamera.org/comment/37049/","msgid":"<176408353576.1350675.13362093487168750865@localhost>","date":"2025-11-25T15:12:15","subject":"Re: [PATCH v2 26/35] pipeline: rkisp1: Enable the dewarper based on\n\tthe tuning file","submitter":{"id":184,"url":"https://patchwork.libcamera.org/api/people/184/","name":"Stefan Klug","email":"stefan.klug@ideasonboard.com"},"content":"Hi Paul,\n\nThank you for the review.\n\nQuoting Paul Elder (2025-11-06 12:43:46)\n> Quoting Stefan Klug (2025-10-23 23:48:27)\n> > To do actual lens dewarping, the dewarper will be configured based on\n> > the tuning file. As a first step implement the basic loading of the\n> > tuning file and enable/disable the dewarper for the given camera based\n> > on the existence of the \"Dewarp\" algorithm in the tuning file.\n> > \n> > Todo: This is an backwards incompatible change in that the dewarper is\n> > currently included in the chain unconditionally. Some users may want to\n> > not use the dewarper, so it is sensible to make that configurable. If it\n> > should be in the algorithms section or in a different one is open for\n> > debate.\n> \n> It indeed does not feel like an algorithm; it's part of the pipeline that\n> implements ScalerCrop... although I suppose in the future it will do actual\n> dewarping and then it will become an algorithm. Thus I think it's fine to put\n> in the algorithms section.\n> \n> As for backwards compatibility, I would imagine that most users would want the\n> dewarper on, but so do they other algorithms that need to be specified in the\n> tuning file to be enabled. Therefore I think it's fine do be backwards\n> incompatible. It's debatable whether or not it's an API change (I would think\n> not). If you're worried about users complaining we could add a warning for a\n> couple versions?\n\nAfter the internal discussion this has now found its place below the\nmodules key. So I hope everyone is happy with that.\n\n> \n> > \n> > Signed-off-by: Stefan Klug \n> \n> I'm not going to bikeshed variable names; they're good enough :)\n\nDo you have proposals? I guess you mean the canUseDwarper and\nusesDewarper pair. I'm not super excited about them either, so ideas are\nwelcome. On the other hand I will try to push changing them to after\nthe series :-)\n\n> \n> Reviewed-by: Paul Elder \n\nThanks. I kept your rby even though I had to rewrite bits of this. I\nhope this is fine.\n\nBest regards,\nStefan\n\n> \n> > \n> > ---\n> > \n> > Changes in v2:\n> > - Drop unused params variable\n> > - Moved patch a bit earlier so canUseDewarper is available for handling\n> > the orientation\n> > ---\n> > src/ipa/libipa/module.h | 4 ++\n> > src/libcamera/pipeline/rkisp1/rkisp1.cpp | 48 +++++++++++++++++++++++-\n> > 2 files changed, 51 insertions(+), 1 deletion(-)\n> > \n> > diff --git a/src/ipa/libipa/module.h b/src/ipa/libipa/module.h\n> > index 0fb51916fff6..84386f901534 100644\n> > --- a/src/ipa/libipa/module.h\n> > +++ b/src/ipa/libipa/module.h\n> > @@ -74,6 +74,10 @@ private:\n> > int createAlgorithm(Context &context, const YamlObject &data)\n> > {\n> > const auto &[name, algoData] = *data.asDict().begin();\n> > +\n> > + if (name == \"Dewarp\")\n> > + return 0;\n> > +\n> > std::unique_ptr> algo = createAlgorithm(name);\n> > if (!algo) {\n> > LOG(IPAModuleAlgo, Error)\n> > diff --git a/src/libcamera/pipeline/rkisp1/rkisp1.cpp b/src/libcamera/pipeline/rkisp1/rkisp1.cpp\n> > index d7bb30f20668..943f26ece974 100644\n> > --- a/src/libcamera/pipeline/rkisp1/rkisp1.cpp\n> > +++ b/src/libcamera/pipeline/rkisp1/rkisp1.cpp\n> > @@ -16,6 +16,7 @@\n> > #include \n> > #include \n> > \n> > +#include \n> > #include \n> > #include \n> > \n> > @@ -47,6 +48,7 @@\n> > #include \"libcamera/internal/v4l2_request.h\"\n> > #include \"libcamera/internal/v4l2_subdevice.h\"\n> > #include \"libcamera/internal/v4l2_videodevice.h\"\n> > +#include \"libcamera/internal/yaml_parser.h\"\n> > \n> > #include \"rkisp1_path.h\"\n> > \n> > @@ -123,6 +125,7 @@ public:\n> > */\n> > MediaPipeline pipe_;\n> > \n> > + bool canUseDewarper_;\n> > bool usesDewarper_;\n> > \n> > private:\n> > @@ -131,6 +134,7 @@ private:\n> > const ControlList &sensorControls);\n> > \n> > void metadataReady(unsigned int frame, const ControlList &metadata);\n> > + int loadTuningFile(const std::string &file);\n> > };\n> > \n> > class RkISP1CameraConfiguration : public CameraConfiguration\n> > @@ -416,6 +420,48 @@ int RkISP1CameraData::loadIPA(unsigned int hwRevision, uint32_t supportedBlocks)\n> > return ret;\n> > }\n> > \n> > + ret = loadTuningFile(ipaTuningFile);\n> > + if (ret < 0) {\n> > + LOG(RkISP1, Error) << \"Failed to load tuning file\";\n> > + return ret;\n> > + }\n> > +\n> > + return 0;\n> > +}\n> > +\n> > +int RkISP1CameraData::loadTuningFile(const std::string &path)\n> > +{\n> > + if (!pipe()->dewarper_)\n> > + /* Nothing to do without dewarper */\n> > + return 0;\n> > +\n> > + LOG(RkISP1, Debug) << \"Load tuning file \" << path;\n> > +\n> > + File file(path);\n> > + if (!file.open(File::OpenModeFlag::ReadOnly)) {\n> > + int ret = file.error();\n> > + LOG(RkISP1, Error)\n> > + << \"Failed to open tuning file \"\n> > + << path << \": \" << strerror(-ret);\n> > + return ret;\n> > + }\n> > +\n> > + std::unique_ptr data = YamlParser::parse(file);\n> > + if (!data)\n> > + return -EINVAL;\n> > +\n> > + if (!data->contains(\"algorithms\")) {\n> > + LOG(RkISP1, Error)\n> > + << \"Tuning file doesn't contain any algorithm\";\n> > + return -EINVAL;\n> > + }\n> > +\n> > + const auto &algos = (*data)[\"algorithms\"].asList();\n> > + for (const auto &algo : algos) {\n> > + if (algo.contains(\"Dewarp\"))\n> > + canUseDewarper_ = true;\n> > + }\n> > +\n> > return 0;\n> > }\n> > \n> > @@ -573,7 +619,7 @@ CameraConfiguration::Status RkISP1CameraConfiguration::validate()\n> > }\n> > \n> > bool useDewarper = false;\n> > - if (pipe->dewarper_ && !isRaw)\n> > + if (data_->canUseDewarper_ && !isRaw)\n> > useDewarper = true;\n> > \n> > /*\n> > -- \n> > 2.48.1\n> >","headers":{"Return-Path":"","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id 20453C3334\n\tfor ;\n\tTue, 25 Nov 2025 15:12:21 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id 474F360A85;\n\tTue, 25 Nov 2025 16:12:20 +0100 (CET)","from perceval.ideasonboard.com (perceval.ideasonboard.com\n\t[IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id AD4F2609D8\n\tfor ;\n\tTue, 25 Nov 2025 16:12:18 +0100 (CET)","from ideasonboard.com (unknown\n\t[IPv6:2a00:6020:448c:6c00:bae1:340c:573c:570b])\n\tby perceval.ideasonboard.com (Postfix) with UTF8SMTPSA id 6B3FB236;\n\tTue, 25 Nov 2025 16:10:09 +0100 (CET)"],"Authentication-Results":"lancelot.ideasonboard.com; dkim=pass (1024-bit key;\n\tunprotected) header.d=ideasonboard.com header.i=@ideasonboard.com\n\theader.b=\"Qmc9Xjay\"; dkim-atps=neutral","DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com;\n\ts=mail; t=1764083409;\n\tbh=+SOsHGIrmvWEZjjnNC308hQgiylC6UYCa42tnE8HG0w=;\n\th=In-Reply-To:References:Subject:From:Cc:To:Date:From;\n\tb=Qmc9Xjayd0xd0+WrkyD65X84pX2NVnMwG8tu0BAVnRzs2KZZMMoA8EOjD13J5GiaN\n\tagNYgeYP9qPZekjcqFmBg/Ms0cQRy/F9hZIjo8/TH9e/JYDl2uP8jLSK962txrMulF\n\tq1Pel61TfHn+34BfJiNFs2Urk+BaWbXs+OecnFJQ=","Content-Type":"text/plain; charset=\"utf-8\"","MIME-Version":"1.0","Content-Transfer-Encoding":"quoted-printable","In-Reply-To":"<176242942679.2116251.2612016294684019248@neptunite.rasen.tech>","References":"<20251023144841.403689-1-stefan.klug@ideasonboard.com>\n\t<20251023144841.403689-27-stefan.klug@ideasonboard.com>\n\t<176242942679.2116251.2612016294684019248@neptunite.rasen.tech>","Subject":"Re: [PATCH v2 26/35] pipeline: rkisp1: Enable the dewarper based on\n\tthe tuning file","From":"Stefan Klug ","Cc":"","To":"Paul Elder ,\n\tlibcamera-devel@lists.libcamera.org","Date":"Tue, 25 Nov 2025 16:12:15 +0100","Message-ID":"<176408353576.1350675.13362093487168750865@localhost>","User-Agent":"alot/0.12.dev8+g2c003385c862.d20250602","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" "}}]