From patchwork Thu Jun 18 10:18:40 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Jacopo Mondi X-Patchwork-Id: 26916 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 16C4CC328C for ; Thu, 18 Jun 2026 10:18:58 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id E6F10629A1; Thu, 18 Jun 2026 12:18:55 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="J5WakPjz"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 3F3C361754 for ; Thu, 18 Jun 2026 12:18:53 +0200 (CEST) Received: from [192.168.125.177] (mob-109-113-4-199.net.vodafone.it [109.113.4.199]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 832B5DF3; Thu, 18 Jun 2026 12:18:17 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1781777898; bh=WwkJjfrF6uYuCHNyMbb4OBFHOUidZxlMMtRfcHXwSLM=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=J5WakPjzxYmzs57OvZ0agjyd2yJTgNKkxAxEPVlxRCFJCp8VDHL4dYEKk1hnZq9LG 6vGt2wxVWVjb0RnpDwEz355010ar5gVxt5CPT/wrdujLBz8X6bR328J8gEgBDED1cK JuPNMoze4/YmbWL5lwXyg9d5zum2blfsRwsXfsww= From: Jacopo Mondi Date: Thu, 18 Jun 2026 12:18:40 +0200 Subject: [PATCH 01/14] libcamera: ipa_manager: Create IPA by name MIME-Version: 1.0 Message-Id: <20260618-rppx1-ipa-v1-1-32337264cfcd@ideasonboard.com> References: <20260618-rppx1-ipa-v1-0-32337264cfcd@ideasonboard.com> In-Reply-To: <20260618-rppx1-ipa-v1-0-32337264cfcd@ideasonboard.com> To: =?utf-8?q?Niklas_S=C3=B6derlund?= , libcamera-devel@lists.libcamera.org Cc: Jacopo Mondi , Hans de Goede , Kieran Bingham X-Mailer: b4 0.14.3 X-Developer-Signature: v=1; a=openpgp-sha256; l=8374; i=jacopo.mondi@ideasonboard.com; h=from:subject:message-id; bh=vuESyGRD9gC1WcSe588Ai3/ECLPgRNN9wbmW63B/y8U=; b=owEBbQKS/ZANAwAKAXI0Bo8WoVY8AcsmYgBqM8YJ492R/nzPIuuD+HrJxEyHuf4ydkH6R0+ld ntefURsa9uJAjMEAAEKAB0WIQS1xD1IgJogio9YOMByNAaPFqFWPAUCajPGCQAKCRByNAaPFqFW PBtlD/0WjS/gn5GpQrDn/rLmNWxPL2az+iWmgIEw74PosfcBJTk6w5hAbE9cI6fK14wgvBy/e+w 1K3KktK2BxC0ZWSIsHumLgSe036kX7/36rXex6WZuhnm9vop/e4XoOEYuaBOoWIeJCDNSq9A4sG 6jmj49oBUWinRJOpY1/L0h0yM/q2jyCi2RjGFqVjtkkZr61BwxEnx7/rsnbpoKPVzB6O47rUdFP 3zpJQo1tNGIKBHHZaEiVzmBMjMiq2bwYxmer0pokADqpm99esMw38FElNLtVYw4CJcqHMr0gLdT GhUAc3VRhB+8THXCnVj+I6tKeQNy68/wIHh5Sb884GRDJivM9c76axVafLkJXSusIFCPeu1uLmM tjVKWiuuSUmN8h2xyfMDK7OrXbNdWkxs556Ear7hXTL0CKiXpaVr1pnSb1z41J9KVAZ4S4WKZcc PkDNgMpDfh57UhBvS2Sc4W+iWmeszYCjEo9A/tgBbndzBVm92KTccIUpWfXwsWhj8fdC0IpCTIe Ra4sVkM+dB0jvAqlQ3qfSpYthvNWi7phJjySJFq1fs/e8H8PcZ+s5k0+tF6ROaGut7agCRmwvSQ vmlLNhjfxuS6Njpi5LTnnGEL9irP69apo3HBPT+tqcc0iAEeWtyLZyRBbolPPkbSPnJq8XAsGna Gck2cSGuIG6TL3g== X-Developer-Key: i=jacopo.mondi@ideasonboard.com; a=openpgp; fpr=72392EDC88144A65C701EA9BA5826A2587AD026B X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" From: Hans de Goede Currently createIPA() / IPAManager::module() assume that there is a 1:1 relationship between pipeline handlers and IPAs and IPA matching is done based on matching the pipe to ipaModuleInfo.pipelineName[]. One way to allow using a single IPA with multiple pipelines would be to allow the IPA to declare itself compatible with more than one pipeline, turning ipaModuleInfo.pipelineName[] into e.g. a vector. But the way ipaModuleInfo is loaded as an ELF symbol requires it to be a simple flat C-struct. Instead, move the IPA creation procedure to be name-based, introducing an overload to IPAManager::createIPA(pipe, name, minVer, maxVer) that allows to specify the name of the IPA module to match. Pipeline handlers that wants to use their name as matching criteria can continue doing so using the already existing createIPA(pipe, minVer, maxVer) overload. Signed-off-by: Hans de Goede Signed-off-by: Jacopo Mondi Reviewed-by: Hans de Goede Reviewed-by: Kieran Bingham Tested-by: Niklas Söderlund --- include/libcamera/internal/ipa_manager.h | 8 +++---- include/libcamera/internal/ipa_module.h | 4 ++-- include/libcamera/internal/pipeline_handler.h | 9 ++++++- src/libcamera/ipa_manager.cpp | 34 ++++++++++++++++++++++----- src/libcamera/ipa_module.cpp | 12 +++++----- 5 files changed, 48 insertions(+), 19 deletions(-) diff --git a/include/libcamera/internal/ipa_manager.h b/include/libcamera/internal/ipa_manager.h index aaa3ca37c493..b9825c38d34e 100644 --- a/include/libcamera/internal/ipa_manager.h +++ b/include/libcamera/internal/ipa_manager.h @@ -25,7 +25,6 @@ LOG_DECLARE_CATEGORY(IPAManager) class CameraManager; class GlobalConfiguration; class IPAModule; -class PipelineHandler; class IPAManager { @@ -34,10 +33,11 @@ public: ~IPAManager(); template - std::unique_ptr createIPA(PipelineHandler *pipe, uint32_t minVersion, + std::unique_ptr createIPA(const char *name, + uint32_t minVersion, uint32_t maxVersion) { - IPAModule *m = module(pipe, minVersion, maxVersion); + IPAModule *m = module(name, minVersion, maxVersion); if (!m) return nullptr; @@ -68,7 +68,7 @@ private: std::vector &files); unsigned int addDir(const char *libDir, unsigned int maxDepth = 0); - IPAModule *module(PipelineHandler *pipe, uint32_t minVersion, + IPAModule *module(const char *name, uint32_t minVersion, uint32_t maxVersion); bool isSignatureValid(IPAModule *ipa) const; diff --git a/include/libcamera/internal/ipa_module.h b/include/libcamera/internal/ipa_module.h index 15f19492c3a0..a0a53764e139 100644 --- a/include/libcamera/internal/ipa_module.h +++ b/include/libcamera/internal/ipa_module.h @@ -36,8 +36,8 @@ public: IPAInterface *createInterface(); - bool match(PipelineHandler *pipe, - uint32_t minVersion, uint32_t maxVersion) const; + bool match(const char *name, uint32_t minVersion, + uint32_t maxVersion) const; protected: std::string logPrefix() const override; diff --git a/include/libcamera/internal/pipeline_handler.h b/include/libcamera/internal/pipeline_handler.h index 6922ce18ec87..d255c149f2f3 100644 --- a/include/libcamera/internal/pipeline_handler.h +++ b/include/libcamera/internal/pipeline_handler.h @@ -76,7 +76,14 @@ public: std::unique_ptr createIPA(uint32_t minVersion, uint32_t maxVersion) { IPAManager *ipaManager = manager_->_d()->ipaManager(); - return ipaManager->createIPA(this, minVersion, maxVersion); + return ipaManager->createIPA(this->name(), minVersion, maxVersion); + } + + template + std::unique_ptr createIPA(const char *name, uint32_t minVersion, uint32_t maxVersion) + { + IPAManager *ipaManager = manager_->_d()->ipaManager(); + return ipaManager->createIPA(name, minVersion, maxVersion); } protected: diff --git a/src/libcamera/ipa_manager.cpp b/src/libcamera/ipa_manager.cpp index 41918e4c2934..e60e947049d5 100644 --- a/src/libcamera/ipa_manager.cpp +++ b/src/libcamera/ipa_manager.cpp @@ -248,15 +248,15 @@ unsigned int IPAManager::addDir(const char *libDir, unsigned int maxDepth) /** * \brief Retrieve an IPA module that matches a given pipeline handler - * \param[in] pipe The pipeline handler + * \param[in] name The IPA module string identifier * \param[in] minVersion Minimum acceptable version of IPA module * \param[in] maxVersion Maximum acceptable version of IPA module */ -IPAModule *IPAManager::module(PipelineHandler *pipe, uint32_t minVersion, +IPAModule *IPAManager::module(const char *name, uint32_t minVersion, uint32_t maxVersion) { for (const auto &module : modules_) { - if (module->match(pipe, minVersion, maxVersion)) + if (module->match(name, minVersion, maxVersion)) return module.get(); } @@ -264,12 +264,34 @@ IPAModule *IPAManager::module(PipelineHandler *pipe, uint32_t minVersion, } /** - * \fn IPAManager::createIPA() - * \brief Create an IPA proxy that matches a given pipeline handler - * \param[in] pipe The pipeline handler that wants a matching IPA proxy + * \fn IPAManager::createIPA(PipelineHandler *pipe, const char *ipaName, uint32_t minVersion, uint32_t maxVersion) + * \brief Create an IPA proxy that matches the requested name and version + * \param[in] pipe The pipeline handler that wants to create the IPA module + * \param[in] ipaName The IPA module name * \param[in] minVersion Minimum acceptable version of IPA module * \param[in] maxVersion Maximum acceptable version of IPA module * + * Create an IPA module using \a name as the matching identifier. This overload + * allows pipeline handlers to create an IPA module by specifying its name + * instead of relying on the fact that the IPA module matches the pipeline + * handler's one. + * + * \return A newly created IPA proxy, or nullptr if no matching IPA module is + * found or if the IPA proxy fails to initialize + */ + +/** + * \fn IPAManager::createIPA(PipelineHandler *pipe, uint32_t minVersion, uint32_t maxVersion) + * \brief Create an IPA proxy that matches the pipeline handler name and the + * requested version + * \param[in] pipe The pipeline handler that wants to create the IPA module + * \param[in] minVersion Minimum acceptable version of IPA module + * \param[in] maxVersion Maximum acceptable version of IPA module + * + * Create an IPA module using the pipeline handler name as the matching + * identifier. This overload allows pipeline handler to create an IPA module + * whose name matches the pipeline handler one. + * * \return A newly created IPA proxy, or nullptr if no matching IPA module is * found or if the IPA proxy fails to initialize */ diff --git a/src/libcamera/ipa_module.cpp b/src/libcamera/ipa_module.cpp index e6ea61e44829..0bd6f14626fe 100644 --- a/src/libcamera/ipa_module.cpp +++ b/src/libcamera/ipa_module.cpp @@ -463,21 +463,21 @@ IPAInterface *IPAModule::createInterface() /** * \brief Verify if the IPA module matches a given pipeline handler - * \param[in] pipe Pipeline handler to match with + * \param[in] name The IPA module name * \param[in] minVersion Minimum acceptable version of IPA module * \param[in] maxVersion Maximum acceptable version of IPA module * - * This function checks if this IPA module matches the \a pipe pipeline handler, + * This function checks if this IPA module matches the requested \a name * and the input version range. * - * \return True if the pipeline handler matches the IPA module, or false otherwise + * \return True if the IPA module matches, or false otherwise */ -bool IPAModule::match(PipelineHandler *pipe, - uint32_t minVersion, uint32_t maxVersion) const +bool IPAModule::match(const char *name, uint32_t minVersion, + uint32_t maxVersion) const { return info_.pipelineVersion >= minVersion && info_.pipelineVersion <= maxVersion && - !strcmp(info_.pipelineName, pipe->name()); + !strcmp(info_.name, name); } std::string IPAModule::logPrefix() const From patchwork Thu Jun 18 10:18:41 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Jacopo Mondi X-Patchwork-Id: 26917 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 118EFBF415 for ; Thu, 18 Jun 2026 10:19:00 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id B4DF1629CE; Thu, 18 Jun 2026 12:18:57 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="Ls2YGaY0"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id BE07B61754 for ; Thu, 18 Jun 2026 12:18:53 +0200 (CEST) Received: from [192.168.125.177] (mob-109-113-4-199.net.vodafone.it [109.113.4.199]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 5BA341894; Thu, 18 Jun 2026 12:18:18 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1781777898; bh=VkqMScwWLjEze/239nhSjPeLBBooBpQmAdhfD+EXPbk=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=Ls2YGaY0r+h7xsFqQXmznwIRylr1Tge/51hRgHRCLbRhXRWFh8Ufj+KPx7hIq3E4a 5HlgU4Qq7OyvLLbGsxDZdIbIRiFf85cWZ/IqUXr/tm20+gOrfpXji0tgxq1OUQLMQb WBNgU15Iyk6BlbyHWLza43Df5tgDZ1h9Vd1GsaK0= From: Jacopo Mondi Date: Thu, 18 Jun 2026 12:18:41 +0200 Subject: [PATCH 02/14] ipa: ipa_module: Remove pipelineName MIME-Version: 1.0 Message-Id: <20260618-rppx1-ipa-v1-2-32337264cfcd@ideasonboard.com> References: <20260618-rppx1-ipa-v1-0-32337264cfcd@ideasonboard.com> In-Reply-To: <20260618-rppx1-ipa-v1-0-32337264cfcd@ideasonboard.com> To: =?utf-8?q?Niklas_S=C3=B6derlund?= , libcamera-devel@lists.libcamera.org Cc: Jacopo Mondi , =?utf-8?b?QmFybmFiw6Fz?= =?utf-8?q?_P=C5=91cze?= , Kieran Bingham X-Mailer: b4 0.14.3 X-Developer-Signature: v=1; a=openpgp-sha256; l=6199; i=jacopo.mondi@ideasonboard.com; h=from:subject:message-id; bh=VkqMScwWLjEze/239nhSjPeLBBooBpQmAdhfD+EXPbk=; b=owEBbQKS/ZANAwAKAXI0Bo8WoVY8AcsmYgBqM8YJbbGymFjv1gnXXB5z6l1JPhYa+Z72/kdGp LrRM9uAeviJAjMEAAEKAB0WIQS1xD1IgJogio9YOMByNAaPFqFWPAUCajPGCQAKCRByNAaPFqFW PPjgEACwtgA1J3he1JIBSYfLAKmL4agM1dgrod7Vdv4uxl51pCAcJ9ebGL5b/WFOD8B8SXvWRcF ZoipeGxdCTSIf0jAfe7b5dMKilDr2bZy2JPkizHcAXqUuEbsP4NCY2Zed4XGkLmWV7wfCAvwPHO AJzFkykyiDNWzDtyzp0vJzk0hdCVTGBMEcatzDBMOunGE5+CdZaWi5kSGGRqceGbGEQ831/lAAu yVxgVOUEbKiPf6ufmoPI0aILjs4F3t9MW6RyBjHoy1EfI/XaCCudSRzPTeCxh5rj5YNe/823lUf SiMOPqtY6AitwHgi5bInukJVqoNHfhsMQhaIn3TgEwy+GPl4s8mj+OlEAnC3n/GZ/Bg/ZXrtbu/ NTki3L40iI0zyq3FZQdCPCb8JVxT8bNOZ4jxjBzW4iDMB0UNnYBuhXl/ROokg1fPTpItGktRieV zf6LGJo9R/Yd09UrFuEPjytKwHcRtt20yeVILmT/qn5YNPk/3C9muW8zKjPxgRk6DELbZ9XYJP9 4lt2DKma0AVq8osThmJjPxPFcdK8JMB0m7WQdIRapNI7x8PfP/JbyYjjsXArDvWylxDq9C4Ac1w T5Yb9tQvyb4Bpn3vWXIWINz0BmQ4SDyPrhESfhikLjqzdlwV+n6eaikkSFzVdZbyY4/KSITaTAh ORCuPtTF3RKArnA== X-Developer-Key: i=jacopo.mondi@ideasonboard.com; a=openpgp; fpr=72392EDC88144A65C701EA9BA5826A2587AD026B X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" All the IPA modules declare a pipelineName that is identical to their name. As we now support creating IPAs by name (either explicitly provided by the pipeline handlers or by using the pipeline name), the duplicated information in IPAModuleInfo is redundant. Remove it. Signed-off-by: Jacopo Mondi Reviewed-by: Barnabás Pőcze Reviewed-by: Kieran Bingham --- include/libcamera/ipa/ipa_module_info.h | 1 - src/ipa/ipu3/ipu3.cpp | 1 - src/ipa/mali-c55/mali-c55.cpp | 1 - src/ipa/rkisp1/rkisp1.cpp | 1 - src/ipa/rpi/pisp/pisp.cpp | 1 - src/ipa/rpi/vc4/vc4.cpp | 1 - src/ipa/simple/soft_simple.cpp | 1 - src/ipa/vimc/vimc.cpp | 1 - src/libcamera/ipa_module.cpp | 15 ++++++--------- test/ipa/ipa_module_test.cpp | 3 --- 10 files changed, 6 insertions(+), 20 deletions(-) diff --git a/include/libcamera/ipa/ipa_module_info.h b/include/libcamera/ipa/ipa_module_info.h index 3507a6d7678a..436ec82d7750 100644 --- a/include/libcamera/ipa/ipa_module_info.h +++ b/include/libcamera/ipa/ipa_module_info.h @@ -16,7 +16,6 @@ namespace libcamera { struct IPAModuleInfo { int moduleAPIVersion; uint32_t pipelineVersion; - char pipelineName[256]; char name[256]; } __attribute__((packed)); diff --git a/src/ipa/ipu3/ipu3.cpp b/src/ipa/ipu3/ipu3.cpp index 4bdc4b7677fe..42d475ecc63a 100644 --- a/src/ipa/ipu3/ipu3.cpp +++ b/src/ipa/ipu3/ipu3.cpp @@ -672,7 +672,6 @@ const struct IPAModuleInfo ipaModuleInfo = { IPA_MODULE_API_VERSION, 1, "ipu3", - "ipu3", }; /** diff --git a/src/ipa/mali-c55/mali-c55.cpp b/src/ipa/mali-c55/mali-c55.cpp index 1d3af0627fdb..47bcd3748f7f 100644 --- a/src/ipa/mali-c55/mali-c55.cpp +++ b/src/ipa/mali-c55/mali-c55.cpp @@ -380,7 +380,6 @@ const struct IPAModuleInfo ipaModuleInfo = { IPA_MODULE_API_VERSION, 1, "mali-c55", - "mali-c55", }; IPAInterface *ipaCreate() diff --git a/src/ipa/rkisp1/rkisp1.cpp b/src/ipa/rkisp1/rkisp1.cpp index 58ef163d85ad..e0dde542c71a 100644 --- a/src/ipa/rkisp1/rkisp1.cpp +++ b/src/ipa/rkisp1/rkisp1.cpp @@ -481,7 +481,6 @@ const struct IPAModuleInfo ipaModuleInfo = { IPA_MODULE_API_VERSION, 1, "rkisp1", - "rkisp1", }; IPAInterface *ipaCreate() diff --git a/src/ipa/rpi/pisp/pisp.cpp b/src/ipa/rpi/pisp/pisp.cpp index de2a6afeb8c1..975d8bfdf8a5 100644 --- a/src/ipa/rpi/pisp/pisp.cpp +++ b/src/ipa/rpi/pisp/pisp.cpp @@ -1145,7 +1145,6 @@ const IPAModuleInfo ipaModuleInfo = { IPA_MODULE_API_VERSION, 1, "rpi/pisp", - "rpi/pisp", }; IPAInterface *ipaCreate() diff --git a/src/ipa/rpi/vc4/vc4.cpp b/src/ipa/rpi/vc4/vc4.cpp index b6ca44e7a0a0..f30eee7ff012 100644 --- a/src/ipa/rpi/vc4/vc4.cpp +++ b/src/ipa/rpi/vc4/vc4.cpp @@ -633,7 +633,6 @@ const struct IPAModuleInfo ipaModuleInfo = { IPA_MODULE_API_VERSION, 1, "rpi/vc4", - "rpi/vc4", }; IPAInterface *ipaCreate() diff --git a/src/ipa/simple/soft_simple.cpp b/src/ipa/simple/soft_simple.cpp index 629e1a32de8a..d4ab91e30bf1 100644 --- a/src/ipa/simple/soft_simple.cpp +++ b/src/ipa/simple/soft_simple.cpp @@ -343,7 +343,6 @@ const struct IPAModuleInfo ipaModuleInfo = { IPA_MODULE_API_VERSION, 0, "simple", - "simple", }; IPAInterface *ipaCreate() diff --git a/src/ipa/vimc/vimc.cpp b/src/ipa/vimc/vimc.cpp index 4162b848f7b1..bae48ea90662 100644 --- a/src/ipa/vimc/vimc.cpp +++ b/src/ipa/vimc/vimc.cpp @@ -183,7 +183,6 @@ const struct IPAModuleInfo ipaModuleInfo = { IPA_MODULE_API_VERSION, 0, "vimc", - "vimc", }; IPAInterface *ipaCreate() diff --git a/src/libcamera/ipa_module.cpp b/src/libcamera/ipa_module.cpp index 0bd6f14626fe..53cea6b187e8 100644 --- a/src/libcamera/ipa_module.cpp +++ b/src/libcamera/ipa_module.cpp @@ -215,18 +215,15 @@ Span elfLoadSymbol(Span elf, const char *symbol) * \var IPAModuleInfo::pipelineVersion * \brief The pipeline handler version that the IPA module is for * - * \var IPAModuleInfo::pipelineName - * \brief The name of the pipeline handler that the IPA module is for - * - * This name is used to match a pipeline handler with the module. - * * \var IPAModuleInfo::name * \brief The name of the IPA module * - * The name may be used to build file system paths to IPA-specific resources. - * It shall only contain printable characters, and may not contain '*', '?' or - * '\'. For IPA modules included in libcamera, it shall match the directory of - * the IPA module in the source tree. + * This name is used to match a the IPA module. + * + * The name may also be used to build file system paths to IPA-specific + * resources. It shall only contain printable characters, and may not contain + * '*', '?' or '\'. For IPA modules included in libcamera, it shall match the + * directory of the IPA module in the source tree. * * \todo Allow user to choose to isolate open source IPAs */ diff --git a/test/ipa/ipa_module_test.cpp b/test/ipa/ipa_module_test.cpp index 1c97da3242a7..af71c3285328 100644 --- a/test/ipa/ipa_module_test.cpp +++ b/test/ipa/ipa_module_test.cpp @@ -37,12 +37,10 @@ protected: cerr << "IPA module information mismatch: expected:" << endl << "moduleAPIVersion = " << testInfo.moduleAPIVersion << endl << "pipelineVersion = " << testInfo.pipelineVersion << endl - << "pipelineName = " << testInfo.pipelineName << endl << "name = " << testInfo.name << "got: " << endl << "moduleAPIVersion = " << info.moduleAPIVersion << endl << "pipelineVersion = " << info.pipelineVersion << endl - << "pipelineName = " << info.pipelineName << endl << "name = " << info.name << endl; } @@ -58,7 +56,6 @@ protected: IPA_MODULE_API_VERSION, 0, "vimc", - "vimc", }; count += runTest("src/ipa/vimc/ipa_vimc.so", testInfo); From patchwork Thu Jun 18 10:18:42 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jacopo Mondi X-Patchwork-Id: 26918 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 63AAEC3301 for ; Thu, 18 Jun 2026 10:19:03 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id D125A6298D; Thu, 18 Jun 2026 12:18:58 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="oyzMiO5F"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id B79256297C for ; Thu, 18 Jun 2026 12:18:54 +0200 (CEST) Received: from [192.168.125.177] (mob-109-113-4-199.net.vodafone.it [109.113.4.199]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 601442393; Thu, 18 Jun 2026 12:18:19 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1781777899; bh=a8B6o1ut5YgA0KtHJ3CBSBbrsVgQBvs05lZoZ2YoL3M=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=oyzMiO5FgL5WoOtOLfpk4BIVaJsrhgb/NPSUsT9ySSPdXsB21gsBXqXKu0Sb+Kbh5 Kv+HEgxIuRtOxjerkgNg5EDfS0qSHO1BPseZ/bWC+CW9dDLWFzddIIv+k22A9RkRfy 5ikZRFCCAJbtv4Y//Qq7AVr2jDvHv4H4QJ6d/NJ8= From: Jacopo Mondi Date: Thu, 18 Jun 2026 12:18:42 +0200 Subject: [PATCH 03/14] ipa: Allow pipelines to have differently named IPA MIME-Version: 1.0 Message-Id: <20260618-rppx1-ipa-v1-3-32337264cfcd@ideasonboard.com> References: <20260618-rppx1-ipa-v1-0-32337264cfcd@ideasonboard.com> In-Reply-To: <20260618-rppx1-ipa-v1-0-32337264cfcd@ideasonboard.com> To: =?utf-8?q?Niklas_S=C3=B6derlund?= , libcamera-devel@lists.libcamera.org Cc: Jacopo Mondi X-Mailer: b4 0.14.3 X-Developer-Signature: v=1; a=openpgp-sha256; l=3861; i=jacopo.mondi@ideasonboard.com; h=from:subject:message-id; bh=a8B6o1ut5YgA0KtHJ3CBSBbrsVgQBvs05lZoZ2YoL3M=; b=owEBbQKS/ZANAwAKAXI0Bo8WoVY8AcsmYgBqM8YJeGAi1Et4e6nFC7gwlvUp3LaxnzEyLKI6p mwvl7A1f3OJAjMEAAEKAB0WIQS1xD1IgJogio9YOMByNAaPFqFWPAUCajPGCQAKCRByNAaPFqFW PGVXEACsAtBPn+oLU3bjYCAYOsXh8d0sVHSGtyeuELqIykzSJX5te3Axrild3iul/ARcHiu/be7 7lxaBY/teseVxZ7hnp+mCW05+444mlUyf8hBTHD/dWd55q4bPC73FKZLi8dYfhPKghv9X0oC8Qi PiWgyTAogIGrSNPd32LE6gjM429bqrMSHS0B3ToVM049Fw7lTLUxC5AYmXRGUAekSL6XvoDOg46 iitej5IbFmTZ1mSwUrVOzW6VmAChTvdVUDsUB8eG8inatCcSFvPDohYfFyu1GGxr+zzLfKqDPMR IpkW3/eCpwxCPdbm/ntRbnqWMd1JoryuKzxIaGIP8nXHBd8hZEos+zcBFAMHXfKxlfZgw3a5o5p BpKhxvgdicbXhDLKIpfHEOxCmuY88sbl6huQf32OKnviKnFUKklc/+YNyO+wA0+R/uHgOXCH8JQ MEtZEzK8xdR9PluIRnrFKntn44WSXfiLTkdHffDx0WWvLVrXywvG3hLMZm0RSnYREyKZ7Sf4pLq yrK7vT/pBvVbqtkH9ZnkefcnppSXs1xelC2snEuXUQGJBm6O2YAfBrP8YyMxaBsY1/fND4B2s/M 2yBTynve88DUt6lfhq46G3d5ofRHuXZct6D32dyUc62eK9RoEudVBZucADz+UXfM8STN88vL7Va c4ECSfzrZaXT6JA== X-Developer-Key: i=jacopo.mondi@ideasonboard.com; a=openpgp; fpr=72392EDC88144A65C701EA9BA5826A2587AD026B X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Right now the build system assumes a 1-to-1 matching between a pipeline handler name and an IPA module. This, as also acknowledged by a \todo comment, is quite a rigid requirement and only allows a 1-to-1 matching between pipeline and IPA names. The more platforms libcamera supports, the more it is likely that a pipeline handler could re-use an IPA module. This is particularly relevant for the softISP IPA module which could theoretically be plugged to any pipeline. Likewise, the forthcoming R-Car Gen4 support uses the RkISP1 IPA and at the moment would require building the 'rkisp1' pipeline in to have the IPA module available. When building IPAs, the build system iterates the list of enabled pipeline handlers and for each of them tries to verify if the 'ipas' list contains a corresponding entry for it. The 'ipas' meson options is an array option and, as no default value is specified for it, it contains by default all its possible choices. In this way if no value is specified for the 'ipas' option, compiling the pipeline handlers ['X','Y', 'Z'] will compile the ['X', 'Y', 'Z'] IPAs. If instead the user specifies '-Dipas=X' during the configuration then only IPA module ['X'] will be built, regardless of which pipeline is enabled. Building an IPA module will anyway require to build a corresponding pipeline with the same name. Relax the 1-to-1 'pipeline'-'IPA' naming requirement by introducing a dictionary that associates pipelines with IPA modules. For each enabled pipeline: 1) Make sure an IPA module exists for it 2) Make sure the IPA module is enabled by the 'ipas' option 3) Make sure the IPA is compiled once only This will require every new pipeline to add an entry to the dictionary and specify which IPA module they would like to use. Signed-off-by: Jacopo Mondi --- src/ipa/meson.build | 40 ++++++++++++++++++++++++++++++---------- 1 file changed, 30 insertions(+), 10 deletions(-) diff --git a/src/ipa/meson.build b/src/ipa/meson.build index eb7846e47888..c583c7efdd35 100644 --- a/src/ipa/meson.build +++ b/src/ipa/meson.build @@ -24,6 +24,16 @@ subdir('libipa') ipa_sign = files('ipa-sign.sh') +supported_ipas = { + 'ipu3': 'ipu3', + 'mali-c55': 'mali-c55', + 'rkisp1': 'rkisp1', + 'rpi/pisp': 'rpi/pisp', + 'rpi/vc4': 'rpi/vc4', + 'simple': 'simple', + 'vimc': 'vimc' +} + ipa_modules = get_option('ipas') # Tests require the vimc IPA, similar to vimc pipline-handler for their @@ -39,24 +49,34 @@ ipa_names = [] subdirs = [] foreach pipeline : pipelines - # The current implementation expects the IPA module name to match the - # pipeline name. - # \todo Make the IPA naming scheme more flexible. - if not ipa_modules.contains(pipeline) + # Make sure an IPA exists for the pipeline + if not supported_ipas.has_key(pipeline) + continue + endif + + ipa = supported_ipas.get(pipeline) + + # Only build IPAs specified with '-Dipas' + if not ipa_modules.contains(ipa) + continue + endif + + # If enabled already do not add it twice + if enabled_ipa_names.contains(ipa) continue endif - enabled_ipa_names += pipeline + enabled_ipa_names += ipa # Allow multi-level directory structuring for the IPAs if needed. - pipeline = pipeline.split('/')[0] - if pipeline in subdirs + ipa = ipa.split('/')[0] + if ipa in subdirs continue endif - subdirs += pipeline - subdir(pipeline) + subdirs += ipa + subdir(ipa) - # Don't reuse the pipeline variable below, the subdirectory may have + # Don't reuse the ipa variable below, the subdirectory may have # overwritten it. endforeach From patchwork Thu Jun 18 10:18:43 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jacopo Mondi X-Patchwork-Id: 26919 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 95857C3302 for ; Thu, 18 Jun 2026 10:19:04 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id EABAF629E8; Thu, 18 Jun 2026 12:18:59 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="H/mrTtFs"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 52F4761754 for ; Thu, 18 Jun 2026 12:18:55 +0200 (CEST) Received: from [192.168.125.177] (mob-109-113-4-199.net.vodafone.it [109.113.4.199]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id E80F7258D; Thu, 18 Jun 2026 12:18:19 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1781777900; bh=qM4mVGKqN5DKdApbDp3WQJBsrNy9s2Ca5BtVuZGvNGE=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=H/mrTtFs3T/jp3GMQX76r69fiVRL9v7eGGZWy3wVMma0Td/O9nrrJTLmx888xwx3z 1mBf4+BIbpgnv/fS5SnLzW28W6thDWd4r++LAptnULDVOizlNJhEWI+sTLhX7Ob+h6 jwotD+mVkfVyK3EUgwh2MOKD4MOL0SPJ+bNE8Sus= From: Jacopo Mondi Date: Thu, 18 Jun 2026 12:18:43 +0200 Subject: [PATCH 04/14] ipa: libipa: awb: Reimplement AwbAlgorithm MIME-Version: 1.0 Message-Id: <20260618-rppx1-ipa-v1-4-32337264cfcd@ideasonboard.com> References: <20260618-rppx1-ipa-v1-0-32337264cfcd@ideasonboard.com> In-Reply-To: <20260618-rppx1-ipa-v1-0-32337264cfcd@ideasonboard.com> To: =?utf-8?q?Niklas_S=C3=B6derlund?= , libcamera-devel@lists.libcamera.org Cc: Jacopo Mondi , Kieran Bingham , Daniel Scally X-Mailer: b4 0.14.3 X-Developer-Signature: v=1; a=openpgp-sha256; l=48850; i=jacopo.mondi@ideasonboard.com; h=from:subject:message-id; bh=qM4mVGKqN5DKdApbDp3WQJBsrNy9s2Ca5BtVuZGvNGE=; b=owEBbQKS/ZANAwAKAXI0Bo8WoVY8AcsmYgBqM8YJfNWhp/JF+4+56XZPowfRPy3nFHIElcOQT oYe3R6XJIiJAjMEAAEKAB0WIQS1xD1IgJogio9YOMByNAaPFqFWPAUCajPGCQAKCRByNAaPFqFW PL8pEAC9LWj2PmytX/cEsHNm8iw16z6oBoGHhE/ESfonFtA92gYtYiSMqWQebnaFapfSpCj8Zj3 YGsx4QtO5uXnQtcV4a4jh6+5jqX3MoK6X2WNhrQVJqftlYNp0GW8RGkiwTeRWUEkBDX+IrPU8Di odq3AtSnquBouSLu74FgH7+w/Y1YTsU4BOWlIo79hFKZrTB2rsDQrm2LAVHgcsNa63GjAdZJjLC IWqv4S4DzBVPyRCjpWa0xCbanl+aSASOJOcRSqr6joWQUlqiJ3QurOOQD/m/p7kBrKKtXNFwJ++ 6spF629/eUMUN0fEtSAfDVuX5ycnNHEnYEG2PdOt40iingGSKx44HEoSlW/RBqTc1Vm87AiN0ZK 8BqzKWsmcLJCh2GeStUMMqQoWjslEce4IeGPFgmepkrRiu1AZBAwCjjZGwKJhobfMLFKUU+v9Ng 0N2KhTJuSrgs2Mo5EQcZe1bKK1fn62rjXtDiH/kNgZb55uOskjyG8wKnhiDjKpdAXBjeUbsJXsc z6kj+xuegNd0Fx/Ph2zEmCfNQfK33uMha4JSSFYD13ddAjjJVAmY0viE1Dz4ENs56yjWzLgigqf /EXzxbZMMfAjyzqU7+6/2S7v/9UJZsB0cLx27+cEB+sxlaK1EeOJfLCE7VXr1LUcmBsnrpJcirp FCTZypJ4nS1zoxw== X-Developer-Key: i=jacopo.mondi@ideasonboard.com; a=openpgp; fpr=72392EDC88144A65C701EA9BA5826A2587AD026B X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" The current implementation of AwbAlgorithm is the following one: AwbAlgorithm o---- RkISP1Awb ^ | |-----| | | Bayes Grey The platform-specific Awb implementation instantiate one of the Bayes or Grey Awb helper classes and both inherit from the AwbAlgorithm base class. The handling of libcamera controls is split between the base and the derived (mostly Bayes) class. We want instead the AwbAlgorithm to handle all-things-libcamera and let the Bayes and Grey classes implement the logic. In order to achieve that use composition also in the AwbAlgorithm class and introduce an AwbImplementation base class from which Bayes and Grey derive from. AwbImplementation o---- AwbAlgorithmBase ^ ^ | | |-----| AwbAlgorithm> o---- RkISP1Awb | | Bayes Grey The AwbAlgorithm<> class is instantiated with a Q template argument that represents the platform-specific register format which is used to initialize controls and clamp gains to the hardware limits. Signed-off-by: Jacopo Mondi Signed-off-by: Kieran Bingham Reviewed-by: Daniel Scally Tested-by: Daniel Scally --- src/ipa/libipa/awb.cpp | 527 +++++++++++++++++++++++++++++++++----- src/ipa/libipa/awb.h | 114 +++++++-- src/ipa/libipa/awb_bayes.cpp | 42 +-- src/ipa/libipa/awb_bayes.h | 11 +- src/ipa/libipa/awb_grey.cpp | 25 +- src/ipa/libipa/awb_grey.h | 5 +- src/ipa/rkisp1/algorithms/awb.cpp | 223 +++------------- src/ipa/rkisp1/algorithms/awb.h | 18 +- src/ipa/rkisp1/ipa_context.h | 29 +-- 9 files changed, 640 insertions(+), 354 deletions(-) diff --git a/src/ipa/libipa/awb.cpp b/src/ipa/libipa/awb.cpp index 29a78069c52c..d1f22d7d57b4 100644 --- a/src/ipa/libipa/awb.cpp +++ b/src/ipa/libipa/awb.cpp @@ -2,7 +2,7 @@ /* * Copyright (C) 2024 Ideas on Board Oy * - * Generic AWB algorithms + * libIPA Awb algorithms */ #include "awb.h" @@ -11,9 +11,16 @@ #include +#include "awb_bayes.h" +#include "awb_grey.h" + +constexpr int32_t kMinColourTemperature = 2500; +constexpr int32_t kMaxColourTemperature = 10000; +constexpr int32_t kDefaultColourTemperature = 5000; + /** * \file awb.h - * \brief Base classes for AWB algorithms + * \brief libipa awb implementation */ namespace libcamera { @@ -22,34 +29,74 @@ LOG_DEFINE_CATEGORY(Awb) namespace ipa { +namespace awb { + /** - * \class AwbResult - * \brief The result of an AWB calculation + * \struct Context + * \brief Awb gains and colour temperature * - * This class holds the result of an auto white balance calculation. + * \var Context::gains + * \brief The white balance gains + * + * \var Context::temperatureK + * \brief The colour temperature, in Kelvin */ /** - * \var AwbResult::gains - * \brief The calculated white balance gains + * \struct ActiveState + * \brief Active awb state shared across frames + * + * \var ActiveState::manual + * \brief The most recent manually requested awb state + * + * \var ActiveState::automatic + * \brief The most recent automatically calculated awb state + * + * \var ActiveState::autoEnabled + * \brief True when automatic awb is selected */ /** - * \var AwbResult::colourTemperature - * \brief The calculated colour temperature in Kelvin + * \struct FrameContext + * \brief Per-frame awb state + * + * \var FrameContext::autoEnabled + * \brief True when automatic awb is in use */ +} /* namespace awb */ + /** * \class AwbStats - * \brief An abstraction class wrapping hardware-specific AWB statistics + * \brief An abstraction class wrapping hardware-specific awb statistics * - * IPA modules using an AWB algorithm based on the AwbAlgorithm class need to - * implement this class to give the algorithm access to the hardware-specific - * statistics data. + * IPA modules shall provide a derived implementation of this class to give the + * awb algorithm access to the hardware-specific statistics data in a common + * format. + */ + +/** + * \fn AwbStats::AwbStats() + * \brief Construct an empty AwbStat + */ + +/** + * \brief Construct Awb statistics from RGB mean values + * \param[in] rgbMeans The RGB mean values + */ +AwbStats::AwbStats(const RGB &rgbMeans) + : rgbMeans_(rgbMeans) +{ + rg_ = rgbMeans_.r() / rgbMeans_.g(); + bg_ = rgbMeans_.b() / rgbMeans_.g(); +} + +/** + * AwbStat::~AwbStat + * \brief Virtual class destructor */ /** - * \fn AwbStats::computeColourError() * \brief Compute an error value for when the given gains would be applied * \param[in] gains The gains to apply * @@ -57,87 +104,383 @@ namespace ipa { * applied. To keep the actual implementations computationally inexpensive, * the squared colour error shall be returned. * - * If the AWB statistics provide multiple zones, the average of the individual + * If the awb statistics provide multiple zones, the average of the individual * squared errors shall be returned. Averaging/normalizing is necessary so that * the numeric dimensions are the same on all hardware platforms. * * \return The computed error value */ +double AwbStats::computeColourError(const RGB &gains) const +{ + /* + * Compute the sum of the squared colour error (non-greyness) as + * it appears in the log likelihood equation. + */ + double deltaR = gains.r() * rg_ - 1.0; + double deltaB = gains.b() * bg_ - 1.0; + double delta2 = deltaR * deltaR + deltaB * deltaB; + + return delta2; +} + +/** + * \brief Retrieve if the awb statistics are valid + * + * If the colour mean values are too small, the AwbAlgorithm class doesn't have + * enough information to meaningfully calculate white-balance gains. Freeze the + * algorithm in that case. + * + * \return True if the awb statistics are valid, false otherwise + */ +bool AwbStats::valid() const +{ + double minValue = minColourValue(); + + return rgbMeans_.r() > minValue || rgbMeans_.g() > minValue || + rgbMeans_.b() > minValue; +} + +/** + * \fn AwbStats::rgRatio() + * \brief Retrieve the Red/Green ratio + * \return The Red/Green ratio + */ /** - * \fn AwbStats::rgbMeans() - * \brief Get RGB means of the statistics + * \fn AwbStats::bgRatio() + * \brief Retrieve the Blue/Green ratio + * \return The Blue/Green ratio + */ + +/** + * \brief Retrieve the RGB means values * * Fetch the RGB means from the statistics. The values of each channel are * dimensionless and only the ratios are used for further calculations. This is - * used by the simple grey world model to calculate the gains to apply. + * used by the AwbGrey to calculate the gains to apply. * * \return The RGB means */ +RGB AwbStats::rgbMeans() const +{ + return rgbMeans_; +} /** - * \class AwbAlgorithm - * \brief A base class for auto white balance algorithms + * \fn AwbStats::minColourValue + * \brief Retrieve the threshold below which a colour information is not valid + * + * Mean colour values as reported by the ISP through the white balance + * statistics are considered valid for colour gain calculation purposes when + * they are above a certain value. The awb algorithm needs to know what is the + * minimum value below which it should ignore the colour mean values. + * + * As different ISP platforms report gains in different formats, the threshold + * is then platform specific, and all IPA implementations that use the + * AwbAlgorithm class should provide that information by implementing this + * function. + * + * The reported minimum value applies to all three colour channels. * - * This class is a base class for auto white balance algorithms. It defines an - * interface for the algorithms to implement, and is used by the IPAs to - * interact with the concrete implementation. + * This function is used by AwbStats::valid(). + * + * \return The minimum valid colour value */ /** - * \fn AwbAlgorithm::init() - * \brief Initialize the algorithm with the given tuning data - * \param[in] tuningData The tuning data to use for the algorithm + * \var AwbStats::rgbMeans_ + * \brief Mean values of colour channels + */ + +/** + * \var AwbStats::rg_ + * \brief Red/Green ratio + */ + +/** + * \var AwbStats::bg_ + * \brief Blue/Green ratio + */ + +/** + * \class AwbImplementation + * \brief Pure virtual base class for awb algorithms implementations * - * \return 0 on success, a negative error code otherwise + * The AwbImplementation class defines the interface for the awb algorithm + * implementations. + * + * It is currently implemented by the AwbGrey and AwbBayes classes. + * + * The interface defines an init() function to initialize the algorithm with the + * content of the tuning file and two functions to compute colour gains + * according to the algorithm operating mode (auto or manual) in use. + * + * The calculateAwb() function calculates colour gains given a set of statistics + * provided by the IPA module. It is used when the algorithm operates in auto + * mode and gains are dynamically computed given a new set of statistics from + * the awb engine. + * + * The gainsFromColourTemperature() function instead interpolates a gain curve + * if specified in the tuning file with the supplied colour temperature. This + * function is used in manual mode where it is expected the application to + * supply a colour temperature and the algorithm to adjust the white balance + * gains to it. + */ + +/** + * \fn AwbImplementation::~AwbImplementation + * \brief Virtual class destructor + */ + +/** + * \fn AwbImplementation::init() + * \param[in] tuningData + * \brief Initialize the algorithm by parsing \a tuningData */ /** - * \fn AwbAlgorithm::calculateAwb() - * \brief Calculate AWB data from the given statistics - * \param[in] stats The statistics to use for the calculation - * \param[in] lux The lux value of the scene + * \fn AwbImplementation::calculateAwb() + * \param[in] stats The awb statistics + * \param[in] lux The estimated lux level + * \param[in] ranges The colour temperature search limits (AwbBayes only) * - * Calculate an AwbResult object from the given statistics and lux value. A \a - * lux value of 0 means it is unknown or invalid and the algorithm shall ignore - * it. + * Calculate a new set of colour gains and a colour temperature given a new + * set of statistics \a stats, an estimated luminance \a lux and a range of + * colour temperature to limit the search (for AwbBayes only). * - * \return The AWB result + * \return An awb::Context computed using the new statistics + */ +/** + * \fn AwbImplementation::gainsFromColourTemperature() + * \param[in] temperatureK Colour temperature in Kelvin + * + * Calculate a new set of colour gains by interpolating a gain curve (if + * provided in the tuning file) with a new colour temperature \a temperatureK. + * + * \return The RGB gain values adjusted to \a temperatureK or std::nullopt if + * no gain curve is specified in the tuning data */ /** - * \fn AwbAlgorithm::gainsFromColourTemperature() - * \brief Compute white balance gains from a colour temperature - * \param[in] colourTemperature The colour temperature in Kelvin + * \class AwbAlgorithmBase + * \brief Base class for AwbAlgorithm for non-templated functions implementation * - * Compute the white balance gains from a \a colourTemperature. This function - * does not take any statistics into account. It is used to compute the colour - * gains when the user manually specifies a colour temperature. + * Base class for AwbAlgorithm where non-templated functions are implemented. + * IPA implementations shall use AwbAlgorithm and not this class. + */ + +/** + * \brief Initialize the algorithm with the given tuning data + * \param[in] tuningData The tuning data to use for the algorithm + * + * Parse \a tuningData to initialize the awb algorithm and register controls. + * IPA modules are expected to call this function as part of their + * implementation of Algorithm::init(). * - * \return The colour gains or std::nullopt if the conversion is not possible + * \return 0 on success, a negative error code otherwise + */ +int AwbAlgorithmBase::init(const ValueNode &tuningData) +{ + bayes_ = false; + + if (!tuningData.contains("algorithm")) + LOG(Awb, Info) << "No Awb algorithm specified, using grey world"; + + auto mode = tuningData["algorithm"].get("grey"); + if (mode == "grey") { + impl_ = std::make_unique(); + } else if (mode == "bayes") { + impl_ = std::make_unique(); + bayes_ = true; + } else { + LOG(Awb, Error) << "Unknown Awb algorithm: " << mode; + return -EINVAL; + } + + LOG(Awb, Debug) << "Using Awb algorithm: " << mode; + + int ret = impl_->init(tuningData); + if (ret) + return ret; + + controls_[&controls::ColourTemperature] = + ControlInfo(kMinColourTemperature, kMaxColourTemperature, + kDefaultColourTemperature); + controls_[&controls::AwbEnable] = ControlInfo(false, true); + + return parseModeConfigs(tuningData, controls::AwbAuto); +} + +/** + * \brief Configure the awb algorithm + * \param[in] state The awb active state + * \return 0 if successful, an error code otherwise */ +int AwbAlgorithmBase::configure(awb::ActiveState &state) +{ + state.manual.gains = RGB{ 1.0 }; + auto gains = impl_->gainsFromColourTemperature(kDefaultColourTemperature); + if (gains) + state.automatic.gains = *gains; + else + state.automatic.gains = RGB{ 1.0 }; + + state.autoEnabled = true; + state.manual.temperatureK = kDefaultColourTemperature; + state.automatic.temperatureK = kDefaultColourTemperature; + + return 0; +} /** - * \fn AwbAlgorithm::controls() - * \brief Get the controls info map for this algorithm + * \brief Queue a Request to the awb algorithm + * \param[in] state The awb active state + * \param[in] frame The frame number + * \param[in] frameContext The awb frame context + * \param[in] controls The list of controls part of the Request + * + * Queue a new Request to the awb algorithm and modify its behaviour according + * to the provided controls. * - * \return The controls info map + * The currently handled controls are: + * - controls::AwbEnable + * - controls::AwbMode + * - controls::ColourGains + * - controls::ColourTemperature */ +void AwbAlgorithmBase::queueRequest(awb::ActiveState &state, + [[maybe_unused]] const uint32_t frame, + awb::FrameContext &frameContext, + const ControlList &controls) +{ + const auto &awbEnable = controls.get(controls::AwbEnable); + if (awbEnable && *awbEnable != state.autoEnabled) { + state.autoEnabled = *awbEnable; + + LOG(Awb, Debug) + << (*awbEnable ? "Enabling" : "Disabling") << " Awb"; + } + + auto mode = controls.get(controls::AwbMode); + if (mode) { + auto it = modes_.find(static_cast(*mode)); + if (it == modes_.end()) { + LOG(Awb, Error) << "Unsupported Awb mode " << *mode; + return; + } + + currentMode_ = &it->second; + } + + frameContext.autoEnabled = state.autoEnabled; + + if (frameContext.autoEnabled) + return; + + const auto &colourGains = controls.get(controls::ColourGains); + const auto &colourTemperature = controls.get(controls::ColourTemperature); + bool update = false; + if (colourGains) { + state.manual.gains.r() = (*colourGains)[0]; + state.manual.gains.b() = (*colourGains)[1]; + /* + * \todo Colour temperature reported in metadata is now + * incorrect, as we can't deduce the temperature from the gains. + * This will be fixed with the bayes AWB algorithm. + */ + update = true; + } else if (colourTemperature) { + state.manual.temperatureK = *colourTemperature; + const auto &gains = impl_->gainsFromColourTemperature(*colourTemperature); + if (gains) { + state.manual.gains.r() = gains->r(); + state.manual.gains.b() = gains->b(); + update = true; + } + } + + if (update) + LOG(Awb, Debug) + << "Set colour gains to " << state.manual.gains; + + frameContext.gains = state.manual.gains; + frameContext.temperatureK = state.manual.temperatureK; +} /** - * \fn AwbAlgorithm::handleControls() - * \param[in] controls The controls to handle - * \brief Handle the controls supplied in a request + * \brief Set the gains and colour temperature values in \a frameContext + * \param[in] state The awb active state + * \param[in] frameContext The awb frame context + * + * If auto mode is enabled, take the most recently computed gains and use them + * for the current frame. Otherwise, if in manual mode, gains and colour + * temperature for a frame are set at queueRequest() time. */ +void AwbAlgorithmBase::prepare(awb::ActiveState &state, + awb::FrameContext &frameContext) +{ + if (frameContext.autoEnabled) { + frameContext.gains = state.automatic.gains; + frameContext.temperatureK = state.automatic.temperatureK; + } +} /** + * \brief Process awb statistics to calculate gains and populate metadata + * \param[in] state The awb active state + * \param[in] frameContext The awb frame context + * \param[in] stats The awb statistics + * \param[in] lux The lux value as estimated by the IPA module + * \param[out] metadata The metadata list + * + * Process \a stats to calculate new gains and colour temperature and populate + * \a metadata with the results. + */ +void AwbAlgorithmBase::process(awb::ActiveState &state, + awb::FrameContext &frameContext, + const AwbStats &stats, unsigned int lux, + ControlList &metadata) +{ + if (!stats.valid()) + return; + + auto awbResult = impl_->calculateAwb(stats, lux, { currentMode_->ctLo, + currentMode_->ctHi }); + + /* + * Clamp the gain values to the hardware, according to the gainMin_ + * and gainMax_ values. + */ + awbResult.gains = awbResult.gains.max(gainMin_).min(gainMax_); + + /* Smooth color gains adjustments. */ + double speed = 0.2; + double ct = awbResult.temperatureK; + ct = ct * speed + state.automatic.temperatureK * (1 - speed); + + state.automatic.temperatureK = awbResult.temperatureK; + state.automatic.gains = awbResult.gains * speed + + state.automatic.gains * (1 - speed); + + /* Populate metadata. */ + metadata.set(controls::AwbEnable, frameContext.autoEnabled); + metadata.set(controls::ColourGains, { static_cast(frameContext.gains.r()), + static_cast(frameContext.gains.b()) }); + metadata.set(controls::ColourTemperature, frameContext.temperatureK); + + LOG(Awb, Debug) << std::showpoint << "Means " << stats.rgbMeans() + << ", gains " << state.automatic.gains + << ", temp " << state.automatic.temperatureK << "K"; +} + +/* * \brief Parse the mode configurations from the tuning data * \param[in] tuningData the ValueNode representing the tuning data * \param[in] def The default value for the AwbMode control * * Utility function to parse the tuning data for an AwbMode entry and read all - * provided modes. It adds controls::AwbMode to AwbAlgorithm::controls_ and - * populates AwbAlgorithm::modes_. For a list of possible modes see \ref + * provided modes. It adds controls::AwbMode to AwbAlgorithmBase::controls_ and + * populates AwbAlgorithmBase::modes_. For a list of possible modes see \ref * controls::AwbModeEnum. * * Each mode entry must contain a "lo" and "hi" key to specify the lower and @@ -156,15 +499,23 @@ namespace ipa { * ... * \endcode * - * If \a def is supplied but not contained in the the \a tuningData, -EINVAL is + * If \a def is supplied but not contained in the \a tuningData, -EINVAL is * returned. * + * AwbModes are only used by the AwbBayes implementation. + * * \sa controls::AwbModeEnum * \return Zero on success, negative error code otherwise */ -int AwbAlgorithm::parseModeConfigs(const ValueNode &tuningData, - const ControlValue &def) +int AwbAlgorithmBase::parseModeConfigs(const ValueNode &tuningData, + const ControlValue &def) { + if (!bayes_) { + /* AwbGrey does not support and does not use modes. */ + currentMode_ = &AwbGreyMode; + return 0; + } + std::vector availableModes; const ValueNode &modes = tuningData[controls::AwbMode.name()]; @@ -227,37 +578,83 @@ int AwbAlgorithm::parseModeConfigs(const ValueNode &tuningData, } controls_[&controls::AwbMode] = ControlInfo(availableModes, def); + currentMode_ = &modes_[controls::AwbAuto]; return 0; } /** - * \class AwbAlgorithm::ModeConfig - * \brief Holds the configuration of a single AWB mode + * \var AwbAlgorithmBase::controls_ + * \brief Controls info map for the controls registered by the awb algorithm + */ + +/** + * \var AwbAlgorithmBase::gainMin_ + * \brief The minimum supported gain value * - * AWB modes limit the regulation of the AWB algorithm to a specific range of - * colour temperatures. + * Minimum gain value used to clamp the awb algorithm calculation results in the + * range supported by the platform awb engine. + * + * The min and max gain values are initialized by AwbAlgorithm::init(). */ /** - * \var AwbAlgorithm::ModeConfig::ctLo + * \var AwbAlgorithmBase::gainMax_ + * \brief The maximum supported gain value + * + * Maximum gain value used to clamp the awb algorithm calculation results in the + * range supported by the platform awb engine. + * + * The min and max gain values are initialized by AwbAlgorithm::init(). + */ + +/* + * \class AwbAlgorithmBase::ModeConfig + * \brief Holds the configuration of a single awb mode + * + * AWB modes limit the regulation of the awb algorithm to a specific range of + * colour temperatures. Use by AwbBayes only. + */ + +/* + * \var AwbAlgorithmBase::ModeConfig::ctLo * \brief The lowest valid colour temperature of that mode */ -/** - * \var AwbAlgorithm::ModeConfig::ctHi +/* + * \var AwbAlgorithmBase::ModeConfig::ctHi * \brief The highest valid colour temperature of that mode */ +/* + * \var AwbAlgorithmBase::modes_ + * \brief Map of all configured modes + * \sa AwbAlgorithmBase::parseModeConfigs + */ + /** - * \var AwbAlgorithm::controls_ - * \brief Controls info map for the controls provided by the algorithm + * \class AwbAlgorithm + * \brief The libipa awb algorithm + * \tparam Q The fixedpoint register representation of gain values + * + * Implement the awb algorithm for libipa. + * + * The AwbAlgorithm class implements an interface similar in spirit to the one + * of the Algorithm class. IPA modules are expected to store an instance of + * AwbAlgorithm as class member, template it with the awb engine gain register + * representation and call its function in their implementations of the + * Algorithm interface. + * + * The AwbAlgorithm instantiates an AwbImplementation implementation (AwbGrey or + * AwbBayes) at init() time by parsing the tuning data and uses it to compute + * the RGB gains and estimate a colour temperature given a set of statistics + * in the form of a AwbStats derived implementation. */ /** - * \var AwbAlgorithm::modes_ - * \brief Map of all configured modes - * \sa AwbAlgorithm::parseModeConfigs + * \fn AwbAlgorithm::init() + * \param[in] controls The info map of the IPA controls + * \copydoc AwbAlgorithmBase::init() */ } /* namespace ipa */ diff --git a/src/ipa/libipa/awb.h b/src/ipa/libipa/awb.h index 09c00e47d604..f453e1f9fd3d 100644 --- a/src/ipa/libipa/awb.h +++ b/src/ipa/libipa/awb.h @@ -2,11 +2,12 @@ /* * Copyright (C) 2024 Ideas on Board Oy * - * Generic AWB algorithms + * libIPA AWB algorithms */ #pragma once +#include #include #include @@ -20,46 +21,119 @@ namespace libcamera { namespace ipa { -struct AwbResult { +namespace awb { + +struct Context { RGB gains; - double colourTemperature; + unsigned int temperatureK; +}; + +struct ActiveState { + Context manual; + Context automatic; + + bool autoEnabled; +}; + +struct FrameContext : public Context { + bool autoEnabled; }; +} /* namespace awb */ + struct AwbStats { - virtual double computeColourError(const RGB &gains) const = 0; - virtual RGB rgbMeans() const = 0; + AwbStats() = default; + AwbStats(const RGB &means); + virtual ~AwbStats() = default; + + bool valid() const; + + virtual double rgRatio() const { return rg_; } + virtual double bgRatio() const { return bg_; } + virtual double computeColourError(const RGB &gains) const; + virtual RGB rgbMeans() const; protected: - ~AwbStats() = default; + virtual double minColourValue() const = 0; + + RGB rgbMeans_; + double rg_; + double bg_; }; -class AwbAlgorithm +class AwbImplementation { public: - virtual ~AwbAlgorithm() = default; - + virtual ~AwbImplementation() = default; virtual int init(const ValueNode &tuningData) = 0; - virtual AwbResult calculateAwb(const AwbStats &stats, unsigned int lux) = 0; - virtual std::optional> gainsFromColourTemperature(double colourTemperature) = 0; + virtual awb::Context calculateAwb(const AwbStats &stats, unsigned int lux, + std::array ranges) = 0; + virtual std::optional> + gainsFromColourTemperature(double temperatureK) = 0; +}; - const ControlInfoMap::Map &controls() const - { - return controls_; - } +class AwbAlgorithmBase +{ +public: + int init(const ValueNode &tuningData); + + int configure(awb::ActiveState &state); + + void queueRequest(awb::ActiveState &state, + const uint32_t frame, + awb::FrameContext &frameContext, + const ControlList &controls); + + void prepare(awb::ActiveState &state, awb::FrameContext &frameContext); - virtual void handleControls([[maybe_unused]] const ControlList &controls) {} + void process(awb::ActiveState &state, awb::FrameContext &frameContext, + const AwbStats &stats, unsigned int lux, + ControlList &metadata); protected: - int parseModeConfigs(const ValueNode &tuningData, - const ControlValue &def = {}); + AwbAlgorithmBase() = default; + ControlInfoMap::Map controls_; + float gainMin_; + float gainMax_; + +private: struct ModeConfig { double ctHi; double ctLo; }; - ControlInfoMap::Map controls_; - std::map modes_; + /* AwbGrey does not support modes; */ + static constexpr ModeConfig AwbGreyMode = { 0.0, 0.0 }; + + int parseModeConfigs(const ValueNode &tuningData, + const ControlValue &def = {}); + + std::map modes_; + const ModeConfig *currentMode_ = nullptr; + std::unique_ptr impl_; + bool bayes_ = false; +}; + +template +class AwbAlgorithm : public AwbAlgorithmBase +{ +public: + int init(const ValueNode &tuningData, ControlInfoMap::Map &controls) + { + AwbAlgorithmBase::init(tuningData); + + gainMin_ = std::max(Q::TraitsType::min, 1.0f); + gainMax_ = Q::TraitsType::max; + + controls_[&controls::ColourGains] = + ControlInfo(gainMin_, gainMax_, + Span{ { 1.0f, 1.0f } }); + + controls.insert(controls_.begin(), controls_.end()); + + return 0; + } }; } /* namespace ipa */ diff --git a/src/ipa/libipa/awb_bayes.cpp b/src/ipa/libipa/awb_bayes.cpp index 9fd85e5a4505..34d86ce6938b 100644 --- a/src/ipa/libipa/awb_bayes.cpp +++ b/src/ipa/libipa/awb_bayes.cpp @@ -140,11 +140,6 @@ void Interpolator::interpolate(const Pwl &a, const Pwl &b, Pwl &dest, doubl * \brief How far to wander off CT curve towards "more green" */ -/** - * \var AwbBayes::currentMode_ - * \brief The currently selected mode - */ - int AwbBayes::init(const ValueNode &tuningData) { int ret = colourGainCurve_.readYaml(tuningData["colourGains"], "ct", "gains"); @@ -172,14 +167,6 @@ int AwbBayes::init(const ValueNode &tuningData) return ret; } - ret = parseModeConfigs(tuningData, controls::AwbAuto); - if (ret) { - LOG(Awb, Error) - << "Failed to parse mode parameter from tuning file"; - return ret; - } - currentMode_ = &modes_[controls::AwbAuto]; - transversePos_ = tuningData["transversePos"].get(0.01); transverseNeg_ = tuningData["transverseNeg"].get(0.01); if (transversePos_ <= 0 || transverseNeg_ <= 0) { @@ -260,18 +247,6 @@ int AwbBayes::readPriors(const ValueNode &tuningData) return 0; } -void AwbBayes::handleControls(const ControlList &controls) -{ - auto mode = controls.get(controls::AwbMode); - if (mode) { - auto it = modes_.find(static_cast(*mode)); - if (it != modes_.end()) - currentMode_ = &it->second; - else - LOG(Awb, Error) << "Unsupported AWB mode " << *mode; - } -} - std::optional> AwbBayes::gainsFromColourTemperature(double colourTemperature) { /* @@ -283,7 +258,9 @@ std::optional> AwbBayes::gainsFromColourTemperature(double colourTem return RGB{ { gains[0], 1.0, gains[1] } }; } -AwbResult AwbBayes::calculateAwb(const AwbStats &stats, unsigned int lux) +awb::Context +AwbBayes::calculateAwb(const AwbStats &stats, unsigned int lux, + std::array ranges) { ipa::Pwl prior; if (lux > 0) { @@ -295,7 +272,7 @@ AwbResult AwbBayes::calculateAwb(const AwbStats &stats, unsigned int lux) prior.append(0, 1.0); } - double t = coarseSearch(prior, stats); + double t = coarseSearch(prior, stats, ranges); double r = ctR_.eval(t); double b = ctB_.eval(t); LOG(Awb, Debug) @@ -316,14 +293,15 @@ AwbResult AwbBayes::calculateAwb(const AwbStats &stats, unsigned int lux) << "After fine search: r " << r << " b " << b << " (gains r " << 1 / r << " b " << 1 / b << ")"; - return { { { 1.0 / r, 1.0, 1.0 / b } }, t }; + return { { { 1.0 / r, 1.0, 1.0 / b } }, static_cast(t) }; } -double AwbBayes::coarseSearch(const ipa::Pwl &prior, const AwbStats &stats) const +double AwbBayes::coarseSearch(const ipa::Pwl &prior, const AwbStats &stats, + std::array ranges) const { std::vector points; size_t bestPoint = 0; - double t = currentMode_->ctLo; + double t = ranges[0]; int spanR = -1; int spanB = -1; LimitsRecorder errorLimits; @@ -345,14 +323,14 @@ double AwbBayes::coarseSearch(const ipa::Pwl &prior, const AwbStats &stats) cons if (points.back().y() < points[bestPoint].y()) bestPoint = points.size() - 1; - if (t == currentMode_->ctHi) + if (t == ranges[1]) break; /* * Ensure even steps along the r/b curve by scaling them by the * current t. */ - t = std::min(t + t / 10 * kSearchStep, currentMode_->ctHi); + t = std::min(t + t / 10 * kSearchStep, ranges[1]); } t = points[bestPoint].x(); diff --git a/src/ipa/libipa/awb_bayes.h b/src/ipa/libipa/awb_bayes.h index 1e3373676bc0..8264bcb09f38 100644 --- a/src/ipa/libipa/awb_bayes.h +++ b/src/ipa/libipa/awb_bayes.h @@ -20,22 +20,23 @@ namespace libcamera { namespace ipa { -class AwbBayes : public AwbAlgorithm +class AwbBayes : public AwbImplementation { public: AwbBayes() = default; int init(const ValueNode &tuningData) override; - AwbResult calculateAwb(const AwbStats &stats, unsigned int lux) override; + awb::Context calculateAwb(const AwbStats &stats, unsigned int lux, + std::array ranges) override; std::optional> gainsFromColourTemperature(double temperatureK) override; - void handleControls(const ControlList &controls) override; private: int readPriors(const ValueNode &tuningData); void fineSearch(double &t, double &r, double &b, ipa::Pwl const &prior, const AwbStats &stats) const; - double coarseSearch(const ipa::Pwl &prior, const AwbStats &stats) const; + double coarseSearch(const ipa::Pwl &prior, const AwbStats &stats, + std::array ranges) const; double interpolateQuadratic(ipa::Pwl::Point const &a, ipa::Pwl::Point const &b, ipa::Pwl::Point const &c) const; @@ -50,8 +51,6 @@ private: double transversePos_; double transverseNeg_; - - ModeConfig *currentMode_ = nullptr; }; } /* namespace ipa */ diff --git a/src/ipa/libipa/awb_grey.cpp b/src/ipa/libipa/awb_grey.cpp index b14b096810ae..2e31492bda3c 100644 --- a/src/ipa/libipa/awb_grey.cpp +++ b/src/ipa/libipa/awb_grey.cpp @@ -60,6 +60,7 @@ int AwbGrey::init(const ValueNode &tuningData) * \brief Calculate AWB data from the given statistics * \param[in] stats The statistics to use for the calculation * \param[in] lux The lux value of the scene + * \param[in] ranges The colour temperature search limits (AwbBayes only) * * The colour temperature is estimated based on the colours::estimateCCT() * function. The gains are calculated purely based on the RGB means provided by @@ -70,20 +71,23 @@ int AwbGrey::init(const ValueNode &tuningData) * * \return The AWB result */ -AwbResult AwbGrey::calculateAwb(const AwbStats &stats, [[maybe_unused]] unsigned int lux) +awb::Context +AwbGrey::calculateAwb(const AwbStats &stats, [[maybe_unused]] unsigned int lux, + [[maybe_unused]] std::array ranges) { - AwbResult result; + awb::Context result; auto means = stats.rgbMeans(); - result.colourTemperature = estimateCCT(means); + result.temperatureK = estimateCCT(means); /* - * Estimate the red and blue gains to apply in a grey world. The green - * gain is hardcoded to 1.0. Avoid divisions by zero by clamping the - * divisor to a minimum value of 1.0. + * Calculate the red and blue gains to apply in a grey world by simply + * inverting the red/green and blue/green ratios as reported in + * statistics. The green gain is hardcoded to 1.0. */ - result.gains.r() = means.g() / std::max(means.r(), 1.0); + result.gains.r() = 1.0 / stats.rgRatio(); result.gains.g() = 1.0; - result.gains.b() = means.g() / std::max(means.b(), 1.0); + result.gains.b() = 1.0 / stats.bgRatio(); + return result; } @@ -96,12 +100,13 @@ AwbResult AwbGrey::calculateAwb(const AwbStats &stats, [[maybe_unused]] unsigned * gains configured in the colour temperature curve. * * \return The colour gains if a colour temperature curve is available, - * [1, 1, 1] otherwise. + * std::nullopt otherwise */ std::optional> AwbGrey::gainsFromColourTemperature(double colourTemperature) { if (!colourGainCurve_) { - LOG(Awb, Error) << "No gains defined"; + LOG(Awb, Info) << "No gains curve defined, " + << "unable to interpolate gains to colour temperature"; return std::nullopt; } diff --git a/src/ipa/libipa/awb_grey.h b/src/ipa/libipa/awb_grey.h index 154a2af97f15..438fb850c223 100644 --- a/src/ipa/libipa/awb_grey.h +++ b/src/ipa/libipa/awb_grey.h @@ -18,13 +18,14 @@ namespace libcamera { namespace ipa { -class AwbGrey : public AwbAlgorithm +class AwbGrey : public AwbImplementation { public: AwbGrey() = default; int init(const ValueNode &tuningData) override; - AwbResult calculateAwb(const AwbStats &stats, unsigned int lux) override; + awb::Context calculateAwb(const AwbStats &stats, unsigned int lux, + std::array ranges) override; std::optional> gainsFromColourTemperature(double colourTemperature) override; private: diff --git a/src/ipa/rkisp1/algorithms/awb.cpp b/src/ipa/rkisp1/algorithms/awb.cpp index 5ae5b6471643..0ad0794b4a0f 100644 --- a/src/ipa/rkisp1/algorithms/awb.cpp +++ b/src/ipa/rkisp1/algorithms/awb.cpp @@ -8,17 +8,12 @@ #include "awb.h" #include -#include #include -#include - #include -#include "libipa/awb_bayes.h" -#include "libipa/awb_grey.h" -#include "libipa/colours.h" +#include "libcamera/internal/vector.h" /** * \file awb.h @@ -28,54 +23,29 @@ namespace libcamera { namespace ipa::rkisp1::algorithms { -/** - * \class Awb - * \brief Manage the white balance with automatic and manual controls - */ - LOG_DEFINE_CATEGORY(RkISP1Awb) -constexpr int32_t kMinColourTemperature = 2500; -constexpr int32_t kMaxColourTemperature = 10000; -constexpr int32_t kDefaultColourTemperature = 5000; - -/* Minimum mean value below which AWB can't operate. */ -constexpr double kMeanMinThreshold = 2.0; - class RkISP1AwbStats final : public AwbStats { public: - RkISP1AwbStats(const RGB &rgbMeans) - : rgbMeans_(rgbMeans) + RkISP1AwbStats() = default; + RkISP1AwbStats(const RGB means) + : AwbStats(means) { - rg_ = rgbMeans_.r() / rgbMeans_.g(); - bg_ = rgbMeans_.b() / rgbMeans_.g(); } - double computeColourError(const RGB &gains) const override + /* Minimum mean value below which AWB can't operate. */ + double minColourValue() const override { - /* - * Compute the sum of the squared colour error (non-greyness) as - * it appears in the log likelihood equation. - */ - double deltaR = gains.r() * rg_ - 1.0; - double deltaB = gains.b() * bg_ - 1.0; - double delta2 = deltaR * deltaR + deltaB * deltaB; - - return delta2; + return 2.0; } - - RGB rgbMeans() const override - { - return rgbMeans_; - } - -private: - RGB rgbMeans_; - double rg_; - double bg_; }; +/** + * \class Awb + * \brief Manage the white balance with automatic and manual controls + */ + Awb::Awb() : rgbMode_(false) { @@ -86,40 +56,7 @@ Awb::Awb() */ int Awb::init(IPAContext &context, const ValueNode &tuningData) { - auto &cmap = context.ctrlMap; - cmap[&controls::ColourTemperature] = ControlInfo(kMinColourTemperature, - kMaxColourTemperature, - kDefaultColourTemperature); - cmap[&controls::AwbEnable] = ControlInfo(false, true); - - cmap[&controls::ColourGains] = ControlInfo(0.0f, 3.996f, - Span{ { 1.0f, 1.0f } }); - - if (!tuningData.contains("algorithm")) - LOG(RkISP1Awb, Info) << "No AWB algorithm specified." - << " Default to grey world"; - - auto mode = tuningData["algorithm"].get("grey"); - if (mode == "grey") { - awbAlgo_ = std::make_unique(); - } else if (mode == "bayes") { - awbAlgo_ = std::make_unique(); - } else { - LOG(RkISP1Awb, Error) << "Unknown AWB algorithm: " << mode; - return -EINVAL; - } - LOG(RkISP1Awb, Debug) << "Using AWB algorithm: " << mode; - - int ret = awbAlgo_->init(tuningData); - if (ret) { - LOG(RkISP1Awb, Error) << "Failed to init AWB algorithm"; - return ret; - } - - const auto &src = awbAlgo_->controls(); - cmap.insert(src.begin(), src.end()); - - return 0; + return awbAlgo_.init(tuningData, context.ctrlMap); } /** @@ -128,16 +65,7 @@ int Awb::init(IPAContext &context, const ValueNode &tuningData) int Awb::configure(IPAContext &context, const IPACameraSensorInfo &configInfo) { - context.activeState.awb.manual.gains = RGB{ 1.0 }; - auto gains = awbAlgo_->gainsFromColourTemperature(kDefaultColourTemperature); - if (gains) - context.activeState.awb.automatic.gains = *gains; - else - context.activeState.awb.automatic.gains = RGB{ 1.0 }; - - context.activeState.awb.autoEnabled = true; - context.activeState.awb.manual.temperatureK = kDefaultColourTemperature; - context.activeState.awb.automatic.temperatureK = kDefaultColourTemperature; + awbAlgo_.configure(context.activeState.awb); /* * Define the measurement window for AWB as a centered rectangle @@ -156,56 +84,12 @@ int Awb::configure(IPAContext &context, /** * \copydoc libcamera::ipa::Algorithm::queueRequest */ -void Awb::queueRequest(IPAContext &context, - [[maybe_unused]] const uint32_t frame, +void Awb::queueRequest(IPAContext &context, const uint32_t frame, IPAFrameContext &frameContext, const ControlList &controls) { - auto &awb = context.activeState.awb; - - const auto &awbEnable = controls.get(controls::AwbEnable); - if (awbEnable && *awbEnable != awb.autoEnabled) { - awb.autoEnabled = *awbEnable; - - LOG(RkISP1Awb, Debug) - << (*awbEnable ? "Enabling" : "Disabling") << " AWB"; - } - - awbAlgo_->handleControls(controls); - - frameContext.awb.autoEnabled = awb.autoEnabled; - - if (awb.autoEnabled) - return; - - const auto &colourGains = controls.get(controls::ColourGains); - const auto &colourTemperature = controls.get(controls::ColourTemperature); - bool update = false; - if (colourGains) { - awb.manual.gains.r() = (*colourGains)[0]; - awb.manual.gains.b() = (*colourGains)[1]; - /* - * \todo Colour temperature reported in metadata is now - * incorrect, as we can't deduce the temperature from the gains. - * This will be fixed with the bayes AWB algorithm. - */ - update = true; - } else if (colourTemperature) { - awb.manual.temperatureK = *colourTemperature; - const auto &gains = awbAlgo_->gainsFromColourTemperature(*colourTemperature); - if (gains) { - awb.manual.gains.r() = gains->r(); - awb.manual.gains.b() = gains->b(); - update = true; - } - } - - if (update) - LOG(RkISP1Awb, Debug) - << "Set colour gains to " << awb.manual.gains; - - frameContext.awb.gains = awb.manual.gains; - frameContext.awb.temperatureK = awb.manual.temperatureK; + awbAlgo_.queueRequest(context.activeState.awb, frame, frameContext.awb, + controls); } /** @@ -214,15 +98,7 @@ void Awb::queueRequest(IPAContext &context, void Awb::prepare(IPAContext &context, const uint32_t frame, IPAFrameContext &frameContext, RkISP1Params *params) { - /* - * This is the latest time we can read the active state. This is the - * most up-to-date automatic values we can read. - */ - if (frameContext.awb.autoEnabled) { - const auto &awb = context.activeState.awb; - frameContext.awb.gains = awb.automatic.gains; - frameContext.awb.temperatureK = awb.automatic.temperatureK; - } + awbAlgo_.prepare(context.activeState.awb, frameContext.awb); auto gainConfig = params->block(); gainConfig.setEnabled(true); @@ -291,68 +167,27 @@ void Awb::process(IPAContext &context, const rkisp1_stat_buffer *stats, ControlList &metadata) { - IPAActiveState &activeState = context.activeState; + RkISP1AwbStats awbStats = calculateRgbMeans(frameContext, stats); - metadata.set(controls::AwbEnable, frameContext.awb.autoEnabled); - metadata.set(controls::ColourGains, { - static_cast(frameContext.awb.gains.r()), - static_cast(frameContext.awb.gains.b()) - }); - metadata.set(controls::ColourTemperature, frameContext.awb.temperatureK); + awbAlgo_.process(context.activeState.awb, frameContext.awb, awbStats, + frameContext.lux.lux, metadata); +} +RkISP1AwbStats Awb::calculateRgbMeans(const IPAFrameContext &frameContext, + const rkisp1_stat_buffer *stats) const +{ if (!stats || !(stats->meas_type & RKISP1_CIF_ISP_STAT_AWB)) { LOG(RkISP1Awb, Error) << "AWB data is missing in statistics"; - return; + return {}; } - const rkisp1_cif_isp_stat *params = &stats->params; - const rkisp1_cif_isp_awb_stat *awb = ¶ms->awb; + const rkisp1_cif_isp_awb_stat *awb = &stats->params.awb; if (awb->awb_mean[0].cnt == 0) { LOG(RkISP1Awb, Debug) << "AWB statistics are empty"; - return; + return {}; } - RGB rgbMeans = calculateRgbMeans(frameContext, awb); - - /* - * If the means are too small we don't have enough information to - * meaningfully calculate gains. Freeze the algorithm in that case. - */ - if (rgbMeans.r() < kMeanMinThreshold && rgbMeans.g() < kMeanMinThreshold && - rgbMeans.b() < kMeanMinThreshold) - return; - - RkISP1AwbStats awbStats{ rgbMeans }; - AwbResult awbResult = awbAlgo_->calculateAwb(awbStats, frameContext.lux.lux); - - /* - * Clamp the gain values to the hardware, which expresses gains as Q2.8 - * unsigned integer values. Set the minimum just above zero to avoid - * divisions by zero when computing the raw means in subsequent - * iterations. - */ - awbResult.gains = awbResult.gains.max(1.0 / 256).min(1023.0 / 256); - - /* Filter the values to avoid oscillations. */ - double speed = 0.2; - double ct = awbResult.colourTemperature; - ct = ct * speed + activeState.awb.automatic.temperatureK * (1 - speed); - awbResult.gains = awbResult.gains * speed + - activeState.awb.automatic.gains * (1 - speed); - - activeState.awb.automatic.temperatureK = static_cast(ct); - activeState.awb.automatic.gains = awbResult.gains; - - LOG(RkISP1Awb, Debug) - << std::showpoint - << "Means " << rgbMeans << ", gains " - << activeState.awb.automatic.gains << ", temp " - << activeState.awb.automatic.temperatureK << "K"; -} - -RGB Awb::calculateRgbMeans(const IPAFrameContext &frameContext, const rkisp1_cif_isp_awb_stat *awb) const -{ Vector rgbMeans; if (rgbMode_) { @@ -419,7 +254,7 @@ RGB Awb::calculateRgbMeans(const IPAFrameContext &frameContext, const rk */ rgbMeans /= frameContext.awb.gains.max(0.01); - return rgbMeans; + return RkISP1AwbStats(rgbMeans); } REGISTER_IPA_ALGORITHM(Awb, "Awb") diff --git a/src/ipa/rkisp1/algorithms/awb.h b/src/ipa/rkisp1/algorithms/awb.h index 60d9ef111495..c7cb59a75bde 100644 --- a/src/ipa/rkisp1/algorithms/awb.h +++ b/src/ipa/rkisp1/algorithms/awb.h @@ -7,17 +7,25 @@ #pragma once -#include "libcamera/internal/vector.h" +#include + +#include + +#include "libcamera/internal/value_node.h" #include "libipa/awb.h" -#include "libipa/interpolator.h" +#include "libipa/fixedpoint.h" #include "algorithm.h" +#include "ipa_context.h" +#include "params.h" namespace libcamera { namespace ipa::rkisp1::algorithms { +class RkISP1AwbStats; + class Awb : public Algorithm { public: @@ -38,10 +46,10 @@ public: ControlList &metadata) override; private: - RGB calculateRgbMeans(const IPAFrameContext &frameContext, - const rkisp1_cif_isp_awb_stat *awb) const; + RkISP1AwbStats calculateRgbMeans(const IPAFrameContext &frameContext, + const rkisp1_stat_buffer *stats) const; - std::unique_ptr awbAlgo_; + AwbAlgorithm> awbAlgo_; bool rgbMode_; }; diff --git a/src/ipa/rkisp1/ipa_context.h b/src/ipa/rkisp1/ipa_context.h index e5f70d934d67..c36c1f7e0084 100644 --- a/src/ipa/rkisp1/ipa_context.h +++ b/src/ipa/rkisp1/ipa_context.h @@ -25,6 +25,7 @@ #include "libcamera/internal/vector.h" #include "libipa/agc_mean_luminance.h" +#include "libipa/awb.h" #include "libipa/camera_sensor_helper.h" #include "libipa/fc_queue.h" #include "libipa/fixedpoint.h" @@ -48,15 +49,17 @@ struct IPAHwSettings { bool compand; }; +struct RKISP1AwbSession { + struct rkisp1_cif_isp_window measureWindow; + bool enabled; +}; + struct IPASessionConfiguration { struct { struct rkisp1_cif_isp_window measureWindow; } agc; - struct { - struct rkisp1_cif_isp_window measureWindow; - bool enabled; - } awb; + struct RKISP1AwbSession awb; struct { bool supported; @@ -100,17 +103,7 @@ struct IPAActiveState { utils::Duration maxFrameDuration; } agc; - struct { - struct AwbState { - RGB gains; - unsigned int temperatureK; - }; - - AwbState manual; - AwbState automatic; - - bool autoEnabled; - } awb; + ipa::awb::ActiveState awb; struct { Matrix manual; @@ -176,11 +169,7 @@ struct IPAFrameContext : public FrameContext { bool autoGainModeChange; } agc; - struct { - RGB gains; - bool autoEnabled; - unsigned int temperatureK; - } awb; + ipa::awb::FrameContext awb; struct { float actualBrightness; From patchwork Thu Jun 18 10:18:44 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jacopo Mondi X-Patchwork-Id: 26920 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 3390CC3303 for ; Thu, 18 Jun 2026 10:19:06 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 24ABD6299E; Thu, 18 Jun 2026 12:19:01 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="O25FVRPY"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 04029629A2 for ; Thu, 18 Jun 2026 12:18:56 +0200 (CEST) Received: from [192.168.125.177] (mob-109-113-4-199.net.vodafone.it [109.113.4.199]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id C7CF82590; Thu, 18 Jun 2026 12:18:20 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1781777901; bh=Z7lE/WfKo0sHH63MuF6EmA1ItDT3yRfiduAdkaiijHw=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=O25FVRPYWixraPWtAU7PQTSIzQyl12WYJn+Iz4IYnokbDkOUhkjruAHtl+RoOTMv0 /cqjI/M7OOogumPDPtR0OA6CCTzPxX/LrUzUsPycKoH11Jf8dbT2ugD60Z1FfhQBNj rCKHLP/Rm09AY6htnWUp48ebC5T8V37KyiH6tLJ4= From: Jacopo Mondi Date: Thu, 18 Jun 2026 12:18:44 +0200 Subject: [PATCH 05/14] ipa: libipa: Add CcmAlgorithm to libipa MIME-Version: 1.0 Message-Id: <20260618-rppx1-ipa-v1-5-32337264cfcd@ideasonboard.com> References: <20260618-rppx1-ipa-v1-0-32337264cfcd@ideasonboard.com> In-Reply-To: <20260618-rppx1-ipa-v1-0-32337264cfcd@ideasonboard.com> To: =?utf-8?q?Niklas_S=C3=B6derlund?= , libcamera-devel@lists.libcamera.org Cc: Jacopo Mondi X-Mailer: b4 0.14.3 X-Developer-Signature: v=1; a=openpgp-sha256; l=12610; i=jacopo.mondi@ideasonboard.com; h=from:subject:message-id; bh=Z7lE/WfKo0sHH63MuF6EmA1ItDT3yRfiduAdkaiijHw=; b=owEBbQKS/ZANAwAKAXI0Bo8WoVY8AcsmYgBqM8YJ3o08FccxEMHSnNKxtFLICcYrfcDgHT8YH MBo0S0bnfeJAjMEAAEKAB0WIQS1xD1IgJogio9YOMByNAaPFqFWPAUCajPGCQAKCRByNAaPFqFW PKhwEACSwAgykIyqFT6qUs9BeS13tloqvx4zhClhMDSwZbW0j3D+ukLvR+tljoHslh/RsxXWeY0 l+qxSj30azunG5qYMr9hZrEJqHb7tXgJzQrJJXJD3muQOJGBHJSdyOxnVHZ95z05BQqgxu6c5rz SiCq8Uwhe3bD2a/x3EV+BAF5ospUs9GcaIdF4cDSC2GsS7e08VA/YLHgKyhEwRz1L6dd3sztLZS M5+Y853qS+pwd2OS9XROJ3Gs8HikszATmtisnZRJ7tdp3kbn/dAuu5Ppj1IwIgd2K7hynrfHW6/ 9jyqkv1sHkvtWDJ3GtVcbaqXW0cMlfPpjv7fxRI4mzVypGun7tzcwmKVzrcnDEpNI5NcL9krine d+X/HhjNxbWcHVQgu54VxzH6JSMRBfSV5sdG1ZgPQG/0Ogg1EFzPAOtOEumM+RBfnTDJlxmI6uY 7uNHu9lk71ylsytj7M4pk1ilixO7MF2FMnmyeCbHRKMAXnO9ZWZn1rLT7g9BCFOlRvejxioCGdW sDA7qes9p6MqaEI6i6e2PShT1Pnr7lnCWE/D+HjaTMXVjPqRGDF4mdC+i+rpEXSwXSnu3msvCE4 ThbJb5alFYAxlrbwfXTG08P607p3BhCX1auUvgQV3MPwKbtIx9gx9Hu//8KRvWHwvLOGpO/zAyf W7Vpb6HAA7YGcog== X-Developer-Key: i=jacopo.mondi@ideasonboard.com; a=openpgp; fpr=72392EDC88144A65C701EA9BA5826A2587AD026B X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Add a CcmAlgorithm to libipa. The CcmAlgorithm performs interpolation of the colour correction matrices as loaded from tuning file on a colour temperature. The implementation is based on the existing RkISP1 CCM algorihtm. Signed-off-by: Jacopo Mondi --- src/ipa/libipa/ccm.cpp | 265 +++++++++++++++++++++++++++++++++++++++++++++ src/ipa/libipa/ccm.h | 84 ++++++++++++++ src/ipa/libipa/meson.build | 2 + 3 files changed, 351 insertions(+) diff --git a/src/ipa/libipa/ccm.cpp b/src/ipa/libipa/ccm.cpp new file mode 100644 index 000000000000..9fc8982e5a95 --- /dev/null +++ b/src/ipa/libipa/ccm.cpp @@ -0,0 +1,265 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2026 Ideas on Board Oy + * + * libIPA CCM algorithm + */ + +#include "ccm.h" + +/** + * \file ccm.h + * \brief libipa ccm (Colour Correction Matrix) algorithm + */ + +namespace libcamera { + +namespace ipa { + +LOG_DEFINE_CATEGORY(Ccm) + +namespace ccm { + +/** + * \struct ActiveState + * \brief Active ccm state + * + * \var ActiveState::manual + * \brief The most recent manually requested ccm state + * + * \var ActiveState::automatic + * \brief The most recent automatically calculated ccm state + */ + +/** + * \struct CcmContext + * \brief Ccm coefficients and offsets + * + * \var CcmContext::ccm + * \brief Matrix of 3x3 ccm coefficients + * + * \var CcmContext::offsets + * \brief Vector of RGB ccm offsets + */ + +/** + * \typedef FrameContext + * \brief Per-frame ccm state + */ + +} /* namespace ccm */ + +/** + * \class CcmAlgorithmBase + * \brief Base class for CcmAlgorithm for non-templated functions implementation + * + * Base class for CcmAlgorithm where non-templated functions are implemented. + * IPA implementations shall use CcmAlgorithm and not this class. + */ + +/** + * \brief Initialize the algorithm with the given tuning data + * \param[in] tuningData The tuning data to use for the algorithm + * + * Parse \a tuningData to initialize the ccm algorithm and register controls. + * IPA modules are expected to call this function as part of their + * implementation of Algorithm::init(). + * + * \return 0 on success, a negative error code otherwise + */ +int CcmAlgorithmBase::init(const ValueNode &tuningData) +{ + int ret = ccm_.readYaml(tuningData["ccms"], "ct", "ccm"); + if (ret < 0) { + LOG(Ccm, Warning) + << "Failed to parse 'ccm' " + << "parameter from tuning file; falling back to unit matrix"; + ccm_.setData({ { 0, Matrix::identity() } }); + } + + ret = offsets_.readYaml(tuningData["ccms"], "ct", "offsets"); + if (ret < 0) { + LOG(Ccm, Warning) + << "Failed to parse 'offsets' " + << "parameter from tuning file; falling back to zero offsets"; + + offsets_.setData({ { 0, Matrix({ 0, 0, 0 }) } }); + } + + return 0; +} + +/** + * \brief Configure the ccm algorithm + * \param[in] state The ccm active state + * \param[in] temperatureK The colour temperature in Kelvin + * + * Configure the ccm algorithm by initializing the manual and automatic + * states in \a state by interpolating the default colour correction matrix + * with the given colour temperature \a temperatureK. + * + * \return 0 if successful, an error code otherwise + */ +int CcmAlgorithmBase::configure(ccm::ActiveState &state, unsigned int temperatureK) +{ + state.manual.ccm = ccm_.getInterpolated(temperatureK); + state.manual.offsets = offsets_.getInterpolated(temperatureK); + state.automatic.ccm = ccm_.getInterpolated(temperatureK); + state.automatic.offsets = offsets_.getInterpolated(temperatureK); + + return 0; +} + +/** + * \brief Queue a Request to the ccm algorithm + * \param[in] state The ccm active state + * \param[in] context The ccm frame context + * \param[in] controls The list of controls part of the Request + * + * Queue a new Request to the ccm algorithm and store the manual colour + * correction matrix and temperature in \a frameContext. + * + * The currently handled controls are: + * - controls::ColourTemperature + * - controls::ColourCorrectionMatrix + * + * When controls::ColourCorrectionMatrix is passed in the supplied matrix is + * stored in \a state and \a context. + * + * When controls::ColourTemperature is passed in, the matrices loaded from + * configuration file are interpolated with the given temperature and the result + * is stored in \a state and \a context. + * + * If the IPA is running in manual mode, the IPA ccm algorithm implementations + * can use the matrix coefficients and offsets directly from \a context after + * calling this function to program the HW ccm engine, without calling prepare(). + */ +void CcmAlgorithmBase::queueRequest(ccm::ActiveState &state, + ccm::FrameContext &context, + const ControlList &controls) +{ + const auto &colourTemperature = controls.get(controls::ColourTemperature); + const auto &ccmMatrix = controls.get(controls::ColourCorrectionMatrix); + if (ccmMatrix) { + state.manual.ccm = Matrix(*ccmMatrix); + LOG(Ccm, Debug) << "Setting manual CCM from CCM control to " + << state.manual.ccm; + } else if (colourTemperature) { + state.manual.ccm = ccm_.getInterpolated(*colourTemperature); + LOG(Ccm, Debug) << "Setting manual CCM from CT control to " + << state.manual.ccm; + } + + context = state.manual; +} + +/** + * \brief Calculate the matrix coefficients for a colour temperature + * \param[in] state The ccm active state + * \param[in] context The ccm frame context + * \param[in] frame The frame number + * \param[in] temperatureK The colour temperature in Kelvin + * + * Interpolate the colour correction matrices as loaded from configuration file + * for colour temperature \a temperatureK. + * + * The function shall only be called if the IPA algorithm is running in auto + * mode. If running in manual mode the application supplied correction matrix is + * stored in \a frameContext at queueRequest() time. + */ +void CcmAlgorithmBase::prepare(ccm::ActiveState &state, + ccm::FrameContext &context, + unsigned int frame, unsigned int temperatureK) +{ + if (frame > 0 && temperatureK == ct_) { + context = state.automatic; + return; + } + + ct_ = temperatureK; + context.ccm = ccm_.getInterpolated(ct_); + context.offsets = offsets_.getInterpolated(ct_); + + state.automatic = context; +} + +/** + * \brief Populate metadata with the latest correction matrix coefficients + * \param[in] context The ccm frame context + * \param[out] metadata The metadata list + */ +void CcmAlgorithmBase::process(ccm::FrameContext &context, ControlList &metadata) +{ + metadata.set(controls::ColourCorrectionMatrix, context.ccm.data()); +} + +/** + * \var CcmAlgorithmBase::coeffMin_ + * \brief The minimum supported coefficients value + * + * Minimum coefficient value used to clamp the ccm algorithm calculation results + * in the range supported by the platform ccm engine. + * + * The min and max gain values are initialized by CcmAlgorithm::init(). + */ + +/** + * \var CcmAlgorithmBase::coeffMax_ + * \brief The maximum supported coefficients value + * + * Maximum coefficient value used to clamp the ccm algorithm calculation results + * in the range supported by the platform ccm engine. + * + * The min and max gain values are initialized by CcmAlgorithm::init(). + */ + +/** + * \class CcmAlgorithm + * \brief The libipa ccm algorithm + * \tparam Q The fixedpoint register representation of the colour correction + * coefficients + * + * Implement the ccm algorithm for libipa. + * + * The CcmAlgorithm class implement an interface similar in spirit to the one + * of the Algorithm class. IPA modules are expected to store an instance of + * CcmAlgorithm as class member, template it with the ccm coefficients register + * representation and call its function in their implementations of the + * Algorithm interface. + * + * The CcmAlgorithm class provides an init() function where tuning data is + * parsed and the per-colour temperature correction matrices are loaded from + * the tuning file. + * + * CcmAlgorithm supports both automatic and manual colour correction operations, + * but doesn't offer a way to select one of them. Enabling or disabling + * automatic ccm operations usually goes through the Awb algorithm + * enable/disable as the two algorithms should work with the same mode. + * + * When the Awb algorithm runs in manual mode a custom colour correction matrix + * or a custom colour temperature can be supplied to the ccm algorithm at + * queueRequest() time. If the Request contains a color correction matrix + * (controls::ColourCorrectionMatrix) then the matrix coefficients gets saved in + * the FrameContext and the IPA module can immediately use them and doesn't need + * to call process(). If a custom colour temperature is provided + * (controls::ColourTemperature) then the matrices loaded from configuration are + * interpolated with it and the result is saved in the FrameContext. In this + * case as well IPA modules can use the result immediately and should avoid + * calling process(). + * + * When the Awb algorithm runs in automatic mode instead, it estimates the scene + * colour temperature. The estimated colour temperature shall be passed to + * process(), where it is used to interpolate the matrices loaded from the + * tuning file. The resulting coefficients are stored in the FrameContext for + * the IPA algorithm to use them to program their ccm engine registers. + */ + +/** + * \fn CcmAlgorithm::init() + * \param[in] controls The info map of the IPA controls + * \copydoc CcmAlgorithmBase::init() + */ + +} /* namespace ipa */ + +} /* namespace libcamera */ diff --git a/src/ipa/libipa/ccm.h b/src/ipa/libipa/ccm.h new file mode 100644 index 000000000000..6bb519d20dda --- /dev/null +++ b/src/ipa/libipa/ccm.h @@ -0,0 +1,84 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2026 Ideas on Board Oy + * + * libIPA CCM algorithm + */ + +#pragma once + +#include +#include + +#include "libcamera/internal/matrix.h" + +#include "fixedpoint.h" +#include "interpolator.h" + +namespace libcamera { + +namespace ipa { + +namespace ccm { + +struct CcmContext { + Matrix ccm; + Matrix offsets; +}; + +struct ActiveState { + struct CcmContext manual; + struct CcmContext automatic; +}; + +using FrameContext = CcmContext; + +} /* namespace ccm */ + +class CcmAlgorithmBase +{ +public: + int init(const ValueNode &tuningData); + int configure(ccm::ActiveState &state, unsigned int temperatureK); + void queueRequest(ccm::ActiveState &state, ccm::FrameContext &context, + const ControlList &controls); + + void prepare(ccm::ActiveState &state, ccm::FrameContext &context, + unsigned int frame, unsigned int temperatureK); + void process(ccm::FrameContext &context, ControlList &metadata); + +protected: + float coeffMin_; + float coeffMax_; + +private: + unsigned int ct_; + Interpolator> ccm_; + Interpolator> offsets_; +}; + +template +class CcmAlgorithm : public CcmAlgorithmBase +{ +public: + int init(const ValueNode &tuningData, ControlInfoMap::Map &controls) + { + int ret = CcmAlgorithmBase::init(tuningData); + if (ret) + return ret; + + coeffMin_ = Q::TraitsType::min; + coeffMax_ = Q::TraitsType::max; + + controls[&controls::ColourCorrectionMatrix] = + ControlInfo(ControlValue(coeffMin_), + ControlValue(coeffMax_), + ControlValue(Matrix::identity().data())); + + return 0; + } +}; + +} /* namespace ipa */ + +} /* namespace libcamera */ diff --git a/src/ipa/libipa/meson.build b/src/ipa/libipa/meson.build index 963c5ee73063..edf8eabd8b78 100644 --- a/src/ipa/libipa/meson.build +++ b/src/ipa/libipa/meson.build @@ -7,6 +7,7 @@ libipa_headers = files([ 'awb_grey.h', 'awb.h', 'camera_sensor_helper.h', + 'ccm.h', 'colours.h', 'exposure_mode_helper.h', 'fc_queue.h', @@ -28,6 +29,7 @@ libipa_sources = files([ 'awb_grey.cpp', 'awb.cpp', 'camera_sensor_helper.cpp', + 'ccm.cpp', 'colours.cpp', 'exposure_mode_helper.cpp', 'fc_queue.cpp', From patchwork Thu Jun 18 10:18:45 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jacopo Mondi X-Patchwork-Id: 26921 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 6ECB8C3304 for ; Thu, 18 Jun 2026 10:19:07 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 969B5629DF; Thu, 18 Jun 2026 12:19:02 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="Vk0iWCAY"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id B3CA162988 for ; Thu, 18 Jun 2026 12:18:56 +0200 (CEST) Received: from [192.168.125.177] (mob-109-113-4-199.net.vodafone.it [109.113.4.199]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 70669DF3; Thu, 18 Jun 2026 12:18:21 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1781777901; bh=7KnAlTUERTSBesKy8vBBMYA7iy8uY418g5ZiePex/Yk=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=Vk0iWCAYJkDmHfqdqCoKktnsnIqXFV1NB2CqrNxONMPg6xcpGtX9SZM7aArILL3MW DN5hJztIhBrstmyLczfmQnGhJpeQRXV1EbDrFhCrWXmjuw5ciyG7MMKUuNUTSLz/Sx sbl11f334gvzjXkp0POCR88FT9+H9UQYl+4v3SFo= From: Jacopo Mondi Date: Thu, 18 Jun 2026 12:18:45 +0200 Subject: [PATCH 06/14] include: linux: Add uAPI file for Dreamchip RPP-X1 MIME-Version: 1.0 Message-Id: <20260618-rppx1-ipa-v1-6-32337264cfcd@ideasonboard.com> References: <20260618-rppx1-ipa-v1-0-32337264cfcd@ideasonboard.com> In-Reply-To: <20260618-rppx1-ipa-v1-0-32337264cfcd@ideasonboard.com> To: =?utf-8?q?Niklas_S=C3=B6derlund?= , libcamera-devel@lists.libcamera.org Cc: Jacopo Mondi X-Mailer: b4 0.14.3 X-Developer-Signature: v=1; a=openpgp-sha256; l=28646; i=jacopo.mondi@ideasonboard.com; h=from:subject:message-id; bh=7KnAlTUERTSBesKy8vBBMYA7iy8uY418g5ZiePex/Yk=; b=owEBbQKS/ZANAwAKAXI0Bo8WoVY8AcsmYgBqM8YJhMGoVn/3RZDrgBklDzv1qTsw7qgbycLHl MYR5fdSAGSJAjMEAAEKAB0WIQS1xD1IgJogio9YOMByNAaPFqFWPAUCajPGCQAKCRByNAaPFqFW POoIEACTSkLJjKiZKjWznsgn++NGU6SX0pYBQik84WftaEUStLg3CS1H55sstu+WoGMhRfsccap /OdFBKfsIlR8KASpHaaLN9yW/a51PveFyFouV5FEyp9uU5waco060GMyR7a6ONoH1S7l66NrHKR hksuZ40LIAc5wUFHARifnvGiun6u/vh76AUb7TxP1HAa9LhmKLvAsygqvCplE64wZ38F0O1QeH1 JlAz2tGf43+u1AEuhH5jwzN4pl77PmjPBADvM3KP7tigPFbJ33LgpVk0iZLxFIwUeqSRrsJeUGG thUOJ+NyMPo22d4/HggpN4wvcNuhXEjRMHBQTlho0z6nUtZKJO2PgQEDkM5wung4We901KGrELS sljBepNcgEpUIHL+BGICMGA2ujco/IyjlmKUsW1zv+mld80C87ZR6l56LJmN6XkTq1mcE7qiiWt b9QpDsk6AIpbFDIvlnhgWVv7xbApN8hwuQ7XHa1n+4YpClyApQc1scX5YZMpYHfi2kM3H00vAU5 9AcFQiA+SA8YZ58RQhXmtwATuyermEkjwQtvK25T1fl5QI+B1A/XmlTml8vr2qVFbk65i/PWR71 NLA/AqOJMHYG+despH+fPTDt54vj87qNHD9M77T/uA4IaL+zHIQ043f25n3g6mjz6PaTCAeCzPE OFFDenoi4olqmwQ== X-Developer-Key: i=jacopo.mondi@ideasonboard.com; a=openpgp; fpr=72392EDC88144A65C701EA9BA5826A2587AD026B X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Add the uAPI header file for the Dreamchip RPP-X1 ISP integrated in the Renesas R-Car V4H SoC. The header is not yet upstream but is currently available at: https://patchwork.linuxtv.org/project/linux-media/list/?series=26362 Signed-off-by: Jacopo Mondi --- include/linux/media/dreamchip/rppx1-config.h | 710 +++++++++++++++++++++++++++ 1 file changed, 710 insertions(+) diff --git a/include/linux/media/dreamchip/rppx1-config.h b/include/linux/media/dreamchip/rppx1-config.h new file mode 100644 index 000000000000..0042c86090f6 --- /dev/null +++ b/include/linux/media/dreamchip/rppx1-config.h @@ -0,0 +1,710 @@ +/* SPDX-License-Identifier: GPL-2.0 WITH Linux-syscall-note */ +/* + * Dreamchip RPP-X1 ISP Driver - Userspace API + * + * Copyright (C) 2026 Renesas Electronics Corp. + * Copyright (C) 2026 Ideas on Board Oy + * Copyright (C) 2026 Ragnatech AB + */ + +#ifndef __UAPI_RPP_X1_CONFIG_H +#define __UAPI_RPP_X1_CONFIG_H + +#include + +/** + * struct rppx1_window - Measurement window + * + * RPP-X1 measurement window. Different blocks use a window or multiple + * windows for measurement purposes. This defines a common type for all of + * them. The number of relevant bits depends on the block where the window is + * used and is specified in the per-block description + * + * @h_offs: horizontal offset from the left of the frame in pixels + * @v_offs: vertical offset from the top of the frame in pixels + * @h_size: horizontal size of the window in pixels + * @v_size: vertical size of the window in pixels + */ +struct rppx1_window { + __u16 h_offs; + __u16 v_offs; + __u16 h_size; + __u16 v_size; +}; + +/** + * enum rppx1_meas_chan - Measurement point for the Histogram and EXM Modules + * + * Measurement points for the RPP-X1 Histogram measurement module and Exposure + * measurement module. + * + * All tap points are available for the PRE1/PRE2 pipes. Only + * RPPX1_MEAS_CHAN_SEL4 and RPPX1_MEAS_CHAN_SEL7 are available for the + * MAIN_POST pipe. + * + * @RPPX1_MEAS_CHAN_SEL0: after input acquisition + * @RPPX1_MEAS_CHAN_SEL1: after black level subtraction + * @RPPX1_MEAS_CHAN_SEL2: after sensor gamma linearization + * @RPPX1_MEAS_CHAN_SEL3: after lens shading correction + * @RPPX1_MEAS_CHAN_SEL4: after auto white balance gains + * @RPPX1_MEAS_CHAN_SEL5: after defect pixel correction + * @RPPX1_MEAS_CHAN_SEL6: after denoise pre-filter + * @RPPX1_MEAS_CHAN_SEL7: after demosaicing + */ +enum rppx1_meas_chan { + RPPX1_MEAS_CHAN_SEL0, + RPPX1_MEAS_CHAN_SEL1, + RPPX1_MEAS_CHAN_SEL2, + RPPX1_MEAS_CHAN_SEL3, + RPPX1_MEAS_CHAN_SEL4, + RPPX1_MEAS_CHAN_SEL5, + RPPX1_MEAS_CHAN_SEL6, + RPPX1_MEAS_CHAN_SEL7, +}; + +/* --------------------------------------------------------------------------- + * Parameter Structures + * + * The same ISP block might be instantiated in multiple pipeliness and operate + * on a different bitdepth/precision. For fields of varying length among + * different instances of the same block, use a data type that can accommodate + * the larger bitdepth/precision. + */ + +/** + * enum rppx1_params_block_type - RPP-X1 extensible params block types + * + * NOTE: Only append to the enumeration as the numbers are uAPI. + * + * @RPPX1_PARAMS_BLOCK_TYPE_WBMEAS_POST: AWB Measurement Configuration + * @RPPX1_PARAMS_BLOCK_TYPE_AWBG_PRE1: PRE1 pipe White Balance Gains + * @RPPX1_PARAMS_BLOCK_TYPE_AWBG_PRE2: PRE2 White Balance Gains + * @RPPX1_PARAMS_BLOCK_TYPE_AWBG_POST: MAIN_POST White Balance Gains + * @RPPX1_PARAMS_BLOCK_TYPE_EXM_PRE1: PRE1 pipe Exposure Measurement + * @RPPX1_PARAMS_BLOCK_TYPE_EXM_PRE2: PRE2 pipe Exposure Measurement + * @RPPX1_PARAMS_BLOCK_TYPE_HIST_PRE1: PRE1 pipe Histogram Measurement + * @RPPX1_PARAMS_BLOCK_TYPE_HIST_PRE2: PRE2 pipe Histogram Measurement + * @RPPX1_PARAMS_BLOCK_TYPE_HIST_POST: POST pipe Histogram Measurement + * @RPPX1_PARAMS_BLOCK_TYPE_BLS_PRE1: PRE1 pipe Black Level Subtraction + * @RPPX1_PARAMS_BLOCK_TYPE_BLS_PRE2: PRE2 pipe Black Level Subtraction + * @RPPX1_PARAMS_BLOCK_TYPE_CCOR_POST: POST pipe Color Correction + * @RPPX1_PARAMS_BLOCK_TYPE_LSC_PRE1: PRE1 pipe Lens Shading Correction + * @RPPX1_PARAMS_BLOCK_TYPE_LSC_PRE2: PRE2 pipe Lens Shading Correction + * @RPPX1_PARAMS_BLOCK_TYPE_GA_HV: Human Vision Pipe Gamma Out Correction + * @RPPX1_PARAMS_BLOCK_TYPE_GA_MV: Machine Vision Gamma Out Correction + * @RPPX1_PARAMS_BLOCK_TYPE_LIN_PRE1: PRE1 pipe Linearization (Sensor De-gamma) + * @RPPX1_PARAMS_BLOCK_TYPE_LIN_PRE2: PRE2 pipe Linearization (Sensor De-gamma) + */ +enum rppx1_params_block_type { + RPPX1_PARAMS_BLOCK_TYPE_WBMEAS_POST, + RPPX1_PARAMS_BLOCK_TYPE_AWBG_PRE1, + RPPX1_PARAMS_BLOCK_TYPE_AWBG_PRE2, + RPPX1_PARAMS_BLOCK_TYPE_AWBG_POST, + RPPX1_PARAMS_BLOCK_TYPE_EXM_PRE1, + RPPX1_PARAMS_BLOCK_TYPE_EXM_PRE2, + RPPX1_PARAMS_BLOCK_TYPE_HIST_PRE1, + RPPX1_PARAMS_BLOCK_TYPE_HIST_PRE2, + RPPX1_PARAMS_BLOCK_TYPE_HIST_POST, + RPPX1_PARAMS_BLOCK_TYPE_BLS_PRE1, + RPPX1_PARAMS_BLOCK_TYPE_BLS_PRE2, + RPPX1_PARAMS_BLOCK_TYPE_CCOR_POST, + RPPX1_PARAMS_BLOCK_TYPE_LSC_PRE1, + RPPX1_PARAMS_BLOCK_TYPE_LSC_PRE2, + RPPX1_PARAMS_BLOCK_TYPE_GA_HV, + RPPX1_PARAMS_BLOCK_TYPE_GA_MV, + RPPX1_PARAMS_BLOCK_TYPE_LIN_PRE1, + RPPX1_PARAMS_BLOCK_TYPE_LIN_PRE2, +}; + +/** + * enum rppx1_wbmeas_mode - AWB measurement mode + * + * @RPPX1_WBMEAS_MODE_YCBCR: YCbCr measurement mode + * @RPPX1_WBMEAS_MODE_RGB: RGB measurement mode + */ +enum rppx1_wbmeas_mode { + RPPX1_WBMEAS_MODE_YCBCR, + RPPX1_WBMEAS_MODE_RGB, +}; + +/** + * struct rppx1_wbmeas_params - AWB measurement configuration + * + * The Auto-White Balance measurement module is available on the MAIN_POST pipe. + * It supports two measurement modes, selected by the @mode field. The + * measurement window is programmed through the @wnd field. + * + * To support measurement in YCbCr mode a color conversion matrix with + * programmable offset is available in the @ccor_coeff and @ccor_offs fields. + * The color conversion matrix coefficients are represented as 16 bits signed + * Q4.12 numbers ranging from -8 to +7.99. The per-color channel offsets are + * represented as 25 bits 2's complement integer numbers ranging from -16777216 + * to +16777215. + * + * @header: block header (type = RPPX1_PARAMS_BLOCK_TYPE_WBMEAS_POST) + * @mode: measurement mode (from enum rppx1_wbmeas_mode) + * @ymax_cmp: enable Y_MAX compare using @max_y + * @wnd: measurement window + * @frames: number of frames for mean value calculation (0 = 1 frame) + * @ref_cr_max_r: reference Cr or max red value in RGB mode, 24 bits + * @ref_cb_max_b: reference Cb or max blue value in RGB mode, 24 bits + * @min_y_max_g: luminance minimum value or max green value in RGB mode, 24 bits + * @max_y: luminance maximum value, only valid if @mode is set to YCbCr and + * @ymax_cmp is set to enabled, 24 bits + * @max_csum: chrominance sum maximum value, 24 bits + * @min_c: chrominance minimum value, 24 bits + * @ccor_coeff: coefficients for color conversion matrix, signed 16 bits Q4.6 + * @ccor_offs: R-G-B color conversion coefficients, signed 25 bits 2's complement + */ +struct rppx1_wbmeas_params { + struct v4l2_isp_params_block_header header; + __u8 mode; + __u8 ymax_cmp; + struct rppx1_window wnd; + __u8 frames; + __u32 ref_cr_max_r; + __u32 ref_cb_max_b; + __u32 min_y_max_g; + __u32 max_y; + __u32 max_csum; + __u32 min_c; + __u16 ccor_coeff[3][3]; + __u32 ccor_offs[3]; +}; + +/** + * struct rppx1_awbg_params - WB gain configuration + * + * The RPP-X1 White Balance Gain module is available in the PRE1 and PRE2 + * pre-fusion pipes and in the MAIN_POST post-fusion pipe. Userspace selects + * which pipe to operate by setting the @header.type field to + * RPPX1_PARAMS_BLOCK_TYPE_AWBG_PRE1, RPPX1_PARAMS_BLOCK_TYPE_AWBG_PRE2 + * or RPPX1_PARAMS_BLOCK_TYPE_AWBG_POST. + * + * The White Balance module allows to specify per-color channel gains, expressed + * as unsigned fixed-point values as 18 bits unsigned integers in Q6.12 format + * with a maximum of 63.999. + * + * @header: block header (type = RPPX1_PARAMS_BLOCK_TYPE_AWBG_PRE1 or + * type = RPPX1_PARAMS_BLOCK_TYPE_AWBG_PRE2 or + * type = RPPX1_PARAMS_BLOCK_TYPE_AWBG_POST) + * @gain_red: gain for red component, 18-bit (unsigned Q6.12) + * @gain_green_r: gain for green component in red lines, 18-bit (unsigned Q6.12) + * @gain_blue: gain for blue component, 18-bit (unsigned Q6.12) + * @gain_green_b: gain for green component in blue lines, 18-bit (unsigned Q6.12) + */ +struct rppx1_awbg_params { + struct v4l2_isp_params_block_header header; + __u32 gain_red; + __u32 gain_green_r; + __u32 gain_blue; + __u32 gain_green_b; +}; + +/** + * enum rppx1_exm_mode - Exposure measurement mode + * + * Exaposure measurement mode selection (RGB/Bayer). + * + * @RPPX1_EXP_MEASURING_MODE_DISABLED: no measurement + * @RPPX1_EXP_MEASURING_MODE_RGB: Y/R/G/B measurement + * @RPPX1_EXP_MEASURING_MODE_BAYER: Bayer RGB measurement + */ +enum rppx1_exm_mode { + RPPX1_EXP_MEASURING_MODE_DISABLED, + RPPX1_EXP_MEASURING_MODE_RGB, + RPPX1_EXP_MEASURING_MODE_BAYER, +}; + +/** + * struct rppx1_exm_params - Exposure measurement configuration + * + * The RPP-X1 Exposure measurement unit is available on the PRE1 and PRE2 + * pre-fusion pipes. Userspace selects which pipe to operate by setting + * the @header.type field to RPPX1_PARAMS_BLOCK_TYPE_EXM_PRE1 or + * RPPX1_PARAMS_BLOCK_TYPE_EXM_PRE2. + * + * Exposure measurement is performed in the RGB or Bayer domain, according to + * the setting of the @mode field. The exposure measurement tap point is + * selected according to the value of @channel_sel. + * + * The exposure measurement is performed on an input window specified in @wnd. + * To each color component a programmable weight coefficient is associated. + * Coefficients are represented as unsigned 8 bits integer values in Q1.7 format + * ranging from 0 to 1.992. + * + * The @last_line fields controls when the exposure measurement completes. It + * is usually programmed to the value of (@wnd.v_offs + @wnd.v_size + 1). + * + * @header: block header (type = RPPX1_PARAMS_BLOCK_TYPE_EXM_PRE1 or + * type = RPPX1_PARAMS_BLOCK_TYPE_EXM_PRE2) + * @mode: exposure measure mode (from enum rppx1_exm_mode) + * @channel_sel: exposure measurement point (see enum rppx1_meas_chan) + * @last_line: line number for which the exposure measurement completes + * @wnd: measurement window coordinates + * @coeff_r: coefficient for the red Bayer sample or red color channel, Q1.7 + * @coeff_g_gr: coefficient for the green/red Bayer sample or green color channel, Q1.7 + * @coeff_b: coefficient for the blue Bayer sample or blue color channel, Q1.7 + * @coeff_gb: coefficient for the green/blue Bayer sample, unused in RGB mode, Q1.7 + */ +struct rppx1_exm_params { + struct v4l2_isp_params_block_header header; + __u32 mode; + __u8 channel_sel; + __u32 last_line; + struct rppx1_window wnd; + __u8 coeff_r; + __u8 coeff_g_gr; + __u8 coeff_b; + __u8 coeff_gb; +}; + +/* Histogram */ +#define RPPX1_HIST_WEIGHT_GRIDS_SIZE 25 + +/** + * enum rppx1_hist_mode - Histogram measurement mode + * + * Histogram measurement mode. Select which channel or combination of channels + * the histogram measurement is performed on. + * + * @RPPX1_HIST_MODE_DISABLE: histogram disabled + * @RPPX1_HIST_MODE_RGB_COMBINED: combined RGB histogram + * @RPPX1_HIST_MODE_R_HISTOGRAM: red channel histogram + * @RPPX1_HIST_MODE_GR_HISTOGRAM: green/red channel histogram + * @RPPX1_HIST_MODE_B_HISTOGRAM: blue channel histogram + * @RPPX1_HIST_MODE_GB_HISTOGRAM: green/blue histogram + */ +enum rppx1_hist_mode { + RPPX1_HIST_MODE_DISABLE, + RPPX1_HIST_MODE_RGB_COMBINED, + RPPX1_HIST_MODE_R_HISTOGRAM, + RPPX1_HIST_MODE_GR_HISTOGRAM, + RPPX1_HIST_MODE_B_HISTOGRAM, + RPPX1_HIST_MODE_GB_HISTOGRAM, +}; + +/** + * struct rppx1_hist_params - Histogram measurement configuration + * + * The RPP-X1 Histogram measurement unit is available on the PRE1, PRE2 and + * MAIN_POST pipes. Userspace selects which pipe to operate by setting the + * @header.type field to RPPX1_PARAMS_BLOCK_TYPE_HIST_PRE1, + * RPPX1_PARAMS_BLOCK_TYPE_HIST_PRE2 or + * RPPX1_PARAMS_BLOCK_TYPE_HIST_POST. + * + * The histogram measurement point is selected using the @channel field while + * histogram measurement mode is selected using the @mode field. + * + * Histogram measurement is performed by programming subsampling factors using + * the @v_stepsize and @h_step_inc fields and by weighted windowing, by + * programming the size of the measurement window @wnd with @weights associated + * to each cell of the 5x5 measurement grid. Weights are represented as 5 bits + * integer values ranging from 0 to 16. + * + * The @last_line fields controls when the histogram measurement completes. It + * is usually programmed to the value of (@wnd.v_offs + @wnd.v_size - 1). + * + * Histogram values are calculated by applying a per-color channel coefficient + * represented as an 8 bits unsigned Q1.7 integer value. The @sample_offs and + * @sample_shift fields allow to reduce the color dynamic range on which + * histogram data are produced. + * + * @header: block header (type = RPPX1_PARAMS_BLOCK_TYPE_HIST_PRE1, + * type = RPPX1_PARAMS_BLOCK_TYPE_HIST_PRE2 or + * type = RPPX1_PARAMS_BLOCK_TYPE_HIST_POST) + * @mode: histogram measurement mode (from enum rppx1_hist_mode) + * @channel_sel: histogram measurement point (see enum rppx1_meas_chan) + * @wnd: measurement window coordinates + * @weights: weighting factors for each sub-window (5x5 grid) + * @last_line: line number for which the histogram measurement completes + * @v_stepsize: vertical subsampling divider, 7 bits + * @h_step_inc: horizontal subsampling step counter, 17 bits + * @coeff: R-G-B coefficients, 8 bits unsigned Q1.7 + * @sample_offs: sample offset, 24 bits + * @sample_shift: sample shift, 4 bits + */ +struct rppx1_hist_params { + struct v4l2_isp_params_block_header header; + __u8 mode; + __u8 channel_sel; + struct rppx1_window wnd; + __u8 weights[RPPX1_HIST_WEIGHT_GRIDS_SIZE]; + __u32 last_line; + __u32 v_stepsize; + __u32 h_step_inc; + __u8 coeff[3]; + __u32 sample_offs; + __u8 sample_shift; +}; + +/** + * struct rppx1_bls_fixed - BLS fixed subtraction values + * + * Fixed black level values subtracted from sensor data per Bayer channel. + * Negative values result in addition. + * + * The PRE1 pipe BLS module operates on a 24-bits input data and fixed black + * levels are stored as a signed 2's complement representation ranging from + * -2^24 to 2^24-1. + * + * The PRE2 pipe BLS module operates on a 12-bits input data and fixed black + * levels are stored as a signed 2's complement representation ranging from + * -2^12 to 2^12-1. + * + * Userspace is expected to provide fixed black level values with a bit-depth + * matching the one of pipe in use. + * + * These subtraction values are matched with the sensor native Bayer components + * ordering according to the cropping configuration on the input port. + * + * @a: subtraction value for channel A + * @b: subtraction value for channel B + * @c: subtraction value for channel C + * @d: subtraction value for channel D + */ +struct rppx1_bls_fixed { + __u32 a; + __u32 b; + __u32 c; + __u32 d; +}; + +/** + * enum rppx1_bls_mode - BLS subtraction mode + * + * Select if subtracted black level come from fixed or measured values. + * + * @RPPX1_BLS_MODE_FIXED: subtract fixed values + * @RPPX1_BLS_MODE_MEAS: subtract measured values + */ +enum rppx1_bls_mode { + RPPX1_BLS_MODE_FIXED, + RPPX1_BLS_MODE_MEAS, +}; + +/** + * enum rppx1_bls_win_en: BLS measurement configuration + * + * Select the measurement window to use for measured black level values. + * + * @RPPX1_BLS_WIN_EN_OFF: disable measurement + * @RPPX1_BLS_WIN_EN_WIN1: Enable measurement from window 1 + * @RPPX1_BLS_WIN_EN_WIN2: enable measurement from window 2 + * @RPPX1_BLS_WIN_EN_WIN12: enable measurement from window 1 and window 2 + */ +enum rppx1_bls_win_en { + RPPX1_BLS_WIN_EN_OFF, + RPPX1_BLS_WIN_EN_WIN1, + RPPX1_BLS_WIN_EN_WIN2, + RPPX1_BLS_WIN_EN_WIN12, +}; + +/** + * struct rppx1_bls_params - RPP-X1 Black Level Subtraction Module + * + * The RPP-X1 Black Level Subtraction module is available on the PRE1 and PRE2 + * pre-fusion pipes. Userspace selects which pipe to operate by setting the + * @header.type field to RPPX1_PARAMS_BLOCK_TYPE_BLS_PRE1 or + * RPPX1_PARAMS_BLOCK_TYPE_BLS_PRE2. + * + * The BLS module operates on fixed or measured data according to the setting of + * the @mode field. When RPPX1_BLS_MODE_FIXED is used userspace shall provide + * the per-channel black levels in @fixed. When RPPX1_BLS_MODE_MEAS is used + * userspace shall configure the measurement windows @window1 and optionally + * @window2 to select the optically black pixels region in the input frame. The + * @samples fields controls how many measure samples are used for averaging the + * measured black levels. + * + * @header: block header (type = RPPX1_PARAMS_BLOCK_TYPE_BLS_PRE1 or + * type == RPPX1_PARAMS_BLOCK_TYPE_BLS_PRE2) + * @mode: BLS subtraction mode (see enum rppx1_bls_mode) + * @en_windows: BLS measurement mode (see rppx1_bls_win_en) + * @samples: log2 of the number of measured pixels per Bayer position + * @window1: BLS measurement window 1 (14 bits) + * @window2: BLS measurement window 2 (14 bits) + * @fixed: fixed subtraction values (see enum rppx1_bls_fixed) + */ +struct rppx1_bls_params { + struct v4l2_isp_params_block_header header; + __u8 mode; + __u8 en_windows; + __u8 samples; + struct rppx1_window window1; + struct rppx1_window window2; + struct rppx1_bls_fixed fixed; +}; + +/** + * struct rppx1_ccor_params - Color CORrection configuration + * + * The CCOR (Color Correction) module is available on the MAIN_POST pipe. It + * performs color space correction on a pixel-per-pixel basis using a 3x3 matrix + * of coefficients and per-color channel offsets. + * + * The matrix coefficients are represented as 16 bits signed fixed point values + * in Q4.12 format ranging from -8 to +7.999. + * + * The per-channel color offsets are represented as 2's complement values + * stored in 25 bits ranging from -16777216 to 16777215. + * + * @header: block header (type = RPPX1_PARAMS_BLOCK_TYPE_CCOR_POST) + * @coeff: color correction matrix coefficients, 16 bits signed Q4.12 + * @offset: R, G, B offsets, 2's complement 25 bits + */ +struct rppx1_ccor_params { + struct v4l2_isp_params_block_header header; + __u16 coeff[3][3]; + __u32 offset[3]; +}; + +/* Lens Shade Correction */ +#define RPPX1_LSC_SAMPLES_MAX 17 +#define RPPX1_LSC_NUM_SECTORS 16 + +/** + * struct rppx1_lsc_params - Lens Shading Correction configuration + * + * The RPP-X1 Lens shading correction module is available on the PRE1 and PRE2 + * pre-fusion pipes. Userspace selects which pipe to operate by setting the + * @header.type field to RPPX1_PARAMS_BLOCK_TYPE_LSC_PRE1 or + * RPPX1_PARAMS_BLOCK_TYPE_LSC_PRE2. + * + * The module applies per-color channel correction factors @r_data, @gr_data, + * @gb_data and @b_data as a 16x16 grid mapped on the image. The size of each + * grid segment is expressed by the @x_sect_size and @y_sect_size arrays. Each + * segment shall be at least 8 pixels in size and the sum of all horizontal + * segments @x_sect_size shall match the input frame size width. + * + * The correction factors values are expressed as unsigned Q2.10 integers + * ranging from 1 to 3.999. + * + * Pre-calculated interpolation factors shall be provided in the @x_grad + * and @y_grad fields, expressed as 12 bits integer values. + * + * @header: block header (type = RPPX1_PARAMS_BLOCK_TYPE_LSC) + * @r_data: correction factors for the red channel in Q2.10 format + * @gr_data: correction factors for the green (red) channel in Q2.10 format + * @gb_data: correction factors for the green (blue) channel in Q2.10 format + * @b_data: correction factors for the blue channel in Q2.10 format + * @x_grad: Interpolation gradients for each horizontal sector (12 bits) + * @y_grad: Interpolation gradients for each vertical sector (12 bits) + * @x_sect_size: Horizontal sectors sizes + * @y_sect_size: Vertical sectors sizes + */ +struct rppx1_lsc_params { + struct v4l2_isp_params_block_header header; + __u16 r_data[RPPX1_LSC_SAMPLES_MAX][RPPX1_LSC_SAMPLES_MAX]; + __u16 gr_data[RPPX1_LSC_SAMPLES_MAX][RPPX1_LSC_SAMPLES_MAX]; + __u16 gb_data[RPPX1_LSC_SAMPLES_MAX][RPPX1_LSC_SAMPLES_MAX]; + __u16 b_data[RPPX1_LSC_SAMPLES_MAX][RPPX1_LSC_SAMPLES_MAX]; + __u16 x_grad[RPPX1_LSC_NUM_SECTORS]; + __u16 y_grad[RPPX1_LSC_NUM_SECTORS]; + __u16 x_sect_size[RPPX1_LSC_NUM_SECTORS]; + __u16 y_sect_size[RPPX1_LSC_NUM_SECTORS]; +}; + +/* Gamma Out */ +#define RPPX1_GA_MAX_SAMPLES 17 + +/** + * enum rppx1_ga_seg_mode - Gamma out curve segmentation mode + * + * Segmentation mode of the 16 input sampling points for the Gamma Out + * Correction module. + * + * @RPPX1_GA_SEG_MODE_LOGARITHMIC: logarithmic-like segmentation mode + * @RPPX1_GA_SEG_MODE_EQUIDISTANT: equidistant segmentation mode + */ +enum rppx1_ga_seg_mode { + RPPX1_GA_SEG_MODE_LOGARITHMIC, + RPPX1_GA_SEG_MODE_EQUIDISTANT +}; + +/** + * struct rppx1_ga_params - Gamma Out Correction configuration + * + * The Gamma Out Correction module is available on the Human Vision Output + * Pipe (HV) and the Machine Vision Output Pipe (MV). Userspace selects + * which pipe to operate by setting the @header.type field to + * RPPX1_PARAMS_BLOCK_TYPE_GA_HV or RPPX1_PARAMS_BLOCK_TYPE_GA_MV. + * + * The module allows to apply a @gamma_y gamma correction curve to RGB data + * represented as a table of 16 entries. The 16 input sampling points can be + * equidistant or segmented using a logarithmic scale according to the value of + * @mode. + * + * The gamma curve values are 12 bits on the HV output pipe and 24 bits on the + * MV output pipe. Userspace is expected to provide the curve values with a + * bit-depth matching the one of pipe in use. + * + * @header: block header (type = RPPX1_PARAMS_BLOCK_TYPE_GA_HV or + * type = RPPX1_PARAMS_BLOCK_TYPE_GA_MV) + * @mode: gamma curve input segmentation mode (see rppx1_ga_seg_mode) + * @gamma_y: gamma out curve y-axis values + */ +struct rppx1_ga_params { + struct v4l2_isp_params_block_header header; + __u8 mode; + __u32 gamma_y[RPPX1_GA_MAX_SAMPLES]; +}; + +/* Linearization (Sensor De-gamma) */ +#define RPPX1_LIN_SAMPLE_POINTS_NUM 16 +#define RPPX1_LIN_DEGAMMA_CURVE_NUM 17 + +/** + * struct rppx1_lin_params - Linearization (Sensor De-gamma) configuration + * + * The RPP-X1 linearization module is available on the PRE1 and PRE2 pre-fusion + * pipes. Userspace selects which pipe to operate by setting the @header.type + * field to RPPX1_PARAMS_BLOCK_TYPE_LIN_PRE1 or + * RPPX1_PARAMS_BLOCK_TYPE_LIN_PRE2. + * + * The LIN module applies the per-color channel de-gamma linearization curves + * @curve_r, @curve_g and @curve_b defined on the input sampling points @dx. + * + * For the PRE1 pipe the de-gamma curves values are 24-bits, for the PRE2 pipe + * the de-gamma curve values are 12-bits. + * + * For the PRE1 pipe de-gamma module sampling points @dx values are in the range + * [0, 15] (4 bits). For the PRE2 pipe de-gamma module sampling points values + * are in the range [0, 7] (3 bits). + * + * Userspace is expected to provide the curve values and sampling points with a + * bit-depth matching the one of pipe in use. + * + * @header: block header (type = RPPX1_PARAMS_BLOCK_TYPE_LIN_PRE1 or + * RPPX1_PARAMS_BLOCK_TYPE_LIN_PRE2) + * @curve_r: de-gamma linearization curve for red channel + * @curve_g: de-gamma linearization curve for green channel + * @curve_b: de-gamma linearization curve for blue channel + * @dx: input sampling points + */ +struct rppx1_lin_params { + struct v4l2_isp_params_block_header header; + __u32 curve_r[RPPX1_LIN_DEGAMMA_CURVE_NUM]; + __u32 curve_g[RPPX1_LIN_DEGAMMA_CURVE_NUM]; + __u32 curve_b[RPPX1_LIN_DEGAMMA_CURVE_NUM]; + __u8 dx[RPPX1_LIN_SAMPLE_POINTS_NUM]; +}; + +/** + * RPPX1_PARAMS_MAX_SIZE - Maximum size of all RPP-X1 parameter blocks + * + * Some types are reported twice as the same block might be instantiated in + * multiple pipes. + */ +#define RPPX1_PARAMS_MAX_SIZE \ + (sizeof(struct rppx1_wbmeas_params) + \ + sizeof(struct rppx1_awbg_params) + \ + sizeof(struct rppx1_awbg_params) + \ + sizeof(struct rppx1_awbg_params) + \ + sizeof(struct rppx1_exm_params) + \ + sizeof(struct rppx1_exm_params) + \ + sizeof(struct rppx1_hist_params) + \ + sizeof(struct rppx1_hist_params) + \ + sizeof(struct rppx1_hist_params) + \ + sizeof(struct rppx1_bls_params) + \ + sizeof(struct rppx1_bls_params) + \ + sizeof(struct rppx1_ccor_params) + \ + sizeof(struct rppx1_lsc_params) + \ + sizeof(struct rppx1_lsc_params) + \ + sizeof(struct rppx1_ga_params) + \ + sizeof(struct rppx1_ga_params) + \ + sizeof(struct rppx1_lin_params) + \ + sizeof(struct rppx1_lin_params)) + +/* --------------------------------------------------------------------------- + * Statistics Structures + * + * The same ISP block might be instantiated in multiple pipeliness and operate + * on a different bitdepth/precision. For fields of varying length among + * different instances of the same block, use a data type that can accommodate + * the larger bitdepth/precision. + */ + +/** + * enum rppx1_stats_block_type - RPP-X1 extensible stats block types + * + * NOTE: Only append to the enumeration as the numbers are uAPI. + * + * @RPPX1_STATS_BLOCK_TYPE_WBMEAS_POST: post-fusion white-balance measurement + * @RPPX1_STATS_BLOCK_TYPE_EXM_PRE1: pre-fusion pipe1 exposure measurement + * @RPPX1_STATS_BLOCK_TYPE_EXM_PRE2: pre-fusion pipe2 exposure measurement + * @RPPX1_STATS_BLOCK_TYPE_HIST_PRE1: pre-fusion pipe1 histogram + * @RPPX1_STATS_BLOCK_TYPE_HIST_PRE2: pre-fusion pipe2 histogram + * @RPPX1_STATS_BLOCK_TYPE_HIST_POST: post-fusion histogram + */ +enum rppx1_stats_block_type { + RPPX1_STATS_BLOCK_TYPE_WBMEAS_POST, + RPPX1_STATS_BLOCK_TYPE_EXM_PRE1, + RPPX1_STATS_BLOCK_TYPE_EXM_PRE2, + RPPX1_STATS_BLOCK_TYPE_HIST_PRE1, + RPPX1_STATS_BLOCK_TYPE_HIST_PRE2, + RPPX1_STATS_BLOCK_TYPE_HIST_POST, +}; + +/** + * struct rppx1_wbmeas_stats - AWB statistics + * + * @header: block header (type = RPPX1_STATS_BLOCK_TYPE_WBMEAS_POST) + * @cnt: Number of pixels matched + * @mean_y_or_g: mean Y (or G in RGB mode) value, 24-bit + * @mean_cb_or_b: mean Cb (or B in RGB mode) value, 24-bit + * @mean_cr_or_r: mean Cr (or R in RGB mode) value, 24-bit + */ +struct rppx1_wbmeas_stats { + struct v4l2_isp_block_header header; + __u32 cnt; + __u32 mean_y_or_g; + __u32 mean_cb_or_b; + __u32 mean_cr_or_r; +}; + +/* Exposure Measurement */ +#define RPPX1_EXM_NUM_WIN 25 + +/** + * struct rppx1_exm_stats - Exposure measurement + * + * RPP-X1 exposure measurement calculates the mean value on 25 programmable + * windows on the input picture. + * + * @header: block header (type = RPPX1_STATS_BLOCK_TYPE_EXM_PRE1) + * @exp_mean: mean luminance values per block, up to 20-bit + */ +struct rppx1_exm_stats { + struct v4l2_isp_block_header header; + __u32 exp_mean[RPPX1_EXM_NUM_WIN]; +}; + +/* Histogram */ +#define RPPX1_HIST_NUM_BINS 32 + +/** + * struct rppx1_hist_stats - Histogram statistics + * + * @header: block header (type = RPPX1_STATS_BLOCK_TYPE_HIST_POST) + * @hist_bins: accumulation histogram results in unsigned 20-bit Q16.4 format + */ +struct rppx1_hist_stats { + struct v4l2_isp_block_header header; + __u32 hist_bins[RPPX1_HIST_NUM_BINS]; +}; + +/** + * RPPX1_STATS_MAX_SIZE - Maximum size of all RPP-X1 statistics + * + * Some types are reported twice as the same block might be instantiated in + * multiple pipes. + */ +#define RPPX1_STATS_MAX_SIZE \ + (sizeof(struct rppx1_wbmeas_stats) + \ + sizeof(struct rppx1_exm_stats) + \ + sizeof(struct rppx1_exm_stats) + \ + sizeof(struct rppx1_hist_stats) + \ + sizeof(struct rppx1_hist_stats) + \ + sizeof(struct rppx1_hist_stats)) + +#endif /* __UAPI_RPP_X1_CONFIG_H */ From patchwork Thu Jun 18 10:18:46 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jacopo Mondi X-Patchwork-Id: 26922 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 548B6C3305 for ; Thu, 18 Jun 2026 10:19:08 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id ACC4F62C6B; Thu, 18 Jun 2026 12:19:04 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="TbXNu3sp"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 9B225629BD for ; Thu, 18 Jun 2026 12:18:57 +0200 (CEST) Received: from [192.168.125.177] (mob-109-113-4-199.net.vodafone.it [109.113.4.199]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 2505D25C5; Thu, 18 Jun 2026 12:18:22 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1781777902; bh=PRmS+gUY99fvynNMzYqy8PtmxKmn5pAiKI+5NfXQssc=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=TbXNu3spwBVB8NgQxGks6Rjx/cAZCpbawwTkGuMl70fvG3F85V1VkXLFPSYkrc9TT BQdSErUpOFlv+2qHgJEzLHFAGsOMjiJmwWz2I59MnQnZBtWO3v9OfnXchrtO4fGZWs iiFzrRKmGi01Ibo0ZOURSvGjikdlPUlSRTVsBbK4= From: Jacopo Mondi Date: Thu, 18 Jun 2026 12:18:46 +0200 Subject: [PATCH 07/14] include: v4l2-isp: Add support for statistics MIME-Version: 1.0 Message-Id: <20260618-rppx1-ipa-v1-7-32337264cfcd@ideasonboard.com> References: <20260618-rppx1-ipa-v1-0-32337264cfcd@ideasonboard.com> In-Reply-To: <20260618-rppx1-ipa-v1-0-32337264cfcd@ideasonboard.com> To: =?utf-8?q?Niklas_S=C3=B6derlund?= , libcamera-devel@lists.libcamera.org Cc: Jacopo Mondi X-Mailer: b4 0.14.3 X-Developer-Signature: v=1; a=openpgp-sha256; l=8728; i=jacopo.mondi@ideasonboard.com; h=from:subject:message-id; bh=PRmS+gUY99fvynNMzYqy8PtmxKmn5pAiKI+5NfXQssc=; b=owEBbQKS/ZANAwAKAXI0Bo8WoVY8AcsmYgBqM8YJOR98k0BXK/HxtvuC1HI7iXmKyWAxUqZLL F/5ph2BBMKJAjMEAAEKAB0WIQS1xD1IgJogio9YOMByNAaPFqFWPAUCajPGCQAKCRByNAaPFqFW PIZBEACOEXOjdA8nBEjE7qPYStvB4yVorPrOFGM8plehT8OIAxNxVqLSHhElHgWdP6N/32aR/oL BuYCmknwEMqFPAnPMLXyPV7GlGsj1psYGOotw5ShEbjfr6uh61L2PdsMbcZs0XUbyR6pAKLyzU9 G9+wD6/dFLW/IbB8V4OekiypwNkyvzgl7U6O5ysZHkRGn7WR++L9iE+aHuflxZX4N1VHdXTO/qZ Ml2RFKQVnO+8WQbOkSxWVk9/NfsF/3VxqoyDjVGzJpD5guXIV368Pg6PuyNGsvQNHDwZGeHG+1r GRqYL/MAZAe36ddpF75+hSjd1DpJfzm02LoodN9g1TTKtwQRTAd4ndLjz/KxecYr00qZWZSMr3k 5s84jypfVL5RMd+ocRo+JPLb+N5LpwbNWNGJSzYFhgxXuWtSmJzXpRzqNm2yqiqZjFs+WPH8DQT UOcio6ul5resNozbclrMW4y1nNzuyGL61aEIl+W+6Xm9cxKx7576UXyYyT7F9pk2r+L6AWzMvRH /UpfdCeLmMPOivD7TTllcsbxNEdTfLe12x7hSd/bN81YGxZo4WRblZMCm/BcEGxX4nQbMmckcy/ beaU2dskw1QkDPjE0AtVDMXpXK9G6k4OZJOehZo00PaS52yvFMWz02qtRGCw3j/px4amFCAb63u db0I/vAk6j9rBAA== X-Developer-Key: i=jacopo.mondi@ideasonboard.com; a=openpgp; fpr=72392EDC88144A65C701EA9BA5826A2587AD026B X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Add support for extensible statistics to v4l2-isp.h. Patches are not upstream yet but are available at: https://patchwork.linuxtv.org/project/linux-media/list/?series=24772 Signed-off-by: Jacopo Mondi --- include/linux/media/v4l2-isp.h | 125 ++++++++++++++++++++++++++--------------- 1 file changed, 79 insertions(+), 46 deletions(-) diff --git a/include/linux/media/v4l2-isp.h b/include/linux/media/v4l2-isp.h index 0cfc4ab94967..60a716177d52 100644 --- a/include/linux/media/v4l2-isp.h +++ b/include/linux/media/v4l2-isp.h @@ -13,25 +13,33 @@ #include /** - * enum v4l2_isp_params_version - V4L2 ISP parameters versioning + * enum v4l2_isp_version - V4L2 ISP serialization format versioning * - * @V4L2_ISP_PARAMS_VERSION_V0: First version of the V4L2 ISP parameters format - * (for compatibility) - * @V4L2_ISP_PARAMS_VERSION_V1: First version of the V4L2 ISP parameters format + * @V4L2_ISP_VERSION_V0: First version of the V4L2 ISP serialization format + * (for compatibility) + * @V4L2_ISP_VERSION_V1: First version of the V4L2 ISP serialization format * * V0 and V1 are identical in order to support drivers compatible with the V4L2 - * ISP parameters format already upstreamed which use either 0 or 1 as their - * versioning identifier. Both V0 and V1 refers to the first version of the - * V4L2 ISP parameters format. + * ISP format already upstreamed which use either 0 or 1 as their versioning + * identifier. Both V0 and V1 refers to the first version of the V4L2 ISP + * serialization format. * - * Future revisions of the V4L2 ISP parameters format should start from the + * Future revisions of the V4L2 ISP serialization format should start from the * value of 2. */ -enum v4l2_isp_params_version { - V4L2_ISP_PARAMS_VERSION_V0 = 0, - V4L2_ISP_PARAMS_VERSION_V1 +enum v4l2_isp_version { + V4L2_ISP_VERSION_V0 = 0, + V4L2_ISP_VERSION_V1 }; +/* + * Compatibility with existing users of v4l2_isp_params which pre-date the + * introduction of v4l2_isp_stats. + */ +#define v4l2_isp_params_version v4l2_isp_version +#define V4L2_ISP_PARAMS_VERSION_V0 V4L2_ISP_VERSION_V0 +#define V4L2_ISP_PARAMS_VERSION_V1 V4L2_ISP_VERSION_V1 + #define V4L2_ISP_PARAMS_FL_BLOCK_DISABLE (1U << 0) #define V4L2_ISP_PARAMS_FL_BLOCK_ENABLE (1U << 1) @@ -39,64 +47,89 @@ enum v4l2_isp_params_version { * Reserve the first 8 bits for V4L2_ISP_PARAMS_FL_* flag. * * Driver-specific flags should be defined as: - * #define DRIVER_SPECIFIC_FLAG0 ((1U << V4L2_ISP_PARAMS_FL_DRIVER_FLAGS(0)) - * #define DRIVER_SPECIFIC_FLAG1 ((1U << V4L2_ISP_PARAMS_FL_DRIVER_FLAGS(1)) + * #define DRIVER_SPECIFIC_FLAG0 ((1U << V4L2_ISP_FL_DRIVER_FLAGS(0)) + * #define DRIVER_SPECIFIC_FLAG1 ((1U << V4L2_ISP_FL_DRIVER_FLAGS(1)) */ -#define V4L2_ISP_PARAMS_FL_DRIVER_FLAGS(n) ((n) + 8) +#define V4L2_ISP_FL_DRIVER_FLAGS(n) ((n) + 8) /** - * struct v4l2_isp_params_block_header - V4L2 extensible parameters block header - * @type: The parameters block type (driver-specific) + * struct v4l2_isp_block_header - V4L2 extensible block header + * @type: The parameters or statistics block type (driver-specific) * @flags: A bitmask of block flags (driver-specific) - * @size: Size (in bytes) of the parameters block, including this header + * @size: Size (in bytes) of the block, including this header * - * This structure represents the common part of all the ISP configuration - * blocks. Each parameters block shall embed an instance of this structure type - * as its first member, followed by the block-specific configuration data. + * This structure represents the common part of all the ISP configuration or + * statistic blocks. Each block shall embed an instance of this structure type + * as its first member, followed by the block-specific configuration or + * statistic data. * * The @type field is an ISP driver-specific value that identifies the block - * type. The @size field specifies the size of the parameters block. + * type. The @size field specifies the size of the block, including this + * header. * - * The @flags field is a bitmask of per-block flags V4L2_PARAMS_ISP_FL_* and - * driver-specific flags specified by the driver header. + * The @flags field is a bitmask of per-block flags. If a block is used for + * configuration parameters this field can be a combination of + * V4L2_ISP_PARAMS_FL_* and driver-specific flags. If a block is used + * for statistics this fields is used to report optional + * driver-specific flags, if any. */ -struct v4l2_isp_params_block_header { +struct v4l2_isp_block_header { __u16 type; __u16 flags; __u32 size; } __attribute__((aligned(8))); /** - * struct v4l2_isp_params_buffer - V4L2 extensible parameters configuration - * @version: The parameters buffer version (driver-specific) - * @data_size: The configuration data effective size, excluding this header - * @data: The configuration data + * v4l2_isp_params_block_header - V4L2 extensible parameters block header + * + * Compatibility with existing users of v4l2_isp_params_block_header + * which pre-date the introduction of v4l2_isp_block_header. + */ +#define v4l2_isp_params_block_header v4l2_isp_block_header + +/** + * struct v4l2_isp_buffer - V4L2 extensible buffer + * @version: The extensible buffer version (driver-specific) + * @data_size: The data effective size, excluding this header + * @data: The configuration or statistics data * - * This structure contains the configuration parameters of the ISP algorithms, - * serialized by userspace into a data buffer. Each configuration parameter - * block is represented by a block-specific structure which contains a - * :c:type:`v4l2_isp_params_block_header` entry as first member. Userspace - * populates the @data buffer with configuration parameters for the blocks that - * it intends to configure. As a consequence, the data buffer effective size - * changes according to the number of ISP blocks that userspace intends to - * configure and is set by userspace in the @data_size field. + * This structure contains ISP configuration parameters or ISP hardware + * statistics serialized into a data buffer. Each block is represented by a + * block-specific structure which contains a :c:type:`v4l2_isp_block_header` + * entry as first member. * - * The parameters buffer is versioned by the @version field to allow modifying - * and extending its definition. Userspace shall populate the @version field to - * inform the driver about the version it intends to use. The driver will parse - * and handle the @data buffer according to the data layout specific to the - * indicated version and return an error if the desired version is not + * When used for ISP parameters, userspace populates the @data buffer with + * configuration parameters for the blocks that it intends to configure. As a + * consequence, the data buffer effective size changes according to the number + * of ISP blocks that userspace intends to configure. + * + * When used to report ISP statistics, the driver populates the @data buffer + * with statistics for each supported measurement block. + * + * The buffer is versioned by the @version field to allow modifying + * and extending its definition. The writer shall populate the @version field + * to inform the reader about the version it intends to use. The reader will + * parse and handle the @data buffer according to the data layout specific to + * the indicated version and return an error if the desired version is not * supported. * - * For each ISP block that userspace wants to configure, a block-specific - * structure is appended to the @data buffer, one after the other without gaps - * in between. Userspace shall populate the @data_size field with the effective - * size, in bytes, of the @data buffer. + * For each ISP block, a block-specific structure is appended to the @data + * buffer, one after the other without gaps in between. The writer shall + * populate the @data_size field with the effective size, in bytes, of the + * @data buffer. */ -struct v4l2_isp_params_buffer { +struct v4l2_isp_buffer { __u32 version; __u32 data_size; __u8 data[] __counted_by(data_size); }; +/** + * v4l2_isp_params_buffer - V4L2 extensible parameters compatibility + * + * Compatibility with existing users of v4l2_isp_params_buffer which + * pre-date the introduction of v4l2_isp_buffer. + */ +#define v4l2_isp_params_buffer v4l2_isp_buffer + #endif /* _V4L2_ISP_H_ */ From patchwork Thu Jun 18 10:18:47 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jacopo Mondi X-Patchwork-Id: 26923 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 62D2EC3306 for ; Thu, 18 Jun 2026 10:19:09 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 1F523629DE; Thu, 18 Jun 2026 12:19:06 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="JDizjgn+"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 2A686629DE for ; Thu, 18 Jun 2026 12:18:58 +0200 (CEST) Received: from [192.168.125.177] (mob-109-113-4-199.net.vodafone.it [109.113.4.199]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 151B31894; Thu, 18 Jun 2026 12:18:22 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1781777903; bh=zFgVj6yNdAlt2Q8ombawCKxzpQUlZoFNp15h6eI4+S4=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=JDizjgn+obFnzY7EFLDDuyCvGyjDy4koGbiofwT0rTbv1knyzuDlNLiUETBIYwrNX W/xMLvlbO11lZgyAzHYSfgf6iRNZ4dXm4wG4cDy4DN3kguadQlkrrAPzm5bbYSAZIK IIuEvslrFc82d4Jxpy1t8Wh23bpDLWwoMpmChOjE= From: Jacopo Mondi Date: Thu, 18 Jun 2026 12:18:47 +0200 Subject: [PATCH 08/14] include: videodev2: Add RPP-X1 meta formats MIME-Version: 1.0 Message-Id: <20260618-rppx1-ipa-v1-8-32337264cfcd@ideasonboard.com> References: <20260618-rppx1-ipa-v1-0-32337264cfcd@ideasonboard.com> In-Reply-To: <20260618-rppx1-ipa-v1-0-32337264cfcd@ideasonboard.com> To: =?utf-8?q?Niklas_S=C3=B6derlund?= , libcamera-devel@lists.libcamera.org Cc: Jacopo Mondi X-Mailer: b4 0.14.3 X-Developer-Signature: v=1; a=openpgp-sha256; l=1214; i=jacopo.mondi@ideasonboard.com; h=from:subject:message-id; bh=zFgVj6yNdAlt2Q8ombawCKxzpQUlZoFNp15h6eI4+S4=; b=owEBbQKS/ZANAwAKAXI0Bo8WoVY8AcsmYgBqM8YKa6o6SPr3EEzqoNAuruEfPHs6Nk8NfvSef UZBhbKRmfiJAjMEAAEKAB0WIQS1xD1IgJogio9YOMByNAaPFqFWPAUCajPGCgAKCRByNAaPFqFW PKpzEACplSI99+BRSt3uRnfiu9VBl8OLwyihY/syqgD2FUgTrRHkYECV/zFMokYIH1FSdwqxzYl wWCTwqV+WKZfkZuOLbQhXRF2I1wlfUtQDiLdybJknPzTHJK0ngR7MT5zv+lLquCsiAHhTxjtZWN O+Fw13Fx0HaANi9nfz0lpk7SLVphXa4+/VDCuHUhsjjnIzt/eNy/IMOPWf5nX4ut61lObZ6/ukx 8fBy8upHOMkPYOD42Va5cLyW92/8l8lxrKghtBU0JJYaFGLDlE1R6eXB0k6By2G+t1PNi9mbKN6 m0IUOHbNrwqxlsVkN0TN8VMUaiqqLkpNhy5Ma1Z0bT+d47wVC5LNIGGsaSgGadBuybhwR4DboX6 RVXUzM1exapt03ebTzgCJKpPQPET18ASYl60bmsYN0rphi/jGo8a1nmXInehap8D2aurhwUKH9p vCj0petwmBnkP2xloeXciGXiL1V/5xKAXVoxVNLqjufrn+gPYiCw3uLvRbQ5xSGEMR4W95bMgWB fyrs8gik3dYSFDMQ9u3BMR4Ib0Pxymw3p4fVVH78yPTiIuh5el/ATqsrlXb1B+6nsqD4QlCgV8E S3mzjVp/t5NiWBvRvRpyYVSpPIYta1QSgvdkgnz5uDiEJWd8Oyy4SWw7gsrlGam+99FY8pjFBze l35stYi+lkz9Q0Q== X-Developer-Key: i=jacopo.mondi@ideasonboard.com; a=openpgp; fpr=72392EDC88144A65C701EA9BA5826A2587AD026B X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Add parameters and statistics meta formats for the RPP-X1 ISP to videodev2.h. The patch that adds the two format to the Linux kernel is not merged upstream yet but is available at: https://patchwork.linuxtv.org/project/linux-media/list/?series=26362 Signed-off-by: Jacopo Mondi --- include/linux/videodev2.h | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/include/linux/videodev2.h b/include/linux/videodev2.h index 8bccad84ff83..dda109575403 100644 --- a/include/linux/videodev2.h +++ b/include/linux/videodev2.h @@ -864,6 +864,10 @@ struct v4l2_pix_format { #define V4L2_META_FMT_MALI_C55_PARAMS v4l2_fourcc('C', '5', '5', 'P') /* ARM Mali-C55 Parameters */ #define V4L2_META_FMT_MALI_C55_STATS v4l2_fourcc('C', '5', '5', 'S') /* ARM Mali-C55 3A Statistics */ +/* Vendor specific - used for Dreamchip RPP-X1 ISP */ +#define V4L2_META_FMT_RPP_X1_PARAMS v4l2_fourcc('D', 'R', '1', 'P') /* Dreamchip RPP-X1 Parameters */ +#define V4L2_META_FMT_RPP_X1_STATS v4l2_fourcc('D', 'R', '1', 'S') /* Dreamchip RPP-X1 Statistics */ + /* * Line-based metadata formats. Remember to update v4l_fill_fmtdesc() when * adding new ones! From patchwork Thu Jun 18 10:18:48 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jacopo Mondi X-Patchwork-Id: 26924 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id EEAF5C328C for ; Thu, 18 Jun 2026 10:19:09 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 3140E629E0; Thu, 18 Jun 2026 12:19:08 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="k3O46RE+"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 131FF629E0 for ; Thu, 18 Jun 2026 12:18:59 +0200 (CEST) Received: from [192.168.125.177] (mob-109-113-4-199.net.vodafone.it [109.113.4.199]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 99FCC2689; Thu, 18 Jun 2026 12:18:23 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1781777904; bh=BCwNM9Uwi/BjvuMp45FftguRa2jcveo6lNW3w56Hse0=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=k3O46RE+6GcoE5KQBTuRXu5Rz1H8WMPA5laElfkRYfla8XAb86LYFrOPlI4tzOtPS kJR/0EvRsCwSHw/hLHD1ZldOoEuqz9DKNOCMAVZ7XptE2aIysSIXEVwArywEKl4dr5 Cvr5B7piygou7qyGT5XyBpF9Lp+v33y0KtYWsg+I= From: Jacopo Mondi Date: Thu, 18 Jun 2026 12:18:48 +0200 Subject: [PATCH 09/14] ipa: libipa: Introduce V4L2Stats MIME-Version: 1.0 Message-Id: <20260618-rppx1-ipa-v1-9-32337264cfcd@ideasonboard.com> References: <20260618-rppx1-ipa-v1-0-32337264cfcd@ideasonboard.com> In-Reply-To: <20260618-rppx1-ipa-v1-0-32337264cfcd@ideasonboard.com> To: =?utf-8?q?Niklas_S=C3=B6derlund?= , libcamera-devel@lists.libcamera.org Cc: Jacopo Mondi , Kieran Bingham X-Mailer: b4 0.14.3 X-Developer-Signature: v=1; a=openpgp-sha256; l=10203; i=jacopo.mondi@ideasonboard.com; h=from:subject:message-id; bh=BCwNM9Uwi/BjvuMp45FftguRa2jcveo6lNW3w56Hse0=; b=owEBbQKS/ZANAwAKAXI0Bo8WoVY8AcsmYgBqM8YKV0+6SIWv5CC9K4hkQvoXCP0MHnMbV9LuU x4/iGHqHZOJAjMEAAEKAB0WIQS1xD1IgJogio9YOMByNAaPFqFWPAUCajPGCgAKCRByNAaPFqFW PLb8D/44Ie784HDZHLCBhSOuHKrt+TGZH7J5cKy15U249Tg3vBozg+sffG7r5NTh9tle+smX0bU 5y+v/hWwPHiNq11vC8aFdK8CvEgoxhz2DFOvWIZf91MJ0qS9LeFuxBiMjCe2DG/Zw7IAFqD8917 uYNtwGDfOvKkXM3PNfpKiDdX6DtDGLkdZq3NLu6kvth7gbnFyGqY10wZgyhZONdjX5Zk7VReYxp w1c4CjwwQDnppxYX9HuLTzxg86r8xvF+WR2/JV1G7yJHt5F/oPdCD4Lh1I0lJ12G608kMi2EIpw 6wY7p94Uf4NwGTtGCzhRVMdGC9pMMyT3aF8sUnEASJ9s2DG/EtN5d/ntc5Nxq9QYwGCIArNjtXr tX+M6d0e/elvQYPzhO6RAnqiGdIEYFxOIcxHoqUdbLRRBQWEYxaLIJoqpBmRbaOoNlzfyVxhMdi d+tzAT7eCy3T723fPFOkUies6p3FUWxqRAEVlace37nfXVxTbzziTroyW3oiSyhyA81RWx9Z5tt tb0EeY4QKGVQ6EOrWMDV0h3IJtJDilDrjQpyv2ZGO0rn3y37PKTWqqLZPrhLGrG0iJzpBW6JMPX xqMhsfOKjesrNTrhJxw4dBWuIT/ODoGIBh69vKJm9nnA10VREQTvki0hA279NhQKb1tkVrEB0m6 myEsIQMj30g42kA== X-Developer-Key: i=jacopo.mondi@ideasonboard.com; a=openpgp; fpr=72392EDC88144A65C701EA9BA5826A2587AD026B X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Add a V4L2Stats class similar in spirit to the existing V4L2Params class to allow IPA modules to easily sub-class it to access ISP statistics blocks serialized into a v4l2_isp_buffer. Signed-off-by: Jacopo Mondi Reviewed-by: Kieran Bingham --- src/ipa/libipa/meson.build | 2 + src/ipa/libipa/v4l2_stats.cpp | 241 ++++++++++++++++++++++++++++++++++++++++++ src/ipa/libipa/v4l2_stats.h | 67 ++++++++++++ 3 files changed, 310 insertions(+) diff --git a/src/ipa/libipa/meson.build b/src/ipa/libipa/meson.build index edf8eabd8b78..635abad778e6 100644 --- a/src/ipa/libipa/meson.build +++ b/src/ipa/libipa/meson.build @@ -20,6 +20,7 @@ libipa_headers = files([ 'pwl.h', 'quantized.h', 'v4l2_params.h', + 'v4l2_stats.h', ]) libipa_sources = files([ @@ -42,6 +43,7 @@ libipa_sources = files([ 'pwl.cpp', 'quantized.cpp', 'v4l2_params.cpp', + 'v4l2_stats.cpp', ]) libipa_includes = include_directories('..') diff --git a/src/ipa/libipa/v4l2_stats.cpp b/src/ipa/libipa/v4l2_stats.cpp new file mode 100644 index 000000000000..64f546d85069 --- /dev/null +++ b/src/ipa/libipa/v4l2_stats.cpp @@ -0,0 +1,241 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2026, Ideas On Board + * + * V4L2 Statistics + */ + +#include "v4l2_stats.h" + +#include + +namespace libcamera { + +namespace ipa { + +LOG_DEFINE_CATEGORY(V4L2Stats) + +/** + * \file v4l2_stats.cpp + * \brief Helper class to handle an ISP statistics buffer compatible with + * the generic V4L2 ISP format + * + * The Linux kernel defines a generic buffer format for ISP statistics. + * The format describes a serialisation method that allows userspace to + * access statistics data from a binary buffer. + * + * The V4L2Stats class implements support for the V4L2 ISP statistics buffer + * format and allows users to retrieve an ISP statistics block. + * + * IPA implementations using these helpers should define an enumeration of ISP + * blocks supported by the IPA module and use a set of common abstractions to + * help their derived implementation of V4L2Stats translate the enumerated ISP + * block identifiers to the actual type of the statistics data as defined by + * the kernel interface. + */ + +/** + * \class V4L2StatsBase + * \brief Base class for V4L2Stats + * + * The V4L2StatsBase is an integral part of V4L2Stats. It serves as a + * container for all code that does not depend on the V4L2Stats template + * arguments, to avoid duplicate copies of inline code. + */ + +/** + * \brief Construct an instance of V4L2StatsBase + * \param[in] data Reference to the v4l2-buffer memory mapped area + * \param[in] version The ISP parameters version the implementation supports + * + * Parse the statistics buffer and construct a cache that maps the block type to + * the memory location of the statistics block in the buffer. + * + * After construction users of this class shall check the validity of the + * constructed instance using operator bool(). + */ +V4L2StatsBase::V4L2StatsBase(Span data, unsigned int version) + : data_(data), valid_(false) +{ + const struct v4l2_isp_buffer *stats = + reinterpret_cast(data_.data()); + + if (data_.size() - sizeof(*stats) < stats->data_size) { + LOG(V4L2Stats, Error) + << "Stats buffer size mismatch: " << stats->data_size; + return; + } + + if (version != stats->version) { + LOG(V4L2Stats, Error) + << "Unsupported v4l2-isp version: " << stats->version; + return; + } + + /* Construct the cache for easier lookup. */ + size_t left = stats->data_size; + const __u8 *d = stats->data; + + while (left > 0) { + const struct v4l2_isp_block_header *header = + reinterpret_cast(d); + + if (left < sizeof(*header) || header->size < sizeof(*header)) { + LOG(V4L2Stats, Error) + << "Block type " << header->type + << " size is not valid"; + return; + } + + if (left < header->size) { + LOG(V4L2Stats, Error) + << "Not enough space for block type " << header->type; + return; + } + + auto [it, inserted] = cache_.try_emplace(header->type, d, header->size); + if (!inserted) { + LOG(V4L2Stats, Error) + << "Duplicated block type " << header->type; + return; + } + + d += header->size; + left -= header->size; + } + + valid_ = true; +} + +/** + * \brief Retrieve an ISP statistics block a returns a reference to it + * \param[in] blockType The kernel-defined ISP block identifier, used to + * identify the block header + * \param[in] blockSize The ISP statistics block size, for validation + * + * Retrieve a span to the statistics block memory location by accessing the + * cache built at class construction time. + * + * \return The memory location of the ISP statistics block, or an empty Span + * if \a blockType is not supported + */ +Span V4L2StatsBase::block(unsigned int blockType, size_t blockSize) const +{ + const auto it = cache_.find(blockType); + if (it == cache_.end()) { + LOG(V4L2Stats, Error) << "Unsupported stats block type: " + << blockType; + return {}; + } + + const struct v4l2_isp_block_header *header = + reinterpret_cast(it->second.data()); + if (header->size != blockSize) { + LOG(V4L2Stats, Error) + << "Block type " << blockType + << " size mistmatch: expected " + << blockSize << " got:" + << header->size; + return {}; + } + + return it->second; +} + +/** + * \fn V4L2StatsBase::operator bool() + * \brief Retrieve if a statistics block is valid + * \return True if the statistics block is valid, false otherwise + */ + +/** + * \class V4L2Stats + * \brief Helper class that represent an ISP statistics buffer + * + * This class represents an ISP statistics buffer. It is constructed with a + * reference to the memory mapped buffer that has been dequeued from the ISP + * driver. + * + * This class is templated with the type of the enumeration of ISP blocks that + * each IPA module is expected to support. IPA modules are expected to derive + * this class by providing a 'stats_traits' type that helps the class associate + * a block type with the actual memory area that represents the ISP statistics + * block. + * + * \code{.cpp} + * + * // Define the supported ISP statistics blocks + * enum class myISPStats { + * Agc, + * Awb, + * ... + * }; + * + * // Maps the C++ enum type to the kernel enum type and concrete parameter type + * template + * struct block_type { + * }; + * + * template<> + * struct block_type { + * using type = struct my_isp_kernel_stats_type_agc; + * static constexpr kernel_enum_type blockType = MY_ISP_STATS_TYPE_AGC; + * }; + * + * template<> + * struct block_type { + * using type = struct my_isp_kernel_stats_type_awb; + * static constexpr kernel_enum_type blockType = MY_ISP_STATS_TYPE_AWB; + * }; + * + * + * // Convenience type to associate a block id to the 'block_type' overload + * struct stats_traits { + * using id_type = myISPStats; + * template using id_to_details = block_type; + * }; + * + * ... + * + * // Derive the V4L2Stats class by providing stats_traits + * class MyISPStats : public V4L2Stats + * { + * public: + * MyISPStats::MyISPStats(Span data, unsigned int version) + * : V4L2Stats(data, version) + * { + * } + * }; + * + * \endcode + * + * Users of this class can then easily access an ISP statistics block using the + * block() function. + * + * \code{.cpp} + * + * MyISPStats stats(data); + * + * auto awb = stats.block(); + * auto mean_r = awb->mean_r; + * auto mean_g = awb->mean_g; + * auto mean_b = awb->mean_b; + * \endcode + */ + +/** + * \fn V4L2Stats::V4L2Stats() + * \brief Construct an instance of V4L2Stats + * \param[in] data Reference to the v4l2-buffer memory mapped area + * \param[in] version The expected V4L2 ISP serialization format version + */ + +/** + * \fn V4L2Stats::block() const + * \brief Retrieve a pointer to an ISP statistics block + * \return A pointer to the ISP statistics block + */ + +} /* namespace ipa */ + +} /* namespace libcamera */ diff --git a/src/ipa/libipa/v4l2_stats.h b/src/ipa/libipa/v4l2_stats.h new file mode 100644 index 000000000000..b96395e5f551 --- /dev/null +++ b/src/ipa/libipa/v4l2_stats.h @@ -0,0 +1,67 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2026, Ideas On Board + * + * V4L2 ISP Statistics + */ + +#pragma once + +#include +#include + +#include + +#include + +namespace libcamera { + +namespace ipa { + +class V4L2StatsBase +{ +public: + V4L2StatsBase(Span data, unsigned int version); + + Span block(unsigned int blockType, size_t blockSize) const; + constexpr explicit operator bool() + { + return valid_; + } + +private: + std::map> cache_; + Span data_; + bool valid_; +}; + +template +class V4L2Stats : public V4L2StatsBase +{ +public: + static_assert(std::is_same_v, uint16_t>); + + V4L2Stats(Span data, unsigned int version) + : V4L2StatsBase(data, version) + { + } + + template + const typename Traits::template id_to_details::type * + block() const + { + using Details = typename Traits::template id_to_details; + + using Type = typename Details::type; + constexpr auto kernelId = Details::blockType; + + auto data = V4L2StatsBase::block(kernelId, sizeof(Type)); + + return data.size() > 0 ? + reinterpret_cast(data.data()) : nullptr; + } +}; + +} /* namespace ipa */ + +} /* namespace libcamera */ From patchwork Thu Jun 18 10:18:49 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jacopo Mondi X-Patchwork-Id: 26925 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id C72F1C3307 for ; Thu, 18 Jun 2026 10:19:10 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 6110662C71; Thu, 18 Jun 2026 12:19:09 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="ryW9D6UO"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id B674C629E6 for ; Thu, 18 Jun 2026 12:18:59 +0200 (CEST) Received: from [192.168.125.177] (mob-109-113-4-199.net.vodafone.it [109.113.4.199]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 84E952393; Thu, 18 Jun 2026 12:18:24 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1781777904; bh=h04KN9mLyfrik8nFAtKM57O5ZCpidFxhIftsvizNnko=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=ryW9D6UOWq9vmU7YeaPkV8ZvTVCVl36xbrWyGHpQGhVFMHe4AIz0X5iQwDF4hGi+t jLIrRZZFsEdiIWAdnt3gOoIZGRXxzzosWcaXr8PJhLLZVRoM4SBI7ARLZnnbxI2iYp p577BTcXH9453dgs7ix7/BPCyEDTD53AHkfuWvPI= From: Jacopo Mondi Date: Thu, 18 Jun 2026 12:18:49 +0200 Subject: [PATCH 10/14] ipa: rppx1: Add RppX1Params and RppX1Stats MIME-Version: 1.0 Message-Id: <20260618-rppx1-ipa-v1-10-32337264cfcd@ideasonboard.com> References: <20260618-rppx1-ipa-v1-0-32337264cfcd@ideasonboard.com> In-Reply-To: <20260618-rppx1-ipa-v1-0-32337264cfcd@ideasonboard.com> To: =?utf-8?q?Niklas_S=C3=B6derlund?= , libcamera-devel@lists.libcamera.org Cc: Jacopo Mondi X-Mailer: b4 0.14.3 X-Developer-Signature: v=1; a=openpgp-sha256; l=4428; i=jacopo.mondi@ideasonboard.com; h=from:subject:message-id; bh=h04KN9mLyfrik8nFAtKM57O5ZCpidFxhIftsvizNnko=; b=owEBbQKS/ZANAwAKAXI0Bo8WoVY8AcsmYgBqM8YKrvJd9kHlySs+WHGLxm0brTfWhahGEtcco 8lLe6uuZG2JAjMEAAEKAB0WIQS1xD1IgJogio9YOMByNAaPFqFWPAUCajPGCgAKCRByNAaPFqFW PDw5D/491rPiSIw5tx4s682pvIw3tY6JQ3qmVopARuTL79hbttal2hYqvp+WnrkZCjMMNfjhGbR TOacnkuPdDdQur3SliiKJhVJDdKS05FNcw5jMrqrboUherovL0RV4gIbxw89T5zqZiVk1ySqbVE Qu9PurWzM/9TRK48qOoWbOGTr3j1BGNOfz5ospi+oJZbjp1azFAZSF+QJGqmKNLpw4UrNp1tSIQ Ii394aImG4e/NL+MLQbiOTf9C1h6oZFwZm/DW+Km11+ReB6pMgm/ursCfdaw8hH3InmQAJ8STZA +IO5n68VEVeDK7cw2Y58FAAM1NFr+DyN6THg6gdN2C4DHhuSjxUw6nmLw8rTBTiLmrNG19FZNk5 crmXQFsvJHPgwneayzyv6QogIWJsP1XZELBNZNVuKx/kXc/yzm7CkeZqcVkDjJsgaRHn5qv86R9 jlgtlyWc07juSEMQaJKSwNcL0mBLnTK5cs3jw8CD5aV5wnR6n27XD9dZ/hajUeC8mPUbfYs369Q OhFEcBCqzf4+wZMGcN1GhJOEh07kUzXm7XgURjpWJq8njx3x5VXjdp4WEMNYZBZtpp/M7PTiSIA V5/c4FZMR7Go3v9SCJKu2w1GnNWbOnEzf3YCITNIcPnSvHewHjOuVwKS70zQ8wxRSHWceAAzUKV wr+EIo82DHyHx8g== X-Developer-Key: i=jacopo.mondi@ideasonboard.com; a=openpgp; fpr=72392EDC88144A65C701EA9BA5826A2587AD026B X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Add RppX1Params and RppX1Stats classes that implement V4L2Params and V4L2Stats for the RPP-X1 ISP. Signed-off-by: Jacopo Mondi --- src/ipa/rppx1/params.h | 108 +++++++++++++++++++++++++++++++++++++++++++++++++ src/ipa/rppx1/stats.h | 62 ++++++++++++++++++++++++++++ 2 files changed, 170 insertions(+) diff --git a/src/ipa/rppx1/params.h b/src/ipa/rppx1/params.h new file mode 100644 index 000000000000..a7f927372c1d --- /dev/null +++ b/src/ipa/rppx1/params.h @@ -0,0 +1,108 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2026, Ideas On Board + * + * RPP-X1 ISP Parameters + */ + +#pragma once + +#include + +#include + +namespace libcamera { + +namespace ipa::rppx1 { + +enum class BlockType : uint16_t { + BlsPre1, + LinPre1, + LscPre1, + AwbgPre1, + CcorPost, + GaHv, + WbmeasPost, + HistPost, + ExmPre1, +}; + +namespace details { + +template +struct block_type { +}; + +#define RPPX1_DEFINE_BLOCK_TYPE(blkType, cfgType, id) \ +template<> \ +struct block_type { \ + using type = struct rppx1_##cfgType##_params; \ + static constexpr rppx1_params_block_type blockType = \ + RPPX1_PARAMS_BLOCK_TYPE_##id; \ +}; + +RPPX1_DEFINE_BLOCK_TYPE(BlsPre1, bls, BLS_PRE1) +RPPX1_DEFINE_BLOCK_TYPE(LinPre1, lin, LIN_PRE1) +RPPX1_DEFINE_BLOCK_TYPE(LscPre1, lsc, LSC_PRE1) +RPPX1_DEFINE_BLOCK_TYPE(AwbgPre1, awbg, AWBG_PRE1) +RPPX1_DEFINE_BLOCK_TYPE(CcorPost, ccor, CCOR_POST) +RPPX1_DEFINE_BLOCK_TYPE(GaHv, ga, GA_HV) +RPPX1_DEFINE_BLOCK_TYPE(WbmeasPost, wbmeas, WBMEAS_POST) +RPPX1_DEFINE_BLOCK_TYPE(HistPost, hist, HIST_POST) +RPPX1_DEFINE_BLOCK_TYPE(ExmPre1, exm, EXM_PRE1) + +struct params_traits { + using id_type = BlockType; + + template + using id_to_details = block_type; +}; + +} /* namespace details */ + +template +class RppX1ParamsBlock final : public V4L2ParamsBlock +{ +public: + RppX1ParamsBlock(const Span data) + : V4L2ParamsBlock(data), + configData_(data.subspan(sizeof(v4l2_isp_block_header))) + { + } + + const T *operator->() const override + { + return reinterpret_cast(configData_.data()); + } + + T *operator->() override + { + return reinterpret_cast(configData_.data()); + } + + const T &operator*() const override + { + return *reinterpret_cast(configData_.data()); + } + + T &operator*() override + { + return *reinterpret_cast(configData_.data()); + } + +private: + Span configData_; +}; + +class RppX1Params : public V4L2Params +{ +public: + RppX1Params(Span data) + : V4L2Params(data, V4L2_ISP_PARAMS_VERSION_V1) + { + } +}; + +} /* namespace ipa::rppx1 */ + +} /* namespace libcamera */ diff --git a/src/ipa/rppx1/stats.h b/src/ipa/rppx1/stats.h new file mode 100644 index 000000000000..71a3e1da505c --- /dev/null +++ b/src/ipa/rppx1/stats.h @@ -0,0 +1,62 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2026, Ideas On Board + * + * RPP-X1 ISP Statistics + */ + +#pragma once + +#include + +#include + +namespace libcamera { + +namespace ipa::rppx1 { + +enum class StatsType : uint16_t { + HistPost, + ExmPre1, + WbmeasPost, +}; + +namespace details { + +template +struct stats_type { +}; + +#define RPPX1_DEFINE_STATS_TYPE(blkType, cfgType, id) \ +template<> \ +struct stats_type { \ + using type = struct rppx1_##cfgType##_stats; \ + static constexpr rppx1_stats_block_type blockType = \ + RPPX1_STATS_BLOCK_TYPE_##id; \ +}; + +RPPX1_DEFINE_STATS_TYPE(HistPost, hist, HIST_POST) +RPPX1_DEFINE_STATS_TYPE(ExmPre1, exm, EXM_PRE1) +RPPX1_DEFINE_STATS_TYPE(WbmeasPost, wbmeas, WBMEAS_POST) + +struct stats_traits { + using id_type = StatsType; + + template + using id_to_details = stats_type; +}; + +} /* namespace details */ + +class RppX1Stats : public V4L2Stats +{ +public: + RppX1Stats(Span data) + : V4L2Stats(data, V4L2_ISP_VERSION_V1) + { + } +}; + +} /* namespace ipa::rppx1 */ + +} /* namespace libcamera */ From patchwork Thu Jun 18 10:18:50 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jacopo Mondi X-Patchwork-Id: 26926 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 06E76C3308 for ; Thu, 18 Jun 2026 10:19:11 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id B0C5562B2E; Thu, 18 Jun 2026 12:19:10 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="GdyoZiod"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id CCCD1629E1 for ; Thu, 18 Jun 2026 12:19:00 +0200 (CEST) Received: from [192.168.125.177] (mob-109-113-4-199.net.vodafone.it [109.113.4.199]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 34B242908; Thu, 18 Jun 2026 12:18:25 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1781777905; bh=FuZvmNCSrt8DCEum16FJpPu5QVKLW50Nfcxw0/qrA5M=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=GdyoZiodwUkP6lTWld0g0Zj3GsF6UTt6h3QWfFoAnJbReDMbnj+ifFYrrH7DgwVeR F/8w6Z7hi4wwjKEQtYd9QHf/kqDScmarK1fDhj+NzUKZqSqJfvWtTCEIIaYjk0+X76 AfaQvbqnJO8Fij032iBfoBRPT/vw45ubu9K0xKCY= From: Jacopo Mondi Date: Thu, 18 Jun 2026 12:18:50 +0200 Subject: [PATCH 11/14] ipa: rppx1: Add RPP-X1 IPA skeleton MIME-Version: 1.0 Message-Id: <20260618-rppx1-ipa-v1-11-32337264cfcd@ideasonboard.com> References: <20260618-rppx1-ipa-v1-0-32337264cfcd@ideasonboard.com> In-Reply-To: <20260618-rppx1-ipa-v1-0-32337264cfcd@ideasonboard.com> To: =?utf-8?q?Niklas_S=C3=B6derlund?= , libcamera-devel@lists.libcamera.org Cc: Jacopo Mondi X-Mailer: b4 0.14.3 X-Developer-Signature: v=1; a=openpgp-sha256; l=17071; i=jacopo.mondi@ideasonboard.com; h=from:subject:message-id; bh=FuZvmNCSrt8DCEum16FJpPu5QVKLW50Nfcxw0/qrA5M=; b=owEBbQKS/ZANAwAKAXI0Bo8WoVY8AcsmYgBqM8YKnIYD382Xq+EpMicwe0ueol0wfpqrFE8AO fZtXe14XsyJAjMEAAEKAB0WIQS1xD1IgJogio9YOMByNAaPFqFWPAUCajPGCgAKCRByNAaPFqFW PHGfD/9GoZDbW/Flyyq+HhMYKzyja8MzKdrCi/iPbW3HWkL9eF4ecwZDMo7PKmISFUsG2vVIwSD Px8gxViN2flKJ0np3Ose1Q7Ktd0vT8bZf47TDyrHTAilbEcsDkcscIMDr3TNhR+4Ej2bUWoMGqd EcpCakVCJZb/Wx0q1+E3A9WL+ZKI1WnxQdyWFZ1x95Hhmrfcky3qJYmzUudn05IFBdmYcYwvehz m/ceig2wU4/iLqS9qWG7e49imhDWdhM5b1Pcni9PcSlQ75h5Ufji1+0rbuHvUkuPuLALWTGS9QM A6qZ6uLNEV9jC/vOO81GglcMBMYW5vTdDI2/i9nBehcmcEf2JZ+piCxYJcgetU3EQwVg10Kdpy7 aYY5jO1SvFjNyc3RXDbZoL60mfRWN4kGxS50wTaCwvORKqU9t3hlcIJOkRFNkxucAeH6zhYvgh+ msTbDB6TQJ5kKkHcOD1dSrUYyntb59pM8TkOhvU1BGWvhLacDy6elULMtt4WZSRViUdVLKDMsyE TBFtGftMkjDw4RCW4Z8058UpB0i1XOB9EbI3f9V3JQp/ZU4uj6uRy4pJiPCEqrYFVKkfPtgSvYA xLyKLN5yT9L1iGXQW8t13FMHCPqidU9oJw0naq2QnbC4zNIoM3/izCRHps9IqJuI09QqXs6Hczt 8P7bKQEMy0g5cqg== X-Developer-Key: i=jacopo.mondi@ideasonboard.com; a=openpgp; fpr=72392EDC88144A65C701EA9BA5826A2587AD026B X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Add the infrastructure for the RPP-X1 without any registered algorithm but with the definition of the IPA module interface and the IPA context. Signed-off-by: Jacopo Mondi --- Documentation/Doxyfile-common.in | 1 + include/libcamera/ipa/meson.build | 1 + include/libcamera/ipa/rppx1.mojom | 41 ++++++ meson_options.txt | 4 +- src/ipa/meson.build | 1 + src/ipa/rppx1/ipa_context.cpp | 63 +++++++++ src/ipa/rppx1/ipa_context.h | 55 ++++++++ src/ipa/rppx1/meson.build | 26 ++++ src/ipa/rppx1/module.h | 27 ++++ src/ipa/rppx1/rppx1.cpp | 280 ++++++++++++++++++++++++++++++++++++++ 10 files changed, 497 insertions(+), 2 deletions(-) diff --git a/Documentation/Doxyfile-common.in b/Documentation/Doxyfile-common.in index f11ec593d5ae..131fdcc6083f 100644 --- a/Documentation/Doxyfile-common.in +++ b/Documentation/Doxyfile-common.in @@ -37,6 +37,7 @@ EXCLUDE_PATTERNS = @TOP_BUILDDIR@/include/libcamera/ipa/*_serializer.h \ @TOP_BUILDDIR@/include/libcamera/ipa/mali-c55_*.h \ @TOP_BUILDDIR@/include/libcamera/ipa/raspberrypi_*.h \ @TOP_BUILDDIR@/include/libcamera/ipa/rkisp1_*.h \ + @TOP_BUILDDIR@/include/libcamera/ipa/rppx1_*.h \ @TOP_BUILDDIR@/include/libcamera/ipa/vimc_*.h EXCLUDE_SYMBOLS = libcamera::BoundMethodArgs \ diff --git a/include/libcamera/ipa/meson.build b/include/libcamera/ipa/meson.build index 3ee3ada303c0..9ac15546102f 100644 --- a/include/libcamera/ipa/meson.build +++ b/include/libcamera/ipa/meson.build @@ -66,6 +66,7 @@ pipeline_ipa_mojom_mapping = { 'ipu3': 'ipu3.mojom', 'mali-c55': 'mali-c55.mojom', 'rkisp1': 'rkisp1.mojom', + 'rcar-gen4': 'rppx1.mojom', 'rpi/pisp': 'raspberrypi.mojom', 'rpi/vc4': 'raspberrypi.mojom', 'simple': 'soft.mojom', diff --git a/include/libcamera/ipa/rppx1.mojom b/include/libcamera/ipa/rppx1.mojom new file mode 100644 index 000000000000..4abbebb24f85 --- /dev/null +++ b/include/libcamera/ipa/rppx1.mojom @@ -0,0 +1,41 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ + +/* + * \todo Document the interface and remove the related EXCLUDE_PATTERNS entry. + */ + +module ipa.rppx1; + +import "include/libcamera/ipa/core.mojom"; + +struct IPAConfigInfo { + libcamera.IPACameraSensorInfo sensorInfo; + libcamera.ControlInfoMap sensorControls; +}; + +interface IPARppX1Interface { + init(libcamera.IPASettings settings, + libcamera.IPACameraSensorInfo sensorInfo, + libcamera.ControlInfoMap sensorControls) + => (int32 ret, libcamera.ControlInfoMap ipaControls); + start() => (int32 ret); + stop(); + + configure(IPAConfigInfo configInfo, + map streamConfig) + => (int32 ret, libcamera.ControlInfoMap ipaControls); + + mapBuffers(array buffers); + unmapBuffers(array ids); + + [async] queueRequest(uint32 frame, libcamera.ControlList reqControls); + [async] computeParams(uint32 frame, uint32 bufferId); + [async] processStats(uint32 frame, uint32 bufferId, + libcamera.ControlList sensorControls); +}; + +interface IPARppX1EventInterface { + paramsComputed(uint32 frame, uint32 bytesused); + setSensorControls(uint32 frame, libcamera.ControlList sensorControls); + metadataReady(uint32 frame, libcamera.ControlList metadata); +}; diff --git a/meson_options.txt b/meson_options.txt index 20baacc4fc65..0aa3aef24edd 100644 --- a/meson_options.txt +++ b/meson_options.txt @@ -48,8 +48,8 @@ option('gstreamer', option('ipas', type : 'array', - choices : ['ipu3', 'mali-c55', 'rkisp1', 'rpi/pisp', 'rpi/vc4', 'simple', - 'vimc'], + choices : ['ipu3', 'mali-c55', 'rkisp1', 'rppx1', 'rpi/pisp', 'rpi/vc4', + 'simple', 'vimc'], description : 'Select which IPA modules to build') option('lc-compliance', diff --git a/src/ipa/meson.build b/src/ipa/meson.build index c583c7efdd35..411d043021f8 100644 --- a/src/ipa/meson.build +++ b/src/ipa/meson.build @@ -27,6 +27,7 @@ ipa_sign = files('ipa-sign.sh') supported_ipas = { 'ipu3': 'ipu3', 'mali-c55': 'mali-c55', + 'rcar-gen4': 'rppx1', 'rkisp1': 'rkisp1', 'rpi/pisp': 'rpi/pisp', 'rpi/vc4': 'rpi/vc4', diff --git a/src/ipa/rppx1/ipa_context.cpp b/src/ipa/rppx1/ipa_context.cpp new file mode 100644 index 000000000000..5ac60dfc7bf6 --- /dev/null +++ b/src/ipa/rppx1/ipa_context.cpp @@ -0,0 +1,63 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2026 Renesas Electronics Corp. + * Copyright (C) 2026 Ideas on Board Oy + * Copyright (C) 2026 Ragnatech AB + * + * RPP-X1 IPA Context + */ + +#include "ipa_context.h" + +/** + * \file ipa_context.h + * \brief Context and state information shared between the algorithms + */ + +namespace libcamera::ipa::rppx1 { + +/** + * \struct IPASessionConfiguration + * \brief Session configuration for the IPA module + */ + +/** + * \struct IPAActiveState + * \brief Active state for algorithms + */ + +/** + * \struct IPAFrameContext + * \brief Per-frame context for algorithms + */ + +/** + * \struct IPAContext + * \brief Global IPA context data shared between all algorithms + * + * \var IPAContext::sensorInfo + * \brief The IPA session sensorInfo, immutable during the session + * + * \var IPAContext::configuration + * \brief The IPA session configuration, immutable during the session + * + * \var IPAContext::activeState + * \brief The IPA active state, storing the latest state for all algorithms + * + * \var IPAContext::frameContexts + * \brief Ring buffer of per-frame contexts + * + * \var IPAContext::ctrlMap + * \brief The IPA map of controls + * + * \var IPAContext::camHelper + * \brief The IPA camera helper + */ + +/** + * \fn IPAContext::IPAContext + * \brief Construct the IPA context + * \param[in] frameContextSize Size of the frame context queue + */ + +} /* namespace libcamera::ipa::rppx1 */ diff --git a/src/ipa/rppx1/ipa_context.h b/src/ipa/rppx1/ipa_context.h new file mode 100644 index 000000000000..f268a35081bc --- /dev/null +++ b/src/ipa/rppx1/ipa_context.h @@ -0,0 +1,55 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2026 Renesas Electronics Corp. + * Copyright (C) 2026 Ideas on Board Oy + * Copyright (C) 2026 Ragnatech AB + * + * RPP-X1 IPA Context + */ + +#pragma once + +#include + +#include + +#include + +#include + +#include +#include + +namespace libcamera { + +namespace ipa::rppx1 { + +struct IPASessionConfiguration { +}; + +struct IPAActiveState { +}; + +struct IPAFrameContext : public FrameContext { +}; + +struct IPAContext { + IPAContext(unsigned int frameContextSize) + : frameContexts(frameContextSize) + { + } + + IPACameraSensorInfo sensorInfo; + IPASessionConfiguration configuration; + IPAActiveState activeState; + + FCQueue frameContexts; + + ControlInfoMap::Map ctrlMap; + + std::unique_ptr camHelper; +}; + +} /* namespace ipa::rppx1 */ + +} /* namespace libcamera*/ diff --git a/src/ipa/rppx1/meson.build b/src/ipa/rppx1/meson.build new file mode 100644 index 000000000000..8034fe24d241 --- /dev/null +++ b/src/ipa/rppx1/meson.build @@ -0,0 +1,26 @@ +# SPDX-License-Identifier: CC0-1.0 + +ipa_name = 'ipa_rppx1' + +rppx1_ipa_sources = files([ + 'ipa_context.cpp', + 'rppx1.cpp', +]) + +mod = shared_module(ipa_name, rppx1_ipa_sources, + name_prefix : '', + include_directories : [ipa_includes], + dependencies : [libcamera_private, libipa_dep], + install : true, + install_dir : ipa_install_dir) + +if ipa_sign_module + custom_target(ipa_name + '.so.sign', + input : mod, + output : ipa_name + '.so.sign', + command : [ipa_sign, ipa_priv_key, '@INPUT@', '@OUTPUT@'], + install : false, + build_by_default : true) +endif + +ipa_names += ipa_name diff --git a/src/ipa/rppx1/module.h b/src/ipa/rppx1/module.h new file mode 100644 index 000000000000..0bacff64de8f --- /dev/null +++ b/src/ipa/rppx1/module.h @@ -0,0 +1,27 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2026 Renesas Electronics Corp. + * Copyright (C) 2026 Ideas on Board Oy + * Copyright (C) 2026 Ragnatech AB + * + * RPP-X1 IPA Module + */ + +#pragma once + +#include + +#include "ipa_context.h" +#include "params.h" +#include "stats.h" + +namespace libcamera { + +namespace ipa::rppx1 { + +using Module = ipa::Module; + +} /* namespace ipa::rppx1 */ + +} /* namespace libcamera*/ diff --git a/src/ipa/rppx1/rppx1.cpp b/src/ipa/rppx1/rppx1.cpp new file mode 100644 index 000000000000..20ca2d292704 --- /dev/null +++ b/src/ipa/rppx1/rppx1.cpp @@ -0,0 +1,280 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2026 Renesas Electronics Corp. + * Copyright (C) 2026 Ideas on Board Oy + * Copyright (C) 2026 Ragnatech AB + * + * RPP-X1 Image Processing Algorithms + */ + +#include +#include +#include + +#include +#include + +#include +#include + +#include +#include +#include + +#include "libcamera/internal/formats.h" +#include "libcamera/internal/mapped_framebuffer.h" +#include "libcamera/internal/yaml_parser.h" + +#include "algorithms/algorithm.h" + +#include "ipa_context.h" +#include "params.h" +#include "stats.h" + +namespace libcamera { + +LOG_DEFINE_CATEGORY(IPARppX1) + +namespace ipa::rppx1 { + +/* Maximum number of frame contexts to be held */ +static constexpr uint32_t kMaxFrameContexts = 16; + +class IPARppX1 : public IPARppX1Interface, public Module +{ +public: + IPARppX1(); + + int init(const IPASettings &settings, + const IPACameraSensorInfo &sensorInfo, + const ControlInfoMap &sensorControls, + ControlInfoMap *ipaControls) override; + int start() override; + void stop() override; + + int configure(const IPAConfigInfo &ipaConfig, + const std::map &streamConfig, + ControlInfoMap *ipaControls) override; + void mapBuffers(const std::vector &buffers) override; + void unmapBuffers(const std::vector &ids) override; + + void queueRequest(const uint32_t frame, const ControlList &controls) override; + void computeParams(const uint32_t frame, const uint32_t bufferId) override; + void processStats(const uint32_t frame, const uint32_t bufferId, + const ControlList &sensorControls) override; + +protected: + std::string logPrefix() const override; + +private: + void updateControls(ControlInfoMap *ipaControls); + + std::map buffers_; + std::map mappedBuffers_; + + /* Local parameter storage */ + struct IPAContext context_; +}; + +IPARppX1::IPARppX1() + : context_(kMaxFrameContexts) +{ +} + +std::string IPARppX1::logPrefix() const +{ + return "rppx1"; +} + +void IPARppX1::updateControls(ControlInfoMap *ipaControls) +{ + ControlInfoMap::Map ctrlMap; + + ctrlMap.insert(context_.ctrlMap.begin(), context_.ctrlMap.end()); + *ipaControls = ControlInfoMap(std::move(ctrlMap), controls::controls); +} + +int IPARppX1::init(const IPASettings &settings, + const IPACameraSensorInfo &sensorInfo, + [[maybe_unused]] const ControlInfoMap &sensorControls, + [[maybe_unused]] ControlInfoMap *ipaControls) +{ + context_.sensorInfo = sensorInfo; + + context_.camHelper = CameraSensorHelperFactoryBase::create(settings.sensorModel); + if (!context_.camHelper) { + LOG(IPARppX1, Error) + << "Failed to create camera sensor helper for " + << settings.sensorModel; + return -ENODEV; + } + + /* Load the tuning data file. */ + File file(settings.configurationFile); + if (!file.open(File::OpenModeFlag::ReadOnly)) { + int ret = file.error(); + LOG(IPARppX1, Error) + << "Failed to open configuration file " + << settings.configurationFile << ": " << strerror(-ret); + return ret; + } + + std::unique_ptr data = YamlParser::parse(file); + if (!data) + return -EINVAL; + + unsigned int version = (*data)["version"].get(0); + if (version != 1) { + LOG(IPARppX1, Error) + << "Invalid tuning file version " << version; + return -EINVAL; + } + + if (!data->contains("algorithms")) { + LOG(IPARppX1, Error) + << "Tuning file doesn't contain any algorithm"; + return -EINVAL; + } + + int ret = createAlgorithms(context_, (*data)["algorithms"]); + if (ret) + return ret; + + /* Initialize controls. */ + updateControls(ipaControls); + + return 0; +} + +int IPARppX1::start() +{ + /* \todo Properly handle startup controls. */ + return 0; +} + +void IPARppX1::stop() +{ + context_.frameContexts.clear(); +} + +int IPARppX1::configure(const IPAConfigInfo &ipaConfig, + [[maybe_unused]] const std::map &streamConfig, + [[maybe_unused]] ControlInfoMap *ipaControls) +{ + /* Clear the IPA context before the streaming session. */ + context_.configuration = {}; + context_.activeState = {}; + context_.frameContexts.clear(); + + const IPACameraSensorInfo &info = ipaConfig.sensorInfo; + + /* Update the camera controls using the new sensor settings. */ + updateControls(ipaControls); + + for (auto const &a : algorithms()) { + Algorithm *algo = static_cast(a.get()); + + int ret = algo->configure(context_, info); + if (ret) + return ret; + } + + return 0; +} + +void IPARppX1::mapBuffers(const std::vector &buffers) +{ + for (const IPABuffer &buffer : buffers) { + auto elem = buffers_.emplace(std::piecewise_construct, + std::forward_as_tuple(buffer.id), + std::forward_as_tuple(buffer.planes)); + const FrameBuffer &fb = elem.first->second; + + MappedFrameBuffer mappedBuffer(&fb, MappedFrameBuffer::MapFlag::ReadWrite); + if (!mappedBuffer.isValid()) { + LOG(IPARppX1, Fatal) << "Failed to mmap buffer: " + << strerror(mappedBuffer.error()); + } + + mappedBuffers_.emplace(buffer.id, std::move(mappedBuffer)); + } +} + +void IPARppX1::unmapBuffers(const std::vector &ids) +{ + for (unsigned int id : ids) { + const auto fb = buffers_.find(id); + if (fb == buffers_.end()) + continue; + + mappedBuffers_.erase(id); + buffers_.erase(id); + } +} + +void IPARppX1::queueRequest(const uint32_t frame, const ControlList &controls) +{ + IPAFrameContext &frameContext = context_.frameContexts.alloc(frame); + + for (auto const &a : algorithms()) { + Algorithm *algo = static_cast(a.get()); + if (algo->disabled_) + continue; + + algo->queueRequest(context_, frame, frameContext, controls); + } +} + +void IPARppX1::computeParams(const uint32_t frame, const uint32_t bufferId) +{ + IPAFrameContext &frameContext = context_.frameContexts.get(frame); + + RppX1Params params(mappedBuffers_.at(bufferId).planes()[0]); + + for (auto const &algo : algorithms()) + algo->prepare(context_, frame, frameContext, ¶ms); + + paramsComputed.emit(frame, params.bytesused()); +} + +void IPARppX1::processStats(const uint32_t frame, const uint32_t bufferId, + [[maybe_unused]] const ControlList &sensorControls) +{ + IPAFrameContext &frameContext = context_.frameContexts.get(frame); + + ControlList metadata(controls::controls); + + auto stats = RppX1Stats(mappedBuffers_.at(bufferId).planes()[0]); + if (!stats) + return; + + for (auto const &a : algorithms()) { + Algorithm *algo = static_cast(a.get()); + if (algo->disabled_) + continue; + algo->process(context_, frame, frameContext, &stats, metadata); + } + + metadataReady.emit(frame, metadata); +} + +} /* namespace ipa::rppx1 */ + +/* + * External IPA module interface + */ + +extern "C" { +const struct IPAModuleInfo ipaModuleInfo = { + IPA_MODULE_API_VERSION, + 1, + "rppx1", +}; + +IPAInterface *ipaCreate() +{ + return new ipa::rppx1::IPARppX1(); +} +} + +} /* namespace libcamera */ From patchwork Thu Jun 18 10:18:51 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Jacopo Mondi X-Patchwork-Id: 26927 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id DA6D1C330A for ; Thu, 18 Jun 2026 10:19:13 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 27C7562C77; Thu, 18 Jun 2026 12:19:13 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="I/KK9le/"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 1E6D862B32 for ; Thu, 18 Jun 2026 12:19:02 +0200 (CEST) Received: from [192.168.125.177] (mob-109-113-4-199.net.vodafone.it [109.113.4.199]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 5E72F3A2; Thu, 18 Jun 2026 12:18:26 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1781777907; bh=uUlwPE2N/1dLbkXLow8fsaI/6yax3sBufgt42FoSy1A=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=I/KK9le/MOfx8LSHXZq7f15WS4dJH9M69vBaxG/0MQ/r6t0NMy0Rqk2svtb3jbblS jPoYYe7f0f55AabS9VCTLbv6etPWkhbV3wlJOtPO36+r2TNhEqtVfCpSr1vQvmUCHX fn1nHxElK0XcYduWb1uaqtr8nReCUfV+wM4jgdgc= From: Jacopo Mondi Date: Thu, 18 Jun 2026 12:18:51 +0200 Subject: [PATCH 12/14] libcamera: pipeline: Add R-Car Gen4 ISP pipeline MIME-Version: 1.0 Message-Id: <20260618-rppx1-ipa-v1-12-32337264cfcd@ideasonboard.com> References: <20260618-rppx1-ipa-v1-0-32337264cfcd@ideasonboard.com> In-Reply-To: <20260618-rppx1-ipa-v1-0-32337264cfcd@ideasonboard.com> To: =?utf-8?q?Niklas_S=C3=B6derlund?= , libcamera-devel@lists.libcamera.org Cc: Jacopo Mondi X-Mailer: b4 0.14.3 X-Developer-Signature: v=1; a=openpgp-sha256; l=57369; i=jacopo.mondi@ideasonboard.com; h=from:subject:message-id; bh=EGFDUpcuV6phkwMkhNczSnI0wY6UWKLZrUrwQnEFais=; b=owEBbQKS/ZANAwAKAXI0Bo8WoVY8AcsmYgBqM8YKbhy2uStw2mkIbv+4cRx95xy+BuXaHGFIs rfYq6Y+8imJAjMEAAEKAB0WIQS1xD1IgJogio9YOMByNAaPFqFWPAUCajPGCgAKCRByNAaPFqFW PMYOD/4taPtuxXXw3IeppmOwiFfcAPb+2EwMju8WN7KS8R4mY1xDv+P+DeSmr0+XiKUfG1haYmQ mfX0VR9Tpeopp+f8A1o42BgHwvUGY1+mxhlngDukUko3OrMdHI4HJmCn1J6KWcEZx9Pz9I8swE6 AAQs0siRC+KvnRl6ySyVJ56hIEwdyM2SCK9Fz8ERmVEl/BE2dJIpAc9BZz2Al9oyPCqyiqYBdaN zh6vi6b1doLsRkKuEwWda6Hlx4a0j91jK0gT/b8cKj6p7T+sioNaSOdM0D1b4pgSlakYEEO0S6n /frA9uMuO4quSA5V3Z+7gpXvtY5/41d+gpxU1uH3gctZW69LvP9jkFIBRnbGmbz2UrX8deRXbok zb8LKSnwznLEqr/4UKiFj2jI9+zmeuDPbGjAJKI8OMexPcr0GaO+1/Zq00fIp2MZfGNYjtxIPFP Nmmt4Qw96LX/xO4sEf9yomZ7Pejv8VCrY/mLqvoqb4EqrTnw+TM8oECrRV6JLu6pb37/EBTgnBN VnnMsoLUJdNzZt1NiE3ZvuGzUldK4gV7pI+33t1rOXFe+sZ3OS/lQk/jBIvyYjJWwd8pZ0yww0C R6WuouOHVytmJ3AKaQfPxKdjOatn61j92Gxyzor0qeCxd9UtH+D0vTKF4uBJj/63QuzvL3p5yt+ KIVK/2XSSbOqvbg== X-Developer-Key: i=jacopo.mondi@ideasonboard.com; a=openpgp; fpr=72392EDC88144A65C701EA9BA5826A2587AD026B X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" From: Niklas Söderlund Add a pipeline handler for R-Car Gen4. The pipeline makes use of the RkISP1 IPA and, depending on the kernel R-Car ISP driver, support the same features as the RkISP1 pipeline handler. The pipeline and IPA is tested with the AWB measurement, Histogram Measurement, Auto Exposure Measurement, AWB gain, Black Level Correction, Color Correction Matrix, Lens Shade Correction and Gamma Correction algorithms and produce a stable and good image. Tested on R-Car V4H Sparrow-Hawk together with IMX219 and IMX462 sensors. Signed-off-by: Niklas Söderlund --- * Changes in v2 - Use the RPP-X1 pipeline - Use RPP-X1 ISP formats - Rebase on recent master * Changes since v1 - Fix camera names. Was 'imx462 2-001a', is now '/base/soc/i2c@e6510000/cam@1a'. - Fix a compiler issue with some versions of gcc. - Add dependency on rkisp1.mojom. - Rebase on latest master branch which requires reworking some interfaces. --- meson.build | 1 + meson_options.txt | 1 + src/libcamera/pipeline/rcar-gen4/frames.cpp | 281 +++++++++ src/libcamera/pipeline/rcar-gen4/frames.h | 86 +++ src/libcamera/pipeline/rcar-gen4/isp.cpp | 228 +++++++ src/libcamera/pipeline/rcar-gen4/isp.h | 45 ++ src/libcamera/pipeline/rcar-gen4/meson.build | 8 + src/libcamera/pipeline/rcar-gen4/rcar-gen4.cpp | 810 +++++++++++++++++++++++++ src/libcamera/pipeline/rcar-gen4/vin.cpp | 391 ++++++++++++ src/libcamera/pipeline/rcar-gen4/vin.h | 67 ++ 10 files changed, 1918 insertions(+) diff --git a/meson.build b/meson.build index 9cd73462f414..ec6b768abef9 100644 --- a/meson.build +++ b/meson.build @@ -218,6 +218,7 @@ pipelines_support = { 'imx8-isi': arch_arm, 'ipu3': arch_x86, 'mali-c55': arch_arm, + 'rcar-gen4': arch_arm, 'rkisp1': arch_arm, 'rpi/pisp': arch_arm, 'rpi/vc4': arch_arm, diff --git a/meson_options.txt b/meson_options.txt index 0aa3aef24edd..f4f085f31d0e 100644 --- a/meson_options.txt +++ b/meson_options.txt @@ -81,6 +81,7 @@ option('pipelines', 'imx8-isi', 'ipu3', 'mali-c55', + 'rcar-gen4', 'rkisp1', 'rpi/pisp', 'rpi/vc4', diff --git a/src/libcamera/pipeline/rcar-gen4/frames.cpp b/src/libcamera/pipeline/rcar-gen4/frames.cpp new file mode 100644 index 000000000000..cb7d2b73c440 --- /dev/null +++ b/src/libcamera/pipeline/rcar-gen4/frames.cpp @@ -0,0 +1,281 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2026 Ideas on Board Oy + * Copyright (C) 2025 Renesas Electronics Co + * Copyright (C) 2025 Niklas Söderlund + * + * Renesas R-Car Gen4 VIN pipeline + */ + +#include "frames.h" + +#include + +#include +#include + +#include "libcamera/internal/framebuffer.h" +#include "libcamera/internal/pipeline_handler.h" + +namespace libcamera { + +LOG_DECLARE_CATEGORY(RCar4) + +int RCar4Frames::start(class RCarISPDevice *isp, class ipa::rppx1::IPAProxyRppX1 *ipa) +{ + unsigned int ipaBufferId = 1; + unsigned int bufferCount; + int ret; + + auto pushBuffers = [&](const std::vector> &buffers, + std::queue &queue) { + for (const std::unique_ptr &buffer : buffers) { + Span planes = buffer->planes(); + + buffer->setCookie(ipaBufferId++); + ipaBuffers_.emplace_back(buffer->cookie(), + std::vector{ planes.begin(), + planes.end() }); + queue.push(buffer.get()); + } + }; + + frameInfo_.clear(); + + bufferCount = std::max({ + rawStream_.configuration().bufferCount, + outputStream_.configuration().bufferCount, + }); + + ret = isp->input_->exportBuffers(bufferCount, &inputBuffers_); + if (ret < 0) { + LOG(RCar4, Error) << "Failed to allocate ISP input buffers"; + goto error; + } + + ret = isp->param_->allocateBuffers(bufferCount, ¶mBuffers_); + if (ret < 0) { + LOG(RCar4, Error) << "Failed to allocate ISP param buffers"; + goto error; + } + + ret = isp->stat_->allocateBuffers(bufferCount, &statBuffers_); + if (ret < 0) { + LOG(RCar4, Error) << "Failed to allocate ISP stat buffers"; + goto error; + } + + ret = isp->output_->exportBuffers(bufferCount, &outputBuffers_); + if (ret < 0) { + LOG(RCar4, Error) << "Failed to allocate ISP output buffers"; + goto error; + } + + for (const std::unique_ptr &buffer : inputBuffers_) + availableInputBuffers_.push(buffer.get()); + + pushBuffers(paramBuffers_, availableParamBuffers_); + pushBuffers(statBuffers_, availableStatBuffers_); + + for (const std::unique_ptr &buffer : outputBuffers_) + availableOutputBuffers_.push(buffer.get()); + + ipa->mapBuffers(ipaBuffers_); + + return 0; +error: + stop(isp, ipa); + return ret; +} + +void RCar4Frames::stop(class RCarISPDevice *isp, class ipa::rppx1::IPAProxyRppX1 *ipa) +{ + std::vector ids; + + availableInputBuffers_ = {}; + availableParamBuffers_ = {}; + availableStatBuffers_ = {}; + availableOutputBuffers_ = {}; + + outputBuffers_.clear(); + statBuffers_.clear(); + paramBuffers_.clear(); + inputBuffers_.clear(); + + for (IPABuffer &ipabuf : ipaBuffers_) + ids.push_back(ipabuf.id); + + ipa->unmapBuffers(ids); + ipaBuffers_.clear(); + + if (isp->output_->releaseBuffers()) + LOG(RCar4, Error) << "Failed to release ISP output buffers"; + + if (isp->stat_->releaseBuffers()) + LOG(RCar4, Error) << "Failed to release ISP stat buffers"; + + if (isp->param_->releaseBuffers()) + LOG(RCar4, Error) << "Failed to release ISP param buffers"; + + if (isp->input_->releaseBuffers()) + LOG(RCar4, Error) << "Failed to release ISP input buffers"; +} + +RCar4Frames::Info *RCar4Frames::create(Request *request) +{ + unsigned int frame = request->sequence(); + + /* Try to get input and output buffers from request. */ + FrameBuffer *inputBuffer = request->findBuffer(&rawStream_); + FrameBuffer *outputBuffer = request->findBuffer(&outputStream_); + + /* Make sure we have enough internal buffers. */ + if (!inputBuffer && availableInputBuffers_.empty()) { + LOG(RCar4, Debug) << "Input buffer underrun"; + return nullptr; + } + + if (availableParamBuffers_.empty()) { + LOG(RCar4, Debug) << "Parameters buffer underrun"; + return nullptr; + } + + if (availableStatBuffers_.empty()) { + LOG(RCar4, Debug) << "Statistics buffer underrun"; + return nullptr; + } + + if (!outputBuffer && availableOutputBuffers_.empty()) { + LOG(RCar4, Debug) << "Output buffer underrun"; + return nullptr; + } + + /* Select buffers to use. */ + if (!inputBuffer) { + inputBuffer = availableInputBuffers_.front(); + availableInputBuffers_.pop(); + inputBuffer->_d()->setRequest(request); + } + + FrameBuffer *paramBuffer = availableParamBuffers_.front(); + availableParamBuffers_.pop(); + paramBuffer->_d()->setRequest(request); + + FrameBuffer *statBuffer = availableStatBuffers_.front(); + availableStatBuffers_.pop(); + statBuffer->_d()->setRequest(request); + + if (!outputBuffer) { + outputBuffer = availableOutputBuffers_.front(); + availableOutputBuffers_.pop(); + outputBuffer->_d()->setRequest(request); + } + + /* Recored the info needed to process one frame. */ + auto [it, inserted] = frameInfo_.try_emplace(frame); + if (!inserted) + return nullptr; + + auto &info = it->second; + + info.frame = frame; + info.request = request; + info.inputBuffer = inputBuffer; + info.paramBuffer = paramBuffer; + info.statBuffer = statBuffer; + info.outputBuffer = outputBuffer; + info.rawDequeued = false; + info.paramDequeued = false; + info.metadataProcessed = false; + info.outputDequeued = false; + + return &info; +} + +void RCar4Frames::remove(RCar4Frames::Info *info) +{ + /* If internal input buffer used, return for reuse. */ + for (const std::unique_ptr &buf : inputBuffers_) { + if (info->inputBuffer == buf.get()) { + availableInputBuffers_.push(info->inputBuffer); + break; + } + } + + /* Return param and stat buffer for reuse. */ + availableParamBuffers_.push(info->paramBuffer); + availableStatBuffers_.push(info->statBuffer); + + /* If internal output buffer used, return for reuse. */ + for (const std::unique_ptr &buf : outputBuffers_) { + if (info->outputBuffer == buf.get()) { + availableOutputBuffers_.push(info->outputBuffer); + break; + } + } + + /* Delete the extended frame information. */ + frameInfo_.erase(info->frame); +} + +bool RCar4Frames::tryComplete(RCar4Frames::Info *info) +{ + Request *request = info->request; + + if (request->hasPendingBuffers()) + return false; + + if (!info->rawDequeued) + return false; + + if (!info->metadataProcessed) + return false; + + if (!info->paramDequeued) + return false; + + if (!info->outputDequeued) + return false; + + remove(info); + + /* Signal new internal buffers available. */ + bufferAvailable.emit(); + + return true; +} + +RCar4Frames::Info *RCar4Frames::find(unsigned int frame) +{ + const auto &itInfo = frameInfo_.find(frame); + + if (itInfo != frameInfo_.end()) + return &itInfo->second; + + LOG(RCar4, Fatal) << "Can't find tracking information for frame " << frame; + + return nullptr; +} + +RCar4Frames::Info *RCar4Frames::find(FrameBuffer *buffer) +{ + for (auto &itInfo : frameInfo_) { + Info *info = &itInfo.second; + + for (auto const itBuffers : info->request->buffers()) + if (itBuffers.second == buffer) + return info; + + if (info->inputBuffer == buffer || + info->paramBuffer == buffer || + info->statBuffer == buffer || + info->outputBuffer == buffer) + return info; + } + + LOG(RCar4, Fatal) << "Can't find tracking information from buffer"; + + return nullptr; +} + +} /* namespace libcamera */ diff --git a/src/libcamera/pipeline/rcar-gen4/frames.h b/src/libcamera/pipeline/rcar-gen4/frames.h new file mode 100644 index 000000000000..63a76bbad4f8 --- /dev/null +++ b/src/libcamera/pipeline/rcar-gen4/frames.h @@ -0,0 +1,86 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2026 Ideas on Board Oy + * Copyright (C) 2025 Renesas Electronics Co + * Copyright (C) 2025 Niklas Söderlund + * + * Renesas R-Car Gen4 VIN pipeline + */ + +#pragma once + +#include +#include +#include +#include + +#include + +#include +#include + +#include +#include +#include + +#include "isp.h" + +namespace libcamera { + +class FrameBuffer; +class Request; + +class RCar4Frames +{ +public: + struct Info { + unsigned int frame; + Request *request; + + FrameBuffer *inputBuffer; + FrameBuffer *paramBuffer; + FrameBuffer *statBuffer; + FrameBuffer *outputBuffer; + + ControlList effectiveSensorControls; + + bool rawDequeued; + bool paramDequeued; + bool metadataProcessed; + bool outputDequeued; + }; + + int start(class RCarISPDevice *isp, class ipa::rppx1::IPAProxyRppX1 *ipa); + void stop(class RCarISPDevice *isp, class ipa::rppx1::IPAProxyRppX1 *ipa); + + Info *create(Request *request); + void remove(Info *info); + bool tryComplete(Info *info); + + Info *find(unsigned int frame); + Info *find(FrameBuffer *buffer); + + Signal<> bufferAvailable; + + Stream rawStream_; + Stream outputStream_; +private: + std::map frameInfo_; + + /* Buffers for internal use, if none is provided in request. */ + std::vector> inputBuffers_; + std::vector> paramBuffers_; + std::vector> statBuffers_; + std::vector> outputBuffers_; + + /* Queues of available internal buffers. */ + std::queue availableInputBuffers_; + std::queue availableParamBuffers_; + std::queue availableStatBuffers_; + std::queue availableOutputBuffers_; + + /* Buffers mapped and shared with IPA. */ + std::vector ipaBuffers_; +}; + +} /* namespace libcamera */ diff --git a/src/libcamera/pipeline/rcar-gen4/isp.cpp b/src/libcamera/pipeline/rcar-gen4/isp.cpp new file mode 100644 index 000000000000..60c27db5aac5 --- /dev/null +++ b/src/libcamera/pipeline/rcar-gen4/isp.cpp @@ -0,0 +1,228 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2026 Ideas on Board Oy + * Copyright (C) 2025 Renesas Electronics Co + * Copyright (C) 2025 Niklas Söderlund + * + * Renesas R-Car Gen4 ISP pipeline + */ + +#include "isp.h" + +#include +#include +#include + +#include + +#include +#include + +#include +#include + +#include "libcamera/internal/media_device.h" +#include "libcamera/internal/v4l2_subdevice.h" + +namespace libcamera { + +LOG_DECLARE_CATEGORY(RCar4) + +int RCarISPDevice::init(const MediaDevice *media, const std::string &pipeId) +{ + const MediaEntity *entity; + const MediaPad *pad, *next; + int ret; + + /* Locate IPSCORE, e.g. rcar_isp fed00000.isp core */ + std::unique_ptr core = + V4L2Subdevice::fromEntityName(media, pipeId + " core"); + if (!core) { + LOG(RCar4, Error) << "Failed to find ISPCORE " << pipeId; + return -EINVAL; + } + + entity = core->entity(); + + /* Use the media links to find all video devices. */ + pad = entity->getPadByIndex(0); + next = pad->links()[0]->source(); + input_ = V4L2VideoDevice::fromEntityName(media, next->entity()->name()); + if (!input_) { + LOG(RCar4, Error) << "Failed to find ISP input entity"; + return -EINVAL; + } + + pad = entity->getPadByIndex(1); + next = pad->links()[0]->source(); + param_ = V4L2VideoDevice::fromEntityName(media, next->entity()->name()); + if (!param_) { + LOG(RCar4, Error) << "Failed to find ISP param entity"; + return -EINVAL; + } + + pad = entity->getPadByIndex(2); + next = pad->links()[0]->sink(); + stat_ = V4L2VideoDevice::fromEntityName(media, next->entity()->name()); + if (!stat_) { + LOG(RCar4, Error) << "Failed to find ISP stat entity"; + return -EINVAL; + } + + pad = entity->getPadByIndex(3); + next = pad->links()[0]->sink(); + output_ = V4L2VideoDevice::fromEntityName(media, next->entity()->name()); + if (!output_) { + LOG(RCar4, Error) << "Failed to find ISP output entity"; + return -EINVAL; + } + + /* Open all devices. */ + ret = input_->open(); + if (ret) + return ret; + + ret = param_->open(); + if (ret) + return ret; + + ret = stat_->open(); + if (ret) + return ret; + + ret = output_->open(); + if (ret) + return ret; + + return 0; +} + +std::vector RCarISPDevice::formats() const +{ + std::vector formats; + for (const auto &[format, sizes] : output_->formats()) + formats.push_back(format.toPixelFormat()); + + return formats; +} + +StreamConfiguration +RCarISPDevice::generateConfiguration(PixelFormat format, Size size) const +{ + StreamConfiguration cfg; + + bool found = false; + for (const auto &pixelFormat : formats()) { + if (pixelFormat == format) + found = true; + } + + cfg.size = size; + cfg.bufferCount = kBufferCount; + cfg.pixelFormat = found ? format : formats::XRGB8888; + + /* Get stride and frame size from device. */ + V4L2DeviceFormat fmt; + fmt.fourcc = output_->toV4L2PixelFormat(cfg.pixelFormat); + fmt.size = cfg.size; + + if (output_->tryFormat(&fmt)) + return {}; + + cfg.stride = fmt.planes[0].bpl; + cfg.frameSize = fmt.planes[0].size; + + return cfg; +} + +int RCarISPDevice::configure(V4L2DeviceFormat *inputFormat, + const PixelFormat &outputPixelFormat) +{ + int ret; + + /* Configure the RAW input. */ + ret = input_->setFormat(inputFormat); + if (ret) + return ret; + + LOG(RCar4, Debug) << "ISP input format = " << *inputFormat; + + /* Configure the image output. */ + V4L2DeviceFormat outputFormat; + outputFormat.fourcc = output_->toV4L2PixelFormat(outputPixelFormat); + outputFormat.size = inputFormat->size; + ret = output_->setFormat(&outputFormat); + if (ret) + return ret; + + LOG(RCar4, Debug) << "ISP output format = " << outputFormat; + + /* Configure paramaters. */ + V4L2DeviceFormat paramFormat; + paramFormat.fourcc = V4L2PixelFormat(V4L2_META_FMT_RPP_X1_PARAMS); + ret = param_->setFormat(¶mFormat); + if (ret) + return ret; + + /* Configure statistics. */ + V4L2DeviceFormat statFormat; + statFormat.fourcc = V4L2PixelFormat(V4L2_META_FMT_RPP_X1_STATS); + ret = stat_->setFormat(&statFormat); + if (ret) + return ret; + + return 0; +} + +int RCarISPDevice::start() +{ + int ret; + + ret = input_->importBuffers(kBufferCount); + if (ret) { + LOG(RCar4, Error) << "Failed to import ISP input buffers"; + return ret; + } + + ret = output_->importBuffers(kBufferCount); + if (ret) { + LOG(RCar4, Error) << "Failed to import ISP output buffers"; + return ret; + } + + ret = output_->streamOn(); + if (ret) { + LOG(RCar4, Error) << "Failed to start ISP output"; + return ret; + } + + ret = param_->streamOn(); + if (ret) { + LOG(RCar4, Error) << "Failed to start ISP param"; + return ret; + } + + ret = stat_->streamOn(); + if (ret) { + LOG(RCar4, Error) << "Failed to start ISP stat"; + return ret; + } + + ret = input_->streamOn(); + if (ret) { + LOG(RCar4, Error) << "Failed to start ISP input"; + return ret; + } + + return 0; +} + +void RCarISPDevice::stop() +{ + output_->streamOff(); + param_->streamOff(); + stat_->streamOff(); + input_->streamOff(); +} + +} /* namespace libcamera */ diff --git a/src/libcamera/pipeline/rcar-gen4/isp.h b/src/libcamera/pipeline/rcar-gen4/isp.h new file mode 100644 index 000000000000..cb46a3c6b3fa --- /dev/null +++ b/src/libcamera/pipeline/rcar-gen4/isp.h @@ -0,0 +1,45 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2026 Ideas on Board Oy + * Copyright (C) 2025 Renesas Electronics Co + * Copyright (C) 2025 Niklas Söderlund + * + * Renesas R-Car Gen4 ISP pipeline + */ + +#pragma once + +#include +#include + +#include "libcamera/internal/v4l2_videodevice.h" + +namespace libcamera { + +class MediaDevice; +class Size; +struct StreamConfiguration; + +class RCarISPDevice +{ +public: + static constexpr unsigned int kBufferCount = 4; + + std::vector formats() const; + + int init(const MediaDevice *media, const std::string &pipeId); + + StreamConfiguration generateConfiguration(PixelFormat format, Size size) const; + + int configure(V4L2DeviceFormat *inputFormat, const PixelFormat &outputPixelFormat); + + int start(); + void stop(); + + std::unique_ptr input_; + std::unique_ptr param_; + std::unique_ptr stat_; + std::unique_ptr output_; +}; + +} /* namespace libcamera */ diff --git a/src/libcamera/pipeline/rcar-gen4/meson.build b/src/libcamera/pipeline/rcar-gen4/meson.build new file mode 100644 index 000000000000..431eb54e2803 --- /dev/null +++ b/src/libcamera/pipeline/rcar-gen4/meson.build @@ -0,0 +1,8 @@ +# SPDX-License-Identifier: CC0-1.0 + +libcamera_internal_sources += files([ + 'frames.cpp', + 'isp.cpp', + 'rcar-gen4.cpp', + 'vin.cpp', +]) diff --git a/src/libcamera/pipeline/rcar-gen4/rcar-gen4.cpp b/src/libcamera/pipeline/rcar-gen4/rcar-gen4.cpp new file mode 100644 index 000000000000..8e231b8115c2 --- /dev/null +++ b/src/libcamera/pipeline/rcar-gen4/rcar-gen4.cpp @@ -0,0 +1,810 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2026 Ideas on Board Oy + * Copyright (C) 2025 Renesas Electronics Co + * Copyright (C) 2025 Niklas Söderlund + * + * Renesas R-Car Gen4 ISP pipeline + */ + +#include +#include +#include +#include + +#include +#include + +#include +#include +#include + +#include "libcamera/internal/camera.h" +#include "libcamera/internal/camera_sensor.h" +#include "libcamera/internal/delayed_controls.h" +#include "libcamera/internal/device_enumerator.h" +#include "libcamera/internal/framebuffer.h" +#include "libcamera/internal/ipa_manager.h" +#include "libcamera/internal/mapped_framebuffer.h" +#include "libcamera/internal/media_device.h" +#include "libcamera/internal/pipeline_handler.h" +#include "libcamera/internal/request.h" +#include "libcamera/internal/v4l2_device.h" +#include "libcamera/internal/v4l2_subdevice.h" +#include "libcamera/internal/v4l2_videodevice.h" + +#include "frames.h" +#include "isp.h" +#include "vin.h" + +namespace libcamera { + +namespace { + +const std::map formatToMediaBus = { + { formats::SBGGR10, MEDIA_BUS_FMT_SBGGR10_1X10 }, + { formats::SGBRG10, MEDIA_BUS_FMT_SGBRG10_1X10 }, + { formats::SGRBG10, MEDIA_BUS_FMT_SGRBG10_1X10 }, + { formats::SRGGB10, MEDIA_BUS_FMT_SRGGB10_1X10 }, +}; + +/* Max supported resolution of VIN and ISP. */ +static constexpr Size MaxResolution = { 4096, 4096 }; +static constexpr unsigned int nBuffers = 4; + +} /* namespace */ + +LOG_DEFINE_CATEGORY(RCar4) + +/* ----------------------------------------------------------------------------- + * Camera Data + */ + +class RCar4CameraData : public Camera::Private +{ +public: + RCar4CameraData(PipelineHandler *pipe) + : Camera::Private(pipe) + { + } + + RCarVINDevice vin_; + RCarISPDevice isp_; + std::unique_ptr ipa_; + + RCar4Frames frames_; + std::unique_ptr delayedCtrls_; + ControlInfoMap ipaControls_; + + void queuePendingRequests(); + void cancelPendingRequests(); + + /* Requests for which no buffer has been queued to the VIN device yet. */ + std::queue pendingRequests_; + + /* Slots for processing ready buffers. */ + void vinBufferReady(FrameBuffer *buffer); + void inputBufferReady(FrameBuffer *buffer); + void paramBufferReady(FrameBuffer *buffer); + void statBufferReady(FrameBuffer *buffer); + void outputBufferReady(FrameBuffer *buffer); + + /* Slots for processing IPA interactions. */ + void paramsComputed(unsigned int frame, unsigned int bytesused); + void setSensorControls(unsigned int frame, + const ControlList &sensorControls); + void metadataReady(unsigned int frame, const ControlList &metadata); +}; + +void RCar4CameraData::queuePendingRequests() +{ + while (!pendingRequests_.empty()) { + Request *request = pendingRequests_.front(); + + RCar4Frames::Info *info = frames_.create(request); + if (!info) + break; + + if (vin_.queueBuffer(info->inputBuffer)) { + /* Remove if raw buffer failed, should not happen. */ + frames_.remove(info); + break; + } + + ipa_->queueRequest(info->frame, request->controls()); + + pendingRequests_.pop(); + } +} + +void RCar4CameraData::cancelPendingRequests() +{ + while (!pendingRequests_.empty()) { + Request *request = pendingRequests_.front(); + + for (auto it : request->buffers()) { + FrameBuffer *buffer = it.second; + buffer->_d()->cancel(); + pipe()->completeBuffer(request, buffer); + } + + pipe()->completeRequest(request); + pendingRequests_.pop(); + } +} + +void RCar4CameraData::vinBufferReady(FrameBuffer *buffer) +{ + RCar4Frames::Info *info = frames_.find(buffer); + if (!info) + return; + + Request *request = info->request; + + /* If the buffer is cancelled force a complete of the whole request. */ + if (buffer->metadata().status == FrameMetadata::FrameCancelled) { + for (auto it : request->buffers()) { + FrameBuffer *b = it.second; + b->_d()->cancel(); + pipe()->completeBuffer(request, b); + } + + frames_.remove(info); + pipe()->completeRequest(request); + return; + } + + /* Record the sensor's timestamp in the request metadata. */ + request->_d()->metadata().set(controls::SensorTimestamp, + buffer->metadata().timestamp); + + ipa_->computeParams(info->frame, info->paramBuffer->cookie()); +} + +void RCar4CameraData::inputBufferReady(FrameBuffer *buffer) +{ + RCar4Frames::Info *info = frames_.find(buffer); + if (!info) + return; + + Request *request = info->request; + + if (request->findBuffer(&frames_.rawStream_)) + pipe()->completeBuffer(request, buffer); + + info->rawDequeued = true; + + if (frames_.tryComplete(info)) + pipe()->completeRequest(request); +} + +void RCar4CameraData::paramBufferReady(FrameBuffer *buffer) +{ + RCar4Frames::Info *info = frames_.find(buffer); + if (!info) + return; + + Request *request = info->request; + + info->paramDequeued = true; + + if (frames_.tryComplete(info)) + pipe()->completeRequest(request); +} + +void RCar4CameraData::statBufferReady(FrameBuffer *buffer) +{ + RCar4Frames::Info *info = frames_.find(buffer); + if (!info) + return; + + Request *request = info->request; + + if (buffer->metadata().status == FrameMetadata::FrameCancelled) { + info->metadataProcessed = true; + + if (frames_.tryComplete(info)) + pipe()->completeRequest(request); + + return; + } + + ipa_->processStats(info->frame, info->statBuffer->cookie(), + delayedCtrls_->get(buffer->metadata().sequence)); +} + +void RCar4CameraData::outputBufferReady(FrameBuffer *buffer) +{ + RCar4Frames::Info *info = frames_.find(buffer); + if (!info) + return; + + Request *request = info->request; + + if (request->findBuffer(&frames_.outputStream_)) + pipe()->completeBuffer(request, buffer); + + request->_d()->metadata().set(controls::draft::PipelineDepth, 3); + + info->outputDequeued = true; + + if (frames_.tryComplete(info)) + pipe()->completeRequest(request); +} + +void RCar4CameraData::paramsComputed(unsigned int frame, unsigned int bytesused) +{ + RCar4Frames::Info *info = frames_.find(frame); + if (!info) + return; + + info->paramBuffer->_d()->metadata().planes()[0].bytesused = bytesused; + + isp_.output_->queueBuffer(info->outputBuffer); + isp_.param_->queueBuffer(info->paramBuffer); + isp_.stat_->queueBuffer(info->statBuffer); + isp_.input_->queueBuffer(info->inputBuffer); +} + +void RCar4CameraData::setSensorControls([[maybe_unused]] unsigned int frame, + const ControlList &sensorControls) +{ + delayedCtrls_->push(sensorControls); +} + +void RCar4CameraData::metadataReady(unsigned int frame, const ControlList &metadata) +{ + RCar4Frames::Info *info = frames_.find(frame); + if (!info) + return; + + Request *request = info->request; + + info->request->_d()->metadata().merge(metadata); + info->metadataProcessed = true; + + if (frames_.tryComplete(info)) + pipe()->completeRequest(request); +} + +/* ----------------------------------------------------------------------------- + * Camera Configuration + */ + +class RCar4CameraConfiguration : public CameraConfiguration +{ +public: + RCar4CameraConfiguration(Camera *camera, RCar4CameraData *data); + + Status validate() override; + + const V4L2SubdeviceFormat &sensorFormat() { return sensorFormat_; } + const Transform &combinedTransform() { return combinedTransform_; } + const PixelFormat &ispOutputFormat() { return ispOutputFormat_; } +private: + std::shared_ptr camera_; + RCar4CameraData *data_; + + V4L2SubdeviceFormat sensorFormat_; + Transform combinedTransform_; + PixelFormat ispOutputFormat_; +}; + +RCar4CameraConfiguration::RCar4CameraConfiguration(Camera *camera, + RCar4CameraData *data) + : CameraConfiguration(), camera_(camera->shared_from_this()), data_(data) +{ +} + +CameraConfiguration::Status RCar4CameraConfiguration::validate() +{ + Status status; + + if (config_.empty()) + return Invalid; + + status = validateColorSpaces(ColorSpaceFlag::StreamsShareColorSpace); + + /* Cap the number of entries to the available streams. */ + if (config_.size() > 2) { + config_.resize(2); + status = Adjusted; + } + + Orientation requestedOrientation = orientation; + combinedTransform_ = data_->vin_.sensor()->computeTransform(&orientation); + if (orientation != requestedOrientation) + status = Adjusted; + + /* Figure out the VIN configuration based on the first stream size. */ + StreamConfiguration vinCfg = data_->vin_.generateConfiguration(config_.at(0).size); + + /* Default ISP output format. */ + ispOutputFormat_ = formats::XRGB8888; + + /* + * Validate there are at max two streams, one output and one RAW. The + * size of two streams must match each other and the sensor output as we + * have no scaler. + */ + unsigned int outputStreams = 0; + unsigned int rawStreams = 0; + for (unsigned int i = 0; i < config_.size(); i++) { + StreamConfiguration &cfg = config_.at(i); + StreamConfiguration newCfg = {}; + const StreamConfiguration originalCfg = cfg; + const PixelFormatInfo &info = PixelFormatInfo::info(cfg.pixelFormat); + + LOG(RCar4, Debug) << "Validating stream: " << cfg.toString(); + + if (info.colourEncoding == PixelFormatInfo::ColourEncodingRAW) { + if (rawStreams++) { + LOG(RCar4, Error) + << "Camera configuration support only one RAW stream"; + return Invalid; + } + + newCfg = vinCfg; + + cfg.setStream(&data_->frames_.rawStream_); + LOG(RCar4, Debug) << "Assigned " << newCfg.toString() + << " to the raw stream"; + } else { + if (outputStreams++) { + LOG(RCar4, Error) + << "Camera configuration support only one output stream"; + return Invalid; + } + + newCfg = data_->isp_.generateConfiguration(cfg.pixelFormat, vinCfg.size); + ispOutputFormat_ = newCfg.pixelFormat; + + cfg.setStream(&data_->frames_.outputStream_); + LOG(RCar4, Debug) << "Assigned " << newCfg.toString() + << " to the output stream"; + } + + cfg.size = newCfg.size; + cfg.bufferCount = newCfg.bufferCount; + cfg.pixelFormat = newCfg.pixelFormat; + cfg.stride = newCfg.stride; + cfg.frameSize = newCfg.frameSize; + + if (!cfg.pixelFormat.isValid()) { + LOG(RCar4, Error) + << "Stream " << i << " can not generate cfg"; + return Invalid; + } + + if (cfg.pixelFormat != originalCfg.pixelFormat || + cfg.size != originalCfg.size) { + LOG(RCar4, Debug) + << "Stream " << i << " configuration adjusted to " + << cfg.toString(); + status = Adjusted; + } + } + + /* Select the sensor format. */ + std::vector mbusCodes = { formatToMediaBus.at(vinCfg.pixelFormat) }; + sensorFormat_ = + data_->vin_.sensor()->getFormat(mbusCodes, vinCfg.size, vinCfg.size); + + return status; +} + +/* ----------------------------------------------------------------------------- + * Pipeline Handler + */ + +class PipelineHandlerRCar4 : public PipelineHandler +{ +public: + PipelineHandlerRCar4(CameraManager *manager); + + std::unique_ptr generateConfiguration(Camera *camera, + Span roles) override; + int configure(Camera *camera, CameraConfiguration *config) override; + + int exportFrameBuffers(Camera *camera, Stream *stream, + std::vector> *buffers) override; + + int start(Camera *camera, const ControlList *controls) override; + void stopDevice(Camera *camera) override; + + int queueRequestDevice(Camera *camera, Request *request) override; + + bool match(DeviceEnumerator *enumerator) override; + + int updateControls(RCar4CameraData *data); + +private: + RCar4CameraData *cameraData(Camera *camera) + { + return static_cast(camera->_d()); + } + + int createCamera(MediaDevice *mdev, const std::string &pipeId); + + StreamConfiguration generateStreamConfiguration(RCar4CameraData *data, + StreamRole role); +}; + +PipelineHandlerRCar4::PipelineHandlerRCar4(CameraManager *manager) + : PipelineHandler(manager) +{ +} + +StreamConfiguration +PipelineHandlerRCar4::generateStreamConfiguration(RCar4CameraData *data, + StreamRole role) +{ + const std::vector &mbusCodes = data->vin_.sensor()->mbusCodes(); + + /* Create the list of supported RAW stream formats. */ + std::map> rawFormats; + unsigned int rawBitsPerPixel = 0; + PixelFormat rawFormat; + Size rawSize = { 0, 0 }; + std::vector rawSizes; + + for (const auto &format : data->vin_.formats()) { + const PixelFormatInfo &info = PixelFormatInfo::info(format); + + /* Populate stream formats for RAW configurations. */ + uint32_t mbusCode = formatToMediaBus.at(format); + + /* Skip formats not supported by sensor. */ + if (std::find(mbusCodes.begin(), mbusCodes.end(), mbusCode) == mbusCodes.end()) + continue; + + /* Add all the RAW sizes the sensor can produce for this code. */ + for (const auto &rawSizeByCode : data->vin_.sensor()->sizes(mbusCode)) { + if (rawSizeByCode.width > MaxResolution.width || + rawSizeByCode.height > MaxResolution.height) + continue; + + rawSizes.push_back({ rawSizeByCode, rawSizeByCode }); + + rawFormats[format].push_back({ rawSizeByCode, rawSizeByCode }); + + /* Cache for later default format. */ + if (info.bitsPerPixel >= rawBitsPerPixel) { + rawBitsPerPixel = info.bitsPerPixel; + rawFormat = format; + + if (rawSizeByCode > rawSize) + rawSize = rawSizeByCode; + } + } + } + + /* If generating for RAW role we are done. */ + if (role == StreamRole::Raw) { + StreamFormats rawStreamFormats(rawFormats); + StreamConfiguration rawCfg(rawStreamFormats); + rawCfg.pixelFormat = rawFormat; + rawCfg.size = rawSize; + rawCfg.bufferCount = nBuffers; + + return rawCfg; + } + + /* Create the list of supported other stream formats. */ + std::map> outputFormats; + std::vector outputSizes(rawSizes.begin(), rawSizes.end()); + + for (const auto &format : data->isp_.formats()) { + const PixelFormatInfo &info = PixelFormatInfo::info(format); + + /* Skip RAW formats. */ + if (info.colourEncoding == PixelFormatInfo::ColourEncodingRAW) + continue; + + outputFormats[format] = { outputSizes }; + } + + StreamFormats outputStreamFormats(outputFormats); + StreamConfiguration outputCfg(outputStreamFormats); + outputCfg.pixelFormat = formats::XRGB8888; + outputCfg.size = rawSize; + + return outputCfg; +} + +std::unique_ptr +PipelineHandlerRCar4::generateConfiguration(Camera *camera, + Span roles) +{ + RCar4CameraData *data = cameraData(camera); + + std::unique_ptr config = + std::make_unique(camera, data); + + if (roles.empty()) + return config; + + for (const StreamRole role : roles) { + std::optional colorSpace; + + switch (role) { + case StreamRole::Raw: + colorSpace = ColorSpace::Raw; + break; + default: + colorSpace = ColorSpace::Rec709; + break; + } + + StreamConfiguration cfg = + generateStreamConfiguration(data, role); + if (!cfg.pixelFormat.isValid()) + return nullptr; + + cfg.colorSpace = colorSpace; + cfg.bufferCount = nBuffers; + config->addConfiguration(cfg); + } + + if (config->validate() == CameraConfiguration::Invalid) + return nullptr; + + return config; +} + +int PipelineHandlerRCar4::configure(Camera *camera, CameraConfiguration *c) +{ + RCar4CameraConfiguration *config = static_cast(c); + RCar4CameraData *data = cameraData(camera); + + V4L2DeviceFormat vinFormat; + int ret; + + /* Configure VIN and propagate format to ISP. */ + ret = data->vin_.configure(config->sensorFormat().size, + config->combinedTransform(), &vinFormat); + if (ret) + return ret; + + ret = data->isp_.configure(&vinFormat, config->ispOutputFormat()); + if (ret) + return ret; + + /* Inform IPA of stream configuration and sensor controls. */ + IPACameraSensorInfo sensorInfo; + ret = data->vin_.sensor()->sensorInfo(&sensorInfo); + if (ret) + return ret; + + ipa::rppx1::IPAConfigInfo ipaConfig{ + sensorInfo, data->vin_.sensor()->controls(), + }; + + std::map streamConfig; + streamConfig[0] = + IPAStream(PixelFormat(config->ispOutputFormat().fourcc()), + config->sensorFormat().size); + + ret = data->ipa_->configure(ipaConfig, streamConfig, &data->ipaControls_); + if (ret) { + LOG(RCar4, Error) << "failed configuring IPA (" << ret << ")"; + return ret; + } + + return updateControls(data); +} + +int PipelineHandlerRCar4::exportFrameBuffers(Camera *camera, Stream *stream, + std::vector> *buffers) +{ + RCar4CameraData *data = cameraData(camera); + unsigned int count = stream->configuration().bufferCount; + + if (stream == &data->frames_.outputStream_) + return data->isp_.output_->exportBuffers(count, buffers); + + if (stream == &data->frames_.rawStream_) + return data->isp_.input_->exportBuffers(count, buffers); + + return -EINVAL; +} + +int PipelineHandlerRCar4::start(Camera *camera, + [[maybe_unused]] const ControlList *controls) +{ + RCar4CameraData *data = cameraData(camera); + int ret; + + data->delayedCtrls_->reset(); + + ret = data->frames_.start(&data->isp_, data->ipa_.get()); + if (ret) + goto error; + + ret = data->ipa_->start(); + if (ret) + goto error; + + ret = data->vin_.start(); + if (ret) + goto error; + + ret = data->isp_.start(); + if (ret) + goto error; + + return 0; +error: + stop(camera); + + return ret; +} + +void PipelineHandlerRCar4::stopDevice(Camera *camera) +{ + RCar4CameraData *data = cameraData(camera); + + data->cancelPendingRequests(); + + data->isp_.stop(); + data->vin_.stop(); + data->ipa_->stop(); + + data->frames_.stop(&data->isp_, data->ipa_.get()); +} + +int PipelineHandlerRCar4::queueRequestDevice(Camera *camera, Request *request) +{ + RCar4CameraData *data = cameraData(camera); + + data->pendingRequests_.push(request); + data->queuePendingRequests(); + + return 0; +} + +int PipelineHandlerRCar4::updateControls(RCar4CameraData *data) +{ + ControlInfoMap::Map controls; + + for (const auto &ipaControl : data->ipaControls_) + controls[ipaControl.first] = ipaControl.second; + + data->controlInfo_ = ControlInfoMap(std::move(controls), + controls::controls); + return 0; +} + +int PipelineHandlerRCar4::createCamera(MediaDevice *mdev, + const std::string &pipeId) +{ + std::unique_ptr data = std::make_unique(this); + IPACameraSensorInfo sensorInfo{}; + int ret; + + ret = data->vin_.init(mdev, pipeId); + if (ret) + return ret; + + ret = data->isp_.init(mdev, pipeId); + if (ret) + return ret; + + /* Load RPP-X1 IPA for use with RCar4. */ + data->ipa_ = createIPA("rppx1", 1, 1); + if (!data->ipa_) { + LOG(RCar4, Error) << "No IPA module found"; + return -ENOENT; + } + + /* The IPA tuning file is made from the sensor name. */ + std::string ipaTuningFile = + data->ipa_->configurationFile(data->vin_.sensor()->model() + ".yaml", "uncalibrated.yaml"); + + ret = data->vin_.sensor()->sensorInfo(&sensorInfo); + if (ret) { + LOG(RCar4, Error) << "Camera sensor information not available"; + return ret; + } + + ret = data->ipa_->init({ ipaTuningFile, data->vin_.sensor()->model() }, + sensorInfo, data->vin_.sensor()->controls(), &data->ipaControls_); + if (ret < 0) { + LOG(RCar4, Error) << "IPA initialization failure"; + return ret; + } + + updateControls(data.get()); + + /* + * Initialize the camera properties. + */ + data->properties_ = data->vin_.sensor()->properties(); + const CameraSensorProperties::SensorDelays &delays = data->vin_.sensor()->sensorDelays(); + std::unordered_map params = { + { V4L2_CID_ANALOGUE_GAIN, { delays.gainDelay, false } }, + { V4L2_CID_EXPOSURE, { delays.exposureDelay, false } }, + { V4L2_CID_VBLANK, { delays.vblankDelay, false } }, + }; + + data->delayedCtrls_ = + std::make_unique(data->vin_.sensor()->device(), + params); + + std::set streams{ + &data->frames_.rawStream_, + &data->frames_.outputStream_, + }; + + /* + * Connect signals to slots to drive the pipeline. + */ + + /* When internal buffers become available try to queue more jobs. */ + data->frames_.bufferAvailable.connect(data.get(), + &RCar4CameraData::queuePendingRequests); + + /* Connect bufferReady for each video device to a handler. */ + data->vin_.bufferReady().connect(data.get(), + &RCar4CameraData::vinBufferReady); + data->isp_.input_->bufferReady.connect(data.get(), + &RCar4CameraData::inputBufferReady); + data->isp_.param_->bufferReady.connect(data.get(), + &RCar4CameraData::paramBufferReady); + data->isp_.stat_->bufferReady.connect(data.get(), + &RCar4CameraData::statBufferReady); + data->isp_.output_->bufferReady.connect(data.get(), + &RCar4CameraData::outputBufferReady); + + /* Connect IPA signals. */ + data->ipa_->setSensorControls.connect(data.get(), + &RCar4CameraData::setSensorControls); + data->ipa_->paramsComputed.connect(data.get(), + &RCar4CameraData::paramsComputed); + data->ipa_->metadataReady.connect(data.get(), + &RCar4CameraData::metadataReady); + + /* Apply controls at start at exposure. */ + data->vin_.frameStart().connect(data->delayedCtrls_.get(), + &DelayedControls::applyControls); + + /* + * Register the camera. + */ + const std::string &id = data->vin_.sensor()->id(); + std::shared_ptr camera = Camera::create(std::move(data), id, streams); + + registerCamera(std::move(camera)); + + return 0; +} + +bool PipelineHandlerRCar4::match(DeviceEnumerator *enumerator) +{ + DeviceMatch dm("rcar_vin"); + + auto media = acquireMediaDevice(enumerator, dm); + if (!media) + return false; + + bool registered = false; + for (const MediaEntity *entity : media->entities()) { + if (entity->name().substr(0, 8) == "rcar_isp" && + entity->name().rfind("core") != std::string::npos) { + /* + * Isolate the unit address that identifies one ISP + * instance. pipeId will look like + * 'rcar_isp fed00000.isp'. + */ + std::string pipeId = entity->name().substr(0, 21); + if (!createCamera(media.get(), pipeId)) + registered = true; + } + } + + return registered; +} + +REGISTER_PIPELINE_HANDLER(PipelineHandlerRCar4, "rcar-gen4") + +} /* namespace libcamera */ diff --git a/src/libcamera/pipeline/rcar-gen4/vin.cpp b/src/libcamera/pipeline/rcar-gen4/vin.cpp new file mode 100644 index 000000000000..76b94a434d84 --- /dev/null +++ b/src/libcamera/pipeline/rcar-gen4/vin.cpp @@ -0,0 +1,391 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright 2025 Renesas Electronics Co + * Copyright 2025 Niklas Söderlund + * + * Renesas R-Car Gen4 VIN pipeline + */ + +#include "vin.h" + +#include +#include + +#include + +#include +#include +#include +#include + +#include "libcamera/internal/camera_sensor.h" +#include "libcamera/internal/framebuffer.h" +#include "libcamera/internal/media_device.h" +#include "libcamera/internal/v4l2_subdevice.h" + +namespace libcamera { + +LOG_DECLARE_CATEGORY(RCar4) + +namespace { + +const std::map mbusCodesToPixelFormat = { + { MEDIA_BUS_FMT_SBGGR10_1X10, formats::SBGGR10 }, + { MEDIA_BUS_FMT_SGBRG10_1X10, formats::SGBRG10 }, + { MEDIA_BUS_FMT_SGRBG10_1X10, formats::SGRBG10 }, + { MEDIA_BUS_FMT_SRGGB10_1X10, formats::SRGGB10 }, +}; + +} /* namespace */ + +/** + * \brief Retrieve the list of supported PixelFormats + * + * Retrieve the list of supported pixel formats by matching the sensor produced + * media bus codes with the formats supported by the VIN unit. + * + * \return The list of supported PixelFormat + */ +std::vector RCarVINDevice::formats() const +{ + if (!sensor_) + return {}; + + std::vector formats; + for (unsigned int code : sensor_->mbusCodes()) { + auto it = mbusCodesToPixelFormat.find(code); + if (it != mbusCodesToPixelFormat.end()) + formats.push_back(it->second); + } + + return formats; +} + +/** + * \brief Retrieve the list of supported size ranges + * \param[in] format The pixel format + * + * Retrieve the list of supported sizes for a particular \a format by matching + * the sensor produced media bus codes formats supported by the VIN unit. + * + * \return A list of supported sizes for the \a format or an empty list + * otherwise + */ +std::vector RCarVINDevice::sizes(const PixelFormat &format) const +{ + int mbusCode = -1; + + if (!sensor_) + return {}; + + std::vector sizes; + for (const auto &iter : mbusCodesToPixelFormat) { + if (iter.second != format) + continue; + + mbusCode = iter.first; + break; + } + + if (mbusCode == -1) + return {}; + + for (const Size &sz : sensor_->sizes(mbusCode)) + sizes.emplace_back(sz); + + return sizes; +} + +int RCarVINDevice::init(const MediaDevice *media, const std::string &pipeId) +{ + const MediaEntity *entity; + const MediaPad *pad, *next; + int ret; + + /* Locate IPS Channel Selector, e.g. rcar_isp fed00000.isp */ + csisp_ = V4L2Subdevice::fromEntityName(media, pipeId); + if (!csisp_) { + LOG(RCar4, Error) << "Failed to find Channel Selector " << pipeId; + return -EINVAL; + } + + /* Use the Channel Selector links to find CSI-2 Rx and Sensor. */ + entity = csisp_->entity(); + pad = entity->getPadByIndex(0); + next = pad->links()[0]->source(); + csi2_ = V4L2Subdevice::fromEntityName(media, next->entity()->name()); + if (!csi2_) { + LOG(RCar4, Error) << "Failed to find CSI-2 Rx entity"; + return -EINVAL; + } + + entity = csi2_->entity(); + pad = entity->getPadByIndex(0); + next = pad->links()[0]->source(); + sensor_ = CameraSensorFactoryBase::create(next->entity()); + if (!sensor_) { + LOG(RCar4, Error) << "Failed to find sensor entity"; + return -EINVAL; + } + + /* + * Make sure the sensor produces at least one format compatible with + * the VIN requirements. + */ + std::vector vinCodes = utils::map_keys(mbusCodesToPixelFormat); + const std::vector &sensorCodes = sensor_->mbusCodes(); + if (!utils::set_overlap(sensorCodes.begin(), sensorCodes.end(), + vinCodes.begin(), vinCodes.end())) { + LOG(RCar4, Error) + << "Sensor " << sensor_->entity()->name() + << " has not format compatible with the VIN"; + return -EINVAL; + } + + /* Use the Channel Selector links to find VIN. */ + entity = csisp_->entity(); + pad = entity->getPadByIndex(1); + next = pad->links()[0]->sink(); + output_ = V4L2VideoDevice::fromEntityName(media, next->entity()->name()); + if (!output_) { + LOG(RCar4, Error) << "Failed to find VIN entity"; + return -EINVAL; + } + + /* Open all devices. */ + ret = csi2_->open(); + if (ret) + return ret; + + ret = csisp_->open(); + if (ret) + return ret; + + ret = output_->open(); + if (ret) + return ret; + + return 0; +} + +int RCarVINDevice::configure(const Size &size, const Transform &transform, + V4L2DeviceFormat *outputFormat) +{ + V4L2SubdeviceFormat sensorFormat; + int ret; + + /* Configure sensor */ + std::vector mbusCodes = utils::map_keys(mbusCodesToPixelFormat); + sensorFormat = getSensorFormat(mbusCodes, size); + ret = sensor_->setFormat(&sensorFormat, transform); + if (ret) + return ret; + + /* Configure CSI-2 */ + ret = csi2_->setFormat(0, &sensorFormat); + if (ret) + return ret; + + if (mbusCodesToPixelFormat.find(sensorFormat.code) == mbusCodesToPixelFormat.end()) + return -EINVAL; + + /* Configure Channel selector. */ + ret = csisp_->setFormat(0, &sensorFormat); + if (ret) + return ret; + + if (mbusCodesToPixelFormat.find(sensorFormat.code) == mbusCodesToPixelFormat.end()) + return -EINVAL; + + /* Configure VIN */ + const auto &itInfo = mbusCodesToPixelFormat.find(sensorFormat.code); + outputFormat->fourcc = output_->toV4L2PixelFormat(itInfo->second); + outputFormat->size = sensorFormat.size; + outputFormat->planesCount = 1; + + ret = output_->setFormat(outputFormat); + if (ret) + return ret; + + LOG(RCar4, Debug) << "VIN output format " << *outputFormat; + + return 0; +} + +StreamConfiguration RCarVINDevice::generateConfiguration(Size size) const +{ + StreamConfiguration cfg; + + /* If no desired size use the sensor resolution. */ + if (size.isNull()) + size = sensor_->resolution(); + + /* Query the sensor static information for closest match. */ + std::vector mbusCodes = utils::map_keys(mbusCodesToPixelFormat); + V4L2SubdeviceFormat sensorFormat = getSensorFormat(mbusCodes, size); + if (!sensorFormat.code) { + LOG(RCar4, Error) << "Sensor does not support mbus code"; + return {}; + } + + cfg.size = sensorFormat.size; + cfg.pixelFormat = mbusCodesToPixelFormat.at(sensorFormat.code); + cfg.bufferCount = kBufferCount; + + /* Get stride and frame size from device. */ + V4L2DeviceFormat fmt; + fmt.fourcc = output_->toV4L2PixelFormat(cfg.pixelFormat); + fmt.size = cfg.size; + + int ret = output_->tryFormat(&fmt); + if (ret) + return {}; + + cfg.stride = fmt.planes[0].bpl; + cfg.frameSize = fmt.planes[0].size; + + return cfg; +} + +/** + * \brief Retrieve the best sensor format for a desired output + * \param[in] mbusCodes The list of acceptable media bus codes + * \param[in] size The desired size + * + * Media bus codes are selected from \a mbusCodes, which lists all acceptable + * codes in decreasing order of preference. Media bus codes supported by the + * sensor but not listed in \a mbusCodes are ignored. If none of the desired + * codes is supported, it returns an error. + * + * \a size indicates the desired size at the output of the sensor. This method + * selects the best media bus code and size supported by the sensor according + * to the following criteria. + * + * - The desired \a size shall fit in the sensor output size to avoid the need + * to up-scale. + * - The aspect ratio of sensor output size shall be as close as possible to + * the sensor's native resolution field of view. + * - The sensor output size shall be as small as possible to lower the required + * bandwidth. + * - The desired \a size shall be supported by one of the media bus code listed + * in \a mbusCodes. + * + * When multiple media bus codes can produce the same size, the code at the + * lowest position in \a mbusCodes is selected. + * + * The returned sensor output format is guaranteed to be acceptable by the + * setFormat() method without any modification. + * + * \return The best sensor output format matching the desired media bus codes + * and size on success, or an empty format otherwise. + */ +V4L2SubdeviceFormat RCarVINDevice::getSensorFormat(const std::vector &mbusCodes, + const Size &size) const +{ + unsigned int desiredArea = size.width * size.height; + unsigned int bestArea = std::numeric_limits::max(); + const Size &resolution = sensor_->resolution(); + float desiredRatio = static_cast(resolution.width) / + resolution.height; + float bestRatio = std::numeric_limits::max(); + Size bestSize; + uint32_t bestCode = 0; + + for (unsigned int code : mbusCodes) { + const auto sizes = sensor_->sizes(code); + if (!sizes.size()) + continue; + + for (const Size &sz : sizes) { + /* No need to check ratios if we have an exact match. */ + if (sz == size) { + bestRatio = 0; + bestArea = 0; + bestSize = sz; + bestCode = code; + break; + } + + if (sz.width < size.width || sz.height < size.height) + continue; + + float ratio = static_cast(sz.width) / sz.height; + /* + * Ratios can differ by small mantissa difference which + * can affect the selection of the sensor output size + * wildly. We are interested in selection of the closest + * size with respect to the desired output size, hence + * comparing it with a single precision digit is enough. + */ + ratio = static_cast(ratio * 10) / 10.0; + float ratioDiff = std::abs(ratio - desiredRatio); + unsigned int area = sz.width * sz.height; + unsigned int areaDiff = area - desiredArea; + + if (ratioDiff > bestRatio) + continue; + + if (ratioDiff < bestRatio || areaDiff < bestArea) { + bestRatio = ratioDiff; + bestArea = areaDiff; + bestSize = sz; + bestCode = code; + } + } + } + + if (bestSize.isNull()) { + LOG(RCar4, Debug) << "No supported format or size found"; + return {}; + } + + V4L2SubdeviceFormat format{}; + format.code = bestCode; + format.size = bestSize; + + return format; +} + +int RCarVINDevice::start() +{ + int ret; + + ret = output_->importBuffers(kBufferCount); + if (ret) { + LOG(RCar4, Error) << "Failed to import VIN buffers"; + return ret; + } + + ret = output_->streamOn(); + if (ret) { + LOG(RCar4, Error) << "Failed to start VIN"; + stop(); + return ret; + } + + ret = output_->setFrameStartEnabled(true); + if (ret) { + LOG(RCar4, Error) << "Failed to enable Frame Start"; + stop(); + return ret; + } + + return 0; +} + +void RCarVINDevice::stop() +{ + output_->setFrameStartEnabled(false); + + output_->streamOff(); + + if (output_->releaseBuffers()) + LOG(RCar4, Error) << "Failed to release VIN buffers"; +} + +int RCarVINDevice::queueBuffer(FrameBuffer *buffer) +{ + return output_->queueBuffer(buffer); +} + +} /* namespace libcamera */ diff --git a/src/libcamera/pipeline/rcar-gen4/vin.h b/src/libcamera/pipeline/rcar-gen4/vin.h new file mode 100644 index 000000000000..ceff9e05076e --- /dev/null +++ b/src/libcamera/pipeline/rcar-gen4/vin.h @@ -0,0 +1,67 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2026 Ideas on Board Oy + * Copyright (C) 2025 Renesas Electronics Co + * Copyright (C) 2025 Niklas Söderlund + * + * Renesas R-Car Gen4 VIN pipeline + */ + +#pragma once + +#include +#include +#include + +#include + +#include "libcamera/internal/v4l2_subdevice.h" +#include "libcamera/internal/v4l2_videodevice.h" + +namespace libcamera { + +class CameraSensor; +class FrameBuffer; +class MediaDevice; +class PixelFormat; +class Request; +class Size; +class SizeRange; +struct StreamConfiguration; +enum class Transform; + +class RCarVINDevice +{ +public: + static constexpr unsigned int kBufferCount = 4; + + std::vector formats() const; + std::vector sizes(const PixelFormat &format) const; + + int init(const MediaDevice *media, const std::string &pipeId); + int configure(const Size &size, const Transform &transform, + V4L2DeviceFormat *outputFormat); + + StreamConfiguration generateConfiguration(Size size) const; + + int start(); + void stop(); + + CameraSensor *sensor() { return sensor_.get(); } + const CameraSensor *sensor() const { return sensor_.get(); } + + int queueBuffer(FrameBuffer *buffer); + + Signal &bufferReady() { return output_->bufferReady; } + Signal &frameStart() { return output_->frameStart; } +private: + V4L2SubdeviceFormat getSensorFormat(const std::vector &mbusCodes, + const Size &size) const; + + std::unique_ptr sensor_; + std::unique_ptr csi2_; + std::unique_ptr csisp_; + std::unique_ptr output_; +}; + +} /* namespace libcamera */ From patchwork Thu Jun 18 10:18:52 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jacopo Mondi X-Patchwork-Id: 26928 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 3EA33C330F for ; Thu, 18 Jun 2026 10:19:15 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id B804E62C6B; Thu, 18 Jun 2026 12:19:14 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="kEaykUTi"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id AFDE8629E2 for ; Thu, 18 Jun 2026 12:19:02 +0200 (CEST) Received: from [192.168.125.177] (mob-109-113-4-199.net.vodafone.it [109.113.4.199]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 7D77ADF3; Thu, 18 Jun 2026 12:18:27 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1781777907; bh=gJjQ+ZviYAXTXT0jc81ax4NVY2j+qk1gkvld9egxiI8=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=kEaykUTi/RmABuM4hrEU1rqvEzMiuxI+lHkzuGiLQu2cAT1f2Ka/RkFLPjBZNxRco 2TZQtnv+0kWvVsM3HricuQ/vZABw+gU9GJ0g21Np48ejoEmgnJg+mf2LkP3IXbwD+5 TVR96TcDuuO+KuHBx7qKS8dQ2fWq/GtcPbmF9L7I= From: Jacopo Mondi Date: Thu, 18 Jun 2026 12:18:52 +0200 Subject: [PATCH 13/14] ipa: rppx1: Add AWB algorithm MIME-Version: 1.0 Message-Id: <20260618-rppx1-ipa-v1-13-32337264cfcd@ideasonboard.com> References: <20260618-rppx1-ipa-v1-0-32337264cfcd@ideasonboard.com> In-Reply-To: <20260618-rppx1-ipa-v1-0-32337264cfcd@ideasonboard.com> To: =?utf-8?q?Niklas_S=C3=B6derlund?= , libcamera-devel@lists.libcamera.org Cc: Jacopo Mondi X-Mailer: b4 0.14.3 X-Developer-Signature: v=1; a=openpgp-sha256; l=13952; i=jacopo.mondi@ideasonboard.com; h=from:subject:message-id; bh=gJjQ+ZviYAXTXT0jc81ax4NVY2j+qk1gkvld9egxiI8=; b=owEBbQKS/ZANAwAKAXI0Bo8WoVY8AcsmYgBqM8YKq3P+O2XCdyZusZBrjyJ9+TpkTCPDwJWgP J15uhH7h0GJAjMEAAEKAB0WIQS1xD1IgJogio9YOMByNAaPFqFWPAUCajPGCgAKCRByNAaPFqFW PFy5D/9Gb3ugTRW7s2JiS5j18EMGkExD3+zZL2qWGgTRQxmWbdTgA+fWAeDc+0BXOCwdh82qZb0 BvGnWAgH6qMiREC7nOeyK/sBZscVOB8ftG2rw0hRPwRposAXK0gr28uKvvNc6mom6IKkB9NiHjq EqgjYpVv31jbYKvV4C7+ZtBBUg3mHXzpB1croY2Tem746pEGFZnx77yobdCiGmrHcU+uOjFftLQ xH45hTpP8lLt8Tpe2WRjYwZsLxHuw53QcBNI8JN8qTPOq6kSz2XGI02/WByz6Be9qNbGyDKBn01 tOO02W3/EjKljgyYIj2cY3G35vf9xSfdCcdQd64zEikmEF0iKzPWZ/IBtSponUvEUOV80W++dfM QZmMjPX45NXF2i8Z5STCsXkZt8kGpnDqZ1fBAZ3nddIZdTc6HtWHJ7kkWkAqNd3HCdYrr25ays3 if1y5nbFSf3Iy1rq01pZxQnlsG9dNEOHDxiQ3yAxARH2/jZgimsu9p2/yJ6sDyeoXnPnA72Dt23 gO1SQcmHcWbeK15502JAWU0jJy1XAzJ6a1Im2TjjzDVyU90mrOdIh1/kTKU+SBGsRpnppN+kILf vDlEKUM2jUuLiQzse4NFBn5cLygN3RWGXfahV4NM1csKdguR5NnLGc60CXvpC3JCePT72uKC2+L KrHWZIH6rAAiSeQ== X-Developer-Key: i=jacopo.mondi@ideasonboard.com; a=openpgp; fpr=72392EDC88144A65C701EA9BA5826A2587AD026B X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Add Awb algorithm to the RPP-X1 IPA. The implementation is based on libIPA AwbAlgorithm. Signed-off-by: Jacopo Mondi --- src/ipa/rppx1/algorithms/awb.cpp | 298 +++++++++++++++++++++++++++++++++++ src/ipa/rppx1/algorithms/awb.h | 48 ++++++ src/ipa/rppx1/algorithms/meson.build | 5 + src/ipa/rppx1/ipa_context.cpp | 30 ++++ src/ipa/rppx1/ipa_context.h | 10 ++ src/ipa/rppx1/meson.build | 4 + 6 files changed, 395 insertions(+) diff --git a/src/ipa/rppx1/algorithms/awb.cpp b/src/ipa/rppx1/algorithms/awb.cpp new file mode 100644 index 000000000000..6316606b4b0f --- /dev/null +++ b/src/ipa/rppx1/algorithms/awb.cpp @@ -0,0 +1,298 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2026, Ideas On Board + * + * RPP-X1 AWB control algorithm + */ + + +#include "awb.h" + +#include +#include + +/** + * \file awb.h + * \brief RPP-X1 Auto White Balance + */ + +namespace libcamera { + +namespace ipa::rppx1::algorithms { + +/* \todo RPP-X1 doesn't support the Lux algorithm. */ +static constexpr unsigned int kDefaultLux = 500; + +LOG_DEFINE_CATEGORY(RppX1Awb) + +namespace { + +/* Quantize a gain to the UQ6.12 register format. */ +constexpr uint32_t quantizeGain(double gain) +{ + return UQ<6, 12>(static_cast(gain)).quantized(); +} + +}; /* namespace */ + +/* + * The RPP-X1 ISP has a bus width of 24 bits and all the measurement limits are + * expressed as 24 bit values. In order to make it simpler to express limits + * in code use an 8-bit value and left-shift by 16 to match the 24 bit width of + * the ISP processing pipeline. + */ +constexpr unsigned int kRPPX1BusWidthShift = 16; + +class RppX1AwbStats final : public AwbStats +{ +public: + RppX1AwbStats() {} + RppX1AwbStats(const RGB &rgbMeans) + : AwbStats(rgbMeans) + { + } + + /* Minimum mean value below which AWB can't operate. */ + double minColourValue() const override + { + return 2.0; + } +}; + +/** + * \class Awb + * \brief RPP-X1 white balance correction algorithm + */ + +/** + * \copydoc libcamera::ipa::Algorithm::init + */ +int Awb::init(IPAContext &context, const ValueNode &tuningData) +{ + return awbAlgo_.init(tuningData, context.ctrlMap); +} + +/** + * \copydoc libcamera::ipa::Algorithm::configure + */ +int Awb::configure(IPAContext &context, + const IPACameraSensorInfo &configInfo) +{ + awbAlgo_.configure(context.activeState.awb); + + /* + * Define the measurement window for AWB as a centered rectangle + * covering 3/4 of the image width and height. + */ + context.configuration.awb.measureWindow.h_offs = configInfo.outputSize.width / 8; + context.configuration.awb.measureWindow.v_offs = configInfo.outputSize.height / 8; + context.configuration.awb.measureWindow.h_size = 3 * configInfo.outputSize.width / 4; + context.configuration.awb.measureWindow.v_size = 3 * configInfo.outputSize.height / 4; + + context.configuration.awb.enabled = true; + + return 0; +} + +/** + * \copydoc libcamera::ipa::Algorithm::queueRequest + */ +void Awb::queueRequest(IPAContext &context, const uint32_t frame, + IPAFrameContext &frameContext, + const ControlList &controls) +{ + awbAlgo_.queueRequest(context.activeState.awb, frame, frameContext.awb, + controls); +} + +/** + * \copydoc libcamera::ipa::Algorithm::prepare + */ +void Awb::prepare(IPAContext &context, const uint32_t frame, + IPAFrameContext &frameContext, RppX1Params *params) +{ + awbAlgo_.prepare(context.activeState.awb, frameContext.awb); + + auto gainConfig = params->block(); + gainConfig.setEnabled(true); + + gainConfig->gain_green_b = quantizeGain(frameContext.awb.gains.g()); + gainConfig->gain_blue = quantizeGain(frameContext.awb.gains.b()); + gainConfig->gain_red = quantizeGain(frameContext.awb.gains.r()); + gainConfig->gain_green_r = quantizeGain(frameContext.awb.gains.g()); + + if (frame > 0) + return; + + /* + * If this is the first frame, program the white balance measurement + * engine. + */ + + auto awbConfig = params->block(); + awbConfig.setEnabled(true); + + /* Configure the measure window for AWB. */ + awbConfig->wnd = context.configuration.awb.measureWindow; + + /* Number of frames to use to estimate the means (0 means 1 frame). */ + awbConfig->frames = 0; + + /* Use YCbCr measurement mode. */ + awbConfig->mode = RPPX1_WBMEAS_MODE_YCBCR; + + /* Set the reference Cr and Cb (AWB target) to white. */ + awbConfig->ref_cb_max_b = 128 << kRPPX1BusWidthShift; + awbConfig->ref_cr_max_r = 128 << kRPPX1BusWidthShift; + + /* + * Filter out pixels based on luminance and chrominance values. + * The acceptable luma values are specified as a [16, 250] + * range, while the acceptable chroma values are specified + * with a minimum of 16 and a maximum Cb+Cr sum of 250; + */ + awbConfig->min_y_max_g = 16 << kRPPX1BusWidthShift; + awbConfig->max_y = 250 << kRPPX1BusWidthShift; + awbConfig->min_c = 16 << kRPPX1BusWidthShift; + awbConfig->max_csum = 250 << kRPPX1BusWidthShift; + + /* \todo Disable ymax_cmp even if we program 'max_y' */ + awbConfig->ymax_cmp = 0; + + /* + * Program coefficients and offsets to perform RGB-to-YCbCr + * conversion according to the BT.601 specification for limited + * range YUV. + * + * Y = 16 + 0.2500 R + 0.5000 G + 0.1094 B + * Cb = 128 - 0.1406 R - 0.2969 G + 0.4375 B + * Cr = 128 + 0.4375 R - 0.3750 G - 0.0625 B + * + * which resuls in the following register values: + * + * - coefficients are signed Q4.12 format + * - their numerical value is then 'reg / 2^12' + * - negative numbers need to reverse the 2's complement + * (!(reg & !BIT(16)) + 1) / 2^12 + * + * coeff G0 0x00000800; = 0.500 + * coeff B0 0x000001c0; = 0.1094 + * coeff R0 0x00000400; = 0.2500 + * coeff G1 0x0000fb40; = -0.2969 + * coeff B1 0x00000700; = 0.4375 + * coeff R1 0x0000fdc0; = -0.1406 + * coeff G2 0x0000fa00; = -0.3750 + * coeff B2 0x0000ff00; = -0.0625 + * coeff R2 0x00000700; = 0.4375 + * offset R 0x00100000; offset_r = (16 << 16) + * offset G 0x00800000; offset_g = (128 << 16) + * offset B 0x00800000; offset_b = (128 << 16) + * + * Use the inverse of this matrix in calculateRgbMeans() to + * reverse the colorspace conversion. + */ + awbConfig->ccor_coeff[0][0] = 0x00000800; + awbConfig->ccor_coeff[0][1] = 0x000001c0; + awbConfig->ccor_coeff[0][2] = 0x00000400; + awbConfig->ccor_coeff[1][0] = 0x0000fb40; + awbConfig->ccor_coeff[1][1] = 0x00000700; + awbConfig->ccor_coeff[1][2] = 0x0000fdc0; + awbConfig->ccor_coeff[2][0] = 0x0000fa00; + awbConfig->ccor_coeff[2][1] = 0x0000ff00; + awbConfig->ccor_coeff[2][2] = 0x00000700; + + awbConfig->ccor_offs[0] = 16 << kRPPX1BusWidthShift; + awbConfig->ccor_offs[1] = 128 << kRPPX1BusWidthShift; + awbConfig->ccor_offs[2] = 128 << kRPPX1BusWidthShift; +} + +/** + * \copydoc libcamera::ipa::Algorithm::process + */ +void Awb::process(IPAContext &context, + [[maybe_unused]] const uint32_t frame, + IPAFrameContext &frameContext, + const RppX1Stats *stats, + ControlList &metadata) +{ + RppX1AwbStats awbStats = calculateRgbMeans(frameContext, stats); + + awbAlgo_.process(context.activeState.awb, frameContext.awb, awbStats, + kDefaultLux, metadata); +} + +RppX1AwbStats Awb::calculateRgbMeans(const IPAFrameContext &frameContext, + const RppX1Stats *stats) const +{ + if (!stats) + return {}; + + const auto awb = stats->block(); + if (!awb) + return {}; + + if (awb->cnt == 0) { + LOG(RppX1Awb, Debug) << "AWB statistics are empty"; + return {}; + } + + Vector rgbMeans; + + /* Get the YCbCr mean values: we use YCbCr measurement mode. */ + Vector yuvMeans({ + static_cast(awb->mean_y_or_g), + static_cast(awb->mean_cb_or_b), + static_cast(awb->mean_cr_or_r) + }); + + /* + * Convert from YCbCr to RGB. The statistics engine has been + * programmed with the following matrix: + * + * Y = 16 + 0.2500 R + 0.5000 G + 0.1094 B + * Cb = 128 - 0.1406 R - 0.2969 G + 0.4375 B + * Cr = 128 + 0.4375 R - 0.3750 G - 0.0625 B + * + * Use the inverse matrix here. + */ + static const Matrix yuv2rgbMatrix({ + 1.1636, -0.0623, 1.6008, + 1.1636, -0.4045, -0.7949, + 1.1636, 1.9912, -0.0250 + }); + static const Vector yuv2rgbOffset({ + 16, 128, 128 + }); + + rgbMeans = yuv2rgbMatrix * (yuvMeans - yuv2rgbOffset); + + /* + * Due to hardware rounding errors in the YCbCr means, the + * calculated RGB means may be negative. This would lead to + * negative gains, messing up calculation. Prevent this by + * clamping the means to positive values. + */ + rgbMeans = rgbMeans.max(0.0); + + /* + * \todo + * The ISP computes the AWB means after applying the CCM. Apply + * the inverse as we want to get the raw means before the colour gains. + * rgbMeans = frameContext.ccm.ccm.inverse() * rgbMeans; + */ + + /* + * The ISP computes the AWB means after applying the colour gains, + * divide by the gains that were used to get the raw means from the + * sensor. Apply a minimum value to avoid divisions by near-zero. + */ + rgbMeans /= frameContext.awb.gains.max(0.01); + + return RppX1AwbStats(rgbMeans); +} + +REGISTER_IPA_ALGORITHM(Awb, "Awb") + +} /* namespace ipa::rppx1::algorithms */ + +} /* namespace libcamera */ diff --git a/src/ipa/rppx1/algorithms/awb.h b/src/ipa/rppx1/algorithms/awb.h new file mode 100644 index 000000000000..2e4f67f11bc0 --- /dev/null +++ b/src/ipa/rppx1/algorithms/awb.h @@ -0,0 +1,48 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2026, Ideas On Board + * + * AWB control algorithm + */ + +#pragma once + +#include "libipa/awb.h" +#include "libipa/fixedpoint.h" + +#include "algorithm.h" + +namespace libcamera { + +namespace ipa::rppx1::algorithms { + +class RppX1AwbStats; + +class Awb : public Algorithm +{ +public: + Awb() = default; + ~Awb() = default; + + int init(IPAContext &context, const ValueNode &tuningData) override; + int configure(IPAContext &context, const IPACameraSensorInfo &configInfo) override; + void queueRequest(IPAContext &context, const uint32_t frame, + IPAFrameContext &frameContext, + const ControlList &controls) override; + void prepare(IPAContext &context, const uint32_t frame, + IPAFrameContext &frameContext, + RppX1Params *params) override; + void process(IPAContext &context, const uint32_t frame, + IPAFrameContext &frameContext, + const RppX1Stats *stats, + ControlList &metadata) override; + +private: + RppX1AwbStats calculateRgbMeans(const IPAFrameContext &frameContext, + const RppX1Stats *stats) const; + + AwbAlgorithm> awbAlgo_; +}; + +} /* namespace ipa::rppx1::algorithms */ +} /* namespace libcamera */ diff --git a/src/ipa/rppx1/algorithms/meson.build b/src/ipa/rppx1/algorithms/meson.build new file mode 100644 index 000000000000..70368ad9bac9 --- /dev/null +++ b/src/ipa/rppx1/algorithms/meson.build @@ -0,0 +1,5 @@ +# SPDX-License-Identifier: CC0-1.0 + +rppx1_ipa_algorithms = files([ + 'awb.cpp', +]) diff --git a/src/ipa/rppx1/ipa_context.cpp b/src/ipa/rppx1/ipa_context.cpp index 5ac60dfc7bf6..ea7008f5685d 100644 --- a/src/ipa/rppx1/ipa_context.cpp +++ b/src/ipa/rppx1/ipa_context.cpp @@ -16,21 +16,51 @@ namespace libcamera::ipa::rppx1 { +/** + * \struct RppX1AwbSession + * \brief RPP-X1 Awb session configuration + */ + +/** + * \var RppX1AwbSession::measureWindow + * \brief Awb measurement window + */ + +/** + * \var RppX1AwbSession::enabled + * \brief Awb enabled flag + */ + /** * \struct IPASessionConfiguration * \brief Session configuration for the IPA module */ +/** + * \var IPASessionConfiguration::awb + * \brief Awb session configuration + */ + /** * \struct IPAActiveState * \brief Active state for algorithms */ +/** + * \var IPAActiveState::awb + * \copydoc ipa::awb::ActiveState + */ + /** * \struct IPAFrameContext * \brief Per-frame context for algorithms */ +/** + * \var IPAFrameContext::awb + * \copydoc ipa::awb::FrameContext + */ + /** * \struct IPAContext * \brief Global IPA context data shared between all algorithms diff --git a/src/ipa/rppx1/ipa_context.h b/src/ipa/rppx1/ipa_context.h index f268a35081bc..57197d865d3f 100644 --- a/src/ipa/rppx1/ipa_context.h +++ b/src/ipa/rppx1/ipa_context.h @@ -20,17 +20,27 @@ #include #include +#include "libipa/awb.h" + namespace libcamera { namespace ipa::rppx1 { +struct RppX1AwbSession { + struct rppx1_window measureWindow; + bool enabled; +}; + struct IPASessionConfiguration { + struct RppX1AwbSession awb; }; struct IPAActiveState { + ipa::awb::ActiveState awb; }; struct IPAFrameContext : public FrameContext { + ipa::awb::FrameContext awb; }; struct IPAContext { diff --git a/src/ipa/rppx1/meson.build b/src/ipa/rppx1/meson.build index 8034fe24d241..8b8c79e997c1 100644 --- a/src/ipa/rppx1/meson.build +++ b/src/ipa/rppx1/meson.build @@ -1,4 +1,6 @@ # SPDX-License-Identifier: CC0-1.0 +# +subdir('algorithms') ipa_name = 'ipa_rppx1' @@ -7,6 +9,8 @@ rppx1_ipa_sources = files([ 'rppx1.cpp', ]) +rppx1_ipa_sources += rppx1_ipa_algorithms + mod = shared_module(ipa_name, rppx1_ipa_sources, name_prefix : '', include_directories : [ipa_includes], From patchwork Thu Jun 18 10:18:53 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jacopo Mondi X-Patchwork-Id: 26929 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 2A560C3310 for ; Thu, 18 Jun 2026 10:19:16 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 75ED562B32; Thu, 18 Jun 2026 12:19:15 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="PZPrBhA8"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id B3210629E3 for ; Thu, 18 Jun 2026 12:19:03 +0200 (CEST) Received: from [192.168.125.177] (mob-109-113-4-199.net.vodafone.it [109.113.4.199]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 382212590; Thu, 18 Jun 2026 12:18:28 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1781777908; bh=Uo14V8xv8NdLqXdbwptWsj5GofZHVGzwazfa8t67n2k=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=PZPrBhA8UTrNMd/90FiqYITcWaQnYlpAnNjuJAb7Ye+HUFD5NvpFKfYFfpXL97AcH aIg0/XYjBDErbApWXPOpjepEFHW4MlMZa6/+KHcU99nt4AzkCx4ynHhP51skH9QPoA lYurhFBGXSn6SUHTJFfHvzv4sw+SXuKd/O3ANBdw= From: Jacopo Mondi Date: Thu, 18 Jun 2026 12:18:53 +0200 Subject: [PATCH 14/14] ipa: rppx1: Add Ccm algorithm MIME-Version: 1.0 Message-Id: <20260618-rppx1-ipa-v1-14-32337264cfcd@ideasonboard.com> References: <20260618-rppx1-ipa-v1-0-32337264cfcd@ideasonboard.com> In-Reply-To: <20260618-rppx1-ipa-v1-0-32337264cfcd@ideasonboard.com> To: =?utf-8?q?Niklas_S=C3=B6derlund?= , libcamera-devel@lists.libcamera.org Cc: Jacopo Mondi X-Mailer: b4 0.14.3 X-Developer-Signature: v=1; a=openpgp-sha256; l=8128; i=jacopo.mondi@ideasonboard.com; h=from:subject:message-id; bh=Uo14V8xv8NdLqXdbwptWsj5GofZHVGzwazfa8t67n2k=; b=owEBbQKS/ZANAwAKAXI0Bo8WoVY8AcsmYgBqM8YKJZmZxVQna/U6TvrSg3aw0EiU3i4xZUzoq RoRQNVlVhiJAjMEAAEKAB0WIQS1xD1IgJogio9YOMByNAaPFqFWPAUCajPGCgAKCRByNAaPFqFW PDnZD/42Xqm7u6JCysWka4KOGxulySLFsTVAjkfBo9UATq4j81q36L/niTFeZwXRb5a640TSB8u tklugP4ocQJYx1Mh+BD+oMBB888COW6+rVD4DC/lt+I5dgIzncTwGUvgJ6z1tFR9GgahLbcS7Yh HttaJU+hjJO5lupFs4q23V4cDXBGJ+KLIKKrF7/9klINev4M4x8EOP//UYc5A9GMqRGWNndM3Rj jKcQw7/HCtoNbKWaTOWJ0fsBLX6Y4LuTQRLTZgKJ/XsVaLTTWl3B1GCYo5UTrBWoBGYEPbjXxb2 PIjBId/IOgXH/vE1yZm6OE3zXKJN80YuK+Ymv02VkYerjKlfcnZAKb+hseOl6umOQxpShXI4TL1 LHQJS2PQKziWznRiemJhHT/NCNWzB2WLVLUfw4chCVxD4yi6pBim2XwXSUS1KS7lsNybcmJfaGm rf+zMEeR5PXqdhT4SL547dbYrcepiKZNrIpEZxyQoy/BgaxhcJUMHNt+h8k3WUaJ/Um50H7Ty+a +J5z0NKIso/ChDnYjcjh/SnKFmx9jmCVBdBPk33GZXJV3Txc/1kGIFwzeqeoDNBPylQne9yngkq NTaYYPuFd5A495lbllmDzVGNFlphGe3ulH34+l1GUjHSyMkIyC+tgnt3ScqJIldH/BXXe2XoKP5 58DAyXICH58MOBQ== X-Developer-Key: i=jacopo.mondi@ideasonboard.com; a=openpgp; fpr=72392EDC88144A65C701EA9BA5826A2587AD026B X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Add Ccm algorithm to the RPP-X1 IPA. The implementation is based on libIPA CcmAlgorithm. Signed-off-by: Jacopo Mondi --- src/ipa/rppx1/algorithms/awb.cpp | 3 +- src/ipa/rppx1/algorithms/ccm.cpp | 124 +++++++++++++++++++++++++++++++++++ src/ipa/rppx1/algorithms/ccm.h | 54 +++++++++++++++ src/ipa/rppx1/algorithms/meson.build | 1 + src/ipa/rppx1/ipa_context.cpp | 10 +++ src/ipa/rppx1/ipa_context.h | 3 + 6 files changed, 193 insertions(+), 2 deletions(-) diff --git a/src/ipa/rppx1/algorithms/awb.cpp b/src/ipa/rppx1/algorithms/awb.cpp index 6316606b4b0f..598e4503513c 100644 --- a/src/ipa/rppx1/algorithms/awb.cpp +++ b/src/ipa/rppx1/algorithms/awb.cpp @@ -275,11 +275,10 @@ RppX1AwbStats Awb::calculateRgbMeans(const IPAFrameContext &frameContext, rgbMeans = rgbMeans.max(0.0); /* - * \todo * The ISP computes the AWB means after applying the CCM. Apply * the inverse as we want to get the raw means before the colour gains. - * rgbMeans = frameContext.ccm.ccm.inverse() * rgbMeans; */ + rgbMeans = frameContext.ccm.ccm.inverse() * rgbMeans; /* * The ISP computes the AWB means after applying the colour gains, diff --git a/src/ipa/rppx1/algorithms/ccm.cpp b/src/ipa/rppx1/algorithms/ccm.cpp new file mode 100644 index 000000000000..4b485d1cff0e --- /dev/null +++ b/src/ipa/rppx1/algorithms/ccm.cpp @@ -0,0 +1,124 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2026, Ideas On Board + * + * RPP-X1 color correction matrix control algorithm + */ + +#include "ccm.h" + +#include + +#include +#include + +#include + +#include + +/** + * \file ccm.h + */ + +namespace libcamera { + +namespace ipa::rppx1::algorithms { + +LOG_DEFINE_CATEGORY(RppX1Ccm) + +/** + * \class Ccm + * \brief Color correction matrix algorithm + */ + +/** + * \copydoc libcamera::ipa::Algorithm::init + */ +int Ccm::init([[maybe_unused]] IPAContext &context, const ValueNode &tuningData) +{ + return ccmAlgo_.init(tuningData, context.ctrlMap); +} + +/** + * \copydoc libcamera::ipa::Algorithm::configure + */ +int Ccm::configure(IPAContext &context, + [[maybe_unused]] const IPACameraSensorInfo &configInfo) +{ + return ccmAlgo_.configure(context.activeState.ccm, + context.activeState.awb.automatic.temperatureK); +} + +void Ccm::queueRequest(IPAContext &context, + [[maybe_unused]] const uint32_t frame, + IPAFrameContext &frameContext, + const ControlList &controls) +{ + /* Nothing to do here, the ccm will be calculated in prepare() */ + if (frameContext.awb.autoEnabled) + return; + + ccmAlgo_.queueRequest(context.activeState.ccm, frameContext.ccm, controls); +} + +void Ccm::setParameters(RppX1Params *params, IPAFrameContext &context) +{ + const Matrix &matrix = context.ccm.ccm; + const Matrix &offsets = context.ccm.offsets; + + auto config = params->block(); + config.setEnabled(true); + + /* + * RPP-X1 coefficients are 16 bits Q4.12 signed fixed-point ranging from + * -8 (0x8000) to +7.9996 (0x7fff). x1 = 0x1000. + */ + for (unsigned int i = 0; i < 3; i++) { + for (unsigned int j = 0; j < 3; j++) + config->coeff[i][j] = Q<4, 12>(matrix[i][j]).quantized(); + } + + /* + * RPP-X1 offsets are 25 bits 2's complement while the CcmAlgorithm + * class uses int16_t. + * + * \todo: Better investigate how negative offsets are handled in the + * offsets interpolation. + */ + for (unsigned int i = 0; i < 3; i++) + config->offset[i] = static_cast(offsets[i][0]); + + LOG(RppX1Ccm, Debug) << "Setting matrix " << matrix; + LOG(RppX1Ccm, Debug) << "Setting offsets " << offsets; +} + +/** + * \copydoc libcamera::ipa::Algorithm::prepare + */ +void Ccm::prepare(IPAContext &context, const uint32_t frame, + IPAFrameContext &frameContext, RppX1Params *params) +{ + if (frameContext.awb.autoEnabled) + ccmAlgo_.prepare(context.activeState.ccm, frameContext.ccm, + frame, frameContext.awb.temperatureK); + + setParameters(params, frameContext); +} + +/** + * \copydoc libcamera::ipa::Algorithm::process + */ +void Ccm::process([[maybe_unused]] IPAContext &context, + [[maybe_unused]] const uint32_t frame, + IPAFrameContext &frameContext, + [[maybe_unused]] const RppX1Stats *stats, + ControlList &metadata) +{ + ccmAlgo_.process(frameContext.ccm, metadata); +} + +REGISTER_IPA_ALGORITHM(Ccm, "Ccm") + +} /* namespace ipa::rppx1::algorithms */ + +} /* namespace libcamera */ diff --git a/src/ipa/rppx1/algorithms/ccm.h b/src/ipa/rppx1/algorithms/ccm.h new file mode 100644 index 000000000000..5e183d255a48 --- /dev/null +++ b/src/ipa/rppx1/algorithms/ccm.h @@ -0,0 +1,54 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2026, Ideas On Board + * + * RPP-X1 color correction matrix control algorithm + */ + +#pragma once + +#include "libcamera/internal/matrix.h" + +#include + +#include "algorithm.h" + +namespace libcamera { + +namespace ipa::rppx1::algorithms { + +class Ccm : public Algorithm +{ +public: + Ccm() {} + ~Ccm() = default; + + int init(IPAContext &context, const ValueNode &tuningData) override; + int configure(IPAContext &context, + const IPACameraSensorInfo &configInfo) override; + void queueRequest(IPAContext &context, + const uint32_t frame, + IPAFrameContext &frameContext, + const ControlList &controls) override; + void prepare(IPAContext &context, const uint32_t frame, + IPAFrameContext &frameContext, + RppX1Params *params) override; + void process(IPAContext &context, const uint32_t frame, + IPAFrameContext &frameContext, + const RppX1Stats *stats, + ControlList &metadata) override; + +private: + void parseYaml(const ValueNode &tuningData); + void setParameters(RppX1Params *params, IPAFrameContext &context); + + unsigned int ct_; + Interpolator> ccm_; + Interpolator> offsets_; + + CcmAlgorithm> ccmAlgo_; +}; + +} /* namespace ipa::rppx1::algorithms */ + +} /* namespace libcamera */ diff --git a/src/ipa/rppx1/algorithms/meson.build b/src/ipa/rppx1/algorithms/meson.build index 70368ad9bac9..ddebbfbecbee 100644 --- a/src/ipa/rppx1/algorithms/meson.build +++ b/src/ipa/rppx1/algorithms/meson.build @@ -2,4 +2,5 @@ rppx1_ipa_algorithms = files([ 'awb.cpp', + 'ccm.cpp' ]) diff --git a/src/ipa/rppx1/ipa_context.cpp b/src/ipa/rppx1/ipa_context.cpp index ea7008f5685d..57a2b56774bc 100644 --- a/src/ipa/rppx1/ipa_context.cpp +++ b/src/ipa/rppx1/ipa_context.cpp @@ -51,6 +51,11 @@ namespace libcamera::ipa::rppx1 { * \copydoc ipa::awb::ActiveState */ +/** + * \var IPAActiveState::ccm + * \copydoc ipa::ccm::ActiveState + */ + /** * \struct IPAFrameContext * \brief Per-frame context for algorithms @@ -61,6 +66,11 @@ namespace libcamera::ipa::rppx1 { * \copydoc ipa::awb::FrameContext */ +/** + * \var IPAFrameContext::ccm + * \copydoc ipa::ccm::FrameContext + */ + /** * \struct IPAContext * \brief Global IPA context data shared between all algorithms diff --git a/src/ipa/rppx1/ipa_context.h b/src/ipa/rppx1/ipa_context.h index 57197d865d3f..bba1d10da73f 100644 --- a/src/ipa/rppx1/ipa_context.h +++ b/src/ipa/rppx1/ipa_context.h @@ -21,6 +21,7 @@ #include #include "libipa/awb.h" +#include "libipa/ccm.h" namespace libcamera { @@ -37,10 +38,12 @@ struct IPASessionConfiguration { struct IPAActiveState { ipa::awb::ActiveState awb; + ipa::ccm::ActiveState ccm; }; struct IPAFrameContext : public FrameContext { ipa::awb::FrameContext awb; + ipa::ccm::FrameContext ccm; }; struct IPAContext {