From patchwork Wed Feb 5 13:04:16 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kieran Bingham X-Patchwork-Id: 2771 Return-Path: Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 03C8760444 for ; Wed, 5 Feb 2020 14:04:26 +0100 (CET) Received: from localhost.localdomain (cpc89242-aztw30-2-0-cust488.18-1.cable.virginm.net [86.31.129.233]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 9507352F; Wed, 5 Feb 2020 14:04:25 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1580907865; bh=t9cqojs86Tp/WE1nUKpMnhO2V9ifQ5SLP+hE2LbOlfU=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=gy7dV5aPaxu6M6pwlewS4gAGaN4yy+mzKZb7KtnholQHqhoNWe4RdpaN5GBRqPXMj lJxgQBoLAUuLOSyXrLDLoUXw/JkJsK2gZMtbIGfwbInRJh4xHZUKaTco/B/+/7xqGC BpOUcq4guWm4mpmy/yntlMp3dI/R1qnVeiz31fFM= From: Kieran Bingham To: LibCamera Devel Date: Wed, 5 Feb 2020 13:04:16 +0000 Message-Id: <20200205130420.8736-2-kieran.bingham@ideasonboard.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20200205130420.8736-1-kieran.bingham@ideasonboard.com> References: <20200205130420.8736-1-kieran.bingham@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH 1/5] libcamera: utils: Provide helper to get dynamic library runpath X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 05 Feb 2020 13:04:26 -0000 Provide a helper that will identify the DT_RUNPATH string from the ELF dynamic library string table entries. Signed-off-by: Kieran Bingham Reviewed-by: Laurent Pinchart --- src/libcamera/include/utils.h | 1 + src/libcamera/utils.cpp | 26 ++++++++++++++++++++++++++ 2 files changed, 27 insertions(+) diff --git a/src/libcamera/include/utils.h b/src/libcamera/include/utils.h index e467eb21c518..069b327bb436 100644 --- a/src/libcamera/include/utils.h +++ b/src/libcamera/include/utils.h @@ -33,6 +33,7 @@ namespace utils { const char *basename(const char *path); char *secure_getenv(const char *name); +const char *get_runpath(); template unsigned int set_overlap(InputIt1 first1, InputIt1 last1, diff --git a/src/libcamera/utils.cpp b/src/libcamera/utils.cpp index 4beffdab5eb6..e48cf2b6a48e 100644 --- a/src/libcamera/utils.cpp +++ b/src/libcamera/utils.cpp @@ -7,7 +7,9 @@ #include "utils.h" +#include #include +#include #include #include #include @@ -70,6 +72,30 @@ char *secure_getenv(const char *name) #endif } +/** + * \brief Obtain the runpath string from the dynamic symbol table + * \returns a const char pointer to the runpath entry in the elf string table + * or NULL if it is not found. + */ +const char *get_runpath() +{ + const ElfW(Dyn) *dyn = _DYNAMIC; + const ElfW(Dyn) *runpath = NULL; + const char *strtab = NULL; + + for (; dyn->d_tag != DT_NULL; ++dyn) { + if (dyn->d_tag == DT_STRTAB) + strtab = reinterpret_cast(dyn->d_un.d_val); + else if (dyn->d_tag == DT_RUNPATH) + runpath = dyn; + } + + if (strtab && runpath) + return strtab + runpath->d_un.d_val; + + return NULL; +} + /** * \fn libcamera::utils::set_overlap(InputIt1 first1, InputIt1 last1, * InputIt2 first2, InputIt2 last2) From patchwork Wed Feb 5 13:04:17 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kieran Bingham X-Patchwork-Id: 2772 Return-Path: Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 3EEB360443 for ; Wed, 5 Feb 2020 14:04:26 +0100 (CET) Received: from localhost.localdomain (cpc89242-aztw30-2-0-cust488.18-1.cable.virginm.net [86.31.129.233]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id DD08FA3B; Wed, 5 Feb 2020 14:04:25 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1580907866; bh=U2Oldkn9fy3GFixU+eQ9nvu7IcEXlTNY+WEr64ZwhE8=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=NwGYpDYCnaISU800BaTQVS4mVuPx/9nRoMaesSY93Z3cR8mvhdvs7U+fFvJ2WAIm0 of27DmGmRWsFWEcJcz5aBdfdGFSuLxBmo05jar30ZRyq+3vqBRh+nSYQnr4l6V+O2N IW9shWk1OkGkvJVoV6C4vQmR9wCUQbbWbEnVYn0g= From: Kieran Bingham To: LibCamera Devel Date: Wed, 5 Feb 2020 13:04:17 +0000 Message-Id: <20200205130420.8736-3-kieran.bingham@ideasonboard.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20200205130420.8736-1-kieran.bingham@ideasonboard.com> References: <20200205130420.8736-1-kieran.bingham@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH 2/5] libcamera: ipa_manager: Split path handling X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 05 Feb 2020 13:04:26 -0000 Move the iteration of a colon separated path to it's own function. This prepares for parsing extra path variables. Signed-off-by: Kieran Bingham --- src/libcamera/include/ipa_manager.h | 1 + src/libcamera/ipa_manager.cpp | 52 +++++++++++++++++++---------- 2 files changed, 36 insertions(+), 17 deletions(-) diff --git a/src/libcamera/include/ipa_manager.h b/src/libcamera/include/ipa_manager.h index 126f8babbc8f..94d35d9062e4 100644 --- a/src/libcamera/include/ipa_manager.h +++ b/src/libcamera/include/ipa_manager.h @@ -33,6 +33,7 @@ private: ~IPAManager(); int addDir(const char *libDir); + int addPath(const char *path); }; } /* namespace libcamera */ diff --git a/src/libcamera/ipa_manager.cpp b/src/libcamera/ipa_manager.cpp index 92adc6c45015..048465c37772 100644 --- a/src/libcamera/ipa_manager.cpp +++ b/src/libcamera/ipa_manager.cpp @@ -110,23 +110,7 @@ IPAManager::IPAManager() return; } - const char *paths = modulePaths; - while (1) { - const char *delim = strchrnul(paths, ':'); - size_t count = delim - paths; - - if (count) { - std::string path(paths, count); - ret = addDir(path.c_str()); - if (ret > 0) - ipaCount += ret; - } - - if (*delim == '\0') - break; - - paths += count + 1; - } + ipaCount += addPath(modulePaths); if (!ipaCount) LOG(IPAManager, Warning) @@ -206,6 +190,40 @@ int IPAManager::addDir(const char *libDir) return count; } +/** + * \brief Load IPA modules from a colon separated PATH variable + * \param[in] path string to split to search for IPA modules + * + * This method tries to create an IPAModule instance for every shared object + * found in the directories described by \a paths. + * + * \return Number of modules loaded by this call, or a negative error code + * otherwise + */ +int IPAManager::addPath(const char *paths) +{ + int ipaCount = 0; + + while (1) { + const char *delim = strchrnul(paths, ':'); + size_t count = delim - paths; + + if (count) { + std::string path(paths, count); + int ret = addDir(path.c_str()); + if (ret > 0) + ipaCount += ret; + } + + if (*delim == '\0') + break; + + paths += count + 1; + } + + return ipaCount; +} + /** * \brief Create an IPA interface that matches a given pipeline handler * \param[in] pipe The pipeline handler that wants a matching IPA interface From patchwork Wed Feb 5 13:04:18 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kieran Bingham X-Patchwork-Id: 2773 Return-Path: Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 8809F60443 for ; Wed, 5 Feb 2020 14:04:26 +0100 (CET) Received: from localhost.localdomain (cpc89242-aztw30-2-0-cust488.18-1.cable.virginm.net [86.31.129.233]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 31E87E48; Wed, 5 Feb 2020 14:04:26 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1580907866; bh=t+AzHqUDfnwtvOSNQDS6KA6QGxbt0Kw3HStE9Q3n54Q=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=Ag6rZftd5sOi8aRsSGNj0buDdMtKJ7ntq6Hk0u8WT2svZwNvj0Snq+TdRNRafkVhB 6k94ACCcOBbkgsOyn66dXL5pMwBZA8ovz6wgnXq+V9cUYa8bZf0wi3IQctznkaBOXL AxZoxXzdvRvTby0xQ42BMzQKla0Z5uZlnylrWf8Y= From: Kieran Bingham To: LibCamera Devel Date: Wed, 5 Feb 2020 13:04:18 +0000 Message-Id: <20200205130420.8736-4-kieran.bingham@ideasonboard.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20200205130420.8736-1-kieran.bingham@ideasonboard.com> References: <20200205130420.8736-1-kieran.bingham@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH 3/5] libcamera: ipa_manager: Allow recursive parsing X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 05 Feb 2020 13:04:26 -0000 Provide an optional means to recurse into subdirectories to search for IPA libraries. This allows IPAs contained within their own build directory to be resolved when loading from a non-installed build. Signed-off-by: Kieran Bingham --- src/libcamera/include/ipa_manager.h | 6 ++-- src/libcamera/ipa_manager.cpp | 54 ++++++++++++++++++++++++----- 2 files changed, 49 insertions(+), 11 deletions(-) diff --git a/src/libcamera/include/ipa_manager.h b/src/libcamera/include/ipa_manager.h index 94d35d9062e4..8243ba5a1f51 100644 --- a/src/libcamera/include/ipa_manager.h +++ b/src/libcamera/include/ipa_manager.h @@ -32,8 +32,10 @@ private: IPAManager(); ~IPAManager(); - int addDir(const char *libDir); - int addPath(const char *path); + int parseDir(const char *libDir, std::vector &paths, + unsigned int subdirs); + int addDir(const char *libDir, unsigned int subdirs = 0); + int addPath(const char *path, unsigned int subdirs = 0); }; } /* namespace libcamera */ diff --git a/src/libcamera/ipa_manager.cpp b/src/libcamera/ipa_manager.cpp index 048465c37772..24fe709108fe 100644 --- a/src/libcamera/ipa_manager.cpp +++ b/src/libcamera/ipa_manager.cpp @@ -140,16 +140,17 @@ IPAManager *IPAManager::instance() } /** - * \brief Load IPA modules from a directory + * \brief Identify shared library objects within a directory * \param[in] libDir directory to search for IPA modules + * \param[in] subdirs maximum number of sub-directories to parse * - * This method tries to create an IPAModule instance for every shared object - * found in \a libDir, and skips invalid IPA modules. + * Search a directory for .so files. allowing recursion down to + * subdirectories no further than the quantity specified by \a subdirs * - * \return Number of modules loaded by this call, or a negative error code - * otherwise + * \return 0 on success or a negative error code otherwise */ -int IPAManager::addDir(const char *libDir) +int IPAManager::parseDir(const char *libDir, std::vector &paths, + unsigned int subdirs) { struct dirent *ent; DIR *dir; @@ -158,8 +159,20 @@ int IPAManager::addDir(const char *libDir) if (!dir) return -errno; - std::vector paths; while ((ent = readdir(dir)) != nullptr) { + if (ent->d_type == DT_DIR && subdirs) { + if (strcmp(ent->d_name, ".") == 0 || + strcmp(ent->d_name, "..") == 0) + continue; + + std::string subdir = std::string(libDir) + "/" + ent->d_name; + + /* Only recurse to the limit specified by subdirs */ + parseDir(subdir.c_str(), paths, subdirs - 1); + + continue; + } + int offset = strlen(ent->d_name) - 3; if (offset < 0) continue; @@ -170,6 +183,28 @@ int IPAManager::addDir(const char *libDir) } closedir(dir); + return 0; +} + +/** + * \brief Load IPA modules from a directory + * \param[in] libDir directory to search for IPA modules + * \param[in] subdirs maximum number of sub-directories to parse + * + * This method tries to create an IPAModule instance for every shared object + * found in \a libDir, and skips invalid IPA modules. + * + * \return Number of modules loaded by this call, or a negative error code + * otherwise + */ +int IPAManager::addDir(const char *libDir, unsigned int subdirs) +{ + std::vector paths; + + int ret = parseDir(libDir, paths, subdirs); + if (ret < 0) + return ret; + /* Ensure a stable ordering of modules. */ std::sort(paths.begin(), paths.end()); @@ -193,6 +228,7 @@ int IPAManager::addDir(const char *libDir) /** * \brief Load IPA modules from a colon separated PATH variable * \param[in] path string to split to search for IPA modules + * \param[in] subdirs maximum number of sub-directories to parse * * This method tries to create an IPAModule instance for every shared object * found in the directories described by \a paths. @@ -200,7 +236,7 @@ int IPAManager::addDir(const char *libDir) * \return Number of modules loaded by this call, or a negative error code * otherwise */ -int IPAManager::addPath(const char *paths) +int IPAManager::addPath(const char *paths, unsigned int subdirs) { int ipaCount = 0; @@ -210,7 +246,7 @@ int IPAManager::addPath(const char *paths) if (count) { std::string path(paths, count); - int ret = addDir(path.c_str()); + int ret = addDir(path.c_str(), subdirs); if (ret > 0) ipaCount += ret; } From patchwork Wed Feb 5 13:04:19 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kieran Bingham X-Patchwork-Id: 2774 Return-Path: Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id D264760912 for ; Wed, 5 Feb 2020 14:04:26 +0100 (CET) Received: from localhost.localdomain (cpc89242-aztw30-2-0-cust488.18-1.cable.virginm.net [86.31.129.233]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 7B3664F8; Wed, 5 Feb 2020 14:04:26 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1580907866; bh=F62auJcz8x0tN8NLTtSNSGfS7N/7zl7eW81saD4rxN0=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=MYfkVZYwxjs7O7GIdPKmOrHvTPQLsUHca6+7TxvXLYA+Ha4gGb2rRWTIYsAlgTkze /uJoz+3Nyk1p4vi1a6RCtLIWh0imS64KFxj6TtJzChKIRjoDVK6ivPeZdgi7JoQQ6N MwEKCmCHB3VvTloN2kzQ24+YR+YBQ45+NQZrzeWE= From: Kieran Bingham To: LibCamera Devel Date: Wed, 5 Feb 2020 13:04:19 +0000 Message-Id: <20200205130420.8736-5-kieran.bingham@ideasonboard.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20200205130420.8736-1-kieran.bingham@ideasonboard.com> References: <20200205130420.8736-1-kieran.bingham@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH 4/5] libcamera: ipa_manager: Search the runpath for IPA libraries X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 05 Feb 2020 13:04:27 -0000 Expose the IPA build path through the libcamera library runpath entry in the elf string table. Utilise this path to search for compiled IPA libraries when loading the IPAManager. The build time path is removed during the library install phase. Signed-off-by: Kieran Bingham --- src/libcamera/ipa_manager.cpp | 9 +++++++++ src/libcamera/meson.build | 1 + src/meson.build | 7 +++++++ 3 files changed, 17 insertions(+) diff --git a/src/libcamera/ipa_manager.cpp b/src/libcamera/ipa_manager.cpp index 24fe709108fe..3dfe84fcf95e 100644 --- a/src/libcamera/ipa_manager.cpp +++ b/src/libcamera/ipa_manager.cpp @@ -102,6 +102,15 @@ IPAManager::IPAManager() if (ret > 0) ipaCount += ret; + /* + * Utilise the Elf runpath to supply the IPA build directory, allowing + * us to load IPA's before they are installed for testing and + * development. + */ + const char *runpath = utils::get_runpath(); + if (runpath) + ipaCount += addPath(runpath, true); + const char *modulePaths = utils::secure_getenv("LIBCAMERA_IPA_MODULE_PATH"); if (!modulePaths) { if (!ipaCount) diff --git a/src/libcamera/meson.build b/src/libcamera/meson.build index 68d89559b290..87c5bbbb17a4 100644 --- a/src/libcamera/meson.build +++ b/src/libcamera/meson.build @@ -107,6 +107,7 @@ libcamera = shared_library('camera', install : true, link_with : libcamera_link_with, include_directories : includes, + build_rpath : ipa_build_path, dependencies : libcamera_deps) libcamera_dep = declare_dependency(sources : [libcamera_api, libcamera_ipa_api, libcamera_h], diff --git a/src/meson.build b/src/meson.build index 5adcd61fd913..42e1eb984ccd 100644 --- a/src/meson.build +++ b/src/meson.build @@ -2,6 +2,13 @@ if get_option('android') subdir('android') endif +# Provide libcamera with a reference to the current build for IPA +# loading. This allows IPAs to be successfully loaded when tests and +# applications are run from the build tree, but is stripped during the +# install phase, ensuring we maintain reproducible builds. + +ipa_build_path = join_paths(meson.current_build_dir(), 'ipa') + subdir('libcamera') subdir('ipa') subdir('cam') From patchwork Wed Feb 5 13:04:20 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kieran Bingham X-Patchwork-Id: 2775 Return-Path: Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 286AB60912 for ; Wed, 5 Feb 2020 14:04:27 +0100 (CET) Received: from localhost.localdomain (cpc89242-aztw30-2-0-cust488.18-1.cable.virginm.net [86.31.129.233]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id C408E52F; Wed, 5 Feb 2020 14:04:26 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1580907866; bh=ykY3ha/bXqaN4XovXqcM2qgrqKpyC2pVKsKKe/340qM=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=hBLgY3an+Zc4n8LVbXBDVa0cb7CCUin9Z3QwprHeeFquD6Hlurjhh6oytfruW/AJ/ FqJUgrwh7kNJPjwOCNVdMDa1VC+km9sudnKqhTCskv6wclLZc5seFhrjUnOKqEm60c D1e1oPsW/S22z1zmYkA9nUNOxMxInJPWltr34VXg= From: Kieran Bingham To: LibCamera Devel Date: Wed, 5 Feb 2020 13:04:20 +0000 Message-Id: <20200205130420.8736-6-kieran.bingham@ideasonboard.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20200205130420.8736-1-kieran.bingham@ideasonboard.com> References: <20200205130420.8736-1-kieran.bingham@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH 5/5] tests: Remove IPA_MODULE_PATH environment variable X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 05 Feb 2020 13:04:27 -0000 The tests declare a hard-coded LIBCAMERA_IPA_MODULE_PATH to allow tests to run from the test-suite. This requires tests to be run only from the root of the build directory, otherwise (for example, by running in their local directory) they will not be able to correctly locate the IPA modules. Now that the build path for the IPA manager is provided through the RUNPATH, we can remove the redundant setting of the LIBCAMERA_IPA_MODULE_PATH for tests. Signed-off-by: Kieran Bingham Reviewed-by: Laurent Pinchart --- test/libtest/test.cpp | 4 ---- 1 file changed, 4 deletions(-) diff --git a/test/libtest/test.cpp b/test/libtest/test.cpp index 333d2160e276..6cd3fbe6df06 100644 --- a/test/libtest/test.cpp +++ b/test/libtest/test.cpp @@ -21,10 +21,6 @@ int Test::execute() { int ret; - ret = setenv("LIBCAMERA_IPA_MODULE_PATH", "src/ipa", 1); - if (ret) - return errno; - ret = setenv("LIBCAMERA_IPA_PROXY_PATH", "src/libcamera/proxy/worker", 1); if (ret) return errno;