{"id":23662,"url":"https://patchwork.libcamera.org/api/1.1/patches/23662/?format=json","web_url":"https://patchwork.libcamera.org/patch/23662/","project":{"id":1,"url":"https://patchwork.libcamera.org/api/1.1/projects/1/?format=json","name":"libcamera","link_name":"libcamera","list_id":"libcamera_core","list_email":"libcamera-devel@lists.libcamera.org","web_url":"","scm_url":"","webscm_url":""},"msgid":"<20250626095944.1746345-5-paul.elder@ideasonboard.com>","date":"2025-06-26T09:59:39","name":"[RFC,4/7] libcamera: layer_manager: Add LayerManager implementation","commit_ref":null,"pull_url":null,"state":"superseded","archived":false,"hash":"80fac2586200153d608bf1c980cac45b1b2d992c","submitter":{"id":17,"url":"https://patchwork.libcamera.org/api/1.1/people/17/?format=json","name":"Paul Elder","email":"paul.elder@ideasonboard.com"},"delegate":null,"mbox":"https://patchwork.libcamera.org/patch/23662/mbox/","series":[{"id":5251,"url":"https://patchwork.libcamera.org/api/1.1/series/5251/?format=json","web_url":"https://patchwork.libcamera.org/project/libcamera/list/?series=5251","date":"2025-06-26T09:59:35","name":"Add Layers support","version":1,"mbox":"https://patchwork.libcamera.org/series/5251/mbox/"}],"comments":"https://patchwork.libcamera.org/api/patches/23662/comments/","check":"pending","checks":"https://patchwork.libcamera.org/api/patches/23662/checks/","tags":{},"headers":{"Return-Path":"","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id 6E2C8BDCBF\n\tfor ;\n\tThu, 26 Jun 2025 10:00:29 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id 1DE3A68E01;\n\tThu, 26 Jun 2025 12:00:29 +0200 (CEST)","from perceval.ideasonboard.com (perceval.ideasonboard.com\n\t[IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id 617BD68DF4\n\tfor ;\n\tThu, 26 Jun 2025 12:00:21 +0200 (CEST)","from neptunite.hamster-moth.ts.net (unknown\n\t[IPv6:2404:7a81:160:2100:258b:9e43:6dff:c39d])\n\tby perceval.ideasonboard.com (Postfix) with ESMTPSA id 7E35118B1;\n\tThu, 26 Jun 2025 12:00:01 +0200 (CEST)"],"Authentication-Results":"lancelot.ideasonboard.com; dkim=pass (1024-bit key;\n\tunprotected) header.d=ideasonboard.com header.i=@ideasonboard.com\n\theader.b=\"EBbc2ZsJ\"; dkim-atps=neutral","DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com;\n\ts=mail; t=1750932002;\n\tbh=IdCHyFP5H/VnkfK2asiXy7JZ5uG17lDxZbRHeHlRYG0=;\n\th=From:To:Cc:Subject:Date:In-Reply-To:References:From;\n\tb=EBbc2ZsJhCCPSlUdfmJvxmwrJl8na/EtPhhIXGSltoVANemcyN4mPNGTPVE94Srgq\n\tTC4xdFJsLsTVCVp+iIJRd02S54tmBDN5X9eXNFon7i4Hy0v5dSizmH9OQsh5OkMPva\n\tG+6AmACS9FyTOw/LDF4sFGRijtIuYIx8ZlP/E2Rs=","From":"Paul Elder ","To":"libcamera-devel@lists.libcamera.org","Cc":"Paul Elder , kieran.bingham@ideasonboard.com","Subject":"[RFC PATCH 4/7] libcamera: layer_manager: Add LayerManager\n\timplementation","Date":"Thu, 26 Jun 2025 18:59:39 +0900","Message-ID":"<20250626095944.1746345-5-paul.elder@ideasonboard.com>","X-Mailer":"git-send-email 2.47.2","In-Reply-To":"<20250626095944.1746345-1-paul.elder@ideasonboard.com>","References":"<20250626095944.1746345-1-paul.elder@ideasonboard.com>","MIME-Version":"1.0","Content-Transfer-Encoding":"8bit","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" "},"content":"We want to be able to implement layers in libcamera, which conceptually\nsit in between the Camera class and the application. This can be useful\nfor implementing things that don't belong inside the Camera/IPA nor inside\nthe application, such as intercepting and translation the AeEnable\ncontrol, or implementing the Sync algorithm.\n\nTo achieve this, first add a LayerManager implementation, which searches\nfor and loads layers from shared object files, and orchestrates\nexecuting them. Actually calling into these functions from the Camera\nclass will be added in the following patch.\n\nSigned-off-by: Paul Elder \n---\n include/libcamera/internal/layer_manager.h | 74 +++++\n include/libcamera/internal/meson.build | 1 +\n include/libcamera/layer.h | 56 ++++\n include/libcamera/meson.build | 1 +\n src/layer/meson.build | 10 +\n src/libcamera/layer_manager.cpp | 314 +++++++++++++++++++++\n src/libcamera/meson.build | 1 +\n src/meson.build | 1 +\n 8 files changed, 458 insertions(+)\n create mode 100644 include/libcamera/internal/layer_manager.h\n create mode 100644 include/libcamera/layer.h\n create mode 100644 src/layer/meson.build\n create mode 100644 src/libcamera/layer_manager.cpp","diff":"diff --git a/include/libcamera/internal/layer_manager.h b/include/libcamera/internal/layer_manager.h\nnew file mode 100644\nindex 000000000000..73ccad01bca0\n--- /dev/null\n+++ b/include/libcamera/internal/layer_manager.h\n@@ -0,0 +1,74 @@\n+/* SPDX-License-Identifier: LGPL-2.1-or-later */\n+/*\n+ * Copyright (C) 2025, Ideas On Board Oy\n+ *\n+ * Layer manager interface\n+ */\n+\n+#pragma once\n+\n+#include \n+#include \n+#include \n+#include \n+\n+#include \n+#include \n+\n+#include \n+#include \n+#include \n+#include \n+#include \n+#include \n+\n+namespace libcamera {\n+\n+LOG_DECLARE_CATEGORY(LayerManager)\n+\n+class LayerManager\n+{\n+public:\n+\tLayerManager();\n+\t~LayerManager();\n+\n+\tvoid bufferCompleted(Request *request, FrameBuffer *buffer);\n+\tvoid requestCompleted(Request *request);\n+\tvoid disconnected();\n+\n+\tvoid acquire();\n+\tvoid release();\n+\n+\tconst ControlInfoMap &controls(const ControlInfoMap &controlInfoMap);\n+\tconst ControlList &properties(const ControlList &properties);\n+\tconst std::set &streams(const std::set &streams);\n+\n+\tvoid generateConfiguration(Span &roles,\n+\t\t\t\t CameraConfiguration *config);\n+\n+\tvoid configure(CameraConfiguration *config);\n+\n+\tvoid createRequest(uint64_t cookie, Request *request);\n+\n+\tvoid queueRequest(Request *request);\n+\n+\tvoid start(const ControlList *controls);\n+\tvoid stop();\n+\n+private:\n+\t/* Extend the layer with information specific to load-handling */\n+\tstruct LayerLoaded\n+\t{\n+\t\tLayer layer;\n+\t\tvoid *dlHandle;\n+\t};\n+\n+\tstd::unique_ptr createLayer(const std::string &file);\n+\tstd::deque> executionQueue_;\n+\n+\tControlInfoMap controlInfoMap_;\n+\tControlList properties_;\n+\tstd::set streams_;\n+};\n+\n+} /* namespace libcamera */\ndiff --git a/include/libcamera/internal/meson.build b/include/libcamera/internal/meson.build\nindex 690f5c5ec9f6..20e6c295601f 100644\n--- a/include/libcamera/internal/meson.build\n+++ b/include/libcamera/internal/meson.build\n@@ -29,6 +29,7 @@ libcamera_internal_headers = files([\n 'ipa_proxy.h',\n 'ipc_pipe.h',\n 'ipc_unixsocket.h',\n+ 'layer_manager.h',\n 'mapped_framebuffer.h',\n 'matrix.h',\n 'media_device.h',\ndiff --git a/include/libcamera/layer.h b/include/libcamera/layer.h\nnew file mode 100644\nindex 000000000000..8fa0ee7d24e6\n--- /dev/null\n+++ b/include/libcamera/layer.h\n@@ -0,0 +1,56 @@\n+/* SPDX-License-Identifier: LGPL-2.1-or-later */\n+/*\n+ * Copyright (C) 2025, Ideas On Board Oy\n+ *\n+ * Layer interface\n+ */\n+\n+#pragma once\n+\n+#include \n+#include \n+\n+#include \n+\n+namespace libcamera {\n+\n+class CameraConfiguration;\n+class ControlInfoMap;\n+class ControlList;\n+class FrameBuffer;\n+class Request;\n+class Stream;\n+enum class StreamRole;\n+\n+struct Layer\n+{\n+\tconst char *name;\n+\tint layerAPIVersion;\n+\n+\tvoid (*init)(const std::string &id);\n+\n+\tvoid (*bufferCompleted)(Request *, FrameBuffer *);\n+\tvoid (*requestCompleted)(Request *);\n+\tvoid (*disconnected)();\n+\n+\tvoid (*acquire)();\n+\tvoid (*release)();\n+\n+\tControlInfoMap::Map (*controls)(ControlInfoMap &);\n+\tControlList (*properties)(ControlList &);\n+\tstd::set (*streams)(std::set &);\n+\n+\tvoid (*generateConfiguration)(Span &,\n+\t\t\t\t CameraConfiguration *);\n+\n+\tvoid (*configure)(CameraConfiguration *);\n+\n+\tvoid (*createRequest)(uint64_t, Request *);\n+\n+\tvoid (*queueRequest)(Request *);\n+\n+\tvoid (*start)(const ControlList *);\n+\tvoid (*stop)();\n+} __attribute__((packed));\n+\n+} /* namespace libcamera */\ndiff --git a/include/libcamera/meson.build b/include/libcamera/meson.build\nindex 30ea76f9470a..552af112abb5 100644\n--- a/include/libcamera/meson.build\n+++ b/include/libcamera/meson.build\n@@ -11,6 +11,7 @@ libcamera_public_headers = files([\n 'framebuffer.h',\n 'framebuffer_allocator.h',\n 'geometry.h',\n+ 'layer.h',\n 'logging.h',\n 'orientation.h',\n 'pixel_format.h',\ndiff --git a/src/layer/meson.build b/src/layer/meson.build\nnew file mode 100644\nindex 000000000000..dee5e5ac5804\n--- /dev/null\n+++ b/src/layer/meson.build\n@@ -0,0 +1,10 @@\n+# SPDX-License-Identifier: CC0-1.0\n+\n+layer_includes = [\n+ libcamera_includes,\n+]\n+\n+layer_install_dir = libcamera_libdir / 'layers'\n+\n+config_h.set('LAYER_DIR',\n+ '\"' + get_option('prefix') / layer_install_dir + '\"')\ndiff --git a/src/libcamera/layer_manager.cpp b/src/libcamera/layer_manager.cpp\nnew file mode 100644\nindex 000000000000..96d53d4fc75d\n--- /dev/null\n+++ b/src/libcamera/layer_manager.cpp\n@@ -0,0 +1,314 @@\n+/* SPDX-License-Identifier: LGPL-2.1-or-later */\n+/*\n+ * Copyright (C) 2025, Ideas On Board Oy\n+ *\n+ * Layer manager\n+ */\n+\n+#include \"libcamera/internal/layer_manager.h\"\n+\n+#include \n+#include \n+#include \n+#include \n+#include \n+#include \n+#include \n+#include \n+#include \n+\n+#include \n+#include \n+#include \n+#include \n+\n+#include \n+#include \n+\n+#include \"libcamera/internal/utils.h\"\n+\n+/**\n+ * \\file layer_manager.h\n+ * \\brief Layer manager\n+ */\n+\n+namespace libcamera {\n+\n+LOG_DEFINE_CATEGORY(LayerManager)\n+\n+/**\n+ * \\class LayerManager\n+ * \\brief Layer manager\n+ *\n+ * The Layer manager discovers layer implementations from disk, and creates\n+ * execution queues for every function that is implemented by each layer and\n+ * executes them. A layer is a layer that sits between libcamera and the\n+ * application, and hooks into the public Camera interface.\n+ */\n+\n+/**\n+ * \\brief Construct a LayerManager instance\n+ *\n+ * The LayerManager class is meant be instantiated by the Camera.\n+ */\n+LayerManager::LayerManager()\n+{\n+\tstd::map> layers;\n+\n+\t/* This returns the number of \"modules\" successfully loaded */\n+\tstd::function addDirHandler =\n+\t[this, &layers](const std::string &file) {\n+\t\tstd::unique_ptr layer = createLayer(file);\n+\t\tif (!layer)\n+\t\t\treturn 0;\n+\n+\t\tLOG(LayerManager, Debug) << \"Loaded layer '\" << file << \"'\";\n+\n+\t\tlayers.insert({std::string(layer->layer.name), std::move(layer)});\n+\n+\t\treturn 1;\n+\t};\n+\n+\t/* User-specified paths take precedence. */\n+\tconst char *layerPaths = utils::secure_getenv(\"LIBCAMERA_LAYER_PATH\");\n+\tif (layerPaths) {\n+\t\tfor (const auto &dir : utils::split(layerPaths, \":\")) {\n+\t\t\tif (dir.empty())\n+\t\t\t\tcontinue;\n+\n+\t\t\tutils::addDir(dir.c_str(), 0, addDirHandler);\n+\t\t}\n+\t}\n+\n+\t/*\n+\t * When libcamera is used before it is installed, load layers from the\n+\t * same build directory as the libcamera library itself.\n+\t */\n+\tstd::string root = utils::libcameraBuildPath();\n+\tif (!root.empty()) {\n+\t\tstd::string layerBuildPath = root + \"src/layer\";\n+\t\tconstexpr int maxDepth = 2;\n+\n+\t\tLOG(LayerManager, Info)\n+\t\t\t<< \"libcamera is not installed. Adding '\"\n+\t\t\t<< layerBuildPath << \"' to the layer search path\";\n+\n+\t\tutils::addDir(layerBuildPath.c_str(), maxDepth, addDirHandler);\n+\t}\n+\n+\t/* Finally try to load layers from the installed system path. */\n+\tutils::addDir(LAYER_DIR, 0, addDirHandler);\n+\n+\t/* Order the layers */\n+\t/* \\todo Document this. First is closer to application, last is closer to libcamera */\n+\tconst char *layerList = utils::secure_getenv(\"LIBCAMERA_LAYERS_ENABLE\");\n+\tif (layerList) {\n+\t\tfor (const auto &layerName : utils::split(layerList, \":\")) {\n+\t\t\tif (layerName.empty())\n+\t\t\t\tcontinue;\n+\n+\t\t\tconst auto &it = layers.find(layerName);\n+\t\t\tif (it == layers.end())\n+\t\t\t\tcontinue;\n+\n+\t\t\texecutionQueue_.push_back(std::move(it->second));\n+\t\t}\n+\t}\n+}\n+\n+LayerManager::~LayerManager()\n+{\n+\tfor (auto &layer : executionQueue_)\n+\t\tdlclose(layer->dlHandle);\n+}\n+\n+std::unique_ptr LayerManager::createLayer(const std::string &filename)\n+{\n+\tFile file{ filename };\n+\tif (!file.open(File::OpenModeFlag::ReadOnly)) {\n+\t\tLOG(LayerManager, Error) << \"Failed to open layer: \"\n+\t\t\t\t\t << strerror(-file.error());\n+\t\treturn nullptr;\n+\t}\n+\n+\tSpan data = file.map();\n+\tint ret = utils::elfVerifyIdent(data);\n+\tif (ret) {\n+\t\tLOG(LayerManager, Error) << \"Layer is not an ELF file\";\n+\t\treturn nullptr;\n+\t}\n+\n+\tSpan info = utils::elfLoadSymbol(data, \"layerInfo\");\n+\tif (info.size() < sizeof(Layer)) {\n+\t\tLOG(LayerManager, Error) << \"Layer has no valid info\";\n+\t\treturn nullptr;\n+\t}\n+\n+\tvoid *dlHandle = dlopen(file.fileName().c_str(), RTLD_LAZY);\n+\tif (!dlHandle) {\n+\t\tLOG(LayerManager, Error)\n+\t\t\t<< \"Failed to open layer shared object: \"\n+\t\t\t<< dlerror();\n+\t\treturn nullptr;\n+\t}\n+\n+\tvoid *symbol = dlsym(dlHandle, \"layerInfo\");\n+\tif (!symbol) {\n+\t\tLOG(LayerManager, Error)\n+\t\t\t<< \"Failed to load layerInfo from layer shared object: \"\n+\t\t\t<< dlerror();\n+\t\tdlclose(dlHandle);\n+\t\tdlHandle = nullptr;\n+\t\treturn nullptr;\n+\t}\n+\n+\tstd::unique_ptr layer =\n+\t\tstd::make_unique();\n+\tlayer->layer = *reinterpret_cast(symbol);\n+\n+\t/* \\todo Implement this. It should come from the libcamera version */\n+\tif (layer->layer.layerAPIVersion != 1) {\n+\t\tLOG(LayerManager, Error) << \"Layer API version mismatch\";\n+\t\treturn nullptr;\n+\t}\n+\n+\t/* \\todo Validate the layer name. */\n+\n+\tlayer->dlHandle = dlHandle;\n+\n+\treturn layer;\n+}\n+\n+void LayerManager::bufferCompleted(Request *request, FrameBuffer *buffer)\n+{\n+\t/* Reverse order because this comes from a Signal emission */\n+\tfor (auto it = executionQueue_.rbegin();\n+\t it != executionQueue_.rend(); it++) {\n+\t\tif ((*it)->layer.bufferCompleted)\n+\t\t\t(*it)->layer.bufferCompleted(request, buffer);\n+\t}\n+}\n+\n+void LayerManager::requestCompleted(Request *request)\n+{\n+\t/* Reverse order because this comes from a Signal emission */\n+\tfor (auto it = executionQueue_.rbegin();\n+\t it != executionQueue_.rend(); it++) {\n+\t\tif ((*it)->layer.requestCompleted)\n+\t\t\t(*it)->layer.requestCompleted(request);\n+\t}\n+}\n+\n+void LayerManager::disconnected()\n+{\n+\t/* Reverse order because this comes from a Signal emission */\n+\tfor (auto it = executionQueue_.rbegin();\n+\t it != executionQueue_.rend(); it++) {\n+\t\tif ((*it)->layer.disconnected)\n+\t\t\t(*it)->layer.disconnected();\n+\t}\n+}\n+\n+void LayerManager::acquire()\n+{\n+\tfor (std::unique_ptr &layer : executionQueue_)\n+\t\tif (layer->layer.acquire)\n+\t\t\tlayer->layer.acquire();\n+}\n+\n+void LayerManager::release()\n+{\n+\tfor (std::unique_ptr &layer : executionQueue_)\n+\t\tif (layer->layer.release)\n+\t\t\tlayer->layer.release();\n+}\n+\n+const ControlInfoMap &LayerManager::controls(const ControlInfoMap &controlInfoMap)\n+{\n+\tcontrolInfoMap_ = controlInfoMap;\n+\n+\t/* \\todo Simplify this once ControlInfoMaps become easier to modify */\n+\tfor (std::unique_ptr &layer : executionQueue_) {\n+\t\tif (layer->layer.controls) {\n+\t\t\tControlInfoMap::Map ret = layer->layer.controls(controlInfoMap_);\n+\t\t\tControlInfoMap::Map map;\n+\t\t\t/* Merge the layer's ret later so that layers can overwrite */\n+\t\t\tfor (auto &pair : controlInfoMap_)\n+\t\t\t\tmap.insert(pair);\n+\t\t\tfor (auto &pair : ret)\n+\t\t\t\tmap.insert(pair);\n+\t\t\tcontrolInfoMap_ = ControlInfoMap(std::move(map),\n+\t\t\t\t\t\t\t libcamera::controls::controls);\n+\t\t}\n+\t}\n+\treturn controlInfoMap_;\n+}\n+\n+const ControlList &LayerManager::properties(const ControlList &properties)\n+{\n+\tproperties_ = properties;\n+\tfor (std::unique_ptr &layer : executionQueue_) {\n+\t\tif (layer->layer.properties) {\n+\t\t\tControlList ret = layer->layer.properties(properties_);\n+\t\t\tproperties_.merge(ret, ControlList::MergePolicy::OverwriteExisting);\n+\t\t}\n+\t}\n+\treturn properties_;\n+}\n+\n+const std::set &LayerManager::streams(const std::set &streams)\n+{\n+\tstreams_ = streams;\n+\tfor (std::unique_ptr &layer : executionQueue_) {\n+\t\tif (layer->layer.streams) {\n+\t\t\tstd::set ret = layer->layer.streams(streams_);\n+\t\t\tstreams_.insert(ret.begin(), ret.end());\n+\t\t}\n+\t}\n+\treturn streams_;\n+}\n+\n+void LayerManager::generateConfiguration(Span &roles,\n+\t\t\t\t\t CameraConfiguration *config)\n+{\n+\tfor (std::unique_ptr &layer : executionQueue_)\n+\t\tif (layer->layer.generateConfiguration)\n+\t\t\tlayer->layer.generateConfiguration(roles, config);\n+}\n+\n+void LayerManager::configure(CameraConfiguration *config)\n+{\n+\tfor (std::unique_ptr &layer : executionQueue_)\n+\t\tif (layer->layer.configure)\n+\t\t\tlayer->layer.configure(config);\n+}\n+\n+void LayerManager::createRequest(uint64_t cookie, Request *request)\n+{\n+\tfor (std::unique_ptr &layer : executionQueue_)\n+\t\tif (layer->layer.createRequest)\n+\t\t\tlayer->layer.createRequest(cookie, request);\n+}\n+\n+void LayerManager::queueRequest(Request *request)\n+{\n+\tfor (std::unique_ptr &layer : executionQueue_)\n+\t\tif (layer->layer.queueRequest)\n+\t\t\tlayer->layer.queueRequest(request);\n+}\n+\n+void LayerManager::start(const ControlList *controls)\n+{\n+\tfor (std::unique_ptr &layer : executionQueue_)\n+\t\tif (layer->layer.start)\n+\t\t\tlayer->layer.start(controls);\n+}\n+\n+void LayerManager::stop()\n+{\n+\tfor (std::unique_ptr &layer : executionQueue_)\n+\t\tif (layer->layer.stop)\n+\t\t\tlayer->layer.stop();\n+}\n+\n+} /* namespace libcamera */\ndiff --git a/src/libcamera/meson.build b/src/libcamera/meson.build\nindex 8e2aa921a620..226a94768514 100644\n--- a/src/libcamera/meson.build\n+++ b/src/libcamera/meson.build\n@@ -40,6 +40,7 @@ libcamera_internal_sources = files([\n 'ipc_pipe.cpp',\n 'ipc_pipe_unixsocket.cpp',\n 'ipc_unixsocket.cpp',\n+ 'layer_manager.cpp',\n 'mapped_framebuffer.cpp',\n 'matrix.cpp',\n 'media_device.cpp',\ndiff --git a/src/meson.build b/src/meson.build\nindex 8eb8f05b362f..37368b01cbf2 100644\n--- a/src/meson.build\n+++ b/src/meson.build\n@@ -63,6 +63,7 @@ subdir('libcamera')\n \n subdir('android')\n subdir('ipa')\n+subdir('layer')\n \n subdir('apps')\n \n","prefixes":["RFC","4/7"]}