From patchwork Thu Jun 26 09:59:39 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Paul Elder X-Patchwork-Id: 23662 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 6E2C8BDCBF for ; Thu, 26 Jun 2025 10:00:29 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 1DE3A68E01; Thu, 26 Jun 2025 12:00:29 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="EBbc2ZsJ"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 617BD68DF4 for ; Thu, 26 Jun 2025 12:00:21 +0200 (CEST) Received: from neptunite.hamster-moth.ts.net (unknown [IPv6:2404:7a81:160:2100:258b:9e43:6dff:c39d]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 7E35118B1; Thu, 26 Jun 2025 12:00:01 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1750932002; bh=IdCHyFP5H/VnkfK2asiXy7JZ5uG17lDxZbRHeHlRYG0=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=EBbc2ZsJhCCPSlUdfmJvxmwrJl8na/EtPhhIXGSltoVANemcyN4mPNGTPVE94Srgq TC4xdFJsLsTVCVp+iIJRd02S54tmBDN5X9eXNFon7i4Hy0v5dSizmH9OQsh5OkMPva G+6AmACS9FyTOw/LDF4sFGRijtIuYIx8ZlP/E2Rs= From: Paul Elder To: libcamera-devel@lists.libcamera.org Cc: Paul Elder , kieran.bingham@ideasonboard.com Subject: [RFC PATCH 4/7] libcamera: layer_manager: Add LayerManager implementation Date: Thu, 26 Jun 2025 18:59:39 +0900 Message-ID: <20250626095944.1746345-5-paul.elder@ideasonboard.com> X-Mailer: git-send-email 2.47.2 In-Reply-To: <20250626095944.1746345-1-paul.elder@ideasonboard.com> References: <20250626095944.1746345-1-paul.elder@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" We want to be able to implement layers in libcamera, which conceptually sit in between the Camera class and the application. This can be useful for implementing things that don't belong inside the Camera/IPA nor inside the application, such as intercepting and translation the AeEnable control, or implementing the Sync algorithm. To achieve this, first add a LayerManager implementation, which searches for and loads layers from shared object files, and orchestrates executing them. Actually calling into these functions from the Camera class will be added in the following patch. Signed-off-by: Paul Elder --- include/libcamera/internal/layer_manager.h | 74 +++++ include/libcamera/internal/meson.build | 1 + include/libcamera/layer.h | 56 ++++ include/libcamera/meson.build | 1 + src/layer/meson.build | 10 + src/libcamera/layer_manager.cpp | 314 +++++++++++++++++++++ src/libcamera/meson.build | 1 + src/meson.build | 1 + 8 files changed, 458 insertions(+) create mode 100644 include/libcamera/internal/layer_manager.h create mode 100644 include/libcamera/layer.h create mode 100644 src/layer/meson.build create mode 100644 src/libcamera/layer_manager.cpp diff --git a/include/libcamera/internal/layer_manager.h b/include/libcamera/internal/layer_manager.h new file mode 100644 index 000000000000..73ccad01bca0 --- /dev/null +++ b/include/libcamera/internal/layer_manager.h @@ -0,0 +1,74 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2025, Ideas On Board Oy + * + * Layer manager interface + */ + +#pragma once + +#include +#include +#include +#include + +#include +#include + +#include +#include +#include +#include +#include +#include + +namespace libcamera { + +LOG_DECLARE_CATEGORY(LayerManager) + +class LayerManager +{ +public: + LayerManager(); + ~LayerManager(); + + void bufferCompleted(Request *request, FrameBuffer *buffer); + void requestCompleted(Request *request); + void disconnected(); + + void acquire(); + void release(); + + const ControlInfoMap &controls(const ControlInfoMap &controlInfoMap); + const ControlList &properties(const ControlList &properties); + const std::set &streams(const std::set &streams); + + void generateConfiguration(Span &roles, + CameraConfiguration *config); + + void configure(CameraConfiguration *config); + + void createRequest(uint64_t cookie, Request *request); + + void queueRequest(Request *request); + + void start(const ControlList *controls); + void stop(); + +private: + /* Extend the layer with information specific to load-handling */ + struct LayerLoaded + { + Layer layer; + void *dlHandle; + }; + + std::unique_ptr createLayer(const std::string &file); + std::deque> executionQueue_; + + ControlInfoMap controlInfoMap_; + ControlList properties_; + std::set streams_; +}; + +} /* namespace libcamera */ diff --git a/include/libcamera/internal/meson.build b/include/libcamera/internal/meson.build index 690f5c5ec9f6..20e6c295601f 100644 --- a/include/libcamera/internal/meson.build +++ b/include/libcamera/internal/meson.build @@ -29,6 +29,7 @@ libcamera_internal_headers = files([ 'ipa_proxy.h', 'ipc_pipe.h', 'ipc_unixsocket.h', + 'layer_manager.h', 'mapped_framebuffer.h', 'matrix.h', 'media_device.h', diff --git a/include/libcamera/layer.h b/include/libcamera/layer.h new file mode 100644 index 000000000000..8fa0ee7d24e6 --- /dev/null +++ b/include/libcamera/layer.h @@ -0,0 +1,56 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2025, Ideas On Board Oy + * + * Layer interface + */ + +#pragma once + +#include +#include + +#include + +namespace libcamera { + +class CameraConfiguration; +class ControlInfoMap; +class ControlList; +class FrameBuffer; +class Request; +class Stream; +enum class StreamRole; + +struct Layer +{ + const char *name; + int layerAPIVersion; + + void (*init)(const std::string &id); + + void (*bufferCompleted)(Request *, FrameBuffer *); + void (*requestCompleted)(Request *); + void (*disconnected)(); + + void (*acquire)(); + void (*release)(); + + ControlInfoMap::Map (*controls)(ControlInfoMap &); + ControlList (*properties)(ControlList &); + std::set (*streams)(std::set &); + + void (*generateConfiguration)(Span &, + CameraConfiguration *); + + void (*configure)(CameraConfiguration *); + + void (*createRequest)(uint64_t, Request *); + + void (*queueRequest)(Request *); + + void (*start)(const ControlList *); + void (*stop)(); +} __attribute__((packed)); + +} /* namespace libcamera */ diff --git a/include/libcamera/meson.build b/include/libcamera/meson.build index 30ea76f9470a..552af112abb5 100644 --- a/include/libcamera/meson.build +++ b/include/libcamera/meson.build @@ -11,6 +11,7 @@ libcamera_public_headers = files([ 'framebuffer.h', 'framebuffer_allocator.h', 'geometry.h', + 'layer.h', 'logging.h', 'orientation.h', 'pixel_format.h', diff --git a/src/layer/meson.build b/src/layer/meson.build new file mode 100644 index 000000000000..dee5e5ac5804 --- /dev/null +++ b/src/layer/meson.build @@ -0,0 +1,10 @@ +# SPDX-License-Identifier: CC0-1.0 + +layer_includes = [ + libcamera_includes, +] + +layer_install_dir = libcamera_libdir / 'layers' + +config_h.set('LAYER_DIR', + '"' + get_option('prefix') / layer_install_dir + '"') diff --git a/src/libcamera/layer_manager.cpp b/src/libcamera/layer_manager.cpp new file mode 100644 index 000000000000..96d53d4fc75d --- /dev/null +++ b/src/libcamera/layer_manager.cpp @@ -0,0 +1,314 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2025, Ideas On Board Oy + * + * Layer manager + */ + +#include "libcamera/internal/layer_manager.h" + +#include +#include +#include +#include +#include +#include +#include +#include +#include + +#include +#include +#include +#include + +#include +#include + +#include "libcamera/internal/utils.h" + +/** + * \file layer_manager.h + * \brief Layer manager + */ + +namespace libcamera { + +LOG_DEFINE_CATEGORY(LayerManager) + +/** + * \class LayerManager + * \brief Layer manager + * + * The Layer manager discovers layer implementations from disk, and creates + * execution queues for every function that is implemented by each layer and + * executes them. A layer is a layer that sits between libcamera and the + * application, and hooks into the public Camera interface. + */ + +/** + * \brief Construct a LayerManager instance + * + * The LayerManager class is meant be instantiated by the Camera. + */ +LayerManager::LayerManager() +{ + std::map> layers; + + /* This returns the number of "modules" successfully loaded */ + std::function addDirHandler = + [this, &layers](const std::string &file) { + std::unique_ptr layer = createLayer(file); + if (!layer) + return 0; + + LOG(LayerManager, Debug) << "Loaded layer '" << file << "'"; + + layers.insert({std::string(layer->layer.name), std::move(layer)}); + + return 1; + }; + + /* User-specified paths take precedence. */ + const char *layerPaths = utils::secure_getenv("LIBCAMERA_LAYER_PATH"); + if (layerPaths) { + for (const auto &dir : utils::split(layerPaths, ":")) { + if (dir.empty()) + continue; + + utils::addDir(dir.c_str(), 0, addDirHandler); + } + } + + /* + * When libcamera is used before it is installed, load layers from the + * same build directory as the libcamera library itself. + */ + std::string root = utils::libcameraBuildPath(); + if (!root.empty()) { + std::string layerBuildPath = root + "src/layer"; + constexpr int maxDepth = 2; + + LOG(LayerManager, Info) + << "libcamera is not installed. Adding '" + << layerBuildPath << "' to the layer search path"; + + utils::addDir(layerBuildPath.c_str(), maxDepth, addDirHandler); + } + + /* Finally try to load layers from the installed system path. */ + utils::addDir(LAYER_DIR, 0, addDirHandler); + + /* Order the layers */ + /* \todo Document this. First is closer to application, last is closer to libcamera */ + const char *layerList = utils::secure_getenv("LIBCAMERA_LAYERS_ENABLE"); + if (layerList) { + for (const auto &layerName : utils::split(layerList, ":")) { + if (layerName.empty()) + continue; + + const auto &it = layers.find(layerName); + if (it == layers.end()) + continue; + + executionQueue_.push_back(std::move(it->second)); + } + } +} + +LayerManager::~LayerManager() +{ + for (auto &layer : executionQueue_) + dlclose(layer->dlHandle); +} + +std::unique_ptr LayerManager::createLayer(const std::string &filename) +{ + File file{ filename }; + if (!file.open(File::OpenModeFlag::ReadOnly)) { + LOG(LayerManager, Error) << "Failed to open layer: " + << strerror(-file.error()); + return nullptr; + } + + Span data = file.map(); + int ret = utils::elfVerifyIdent(data); + if (ret) { + LOG(LayerManager, Error) << "Layer is not an ELF file"; + return nullptr; + } + + Span info = utils::elfLoadSymbol(data, "layerInfo"); + if (info.size() < sizeof(Layer)) { + LOG(LayerManager, Error) << "Layer has no valid info"; + return nullptr; + } + + void *dlHandle = dlopen(file.fileName().c_str(), RTLD_LAZY); + if (!dlHandle) { + LOG(LayerManager, Error) + << "Failed to open layer shared object: " + << dlerror(); + return nullptr; + } + + void *symbol = dlsym(dlHandle, "layerInfo"); + if (!symbol) { + LOG(LayerManager, Error) + << "Failed to load layerInfo from layer shared object: " + << dlerror(); + dlclose(dlHandle); + dlHandle = nullptr; + return nullptr; + } + + std::unique_ptr layer = + std::make_unique(); + layer->layer = *reinterpret_cast(symbol); + + /* \todo Implement this. It should come from the libcamera version */ + if (layer->layer.layerAPIVersion != 1) { + LOG(LayerManager, Error) << "Layer API version mismatch"; + return nullptr; + } + + /* \todo Validate the layer name. */ + + layer->dlHandle = dlHandle; + + return layer; +} + +void LayerManager::bufferCompleted(Request *request, FrameBuffer *buffer) +{ + /* Reverse order because this comes from a Signal emission */ + for (auto it = executionQueue_.rbegin(); + it != executionQueue_.rend(); it++) { + if ((*it)->layer.bufferCompleted) + (*it)->layer.bufferCompleted(request, buffer); + } +} + +void LayerManager::requestCompleted(Request *request) +{ + /* Reverse order because this comes from a Signal emission */ + for (auto it = executionQueue_.rbegin(); + it != executionQueue_.rend(); it++) { + if ((*it)->layer.requestCompleted) + (*it)->layer.requestCompleted(request); + } +} + +void LayerManager::disconnected() +{ + /* Reverse order because this comes from a Signal emission */ + for (auto it = executionQueue_.rbegin(); + it != executionQueue_.rend(); it++) { + if ((*it)->layer.disconnected) + (*it)->layer.disconnected(); + } +} + +void LayerManager::acquire() +{ + for (std::unique_ptr &layer : executionQueue_) + if (layer->layer.acquire) + layer->layer.acquire(); +} + +void LayerManager::release() +{ + for (std::unique_ptr &layer : executionQueue_) + if (layer->layer.release) + layer->layer.release(); +} + +const ControlInfoMap &LayerManager::controls(const ControlInfoMap &controlInfoMap) +{ + controlInfoMap_ = controlInfoMap; + + /* \todo Simplify this once ControlInfoMaps become easier to modify */ + for (std::unique_ptr &layer : executionQueue_) { + if (layer->layer.controls) { + ControlInfoMap::Map ret = layer->layer.controls(controlInfoMap_); + ControlInfoMap::Map map; + /* Merge the layer's ret later so that layers can overwrite */ + for (auto &pair : controlInfoMap_) + map.insert(pair); + for (auto &pair : ret) + map.insert(pair); + controlInfoMap_ = ControlInfoMap(std::move(map), + libcamera::controls::controls); + } + } + return controlInfoMap_; +} + +const ControlList &LayerManager::properties(const ControlList &properties) +{ + properties_ = properties; + for (std::unique_ptr &layer : executionQueue_) { + if (layer->layer.properties) { + ControlList ret = layer->layer.properties(properties_); + properties_.merge(ret, ControlList::MergePolicy::OverwriteExisting); + } + } + return properties_; +} + +const std::set &LayerManager::streams(const std::set &streams) +{ + streams_ = streams; + for (std::unique_ptr &layer : executionQueue_) { + if (layer->layer.streams) { + std::set ret = layer->layer.streams(streams_); + streams_.insert(ret.begin(), ret.end()); + } + } + return streams_; +} + +void LayerManager::generateConfiguration(Span &roles, + CameraConfiguration *config) +{ + for (std::unique_ptr &layer : executionQueue_) + if (layer->layer.generateConfiguration) + layer->layer.generateConfiguration(roles, config); +} + +void LayerManager::configure(CameraConfiguration *config) +{ + for (std::unique_ptr &layer : executionQueue_) + if (layer->layer.configure) + layer->layer.configure(config); +} + +void LayerManager::createRequest(uint64_t cookie, Request *request) +{ + for (std::unique_ptr &layer : executionQueue_) + if (layer->layer.createRequest) + layer->layer.createRequest(cookie, request); +} + +void LayerManager::queueRequest(Request *request) +{ + for (std::unique_ptr &layer : executionQueue_) + if (layer->layer.queueRequest) + layer->layer.queueRequest(request); +} + +void LayerManager::start(const ControlList *controls) +{ + for (std::unique_ptr &layer : executionQueue_) + if (layer->layer.start) + layer->layer.start(controls); +} + +void LayerManager::stop() +{ + for (std::unique_ptr &layer : executionQueue_) + if (layer->layer.stop) + layer->layer.stop(); +} + +} /* namespace libcamera */ diff --git a/src/libcamera/meson.build b/src/libcamera/meson.build index 8e2aa921a620..226a94768514 100644 --- a/src/libcamera/meson.build +++ b/src/libcamera/meson.build @@ -40,6 +40,7 @@ libcamera_internal_sources = files([ 'ipc_pipe.cpp', 'ipc_pipe_unixsocket.cpp', 'ipc_unixsocket.cpp', + 'layer_manager.cpp', 'mapped_framebuffer.cpp', 'matrix.cpp', 'media_device.cpp', diff --git a/src/meson.build b/src/meson.build index 8eb8f05b362f..37368b01cbf2 100644 --- a/src/meson.build +++ b/src/meson.build @@ -63,6 +63,7 @@ subdir('libcamera') subdir('android') subdir('ipa') +subdir('layer') subdir('apps')