[{"id":33906,"web_url":"https://patchwork.libcamera.org/comment/33906/","msgid":"","date":"2025-04-03T10:28:49","subject":"Re: [PATCH v4 2/3] libcamera: internal: Add MediaPipeline helper","submitter":{"id":216,"url":"https://patchwork.libcamera.org/api/people/216/","name":"Barnabás Pőcze","email":"barnabas.pocze@ideasonboard.com"},"content":"Hi\n\n\n2025. 04. 02. 9:39 keltezéssel, Paul Elder írta:\n> From: Kieran Bingham \n> \n> Provide a MediaPipeline class to help identifing and managing pipelines across\n> a MediaDevice graph.\n> \n> Signed-off-by: Kieran Bingham \n> Reviewed-by: Umang Jain \n> Reviewed-by: Jacopo Mondi \n> Signed-off-by: Paul Elder \n> ---\n> include/libcamera/internal/media_pipeline.h | 60 ++++\n> include/libcamera/internal/meson.build | 1 +\n> src/libcamera/media_pipeline.cpp | 302 ++++++++++++++++++++\n> src/libcamera/meson.build | 1 +\n> 4 files changed, 364 insertions(+)\n> create mode 100644 include/libcamera/internal/media_pipeline.h\n> create mode 100644 src/libcamera/media_pipeline.cpp\n> \n> diff --git a/include/libcamera/internal/media_pipeline.h b/include/libcamera/internal/media_pipeline.h\n> new file mode 100644\n> index 000000000000..b5a224e7172f\n> --- /dev/null\n> +++ b/include/libcamera/internal/media_pipeline.h\n> @@ -0,0 +1,60 @@\n> +/* SPDX-License-Identifier: LGPL-2.1-or-later */\n> +/*\n> + * Copyright (C) 2024, Ideas on Board Oy\n> + *\n> + * Media pipeline support\n> + */\n> +\n> +#pragma once\n> +\n> +#include \n> +#include \n> +\n> +#include \n> +\n> +namespace libcamera {\n> +\n> +class CameraSensor;\n> +class MediaEntity;\n> +class MediaLink;\n> +class MediaPad;\n> +struct V4L2SubdeviceFormat;\n> +\n> +class MediaPipeline\n> +{\n> +public:\n> +\tint init(MediaEntity *source, std::string sink);\n\nOk, sorry, I know this is my pet peeve, but could `sink` be an `std::string_view`\nor at least `const std::string&`? Or am I missing why it is needed by value?\n\n\n> +\tint initLinks();\n> +\tint configure(CameraSensor *sensor, V4L2SubdeviceFormat *);\n> +\n> +private:\n> +\tstruct Entity {\n> +\t\t/* The media entity, always valid. */\n> +\t\tMediaEntity *entity;\n> +\t\t/*\n> +\t\t * Whether or not the entity is a subdev that supports the\n> +\t\t * routing API.\n> +\t\t */\n> +\t\tbool supportsRouting;\n> +\t\t/*\n> +\t\t * The local sink pad connected to the upstream entity, null for\n> +\t\t * the camera sensor at the beginning of the pipeline.\n> +\t\t */\n> +\t\tconst MediaPad *sink;\n> +\t\t/*\n> +\t\t * The local source pad connected to the downstream entity, null\n> +\t\t * for the video node at the end of the pipeline.\n> +\t\t */\n> +\t\tconst MediaPad *source;\n> +\t\t/*\n> +\t\t * The link on the source pad, to the downstream entity, null\n> +\t\t * for the video node at the end of the pipeline.\n> +\t\t */\n> +\t\tMediaLink *sourceLink;\n> +\t};\n> +\n> +\tstd::vector routedSourcePads(MediaPad *sink);\n\nCan this be `static`?\n\n\n> +\tstd::list entities_;\n\nWhy not a vector? Because of `push_front()`?\n\n\n> +};\n> +\n> +} /* namespace libcamera */\n> diff --git a/include/libcamera/internal/meson.build b/include/libcamera/internal/meson.build\n> index 45408b313848..33f318b2b602 100644\n> --- a/include/libcamera/internal/meson.build\n> +++ b/include/libcamera/internal/meson.build\n> @@ -32,6 +32,7 @@ libcamera_internal_headers = files([\n> 'matrix.h',\n> 'media_device.h',\n> 'media_object.h',\n> + 'media_pipeline.h',\n> 'pipeline_handler.h',\n> 'process.h',\n> 'pub_key.h',\n> diff --git a/src/libcamera/media_pipeline.cpp b/src/libcamera/media_pipeline.cpp\n> new file mode 100644\n> index 000000000000..c9a40e44a81d\n> --- /dev/null\n> +++ b/src/libcamera/media_pipeline.cpp\n> @@ -0,0 +1,302 @@\n> +/* SPDX-License-Identifier: LGPL-2.1-or-later */\n> +/*\n> + * Copyright (C) 2024, Ideas on Board Oy\n> + *\n> + * Media pipeline support\n> + */\n> +\n> +#include \"libcamera/internal/media_pipeline.h\"\n> +\n> +#include \n> +#include \n> +#include \n> +#include \n> +#include \n> +#include \n> +#include \n> +\n> +#include \n> +\n> +#include \n> +\n> +#include \"libcamera/internal/camera_sensor.h\"\n> +#include \"libcamera/internal/media_device.h\"\n> +#include \"libcamera/internal/media_object.h\"\n> +#include \"libcamera/internal/v4l2_subdevice.h\"\n> +\n> +/**\n> + * \\file media_pipeline.h\n> + * \\brief Provide a representation of a pipeline of devices using the Media\n> + * Controller.\n> + */\n> +\n> +namespace libcamera {\n> +\n> +LOG_DEFINE_CATEGORY(MediaPipeline)\n> +\n> +/**\n> + * \\class MediaPipeline\n> + * \\brief The MediaPipeline represents a set of entities that together form a\n> + * data path for stream data.\n> + *\n> + * A MediaPipeline instance is constructed from a sink and a source between\n> + * two entities in a media graph.\n> + */\n> +\n> +/**\n> + * \\brief Retrieve all source pads connected to a sink pad through active routes\n> + *\n> + * Examine the entity using the V4L2 Subdevice Routing API to collect all the\n> + * source pads which are connected with an active route to the sink pad.\n> + *\n> + * \\return A vector of source MediaPads\n> + */\n> +std::vector MediaPipeline::routedSourcePads(MediaPad *sink)\n> +{\n> +\tMediaEntity *entity = sink->entity();\n> +\tstd::unique_ptr subdev =\n> +\t\tstd::make_unique(entity);\n> +\n> +\tint ret = subdev->open();\n> +\tif (ret < 0)\n> +\t\treturn {};\n> +\n> +\tV4L2Subdevice::Routing routing = {};\n> +\tret = subdev->getRouting(&routing, V4L2Subdevice::ActiveFormat);\n> +\tif (ret < 0)\n> +\t\treturn {};\n> +\n> +\tstd::vector pads;\n> +\n> +\tfor (const V4L2Subdevice::Route &route : routing) {\n> +\t\tif (sink->index() != route.sink.pad ||\n> +\t\t !(route.flags & V4L2_SUBDEV_ROUTE_FL_ACTIVE))\n> +\t\t\tcontinue;\n> +\n> +\t\tconst MediaPad *pad = entity->getPadByIndex(route.source.pad);\n> +\t\tif (!pad) {\n> +\t\t\tLOG(MediaPipeline, Error)\n> +\t\t\t\t<< \"Entity \" << entity->name()\n> +\t\t\t\t<< \" has invalid route source pad \"\n> +\t\t\t\t<< route.source.pad;\n> +\t\t\treturn {};\n> +\t\t}\n> +\n> +\t\tpads.push_back(pad);\n> +\t}\n> +\n> +\treturn pads;\n> +}\n> +\n> +/**\n> + * \\brief Find the path from source to sink\n> + *\n> + * Starting from a source entity, determine the shortest path to the target\n> + * described by \\a sink.\n> + *\n> + * If \\a sink can not be found, or a route from source to sink can not be\n> + * achieved an error of -ENOLINK will be returned.\n> + *\n> + * When successful, the MediaPipeline will internally store the representation\n> + * of entities and links to describe the path between the two entities.\n> + *\n> + * \\return 0 on success, a negative errno otherwise\n> + */\n> +int MediaPipeline::init(MediaEntity *source, std::string sink)\n> +{\n> +\t/*\n> +\t * Find the shortest path between from the Camera Sensor and the\n> +\t * target entity.\n> +\t */\n> +\tstd::unordered_set visited;\n> +\tstd::queue> queue;\n\nI'd have used `std::pair` since that is usually simpler, but it doesn't really\nmatter, both work with `std::tie()`, etc.\n\n\n> +\n> +\t/* Remember at each entity where we came from. */\n> +\tstd::unordered_map parents;\n> +\tMediaEntity *entity = nullptr;\n> +\tMediaEntity *target = nullptr;\n> +\tMediaPad *sinkPad;\n> +\n> +\tqueue.push({ source, nullptr });\n> +\n> +\twhile (!queue.empty()) {\n> +\t\tstd::tie(entity, sinkPad) = queue.front();\n> +\t\tqueue.pop();\n> +\n> +\t\t/* Found the target device. */\n> +\t\tif (entity->name() == sink) {\n> +\t\t\tLOG(MediaPipeline, Debug)\n> +\t\t\t\t<< \"Found Pipeline target \" << entity->name();\n> +\t\t\ttarget = entity;\n> +\t\t\tbreak;\n> +\t\t}\n> +\n> +\t\tvisited.insert(entity);\n> +\n> +\t\t/*\n> +\t\t * Add direct downstream entities to the search queue. If the\n> +\t\t * current entity supports the subdev internal routing API,\n> +\t\t * restrict the search to downstream entities reachable through\n> +\t\t * active routes.\n> +\t\t */\n> +\n> +\t\tstd::vector pads;\n> +\t\tbool supportsRouting = false;\n> +\n> +\t\tif (sinkPad) {\n> +\t\t\tpads = routedSourcePads(sinkPad);\n> +\t\t\tif (!pads.empty())\n> +\t\t\t\tsupportsRouting = true;\n> +\t\t}\n> +\n> +\t\tif (pads.empty()) {\n> +\t\t\tfor (const MediaPad *pad : entity->pads()) {\n> +\t\t\t\tif (!(pad->flags() & MEDIA_PAD_FL_SOURCE))\n> +\t\t\t\t\tcontinue;\n> +\t\t\t\tpads.push_back(pad);\n> +\t\t\t}\n> +\t\t}\n> +\n> +\t\tfor (const MediaPad *pad : pads) {\n> +\t\t\tfor (MediaLink *link : pad->links()) {\n> +\t\t\t\tMediaEntity *next = link->sink()->entity();\n> +\t\t\t\tif (visited.find(next) == visited.end()) {\n\nAs far as I can see an entity is only marked `visited` when it is popped from the queue.\nSo it seems to me that an entity can be added to the queue multiple times.\nIs that not an issue? Or are there restrictions on the topology?\n\n\n> +\t\t\t\t\tqueue.push({ next, link->sink() });\n> +\n> +\t\t\t\t\tEntity e{ entity, supportsRouting, sinkPad, pad, link };\n> +\t\t\t\t\tparents.insert({ next, e });\n> +\t\t\t\t}\n> +\t\t\t}\n> +\t\t}\n> +\t}\n> +\n> +\tif (!target) {\n> +\t\tLOG(MediaPipeline, Error) << \"Failed to connect \" << source->name();\n\nShould this message maybe include `sink` as well?\n\n\nRegards,\nBarnabás Pőcze\n\n> +\t\treturn -ENOLINK;\n> +\t}\n> +\n> +\t/*\n> +\t * With the parents, we can follow back our way from the capture device\n> +\t * to the sensor. Store all the entities in the pipeline, from the\n> +\t * camera sensor to the video node, in entities_.\n> +\t */\n> +\tentities_.push_front({ entity, false, sinkPad, nullptr, nullptr });\n> +\n> +\tfor (auto it = parents.find(entity); it != parents.end();\n> +\t it = parents.find(entity)) {\n> +\t\tconst Entity &e = it->second;\n> +\t\tentities_.push_front(e);\n> +\t\tentity = e.entity;\n> +\t}\n> +\n> +\tLOG(MediaPipeline, Info)\n> +\t\t<< \"Found pipeline: \"\n> +\t\t<< utils::join(entities_, \" -> \",\n> +\t\t\t [](const Entity &e) {\n> +\t\t\t\t std::string s = \"[\";\n> +\t\t\t\t if (e.sink)\n> +\t\t\t\t\t s += std::to_string(e.sink->index()) + \"|\";\n> +\t\t\t\t s += e.entity->name();\n> +\t\t\t\t if (e.source)\n> +\t\t\t\t\t s += \"|\" + std::to_string(e.source->index());\n> +\t\t\t\t s += \"]\";\n> +\t\t\t\t return s;\n> +\t\t\t });\n> +\n> +\treturn 0;\n> +}\n> +\n> +/**\n> + * \\brief Initialise and enable all links through the MediaPipeline\n> + * \\return 0 on success, or a negative errno otherwise\n> + */\n> +int MediaPipeline::initLinks()\n> +{\n> +\tint ret = 0;\n> +\n> +\tMediaLink *sinkLink = nullptr;\n> +\tfor (Entity &e : entities_) {\n> +\t\t/* Sensor entities have no connected sink. */\n> +\t\tif (!sinkLink) {\n> +\t\t\tsinkLink = e.sourceLink;\n> +\t\t\tcontinue;\n> +\t\t}\n> +\n> +\t\tLOG(MediaPipeline, Debug) << \"Enabling : \" << *sinkLink;\n> +\n> +\t\tif (!(sinkLink->flags() & MEDIA_LNK_FL_ENABLED)) {\n> +\t\t\tret = sinkLink->setEnabled(true);\n> +\t\t\tif (ret < 0)\n> +\t\t\t\treturn ret;\n> +\t\t}\n> +\n> +\t\tsinkLink = e.sourceLink;\n> +\t}\n> +\n> +\treturn ret;\n> +}\n> +\n> +/**\n> + * \\brief Configure the entities of this MediaPipeline\n> + *\n> + * Propagate formats through each of the entities of the Pipeline, validating\n> + * that each one was not adjusted by the driver from the desired format.\n> + *\n> + * \\return 0 on success or a negative errno otherwise\n> + */\n> +int MediaPipeline::configure(CameraSensor *sensor, V4L2SubdeviceFormat *format)\n> +{\n> +\tint ret;\n> +\n> +\tfor (const Entity &e : entities_) {\n> +\t\t/* The sensor is configured through the CameraSensor */\n> +\t\tif (!e.sourceLink)\n> +\t\t\tbreak;\n> +\n> +\t\tMediaLink *link = e.sourceLink;\n> +\t\tMediaPad *source = link->source();\n> +\t\tMediaPad *sink = link->sink();\n> +\n> +\t\t/* 'format' already contains the sensor configuration */\n> +\t\tif (source->entity() != sensor->entity()) {\n> +\t\t\t/* \\todo Add MediaDevice cache to reduce FD pressure */\n> +\t\t\tV4L2Subdevice subdev(source->entity());\n> +\t\t\tret = subdev.open();\n> +\t\t\tif (ret)\n> +\t\t\t\treturn ret;\n> +\n> +\t\t\tret = subdev.getFormat(source->index(), format);\n> +\t\t\tif (ret < 0)\n> +\t\t\t\treturn ret;\n> +\t\t}\n> +\n> +\t\tV4L2SubdeviceFormat sourceFormat = *format;\n> +\t\t/* \\todo Add MediaDevice cache to reduce FD pressure */\n> +\t\tV4L2Subdevice subdev(sink->entity());\n> +\t\tret = subdev.open();\n> +\t\tif (ret)\n> +\t\t\treturn ret;\n> +\n> +\t\tret = subdev.setFormat(sink->index(), format);\n> +\t\tif (ret < 0)\n> +\t\t\treturn ret;\n> +\n> +\t\tif (format->code != sourceFormat.code ||\n> +\t\t format->size != sourceFormat.size) {\n> +\t\t\tLOG(MediaPipeline, Debug)\n> +\t\t\t\t<< \"Source '\" << *source\n> +\t\t\t\t<< \" produces \" << sourceFormat\n> +\t\t\t\t<< \", sink '\" << *sink\n> +\t\t\t\t<< \" requires \" << format;\n> +\t\t\treturn -EINVAL;\n> +\t\t}\n> +\n> +\t\tLOG(MediaPipeline, Debug)\n> +\t\t\t<< \"Link \" << *link << \" configured with format \"\n> +\t\t\t<< format;\n> +\t}\n> +\n> +\treturn 0;\n> +}\n> +\n> +} /* namespace libcamera */\n> diff --git a/src/libcamera/meson.build b/src/libcamera/meson.build\n> index de22b8e60dde..800d408077be 100644\n> --- a/src/libcamera/meson.build\n> +++ b/src/libcamera/meson.build\n> @@ -43,6 +43,7 @@ libcamera_internal_sources = files([\n> 'matrix.cpp',\n> 'media_device.cpp',\n> 'media_object.cpp',\n> + 'media_pipeline.cpp',\n> 'pipeline_handler.cpp',\n> 'process.cpp',\n> 'pub_key.cpp',","headers":{"Return-Path":"","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id 49345C3213\n\tfor ;\n\tThu, 3 Apr 2025 10:28:56 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id 8107568994;\n\tThu, 3 Apr 2025 12:28:55 +0200 (CEST)","from perceval.ideasonboard.com (perceval.ideasonboard.com\n\t[213.167.242.64])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id E871A61867\n\tfor ;\n\tThu, 3 Apr 2025 12:28:53 +0200 (CEST)","from [192.168.33.20] (185.221.143.221.nat.pool.zt.hu\n\t[185.221.143.221])\n\tby perceval.ideasonboard.com (Postfix) with ESMTPSA id 11C5F105D;\n\tThu, 3 Apr 2025 12:27:00 +0200 (CEST)"],"Authentication-Results":"lancelot.ideasonboard.com; dkim=pass (1024-bit key;\n\tunprotected) header.d=ideasonboard.com header.i=@ideasonboard.com\n\theader.b=\"h01FZxpB\"; dkim-atps=neutral","DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com;\n\ts=mail; t=1743676020;\n\tbh=C38iNscp4g3ETpAZ6pCOOvK+3N9N6+IWsBhUQ/eqhWw=;\n\th=Date:Subject:To:Cc:References:From:In-Reply-To:From;\n\tb=h01FZxpB4Apt+GcUUzGGedl3AoQkdp/31NyeiNG0rsmxzsGSTlNm77TxrtPvisWj2\n\t9q/PBmP5tm3J4bxmaf1LBuQUFxyGUJZY/B1zDhEB7dWu1P6zZnXNuWdG/ZszN7xAaD\n\t6OxS+zQTN3N7aqffKUoLeSPMKMzu/MpdeO+moxfI=","Message-ID":"","Date":"Thu, 3 Apr 2025 12:28:49 +0200","MIME-Version":"1.0","User-Agent":"Mozilla Thunderbird","Subject":"Re: [PATCH v4 2/3] libcamera: internal: Add MediaPipeline helper","To":"Paul Elder ,\n\tlibcamera-devel@lists.libcamera.org","Cc":"Kieran Bingham ,\n\tUmang Jain ,\n\tJacopo Mondi ","References":"<20250402073919.183330-1-paul.elder@ideasonboard.com>\n\t<20250402073919.183330-3-paul.elder@ideasonboard.com>","From":"=?utf-8?q?Barnab=C3=A1s_P=C5=91cze?= ","Content-Language":"en-US, hu-HU","In-Reply-To":"<20250402073919.183330-3-paul.elder@ideasonboard.com>","Content-Type":"text/plain; charset=UTF-8; format=flowed","Content-Transfer-Encoding":"8bit","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" "}},{"id":33907,"web_url":"https://patchwork.libcamera.org/comment/33907/","msgid":"","date":"2025-04-03T10:29:51","subject":"Re: [PATCH v4 2/3] libcamera: internal: Add MediaPipeline helper","submitter":{"id":184,"url":"https://patchwork.libcamera.org/api/people/184/","name":"Stefan Klug","email":"stefan.klug@ideasonboard.com"},"content":"On Wed, Apr 02, 2025 at 04:39:17PM +0900, Paul Elder wrote:\n> From: Kieran Bingham \n> \n> Provide a MediaPipeline class to help identifing and managing pipelines across\n> a MediaDevice graph.\n> \n> Signed-off-by: Kieran Bingham \n> Reviewed-by: Umang Jain \n> Reviewed-by: Jacopo Mondi \n> Signed-off-by: Paul Elder \n\nI like the encapsulation this brings.\nReviewed-by: Stefan Klug \n\n\n> ---\n> include/libcamera/internal/media_pipeline.h | 60 ++++\n> include/libcamera/internal/meson.build | 1 +\n> src/libcamera/media_pipeline.cpp | 302 ++++++++++++++++++++\n> src/libcamera/meson.build | 1 +\n> 4 files changed, 364 insertions(+)\n> create mode 100644 include/libcamera/internal/media_pipeline.h\n> create mode 100644 src/libcamera/media_pipeline.cpp\n> \n> diff --git a/include/libcamera/internal/media_pipeline.h b/include/libcamera/internal/media_pipeline.h\n> new file mode 100644\n> index 000000000000..b5a224e7172f\n> --- /dev/null\n> +++ b/include/libcamera/internal/media_pipeline.h\n> @@ -0,0 +1,60 @@\n> +/* SPDX-License-Identifier: LGPL-2.1-or-later */\n> +/*\n> + * Copyright (C) 2024, Ideas on Board Oy\n> + *\n> + * Media pipeline support\n> + */\n> +\n> +#pragma once\n> +\n> +#include \n> +#include \n> +\n> +#include \n> +\n> +namespace libcamera {\n> +\n> +class CameraSensor;\n> +class MediaEntity;\n> +class MediaLink;\n> +class MediaPad;\n> +struct V4L2SubdeviceFormat;\n> +\n> +class MediaPipeline\n> +{\n> +public:\n> +\tint init(MediaEntity *source, std::string sink);\n> +\tint initLinks();\n> +\tint configure(CameraSensor *sensor, V4L2SubdeviceFormat *);\n> +\n> +private:\n> +\tstruct Entity {\n> +\t\t/* The media entity, always valid. */\n> +\t\tMediaEntity *entity;\n> +\t\t/*\n> +\t\t * Whether or not the entity is a subdev that supports the\n> +\t\t * routing API.\n> +\t\t */\n> +\t\tbool supportsRouting;\n> +\t\t/*\n> +\t\t * The local sink pad connected to the upstream entity, null for\n> +\t\t * the camera sensor at the beginning of the pipeline.\n> +\t\t */\n> +\t\tconst MediaPad *sink;\n> +\t\t/*\n> +\t\t * The local source pad connected to the downstream entity, null\n> +\t\t * for the video node at the end of the pipeline.\n> +\t\t */\n> +\t\tconst MediaPad *source;\n> +\t\t/*\n> +\t\t * The link on the source pad, to the downstream entity, null\n> +\t\t * for the video node at the end of the pipeline.\n> +\t\t */\n> +\t\tMediaLink *sourceLink;\n> +\t};\n> +\n> +\tstd::vector routedSourcePads(MediaPad *sink);\n> +\tstd::list entities_;\n> +};\n> +\n> +} /* namespace libcamera */\n> diff --git a/include/libcamera/internal/meson.build b/include/libcamera/internal/meson.build\n> index 45408b313848..33f318b2b602 100644\n> --- a/include/libcamera/internal/meson.build\n> +++ b/include/libcamera/internal/meson.build\n> @@ -32,6 +32,7 @@ libcamera_internal_headers = files([\n> 'matrix.h',\n> 'media_device.h',\n> 'media_object.h',\n> + 'media_pipeline.h',\n> 'pipeline_handler.h',\n> 'process.h',\n> 'pub_key.h',\n> diff --git a/src/libcamera/media_pipeline.cpp b/src/libcamera/media_pipeline.cpp\n> new file mode 100644\n> index 000000000000..c9a40e44a81d\n> --- /dev/null\n> +++ b/src/libcamera/media_pipeline.cpp\n> @@ -0,0 +1,302 @@\n> +/* SPDX-License-Identifier: LGPL-2.1-or-later */\n> +/*\n> + * Copyright (C) 2024, Ideas on Board Oy\n> + *\n> + * Media pipeline support\n> + */\n> +\n> +#include \"libcamera/internal/media_pipeline.h\"\n> +\n> +#include \n> +#include \n> +#include \n> +#include \n> +#include \n> +#include \n> +#include \n> +\n> +#include \n> +\n> +#include \n> +\n> +#include \"libcamera/internal/camera_sensor.h\"\n> +#include \"libcamera/internal/media_device.h\"\n> +#include \"libcamera/internal/media_object.h\"\n> +#include \"libcamera/internal/v4l2_subdevice.h\"\n> +\n> +/**\n> + * \\file media_pipeline.h\n> + * \\brief Provide a representation of a pipeline of devices using the Media\n> + * Controller.\n> + */\n> +\n> +namespace libcamera {\n> +\n> +LOG_DEFINE_CATEGORY(MediaPipeline)\n> +\n> +/**\n> + * \\class MediaPipeline\n> + * \\brief The MediaPipeline represents a set of entities that together form a\n> + * data path for stream data.\n> + *\n> + * A MediaPipeline instance is constructed from a sink and a source between\n> + * two entities in a media graph.\n> + */\n> +\n> +/**\n> + * \\brief Retrieve all source pads connected to a sink pad through active routes\n> + *\n> + * Examine the entity using the V4L2 Subdevice Routing API to collect all the\n> + * source pads which are connected with an active route to the sink pad.\n> + *\n> + * \\return A vector of source MediaPads\n> + */\n> +std::vector MediaPipeline::routedSourcePads(MediaPad *sink)\n> +{\n> +\tMediaEntity *entity = sink->entity();\n> +\tstd::unique_ptr subdev =\n> +\t\tstd::make_unique(entity);\n> +\n> +\tint ret = subdev->open();\n> +\tif (ret < 0)\n> +\t\treturn {};\n> +\n> +\tV4L2Subdevice::Routing routing = {};\n> +\tret = subdev->getRouting(&routing, V4L2Subdevice::ActiveFormat);\n> +\tif (ret < 0)\n> +\t\treturn {};\n> +\n> +\tstd::vector pads;\n> +\n> +\tfor (const V4L2Subdevice::Route &route : routing) {\n> +\t\tif (sink->index() != route.sink.pad ||\n> +\t\t !(route.flags & V4L2_SUBDEV_ROUTE_FL_ACTIVE))\n> +\t\t\tcontinue;\n> +\n> +\t\tconst MediaPad *pad = entity->getPadByIndex(route.source.pad);\n> +\t\tif (!pad) {\n> +\t\t\tLOG(MediaPipeline, Error)\n> +\t\t\t\t<< \"Entity \" << entity->name()\n> +\t\t\t\t<< \" has invalid route source pad \"\n> +\t\t\t\t<< route.source.pad;\n> +\t\t\treturn {};\n> +\t\t}\n> +\n> +\t\tpads.push_back(pad);\n> +\t}\n> +\n> +\treturn pads;\n> +}\n> +\n> +/**\n> + * \\brief Find the path from source to sink\n> + *\n> + * Starting from a source entity, determine the shortest path to the target\n> + * described by \\a sink.\n> + *\n> + * If \\a sink can not be found, or a route from source to sink can not be\n> + * achieved an error of -ENOLINK will be returned.\n> + *\n> + * When successful, the MediaPipeline will internally store the representation\n> + * of entities and links to describe the path between the two entities.\n> + *\n> + * \\return 0 on success, a negative errno otherwise\n> + */\n> +int MediaPipeline::init(MediaEntity *source, std::string sink)\n> +{\n> +\t/*\n> +\t * Find the shortest path between from the Camera Sensor and the\n> +\t * target entity.\n> +\t */\n> +\tstd::unordered_set visited;\n> +\tstd::queue> queue;\n> +\n> +\t/* Remember at each entity where we came from. */\n> +\tstd::unordered_map parents;\n> +\tMediaEntity *entity = nullptr;\n> +\tMediaEntity *target = nullptr;\n> +\tMediaPad *sinkPad;\n> +\n> +\tqueue.push({ source, nullptr });\n> +\n> +\twhile (!queue.empty()) {\n> +\t\tstd::tie(entity, sinkPad) = queue.front();\n> +\t\tqueue.pop();\n> +\n> +\t\t/* Found the target device. */\n> +\t\tif (entity->name() == sink) {\n> +\t\t\tLOG(MediaPipeline, Debug)\n> +\t\t\t\t<< \"Found Pipeline target \" << entity->name();\n> +\t\t\ttarget = entity;\n> +\t\t\tbreak;\n> +\t\t}\n> +\n> +\t\tvisited.insert(entity);\n> +\n> +\t\t/*\n> +\t\t * Add direct downstream entities to the search queue. If the\n> +\t\t * current entity supports the subdev internal routing API,\n> +\t\t * restrict the search to downstream entities reachable through\n> +\t\t * active routes.\n> +\t\t */\n> +\n> +\t\tstd::vector pads;\n> +\t\tbool supportsRouting = false;\n> +\n> +\t\tif (sinkPad) {\n> +\t\t\tpads = routedSourcePads(sinkPad);\n> +\t\t\tif (!pads.empty())\n> +\t\t\t\tsupportsRouting = true;\n> +\t\t}\n> +\n> +\t\tif (pads.empty()) {\n> +\t\t\tfor (const MediaPad *pad : entity->pads()) {\n> +\t\t\t\tif (!(pad->flags() & MEDIA_PAD_FL_SOURCE))\n> +\t\t\t\t\tcontinue;\n> +\t\t\t\tpads.push_back(pad);\n> +\t\t\t}\n> +\t\t}\n> +\n> +\t\tfor (const MediaPad *pad : pads) {\n> +\t\t\tfor (MediaLink *link : pad->links()) {\n> +\t\t\t\tMediaEntity *next = link->sink()->entity();\n> +\t\t\t\tif (visited.find(next) == visited.end()) {\n> +\t\t\t\t\tqueue.push({ next, link->sink() });\n> +\n> +\t\t\t\t\tEntity e{ entity, supportsRouting, sinkPad, pad, link };\n> +\t\t\t\t\tparents.insert({ next, e });\n> +\t\t\t\t}\n> +\t\t\t}\n> +\t\t}\n> +\t}\n> +\n> +\tif (!target) {\n> +\t\tLOG(MediaPipeline, Error) << \"Failed to connect \" << source->name();\n> +\t\treturn -ENOLINK;\n> +\t}\n> +\n> +\t/*\n> +\t * With the parents, we can follow back our way from the capture device\n> +\t * to the sensor. Store all the entities in the pipeline, from the\n> +\t * camera sensor to the video node, in entities_.\n> +\t */\n> +\tentities_.push_front({ entity, false, sinkPad, nullptr, nullptr });\n> +\n> +\tfor (auto it = parents.find(entity); it != parents.end();\n> +\t it = parents.find(entity)) {\n> +\t\tconst Entity &e = it->second;\n> +\t\tentities_.push_front(e);\n> +\t\tentity = e.entity;\n> +\t}\n> +\n> +\tLOG(MediaPipeline, Info)\n> +\t\t<< \"Found pipeline: \"\n> +\t\t<< utils::join(entities_, \" -> \",\n> +\t\t\t [](const Entity &e) {\n> +\t\t\t\t std::string s = \"[\";\n> +\t\t\t\t if (e.sink)\n> +\t\t\t\t\t s += std::to_string(e.sink->index()) + \"|\";\n> +\t\t\t\t s += e.entity->name();\n> +\t\t\t\t if (e.source)\n> +\t\t\t\t\t s += \"|\" + std::to_string(e.source->index());\n> +\t\t\t\t s += \"]\";\n> +\t\t\t\t return s;\n> +\t\t\t });\n> +\n> +\treturn 0;\n> +}\n> +\n> +/**\n> + * \\brief Initialise and enable all links through the MediaPipeline\n> + * \\return 0 on success, or a negative errno otherwise\n> + */\n> +int MediaPipeline::initLinks()\n> +{\n> +\tint ret = 0;\n> +\n> +\tMediaLink *sinkLink = nullptr;\n> +\tfor (Entity &e : entities_) {\n> +\t\t/* Sensor entities have no connected sink. */\n> +\t\tif (!sinkLink) {\n> +\t\t\tsinkLink = e.sourceLink;\n> +\t\t\tcontinue;\n> +\t\t}\n> +\n> +\t\tLOG(MediaPipeline, Debug) << \"Enabling : \" << *sinkLink;\n> +\n> +\t\tif (!(sinkLink->flags() & MEDIA_LNK_FL_ENABLED)) {\n> +\t\t\tret = sinkLink->setEnabled(true);\n> +\t\t\tif (ret < 0)\n> +\t\t\t\treturn ret;\n> +\t\t}\n> +\n> +\t\tsinkLink = e.sourceLink;\n> +\t}\n> +\n> +\treturn ret;\n> +}\n> +\n> +/**\n> + * \\brief Configure the entities of this MediaPipeline\n> + *\n> + * Propagate formats through each of the entities of the Pipeline, validating\n> + * that each one was not adjusted by the driver from the desired format.\n> + *\n> + * \\return 0 on success or a negative errno otherwise\n> + */\n> +int MediaPipeline::configure(CameraSensor *sensor, V4L2SubdeviceFormat *format)\n> +{\n> +\tint ret;\n> +\n> +\tfor (const Entity &e : entities_) {\n> +\t\t/* The sensor is configured through the CameraSensor */\n> +\t\tif (!e.sourceLink)\n> +\t\t\tbreak;\n> +\n> +\t\tMediaLink *link = e.sourceLink;\n> +\t\tMediaPad *source = link->source();\n> +\t\tMediaPad *sink = link->sink();\n> +\n> +\t\t/* 'format' already contains the sensor configuration */\n> +\t\tif (source->entity() != sensor->entity()) {\n> +\t\t\t/* \\todo Add MediaDevice cache to reduce FD pressure */\n> +\t\t\tV4L2Subdevice subdev(source->entity());\n> +\t\t\tret = subdev.open();\n> +\t\t\tif (ret)\n> +\t\t\t\treturn ret;\n> +\n> +\t\t\tret = subdev.getFormat(source->index(), format);\n> +\t\t\tif (ret < 0)\n> +\t\t\t\treturn ret;\n> +\t\t}\n> +\n> +\t\tV4L2SubdeviceFormat sourceFormat = *format;\n> +\t\t/* \\todo Add MediaDevice cache to reduce FD pressure */\n> +\t\tV4L2Subdevice subdev(sink->entity());\n> +\t\tret = subdev.open();\n> +\t\tif (ret)\n> +\t\t\treturn ret;\n> +\n> +\t\tret = subdev.setFormat(sink->index(), format);\n> +\t\tif (ret < 0)\n> +\t\t\treturn ret;\n> +\n> +\t\tif (format->code != sourceFormat.code ||\n> +\t\t format->size != sourceFormat.size) {\n> +\t\t\tLOG(MediaPipeline, Debug)\n> +\t\t\t\t<< \"Source '\" << *source\n> +\t\t\t\t<< \" produces \" << sourceFormat\n> +\t\t\t\t<< \", sink '\" << *sink\n> +\t\t\t\t<< \" requires \" << format;\n> +\t\t\treturn -EINVAL;\n> +\t\t}\n> +\n> +\t\tLOG(MediaPipeline, Debug)\n> +\t\t\t<< \"Link \" << *link << \" configured with format \"\n> +\t\t\t<< format;\n> +\t}\n> +\n> +\treturn 0;\n> +}\n> +\n> +} /* namespace libcamera */\n> diff --git a/src/libcamera/meson.build b/src/libcamera/meson.build\n> index de22b8e60dde..800d408077be 100644\n> --- a/src/libcamera/meson.build\n> +++ b/src/libcamera/meson.build\n> @@ -43,6 +43,7 @@ libcamera_internal_sources = files([\n> 'matrix.cpp',\n> 'media_device.cpp',\n> 'media_object.cpp',\n> + 'media_pipeline.cpp',\n> 'pipeline_handler.cpp',\n> 'process.cpp',\n> 'pub_key.cpp',\n> -- \n> 2.47.2\n>","headers":{"Return-Path":"","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id 037E5C3213\n\tfor ;\n\tThu, 3 Apr 2025 10:29:57 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id B091C68980;\n\tThu, 3 Apr 2025 12:29:56 +0200 (CEST)","from perceval.ideasonboard.com (perceval.ideasonboard.com\n\t[213.167.242.64])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id B938D61867\n\tfor ;\n\tThu, 3 Apr 2025 12:29:54 +0200 (CEST)","from ideasonboard.com (unknown\n\t[IPv6:2a00:6020:448c:6c00:6d9d:9854:3fc1:4bb2])\n\tby perceval.ideasonboard.com (Postfix) with ESMTPSA id 44B7D105D;\n\tThu, 3 Apr 2025 12:28:01 +0200 (CEST)"],"Authentication-Results":"lancelot.ideasonboard.com; dkim=pass (1024-bit key;\n\tunprotected) header.d=ideasonboard.com header.i=@ideasonboard.com\n\theader.b=\"dlg9ZAER\"; dkim-atps=neutral","DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com;\n\ts=mail; t=1743676081;\n\tbh=vfKPnKwXwnAn+JGfM91xwtShd/w2+kEiFFv4xFRMnwI=;\n\th=Date:From:To:Cc:Subject:References:In-Reply-To:From;\n\tb=dlg9ZAERlCkMey/7ow7KQXZeWr/mKOFOq117Xy/yWaQ/D6IoiXLNWs22ZlDK5Vw5h\n\tVtkC6bJa8R6doOSfeBNgIfJm8EtfGJkotLznLOoVbv+/NLC7XuDfa07YA5K6Xg4H5I\n\to6TJNivII+uJC+3xWeqBAPFN6TMBv0XDmz3gKwoA=","Date":"Thu, 3 Apr 2025 12:29:51 +0200","From":"Stefan Klug ","To":"Paul Elder ","Cc":"libcamera-devel@lists.libcamera.org, \n\tKieran Bingham ,\n\tUmang Jain , \n\tJacopo Mondi ","Subject":"Re: [PATCH v4 2/3] libcamera: internal: Add MediaPipeline helper","Message-ID":"","References":"<20250402073919.183330-1-paul.elder@ideasonboard.com>\n\t<20250402073919.183330-3-paul.elder@ideasonboard.com>","MIME-Version":"1.0","Content-Type":"text/plain; charset=utf-8","Content-Disposition":"inline","In-Reply-To":"<20250402073919.183330-3-paul.elder@ideasonboard.com>","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" "}}]