From patchwork Tue Oct 8 23:13:11 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kieran Bingham X-Patchwork-Id: 21547 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id C8CC0C32DE for ; Tue, 8 Oct 2024 23:13:37 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 1F5E56536C; Wed, 9 Oct 2024 01:13:36 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="AH626oe7"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id E02476353A for ; Wed, 9 Oct 2024 01:13:22 +0200 (CEST) Received: from Monstersaurus.tail69b4.ts.net (cpc89244-aztw30-2-0-cust6594.18-1.cable.virginm.net [86.31.185.195]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 91F8282A; Wed, 9 Oct 2024 01:11:45 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1728429105; bh=RqzSxwqprrcsw057g2saUwu1P2u9vHUaa9Y5qzMfFNU=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=AH626oe77q90at7XaDG5a8MmAwsc4uiefiqEmog/yYzvb9YDWf12pz4JjF8rBETSf A55mCG9llkybLq2rPl/pGzBjRF7RqyOWjv3GBM2C7ih+8lT8RVPOiyr9Wfle87weed i9eAZNT8JbakZ1ZWL071MMT6AlgMY5a8y5JAMPQU= From: Kieran Bingham To: libcamera devel Cc: Kieran Bingham , Umang Jain , Jacopo Mondi , Daniel Scally Subject: [PATCH v3 1/4] libcamera: media_device: Add helper to return matching entities Date: Wed, 9 Oct 2024 00:13:11 +0100 Message-Id: <20241008231314.744556-2-kieran.bingham@ideasonboard.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20241008231314.744556-1-kieran.bingham@ideasonboard.com> References: <20241008231314.744556-1-kieran.bingham@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Provide a helper on the MediaDevice to return a list of all available entities which match a given function in the graph. As a drive by, also fix a whitespace error in the documentation of MediaDevice::setupLink. Reviewed-by: Umang Jain Reviewed-by: Jacopo Mondi Reviewed-by: Daniel Scally Signed-off-by: Kieran Bingham --- include/libcamera/internal/media_device.h | 2 ++ src/libcamera/media_device.cpp | 25 ++++++++++++++++++++++- 2 files changed, 26 insertions(+), 1 deletion(-) diff --git a/include/libcamera/internal/media_device.h b/include/libcamera/internal/media_device.h index e412d3a0b7e3..b3a48b98d64b 100644 --- a/include/libcamera/internal/media_device.h +++ b/include/libcamera/internal/media_device.h @@ -55,6 +55,8 @@ public: Signal<> disconnected; + std::vector locateEntities(unsigned int function); + protected: std::string logPrefix() const override; diff --git a/src/libcamera/media_device.cpp b/src/libcamera/media_device.cpp index d71dad74df70..65080a4afdca 100644 --- a/src/libcamera/media_device.cpp +++ b/src/libcamera/media_device.cpp @@ -793,7 +793,7 @@ void MediaDevice::fixupEntityFlags(struct media_v2_entity *entity) * low-level link setup as it performs no checks on the validity of the \a * flags, and assumes that the supplied \a flags are valid for the link (e.g. * immutable links cannot be disabled). -* + * * \sa MediaLink::setEnabled(bool enable) * * \return 0 on success or a negative error code otherwise @@ -828,4 +828,27 @@ int MediaDevice::setupLink(const MediaLink *link, unsigned int flags) return 0; } +/** + * \brief Identify all entities of a common function in the MediaDevice + * \param[in] function The entity function to search for + * + * Search all entities within the graph of the MediaDevice and return + * a vector of those which match the given function as defined by the + * MEDIA_ENT_F_* macros of the Media Controller API. + * + * \return A vector of matching entities + */ +std::vector MediaDevice::locateEntities(unsigned int function) +{ + std::vector found; + + /* Gather all the entities matching the function they expose. */ + for (MediaEntity *entity : entities()) { + if (entity->function() == function) + found.push_back(entity); + } + + return found; +} + } /* namespace libcamera */ From patchwork Tue Oct 8 23:13:12 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kieran Bingham X-Patchwork-Id: 21548 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id C1FF1C32DE for ; Tue, 8 Oct 2024 23:13:39 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 8F50465373; Wed, 9 Oct 2024 01:13:38 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="K//NS9qN"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 693906536A for ; Wed, 9 Oct 2024 01:13:23 +0200 (CEST) Received: from Monstersaurus.tail69b4.ts.net (cpc89244-aztw30-2-0-cust6594.18-1.cable.virginm.net [86.31.185.195]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id F0DA3D21; Wed, 9 Oct 2024 01:11:45 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1728429106; bh=b1bCuUEWIxg0c9z/dqJMBUBciQv78Kl+YURbzgCbQ/0=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=K//NS9qNQL1msQzV84Qk04W9OPKTnrhmn78cqD4nrzLoq5jR6k1KvfjPjJN6eC+pm R+VHXaaSLjrK1rgW3BZsG17FbRwzF6DbZscYJ8ZJKofkCsGGBKCKa3OJqE/aKh0s/p HFiFCamEOzDOq+qQ/qDLoiZnB0kMfcTzUBr99zi4= From: Kieran Bingham To: libcamera devel Cc: Kieran Bingham , Umang Jain , Jacopo Mondi Subject: [PATCH v3 2/4] libcamera: pipeline: rkisp1: Fix typo in todo task Date: Wed, 9 Oct 2024 00:13:12 +0100 Message-Id: <20241008231314.744556-3-kieran.bingham@ideasonboard.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20241008231314.744556-1-kieran.bingham@ideasonboard.com> References: <20241008231314.744556-1-kieran.bingham@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" The delay values should be managed correctly. Not the dealys. Correct accordingly. Reviewed-by: Umang Jain Reviewed-by: Jacopo Mondi Signed-off-by: Kieran Bingham --- src/libcamera/pipeline/rkisp1/rkisp1.cpp | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/libcamera/pipeline/rkisp1/rkisp1.cpp b/src/libcamera/pipeline/rkisp1/rkisp1.cpp index c02c7cf374e9..2fee84e56d4d 100644 --- a/src/libcamera/pipeline/rkisp1/rkisp1.cpp +++ b/src/libcamera/pipeline/rkisp1/rkisp1.cpp @@ -1103,7 +1103,7 @@ int PipelineHandlerRkISP1::createCamera(MediaEntity *sensor) data->properties_ = data->sensor_->properties(); /* - * \todo Read dealy values from the sensor itself or from a + * \todo Read delay values from the sensor itself or from a * a sensor database. For now use generic values taken from * the Raspberry Pi and listed as generic values. */ From patchwork Tue Oct 8 23:13:13 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kieran Bingham X-Patchwork-Id: 21549 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 35B9CC32DF for ; Tue, 8 Oct 2024 23:13:41 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id B0FDD6536F; Wed, 9 Oct 2024 01:13:39 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="B6nD/cwA"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 7ADC46536B for ; Wed, 9 Oct 2024 01:13:23 +0200 (CEST) Received: from Monstersaurus.tail69b4.ts.net (cpc89244-aztw30-2-0-cust6594.18-1.cable.virginm.net [86.31.185.195]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 559C24D4; Wed, 9 Oct 2024 01:11:46 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1728429106; bh=ws/WBB+UwyJShsgbTSLSIYDZv2rTxDZ+cd2b+o9J0P0=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=B6nD/cwAYMjvzmQ9N2NXNTDRwFpZ/GHQ002URnZYwmYFANzx05NdHAf1QgfM+0Cvk EljmecaI44mNCS4QJGKrcipHA1XXdC6N/1Dq3kjjlun9tZkgQV5bwDY2FDNQBAaOe7 noJ4+Qs/ttksZsJeVE1kXsmwv6vr6MJsHXs57Np4= From: Kieran Bingham To: libcamera devel Cc: Kieran Bingham , Umang Jain Subject: [PATCH v3 3/4] libcamera: internal: Add MediaPipeline helper Date: Wed, 9 Oct 2024 00:13:13 +0100 Message-Id: <20241008231314.744556-4-kieran.bingham@ideasonboard.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20241008231314.744556-1-kieran.bingham@ideasonboard.com> References: <20241008231314.744556-1-kieran.bingham@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Provide a MediaPipeline class to help identifing and managing pipelines across a MediaDevice graph. Reviewed-by: Umang Jain Signed-off-by: Kieran Bingham Reviewed-by: Jacopo Mondi --- v2: - use srcPads to clearly identify which pads are managed - Only report enabling links when a change is made - fix header includes - Fix includes - Remove period at end of briefs - Document function parameters - expand documentation throughout - Fix debug log capitalisation - reduce scope of single use 'ret' v3: - Add doxygen documentation for the format parameter of configure() - Add 'format' identifier to configure parameter include/libcamera/internal/media_pipeline.h | 59 ++++ include/libcamera/internal/meson.build | 1 + src/libcamera/media_pipeline.cpp | 311 ++++++++++++++++++++ src/libcamera/meson.build | 1 + 4 files changed, 372 insertions(+) create mode 100644 include/libcamera/internal/media_pipeline.h create mode 100644 src/libcamera/media_pipeline.cpp diff --git a/include/libcamera/internal/media_pipeline.h b/include/libcamera/internal/media_pipeline.h new file mode 100644 index 000000000000..ee773b892719 --- /dev/null +++ b/include/libcamera/internal/media_pipeline.h @@ -0,0 +1,59 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2024, Ideas on Board Oy + * + * Media pipeline handler + */ + +#pragma once + +#include +#include +#include + +namespace libcamera { + +class CameraSensor; +class MediaEntity; +class MediaLink; +class MediaPad; +struct V4L2SubdeviceFormat; + +class MediaPipeline +{ +public: + int init(MediaEntity *source, std::string sink); + int initLinks(); + int configure(CameraSensor *sensor, V4L2SubdeviceFormat *format); + +private: + struct Entity { + /* The media entity, always valid. */ + MediaEntity *entity; + /* + * Whether or not the entity is a subdev that supports the + * routing API. + */ + bool supportsRouting; + /* + * The local sink pad connected to the upstream entity, null for + * the camera sensor at the beginning of the pipeline. + */ + const MediaPad *sink; + /* + * The local source pad connected to the downstream entity, null + * for the video node at the end of the pipeline. + */ + const MediaPad *source; + /* + * The link on the source pad, to the downstream entity, null + * for the video node at the end of the pipeline. + */ + MediaLink *sourceLink; + }; + + std::vector routedSourcePads(MediaPad *sink); + std::list entities_; +}; + +} /* namespace libcamera */ diff --git a/include/libcamera/internal/meson.build b/include/libcamera/internal/meson.build index 1c5eef9cab80..60a35d3e0357 100644 --- a/include/libcamera/internal/meson.build +++ b/include/libcamera/internal/meson.build @@ -30,6 +30,7 @@ libcamera_internal_headers = files([ 'mapped_framebuffer.h', 'media_device.h', 'media_object.h', + 'media_pipeline.h', 'pipeline_handler.h', 'process.h', 'pub_key.h', diff --git a/src/libcamera/media_pipeline.cpp b/src/libcamera/media_pipeline.cpp new file mode 100644 index 000000000000..ec78b78e2f75 --- /dev/null +++ b/src/libcamera/media_pipeline.cpp @@ -0,0 +1,311 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2024, Ideas on Board Oy + * + * Media pipeline support + */ + +#include "libcamera/internal/media_pipeline.h" + +#include +#include +#include +#include +#include +#include +#include + +#include + +#include + +#include "libcamera/internal/camera_sensor.h" +#include "libcamera/internal/media_device.h" +#include "libcamera/internal/media_object.h" +#include "libcamera/internal/v4l2_subdevice.h" + +/** + * \file media_pipeline.h + * \brief Provide a representation of a pipeline of devices using the Media + * Controller + */ + +namespace libcamera { + +LOG_DEFINE_CATEGORY(MediaPipeline) + +/** + * \class MediaPipeline + * \brief The MediaPipeline represents a set of entities that together form a + * data path for stream data + * + * A MediaPipeline instance is constructed from a sink and a source between + * two entities in a media graph. + */ + +/** + * \brief Retrieve all source pads connected to a sink pad through active routes + * \param[in] sink The sink pad to examine + * + * Examine the entity using the V4L2 Subdevice Routing API to collect all the + * source pads which are connected with an active route to the sink pad. + * + * \return A vector of source MediaPads + */ +std::vector MediaPipeline::routedSourcePads(MediaPad *sink) +{ + MediaEntity *entity = sink->entity(); + std::unique_ptr subdev = + std::make_unique(entity); + + int ret = subdev->open(); + if (ret < 0) + return {}; + + V4L2Subdevice::Routing routing = {}; + ret = subdev->getRouting(&routing, V4L2Subdevice::ActiveFormat); + if (ret < 0) + return {}; + + std::vector pads; + + for (const V4L2Subdevice::Route &route : routing) { + if (sink->index() != route.sink.pad || + !(route.flags & V4L2_SUBDEV_ROUTE_FL_ACTIVE)) + continue; + + const MediaPad *pad = entity->getPadByIndex(route.source.pad); + if (!pad) { + LOG(MediaPipeline, Warning) + << "Entity " << entity->name() + << " has invalid route source pad " + << route.source.pad; + } + + pads.push_back(pad); + } + + return pads; +} + +/** + * \brief Find the path from source to sink + * \param[in] source The source entity to start from + * \param[in] sink The sink entity name to search for + * + * Starting from a source entity, determine the shortest path to the target + * described by sink. + * + * If sink can not be found, or a route from source to sink can not be achieved, + * an error of -ENOLINK will be returned. + * + * When successful, the MediaPipeline will internally store the representation + * of entities and links to describe the path between the two entities. + * + * It is expected that the Source entity is a sensor represented by the + * CameraSensor class. + * + * \return 0 on success, a negative errno otherwise + */ +int MediaPipeline::init(MediaEntity *source, std::string sink) +{ + /* + * Find the shortest path between from the source and the + * target entity. + */ + std::unordered_set visited; + std::queue> queue; + + /* Remember at each entity where we came from. */ + std::unordered_map parents; + MediaEntity *entity = nullptr; + MediaEntity *target = nullptr; + MediaPad *sinkPad; + + queue.push({ source, nullptr }); + + while (!queue.empty()) { + std::tie(entity, sinkPad) = queue.front(); + queue.pop(); + + /* Found the target device. */ + if (entity->name() == sink) { + LOG(MediaPipeline, Debug) + << "Found pipeline target " << entity->name(); + target = entity; + break; + } + + visited.insert(entity); + + /* + * Add direct downstream entities to the search queue. If the + * current entity supports the subdev internal routing API, + * restrict the search to downstream entities reachable through + * active routes. + */ + + std::vector srcPads; + bool supportsRouting = false; + + if (sinkPad) { + srcPads = routedSourcePads(sinkPad); + if (!srcPads.empty()) + supportsRouting = true; + } + + if (srcPads.empty()) { + for (const MediaPad *pad : entity->pads()) { + if (!(pad->flags() & MEDIA_PAD_FL_SOURCE)) + continue; + srcPads.push_back(pad); + } + } + + for (const MediaPad *srcPad : srcPads) { + for (MediaLink *link : srcPad->links()) { + MediaEntity *next = link->sink()->entity(); + if (visited.find(next) == visited.end()) { + queue.push({ next, link->sink() }); + + Entity e{ entity, supportsRouting, sinkPad, + srcPad, link }; + parents.insert({ next, e }); + } + } + } + } + + if (!target) { + LOG(MediaPipeline, Error) << "Failed to connect " << source->name(); + return -ENOLINK; + } + + /* + * With the parents, we can follow back our way from the capture device + * to the sensor. Store all the entities in the pipeline, from the + * camera sensor to the video node, in entities_. + */ + entities_.push_front({ entity, false, sinkPad, nullptr, nullptr }); + + for (auto it = parents.find(entity); it != parents.end(); + it = parents.find(entity)) { + const Entity &e = it->second; + entities_.push_front(e); + entity = e.entity; + } + + LOG(MediaPipeline, Info) + << "Found pipeline: " + << utils::join(entities_, " -> ", + [](const Entity &e) { + std::string s = "["; + if (e.sink) + s += std::to_string(e.sink->index()) + "|"; + s += e.entity->name(); + if (e.source) + s += "|" + std::to_string(e.source->index()); + s += "]"; + return s; + }); + + return 0; +} + +/** + * \brief Initialise and enable all links through the MediaPipeline + * \return 0 on success, or a negative errno otherwise + */ +int MediaPipeline::initLinks() +{ + MediaLink *sinkLink = nullptr; + for (Entity &e : entities_) { + /* Sensor entities have no connected sink. */ + if (!sinkLink) { + sinkLink = e.sourceLink; + continue; + } + + if (!(sinkLink->flags() & MEDIA_LNK_FL_ENABLED)) { + LOG(MediaPipeline, Debug) << "Enabling : " << *sinkLink; + + int ret = sinkLink->setEnabled(true); + if (ret < 0) + return ret; + } + + sinkLink = e.sourceLink; + } + + return 0; +} + +/** + * \brief Configure the entities of this MediaPipeline + * \param[in] sensor The configured CameraSensor to propogate + * \param[inout] format The format to propogate through the pipeline + * + * Propagate formats through each of the entities of the Pipeline, validating + * that each one was not adjusted by the driver from the desired format. + * + * The format is updated with the final accepted format of the last entity of + * the pipeline. + * + * \return 0 on success or a negative errno otherwise + */ +int MediaPipeline::configure(CameraSensor *sensor, V4L2SubdeviceFormat *format) +{ + int ret; + + for (const Entity &e : entities_) { + /* The sensor is configured through the CameraSensor */ + if (!e.sourceLink) + break; + + MediaLink *link = e.sourceLink; + MediaPad *source = link->source(); + MediaPad *sink = link->sink(); + + /* 'format' already contains the sensor configuration */ + if (source->entity() != sensor->entity()) { + /* todo: Add MediaDevice cache to reduce FD pressure */ + V4L2Subdevice subdev(source->entity()); + ret = subdev.open(); + if (ret) + return ret; + + ret = subdev.getFormat(source->index(), format); + if (ret < 0) + return ret; + } + + V4L2SubdeviceFormat sourceFormat = *format; + /* todo: Add MediaDevice cache to reduce FD pressure */ + V4L2Subdevice subdev(sink->entity()); + ret = subdev.open(); + if (ret) + return ret; + + ret = subdev.setFormat(sink->index(), format); + if (ret < 0) + return ret; + + if (format->code != sourceFormat.code || + format->size != sourceFormat.size) { + LOG(MediaPipeline, Debug) + << "Source '" << *source + << " produces " << sourceFormat + << ", sink '" << *sink + << " requires " << format; + return -EINVAL; + } + + LOG(MediaPipeline, Debug) + << "Link " << *link << " configured with format " + << format; + } + + return 0; +} + +} /* namespace libcamera */ diff --git a/src/libcamera/meson.build b/src/libcamera/meson.build index aa9ab0291854..2c0f8603b231 100644 --- a/src/libcamera/meson.build +++ b/src/libcamera/meson.build @@ -41,6 +41,7 @@ libcamera_internal_sources = files([ 'mapped_framebuffer.cpp', 'media_device.cpp', 'media_object.cpp', + 'media_pipeline.cpp', 'pipeline_handler.cpp', 'process.cpp', 'pub_key.cpp', From patchwork Tue Oct 8 23:13:14 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kieran Bingham X-Patchwork-Id: 21550 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 9C96CC32DE for ; Tue, 8 Oct 2024 23:13:42 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id DEBFC65371; Wed, 9 Oct 2024 01:13:41 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="Nkj0HWYi"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id A294E6536C for ; Wed, 9 Oct 2024 01:13:23 +0200 (CEST) Received: from Monstersaurus.tail69b4.ts.net (cpc89244-aztw30-2-0-cust6594.18-1.cable.virginm.net [86.31.185.195]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id A694D10C4; Wed, 9 Oct 2024 01:11:46 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1728429106; bh=Pm8Iknascaa0ri/DTSI9S8pVjcDXS4L5qWWLsr37y8E=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=Nkj0HWYi7qUp3BTLlPB4QkPHflH67Nfy4qTc1D4ttFPNA/cnkEg43dXqHZnrtHb4v e9jr10TFKTfoc5YVuGBa3tgVDIjHwEOABtzine+beDODG5bV2BPa5qlORMg3liZvcU tyniaVJKHVvp75ckvNqG+PCCJuPgD2eE+jGOo/6k= From: Kieran Bingham To: libcamera devel Cc: Kieran Bingham , Umang Jain Subject: [PATCH v3 4/4] libcamera: pipeline: rkisp1: Convert to use MediaPipeline Date: Wed, 9 Oct 2024 00:13:14 +0100 Message-Id: <20241008231314.744556-5-kieran.bingham@ideasonboard.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20241008231314.744556-1-kieran.bingham@ideasonboard.com> References: <20241008231314.744556-1-kieran.bingham@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Use the new MediaPipeline to manage and identify all sensors connected to complex pipelines that can connect to the CSI2 receiver before the ISP. This can include chained multiplexers that supply multiple cameras, so make use of the MediaDevice::locateEntities to search for all cameras and construct a pipeline for each. Reviewed-by: Umang Jain Signed-off-by: Kieran Bingham Reviewed-by: Daniel Scally Reviewed-by: Jacopo Mondi --- src/libcamera/pipeline/rkisp1/rkisp1.cpp | 86 +++++++++--------------- 1 file changed, 32 insertions(+), 54 deletions(-) diff --git a/src/libcamera/pipeline/rkisp1/rkisp1.cpp b/src/libcamera/pipeline/rkisp1/rkisp1.cpp index 2fee84e56d4d..e94f047ba70c 100644 --- a/src/libcamera/pipeline/rkisp1/rkisp1.cpp +++ b/src/libcamera/pipeline/rkisp1/rkisp1.cpp @@ -37,6 +37,7 @@ #include "libcamera/internal/framebuffer.h" #include "libcamera/internal/ipa_manager.h" #include "libcamera/internal/media_device.h" +#include "libcamera/internal/media_pipeline.h" #include "libcamera/internal/pipeline_handler.h" #include "libcamera/internal/v4l2_subdevice.h" #include "libcamera/internal/v4l2_videodevice.h" @@ -108,6 +109,11 @@ public: std::unique_ptr ipa_; + /* + * All entities in the pipeline, from the camera sensor to the RkISP1. + */ + MediaPipeline pipe_; + private: void paramFilled(unsigned int frame, unsigned int bytesused); void setSensorControls(unsigned int frame, @@ -171,8 +177,7 @@ private: friend RkISP1CameraData; friend RkISP1Frames; - int initLinks(Camera *camera, const CameraSensor *sensor, - const RkISP1CameraConfiguration &config); + int initLinks(Camera *camera, const RkISP1CameraConfiguration &config); int createCamera(MediaEntity *sensor); void tryCompleteRequest(RkISP1FrameInfo *info); void bufferReady(FrameBuffer *buffer); @@ -187,7 +192,6 @@ private: std::unique_ptr isp_; std::unique_ptr param_; std::unique_ptr stat_; - std::unique_ptr csi_; bool hasSelfPath_; bool isRaw_; @@ -201,8 +205,6 @@ private: std::queue availableStatBuffers_; Camera *activeCamera_; - - const MediaPad *ispSink_; }; RkISP1Frames::RkISP1Frames(PipelineHandler *pipe) @@ -712,7 +714,7 @@ int PipelineHandlerRkISP1::configure(Camera *camera, CameraConfiguration *c) CameraSensor *sensor = data->sensor_.get(); int ret; - ret = initLinks(camera, sensor, *config); + ret = initLinks(camera, *config); if (ret) return ret; @@ -729,12 +731,12 @@ int PipelineHandlerRkISP1::configure(Camera *camera, CameraConfiguration *c) LOG(RkISP1, Debug) << "Sensor configured with " << format; - if (csi_) { - ret = csi_->setFormat(0, &format); - if (ret < 0) - return ret; - } + /* Propagate format through the internal media pipeline up to the ISP */ + ret = data->pipe_.configure(sensor, &format); + if (ret < 0) + return ret; + LOG(RkISP1, Debug) << "Configuring ISP with : " << format; ret = isp_->setFormat(0, &format); if (ret < 0) return ret; @@ -1035,7 +1037,6 @@ int PipelineHandlerRkISP1::queueRequestDevice(Camera *camera, Request *request) */ int PipelineHandlerRkISP1::initLinks(Camera *camera, - const CameraSensor *sensor, const RkISP1CameraConfiguration &config) { RkISP1CameraData *data = cameraData(camera); @@ -1046,31 +1047,16 @@ int PipelineHandlerRkISP1::initLinks(Camera *camera, return ret; /* - * Configure the sensor links: enable the link corresponding to this - * camera. + * Configure the sensor links: enable the links corresponding to this + * pipeline all the way up to the ISP, through any connected CSI receiver. */ - for (MediaLink *link : ispSink_->links()) { - if (link->source()->entity() != sensor->entity()) - continue; - - LOG(RkISP1, Debug) - << "Enabling link from sensor '" - << link->source()->entity()->name() - << "' to ISP"; - - ret = link->setEnabled(true); - if (ret < 0) - return ret; - } - - if (csi_) { - MediaLink *link = isp_->entity()->getPadByIndex(0)->links().at(0); - - ret = link->setEnabled(true); - if (ret < 0) - return ret; + ret = data->pipe_.initLinks(); + if (ret) { + LOG(RkISP1, Error) << "Failed to set up pipe links"; + return ret; } + /* Configure the paths after the ISP */ for (const StreamConfiguration &cfg : config) { if (cfg.stream() == &data->mainPathStream_) ret = data->mainPath_->setEnabled(true); @@ -1094,6 +1080,13 @@ int PipelineHandlerRkISP1::createCamera(MediaEntity *sensor) std::make_unique(this, &mainPath_, hasSelfPath_ ? &selfPath_ : nullptr); + /* Identify the pipeline path between the sensor and the rkisp1_isp */ + ret = data->pipe_.init(sensor, "rkisp1_isp"); + if (ret) { + LOG(RkISP1, Error) << "Failed to identify path from sensor to rkisp1_isp"; + return ret; + } + data->sensor_ = std::make_unique(sensor); ret = data->sensor_->init(); if (ret) @@ -1129,6 +1122,7 @@ int PipelineHandlerRkISP1::createCamera(MediaEntity *sensor) const std::string &id = data->sensor_->id(); std::shared_ptr camera = Camera::create(std::move(data), id, streams); + registerCamera(std::move(camera)); return 0; @@ -1136,8 +1130,6 @@ int PipelineHandlerRkISP1::createCamera(MediaEntity *sensor) bool PipelineHandlerRkISP1::match(DeviceEnumerator *enumerator) { - const MediaPad *pad; - DeviceMatch dm("rkisp1"); dm.add("rkisp1_isp"); dm.add("rkisp1_resizer_mainpath"); @@ -1162,22 +1154,6 @@ bool PipelineHandlerRkISP1::match(DeviceEnumerator *enumerator) if (isp_->open() < 0) return false; - /* Locate and open the optional CSI-2 receiver. */ - ispSink_ = isp_->entity()->getPadByIndex(0); - if (!ispSink_ || ispSink_->links().empty()) - return false; - - pad = ispSink_->links().at(0)->source(); - if (pad->entity()->function() == MEDIA_ENT_F_VID_IF_BRIDGE) { - csi_ = std::make_unique(pad->entity()); - if (csi_->open() < 0) - return false; - - ispSink_ = csi_->entity()->getPadByIndex(0); - if (!ispSink_) - return false; - } - /* Locate and open the stats and params video nodes. */ stat_ = V4L2VideoDevice::fromEntityName(media_, "rkisp1_stats"); if (stat_->open() < 0) @@ -1205,8 +1181,10 @@ bool PipelineHandlerRkISP1::match(DeviceEnumerator *enumerator) * camera instance for each of them. */ bool registered = false; - for (MediaLink *link : ispSink_->links()) { - if (!createCamera(link->source()->entity())) + + for (MediaEntity *entity : media_->locateEntities(MEDIA_ENT_F_CAM_SENSOR)) { + LOG(RkISP1, Info) << "Identified " << entity->name(); + if (!createCamera(entity)) registered = true; }