From patchwork Mon Sep 16 14:02:38 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kieran Bingham X-Patchwork-Id: 21281 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id B44D7C3257 for ; Mon, 16 Sep 2024 14:02:54 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 93941634FF; Mon, 16 Sep 2024 16:02:51 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="qcBXRs1b"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id CC7A5634FC for ; Mon, 16 Sep 2024 16:02:48 +0200 (CEST) Received: from charm.lan (unknown [IPv6:2001:4bc9:a45:b0af:726e:a88f:9f07:34be]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id A325582A; Mon, 16 Sep 2024 16:01:27 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1726495287; bh=uiBSR0tOq4Y0tvXZp5jRAZLAjaMTGxsDzQBGB7knc9s=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=qcBXRs1bcdyUCByUrZopEXojRDCMcZ1xKFtBe58XCCRtEyy8sDqV9+X/DhsG36CUi k/dUGTI6bIblEbxFkJH4s8t6Q7IjO6B9ZVDla1k+SYlzv4aD9PCCsMIHpkyeCnakxr hFCejMd+uGU1uNw82HMLqrlKqM58563BzGx51x8g= From: Kieran Bingham To: libcamera devel Cc: Kieran Bingham Subject: [PATCH 1/4] libcamera: media_device: Add helper to return matching entities Date: Mon, 16 Sep 2024 16:02:38 +0200 Message-ID: <20240916140241.47845-2-kieran.bingham@ideasonboard.com> X-Mailer: git-send-email 2.46.0 In-Reply-To: <20240916140241.47845-1-kieran.bingham@ideasonboard.com> References: <20240916140241.47845-1-kieran.bingham@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Provide a helper on the MediaDevice to return a list of all available entities which match a given function in the graph. As a drive by, also fix a whitespace error in the documentation of MediaDevice::setupLink. Signed-off-by: Kieran Bingham Reviewed-by: Umang Jain Reviewed-by: Jacopo Mondi Reviewed-by: Daniel Scally --- include/libcamera/internal/media_device.h | 2 ++ src/libcamera/media_device.cpp | 24 ++++++++++++++++++++++- 2 files changed, 25 insertions(+), 1 deletion(-) diff --git a/include/libcamera/internal/media_device.h b/include/libcamera/internal/media_device.h index e412d3a0b7e3..b3a48b98d64b 100644 --- a/include/libcamera/internal/media_device.h +++ b/include/libcamera/internal/media_device.h @@ -55,6 +55,8 @@ public: Signal<> disconnected; + std::vector locateEntities(unsigned int function); + protected: std::string logPrefix() const override; diff --git a/src/libcamera/media_device.cpp b/src/libcamera/media_device.cpp index d71dad74df70..d1e0dee2ff0f 100644 --- a/src/libcamera/media_device.cpp +++ b/src/libcamera/media_device.cpp @@ -793,7 +793,7 @@ void MediaDevice::fixupEntityFlags(struct media_v2_entity *entity) * low-level link setup as it performs no checks on the validity of the \a * flags, and assumes that the supplied \a flags are valid for the link (e.g. * immutable links cannot be disabled). -* + * * \sa MediaLink::setEnabled(bool enable) * * \return 0 on success or a negative error code otherwise @@ -828,4 +828,26 @@ int MediaDevice::setupLink(const MediaLink *link, unsigned int flags) return 0; } +/** + * \brief Identify all entities of a common function in the MediaDevice + * \param[in] function The entity function to search for + * + * Search all entities within the graph of the MediaDevice and return + * a vector of those which match the given function. + * + * \return A vector of matching entities + */ +std::vector MediaDevice::locateEntities(unsigned int function) +{ + std::vector found; + + /* Gather all the entities matching the function they expose. */ + for (MediaEntity *entity : entities()) { + if (entity->function() == function) + found.push_back(entity); + } + + return found; +} + } /* namespace libcamera */ From patchwork Mon Sep 16 14:02:39 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kieran Bingham X-Patchwork-Id: 21282 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id AA40DC3257 for ; Mon, 16 Sep 2024 14:02:56 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id D71E363502; Mon, 16 Sep 2024 16:02:53 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="J6ferGuL"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 239F3634F5 for ; Mon, 16 Sep 2024 16:02:49 +0200 (CEST) Received: from charm.lan (unknown [IPv6:2001:4bc9:a45:b0af:726e:a88f:9f07:34be]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 07923DF3; Mon, 16 Sep 2024 16:01:27 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1726495288; bh=ErhgNr2NGcVwE/J0bp++WnI+oorHAvu55mM4+1Mhweo=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=J6ferGuL3BoA1j7nJugcVDUAhijS7VwMVx41KlKtOWWE37GnUmvNtlAvIRuZFdo3r W7uv3kYJxHaTwcd7eXSJEdza/IsNIG7ib9TUMp1SvUGa7y5WIJ/CbhCEfOCw2ezFoF CV63AWvq4WWFzA+/Sii74ju8SWrKaUg263DHA/YM= From: Kieran Bingham To: libcamera devel Cc: Kieran Bingham Subject: [PATCH 2/4] libcamera: pipeline: rkisp1: Fix typo in todo task Date: Mon, 16 Sep 2024 16:02:39 +0200 Message-ID: <20240916140241.47845-3-kieran.bingham@ideasonboard.com> X-Mailer: git-send-email 2.46.0 In-Reply-To: <20240916140241.47845-1-kieran.bingham@ideasonboard.com> References: <20240916140241.47845-1-kieran.bingham@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" The delay values should be managed correctly. Not the dealys. Correct accordingly, but I also hope we'll resolve this task soon too. Signed-off-by: Kieran Bingham Reviewed-by: Umang Jain Reviewed-by: Jacopo Mondi --- src/libcamera/pipeline/rkisp1/rkisp1.cpp | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/libcamera/pipeline/rkisp1/rkisp1.cpp b/src/libcamera/pipeline/rkisp1/rkisp1.cpp index c02c7cf374e9..2fee84e56d4d 100644 --- a/src/libcamera/pipeline/rkisp1/rkisp1.cpp +++ b/src/libcamera/pipeline/rkisp1/rkisp1.cpp @@ -1103,7 +1103,7 @@ int PipelineHandlerRkISP1::createCamera(MediaEntity *sensor) data->properties_ = data->sensor_->properties(); /* - * \todo Read dealy values from the sensor itself or from a + * \todo Read delay values from the sensor itself or from a * a sensor database. For now use generic values taken from * the Raspberry Pi and listed as generic values. */ From patchwork Mon Sep 16 14:02:40 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kieran Bingham X-Patchwork-Id: 21283 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 7E52EC3257 for ; Mon, 16 Sep 2024 14:02:58 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 9D88063507; Mon, 16 Sep 2024 16:02:56 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="ZIXiwMx2"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 4EF8063502 for ; Mon, 16 Sep 2024 16:02:50 +0200 (CEST) Received: from charm.lan (unknown [IPv6:2001:4bc9:a45:b0af:726e:a88f:9f07:34be]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 6846A480; Mon, 16 Sep 2024 16:01:28 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1726495288; bh=YPo+fQnSekB/ZczCl+huJ/SLvdLMrO31aVQVLR2jDOA=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=ZIXiwMx2V+UE3pn3cNhSYI54d1q7impGVsjBfcJLmRFvnkHWE+QsXA6ugGlr1GtSd 8yl0y6EZvHeBWmLqkLvo0Xjymml9CfLmEZLcFcSzNKOVOaWIOh8WYKiN5177PBySxU L5XcTI2PxztYSCcTimUND8WsyteYjl3z/oJXMKzU= From: Kieran Bingham To: libcamera devel Cc: Kieran Bingham Subject: [PATCH 3/4] libcamera: internal: Add MediaPipeline helper Date: Mon, 16 Sep 2024 16:02:40 +0200 Message-ID: <20240916140241.47845-4-kieran.bingham@ideasonboard.com> X-Mailer: git-send-email 2.46.0 In-Reply-To: <20240916140241.47845-1-kieran.bingham@ideasonboard.com> References: <20240916140241.47845-1-kieran.bingham@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Provide a MediaPipeline class to help identifing and managing pipelines across a MediaDevice graph. Signed-off-by: Kieran Bingham --- include/libcamera/internal/media_pipeline.h | 60 ++++ include/libcamera/internal/meson.build | 1 + src/libcamera/media_pipeline.cpp | 301 ++++++++++++++++++++ src/libcamera/meson.build | 1 + 4 files changed, 363 insertions(+) create mode 100644 include/libcamera/internal/media_pipeline.h create mode 100644 src/libcamera/media_pipeline.cpp diff --git a/include/libcamera/internal/media_pipeline.h b/include/libcamera/internal/media_pipeline.h new file mode 100644 index 000000000000..be8a611e20ca --- /dev/null +++ b/include/libcamera/internal/media_pipeline.h @@ -0,0 +1,60 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2024, Ideas on Board Oy + * + * Media pipeline handler + */ + +#pragma once + +#include +#include + +#include + +namespace libcamera { + +class CameraSensor; +class MediaEntity; +class MediaLink; +class MediaPad; +struct V4L2SubdeviceFormat; + +class MediaPipeline +{ +public: + int init(MediaEntity *source, std::string sink); + int initLinks(); + int configure(CameraSensor *sensor, V4L2SubdeviceFormat *); + +private: + struct Entity { + /* The media entity, always valid. */ + MediaEntity *entity; + /* + * Whether or not the entity is a subdev that supports the + * routing API. + */ + bool supportsRouting; + /* + * The local sink pad connected to the upstream entity, null for + * the camera sensor at the beginning of the pipeline. + */ + const MediaPad *sink; + /* + * The local source pad connected to the downstream entity, null + * for the video node at the end of the pipeline. + */ + const MediaPad *source; + /* + * The link on the source pad, to the downstream entity, null + * for the video node at the end of the pipeline. + */ + MediaLink *sourceLink; + }; + + std::vector routedSourcePads(MediaPad *sink); + std::list entities_; +}; + +} /* namespace libcamera */ diff --git a/include/libcamera/internal/meson.build b/include/libcamera/internal/meson.build index 1c5eef9cab80..60a35d3e0357 100644 --- a/include/libcamera/internal/meson.build +++ b/include/libcamera/internal/meson.build @@ -30,6 +30,7 @@ libcamera_internal_headers = files([ 'mapped_framebuffer.h', 'media_device.h', 'media_object.h', + 'media_pipeline.h', 'pipeline_handler.h', 'process.h', 'pub_key.h', diff --git a/src/libcamera/media_pipeline.cpp b/src/libcamera/media_pipeline.cpp new file mode 100644 index 000000000000..e14cc1ff4773 --- /dev/null +++ b/src/libcamera/media_pipeline.cpp @@ -0,0 +1,301 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2024, Ideas on Board Oy + * + * Media pipeline support + */ + +#include "libcamera/internal/media_pipeline.h" + +#include +#include +#include +#include +#include +#include +#include + +#include + +#include + +#include "libcamera/internal/camera_sensor.h" +#include "libcamera/internal/media_device.h" +#include "libcamera/internal/media_object.h" +#include "libcamera/internal/v4l2_subdevice.h" + +/** + * \file media_pipeline.h + * \brief Provide a representation of a pipeline of devices using the Media + * Controller. + */ + +namespace libcamera { + +LOG_DEFINE_CATEGORY(MediaPipeline) + +/** + * \class MediaPipeline + * \brief The MediaPipeline represents a set of entities that together form a + * data path for stream data. + * + * A MediaPipeline instance is constructed from a sink and a source between + * two entities in a media graph. + */ + +/** + * \brief Retrieve all source pads connected to a sink pad through active routes + * + * Examine the entity using the V4L2 Subdevice Routing API to collect all the + * source pads which are connected with an active route to the sink pad. + * + * \return A vector of source MediaPads + */ +std::vector MediaPipeline::routedSourcePads(MediaPad *sink) +{ + MediaEntity *entity = sink->entity(); + std::unique_ptr subdev = + std::make_unique(entity); + + int ret = subdev->open(); + if (ret < 0) + return {}; + + V4L2Subdevice::Routing routing = {}; + ret = subdev->getRouting(&routing, V4L2Subdevice::ActiveFormat); + if (ret < 0) + return {}; + + std::vector pads; + + for (const V4L2Subdevice::Route &route : routing) { + if (sink->index() != route.sink.pad || + !(route.flags & V4L2_SUBDEV_ROUTE_FL_ACTIVE)) + continue; + + const MediaPad *pad = entity->getPadByIndex(route.source.pad); + if (!pad) { + LOG(MediaPipeline, Warning) + << "Entity " << entity->name() + << " has invalid route source pad " + << route.source.pad; + } + + pads.push_back(pad); + } + + return pads; +} + +/** + * \brief Find the path from source to sink + * + * Starting from a source entity, deteremine the shortest path to the + * target described by sink. + * + * If sink can not be found, or a route from source to sink can not be achieved + * an error of -ENOLINK will be returned. + * + * When successful, the MediaPipeline will internally store the representation + * of entities and links to describe the path between the two entities. + * + * \return 0 on success, a negative errno otherwise + */ +int MediaPipeline::init(MediaEntity *source, std::string sink) +{ + /* + * Find the shortest path between from the Camera Sensor and the + * target entity. + */ + std::unordered_set visited; + std::queue> queue; + + /* Remember at each entity where we came from. */ + std::unordered_map parents; + MediaEntity *entity = nullptr; + MediaEntity *target = nullptr; + MediaPad *sinkPad; + + queue.push({ source, nullptr }); + + while (!queue.empty()) { + std::tie(entity, sinkPad) = queue.front(); + queue.pop(); + + /* Found the target device. */ + if (entity->name() == sink) { + LOG(MediaPipeline, Debug) + << "Found Pipeline target " << entity->name(); + target = entity; + break; + } + + visited.insert(entity); + + /* + * Add direct downstream entities to the search queue. If the + * current entity supports the subdev internal routing API, + * restrict the search to downstream entities reachable through + * active routes. + */ + + std::vector pads; + bool supportsRouting = false; + + if (sinkPad) { + pads = routedSourcePads(sinkPad); + if (!pads.empty()) + supportsRouting = true; + } + + if (pads.empty()) { + for (const MediaPad *pad : entity->pads()) { + if (!(pad->flags() & MEDIA_PAD_FL_SOURCE)) + continue; + pads.push_back(pad); + } + } + + for (const MediaPad *pad : pads) { + for (MediaLink *link : pad->links()) { + MediaEntity *next = link->sink()->entity(); + if (visited.find(next) == visited.end()) { + queue.push({ next, link->sink() }); + + Entity e{ entity, supportsRouting, sinkPad, pad, link }; + parents.insert({ next, e }); + } + } + } + } + + if (!target) { + LOG(MediaPipeline, Error) << "Failed to connect " << source->name(); + return -ENOLINK; + } + + /* + * With the parents, we can follow back our way from the capture device + * to the sensor. Store all the entities in the pipeline, from the + * camera sensor to the video node, in entities_. + */ + entities_.push_front({ entity, false, sinkPad, nullptr, nullptr }); + + for (auto it = parents.find(entity); it != parents.end(); + it = parents.find(entity)) { + const Entity &e = it->second; + entities_.push_front(e); + entity = e.entity; + } + + LOG(MediaPipeline, Info) + << "Found pipeline: " + << utils::join(entities_, " -> ", + [](const Entity &e) { + std::string s = "["; + if (e.sink) + s += std::to_string(e.sink->index()) + "|"; + s += e.entity->name(); + if (e.source) + s += "|" + std::to_string(e.source->index()); + s += "]"; + return s; + }); + + return 0; +} + +/** + * \brief Initialise and enable all links through the MediaPipeline + * \return 0 on success, or a negative errno otherwise + */ +int MediaPipeline::initLinks() +{ + int ret = 0; + + MediaLink *sinkLink = nullptr; + for (Entity &e : entities_) { + /* Sensor entities have no connected sink. */ + if (!sinkLink) { + sinkLink = e.sourceLink; + continue; + } + + LOG(MediaPipeline, Debug) << "Enabling : " << *sinkLink; + + if (!(sinkLink->flags() & MEDIA_LNK_FL_ENABLED)) { + ret = sinkLink->setEnabled(true); + if (ret < 0) + return ret; + } + + sinkLink = e.sourceLink; + } + + return ret; +} + +/** + * \brief Configure the entities of this MediaPipeline + * + * Propagate formats through each of the entities of the Pipeline, validating + * that each one was not adjusted by the driver from the desired format. + * + * \return 0 on success or a negative errno otherwise + */ +int MediaPipeline::configure(CameraSensor *sensor, V4L2SubdeviceFormat *format) +{ + int ret; + + for (const Entity &e : entities_) { + /* The sensor is configured through the CameraSensor */ + if (!e.sourceLink) + break; + + MediaLink *link = e.sourceLink; + MediaPad *source = link->source(); + MediaPad *sink = link->sink(); + + /* 'format' already contains the sensor configuration */ + if (source->entity() != sensor->entity()) { + /* Todo: Add MediaDevice cache to reduce FD pressure */ + V4L2Subdevice subdev(source->entity()); + ret = subdev.open(); + if (ret) + return ret; + + ret = subdev.getFormat(source->index(), format); + if (ret < 0) + return ret; + } + + V4L2SubdeviceFormat sourceFormat = *format; + /* Todo: Add MediaDevice cache to reduce FD pressure */ + V4L2Subdevice subdev(sink->entity()); + ret = subdev.open(); + if (ret) + return ret; + + ret = subdev.setFormat(sink->index(), format); + if (ret < 0) + return ret; + + if (format->code != sourceFormat.code || + format->size != sourceFormat.size) { + LOG(MediaPipeline, Debug) + << "Source '" << *source + << " produces " << sourceFormat + << ", sink '" << *sink + << " requires " << format; + return -EINVAL; + } + + LOG(MediaPipeline, Debug) + << "Link " << *link << " configured with format " + << format; + } + + return 0; +} + +} /* namespace libcamera */ diff --git a/src/libcamera/meson.build b/src/libcamera/meson.build index aa9ab0291854..2c0f8603b231 100644 --- a/src/libcamera/meson.build +++ b/src/libcamera/meson.build @@ -41,6 +41,7 @@ libcamera_internal_sources = files([ 'mapped_framebuffer.cpp', 'media_device.cpp', 'media_object.cpp', + 'media_pipeline.cpp', 'pipeline_handler.cpp', 'process.cpp', 'pub_key.cpp', From patchwork Mon Sep 16 14:02:41 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kieran Bingham X-Patchwork-Id: 21284 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 0A4FFC3261 for ; Mon, 16 Sep 2024 14:02:59 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 06BC663502; Mon, 16 Sep 2024 16:02:59 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="BVdPu6R/"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id E7380634F5 for ; Mon, 16 Sep 2024 16:02:50 +0200 (CEST) Received: from charm.lan (unknown [IPv6:2001:4bc9:a45:b0af:726e:a88f:9f07:34be]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id BD9F882A; Mon, 16 Sep 2024 16:01:29 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1726495289; bh=knY69wIVrTmSm8+G44lJgwCYK/KEGjz87/uwNLrxMYs=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=BVdPu6R/1Wp1jn1Bw5BDhCwKG3avik+VCLj0V+Y64o5h/Y3z+guYPI+qhybxaLHKK UKDPwNngHSnG0Xx6+/pZdXLVufosBKyZcDCyTscCJ9i/bInJ39fGL3ewCFZ7Dmry+y XQqDAhVI2/CnF8OGPugIB0L4JI1eh4gtrXCv1nXU= From: Kieran Bingham To: libcamera devel Cc: Kieran Bingham Subject: [PATCH 4/4] libcamera: pipeline: rkisp1: Convert to use MediaPipeline Date: Mon, 16 Sep 2024 16:02:41 +0200 Message-ID: <20240916140241.47845-5-kieran.bingham@ideasonboard.com> X-Mailer: git-send-email 2.46.0 In-Reply-To: <20240916140241.47845-1-kieran.bingham@ideasonboard.com> References: <20240916140241.47845-1-kieran.bingham@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Use the new MediaPipeline to manage and identify all sensors connected to complex pipelines that can connect to the CSI2 receiver before the ISP. This can include chained multiplexors that supply multiple cameras, so make use of the MediaDevice::locateEntities to search for all cameras and construct a pipeline for each. Signed-off-by: Kieran Bingham Reviewed-by: Umang Jain --- src/libcamera/pipeline/rkisp1/rkisp1.cpp | 65 ++++++++++++------------ 1 file changed, 32 insertions(+), 33 deletions(-) diff --git a/src/libcamera/pipeline/rkisp1/rkisp1.cpp b/src/libcamera/pipeline/rkisp1/rkisp1.cpp index 2fee84e56d4d..914f5181b51d 100644 --- a/src/libcamera/pipeline/rkisp1/rkisp1.cpp +++ b/src/libcamera/pipeline/rkisp1/rkisp1.cpp @@ -37,6 +37,7 @@ #include "libcamera/internal/framebuffer.h" #include "libcamera/internal/ipa_manager.h" #include "libcamera/internal/media_device.h" +#include "libcamera/internal/media_pipeline.h" #include "libcamera/internal/pipeline_handler.h" #include "libcamera/internal/v4l2_subdevice.h" #include "libcamera/internal/v4l2_videodevice.h" @@ -108,6 +109,11 @@ public: std::unique_ptr ipa_; + /* + * All entities in the pipeline, from the camera sensor to the RKISP1. + */ + MediaPipeline pipe_; + private: void paramFilled(unsigned int frame, unsigned int bytesused); void setSensorControls(unsigned int frame, @@ -171,8 +177,7 @@ private: friend RkISP1CameraData; friend RkISP1Frames; - int initLinks(Camera *camera, const CameraSensor *sensor, - const RkISP1CameraConfiguration &config); + int initLinks(Camera *camera, const RkISP1CameraConfiguration &config); int createCamera(MediaEntity *sensor); void tryCompleteRequest(RkISP1FrameInfo *info); void bufferReady(FrameBuffer *buffer); @@ -712,7 +717,7 @@ int PipelineHandlerRkISP1::configure(Camera *camera, CameraConfiguration *c) CameraSensor *sensor = data->sensor_.get(); int ret; - ret = initLinks(camera, sensor, *config); + ret = initLinks(camera, *config); if (ret) return ret; @@ -729,12 +734,12 @@ int PipelineHandlerRkISP1::configure(Camera *camera, CameraConfiguration *c) LOG(RkISP1, Debug) << "Sensor configured with " << format; - if (csi_) { - ret = csi_->setFormat(0, &format); - if (ret < 0) - return ret; - } + /* Propagate format through the internal media pipeline up to the ISP */ + ret = data->pipe_.configure(sensor, &format); + if (ret < 0) + return ret; + LOG(RkISP1, Debug) << "Configuring ISP with : " << format; ret = isp_->setFormat(0, &format); if (ret < 0) return ret; @@ -1035,7 +1040,6 @@ int PipelineHandlerRkISP1::queueRequestDevice(Camera *camera, Request *request) */ int PipelineHandlerRkISP1::initLinks(Camera *camera, - const CameraSensor *sensor, const RkISP1CameraConfiguration &config) { RkISP1CameraData *data = cameraData(camera); @@ -1046,31 +1050,16 @@ int PipelineHandlerRkISP1::initLinks(Camera *camera, return ret; /* - * Configure the sensor links: enable the link corresponding to this - * camera. + * Configure the sensor links: enable the links corresponding to this + * pipeline all the way up to the ISP, through any connected CSI receiver. */ - for (MediaLink *link : ispSink_->links()) { - if (link->source()->entity() != sensor->entity()) - continue; - - LOG(RkISP1, Debug) - << "Enabling link from sensor '" - << link->source()->entity()->name() - << "' to ISP"; - - ret = link->setEnabled(true); - if (ret < 0) - return ret; - } - - if (csi_) { - MediaLink *link = isp_->entity()->getPadByIndex(0)->links().at(0); - - ret = link->setEnabled(true); - if (ret < 0) - return ret; + ret = data->pipe_.initLinks(); + if (ret) { + LOG(RkISP1, Error) << "Failed to set up pipe links"; + return ret; } + /* Configure the paths after the ISP */ for (const StreamConfiguration &cfg : config) { if (cfg.stream() == &data->mainPathStream_) ret = data->mainPath_->setEnabled(true); @@ -1094,6 +1083,13 @@ int PipelineHandlerRkISP1::createCamera(MediaEntity *sensor) std::make_unique(this, &mainPath_, hasSelfPath_ ? &selfPath_ : nullptr); + /* Identify the pipeline path between the sensor and the rkisp1_isp */ + ret = data->pipe_.init(sensor, "rkisp1_isp"); + if (ret) { + LOG(RkISP1, Error) << "Failed to identify path from sensor to sink"; + return ret; + } + data->sensor_ = std::make_unique(sensor); ret = data->sensor_->init(); if (ret) @@ -1129,6 +1125,7 @@ int PipelineHandlerRkISP1::createCamera(MediaEntity *sensor) const std::string &id = data->sensor_->id(); std::shared_ptr camera = Camera::create(std::move(data), id, streams); + registerCamera(std::move(camera)); return 0; @@ -1205,8 +1202,10 @@ bool PipelineHandlerRkISP1::match(DeviceEnumerator *enumerator) * camera instance for each of them. */ bool registered = false; - for (MediaLink *link : ispSink_->links()) { - if (!createCamera(link->source()->entity())) + + for (MediaEntity *entity : media_->locateEntities(MEDIA_ENT_F_CAM_SENSOR)) { + LOG(RkISP1, Info) << "Identified " << entity->name(); + if (!createCamera(entity)) registered = true; }