From patchwork Wed Oct 2 21:26:29 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kieran Bingham X-Patchwork-Id: 21482 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id AC2E1C3257 for ; Wed, 2 Oct 2024 21:26:45 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id B74D963510; Wed, 2 Oct 2024 23:26:42 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="gWudTBsF"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 125406351F for ; Wed, 2 Oct 2024 23:26:38 +0200 (CEST) Received: from Monstersaurus.tail69b4.ts.net (cpc89244-aztw30-2-0-cust6594.18-1.cable.virginm.net [86.31.185.195]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 179BA968; Wed, 2 Oct 2024 23:25:05 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1727904305; bh=r8owuYlalLJ22tjZS0HNJ5QAf+oxLDCm5akMRV85+Bw=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=gWudTBsFforxob2AAge4JYzYfYq2h3szZlhrQMo7sDk8UPje8co0eRqidDwBWIe7U citehGsIeR6onXXYpOTqgn121mUccvDrYQQmt1eIBtbFwwzbOvl5fdm64l4t6Xdz2m lTG3CVsVdMnfQHErT22yv8tDU6OvzwdljplpE9uE= From: Kieran Bingham To: libcamera devel Cc: Kieran Bingham , Umang Jain , Jacopo Mondi Subject: [PATCH v2 1/4] libcamera: media_device: Add helper to return matching entities Date: Wed, 2 Oct 2024 22:26:29 +0100 Message-Id: <20241002212632.2463458-2-kieran.bingham@ideasonboard.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20241002212632.2463458-1-kieran.bingham@ideasonboard.com> References: <20241002212632.2463458-1-kieran.bingham@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Provide a helper on the MediaDevice to return a list of all available entities which match a given function in the graph. As a drive by, also fix a whitespace error in the documentation of MediaDevice::setupLink. Signed-off-by: Kieran Bingham Reviewed-by: Umang Jain Reviewed-by: Jacopo Mondi Signed-off-by: Kieran Bingham --- include/libcamera/internal/media_device.h | 2 ++ src/libcamera/media_device.cpp | 25 ++++++++++++++++++++++- 2 files changed, 26 insertions(+), 1 deletion(-) diff --git a/include/libcamera/internal/media_device.h b/include/libcamera/internal/media_device.h index e412d3a0b7e3..b3a48b98d64b 100644 --- a/include/libcamera/internal/media_device.h +++ b/include/libcamera/internal/media_device.h @@ -55,6 +55,8 @@ public: Signal<> disconnected; + std::vector locateEntities(unsigned int function); + protected: std::string logPrefix() const override; diff --git a/src/libcamera/media_device.cpp b/src/libcamera/media_device.cpp index d71dad74df70..65080a4afdca 100644 --- a/src/libcamera/media_device.cpp +++ b/src/libcamera/media_device.cpp @@ -793,7 +793,7 @@ void MediaDevice::fixupEntityFlags(struct media_v2_entity *entity) * low-level link setup as it performs no checks on the validity of the \a * flags, and assumes that the supplied \a flags are valid for the link (e.g. * immutable links cannot be disabled). -* + * * \sa MediaLink::setEnabled(bool enable) * * \return 0 on success or a negative error code otherwise @@ -828,4 +828,27 @@ int MediaDevice::setupLink(const MediaLink *link, unsigned int flags) return 0; } +/** + * \brief Identify all entities of a common function in the MediaDevice + * \param[in] function The entity function to search for + * + * Search all entities within the graph of the MediaDevice and return + * a vector of those which match the given function as defined by the + * MEDIA_ENT_F_* macros of the Media Controller API. + * + * \return A vector of matching entities + */ +std::vector MediaDevice::locateEntities(unsigned int function) +{ + std::vector found; + + /* Gather all the entities matching the function they expose. */ + for (MediaEntity *entity : entities()) { + if (entity->function() == function) + found.push_back(entity); + } + + return found; +} + } /* namespace libcamera */ From patchwork Wed Oct 2 21:26:30 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kieran Bingham X-Patchwork-Id: 21483 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 87ED4C3257 for ; Wed, 2 Oct 2024 21:26:47 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 6C1FF6352C; Wed, 2 Oct 2024 23:26:44 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="ajH+w69B"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 27CD463522 for ; Wed, 2 Oct 2024 23:26:38 +0200 (CEST) Received: from Monstersaurus.tail69b4.ts.net (cpc89244-aztw30-2-0-cust6594.18-1.cable.virginm.net [86.31.185.195]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 60126A44; Wed, 2 Oct 2024 23:25:05 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1727904305; bh=lQj9d/yLrBdgWbTScxcUTqS8eGShv7JmIi1LDhdBuds=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=ajH+w69BhrohBOzcqiaIX1eY7FSr//JaT8/wbTwB3F8NZ5VpviUp6G9fVvsqrE1hH Ay/KbzPBM6Hi/c1pJjG6ho+S8fEQs4Ak3paw7DlnhM2sWyUhIqAI/SeL98vLA4xRcy vUgJStf5Z9q2U2FdZ+IMIadjNHwirABYf2tv0Zok= From: Kieran Bingham To: libcamera devel Cc: Kieran Bingham , Umang Jain , Jacopo Mondi Subject: [PATCH v2 2/4] libcamera: pipeline: rkisp1: Fix typo in todo task Date: Wed, 2 Oct 2024 22:26:30 +0100 Message-Id: <20241002212632.2463458-3-kieran.bingham@ideasonboard.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20241002212632.2463458-1-kieran.bingham@ideasonboard.com> References: <20241002212632.2463458-1-kieran.bingham@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" The delay values should be managed correctly. Not the dealys. Correct accordingly. Signed-off-by: Kieran Bingham Reviewed-by: Umang Jain Reviewed-by: Jacopo Mondi Signed-off-by: Kieran Bingham --- src/libcamera/pipeline/rkisp1/rkisp1.cpp | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/libcamera/pipeline/rkisp1/rkisp1.cpp b/src/libcamera/pipeline/rkisp1/rkisp1.cpp index c02c7cf374e9..2fee84e56d4d 100644 --- a/src/libcamera/pipeline/rkisp1/rkisp1.cpp +++ b/src/libcamera/pipeline/rkisp1/rkisp1.cpp @@ -1103,7 +1103,7 @@ int PipelineHandlerRkISP1::createCamera(MediaEntity *sensor) data->properties_ = data->sensor_->properties(); /* - * \todo Read dealy values from the sensor itself or from a + * \todo Read delay values from the sensor itself or from a * a sensor database. For now use generic values taken from * the Raspberry Pi and listed as generic values. */ From patchwork Wed Oct 2 21:26:31 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kieran Bingham X-Patchwork-Id: 21484 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 118DAC3263 for ; Wed, 2 Oct 2024 21:26:49 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 78B086352F; Wed, 2 Oct 2024 23:26:45 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="vjVUgGPO"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 54EB063525 for ; Wed, 2 Oct 2024 23:26:38 +0200 (CEST) Received: from Monstersaurus.tail69b4.ts.net (cpc89244-aztw30-2-0-cust6594.18-1.cable.virginm.net [86.31.185.195]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id A9D10581; Wed, 2 Oct 2024 23:25:05 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1727904305; bh=3B3ZRbn1OA0rh9GtaEVnod5Z+qdsXoWMeo6A6OE2URY=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=vjVUgGPOCm3lEO8Pek9tyvWgoxF2coT/E4K1P0xhvSCwSk3P9JTGl2hlH2uChuMPo H7rEIF4yq26MmTsCztOZ64jjQHEI7xnWETAM4psgfdSzKZ0+u3UEbeLL4k1xzm8cyh m1f2SlMYQin642eKRJrN60L2ImfRY3Snc1tT3sb0= From: Kieran Bingham To: libcamera devel Cc: Kieran Bingham Subject: [PATCH v2 3/4] libcamera: internal: Add MediaPipeline helper Date: Wed, 2 Oct 2024 22:26:31 +0100 Message-Id: <20241002212632.2463458-4-kieran.bingham@ideasonboard.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20241002212632.2463458-1-kieran.bingham@ideasonboard.com> References: <20241002212632.2463458-1-kieran.bingham@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Provide a MediaPipeline class to help identifing and managing pipelines across a MediaDevice graph. Signed-off-by: Kieran Bingham Reviewed-by: Umang Jain --- v2: - use srcPads to clearly identify which pads are managed - Only report enabling links when a change is made - fix header includes - Fix includes - Remove period at end of briefs - Document function parameters - expand documentation throughout - Fix debug log capitalisation - reduce scope of single use 'ret' Signed-off-by: Kieran Bingham --- include/libcamera/internal/media_pipeline.h | 59 ++++ include/libcamera/internal/meson.build | 1 + src/libcamera/media_pipeline.cpp | 307 ++++++++++++++++++++ src/libcamera/meson.build | 1 + 4 files changed, 368 insertions(+) create mode 100644 include/libcamera/internal/media_pipeline.h create mode 100644 src/libcamera/media_pipeline.cpp diff --git a/include/libcamera/internal/media_pipeline.h b/include/libcamera/internal/media_pipeline.h new file mode 100644 index 000000000000..ca4a86a88a1e --- /dev/null +++ b/include/libcamera/internal/media_pipeline.h @@ -0,0 +1,59 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2024, Ideas on Board Oy + * + * Media pipeline handler + */ + +#pragma once + +#include +#include +#include + +namespace libcamera { + +class CameraSensor; +class MediaEntity; +class MediaLink; +class MediaPad; +struct V4L2SubdeviceFormat; + +class MediaPipeline +{ +public: + int init(MediaEntity *source, std::string sink); + int initLinks(); + int configure(CameraSensor *sensor, V4L2SubdeviceFormat *); + +private: + struct Entity { + /* The media entity, always valid. */ + MediaEntity *entity; + /* + * Whether or not the entity is a subdev that supports the + * routing API. + */ + bool supportsRouting; + /* + * The local sink pad connected to the upstream entity, null for + * the camera sensor at the beginning of the pipeline. + */ + const MediaPad *sink; + /* + * The local source pad connected to the downstream entity, null + * for the video node at the end of the pipeline. + */ + const MediaPad *source; + /* + * The link on the source pad, to the downstream entity, null + * for the video node at the end of the pipeline. + */ + MediaLink *sourceLink; + }; + + std::vector routedSourcePads(MediaPad *sink); + std::list entities_; +}; + +} /* namespace libcamera */ diff --git a/include/libcamera/internal/meson.build b/include/libcamera/internal/meson.build index 1c5eef9cab80..60a35d3e0357 100644 --- a/include/libcamera/internal/meson.build +++ b/include/libcamera/internal/meson.build @@ -30,6 +30,7 @@ libcamera_internal_headers = files([ 'mapped_framebuffer.h', 'media_device.h', 'media_object.h', + 'media_pipeline.h', 'pipeline_handler.h', 'process.h', 'pub_key.h', diff --git a/src/libcamera/media_pipeline.cpp b/src/libcamera/media_pipeline.cpp new file mode 100644 index 000000000000..dba8084ddc97 --- /dev/null +++ b/src/libcamera/media_pipeline.cpp @@ -0,0 +1,307 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2024, Ideas on Board Oy + * + * Media pipeline support + */ + +#include "libcamera/internal/media_pipeline.h" + +#include +#include +#include +#include +#include +#include +#include + +#include + +#include + +#include "libcamera/internal/camera_sensor.h" +#include "libcamera/internal/media_device.h" +#include "libcamera/internal/media_object.h" +#include "libcamera/internal/v4l2_subdevice.h" + +/** + * \file media_pipeline.h + * \brief Provide a representation of a pipeline of devices using the Media + * Controller + */ + +namespace libcamera { + +LOG_DEFINE_CATEGORY(MediaPipeline) + +/** + * \class MediaPipeline + * \brief The MediaPipeline represents a set of entities that together form a + * data path for stream data + * + * A MediaPipeline instance is constructed from a sink and a source between + * two entities in a media graph. + */ + +/** + * \brief Retrieve all source pads connected to a sink pad through active routes + * \param[in] sink The sink pad to examine + * + * Examine the entity using the V4L2 Subdevice Routing API to collect all the + * source pads which are connected with an active route to the sink pad. + * + * \return A vector of source MediaPads + */ +std::vector MediaPipeline::routedSourcePads(MediaPad *sink) +{ + MediaEntity *entity = sink->entity(); + std::unique_ptr subdev = + std::make_unique(entity); + + int ret = subdev->open(); + if (ret < 0) + return {}; + + V4L2Subdevice::Routing routing = {}; + ret = subdev->getRouting(&routing, V4L2Subdevice::ActiveFormat); + if (ret < 0) + return {}; + + std::vector pads; + + for (const V4L2Subdevice::Route &route : routing) { + if (sink->index() != route.sink.pad || + !(route.flags & V4L2_SUBDEV_ROUTE_FL_ACTIVE)) + continue; + + const MediaPad *pad = entity->getPadByIndex(route.source.pad); + if (!pad) { + LOG(MediaPipeline, Warning) + << "Entity " << entity->name() + << " has invalid route source pad " + << route.source.pad; + } + + pads.push_back(pad); + } + + return pads; +} + +/** + * \brief Find the path from source to sink + * \param[in] source The source entity to start from + * \param[in] sink The sink entity name to search for + * + * Starting from a source entity, determine the shortest path to the target + * described by sink. + * + * If sink can not be found, or a route from source to sink can not be achieved, + * an error of -ENOLINK will be returned. + * + * When successful, the MediaPipeline will internally store the representation + * of entities and links to describe the path between the two entities. + * + * It is expected that the Source entity is a sensor represented by the + * CameraSensor class. + * + * \return 0 on success, a negative errno otherwise + */ +int MediaPipeline::init(MediaEntity *source, std::string sink) +{ + /* + * Find the shortest path between from the source and the + * target entity. + */ + std::unordered_set visited; + std::queue> queue; + + /* Remember at each entity where we came from. */ + std::unordered_map parents; + MediaEntity *entity = nullptr; + MediaEntity *target = nullptr; + MediaPad *sinkPad; + + queue.push({ source, nullptr }); + + while (!queue.empty()) { + std::tie(entity, sinkPad) = queue.front(); + queue.pop(); + + /* Found the target device. */ + if (entity->name() == sink) { + LOG(MediaPipeline, Debug) + << "Found pipeline target " << entity->name(); + target = entity; + break; + } + + visited.insert(entity); + + /* + * Add direct downstream entities to the search queue. If the + * current entity supports the subdev internal routing API, + * restrict the search to downstream entities reachable through + * active routes. + */ + + std::vector srcPads; + bool supportsRouting = false; + + if (sinkPad) { + srcPads = routedSourcePads(sinkPad); + if (!srcPads.empty()) + supportsRouting = true; + } + + if (srcPads.empty()) { + for (const MediaPad *pad : entity->pads()) { + if (!(pad->flags() & MEDIA_PAD_FL_SOURCE)) + continue; + srcPads.push_back(pad); + } + } + + for (const MediaPad *srcPad : srcPads) { + for (MediaLink *link : srcPad->links()) { + MediaEntity *next = link->sink()->entity(); + if (visited.find(next) == visited.end()) { + queue.push({ next, link->sink() }); + + Entity e{ entity, supportsRouting, sinkPad, + srcPad, link }; + parents.insert({ next, e }); + } + } + } + } + + if (!target) { + LOG(MediaPipeline, Error) << "Failed to connect " << source->name(); + return -ENOLINK; + } + + /* + * With the parents, we can follow back our way from the capture device + * to the sensor. Store all the entities in the pipeline, from the + * camera sensor to the video node, in entities_. + */ + entities_.push_front({ entity, false, sinkPad, nullptr, nullptr }); + + for (auto it = parents.find(entity); it != parents.end(); + it = parents.find(entity)) { + const Entity &e = it->second; + entities_.push_front(e); + entity = e.entity; + } + + LOG(MediaPipeline, Info) + << "Found pipeline: " + << utils::join(entities_, " -> ", + [](const Entity &e) { + std::string s = "["; + if (e.sink) + s += std::to_string(e.sink->index()) + "|"; + s += e.entity->name(); + if (e.source) + s += "|" + std::to_string(e.source->index()); + s += "]"; + return s; + }); + + return 0; +} + +/** + * \brief Initialise and enable all links through the MediaPipeline + * \return 0 on success, or a negative errno otherwise + */ +int MediaPipeline::initLinks() +{ + MediaLink *sinkLink = nullptr; + for (Entity &e : entities_) { + /* Sensor entities have no connected sink. */ + if (!sinkLink) { + sinkLink = e.sourceLink; + continue; + } + + if (!(sinkLink->flags() & MEDIA_LNK_FL_ENABLED)) { + LOG(MediaPipeline, Debug) << "Enabling : " << *sinkLink; + + int ret = sinkLink->setEnabled(true); + if (ret < 0) + return ret; + } + + sinkLink = e.sourceLink; + } + + return 0; +} + +/** + * \brief Configure the entities of this MediaPipeline + * \param[in] sensor The configured CameraSensor to propogate + * + * Propagate formats through each of the entities of the Pipeline, validating + * that each one was not adjusted by the driver from the desired format. + * + * \return 0 on success or a negative errno otherwise + */ +int MediaPipeline::configure(CameraSensor *sensor, V4L2SubdeviceFormat *format) +{ + int ret; + + for (const Entity &e : entities_) { + /* The sensor is configured through the CameraSensor */ + if (!e.sourceLink) + break; + + MediaLink *link = e.sourceLink; + MediaPad *source = link->source(); + MediaPad *sink = link->sink(); + + /* 'format' already contains the sensor configuration */ + if (source->entity() != sensor->entity()) { + /* todo: Add MediaDevice cache to reduce FD pressure */ + V4L2Subdevice subdev(source->entity()); + ret = subdev.open(); + if (ret) + return ret; + + ret = subdev.getFormat(source->index(), format); + if (ret < 0) + return ret; + } + + V4L2SubdeviceFormat sourceFormat = *format; + /* todo: Add MediaDevice cache to reduce FD pressure */ + V4L2Subdevice subdev(sink->entity()); + ret = subdev.open(); + if (ret) + return ret; + + ret = subdev.setFormat(sink->index(), format); + if (ret < 0) + return ret; + + if (format->code != sourceFormat.code || + format->size != sourceFormat.size) { + LOG(MediaPipeline, Debug) + << "Source '" << *source + << " produces " << sourceFormat + << ", sink '" << *sink + << " requires " << format; + return -EINVAL; + } + + LOG(MediaPipeline, Debug) + << "Link " << *link << " configured with format " + << format; + } + + return 0; +} + +} /* namespace libcamera */ diff --git a/src/libcamera/meson.build b/src/libcamera/meson.build index aa9ab0291854..2c0f8603b231 100644 --- a/src/libcamera/meson.build +++ b/src/libcamera/meson.build @@ -41,6 +41,7 @@ libcamera_internal_sources = files([ 'mapped_framebuffer.cpp', 'media_device.cpp', 'media_object.cpp', + 'media_pipeline.cpp', 'pipeline_handler.cpp', 'process.cpp', 'pub_key.cpp', From patchwork Wed Oct 2 21:26:32 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kieran Bingham X-Patchwork-Id: 21485 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id E6249C3257 for ; Wed, 2 Oct 2024 21:26:49 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 7CD6A63528; Wed, 2 Oct 2024 23:26:46 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="tgHS8R5h"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 8CACF63526 for ; Wed, 2 Oct 2024 23:26:38 +0200 (CEST) Received: from Monstersaurus.tail69b4.ts.net (cpc89244-aztw30-2-0-cust6594.18-1.cable.virginm.net [86.31.185.195]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id E74C9D52; Wed, 2 Oct 2024 23:25:05 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1727904306; bh=sEWWTuT3IY/o/vSV5Oolf5bQD+PHUkQsbamcUbKVLZI=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=tgHS8R5h07IVORkftZbCegA6MRFGAyvDAxnBS6VMekqJMXrGNrflUXkQEmqubPxaT w0JJkjApQe200pLgl1/Jwov263fTcsw+TAlouiGlJ4Jz2WD7XWJbwfeNA/4+fu3Enb U8Gv2YyKyU1vrEqNOdCa2PshPgvJqMECHiI9PIio= From: Kieran Bingham To: libcamera devel Cc: Kieran Bingham , Umang Jain Subject: [PATCH v2 4/4] libcamera: pipeline: rkisp1: Convert to use MediaPipeline Date: Wed, 2 Oct 2024 22:26:32 +0100 Message-Id: <20241002212632.2463458-5-kieran.bingham@ideasonboard.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20241002212632.2463458-1-kieran.bingham@ideasonboard.com> References: <20241002212632.2463458-1-kieran.bingham@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Use the new MediaPipeline to manage and identify all sensors connected to complex pipelines that can connect to the CSI2 receiver before the ISP. This can include chained multiplexers that supply multiple cameras, so make use of the MediaDevice::locateEntities to search for all cameras and construct a pipeline for each. Signed-off-by: Kieran Bingham Reviewed-by: Umang Jain Signed-off-by: Kieran Bingham --- src/libcamera/pipeline/rkisp1/rkisp1.cpp | 86 +++++++++--------------- 1 file changed, 32 insertions(+), 54 deletions(-) diff --git a/src/libcamera/pipeline/rkisp1/rkisp1.cpp b/src/libcamera/pipeline/rkisp1/rkisp1.cpp index 2fee84e56d4d..e94f047ba70c 100644 --- a/src/libcamera/pipeline/rkisp1/rkisp1.cpp +++ b/src/libcamera/pipeline/rkisp1/rkisp1.cpp @@ -37,6 +37,7 @@ #include "libcamera/internal/framebuffer.h" #include "libcamera/internal/ipa_manager.h" #include "libcamera/internal/media_device.h" +#include "libcamera/internal/media_pipeline.h" #include "libcamera/internal/pipeline_handler.h" #include "libcamera/internal/v4l2_subdevice.h" #include "libcamera/internal/v4l2_videodevice.h" @@ -108,6 +109,11 @@ public: std::unique_ptr ipa_; + /* + * All entities in the pipeline, from the camera sensor to the RkISP1. + */ + MediaPipeline pipe_; + private: void paramFilled(unsigned int frame, unsigned int bytesused); void setSensorControls(unsigned int frame, @@ -171,8 +177,7 @@ private: friend RkISP1CameraData; friend RkISP1Frames; - int initLinks(Camera *camera, const CameraSensor *sensor, - const RkISP1CameraConfiguration &config); + int initLinks(Camera *camera, const RkISP1CameraConfiguration &config); int createCamera(MediaEntity *sensor); void tryCompleteRequest(RkISP1FrameInfo *info); void bufferReady(FrameBuffer *buffer); @@ -187,7 +192,6 @@ private: std::unique_ptr isp_; std::unique_ptr param_; std::unique_ptr stat_; - std::unique_ptr csi_; bool hasSelfPath_; bool isRaw_; @@ -201,8 +205,6 @@ private: std::queue availableStatBuffers_; Camera *activeCamera_; - - const MediaPad *ispSink_; }; RkISP1Frames::RkISP1Frames(PipelineHandler *pipe) @@ -712,7 +714,7 @@ int PipelineHandlerRkISP1::configure(Camera *camera, CameraConfiguration *c) CameraSensor *sensor = data->sensor_.get(); int ret; - ret = initLinks(camera, sensor, *config); + ret = initLinks(camera, *config); if (ret) return ret; @@ -729,12 +731,12 @@ int PipelineHandlerRkISP1::configure(Camera *camera, CameraConfiguration *c) LOG(RkISP1, Debug) << "Sensor configured with " << format; - if (csi_) { - ret = csi_->setFormat(0, &format); - if (ret < 0) - return ret; - } + /* Propagate format through the internal media pipeline up to the ISP */ + ret = data->pipe_.configure(sensor, &format); + if (ret < 0) + return ret; + LOG(RkISP1, Debug) << "Configuring ISP with : " << format; ret = isp_->setFormat(0, &format); if (ret < 0) return ret; @@ -1035,7 +1037,6 @@ int PipelineHandlerRkISP1::queueRequestDevice(Camera *camera, Request *request) */ int PipelineHandlerRkISP1::initLinks(Camera *camera, - const CameraSensor *sensor, const RkISP1CameraConfiguration &config) { RkISP1CameraData *data = cameraData(camera); @@ -1046,31 +1047,16 @@ int PipelineHandlerRkISP1::initLinks(Camera *camera, return ret; /* - * Configure the sensor links: enable the link corresponding to this - * camera. + * Configure the sensor links: enable the links corresponding to this + * pipeline all the way up to the ISP, through any connected CSI receiver. */ - for (MediaLink *link : ispSink_->links()) { - if (link->source()->entity() != sensor->entity()) - continue; - - LOG(RkISP1, Debug) - << "Enabling link from sensor '" - << link->source()->entity()->name() - << "' to ISP"; - - ret = link->setEnabled(true); - if (ret < 0) - return ret; - } - - if (csi_) { - MediaLink *link = isp_->entity()->getPadByIndex(0)->links().at(0); - - ret = link->setEnabled(true); - if (ret < 0) - return ret; + ret = data->pipe_.initLinks(); + if (ret) { + LOG(RkISP1, Error) << "Failed to set up pipe links"; + return ret; } + /* Configure the paths after the ISP */ for (const StreamConfiguration &cfg : config) { if (cfg.stream() == &data->mainPathStream_) ret = data->mainPath_->setEnabled(true); @@ -1094,6 +1080,13 @@ int PipelineHandlerRkISP1::createCamera(MediaEntity *sensor) std::make_unique(this, &mainPath_, hasSelfPath_ ? &selfPath_ : nullptr); + /* Identify the pipeline path between the sensor and the rkisp1_isp */ + ret = data->pipe_.init(sensor, "rkisp1_isp"); + if (ret) { + LOG(RkISP1, Error) << "Failed to identify path from sensor to rkisp1_isp"; + return ret; + } + data->sensor_ = std::make_unique(sensor); ret = data->sensor_->init(); if (ret) @@ -1129,6 +1122,7 @@ int PipelineHandlerRkISP1::createCamera(MediaEntity *sensor) const std::string &id = data->sensor_->id(); std::shared_ptr camera = Camera::create(std::move(data), id, streams); + registerCamera(std::move(camera)); return 0; @@ -1136,8 +1130,6 @@ int PipelineHandlerRkISP1::createCamera(MediaEntity *sensor) bool PipelineHandlerRkISP1::match(DeviceEnumerator *enumerator) { - const MediaPad *pad; - DeviceMatch dm("rkisp1"); dm.add("rkisp1_isp"); dm.add("rkisp1_resizer_mainpath"); @@ -1162,22 +1154,6 @@ bool PipelineHandlerRkISP1::match(DeviceEnumerator *enumerator) if (isp_->open() < 0) return false; - /* Locate and open the optional CSI-2 receiver. */ - ispSink_ = isp_->entity()->getPadByIndex(0); - if (!ispSink_ || ispSink_->links().empty()) - return false; - - pad = ispSink_->links().at(0)->source(); - if (pad->entity()->function() == MEDIA_ENT_F_VID_IF_BRIDGE) { - csi_ = std::make_unique(pad->entity()); - if (csi_->open() < 0) - return false; - - ispSink_ = csi_->entity()->getPadByIndex(0); - if (!ispSink_) - return false; - } - /* Locate and open the stats and params video nodes. */ stat_ = V4L2VideoDevice::fromEntityName(media_, "rkisp1_stats"); if (stat_->open() < 0) @@ -1205,8 +1181,10 @@ bool PipelineHandlerRkISP1::match(DeviceEnumerator *enumerator) * camera instance for each of them. */ bool registered = false; - for (MediaLink *link : ispSink_->links()) { - if (!createCamera(link->source()->entity())) + + for (MediaEntity *entity : media_->locateEntities(MEDIA_ENT_F_CAM_SENSOR)) { + LOG(RkISP1, Info) << "Identified " << entity->name(); + if (!createCamera(entity)) registered = true; }