From patchwork Wed Dec 8 15:15:26 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Naushir Patuck X-Patchwork-Id: 15082 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 7FA0DC324B for ; Wed, 8 Dec 2021 15:15:35 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 2C00160822; Wed, 8 Dec 2021 16:15:35 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="SZ+ja6Bu"; dkim-atps=neutral Received: from mail-wr1-x429.google.com (mail-wr1-x429.google.com [IPv6:2a00:1450:4864:20::429]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 2C96860725 for ; Wed, 8 Dec 2021 16:15:33 +0100 (CET) Received: by mail-wr1-x429.google.com with SMTP id a18so4641637wrn.6 for ; Wed, 08 Dec 2021 07:15:33 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=cqUEfJUvmoLjYb1FisS+gsYKY5V17Z8BcEWse6YTuFg=; b=SZ+ja6Bu7Kiowovq+z5zTelKqNhxofsZbkaHj4hfDU79U1Guw6/per8JtCggCdzu3d ft5pKmr1v/Z7lmOEhsjTrrVdcSJZskCHLhzYmjzG0DllqJrmsyXJX1q9h9RQOxQLY0Oy obqgZtLo9GpT4g7hxLHSli8H23dTAV+5NCcJos+C5gyJj5lgzREl3dS62+vkzAB1D/Zv SSWRnCDOrwDhdRdB2cFW9JEh3s3konn6o2COlsp7hLH+OLAH4yFh4q786dE+VYxGuF5o rHNe1LcJIcEDeI4D8y/KTAt9hWFYvxJS7OdSZxqVplotez15dMZLjPYU6hOnZ4hIQLJj yXBw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=cqUEfJUvmoLjYb1FisS+gsYKY5V17Z8BcEWse6YTuFg=; b=qS86Uc4khEgk+cFTmDtCy6Yw3klYaPndXkFspVqsW8pXt58zy+VAiudj3eu+sPjYMJ JIIJ7KM6sMCVDLPAOwkpREt8wM6CrXdDLUEw073waHYU2r4yIYnBEwySe24m8j4ZVABL U0O2aRSLZqGCxNN4HAHyJwYNq61+TzHHu1VtWHwcFtwzpVmrJn5mnk0yWAidH5I0ECpt 2DS8G3dbDwGMdj0CcjeJEkVMDdJavqZHTR/LojBfdrGFkCTPLyY/2uUOpFCgMDJzkBP5 Ojyz/WFXv0gEvdWdZ6Fgiy9iZbN/yCsMTzSjM5QLyFs1hWp1im6ZpOhf0N+HtsuhIOAb Rm0Q== X-Gm-Message-State: AOAM532E+N2skKqOJf5E4G6ADFGE9a7dl30De66/KAip0d676i3yWiDw QxrjxJixsmh1nao2l2JqGMqyd1rYVkKsujGM X-Google-Smtp-Source: ABdhPJwE92UpEmfR7baQvGrWc7MPB9WyeLiIiZQcMbZGCbqIa5sGqshEpVuPfUXFNqu6fgwZHk0IcA== X-Received: by 2002:a5d:668c:: with SMTP id l12mr61677928wru.19.1638976532388; Wed, 08 Dec 2021 07:15:32 -0800 (PST) Received: from naush-laptop.pitowers.org ([2a00:1098:3142:14:27d5:9305:f572:6623]) by smtp.gmail.com with ESMTPSA id 10sm3998857wrb.75.2021.12.08.07.15.31 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 08 Dec 2021 07:15:31 -0800 (PST) From: Naushir Patuck To: libcamera-devel@lists.libcamera.org Date: Wed, 8 Dec 2021 15:15:26 +0000 Message-Id: <20211208151527.1404715-2-naush@raspberrypi.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20211208151527.1404715-1-naush@raspberrypi.com> References: <20211208151527.1404715-1-naush@raspberrypi.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v2 1/2] pipeline: raspberrypi: Move sensor entity detection out of registerCamera() X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Enumerate the sensor device entities in PipelineHandlerRPi::match() and loop over PipelineHandlerRPi::registerCamera() for each sensor found. This will allow the pipeline handler to register multiple cameras attached to a single Unicam instance with a Video Mux device. Signed-off-by: Naushir Patuck Reviewed-by: Kieran Bingham --- .../pipeline/raspberrypi/raspberrypi.cpp | 35 +++++++++++-------- 1 file changed, 20 insertions(+), 15 deletions(-) diff --git a/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp b/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp index 86851ac467ad..756878c70036 100644 --- a/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp +++ b/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp @@ -310,7 +310,7 @@ private: return static_cast(camera->_d()); } - int registerCamera(MediaDevice *unicam, MediaDevice *isp); + int registerCamera(MediaDevice *unicam, MediaDevice *isp, MediaEntity *sensorEntity); int queueAllBuffers(Camera *camera); int prepareBuffers(Camera *camera); void freeBuffers(Camera *camera); @@ -1029,16 +1029,28 @@ bool PipelineHandlerRPi::match(DeviceEnumerator *enumerator) return false; } - int ret = registerCamera(unicamDevice, ispDevice); - if (ret) { - LOG(RPI, Error) << "Failed to register camera: " << ret; - return false; + /* + * The loop below is used to register multiple cameras behind one or more + * video mux devices that are attaced to a particular Unicam instance. + * Obviously these cameras cannot be used simultaneously. + */ + unsigned int numCameras = 0; + for (MediaEntity *entity : unicamDevice->entities()) { + if (entity->function() != MEDIA_ENT_F_CAM_SENSOR) + continue; + + int ret = registerCamera(unicamDevice, ispDevice, entity); + if (ret) + LOG(RPI, Error) << "Failed to register camera " + << entity->name() << ": " << ret; + else + numCameras++; } - return true; + return !!numCameras; } -int PipelineHandlerRPi::registerCamera(MediaDevice *unicam, MediaDevice *isp) +int PipelineHandlerRPi::registerCamera(MediaDevice *unicam, MediaDevice *isp, MediaEntity *sensorEntity) { std::unique_ptr data = std::make_unique(this); @@ -1079,14 +1091,7 @@ int PipelineHandlerRPi::registerCamera(MediaDevice *unicam, MediaDevice *isp) data->isp_[Isp::Output1].dev()->bufferReady.connect(data.get(), &RPiCameraData::ispOutputDequeue); data->isp_[Isp::Stats].dev()->bufferReady.connect(data.get(), &RPiCameraData::ispOutputDequeue); - /* Identify the sensor. */ - for (MediaEntity *entity : unicam->entities()) { - if (entity->function() == MEDIA_ENT_F_CAM_SENSOR) { - data->sensor_ = std::make_unique(entity); - break; - } - } - + data->sensor_ = std::make_unique(sensorEntity); if (!data->sensor_) return -EINVAL; From patchwork Wed Dec 8 15:15:27 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Naushir Patuck X-Patchwork-Id: 15083 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id D5762BF415 for ; Wed, 8 Dec 2021 15:15:37 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 842ED6088E; Wed, 8 Dec 2021 16:15:37 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="kC8s62K+"; dkim-atps=neutral Received: from mail-wm1-x32b.google.com (mail-wm1-x32b.google.com [IPv6:2a00:1450:4864:20::32b]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 4BEC660725 for ; Wed, 8 Dec 2021 16:15:34 +0100 (CET) Received: by mail-wm1-x32b.google.com with SMTP id p18so2000996wmq.5 for ; Wed, 08 Dec 2021 07:15:34 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=KRWYQebGKAW4ly9N4vPVLysxEY4G5yLfOO4ApQ+gj60=; b=kC8s62K+LND9eqn2ugLndMVaLiBarhqmfTEMxOqprDqm4geqFGVIzmmvzsqJJIG7nD 5tQxcQIteyzNwZfd0bq6i9FfQw5g3zgqroyzXlm1HjWQaAP93VXo/AhqcSnNlLO3MB0b 2b0W+Kkn1MhH3zeukfW1gY+l8m3EUj9fFLI0qv0LZbuYS9HghXtPXNLf2mLx6tiA5ZzP ur3VmlUsmy3xpg/ttbk1K9guwVCn81dAVw/K8A4XehI/czPJbwR/bZX7P4XyfQNb763m 0LEP6HFB2XP7zuRF9uQHuEIiY7lRGNya/L3pPkjW5D7uf/6tUACh5smvCWQF2oE8NgCB /CEA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=KRWYQebGKAW4ly9N4vPVLysxEY4G5yLfOO4ApQ+gj60=; b=mOMnnlTZW8Zm+inA9disvuldbdjY8Bwkn9V8op8gZwpaUm6s3NfcAtKOI4vP1u0C6v 6KUpBtoIA/igfQlrYt65MZG4Jp+MZPZcHgqnpQUYf5WZpnanAMWfRZRMfHFKgtfDOc6j RQFmk5qWteJ1XmOQy1uP19ejFzsi1Q9Ia6bLR+e31DCIXmS+NqRhLe/gkevX6bwohv3M 864TqDnTl4HzPyABC5ciHhqLPaUdMtHdMuv/y3FEcsZ7YSPWEGFfKb9sa3QHmNkRwedA e8qukDsslBN77fn3ota0bCWNX6EcARp3MPJl5LcQM9Pc3GvFF8YDonY93f2DMBW6gz9W F5aQ== X-Gm-Message-State: AOAM532B3dyVhTxJp3gcLAg+opO9NR0ocuVAZGKQ9tXQOxKXjEdYaAml G7t1vkgGKMRFf6S9x23+nzqvwjIWz7K0j7FN X-Google-Smtp-Source: ABdhPJyxgGBVVu3yaG7ceENTLsr+q4hq82aATIt7ph8u/SKcf6EpXDGmDUJMWbSVcZmyrP5rwGL/uA== X-Received: by 2002:a05:600c:1c1a:: with SMTP id j26mr17284377wms.28.1638976533529; Wed, 08 Dec 2021 07:15:33 -0800 (PST) Received: from naush-laptop.pitowers.org ([2a00:1098:3142:14:27d5:9305:f572:6623]) by smtp.gmail.com with ESMTPSA id 10sm3998857wrb.75.2021.12.08.07.15.32 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 08 Dec 2021 07:15:32 -0800 (PST) From: Naushir Patuck To: libcamera-devel@lists.libcamera.org Date: Wed, 8 Dec 2021 15:15:27 +0000 Message-Id: <20211208151527.1404715-3-naush@raspberrypi.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20211208151527.1404715-1-naush@raspberrypi.com> References: <20211208151527.1404715-1-naush@raspberrypi.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v2 2/2] pipeline: raspberrypi: Add support for Video Mux and Bridge devices X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" This change will allow the pipeline handler to enumerate and control Video Mux or Bridge devices that may be attached between sensors and a particular Unicam instance. Cascaded mux or bridge devices are also handled. A new member function enumerateVideoDevices(), called from registerCamera(), is used to identify and open all mux and bridge subdevices present in the sensor -> Unicam link. Relevent links are enabled/disabled and pad formats correctly set in configure() before the camera is started. Signed-off-by: Naushir Patuck Reviewed-by: Kieran Bingham --- .../pipeline/raspberrypi/raspberrypi.cpp | 78 +++++++++++++++++++ 1 file changed, 78 insertions(+) diff --git a/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp b/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp index 756878c70036..ca176ecb40ec 100644 --- a/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp +++ b/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp @@ -12,6 +12,7 @@ #include #include #include +#include #include #include @@ -220,6 +221,11 @@ public: std::vector streams_; /* Stores the ids of the buffers mapped in the IPA. */ std::unordered_set ipaBuffers_; + /* + * Stores a cascade of Video Mux or Bridge devices between the sensor and + * Unicam together with media link across the entities. + */ + std::vector, MediaLink *>> bridgeDevices_; /* DMAHEAP allocation helper. */ RPi::DmaHeap dmaHeap_; @@ -311,6 +317,7 @@ private: } int registerCamera(MediaDevice *unicam, MediaDevice *isp, MediaEntity *sensorEntity); + void enumerateVideoDevices(RPiCameraData *data, MediaLink *link); int queueAllBuffers(Camera *camera); int prepareBuffers(Camera *camera); void freeBuffers(Camera *camera); @@ -868,6 +875,25 @@ int PipelineHandlerRPi::configure(Camera *camera, CameraConfiguration *config) */ data->properties_.set(properties::ScalerCropMaximum, data->sensorInfo_.analogCrop); + /* Setup the Video Mux/Bridge entities. */ + for (auto &[device, link] : data->bridgeDevices_) { + /* Start by disabling all the sink pad links on the devices in the cascade. */ + for (const MediaPad *p : device->entity()->pads()) { + if (!(p->flags() & MEDIA_PAD_FL_SINK)) + continue; + + for (MediaLink *l : p->links()) + l->setEnabled(false); + } + + /* Finally enable the link, and setup the pad format. */ + link->setEnabled(true); + const MediaPad *srcPad = link->sink(); + ret = device->setFormat(srcPad->index(), &sensorFormat); + LOG(RPI, Info) << "Configured media link on device " << device->entity()->name() + << " at pad " << srcPad->index(); + } + return ret; } @@ -1098,6 +1124,13 @@ int PipelineHandlerRPi::registerCamera(MediaDevice *unicam, MediaDevice *isp, Me if (data->sensor_->init()) return -EINVAL; + /* + * Enumerate all the Video Mux/Bridge devices across the sensor -> unicam + * link. There may be a cascade of devices in this link! + */ + MediaLink *link = sensorEntity->getPadByIndex(0)->links()[0]; + enumerateVideoDevices(data.get(), link); + data->sensorFormats_ = populateSensorFormats(data->sensor_); ipa::RPi::SensorConfig sensorConfig; @@ -1224,6 +1257,51 @@ int PipelineHandlerRPi::registerCamera(MediaDevice *unicam, MediaDevice *isp, Me return 0; } +void PipelineHandlerRPi::enumerateVideoDevices(RPiCameraData *data, MediaLink *link) +{ + const MediaPad *sinkPad = link->sink(); + const MediaEntity *entity = sinkPad->entity(); + bool unicamFound = false; + + /* We only deal with Video Mux and Bridge devices in cascade. */ + if (entity->function() != MEDIA_ENT_F_VID_MUX && + entity->function() != MEDIA_ENT_F_VID_IF_BRIDGE) + return; + + LOG(RPI, Info) << "Found video mux device " << entity->name() + << " linked to sink pad " << sinkPad->index(); + + data->bridgeDevices_.emplace_back(std::make_unique(entity), link); + data->bridgeDevices_.back().first->open(); + + for (const MediaPad *pad : entity->pads()) { + /* + * Iterate through all the sink pads down the cascade to find any + * other Video Mux and Bridge devices. + */ + if (!(pad->flags() & MEDIA_PAD_FL_SOURCE)) + continue; + + for (MediaLink *l : pad->links()) { + enumerateVideoDevices(data, l); + if (l->sink()->entity()->name() == "unicam-image") + unicamFound = true; + } + } + + /* This identifies the end of our entity enumeration recursion. */ + if (link->source()->entity()->function() == MEDIA_ENT_F_CAM_SENSOR) { + /* + * If Unicam is not at the end of this cascade, we cannot configure + * this topology automatically, so remove all entity references. + */ + if (!unicamFound) { + LOG(RPI, Warning) << "Cannot automatically configure this MC topology!"; + data->bridgeDevices_.clear(); + } + } +} + int PipelineHandlerRPi::queueAllBuffers(Camera *camera) { RPiCameraData *data = cameraData(camera);