From patchwork Wed Dec 1 11:57:10 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Naushir Patuck X-Patchwork-Id: 14957 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id BE590C3250 for ; Wed, 1 Dec 2021 11:57:24 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 5B5EB6059C; Wed, 1 Dec 2021 12:57:24 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="N2EhLhdM"; dkim-atps=neutral Received: from mail-wr1-x436.google.com (mail-wr1-x436.google.com [IPv6:2a00:1450:4864:20::436]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 1B3A46011A for ; Wed, 1 Dec 2021 12:57:23 +0100 (CET) Received: by mail-wr1-x436.google.com with SMTP id t9so34685231wrx.7 for ; Wed, 01 Dec 2021 03:57:23 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=49BdeSGrYkKnPrwypx/xXitVPSiEQrgJRyauOS4rVKk=; b=N2EhLhdM4V/UCs7rKPwLdbGOsIOIGRC9NzE2y6adyYSWSmJRCGTR3C17kySEB0cbQI xl/kavMlG757YjOBr3lFywwhaGKY1lgzhlMfT9ndLu4cCo1oRBq6J3785q2A9uUNwbwb J2nAQkKVmoBqhyfPC4PQSL5i1sEssTZRgZU/XOSnARJ/RFC1FvvELt06iPODzrKYY5kP gHLEnt1GP8iYk8S+6f4h72ZIuu6fYFa35/B3bjOt1i6vmToVMvjWA3qnDCevF4ZH0skV NkWOiPfl6ynyKKapXQs5uexX4Uh/697fQNYPKZBOrgezGAnkF9Q27G/kpn5sSVTnJR26 CDzg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=49BdeSGrYkKnPrwypx/xXitVPSiEQrgJRyauOS4rVKk=; b=d1LiSOBw8uzNipg+KOK/xN3tV3PMG5tAnhcIOi9SSGmUHXY3NPIFSAXKiZtK61SQ3R r5kiA2SGVCrXY+oEfA29pweJPx/g83UVwVY6Y+xEyOLBIWmr26wyCZZF4sboSjMnWs83 cx9UxY19NjtC+Eetptz+yXwDVL4zpqcdfAtG+hV8tz3gf1SFyX6jk6GZAOt5nYXsrLl1 G9s3Cq2zRRDSobfC86frKm5nkds2q0QwxTvpcRp3GwoNVfuNeCOj44IVUpKupOVlR3HN 0zYVT6ESF7Mhez3esOJAToZ6DElzN96O7Ij7uO6w15NmciQKLZRCgpyqCeT1iawEwzlW myBA== X-Gm-Message-State: AOAM531YbKUoJ/2b7TVgaH/ErpO0qDSM8kx75Sd7zVGMTPs3TUMrntBu /2HZST+14v2D4SSMD7qe4uY+5XWZ1F5VG8x1 X-Google-Smtp-Source: ABdhPJz3IYmo9Vo01X35Sipk10+KNhL3FlRvO/nYO1i/CwMgOCs8hIpLMK8Xxg8mXxau30tb2noIDA== X-Received: by 2002:a5d:4a0a:: with SMTP id m10mr5962292wrq.221.1638359842544; Wed, 01 Dec 2021 03:57:22 -0800 (PST) Received: from naush-laptop.pitowers.org ([2a00:1098:3142:14:9907:13eb:bd49:5862]) by smtp.gmail.com with ESMTPSA id f13sm1139451wmq.29.2021.12.01.03.57.21 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 01 Dec 2021 03:57:22 -0800 (PST) From: Naushir Patuck To: libcamera-devel@lists.libcamera.org Date: Wed, 1 Dec 2021 11:57:10 +0000 Message-Id: <20211201115711.1017822-2-naush@raspberrypi.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20211201115711.1017822-1-naush@raspberrypi.com> References: <20211201115711.1017822-1-naush@raspberrypi.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v1 1/2] pipeline: raspberrypi: Move sensor entity detection out of registerCamera() X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Enumerate the sensor device entities in PipelineHandlerRPi::match() and loop over PipelineHandlerRPi::registerCamera() for each sensor found. This will allow the pipeline handler to register multiple cameras attached to a single Unicam instance with a Video Mux device. Signed-off-by: Naushir Patuck Reviewed-by: Kieran Bingham --- .../pipeline/raspberrypi/raspberrypi.cpp | 35 +++++++++++-------- 1 file changed, 20 insertions(+), 15 deletions(-) diff --git a/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp b/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp index e31fa3b23859..811160490f5c 100644 --- a/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp +++ b/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp @@ -314,7 +314,7 @@ private: return static_cast(camera->_d()); } - int registerCamera(MediaDevice *unicam, MediaDevice *isp); + int registerCamera(MediaDevice *unicam, MediaDevice *isp, MediaEntity *sensorEntity); int queueAllBuffers(Camera *camera); int prepareBuffers(Camera *camera); void freeBuffers(Camera *camera); @@ -1009,16 +1009,28 @@ bool PipelineHandlerRPi::match(DeviceEnumerator *enumerator) return false; } - int ret = registerCamera(unicamDevice, ispDevice); - if (ret) { - LOG(RPI, Error) << "Failed to register camera: " << ret; - return false; + /* + * The loop below is used to register multiple cameras behind one or more + * video mux devices that are attaced to a particular Unicam instance. + * Obviously these cameras cannot be used simultaneously. + */ + unsigned int numCameras = 0; + for (MediaEntity *entity : unicamDevice->entities()) { + if (entity->function() != MEDIA_ENT_F_CAM_SENSOR) + continue; + + int ret = registerCamera(unicamDevice, ispDevice, entity); + if (ret) + LOG(RPI, Error) << "Failed to register camera " + << entity->name() << ": " << ret; + else + numCameras++; } - return true; + return !!numCameras; } -int PipelineHandlerRPi::registerCamera(MediaDevice *unicam, MediaDevice *isp) +int PipelineHandlerRPi::registerCamera(MediaDevice *unicam, MediaDevice *isp, MediaEntity *sensorEntity) { std::unique_ptr data = std::make_unique(this); @@ -1059,14 +1071,7 @@ int PipelineHandlerRPi::registerCamera(MediaDevice *unicam, MediaDevice *isp) data->isp_[Isp::Output1].dev()->bufferReady.connect(data.get(), &RPiCameraData::ispOutputDequeue); data->isp_[Isp::Stats].dev()->bufferReady.connect(data.get(), &RPiCameraData::ispOutputDequeue); - /* Identify the sensor. */ - for (MediaEntity *entity : unicam->entities()) { - if (entity->function() == MEDIA_ENT_F_CAM_SENSOR) { - data->sensor_ = std::make_unique(entity); - break; - } - } - + data->sensor_ = std::make_unique(sensorEntity); if (!data->sensor_) return -EINVAL; From patchwork Wed Dec 1 11:57:11 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Naushir Patuck X-Patchwork-Id: 14958 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 3ECD4BF415 for ; Wed, 1 Dec 2021 11:57:26 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id E1A516072B; Wed, 1 Dec 2021 12:57:25 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="h7UfYQCn"; dkim-atps=neutral Received: from mail-wr1-x430.google.com (mail-wr1-x430.google.com [IPv6:2a00:1450:4864:20::430]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id D0FD460729 for ; Wed, 1 Dec 2021 12:57:23 +0100 (CET) Received: by mail-wr1-x430.google.com with SMTP id u1so51619362wru.13 for ; Wed, 01 Dec 2021 03:57:23 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=XVI1aAlrGYw5+gRkS8jX1UKJLOe2buBKYIv9RhJ0V4I=; b=h7UfYQCnegSZM7gPvFg8mMw9aFhEjJOsj9p1kVOHmkrbDiGnMzw/2HskWHZwyGl14I d0VeQp/4A4VYmDv7lCjGqZEEP604+FjPkVE+T7DezEwk5+CjoMp4oSra2C6jvJBPiE9T p+bThoTdBs0TItTbZtQr3sITMvYK13L5+ZrGId2EQnzV7JaiR395hsBNRXMyM0tEpTmP DKAhOEoZ/WkmX8oXspI8wViflFslgw08+9yu01Nx1UQs39UvjPRCdGW3PSHGGQp/PBys Pmu4SfNKuYZvAoLquCTF1rTEvVBk5K+yBWUYQbhiaS2eet7ne+8MySzqaB0ouumJ4FtT Q2NA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=XVI1aAlrGYw5+gRkS8jX1UKJLOe2buBKYIv9RhJ0V4I=; b=it4v4BLGnOMynNpxc3gscJj/jv97d0cL9G/lnDBYfvCFuzFObn5GVWqQkSTrq+sKzb xa58yaH0vtf5YTNF8W/8HP/5A8n/3JcmnmCKkAZCm3pIOvbUsQjECKaWUscTKyZhBDua ODRKg9dHHEKRXmwXXshSAqO8fCvZ79Ux1EvtaGHtpi/C0M0J4YKJEJeb1qHttTSprCp6 yeFM2JlsYyBp/k+oBQ1tpBvLtpZIE/qPCd7jVn7n7H9AxxzKiBSz22JMnNdCjspDXyn2 QAxf0gmpN/tATnpK2NP9SvmshuaSYNTLVmoDaJ0VYrmNvw3KJPHK/O3zTLaLuxqv0eOb Rshg== X-Gm-Message-State: AOAM533E37XlOLSTqP9TK8jPrf8Pqf1jRdX6KHGWz4iAbNTeNMoGYLgJ TlHnXVwHeebxghLU+ra5DoYUBkEf7a1/nEwj X-Google-Smtp-Source: ABdhPJx2NZipyKwJlesWgfqkRKzW7FTmUQ9sJZUxV+CRCIP40tj8pMrYu51bWbzSLYX4Yf20V+cmTw== X-Received: by 2002:adf:a404:: with SMTP id d4mr6261264wra.556.1638359843312; Wed, 01 Dec 2021 03:57:23 -0800 (PST) Received: from naush-laptop.pitowers.org ([2a00:1098:3142:14:9907:13eb:bd49:5862]) by smtp.gmail.com with ESMTPSA id f13sm1139451wmq.29.2021.12.01.03.57.22 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 01 Dec 2021 03:57:22 -0800 (PST) From: Naushir Patuck To: libcamera-devel@lists.libcamera.org Date: Wed, 1 Dec 2021 11:57:11 +0000 Message-Id: <20211201115711.1017822-3-naush@raspberrypi.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20211201115711.1017822-1-naush@raspberrypi.com> References: <20211201115711.1017822-1-naush@raspberrypi.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v1 2/2] pipeline: raspberrypi: Add support for Video Mux and Bridge devices X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" This change will allow the pipeline handler to enumerate and control Video Mux or Bridge devices that may be attached between sensors and a particular Unicam instance. Cascaded mux or bridge devices are also handled. A new member function enumerateVideoDevices(), called from registerCamera(), is used to identify and open all mux and bridge subdevices present in the sensor -> Unicam link. Relevent links are enabled/disabled and pad formats correctly set in configure() before the camera is started. Signed-off-by: Naushir Patuck Reviewed-by: Kieran Bingham --- .../pipeline/raspberrypi/raspberrypi.cpp | 60 +++++++++++++++++++ 1 file changed, 60 insertions(+) diff --git a/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp b/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp index 811160490f5c..2512d0746d4a 100644 --- a/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp +++ b/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp @@ -11,6 +11,7 @@ #include #include #include +#include #include #include @@ -224,6 +225,11 @@ public: std::vector streams_; /* Stores the ids of the buffers mapped in the IPA. */ std::unordered_set ipaBuffers_; + /* + * Stores a cascade of Video Mux or Bridge devices between the sensor and + * Unicam together with the sink pad of the entity. + */ + std::vector, const MediaPad *>> videoDevices; /* DMAHEAP allocation helper. */ RPi::DmaHeap dmaHeap_; @@ -315,6 +321,7 @@ private: } int registerCamera(MediaDevice *unicam, MediaDevice *isp, MediaEntity *sensorEntity); + void enumerateVideoDevices(RPiCameraData *data, const MediaPad *sinkPad); int queueAllBuffers(Camera *camera); int prepareBuffers(Camera *camera); void freeBuffers(Camera *camera); @@ -848,6 +855,24 @@ int PipelineHandlerRPi::configure(Camera *camera, CameraConfiguration *config) */ data->properties_.set(properties::ScalerCropMaximum, data->sensorInfo_.analogCrop); + /* Setup the Video Mux/Bridge entities. */ + for (auto &[device, pad] : data->videoDevices) { + /* Start by disabling all the sink pad links on the devices in the cascade. */ + for (const MediaPad *p : device->entity()->pads()) { + if (!(p->flags() & MEDIA_PAD_FL_SINK)) + continue; + + for (MediaLink *l : p->links()) + l->setEnabled(false); + } + + /* Finally enable the link to this camera, and setup the pad format. */ + data->sensor_->entity()->pads()[0]->links()[0]->setEnabled(true); + ret = device->setFormat(pad->index(), &sensorFormat); + LOG(RPI, Info) << "Configured media link on device " << device->entity()->name() + << " at pad " << pad->index(); + } + return ret; } @@ -1078,6 +1103,13 @@ int PipelineHandlerRPi::registerCamera(MediaDevice *unicam, MediaDevice *isp, Me if (data->sensor_->init()) return -EINVAL; + /* + * Enumerate all the Video Mux/Bridge devices across the sensor -> unicam + * link. There may be a cascade of devices in this link! + */ + MediaPad *sensorSinkPad = sensorEntity->getPadByIndex(0)->links()[0]->sink(); + enumerateVideoDevices(data.get(), sensorSinkPad); + data->sensorFormats_ = populateSensorFormats(data->sensor_); ipa::RPi::SensorConfig sensorConfig; @@ -1204,6 +1236,34 @@ int PipelineHandlerRPi::registerCamera(MediaDevice *unicam, MediaDevice *isp, Me return 0; } +void PipelineHandlerRPi::enumerateVideoDevices(RPiCameraData *data, const MediaPad *sinkPad) +{ + const MediaEntity *entity = sinkPad->entity(); + + /* We only deal with Video Mux and Bridge devices in cascade. */ + if (entity->function() != MEDIA_ENT_F_VID_MUX && + entity->function() != MEDIA_ENT_F_VID_IF_BRIDGE) + return; + + LOG(RPI, Info) << "Found video mux device " << entity->name() + << " linked to sink pad " << sinkPad->index(); + + data->videoDevices.emplace_back(std::make_unique(entity), sinkPad); + data->videoDevices.back().first->open(); + + for (const MediaPad *pad : entity->pads()) { + /* + * Iterate through all the sink pads down the cascade to find any + * other Video Mux and Bridge devices. + */ + if (!(pad->flags() & MEDIA_PAD_FL_SOURCE)) + continue; + + for (const MediaLink *l : pad->links()) + enumerateVideoDevices(data, l->sink()); + } +} + int PipelineHandlerRPi::queueAllBuffers(Camera *camera) { RPiCameraData *data = cameraData(camera);