{"id":13238,"url":"https://patchwork.libcamera.org/api/1.1/patches/13238/?format=json","web_url":"https://patchwork.libcamera.org/patch/13238/","project":{"id":1,"url":"https://patchwork.libcamera.org/api/1.1/projects/1/?format=json","name":"libcamera","link_name":"libcamera","list_id":"libcamera_core","list_email":"libcamera-devel@lists.libcamera.org","web_url":"","scm_url":"","webscm_url":""},"msgid":"<20210805222436.6263-8-laurent.pinchart@ideasonboard.com>","date":"2021-08-05T22:24:33","name":"[libcamera-devel,07/10] libcamera: pipeline: simple: Add pipeline pad reservation mechanism","commit_ref":null,"pull_url":null,"state":"accepted","archived":false,"hash":"35fd4de416288f012e8d4745bb7fcd234212e626","submitter":{"id":2,"url":"https://patchwork.libcamera.org/api/1.1/people/2/?format=json","name":"Laurent Pinchart","email":"laurent.pinchart@ideasonboard.com"},"delegate":null,"mbox":"https://patchwork.libcamera.org/patch/13238/mbox/","series":[{"id":2314,"url":"https://patchwork.libcamera.org/api/1.1/series/2314/?format=json","web_url":"https://patchwork.libcamera.org/project/libcamera/list/?series=2314","date":"2021-08-05T22:24:26","name":"Concurrent camera support in simple pipeline handler","version":1,"mbox":"https://patchwork.libcamera.org/series/2314/mbox/"}],"comments":"https://patchwork.libcamera.org/api/patches/13238/comments/","check":"pending","checks":"https://patchwork.libcamera.org/api/patches/13238/checks/","tags":{},"headers":{"Return-Path":"","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id 1C1EFC323C\n\tfor ;\n\tThu, 5 Aug 2021 22:25:04 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id 073D0688CA;\n\tFri, 6 Aug 2021 00:25:02 +0200 (CEST)","from perceval.ideasonboard.com (perceval.ideasonboard.com\n\t[213.167.242.64])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id 127186884E\n\tfor ;\n\tFri, 6 Aug 2021 00:24:58 +0200 (CEST)","from pendragon.lan (62-78-145-57.bb.dnainternet.fi [62.78.145.57])\n\tby perceval.ideasonboard.com (Postfix) with ESMTPSA id AF7DE4FB\n\tfor ;\n\tFri, 6 Aug 2021 00:24:57 +0200 (CEST)"],"Authentication-Results":"lancelot.ideasonboard.com;\n\tdkim=fail reason=\"signature verification failed\" (1024-bit key;\n\tunprotected) header.d=ideasonboard.com header.i=@ideasonboard.com\n\theader.b=\"kwEcaz9A\"; dkim-atps=neutral","DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com;\n\ts=mail; t=1628202297;\n\tbh=yLlL0DLW9bNfQFknGVf71ywNPPHIqC9SNrVsLNOSPlE=;\n\th=From:To:Subject:Date:In-Reply-To:References:From;\n\tb=kwEcaz9AjdHgHb2J3Ajr3UhbyNK29Hj/VgIzkzniKeDnmNRVd9vSfya4DYJVbBYEY\n\tdRkat+FoMs4tLr2kZb9s0vcGwUzVdFSk88olw5UbRvhpbmDCO1jcPaq/zWeGhqnMvA\n\tQx861dVKFq0rnzu71x2uXYAt+efbSRSZHCNjvt0k=","From":"Laurent Pinchart ","To":"libcamera-devel@lists.libcamera.org","Date":"Fri, 6 Aug 2021 01:24:33 +0300","Message-Id":"<20210805222436.6263-8-laurent.pinchart@ideasonboard.com>","X-Mailer":"git-send-email 2.31.1","In-Reply-To":"<20210805222436.6263-1-laurent.pinchart@ideasonboard.com>","References":"<20210805222436.6263-1-laurent.pinchart@ideasonboard.com>","MIME-Version":"1.0","Content-Transfer-Encoding":"8bit","Subject":"[libcamera-devel] [PATCH 07/10] libcamera: pipeline: simple: Add\n\tpipeline pad reservation mechanism","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" "},"content":"The cameras created by the same pipeline handler instance may share\nhardware resources, prohibiting usage of multiple cameras concurrently.\nImplement a heuristic to reserve resources and handle mutual exclusiong\nin a generic way.\n\nSigned-off-by: Laurent Pinchart \n---\n src/libcamera/pipeline/simple/simple.cpp | 95 +++++++++++++++++++++++-\n 1 file changed, 93 insertions(+), 2 deletions(-)","diff":"diff --git a/src/libcamera/pipeline/simple/simple.cpp b/src/libcamera/pipeline/simple/simple.cpp\nindex 17ef3f43ad41..b32369cae701 100644\n--- a/src/libcamera/pipeline/simple/simple.cpp\n+++ b/src/libcamera/pipeline/simple/simple.cpp\n@@ -121,6 +121,28 @@ LOG_DEFINE_CATEGORY(SimplePipeline)\n * the pixel formats and sizes that the converter can produce for the output of\n * the capture video node, and stores the information in the outputFormats and\n * outputSizes of the SimpleCameraData::Configuration structure.\n+ *\n+ * Concurrent Access to Cameras\n+ * ----------------------------\n+ *\n+ * The cameras created by the same pipeline handler instance may share hardware\n+ * resources. For instances, a platform may have multiple CSI-2 receivers but a\n+ * single DMA engine, prohibiting usage of multiple cameras concurrently. This\n+ * depends heavily on the hardware architecture, which the simple pipeline\n+ * handler has no a priori knowledge of. The pipeline handler thus implements a\n+ * heuristic to handle sharing of hardware resources in a generic fashion.\n+ *\n+ * Two cameras are considered to be mutually exclusive if their share common\n+ * pads along the pipeline from the camera sensor to the video node. An entity\n+ * can thus be used concurrently by multiple cameras, as long as pads are\n+ * distinct.\n+ *\n+ * A resource reservation mechanism is implemented by the SimplePipelineHandler\n+ * acquirePipeline() and releasePipeline() functions to manage exclusive access\n+ * to pads. A camera reserves all the pads present in its pipeline when it is\n+ * started, and the start() function returns an error if any of the required\n+ * pads is already in use. When the camera is stopped, the pads it has reserved\n+ * are released.\n */\n \n class SimplePipelineHandler;\n@@ -266,6 +288,7 @@ private:\n \tstruct EntityData {\n \t\tstd::unique_ptr video;\n \t\tstd::unique_ptr subdev;\n+\t\tstd::map owners;\n \t};\n \n \tSimpleCameraData *cameraData(const Camera *camera)\n@@ -276,6 +299,9 @@ private:\n \n \tstd::vector locateSensors();\n \n+\tconst MediaPad *acquirePipeline(SimpleCameraData *data);\n+\tvoid releasePipeline(SimpleCameraData *data);\n+\n \tvoid bufferReady(FrameBuffer *buffer);\n \tvoid converterInputDone(FrameBuffer *buffer);\n \tvoid converterOutputDone(FrameBuffer *buffer);\n@@ -841,6 +867,14 @@ int SimplePipelineHandler::start(Camera *camera, [[maybe_unused]] const ControlL\n \tV4L2VideoDevice *video = data->video_;\n \tint ret;\n \n+\tconst MediaPad *pad = acquirePipeline(data);\n+\tif (pad) {\n+\t\tLOG(SimplePipeline, Info)\n+\t\t\t<< \"Failed to acquire pipeline, entity \"\n+\t\t\t<< pad->entity()->name() << \" in use\";\n+\t\treturn -EBUSY;\n+\t}\n+\n \tif (data->useConverter_) {\n \t\t/*\n \t\t * When using the converter allocate a fixed number of internal\n@@ -853,8 +887,10 @@ int SimplePipelineHandler::start(Camera *camera, [[maybe_unused]] const ControlL\n \t\tStream *stream = &data->streams_[0];\n \t\tret = video->importBuffers(stream->configuration().bufferCount);\n \t}\n-\tif (ret < 0)\n+\tif (ret < 0) {\n+\t\treleasePipeline(data);\n \t\treturn ret;\n+\t}\n \n \tret = video->streamOn();\n \tif (ret < 0) {\n@@ -892,6 +928,8 @@ void SimplePipelineHandler::stop(Camera *camera)\n \n \tdata->converterBuffers_.clear();\n \tactiveCamera_ = nullptr;\n+\n+\treleasePipeline(data);\n }\n \n int SimplePipelineHandler::queueRequestDevice(Camera *camera, Request *request)\n@@ -1101,7 +1139,7 @@ bool SimplePipelineHandler::match(DeviceEnumerator *enumerator)\n \t\t\tbreak;\n \t\t}\n \n-\t\tentities_[entity] = { std::move(video), std::move(subdev) };\n+\t\tentities_[entity] = { std::move(video), std::move(subdev), {} };\n \t}\n \n \t/* Initialize each pipeline and register a corresponding camera. */\n@@ -1144,6 +1182,59 @@ V4L2Subdevice *SimplePipelineHandler::subdev(const MediaEntity *entity)\n \treturn iter->second.subdev.get();\n }\n \n+/**\n+ * \\brief Acquire all resources needed by the camera pipeline\n+ * \\return nullptr on success, a pointer to the contended pad on error\n+ */\n+const MediaPad *SimplePipelineHandler::acquirePipeline(SimpleCameraData *data)\n+{\n+\tfor (const SimpleCameraData::Entity &entity : data->entities_) {\n+\t\tconst EntityData &edata = entities_[entity.entity];\n+\n+\t\tif (entity.sink) {\n+\t\t\tauto iter = edata.owners.find(entity.sink);\n+\t\t\tif (iter != edata.owners.end() && iter->second != data)\n+\t\t\t\treturn entity.sink;\n+\t\t}\n+\n+\t\tif (entity.source) {\n+\t\t\tauto iter = edata.owners.find(entity.source);\n+\t\t\tif (iter != edata.owners.end() && iter->second != data)\n+\t\t\t\treturn entity.source;\n+\t\t}\n+\t}\n+\n+\tfor (const SimpleCameraData::Entity &entity : data->entities_) {\n+\t\tEntityData &edata = entities_[entity.entity];\n+\n+\t\tif (entity.sink)\n+\t\t\tedata.owners[entity.sink] = data;\n+\t\tif (entity.source)\n+\t\t\tedata.owners[entity.source] = data;\n+\t}\n+\n+\treturn nullptr;\n+}\n+\n+void SimplePipelineHandler::releasePipeline(SimpleCameraData *data)\n+{\n+\tfor (const SimpleCameraData::Entity &entity : data->entities_) {\n+\t\tEntityData &edata = entities_[entity.entity];\n+\n+\t\tif (entity.sink) {\n+\t\t\tauto iter = edata.owners.find(entity.sink);\n+\t\t\tASSERT(iter->second == data);\n+\t\t\tedata.owners.erase(iter);\n+\t\t}\n+\n+\t\tif (entity.source) {\n+\t\t\tauto iter = edata.owners.find(entity.source);\n+\t\t\tASSERT(iter->second == data);\n+\t\t\tedata.owners.erase(iter);\n+\t\t}\n+\t}\n+}\n+\n /* -----------------------------------------------------------------------------\n * Buffer Handling\n */\n","prefixes":["libcamera-devel","07/10"]}