{"id":17982,"url":"https://patchwork.libcamera.org/api/1.1/patches/17982/?format=json","web_url":"https://patchwork.libcamera.org/patch/17982/","project":{"id":1,"url":"https://patchwork.libcamera.org/api/1.1/projects/1/?format=json","name":"libcamera","link_name":"libcamera","list_id":"libcamera_core","list_email":"libcamera-devel@lists.libcamera.org","web_url":"","scm_url":"","webscm_url":""},"msgid":"<20221212150256.69004-3-umang.jain@ideasonboard.com>","date":"2022-12-12T15:02:56","name":"[libcamera-devel,2/2] libcamera: pipeline-handler: Consider max in-flight requests constraint","commit_ref":null,"pull_url":null,"state":"deferred","archived":false,"hash":"60b2ba8e02e90d5e93b0dd795b3f8e38db5b7bdf","submitter":{"id":86,"url":"https://patchwork.libcamera.org/api/1.1/people/86/?format=json","name":"Umang Jain","email":"umang.jain@ideasonboard.com"},"delegate":null,"mbox":"https://patchwork.libcamera.org/patch/17982/mbox/","series":[{"id":3667,"url":"https://patchwork.libcamera.org/api/1.1/series/3667/?format=json","web_url":"https://patchwork.libcamera.org/project/libcamera/list/?series=3667","date":"2022-12-12T15:02:54","name":"pipeline-handler: Consider in-flight max requests","version":1,"mbox":"https://patchwork.libcamera.org/series/3667/mbox/"}],"comments":"https://patchwork.libcamera.org/api/patches/17982/comments/","check":"pending","checks":"https://patchwork.libcamera.org/api/patches/17982/checks/","tags":{},"headers":{"Return-Path":"","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id C38E0BDE6B\n\tfor ;\n\tMon, 12 Dec 2022 15:03:12 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id 8C58763368;\n\tMon, 12 Dec 2022 16:03:12 +0100 (CET)","from perceval.ideasonboard.com (perceval.ideasonboard.com\n\t[IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id 3AB276333F\n\tfor ;\n\tMon, 12 Dec 2022 16:03:11 +0100 (CET)","from umang.jainideasonboard.com (unknown\n\t[IPv6:2401:4900:1f3e:7d24:3f0:3e81:fb16:ab4d])\n\tby perceval.ideasonboard.com (Postfix) with ESMTPSA id C182B6CF;\n\tMon, 12 Dec 2022 16:03:09 +0100 (CET)"],"DKIM-Signature":["v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org;\n\ts=mail; t=1670857392;\n\tbh=XcTwOEslkOFI0txG9wzzN/XmWGaYwB5fIdS3RRnzCKI=;\n\th=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe:\n\tList-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To:\n\tFrom;\n\tb=Yu+g8PIX1Xq2DndYyslszFvaofUcwJUr04dpTqafs0UCmgKlPKIwbMLCax+yHlIzG\n\tlAdvORBhhslDA8RPLpqjVdG2FegoGcGetQkf7Innrtx9hIuqhqiEHRMGS8lgZyhFwl\n\tIJuGAXwq1qhY8QZ66Of6oprxFfs3MtkVpQKmWpiRf+2boIED3VRmWfv2Sg5qzmVSkF\n\t8tX0eoE9yNgFW5S0cHIfUv+ggL4CDW5Z7FlsfgKvtY87AxKXM85njRZGhkGSWY838s\n\t447uYceFZzUxg9jSor96jGbKlADjUJIZeVHtkA3Sp6k9MgjbzwkgN+Seo2fHQNfL0r\n\t09Cj+YXSICPKQ==","v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com;\n\ts=mail; t=1670857391;\n\tbh=XcTwOEslkOFI0txG9wzzN/XmWGaYwB5fIdS3RRnzCKI=;\n\th=From:To:Cc:Subject:Date:In-Reply-To:References:From;\n\tb=DkFZdc83c5CCiheDk7KacCD41p3ERey/DGcV/eRbpNOXdktbEQwDLNz3fIk+5aLRG\n\t7qDNyyccY1hvjPwyXkdMGrvX2vfR8sDoD12FJtAIJMkIsLvzggRM3MtsMBScsg20HC\n\tElXqjNzx5hf1Y89FiluIZc+jGgAnmUn/gl3LXcdU="],"Authentication-Results":"lancelot.ideasonboard.com; dkim=pass (1024-bit key; \n\tunprotected) header.d=ideasonboard.com\n\theader.i=@ideasonboard.com\n\theader.b=\"DkFZdc83\"; dkim-atps=neutral","To":"libcamera-devel@lists.libcamera.org","Date":"Mon, 12 Dec 2022 20:32:56 +0530","Message-Id":"<20221212150256.69004-3-umang.jain@ideasonboard.com>","X-Mailer":"git-send-email 2.38.1","In-Reply-To":"<20221212150256.69004-1-umang.jain@ideasonboard.com>","References":"<20221212150256.69004-1-umang.jain@ideasonboard.com>","MIME-Version":"1.0","Content-Transfer-Encoding":"8bit","Subject":"[libcamera-devel] [PATCH 2/2] libcamera: pipeline-handler: Consider\n\tmax in-flight requests constraint","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","From":"Umang Jain via libcamera-devel ","Reply-To":"Umang Jain ","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" "},"content":"This patch introduces a new constraint in the pipeline handler base\nclass which allows various derived pipeline handlers to set and\nrestrict the number of maximum in-flight requests that can be queued to\nthe underlying components (for e.g. the IPA).\n\nThe pipeline handler is now equipped with the responsibility of not to\nover queue the requests to the underlying layers. The derived\npipeline handler (or even IPA) can also have various kind of requests\nqueue(arrays, queues or ring-buffer) hence, it might be an issue where\nthe application queues the requests at a rate where these kind of\nqueues can over-flow. The patch ensures that the base PipelineHandler\nwill never let the requests overflow to underlying layers, once the\nderived pipeline handler sets its maximum capacity of handling\nin-flight requests using PipelineHandler::setMaxQueueRequests().\n\nThe queue request management introduced in the pipeline handler base\nclass is now used by the IPU3 and RkISP1 pipeline handlers. This will\nprevent over-writing of frame contexts (i.e. FCQueue) in these two\npipeline handlers.\n\nSigned-off-by: Umang Jain \n---\n include/libcamera/internal/pipeline_handler.h | 4 ++\n src/libcamera/pipeline/ipu3/ipu3.cpp | 1 +\n src/libcamera/pipeline/rkisp1/rkisp1.cpp | 1 +\n src/libcamera/pipeline_handler.cpp | 51 ++++++++++++++++++-\n 4 files changed, 56 insertions(+), 1 deletion(-)","diff":"diff --git a/include/libcamera/internal/pipeline_handler.h b/include/libcamera/internal/pipeline_handler.h\nindex ec4f662d..83f8bd9f 100644\n--- a/include/libcamera/internal/pipeline_handler.h\n+++ b/include/libcamera/internal/pipeline_handler.h\n@@ -70,6 +70,7 @@ public:\n protected:\n \tvoid registerCamera(std::shared_ptr camera);\n \tvoid hotplugMediaDevice(MediaDevice *media);\n+\tvoid setMaxQueueRequests(uint32_t maxRequests);\n \n \tvirtual int queueRequestDevice(Camera *camera, Request *request) = 0;\n \tvirtual void stopDevice(Camera *camera) = 0;\n@@ -97,6 +98,9 @@ private:\n \tMutex lock_;\n \tunsigned int useCount_ LIBCAMERA_TSA_GUARDED_BY(lock_);\n \n+\tuint32_t maxQueueRequests_;\n+\tuint32_t requestsQueueCounter_;\n+\n \tfriend class PipelineHandlerFactoryBase;\n };\n \ndiff --git a/src/libcamera/pipeline/ipu3/ipu3.cpp b/src/libcamera/pipeline/ipu3/ipu3.cpp\nindex e4d79ea4..d1d42f78 100644\n--- a/src/libcamera/pipeline/ipu3/ipu3.cpp\n+++ b/src/libcamera/pipeline/ipu3/ipu3.cpp\n@@ -422,6 +422,7 @@ CameraConfiguration::Status IPU3CameraConfiguration::validate()\n PipelineHandlerIPU3::PipelineHandlerIPU3(CameraManager *manager)\n \t: PipelineHandler(manager), cio2MediaDev_(nullptr), imguMediaDev_(nullptr)\n {\n+\tsetMaxQueueRequests(ipa::ipu3::kMaxFrameContexts);\n }\n \n std::unique_ptr\ndiff --git a/src/libcamera/pipeline/rkisp1/rkisp1.cpp b/src/libcamera/pipeline/rkisp1/rkisp1.cpp\nindex eb9ad65c..a48adba9 100644\n--- a/src/libcamera/pipeline/rkisp1/rkisp1.cpp\n+++ b/src/libcamera/pipeline/rkisp1/rkisp1.cpp\n@@ -599,6 +599,7 @@ CameraConfiguration::Status RkISP1CameraConfiguration::validate()\n PipelineHandlerRkISP1::PipelineHandlerRkISP1(CameraManager *manager)\n \t: PipelineHandler(manager), hasSelfPath_(true)\n {\n+\tsetMaxQueueRequests(ipa::rkisp1::kMaxFrameContexts);\n }\n \n std::unique_ptr\ndiff --git a/src/libcamera/pipeline_handler.cpp b/src/libcamera/pipeline_handler.cpp\nindex cfade490..103f9db0 100644\n--- a/src/libcamera/pipeline_handler.cpp\n+++ b/src/libcamera/pipeline_handler.cpp\n@@ -67,7 +67,8 @@ LOG_DEFINE_CATEGORY(Pipeline)\n * through the PipelineHandlerFactoryBase::create() function.\n */\n PipelineHandler::PipelineHandler(CameraManager *manager)\n-\t: manager_(manager), useCount_(0)\n+\t: manager_(manager), useCount_(0), maxQueueRequests_(0),\n+\t requestsQueueCounter_(0)\n {\n }\n \n@@ -428,6 +429,9 @@ void PipelineHandler::doQueueRequest(Request *request)\n \tCamera::Private *data = camera->_d();\n \tdata->queuedRequests_.push_back(request);\n \n+\tif (maxQueueRequests_)\n+\t\trequestsQueueCounter_++;\n+\n \trequest->_d()->sequence_ = data->requestSequence_++;\n \n \tif (request->_d()->cancelled_) {\n@@ -455,6 +459,10 @@ void PipelineHandler::doQueueRequests()\n \t\tif (!request->_d()->prepared_)\n \t\t\tbreak;\n \n+\t\tif (maxQueueRequests_ &&\n+\t\t requestsQueueCounter_ >= maxQueueRequests_)\n+\t\t\tbreak;\n+\n \t\tdoQueueRequest(request);\n \t\twaitingRequests_.pop();\n \t}\n@@ -531,6 +539,9 @@ void PipelineHandler::completeRequest(Request *request)\n \t\tASSERT(!req->hasPendingBuffers());\n \t\tdata->queuedRequests_.pop_front();\n \t\tcamera->requestComplete(req);\n+\n+\t\tif (maxQueueRequests_)\n+\t\t\trequestsQueueCounter_--;\n \t}\n }\n \n@@ -647,6 +658,44 @@ void PipelineHandler::disconnect()\n * constant for the whole lifetime of the pipeline handler.\n */\n \n+/**\n+ * \\var PipelineHandler::maxQueueRequests_\n+ * \\brief Maximum number of in-flight requests that can be queued\n+ *\n+ * A hardware can handle a certain number of maximum requests at a given\n+ * point. If such a constraint exists, set maxQueueRequests_ via\n+ * \\a setMaxQueueRequests() in the derived pipeline handler.\n+ *\n+ * The derived pipeline handler can choose not to define such constraint as\n+ * well. In that case, the derived pipeline handler can avoid setting\n+ * \\a setMaxQueueReqeuests(), hence \\a maxQueueRequests_ and\n+ * \\a requestsQueueCounter_ will be 0.\n+ */\n+\n+/**\n+ * \\var PipelineHandler::requestsQueueCounter_\n+ * \\brief Number of requests queued to the underlying hardware\n+ *\n+ * If \\a setMaxQueueRequests() is set by the derived pipeline handler,\n+ * requestsQueueCounter_ reflects the number of requests queued\n+ * to the underlying hardware by the pipeline handler.\n+ */\n+\n+/**\n+ * \\brief Sets the maximum number of requests that can be queued\n+ * \\param[in] maxRequests Maximum number of in-flight requests\n+ *\n+ * A hardware can handle a certain number of requests at a given point.\n+ * This function sets the maximum number of in-flight requests that can\n+ * be queued to the hardware by the pipeline handler. Each derived pipeline\n+ * handler should set the maximum number of in-flight requests it can handle\n+ * at a given point using this function, if at all such a constraint exists.\n+ */\n+void PipelineHandler::setMaxQueueRequests(uint32_t maxRequests)\n+{\n+\tmaxQueueRequests_ = maxRequests;\n+}\n+\n /**\n * \\fn PipelineHandler::name()\n * \\brief Retrieve the pipeline handler name\n","prefixes":["libcamera-devel","2/2"]}