{"id":25462,"url":"https://patchwork.libcamera.org/api/1.1/patches/25462/?format=json","web_url":"https://patchwork.libcamera.org/patch/25462/","project":{"id":1,"url":"https://patchwork.libcamera.org/api/1.1/projects/1/?format=json","name":"libcamera","link_name":"libcamera","list_id":"libcamera_core","list_email":"libcamera-devel@lists.libcamera.org","web_url":"","scm_url":"","webscm_url":""},"msgid":"<20251210-mali-cru-v2-5-e26421de202b@ideasonboard.com>","date":"2025-12-10T14:39:21","name":"[v2,5/7] libcamera: mali-c55: Implement capture for memory-to-memory","commit_ref":null,"pull_url":null,"state":"superseded","archived":false,"hash":"4a0dbde06cdfad48236cf21e86995b485dd372f8","submitter":{"id":143,"url":"https://patchwork.libcamera.org/api/1.1/people/143/?format=json","name":"Jacopo Mondi","email":"jacopo.mondi@ideasonboard.com"},"delegate":null,"mbox":"https://patchwork.libcamera.org/patch/25462/mbox/","series":[{"id":5649,"url":"https://patchwork.libcamera.org/api/1.1/series/5649/?format=json","web_url":"https://patchwork.libcamera.org/project/libcamera/list/?series=5649","date":"2025-12-10T14:39:16","name":"libcamera: mali-c55: Add support for memory-to-memory","version":2,"mbox":"https://patchwork.libcamera.org/series/5649/mbox/"}],"comments":"https://patchwork.libcamera.org/api/patches/25462/comments/","check":"pending","checks":"https://patchwork.libcamera.org/api/patches/25462/checks/","tags":{},"headers":{"Return-Path":"","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id EA2ABC326B\n\tfor ;\n\tWed, 10 Dec 2025 14:39:40 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id 63CA661495;\n\tWed, 10 Dec 2025 15:39:39 +0100 (CET)","from perceval.ideasonboard.com (perceval.ideasonboard.com\n\t[213.167.242.64])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id E5E8961499\n\tfor ;\n\tWed, 10 Dec 2025 15:39:33 +0100 (CET)","from [192.168.1.106] (mob-5-90-55-146.net.vodafone.it\n\t[5.90.55.146])\n\tby perceval.ideasonboard.com (Postfix) with ESMTPSA id 7F0D610C4;\n\tWed, 10 Dec 2025 15:39:32 +0100 (CET)"],"Authentication-Results":"lancelot.ideasonboard.com; dkim=pass (1024-bit key;\n\tunprotected) header.d=ideasonboard.com header.i=@ideasonboard.com\n\theader.b=\"FUZFw9Pc\"; dkim-atps=neutral","DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com;\n\ts=mail; t=1765377572;\n\tbh=iqmzamUpUgXPzSUkiK1h0Gp1XC6pXJetUbGunDuyNzU=;\n\th=From:Date:Subject:References:In-Reply-To:To:Cc:From;\n\tb=FUZFw9Pcw6oqeQ4FOIhGgSK5Fr/UMsrdty2OsRxVtrynx0rb1xzIPImg3XR7YNZK6\n\tzx37tWiIfAKOQkvXZ9SlYMXqC1lX/FrDnnyp/pVIUIT81H92U5RQJLn95vGsGRwCI9\n\t8BH3Nc6bF7TsVEbN8Sv5mOONZOhmWNkaNSbjSsrg=","From":"Jacopo Mondi ","Date":"Wed, 10 Dec 2025 15:39:21 +0100","Subject":"[PATCH v2 5/7] libcamera: mali-c55: Implement capture for\n\tmemory-to-memory","MIME-Version":"1.0","Content-Type":"text/plain; charset=\"utf-8\"","Content-Transfer-Encoding":"7bit","Message-Id":"<20251210-mali-cru-v2-5-e26421de202b@ideasonboard.com>","References":"<20251210-mali-cru-v2-0-e26421de202b@ideasonboard.com>","In-Reply-To":"<20251210-mali-cru-v2-0-e26421de202b@ideasonboard.com>","To":"Daniel Scally , \n\tlibcamera-devel@lists.libcamera.org","Cc":"Jacopo Mondi ","X-Mailer":"b4 0.14.2","X-Developer-Signature":"v=1; a=openpgp-sha256; l=9438;\n\ti=jacopo.mondi@ideasonboard.com; h=from:subject:message-id;\n\tbh=IWn36FKOAlst4y5gWEgsituAx7g1CDa+O1cF2zmWej4=;\n\tb=owEBbQKS/ZANAwAKAXI0Bo8WoVY8AcsmYgBpOYYjx+J/n2otU2sebkVQmtV0KIprL+8Lxa92I\n\tCmXDvQqSpSJAjMEAAEKAB0WIQS1xD1IgJogio9YOMByNAaPFqFWPAUCaTmGIwAKCRByNAaPFqFW\n\tPAjkD/0Q9cBCmBijnzRUqDNmvMoAADYBfdXEFzjRzP4iJb7tgwBbWOrY5CeNVgko9hHHf6Z7bkP\n\tC0lQcscVOzGmZqhqhvsXhT2xLCNT2i7TL5YizIDV+IyaMOen6G6AnIBsxFrRocyrcEO6U/PTkg6\n\tn0f9JquU37vgqTBCLnavwm9HyGXIoZZJs3kdqsP5VOLgwe5GNiWhWWalJ/CN54LQRHq0nekQFyt\n\tlloBB8iro8RRPVIcMHuWIwIUDv85wZ9x1BZGTmILJ6t0ZIeIpYI0svzd2soYyHTkwysa2nXhmGQ\n\t7mht6zlhpHN6wMqSIa2n0Yk46veKAsyAMBYYlfglGC4r1hdP2OF8aO+aZoEB8m2Jcq5oMtH2sbW\n\t6rAE8AtoYFxBu7Dtafb63LnJ/hseREHaIOJvy6KOdvneVzR9OpSu9fA4a9/GhcsZSTa7ZH616xC\n\tYbWhMo7e0qBFIsZdKaeJ3M1HKJsQ/oKGEvyAk6KAFq7y4jmoxZClhywp5GZ1L36GlrszPCpO3MG\n\tSUB6qwXQ7Z70q4RVX1OMRxxSXrkg5M5UEONir+amfylEnFmIVxz+M4ArqelBvEd/74HTjnLfSsd\n\t2AkXRyk5aLZ5Bl1H9LOeHxSTsiJhEWut2KQ+U/g9eRQ/N/U/MUZMiRu9uEBn5e4O4Wr31OqF5Lt\n\twZZP5/MjPjaiBhQ==","X-Developer-Key":"i=jacopo.mondi@ideasonboard.com; a=openpgp;\n\tfpr=72392EDC88144A65C701EA9BA5826A2587AD026B","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" "},"content":"From: Daniel Scally \n\nPlumb in the MaliC55 pipeline handler support for capturing frames\nfrom memory using the CRU.\n\nIntroduce a data flow which uses the CRU to feed the ISP through\nthe IVC.\n\nIn detail:\n\n- push incoming request to a pending queue until a buffer from the CRU\n is available\n- delay the call to ipa_->fillParams() to the CRU buffer ready even\n- once the IPA has computed parameters feed the ISP through the IVC\n with buffers from the CRU, params and statistics.\n\nSigned-off-by: Daniel Scally \nSigned-off-by: Jacopo Mondi \n---\n src/libcamera/pipeline/mali-c55/mali-c55.cpp | 166 ++++++++++++++++++++++++++-\n 1 file changed, 161 insertions(+), 5 deletions(-)","diff":"diff --git a/src/libcamera/pipeline/mali-c55/mali-c55.cpp b/src/libcamera/pipeline/mali-c55/mali-c55.cpp\nindex 8dcc51e27020c754004cd98cbdf73e771275b059..83cb10af5d3c0588b25e4a3190f561f09627d239 100644\n--- a/src/libcamera/pipeline/mali-c55/mali-c55.cpp\n+++ b/src/libcamera/pipeline/mali-c55/mali-c55.cpp\n@@ -19,6 +19,7 @@\n #include \n \n #include \n+#include \n #include \n #include \n #include \n@@ -86,6 +87,7 @@ struct MaliC55FrameInfo {\n \n \tFrameBuffer *paramBuffer;\n \tFrameBuffer *statBuffer;\n+\tFrameBuffer *rawBuffer;\n \n \tbool paramsDone;\n \tbool statsDone;\n@@ -775,11 +777,14 @@ public:\n \tint start(Camera *camera, const ControlList *controls) override;\n \tvoid stopDevice(Camera *camera) override;\n \n+\tint queuePendingRequests(MaliC55CameraData *data);\n+\tvoid cancelPendingRequests();\n \tint queueRequestDevice(Camera *camera, Request *request) override;\n \n \tvoid imageBufferReady(FrameBuffer *buffer);\n \tvoid paramsBufferReady(FrameBuffer *buffer);\n \tvoid statsBufferReady(FrameBuffer *buffer);\n+\tvoid cruBufferReady(FrameBuffer *buffer);\n \tvoid paramsComputed(unsigned int requestId, uint32_t bytesused);\n \tvoid statsProcessed(unsigned int requestId, const ControlList &metadata);\n \n@@ -861,6 +866,11 @@ private:\n \n \tstd::map frameInfoMap_;\n \n+\t/* Requests for which no buffer has been queued to the CRU device yet. */\n+\tstd::queue pendingRequests_;\n+\t/* Requests queued to the CRU device but not yet processed by the ISP. */\n+\tstd::queue processingRequests_;\n+\n \tstd::array pipes_;\n \n \tbool dsFitted_;\n@@ -1320,6 +1330,11 @@ void PipelineHandlerMaliC55::freeBuffers(Camera *camera)\n \tif (params_->releaseBuffers())\n \t\tLOG(MaliC55, Error) << \"Failed to release params buffers\";\n \n+\tif (data->type() == MaliC55CameraData::CameraType::Memory) {\n+\t\tif (input_->releaseBuffers())\n+\t\t\tLOG(MaliC55, Error) << \"Failed to release input buffers\";\n+\t}\n+\n \treturn;\n }\n \n@@ -1349,6 +1364,12 @@ int PipelineHandlerMaliC55::allocateBuffers(Camera *camera)\n \t\t}\n \t};\n \n+\tif (input_) {\n+\t\tret = input_->importBuffers(RZG2LCRU::kBufferCount);\n+\t\tif (ret < 0)\n+\t\t\treturn ret;\n+\t}\n+\n \tret = stats_->allocateBuffers(bufferCount, &statsBuffers_);\n \tif (ret < 0)\n \t\treturn ret;\n@@ -1380,6 +1401,24 @@ int PipelineHandlerMaliC55::start(Camera *camera, [[maybe_unused]] const Control\n \tif (ret)\n \t\treturn ret;\n \n+\tif (data->type() == MaliC55CameraData::CameraType::Memory) {\n+\t\tret = data->cru()->start();\n+\t\tif (ret) {\n+\t\t\tLOG(MaliC55, Error)\n+\t\t\t\t<< \"Failed to start CRU \" << camera->id();\n+\t\t\tfreeBuffers(camera);\n+\t\t\treturn ret;\n+\t\t}\n+\n+\t\tret = input_->streamOn();\n+\t\tif (ret) {\n+\t\t\tLOG(MaliC55, Error)\n+\t\t\t\t<< \"Failed to start IVC\" << camera->id();\n+\t\t\tfreeBuffers(camera);\n+\t\t\treturn ret;\n+\t\t}\n+\t}\n+\n \tif (data->ipa_) {\n \t\tret = data->ipa_->start();\n \t\tif (ret) {\n@@ -1469,6 +1508,12 @@ void PipelineHandlerMaliC55::stopDevice(Camera *camera)\n \t\tpipe.cap->releaseBuffers();\n \t}\n \n+\tif (data->type() == MaliC55CameraData::CameraType::Memory) {\n+\t\tcancelPendingRequests();\n+\t\tinput_->streamOff();\n+\t\tdata->cru()->stop();\n+\t}\n+\n \tstats_->streamOff();\n \tparams_->streamOff();\n \tif (data->ipa_)\n@@ -1572,10 +1617,88 @@ void PipelineHandlerMaliC55::applyScalerCrop(Camera *camera,\n \t}\n }\n \n+void PipelineHandlerMaliC55::cancelPendingRequests()\n+{\n+\tprocessingRequests_ = {};\n+\n+\twhile (!pendingRequests_.empty()) {\n+\t\tRequest *request = pendingRequests_.front();\n+\n+\t\tcompleteRequest(request);\n+\t\tpendingRequests_.pop();\n+\t}\n+}\n+\n+int PipelineHandlerMaliC55::queuePendingRequests(MaliC55CameraData *data)\n+{\n+\tASSERT(data->type() == MaliC55CameraData::CameraType::Memory);\n+\n+\twhile (!pendingRequests_.empty()) {\n+\t\tRequest *request = pendingRequests_.front();\n+\n+\t\tif (availableStatsBuffers_.empty()) {\n+\t\t\tLOG(MaliC55, Error) << \"Stats buffer underrun\";\n+\t\t\treturn -ENOENT;\n+\t\t}\n+\n+\t\tif (availableParamsBuffers_.empty()) {\n+\t\t\tLOG(MaliC55, Error) << \"Params buffer underrun\";\n+\t\t\treturn -ENOENT;\n+\t\t}\n+\n+\t\tMaliC55FrameInfo frameInfo;\n+\t\tframeInfo.request = request;\n+\n+\t\tframeInfo.rawBuffer = data->cru()->queueBuffer(request);\n+\t\tif (!frameInfo.rawBuffer)\n+\t\t\treturn -ENOENT;\n+\n+\t\tframeInfo.statBuffer = availableStatsBuffers_.front();\n+\t\tavailableStatsBuffers_.pop();\n+\t\tframeInfo.paramBuffer = availableParamsBuffers_.front();\n+\t\tavailableParamsBuffers_.pop();\n+\n+\t\tframeInfo.paramsDone = false;\n+\t\tframeInfo.statsDone = false;\n+\n+\t\tframeInfoMap_[request->sequence()] = frameInfo;\n+\n+\t\tfor (auto &[stream, buffer] : request->buffers()) {\n+\t\t\tMaliC55Pipe *pipe = pipeFromStream(data, stream);\n+\n+\t\t\tpipe->cap->queueBuffer(buffer);\n+\t\t}\n+\n+\t\tdata->ipa_->queueRequest(request->sequence(), request->controls());\n+\n+\t\tpendingRequests_.pop();\n+\t\tprocessingRequests_.push(request);\n+\t}\n+\n+\treturn 0;\n+}\n+\n int PipelineHandlerMaliC55::queueRequestDevice(Camera *camera, Request *request)\n {\n \tMaliC55CameraData *data = cameraData(camera);\n \n+\t/*\n+\t * If we're in memory input mode, we need to pop the requests onto the\n+\t * pending list until a CRU buffer is ready...otherwise we can just do\n+\t * everything immediately.\n+\t */\n+\tif (data->type() == MaliC55CameraData::CameraType::Memory) {\n+\t\tpendingRequests_.push(request);\n+\n+\t\tint ret = queuePendingRequests(data);\n+\t\tif (ret) {\n+\t\t\tpendingRequests_.pop();\n+\t\t\treturn ret;\n+\t\t}\n+\n+\t\treturn 0;\n+\t}\n+\n \t/* Do not run the IPA if the TPG is in use. */\n \tif (!data->ipa_) {\n \t\tMaliC55FrameInfo frameInfo;\n@@ -1640,7 +1763,8 @@ MaliC55FrameInfo *PipelineHandlerMaliC55::findFrameInfo(FrameBuffer *buffer)\n {\n \tfor (auto &[sequence, info] : frameInfoMap_) {\n \t\tif (info.paramBuffer == buffer ||\n-\t\t info.statBuffer == buffer)\n+\t\t info.statBuffer == buffer ||\n+\t\t info.rawBuffer == buffer)\n \t\t\treturn &info;\n \t}\n \n@@ -1702,6 +1826,26 @@ void PipelineHandlerMaliC55::statsBufferReady(FrameBuffer *buffer)\n \t\t\t\t sensorControls);\n }\n \n+void PipelineHandlerMaliC55::cruBufferReady(FrameBuffer *buffer)\n+{\n+\tMaliC55FrameInfo *info = findFrameInfo(buffer);\n+\tRequest *request = info->request;\n+\tASSERT(info);\n+\n+\tif (buffer->metadata().status == FrameMetadata::FrameCancelled) {\n+\t\tframeInfoMap_.erase(request->sequence());\n+\t\tcompleteRequest(request);\n+\t\treturn;\n+\t}\n+\n+\trequest->metadata().set(controls::SensorTimestamp,\n+\t\t\t\tbuffer->metadata().timestamp);\n+\n+\t/* Ought we do something with the sensor's controls here...? */\n+\tMaliC55CameraData *data = cameraData(request->_d()->camera());\n+\tdata->ipa_->fillParams(request->sequence(), info->paramBuffer->cookie());\n+}\n+\n void PipelineHandlerMaliC55::paramsComputed(unsigned int requestId, uint32_t bytesused)\n {\n \tMaliC55FrameInfo &frameInfo = frameInfoMap_[requestId];\n@@ -1710,18 +1854,27 @@ void PipelineHandlerMaliC55::paramsComputed(unsigned int requestId, uint32_t byt\n \n \t/*\n \t * Queue buffers for stats and params, then queue buffers to the capture\n-\t * video devices.\n+\t * video devices if we're running in Inline mode or with the TPG.\n+\t *\n+\t * If we're running in M2M buffers have been queued to the capture\n+\t * devices at queuePendingRequests() time and here we only have to queue\n+\t * buffers to the IVC input to start a transfer.\n \t */\n \n \tframeInfo.paramBuffer->_d()->metadata().planes()[0].bytesused = bytesused;\n \tparams_->queueBuffer(frameInfo.paramBuffer);\n \tstats_->queueBuffer(frameInfo.statBuffer);\n \n-\tfor (auto &[stream, buffer] : request->buffers()) {\n-\t\tMaliC55Pipe *pipe = pipeFromStream(data, stream);\n+\tif (data->type() != MaliC55CameraData::CameraType::Memory) {\n+\t\tfor (auto &[stream, buffer] : request->buffers()) {\n+\t\t\tMaliC55Pipe *pipe = pipeFromStream(data, stream);\n \n-\t\tpipe->cap->queueBuffer(buffer);\n+\t\t\tpipe->cap->queueBuffer(buffer);\n+\t\t}\n \t}\n+\n+\tif (data->type() == MaliC55CameraData::CameraType::Memory)\n+\t\tinput_->queueBuffer(frameInfo.rawBuffer);\n }\n \n void PipelineHandlerMaliC55::statsProcessed(unsigned int requestId,\n@@ -1863,6 +2016,9 @@ bool PipelineHandlerMaliC55::registerMemoryInputCamera()\n \n \tinput_->bufferReady.connect(memoryData->cru(), &RZG2LCRU::cruReturnBuffer);\n \n+\tV4L2VideoDevice *cruOutput = memoryData->cru()->output();\n+\tcruOutput->bufferReady.connect(this, &PipelineHandlerMaliC55::cruBufferReady);\n+\n \tstd::unique_ptr data(memoryData);\n \tif (!registerMaliCamera(std::move(data), sensor->device()->entity()->name()))\n \t\treturn false;\n","prefixes":["v2","5/7"]}