{"id":25375,"url":"https://patchwork.libcamera.org/api/1.1/patches/25375/?format=json","web_url":"https://patchwork.libcamera.org/patch/25375/","project":{"id":1,"url":"https://patchwork.libcamera.org/api/1.1/projects/1/?format=json","name":"libcamera","link_name":"libcamera","list_id":"libcamera_core","list_email":"libcamera-devel@lists.libcamera.org","web_url":"","scm_url":"","webscm_url":""},"msgid":"<20251205-mali-cru-v1-5-d81bb5ffe73a@ideasonboard.com>","date":"2025-12-05T14:52:11","name":"[5/7] libcamera: mali-c55: Implement capture for memory-to-memory","commit_ref":null,"pull_url":null,"state":"superseded","archived":false,"hash":"34254b03306194918e2afd6af3baf10f25a7ef1b","submitter":{"id":143,"url":"https://patchwork.libcamera.org/api/1.1/people/143/?format=json","name":"Jacopo Mondi","email":"jacopo.mondi@ideasonboard.com"},"delegate":null,"mbox":"https://patchwork.libcamera.org/patch/25375/mbox/","series":[{"id":5640,"url":"https://patchwork.libcamera.org/api/1.1/series/5640/?format=json","web_url":"https://patchwork.libcamera.org/project/libcamera/list/?series=5640","date":"2025-12-05T14:52:06","name":"libcamera: mali-c55: Add support for memory-to-memory","version":1,"mbox":"https://patchwork.libcamera.org/series/5640/mbox/"}],"comments":"https://patchwork.libcamera.org/api/patches/25375/comments/","check":"pending","checks":"https://patchwork.libcamera.org/api/patches/25375/checks/","tags":{},"headers":{"Return-Path":"<libcamera-devel-bounces@lists.libcamera.org>","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id DF2EFC32AF\n\tfor <parsemail@patchwork.libcamera.org>;\n\tFri,  5 Dec 2025 14:52:40 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id 314DF613EB;\n\tFri,  5 Dec 2025 15:52:37 +0100 (CET)","from perceval.ideasonboard.com (perceval.ideasonboard.com\n\t[213.167.242.64])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id 941A8606A0\n\tfor <libcamera-devel@lists.libcamera.org>;\n\tFri,  5 Dec 2025 15:52:31 +0100 (CET)","from [192.168.1.4] (net-93-65-100-155.cust.vodafonedsl.it\n\t[93.65.100.155])\n\tby perceval.ideasonboard.com (Postfix) with ESMTPSA id 6C47D1340;\n\tFri,  5 Dec 2025 15:50:15 +0100 (CET)"],"Authentication-Results":"lancelot.ideasonboard.com; dkim=pass (1024-bit key;\n\tunprotected) header.d=ideasonboard.com header.i=@ideasonboard.com\n\theader.b=\"vJFfz/Es\"; dkim-atps=neutral","DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com;\n\ts=mail; t=1764946215;\n\tbh=A/dnpEvqJ410WD7m1lspdx6fWRD2/0pPEEWUV8JhJ4g=;\n\th=From:Date:Subject:References:In-Reply-To:To:Cc:From;\n\tb=vJFfz/EsYaCCrP8FlAgr7r7I19X0nKe3n6syEKGNepa5J2+8I5L+8/s3FhoQkFpJI\n\tHuzSu6ohYRSLSdzJuTqcfmlgLPWO4auYcaltfktvzRsHlcOoku/jKr+LsCsYlP1Bbg\n\t/XmPDaHxZ07COlY1Ed6vUADTHic1em7PHgiacDOU=","From":"Jacopo Mondi <jacopo.mondi@ideasonboard.com>","Date":"Fri, 05 Dec 2025 15:52:11 +0100","Subject":"[PATCH 5/7] libcamera: mali-c55: Implement capture for\n\tmemory-to-memory","MIME-Version":"1.0","Content-Type":"text/plain; charset=\"utf-8\"","Content-Transfer-Encoding":"7bit","Message-Id":"<20251205-mali-cru-v1-5-d81bb5ffe73a@ideasonboard.com>","References":"<20251205-mali-cru-v1-0-d81bb5ffe73a@ideasonboard.com>","In-Reply-To":"<20251205-mali-cru-v1-0-d81bb5ffe73a@ideasonboard.com>","To":"Daniel Scally <dan.scally@ideasonboard.com>, \n\tlibcamera-devel@lists.libcamera.org","Cc":"Jacopo Mondi <jacopo.mondi@ideasonboard.com>","X-Mailer":"b4 0.14.2","X-Developer-Signature":"v=1; a=openpgp-sha256; l=9311;\n\ti=jacopo.mondi@ideasonboard.com; h=from:subject:message-id;\n\tbh=4UcvgR4scTrxJYrijGo+f5eFnqOvmZWIz+luVNrEiCo=;\n\tb=owEBbQKS/ZANAwAKAXI0Bo8WoVY8AcsmYgBpMvGtixsOn0mJ0Y4bp6ty0X9Tc4GwwAdWkIxwZ\n\tY0oiQgUhh+JAjMEAAEKAB0WIQS1xD1IgJogio9YOMByNAaPFqFWPAUCaTLxrQAKCRByNAaPFqFW\n\tPMx4D/0TYj/sO5NvolfcYCGsm+IJZ1PnM+Tz+HvGaPO3WC4hQtRhAUvWpwXu71PidXzqiOHXTe/\n\t0Md/DwMSzJ1wmjuFi9UNwF+hTGN2Eek3pjL8yV2n7HmgV1WP99WcFsZdksKSoeiqWPOTI54PBlr\n\tWXURmugqHHU1eEUNFpRhC3eZ1iZQsoLQOcfZDfyGsakr+9GUpT5WTidfsred9QMToItGJUO2LCX\n\tIfPv2skq2TTRTq7/nsWxl8XxhP3xgj5Et0SI1S+znE4pFYzsY6YU9/EwFQQ7MMLqxQTfNtVJUBh\n\t1FGFPXRN1VF/FnGRyyTFnUmZ/bOZfI0S2ja2N5Qn67tKGYVxC+ctXe0ePRQEYGOyDg+DuVns7og\n\tChAwh/+tuGyPxV28upf03XJfAaw6dP58AMTd7T+ChhuBV8+u6g70Em9QLJAIeNwt+4Hup/wZCQk\n\tjqXdvfK05CXojcS7GewSIEW03Z2rV9x9kUp/mBjIakLajrqlwcFuQuVWLxz6bco7HqcrYQkDMF4\n\te0N82BIiQIj2fBBTBFOarjt/3Qv3j3cnxWjmA/jA9K7dPoTzhr8vzB7cHRqNA4xIifFXzurkvJx\n\ta1PNlE0AEEGv49UIuZUy84cuoSf9yw3ZGmkY+wA4MxcHPmQ2eGnl1w7v4GCATeC55i2qCkhSxkn\n\tlfcZPHhYzgLbMeg==","X-Developer-Key":"i=jacopo.mondi@ideasonboard.com; a=openpgp;\n\tfpr=72392EDC88144A65C701EA9BA5826A2587AD026B","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"<libcamera-devel.lists.libcamera.org>","List-Unsubscribe":"<https://lists.libcamera.org/options/libcamera-devel>,\n\t<mailto:libcamera-devel-request@lists.libcamera.org?subject=unsubscribe>","List-Archive":"<https://lists.libcamera.org/pipermail/libcamera-devel/>","List-Post":"<mailto:libcamera-devel@lists.libcamera.org>","List-Help":"<mailto:libcamera-devel-request@lists.libcamera.org?subject=help>","List-Subscribe":"<https://lists.libcamera.org/listinfo/libcamera-devel>,\n\t<mailto:libcamera-devel-request@lists.libcamera.org?subject=subscribe>","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" <libcamera-devel-bounces@lists.libcamera.org>"},"content":"From: Daniel Scally <dan.scally@ideasonboard.com>\n\nPlumb in the MaliC55 pipeline handler support for capturing frames\nfrom memory using the CRU.\n\nIntroduce a data flow which uses the CRU to feed the ISP through\nthe IVC.\n\nIn detail:\n\n- push incoming request to a pending queue until a buffer from the CRU\n  is available\n- delay the call to ipa_->fillParams() to the CRU buffer ready even\n- once the IPA has computed parameters feed the ISP through the IVC\n  with buffers from the CRU, params and statistics.\n\nSigned-off-by: Daniel Scally <dan.scally@ideasonboard.com>\nSigned-off-by: Jacopo Mondi <jacopo.mondi@ideasonboard.com>\n---\n src/libcamera/pipeline/mali-c55/mali-c55.cpp | 164 ++++++++++++++++++++++++++-\n 1 file changed, 159 insertions(+), 5 deletions(-)","diff":"diff --git a/src/libcamera/pipeline/mali-c55/mali-c55.cpp b/src/libcamera/pipeline/mali-c55/mali-c55.cpp\nindex 6dce315c1c82db3554e8c0eae727cba9d632ca82..46ef5e7735a30a0d4ae9bb6f8d671bbd2dff3f51 100644\n--- a/src/libcamera/pipeline/mali-c55/mali-c55.cpp\n+++ b/src/libcamera/pipeline/mali-c55/mali-c55.cpp\n@@ -19,6 +19,7 @@\n #include <libcamera/base/log.h>\n \n #include <libcamera/camera.h>\n+#include <libcamera/controls.h>\n #include <libcamera/formats.h>\n #include <libcamera/geometry.h>\n #include <libcamera/property_ids.h>\n@@ -86,6 +87,7 @@ struct MaliC55FrameInfo {\n \n \tFrameBuffer *paramBuffer;\n \tFrameBuffer *statBuffer;\n+\tFrameBuffer *rawBuffer;\n \n \tbool paramsDone;\n \tbool statsDone;\n@@ -692,11 +694,14 @@ public:\n \tint start(Camera *camera, const ControlList *controls) override;\n \tvoid stopDevice(Camera *camera) override;\n \n+\tint queuePendingRequests();\n+\tvoid cancelPendingRequests();\n \tint queueRequestDevice(Camera *camera, Request *request) override;\n \n \tvoid imageBufferReady(FrameBuffer *buffer);\n \tvoid paramsBufferReady(FrameBuffer *buffer);\n \tvoid statsBufferReady(FrameBuffer *buffer);\n+\tvoid cruBufferReady(FrameBuffer *buffer);\n \tvoid paramsComputed(unsigned int requestId, uint32_t bytesused);\n \tvoid statsProcessed(unsigned int requestId, const ControlList &metadata);\n \n@@ -778,6 +783,11 @@ private:\n \n \tstd::map<unsigned int, MaliC55FrameInfo> frameInfoMap_;\n \n+\t/* Requests for which no buffer has been queued to the CRU device yet. */\n+\tstd::queue<Request *> pendingRequests_;\n+\t/* Requests queued to the CRU device but not yet processed by the ISP. */\n+\tstd::queue<Request *> processingRequests_;\n+\n \tstd::array<MaliC55Pipe, MaliC55NumPipes> pipes_;\n \n \tbool dsFitted_;\n@@ -1235,6 +1245,11 @@ void PipelineHandlerMaliC55::freeBuffers(Camera *camera)\n \tif (params_->releaseBuffers())\n \t\tLOG(MaliC55, Error) << \"Failed to release params buffers\";\n \n+\tif (data->input_ == MaliC55CameraData::Memory) {\n+\t\tif (input_->releaseBuffers())\n+\t\t\tLOG(MaliC55, Error) << \"Failed to release input buffers\";\n+\t}\n+\n \treturn;\n }\n \n@@ -1264,6 +1279,12 @@ int PipelineHandlerMaliC55::allocateBuffers(Camera *camera)\n \t\t}\n \t};\n \n+\tif (input_) {\n+\t\tret = input_->importBuffers(RZG2LCRU::kBufferCount);\n+\t\tif (ret < 0)\n+\t\t\treturn ret;\n+\t}\n+\n \tret = stats_->allocateBuffers(bufferCount, &statsBuffers_);\n \tif (ret < 0)\n \t\treturn ret;\n@@ -1295,6 +1316,24 @@ int PipelineHandlerMaliC55::start(Camera *camera, [[maybe_unused]] const Control\n \tif (ret)\n \t\treturn ret;\n \n+\tif (data->input_ == MaliC55CameraData::Memory) {\n+\t\tret = data->memoryInput.cru_->start();\n+\t\tif (ret) {\n+\t\t\tLOG(MaliC55, Error)\n+\t\t\t\t<< \"Failed to start CRU \" << camera->id();\n+\t\t\tfreeBuffers(camera);\n+\t\t\treturn ret;\n+\t\t}\n+\n+\t\tret = input_->streamOn();\n+\t\tif (ret) {\n+\t\t\tLOG(MaliC55, Error)\n+\t\t\t\t<< \"Failed to start IVC\" << camera->id();\n+\t\t\tfreeBuffers(camera);\n+\t\t\treturn ret;\n+\t\t}\n+\t}\n+\n \tif (data->ipa_) {\n \t\tret = data->ipa_->start();\n \t\tif (ret) {\n@@ -1384,6 +1423,12 @@ void PipelineHandlerMaliC55::stopDevice(Camera *camera)\n \t\tpipe.cap->releaseBuffers();\n \t}\n \n+\tif (data->input_ == MaliC55CameraData::Memory) {\n+\t\tcancelPendingRequests();\n+\t\tinput_->streamOff();\n+\t\tdata->memoryInput.cru_->stop();\n+\t}\n+\n \tstats_->streamOff();\n \tparams_->streamOff();\n \tif (data->ipa_)\n@@ -1488,10 +1533,87 @@ void PipelineHandlerMaliC55::applyScalerCrop(Camera *camera,\n \t}\n }\n \n+void PipelineHandlerMaliC55::cancelPendingRequests()\n+{\n+\tprocessingRequests_ = {};\n+\n+\twhile (!pendingRequests_.empty()) {\n+\t\tRequest *request = pendingRequests_.front();\n+\n+\t\tcompleteRequest(request);\n+\t\tpendingRequests_.pop();\n+\t}\n+}\n+\n+int PipelineHandlerMaliC55::queuePendingRequests()\n+{\n+\twhile (!pendingRequests_.empty()) {\n+\t\tRequest *request = pendingRequests_.front();\n+\n+\t\tif (availableStatsBuffers_.empty()) {\n+\t\t\tLOG(MaliC55, Error) << \"Stats buffer underrun\";\n+\t\t\treturn -ENOENT;\n+\t\t}\n+\n+\t\tif (availableParamsBuffers_.empty()) {\n+\t\t\tLOG(MaliC55, Error) << \"Params buffer underrun\";\n+\t\t\treturn -ENOENT;\n+\t\t}\n+\n+\t\tMaliC55FrameInfo frameInfo;\n+\t\tframeInfo.request = request;\n+\n+\t\tMaliC55CameraData *data = cameraData(request->_d()->camera());\n+\t\tframeInfo.rawBuffer = data->memoryInput.cru_->queueBuffer(request);\n+\t\tif (!frameInfo.rawBuffer)\n+\t\t\treturn -ENOENT;\n+\n+\t\tframeInfo.statBuffer = availableStatsBuffers_.front();\n+\t\tavailableStatsBuffers_.pop();\n+\t\tframeInfo.paramBuffer = availableParamsBuffers_.front();\n+\t\tavailableParamsBuffers_.pop();\n+\n+\t\tframeInfo.paramsDone = false;\n+\t\tframeInfo.statsDone = false;\n+\n+\t\tframeInfoMap_[request->sequence()] = frameInfo;\n+\n+\t\tfor (auto &[stream, buffer] : request->buffers()) {\n+\t\t\tMaliC55Pipe *pipe = pipeFromStream(data, stream);\n+\n+\t\t\tpipe->cap->queueBuffer(buffer);\n+\t\t}\n+\n+\t\tdata->ipa_->queueRequest(request->sequence(), request->controls());\n+\n+\t\tpendingRequests_.pop();\n+\t\tprocessingRequests_.push(request);\n+\t}\n+\n+\treturn 0;\n+}\n+\n int PipelineHandlerMaliC55::queueRequestDevice(Camera *camera, Request *request)\n {\n \tMaliC55CameraData *data = cameraData(camera);\n \n+\t/*\n+\t * If we're in memory input mode, we need to pop the requests onto the\n+\t * pending list until a CRU buffer is ready...otherwise we can just do\n+\t * everything immediately.\n+\t */\n+\tif (data->input_ == MaliC55CameraData::Memory) {\n+\t\tpendingRequests_.push(request);\n+\n+\t\tint ret = queuePendingRequests();\n+\t\tif (ret) {\n+\t\t\tpendingRequests_.pop();\n+\t\t\treturn ret;\n+\t\t}\n+\n+\t\treturn 0;\n+\t}\n+\n \t/* Do not run the IPA if the TPG is in use. */\n \tif (!data->ipa_) {\n \t\tMaliC55FrameInfo frameInfo;\n@@ -1556,7 +1678,8 @@ MaliC55FrameInfo *PipelineHandlerMaliC55::findFrameInfo(FrameBuffer *buffer)\n {\n \tfor (auto &[sequence, info] : frameInfoMap_) {\n \t\tif (info.paramBuffer == buffer ||\n-\t\t    info.statBuffer == buffer)\n+\t\t    info.statBuffer == buffer ||\n+\t\t    info.rawBuffer == buffer)\n \t\t\treturn &info;\n \t}\n \n@@ -1618,6 +1741,26 @@ void PipelineHandlerMaliC55::statsBufferReady(FrameBuffer *buffer)\n \t\t\t\t sensorControls);\n }\n \n+void PipelineHandlerMaliC55::cruBufferReady(FrameBuffer *buffer)\n+{\n+\tMaliC55FrameInfo *info = findFrameInfo(buffer);\n+\tRequest *request = info->request;\n+\tASSERT(info);\n+\n+\tif (buffer->metadata().status == FrameMetadata::FrameCancelled) {\n+\t\tframeInfoMap_.erase(request->sequence());\n+\t\tcompleteRequest(request);\n+\t\treturn;\n+\t}\n+\n+\trequest->metadata().set(controls::SensorTimestamp,\n+\t\t\t\tbuffer->metadata().timestamp);\n+\n+\t/* Ought we do something with the sensor's controls here...? */\n+\tMaliC55CameraData *data = cameraData(request->_d()->camera());\n+\tdata->ipa_->fillParams(request->sequence(), info->paramBuffer->cookie());\n+}\n+\n void PipelineHandlerMaliC55::paramsComputed(unsigned int requestId, uint32_t bytesused)\n {\n \tMaliC55FrameInfo &frameInfo = frameInfoMap_[requestId];\n@@ -1626,18 +1769,27 @@ void PipelineHandlerMaliC55::paramsComputed(unsigned int requestId, uint32_t byt\n \n \t/*\n \t * Queue buffers for stats and params, then queue buffers to the capture\n-\t * video devices.\n+\t * video devices if we're running in Inline mode or with the TPG.\n+\t *\n+\t * If we're running in M2M buffers have been queued to the capture\n+\t * devices at queuePendingRequests() time and here we only have to queue\n+\t * buffers to the IVC input to start a transfer.\n \t */\n \n \tframeInfo.paramBuffer->_d()->metadata().planes()[0].bytesused = bytesused;\n \tparams_->queueBuffer(frameInfo.paramBuffer);\n \tstats_->queueBuffer(frameInfo.statBuffer);\n \n-\tfor (auto &[stream, buffer] : request->buffers()) {\n-\t\tMaliC55Pipe *pipe = pipeFromStream(data, stream);\n+\tif (data->input_ != MaliC55CameraData::Memory) {\n+\t\tfor (auto &[stream, buffer] : request->buffers()) {\n+\t\t\tMaliC55Pipe *pipe = pipeFromStream(data, stream);\n \n-\t\tpipe->cap->queueBuffer(buffer);\n+\t\t\tpipe->cap->queueBuffer(buffer);\n+\t\t}\n \t}\n+\n+\tif (data->input_ == MaliC55CameraData::Memory)\n+\t\tinput_->queueBuffer(frameInfo.rawBuffer);\n }\n \n void PipelineHandlerMaliC55::statsProcessed(unsigned int requestId,\n@@ -1768,6 +1920,8 @@ bool PipelineHandlerMaliC55::registerMemoryInputCamera()\n \tisp_->frameStart.connect(data->delayedCtrls_.get(),\n \t\t\t\t &DelayedControls::applyControls);\n \n+\tV4L2VideoDevice *cruOutput = data->memoryInput.cru_->output();\n+\tcruOutput->bufferReady.connect(this, &PipelineHandlerMaliC55::cruBufferReady);\n \tinput_->bufferReady.connect(data->memoryInput.cru_.get(),\n \t\t\t\t    &RZG2LCRU::cruReturnBuffer);\n \n","prefixes":["5/7"]}