[{"id":37224,"web_url":"https://patchwork.libcamera.org/comment/37224/","msgid":"<80eba103-a36d-4bc4-8d01-d7e9b837814f@ideasonboard.com>","date":"2025-12-08T10:13:22","subject":"Re: [PATCH 5/7] libcamera: mali-c55: Implement capture for\n\tmemory-to-memory","submitter":{"id":216,"url":"https://patchwork.libcamera.org/api/people/216/","name":"Barnabás Pőcze","email":"barnabas.pocze@ideasonboard.com"},"content":"Hi\n\n2025. 12. 05. 15:52 keltezéssel, Jacopo Mondi írta:\n> From: Daniel Scally \n> \n> Plumb in the MaliC55 pipeline handler support for capturing frames\n> from memory using the CRU.\n> \n> Introduce a data flow which uses the CRU to feed the ISP through\n> the IVC.\n> \n> In detail:\n> \n> - push incoming request to a pending queue until a buffer from the CRU\n> is available\n> - delay the call to ipa_->fillParams() to the CRU buffer ready even\n> - once the IPA has computed parameters feed the ISP through the IVC\n> with buffers from the CRU, params and statistics.\n> \n> Signed-off-by: Daniel Scally \n> Signed-off-by: Jacopo Mondi \n> ---\n> src/libcamera/pipeline/mali-c55/mali-c55.cpp | 164 ++++++++++++++++++++++++++-\n> 1 file changed, 159 insertions(+), 5 deletions(-)\n> \n> diff --git a/src/libcamera/pipeline/mali-c55/mali-c55.cpp b/src/libcamera/pipeline/mali-c55/mali-c55.cpp\n> index 6dce315c1c82db3554e8c0eae727cba9d632ca82..46ef5e7735a30a0d4ae9bb6f8d671bbd2dff3f51 100644\n> --- a/src/libcamera/pipeline/mali-c55/mali-c55.cpp\n> +++ b/src/libcamera/pipeline/mali-c55/mali-c55.cpp\n> @@ -19,6 +19,7 @@\n> [...]\n> +int PipelineHandlerMaliC55::queuePendingRequests()\n> +{\n> +\twhile (!pendingRequests_.empty()) {\n> +\t\tRequest *request = pendingRequests_.front();\n> +\n> +\t\tif (availableStatsBuffers_.empty()) {\n> +\t\t\tLOG(MaliC55, Error) << \"Stats buffer underrun\";\n> +\t\t\treturn -ENOENT;\n> +\t\t}\n> +\n> +\t\tif (availableParamsBuffers_.empty()) {\n> +\t\t\tLOG(MaliC55, Error) << \"Params buffer underrun\";\n> +\t\t\treturn -ENOENT;\n> +\t\t}\n> +\n> +\t\tMaliC55FrameInfo frameInfo;\n> +\t\tframeInfo.request = request;\n> +\n> +\t\tMaliC55CameraData *data = cameraData(request->_d()->camera());\n> +\t\tframeInfo.rawBuffer = data->memoryInput.cru_->queueBuffer(request);\n> +\t\tif (!frameInfo.rawBuffer)\n> +\t\t\treturn -ENOENT;\n> +\n> +\t\tframeInfo.statBuffer = availableStatsBuffers_.front();\n> +\t\tavailableStatsBuffers_.pop();\n> +\t\tframeInfo.paramBuffer = availableParamsBuffers_.front();\n> +\t\tavailableParamsBuffers_.pop();\n> +\n> +\t\tframeInfo.paramsDone = false;\n> +\t\tframeInfo.statsDone = false;\n> +\n> +\t\tframeInfoMap_[request->sequence()] = frameInfo;\n\nIf I understand it correctly, there is just a single memory-to-memory\ncamera per pipeline handler, that is why it is fine to store e.g.\n`pendingBuffers_` and `frameInfoMap_` in the pipeline handler and not\nin the camera, right?\n\n\n> +\n> +\t\tfor (auto &[stream, buffer] : request->buffers()) {\n> +\t\t\tMaliC55Pipe *pipe = pipeFromStream(data, stream);\n> +\n> +\t\t\tpipe->cap->queueBuffer(buffer);\n> +\t\t}\n> +\n> +\t\tdata->ipa_->queueRequest(request->sequence(), request->controls());\n> +\n> +\t\tpendingRequests_.pop();\n> +\t\tprocessingRequests_.push(request);\n> +\t}\n> +\n> +\treturn 0;\n> +}\n> +\n> int PipelineHandlerMaliC55::queueRequestDevice(Camera *camera, Request *request)\n> {\n> \tMaliC55CameraData *data = cameraData(camera);\n> \n> +\t/*\n> +\t * If we're in memory input mode, we need to pop the requests onto the\n> +\t * pending list until a CRU buffer is ready...otherwise we can just do\n> +\t * everything immediately.\n> +\t */\n> +\tif (data->input_ == MaliC55CameraData::Memory) {\n> +\t\tpendingRequests_.push(request);\n> +\n> +\t\tint ret = queuePendingRequests();\n> +\t\tif (ret) {\n> +\t\t\tpendingRequests_.pop();\n> +\t\t\treturn ret;\n> +\t\t}\n> +\n> +\t\treturn 0;\n> +\t}\n> +\n> \t/* Do not run the IPA if the TPG is in use. */\n> \tif (!data->ipa_) {\n> \t\tMaliC55FrameInfo frameInfo;\n> @@ -1556,7 +1678,8 @@ MaliC55FrameInfo *PipelineHandlerMaliC55::findFrameInfo(FrameBuffer *buffer)\n> {\n> \tfor (auto &[sequence, info] : frameInfoMap_) {\n> \t\tif (info.paramBuffer == buffer ||\n> -\t\t info.statBuffer == buffer)\n> +\t\t info.statBuffer == buffer ||\n> +\t\t info.rawBuffer == buffer)\n> \t\t\treturn &info;\n> \t}\n> \n> @@ -1618,6 +1741,26 @@ void PipelineHandlerMaliC55::statsBufferReady(FrameBuffer *buffer)\n> \t\t\t\t sensorControls);\n> }\n> [...]","headers":{"Return-Path":"","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id D3F58BDDFC\n\tfor ;\n\tMon, 8 Dec 2025 10:13:28 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id 064E1613FA;\n\tMon, 8 Dec 2025 11:13:28 +0100 (CET)","from perceval.ideasonboard.com (perceval.ideasonboard.com\n\t[IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id CA79C60C8A\n\tfor ;\n\tMon, 8 Dec 2025 11:13:26 +0100 (CET)","from [192.168.33.35] (185.221.142.68.nat.pool.zt.hu\n\t[185.221.142.68])\n\tby perceval.ideasonboard.com (Postfix) with ESMTPSA id 516B32E0;\n\tMon, 8 Dec 2025 11:11:07 +0100 (CET)"],"Authentication-Results":"lancelot.ideasonboard.com; dkim=pass (1024-bit key;\n\tunprotected) header.d=ideasonboard.com header.i=@ideasonboard.com\n\theader.b=\"vfb8+YtH\"; dkim-atps=neutral","DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com;\n\ts=mail; t=1765188667;\n\tbh=DzWQTL+mVBlKPqAHoVTRXJeMf77I2OgW+UNHJs3K/p8=;\n\th=Date:Subject:To:References:From:In-Reply-To:From;\n\tb=vfb8+YtHClmL2Q1MjzLqO/SNWqYf61tLYgB8xTjjLq4HZkxDlzopUXH/XB3o+1Crg\n\ti3QeawS21j9DoYcWOnUmgFg86+w8pMvHZFDKyPfNNnDoqHSNpQkdoUKRTgG+RPuLqi\n\tOKe4KUdXOlmK3gftdUS06xlVl11UVCiP6EuOYo6U=","Message-ID":"<80eba103-a36d-4bc4-8d01-d7e9b837814f@ideasonboard.com>","Date":"Mon, 8 Dec 2025 11:13:22 +0100","MIME-Version":"1.0","User-Agent":"Mozilla Thunderbird","Subject":"Re: [PATCH 5/7] libcamera: mali-c55: Implement capture for\n\tmemory-to-memory","To":"Jacopo Mondi ,\n\tDaniel Scally ,\n\tlibcamera-devel@lists.libcamera.org","References":"<20251205-mali-cru-v1-0-d81bb5ffe73a@ideasonboard.com>\n\t\n\t<20251205-mali-cru-v1-5-d81bb5ffe73a@ideasonboard.com>","From":"=?utf-8?q?Barnab=C3=A1s_P=C5=91cze?= ","Content-Language":"en-US, hu-HU","In-Reply-To":"<20251205-mali-cru-v1-5-d81bb5ffe73a@ideasonboard.com>","Content-Type":"text/plain; charset=UTF-8; format=flowed","Content-Transfer-Encoding":"8bit","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" "}},{"id":37255,"web_url":"https://patchwork.libcamera.org/comment/37255/","msgid":"","date":"2025-12-10T14:38:47","subject":"Re: [PATCH 5/7] libcamera: mali-c55: Implement capture for\n\tmemory-to-memory","submitter":{"id":143,"url":"https://patchwork.libcamera.org/api/people/143/","name":"Jacopo Mondi","email":"jacopo.mondi@ideasonboard.com"},"content":"Hi Barnabás\n\nOn Mon, Dec 08, 2025 at 11:13:22AM +0100, Barnabás Pőcze wrote:\n> Hi\n>\n> 2025. 12. 05. 15:52 keltezéssel, Jacopo Mondi írta:\n> > From: Daniel Scally \n> >\n> > Plumb in the MaliC55 pipeline handler support for capturing frames\n> > from memory using the CRU.\n> >\n> > Introduce a data flow which uses the CRU to feed the ISP through\n> > the IVC.\n> >\n> > In detail:\n> >\n> > - push incoming request to a pending queue until a buffer from the CRU\n> > is available\n> > - delay the call to ipa_->fillParams() to the CRU buffer ready even\n> > - once the IPA has computed parameters feed the ISP through the IVC\n> > with buffers from the CRU, params and statistics.\n> >\n> > Signed-off-by: Daniel Scally \n> > Signed-off-by: Jacopo Mondi \n> > ---\n> > src/libcamera/pipeline/mali-c55/mali-c55.cpp | 164 ++++++++++++++++++++++++++-\n> > 1 file changed, 159 insertions(+), 5 deletions(-)\n> >\n> > diff --git a/src/libcamera/pipeline/mali-c55/mali-c55.cpp b/src/libcamera/pipeline/mali-c55/mali-c55.cpp\n> > index 6dce315c1c82db3554e8c0eae727cba9d632ca82..46ef5e7735a30a0d4ae9bb6f8d671bbd2dff3f51 100644\n> > --- a/src/libcamera/pipeline/mali-c55/mali-c55.cpp\n> > +++ b/src/libcamera/pipeline/mali-c55/mali-c55.cpp\n> > @@ -19,6 +19,7 @@\n> > [...]\n> > +int PipelineHandlerMaliC55::queuePendingRequests()\n> > +{\n> > +\twhile (!pendingRequests_.empty()) {\n> > +\t\tRequest *request = pendingRequests_.front();\n> > +\n> > +\t\tif (availableStatsBuffers_.empty()) {\n> > +\t\t\tLOG(MaliC55, Error) << \"Stats buffer underrun\";\n> > +\t\t\treturn -ENOENT;\n> > +\t\t}\n> > +\n> > +\t\tif (availableParamsBuffers_.empty()) {\n> > +\t\t\tLOG(MaliC55, Error) << \"Params buffer underrun\";\n> > +\t\t\treturn -ENOENT;\n> > +\t\t}\n> > +\n> > +\t\tMaliC55FrameInfo frameInfo;\n> > +\t\tframeInfo.request = request;\n> > +\n> > +\t\tMaliC55CameraData *data = cameraData(request->_d()->camera());\n> > +\t\tframeInfo.rawBuffer = data->memoryInput.cru_->queueBuffer(request);\n> > +\t\tif (!frameInfo.rawBuffer)\n> > +\t\t\treturn -ENOENT;\n> > +\n> > +\t\tframeInfo.statBuffer = availableStatsBuffers_.front();\n> > +\t\tavailableStatsBuffers_.pop();\n> > +\t\tframeInfo.paramBuffer = availableParamsBuffers_.front();\n> > +\t\tavailableParamsBuffers_.pop();\n> > +\n> > +\t\tframeInfo.paramsDone = false;\n> > +\t\tframeInfo.statsDone = false;\n> > +\n> > +\t\tframeInfoMap_[request->sequence()] = frameInfo;\n>\n> If I understand it correctly, there is just a single memory-to-memory\n> camera per pipeline handler, that is why it is fine to store e.g.\n> `pendingBuffers_` and `frameInfoMap_` in the pipeline handler and not\n> in the camera, right?\n>\n\nSorry, I missed this question.\n\nYes, at the moment we don't have kernel interfaces in place to handle\nmultiple cameras.\n\nThe variables you mentioned, as well as the IVC/ISP devices will have\nto moved to CameraData, but to do we need to have support for contexts\nfirst, otherwise we could potentially end up configure the IVC and ISP\nwith different formats/sizes at every frame.\n\n>\n> > +\n> > +\t\tfor (auto &[stream, buffer] : request->buffers()) {\n> > +\t\t\tMaliC55Pipe *pipe = pipeFromStream(data, stream);\n> > +\n> > +\t\t\tpipe->cap->queueBuffer(buffer);\n> > +\t\t}\n> > +\n> > +\t\tdata->ipa_->queueRequest(request->sequence(), request->controls());\n> > +\n> > +\t\tpendingRequests_.pop();\n> > +\t\tprocessingRequests_.push(request);\n> > +\t}\n> > +\n> > +\treturn 0;\n> > +}\n> > +\n> > int PipelineHandlerMaliC55::queueRequestDevice(Camera *camera, Request *request)\n> > {\n> > \tMaliC55CameraData *data = cameraData(camera);\n> >\n> > +\t/*\n> > +\t * If we're in memory input mode, we need to pop the requests onto the\n> > +\t * pending list until a CRU buffer is ready...otherwise we can just do\n> > +\t * everything immediately.\n> > +\t */\n> > +\tif (data->input_ == MaliC55CameraData::Memory) {\n> > +\t\tpendingRequests_.push(request);\n> > +\n> > +\t\tint ret = queuePendingRequests();\n> > +\t\tif (ret) {\n> > +\t\t\tpendingRequests_.pop();\n> > +\t\t\treturn ret;\n> > +\t\t}\n> > +\n> > +\t\treturn 0;\n> > +\t}\n> > +\n> > \t/* Do not run the IPA if the TPG is in use. */\n> > \tif (!data->ipa_) {\n> > \t\tMaliC55FrameInfo frameInfo;\n> > @@ -1556,7 +1678,8 @@ MaliC55FrameInfo *PipelineHandlerMaliC55::findFrameInfo(FrameBuffer *buffer)\n> > {\n> > \tfor (auto &[sequence, info] : frameInfoMap_) {\n> > \t\tif (info.paramBuffer == buffer ||\n> > -\t\t info.statBuffer == buffer)\n> > +\t\t info.statBuffer == buffer ||\n> > +\t\t info.rawBuffer == buffer)\n> > \t\t\treturn &info;\n> > \t}\n> >\n> > @@ -1618,6 +1741,26 @@ void PipelineHandlerMaliC55::statsBufferReady(FrameBuffer *buffer)\n> > \t\t\t\t sensorControls);\n> > }\n> > [...]","headers":{"Return-Path":"","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id 5F196C3257\n\tfor ;\n\tWed, 10 Dec 2025 14:38:58 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id 9916061484;\n\tWed, 10 Dec 2025 15:38:57 +0100 (CET)","from perceval.ideasonboard.com (perceval.ideasonboard.com\n\t[IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id E412B613CB\n\tfor ;\n\tWed, 10 Dec 2025 15:38:55 +0100 (CET)","from ideasonboard.com (mob-5-90-55-146.net.vodafone.it\n\t[5.90.55.146])\n\tby perceval.ideasonboard.com (Postfix) with ESMTPSA id 59EA16DC;\n\tWed, 10 Dec 2025 15:38:53 +0100 (CET)"],"Authentication-Results":"lancelot.ideasonboard.com; dkim=pass (1024-bit key;\n\tunprotected) header.d=ideasonboard.com header.i=@ideasonboard.com\n\theader.b=\"oLdmvtiJ\"; dkim-atps=neutral","DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com;\n\ts=mail; t=1765377534;\n\tbh=9tJUtzps4x7W0F1/fXAAtKvJXu5xDXcBLIh6w0DzyU4=;\n\th=Date:From:To:Cc:Subject:References:In-Reply-To:From;\n\tb=oLdmvtiJrdtiUtnMwKoXlRGFl/ypP/nXhZvhSur9nae4yLq0iz8g9mcSqWrdsqyII\n\t2UB7pNBDcgFnK/9naTLy3bFP7SJmyg/CQoccG1oKLk6zL8m3Gb6J7LYp7fQyLv5qBq\n\tYVjQHzgB+tacKvN6HYVKatSf0ypWm79EyuMoyzUM=","Date":"Wed, 10 Dec 2025 15:38:47 +0100","From":"Jacopo Mondi ","To":"=?utf-8?q?Barnab=C3=A1s_P=C5=91cze?= ","Cc":"Jacopo Mondi , \n\tDaniel Scally ,\n\tlibcamera-devel@lists.libcamera.org","Subject":"Re: [PATCH 5/7] libcamera: mali-c55: Implement capture for\n\tmemory-to-memory","Message-ID":"","References":"<20251205-mali-cru-v1-0-d81bb5ffe73a@ideasonboard.com>\n\t\n\t<20251205-mali-cru-v1-5-d81bb5ffe73a@ideasonboard.com>\n\t<80eba103-a36d-4bc4-8d01-d7e9b837814f@ideasonboard.com>","MIME-Version":"1.0","Content-Type":"text/plain; charset=utf-8","Content-Disposition":"inline","Content-Transfer-Encoding":"8bit","In-Reply-To":"<80eba103-a36d-4bc4-8d01-d7e9b837814f@ideasonboard.com>","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" "}}]