From patchwork Fri Mar 13 16:14:40 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jacopo Mondi X-Patchwork-Id: 26297 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 27B80C32EF for ; Fri, 13 Mar 2026 16:14:58 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id B015562730; Fri, 13 Mar 2026 17:14:57 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="pvnevIlt"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 0E54B626BE for ; Fri, 13 Mar 2026 17:14:56 +0100 (CET) Received: from [192.168.224.131] (unknown [37.159.122.93]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 0ACD21E20; Fri, 13 Mar 2026 17:13:42 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1773418426; bh=EwM0lPDrHo3KScdzd1O4arRS97/pjfMTXAzsxqxebOo=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=pvnevIltH3xz313fOyTI/BmByAWayOIbNiGHTkGnhtwmU9DJl7DcQ+juemYuVtCdx xLrs86SL+rpCfmvDU0K21H5xdgk3hQI1f6IPVyna93kUSFpBHefH73V5TxHzVSPfcm 6B4GvjUNIUSqgS8PpBWwfTb5D9rnoZfQIkL8fKDM= From: Jacopo Mondi Date: Fri, 13 Mar 2026 17:14:40 +0100 Subject: [PATCH v5 6/7] libcamera: mali-c55: Implement capture for memory-to-memory MIME-Version: 1.0 Message-Id: <20260313-mali-cru-v5-6-48f93e431294@ideasonboard.com> References: <20260313-mali-cru-v5-0-48f93e431294@ideasonboard.com> In-Reply-To: <20260313-mali-cru-v5-0-48f93e431294@ideasonboard.com> To: Daniel Scally , libcamera-devel@lists.libcamera.org Cc: Jacopo Mondi X-Mailer: b4 0.14.3 X-Developer-Signature: v=1; a=openpgp-sha256; l=9435; i=jacopo.mondi@ideasonboard.com; h=from:subject:message-id; bh=mGB/JK1uaiNUsm5JpSdV97aWgFIcj2ZI7a1IZpjhjX0=; b=owEBbQKS/ZANAwAKAXI0Bo8WoVY8AcsmYgBptDfz9LLvNbB0g/OtUm8noa2XBVB3a1FwGc3V2 ERsSe6O/Z6JAjMEAAEKAB0WIQS1xD1IgJogio9YOMByNAaPFqFWPAUCabQ38wAKCRByNAaPFqFW POWuD/0c+imOFOXwWwQ7agIZ3EiK4Q+TGqZGsJMlV7SMf7l6SLsm9Cm9FmWE9DBUx6+fcEm7HmQ V6nkSxS2pzO3tKQLOS57iGS+o5zEc2yML2OyVFTVBHC6pWyiUaPqqoVQkZdCNjFDIU8ijq9X0HJ Q5OaQBAYnFhQiTkly0PNxK++sYLaLAqzpc91Uq+0H7mRPzdgJjjy7CZVrH8jGm4KQyBTao3HG/5 NyR/aKItMS06cw6hL7plYb95jpS1A8Elrs/EZ+2NT6LsxZnp8xh2t2I700I4r8GAukVItr63zSY Jzrn2W+t50329AIPnVrRT8cjMulIx2ZnFocj9gkZahAb+ahrX/JiN69/tjRZZOXpQRBXb15rEpE uMMVoG2nVygudI5Hrph6MEsMMeFV4znRWo2sEmpRnW41atCuRPhZHdzpgXtqOFbJeEOBE+JrDBe nIHnFYwQF9PURoYw5sfvAiPtvNCB7SUkgzI3JQWkBBa4OD/7x2Sf2Q4Yi2Yg26U4tRB7ljWkHyb U8gsxvPxucpvRbt9xJShsT2XbpOx1r+SvPUBjrhS5bNxKDqHB66Qe+82Z3IUBd/cWuoO4Wi/Wq7 tXazUiWSBfpXfOcG1xkHS6IqarJOk5ec6wTAFw0r1AgAgPigWQO+C2K2do0hzJvVJ5wkqDnsQkZ qyfEt3qcCf4ZomQ== X-Developer-Key: i=jacopo.mondi@ideasonboard.com; a=openpgp; fpr=72392EDC88144A65C701EA9BA5826A2587AD026B X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" From: Daniel Scally Plumb in the MaliC55 pipeline handler support for capturing frames from memory using the CRU. Introduce a data flow which uses the CRU to feed the ISP through the IVC. In detail: - push incoming request to a pending queue until a buffer from the CRU is available - delay the call to ipa_->fillParams() to the CRU buffer ready even - once the IPA has computed parameters feed the ISP through the IVC with buffers from the CRU, params and statistics. Signed-off-by: Daniel Scally Signed-off-by: Jacopo Mondi --- src/libcamera/pipeline/mali-c55/mali-c55.cpp | 166 ++++++++++++++++++++++++++- 1 file changed, 161 insertions(+), 5 deletions(-) diff --git a/src/libcamera/pipeline/mali-c55/mali-c55.cpp b/src/libcamera/pipeline/mali-c55/mali-c55.cpp index 9b83927d8441..84c2a030b470 100644 --- a/src/libcamera/pipeline/mali-c55/mali-c55.cpp +++ b/src/libcamera/pipeline/mali-c55/mali-c55.cpp @@ -21,6 +21,7 @@ #include #include +#include #include #include #include @@ -88,6 +89,7 @@ struct MaliC55FrameInfo { FrameBuffer *paramBuffer; FrameBuffer *statBuffer; + FrameBuffer *rawBuffer; bool paramsDone; bool statsDone; @@ -721,11 +723,14 @@ public: int start(Camera *camera, const ControlList *controls) override; void stopDevice(Camera *camera) override; + int queuePendingRequests(MaliC55CameraData *data); + void cancelPendingRequests(); int queueRequestDevice(Camera *camera, Request *request) override; void imageBufferReady(FrameBuffer *buffer); void paramsBufferReady(FrameBuffer *buffer); void statsBufferReady(FrameBuffer *buffer); + void cruBufferReady(FrameBuffer *buffer); void paramsComputed(unsigned int requestId, uint32_t bytesused); void statsProcessed(unsigned int requestId, const ControlList &metadata); @@ -807,6 +812,11 @@ private: std::map frameInfoMap_; + /* Requests for which no buffer has been queued to the CRU device yet. */ + std::queue pendingRequests_; + /* Requests queued to the CRU device but not yet processed by the ISP. */ + std::queue processingRequests_; + std::array pipes_; bool dsFitted_; @@ -1255,6 +1265,11 @@ void PipelineHandlerMaliC55::freeBuffers(Camera *camera) if (params_->releaseBuffers()) LOG(MaliC55, Error) << "Failed to release params buffers"; + if (std::holds_alternative(data->input_)) { + if (ivc_->releaseBuffers()) + LOG(MaliC55, Error) << "Failed to release input buffers"; + } + return; } @@ -1284,6 +1299,12 @@ int PipelineHandlerMaliC55::allocateBuffers(Camera *camera) } }; + if (std::holds_alternative(data->input_)) { + ret = ivc_->importBuffers(RZG2LCRU::kBufferCount); + if (ret < 0) + return ret; + } + ret = stats_->allocateBuffers(bufferCount, &statsBuffers_); if (ret < 0) return ret; @@ -1315,6 +1336,24 @@ int PipelineHandlerMaliC55::start(Camera *camera, [[maybe_unused]] const Control if (ret) return ret; + if (std::holds_alternative(data->input_)) { + ret = data->cru()->start(); + if (ret) { + LOG(MaliC55, Error) + << "Failed to start CRU " << camera->id(); + freeBuffers(camera); + return ret; + } + + ret = ivc_->streamOn(); + if (ret) { + LOG(MaliC55, Error) + << "Failed to start IVC" << camera->id(); + freeBuffers(camera); + return ret; + } + } + if (data->ipa_) { ret = data->ipa_->start(); if (ret) { @@ -1404,6 +1443,12 @@ void PipelineHandlerMaliC55::stopDevice(Camera *camera) pipe.cap->releaseBuffers(); } + if (std::holds_alternative(data->input_)) { + cancelPendingRequests(); + ivc_->streamOff(); + data->cru()->stop(); + } + stats_->streamOff(); params_->streamOff(); if (data->ipa_) @@ -1507,10 +1552,88 @@ void PipelineHandlerMaliC55::applyScalerCrop(Camera *camera, } } +void PipelineHandlerMaliC55::cancelPendingRequests() +{ + processingRequests_ = {}; + + while (!pendingRequests_.empty()) { + Request *request = pendingRequests_.front(); + + completeRequest(request); + pendingRequests_.pop(); + } +} + +int PipelineHandlerMaliC55::queuePendingRequests(MaliC55CameraData *data) +{ + ASSERT(std::holds_alternative(data->input_)); + + while (!pendingRequests_.empty()) { + Request *request = pendingRequests_.front(); + + if (availableStatsBuffers_.empty()) { + LOG(MaliC55, Error) << "Stats buffer underrun"; + return -ENOENT; + } + + if (availableParamsBuffers_.empty()) { + LOG(MaliC55, Error) << "Params buffer underrun"; + return -ENOENT; + } + + MaliC55FrameInfo frameInfo; + frameInfo.request = request; + + frameInfo.rawBuffer = data->cru()->queueBuffer(request); + if (!frameInfo.rawBuffer) + return -ENOENT; + + frameInfo.statBuffer = availableStatsBuffers_.front(); + availableStatsBuffers_.pop(); + frameInfo.paramBuffer = availableParamsBuffers_.front(); + availableParamsBuffers_.pop(); + + frameInfo.paramsDone = false; + frameInfo.statsDone = false; + + frameInfoMap_[request->sequence()] = frameInfo; + + for (auto &[stream, buffer] : request->buffers()) { + MaliC55Pipe *pipe = pipeFromStream(data, stream); + + pipe->cap->queueBuffer(buffer); + } + + data->ipa_->queueRequest(request->sequence(), request->controls()); + + pendingRequests_.pop(); + processingRequests_.push(request); + } + + return 0; +} + int PipelineHandlerMaliC55::queueRequestDevice(Camera *camera, Request *request) { MaliC55CameraData *data = cameraData(camera); + /* + * If we're in memory input mode, we need to pop the requests onto the + * pending list until a CRU buffer is ready...otherwise we can just do + * everything immediately. + */ + if (std::holds_alternative(data->input_)) { + pendingRequests_.push(request); + + int ret = queuePendingRequests(data); + if (ret) { + pendingRequests_.pop(); + return ret; + } + + return 0; + } + /* Do not run the IPA if the TPG is in use. */ if (!data->ipa_) { MaliC55FrameInfo frameInfo; @@ -1575,7 +1698,8 @@ MaliC55FrameInfo *PipelineHandlerMaliC55::findFrameInfo(FrameBuffer *buffer) { for (auto &[sequence, info] : frameInfoMap_) { if (info.paramBuffer == buffer || - info.statBuffer == buffer) + info.statBuffer == buffer || + info.rawBuffer == buffer) return &info; } @@ -1637,6 +1761,26 @@ void PipelineHandlerMaliC55::statsBufferReady(FrameBuffer *buffer) sensorControls); } +void PipelineHandlerMaliC55::cruBufferReady(FrameBuffer *buffer) +{ + MaliC55FrameInfo *info = findFrameInfo(buffer); + Request *request = info->request; + ASSERT(info); + + if (buffer->metadata().status == FrameMetadata::FrameCancelled) { + frameInfoMap_.erase(request->sequence()); + completeRequest(request); + return; + } + + request->_d()->metadata().set(controls::SensorTimestamp, + buffer->metadata().timestamp); + + /* Ought we do something with the sensor's controls here...? */ + MaliC55CameraData *data = cameraData(request->_d()->camera()); + data->ipa_->fillParams(request->sequence(), info->paramBuffer->cookie()); +} + void PipelineHandlerMaliC55::paramsComputed(unsigned int requestId, uint32_t bytesused) { MaliC55FrameInfo &frameInfo = frameInfoMap_[requestId]; @@ -1645,18 +1789,27 @@ void PipelineHandlerMaliC55::paramsComputed(unsigned int requestId, uint32_t byt /* * Queue buffers for stats and params, then queue buffers to the capture - * video devices. + * video devices if we're running in Inline mode or with the TPG. + * + * If we're running in M2M buffers have been queued to the capture + * devices at queuePendingRequests() time and here we only have to queue + * buffers to the IVC input to start a transfer. */ frameInfo.paramBuffer->_d()->metadata().planes()[0].bytesused = bytesused; params_->queueBuffer(frameInfo.paramBuffer); stats_->queueBuffer(frameInfo.statBuffer); - for (auto &[stream, buffer] : request->buffers()) { - MaliC55Pipe *pipe = pipeFromStream(data, stream); + if (!std::holds_alternative(data->input_)) { + for (auto &[stream, buffer] : request->buffers()) { + MaliC55Pipe *pipe = pipeFromStream(data, stream); - pipe->cap->queueBuffer(buffer); + pipe->cap->queueBuffer(buffer); + } } + + if (std::holds_alternative(data->input_)) + ivc_->queueBuffer(frameInfo.rawBuffer); } void PipelineHandlerMaliC55::statsProcessed(unsigned int requestId, @@ -1793,6 +1946,9 @@ bool PipelineHandlerMaliC55::registerMemoryInputCamera() ivc_->bufferReady.connect(data->cru(), &RZG2LCRU::cruReturnBuffer); + V4L2VideoDevice *cruOutput = data->cru()->output(); + cruOutput->bufferReady.connect(this, &PipelineHandlerMaliC55::cruBufferReady); + return registerMaliCamera(std::move(data), sensor->device()->entity()->name()); }