From patchwork Wed Dec 10 14:39:21 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jacopo Mondi X-Patchwork-Id: 25462 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id EA2ABC326B for ; Wed, 10 Dec 2025 14:39:40 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 63CA661495; Wed, 10 Dec 2025 15:39:39 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="FUZFw9Pc"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id E5E8961499 for ; Wed, 10 Dec 2025 15:39:33 +0100 (CET) Received: from [192.168.1.106] (mob-5-90-55-146.net.vodafone.it [5.90.55.146]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 7F0D610C4; Wed, 10 Dec 2025 15:39:32 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1765377572; bh=iqmzamUpUgXPzSUkiK1h0Gp1XC6pXJetUbGunDuyNzU=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=FUZFw9Pcw6oqeQ4FOIhGgSK5Fr/UMsrdty2OsRxVtrynx0rb1xzIPImg3XR7YNZK6 zx37tWiIfAKOQkvXZ9SlYMXqC1lX/FrDnnyp/pVIUIT81H92U5RQJLn95vGsGRwCI9 8BH3Nc6bF7TsVEbN8Sv5mOONZOhmWNkaNSbjSsrg= From: Jacopo Mondi Date: Wed, 10 Dec 2025 15:39:21 +0100 Subject: [PATCH v2 5/7] libcamera: mali-c55: Implement capture for memory-to-memory MIME-Version: 1.0 Message-Id: <20251210-mali-cru-v2-5-e26421de202b@ideasonboard.com> References: <20251210-mali-cru-v2-0-e26421de202b@ideasonboard.com> In-Reply-To: <20251210-mali-cru-v2-0-e26421de202b@ideasonboard.com> To: Daniel Scally , libcamera-devel@lists.libcamera.org Cc: Jacopo Mondi X-Mailer: b4 0.14.2 X-Developer-Signature: v=1; a=openpgp-sha256; l=9438; i=jacopo.mondi@ideasonboard.com; h=from:subject:message-id; bh=IWn36FKOAlst4y5gWEgsituAx7g1CDa+O1cF2zmWej4=; b=owEBbQKS/ZANAwAKAXI0Bo8WoVY8AcsmYgBpOYYjx+J/n2otU2sebkVQmtV0KIprL+8Lxa92I CmXDvQqSpSJAjMEAAEKAB0WIQS1xD1IgJogio9YOMByNAaPFqFWPAUCaTmGIwAKCRByNAaPFqFW PAjkD/0Q9cBCmBijnzRUqDNmvMoAADYBfdXEFzjRzP4iJb7tgwBbWOrY5CeNVgko9hHHf6Z7bkP C0lQcscVOzGmZqhqhvsXhT2xLCNT2i7TL5YizIDV+IyaMOen6G6AnIBsxFrRocyrcEO6U/PTkg6 n0f9JquU37vgqTBCLnavwm9HyGXIoZZJs3kdqsP5VOLgwe5GNiWhWWalJ/CN54LQRHq0nekQFyt lloBB8iro8RRPVIcMHuWIwIUDv85wZ9x1BZGTmILJ6t0ZIeIpYI0svzd2soYyHTkwysa2nXhmGQ 7mht6zlhpHN6wMqSIa2n0Yk46veKAsyAMBYYlfglGC4r1hdP2OF8aO+aZoEB8m2Jcq5oMtH2sbW 6rAE8AtoYFxBu7Dtafb63LnJ/hseREHaIOJvy6KOdvneVzR9OpSu9fA4a9/GhcsZSTa7ZH616xC YbWhMo7e0qBFIsZdKaeJ3M1HKJsQ/oKGEvyAk6KAFq7y4jmoxZClhywp5GZ1L36GlrszPCpO3MG SUB6qwXQ7Z70q4RVX1OMRxxSXrkg5M5UEONir+amfylEnFmIVxz+M4ArqelBvEd/74HTjnLfSsd 2AkXRyk5aLZ5Bl1H9LOeHxSTsiJhEWut2KQ+U/g9eRQ/N/U/MUZMiRu9uEBn5e4O4Wr31OqF5Lt wZZP5/MjPjaiBhQ== X-Developer-Key: i=jacopo.mondi@ideasonboard.com; a=openpgp; fpr=72392EDC88144A65C701EA9BA5826A2587AD026B X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" From: Daniel Scally Plumb in the MaliC55 pipeline handler support for capturing frames from memory using the CRU. Introduce a data flow which uses the CRU to feed the ISP through the IVC. In detail: - push incoming request to a pending queue until a buffer from the CRU is available - delay the call to ipa_->fillParams() to the CRU buffer ready even - once the IPA has computed parameters feed the ISP through the IVC with buffers from the CRU, params and statistics. Signed-off-by: Daniel Scally Signed-off-by: Jacopo Mondi --- src/libcamera/pipeline/mali-c55/mali-c55.cpp | 166 ++++++++++++++++++++++++++- 1 file changed, 161 insertions(+), 5 deletions(-) diff --git a/src/libcamera/pipeline/mali-c55/mali-c55.cpp b/src/libcamera/pipeline/mali-c55/mali-c55.cpp index 8dcc51e27020c754004cd98cbdf73e771275b059..83cb10af5d3c0588b25e4a3190f561f09627d239 100644 --- a/src/libcamera/pipeline/mali-c55/mali-c55.cpp +++ b/src/libcamera/pipeline/mali-c55/mali-c55.cpp @@ -19,6 +19,7 @@ #include #include +#include #include #include #include @@ -86,6 +87,7 @@ struct MaliC55FrameInfo { FrameBuffer *paramBuffer; FrameBuffer *statBuffer; + FrameBuffer *rawBuffer; bool paramsDone; bool statsDone; @@ -775,11 +777,14 @@ public: int start(Camera *camera, const ControlList *controls) override; void stopDevice(Camera *camera) override; + int queuePendingRequests(MaliC55CameraData *data); + void cancelPendingRequests(); int queueRequestDevice(Camera *camera, Request *request) override; void imageBufferReady(FrameBuffer *buffer); void paramsBufferReady(FrameBuffer *buffer); void statsBufferReady(FrameBuffer *buffer); + void cruBufferReady(FrameBuffer *buffer); void paramsComputed(unsigned int requestId, uint32_t bytesused); void statsProcessed(unsigned int requestId, const ControlList &metadata); @@ -861,6 +866,11 @@ private: std::map frameInfoMap_; + /* Requests for which no buffer has been queued to the CRU device yet. */ + std::queue pendingRequests_; + /* Requests queued to the CRU device but not yet processed by the ISP. */ + std::queue processingRequests_; + std::array pipes_; bool dsFitted_; @@ -1320,6 +1330,11 @@ void PipelineHandlerMaliC55::freeBuffers(Camera *camera) if (params_->releaseBuffers()) LOG(MaliC55, Error) << "Failed to release params buffers"; + if (data->type() == MaliC55CameraData::CameraType::Memory) { + if (input_->releaseBuffers()) + LOG(MaliC55, Error) << "Failed to release input buffers"; + } + return; } @@ -1349,6 +1364,12 @@ int PipelineHandlerMaliC55::allocateBuffers(Camera *camera) } }; + if (input_) { + ret = input_->importBuffers(RZG2LCRU::kBufferCount); + if (ret < 0) + return ret; + } + ret = stats_->allocateBuffers(bufferCount, &statsBuffers_); if (ret < 0) return ret; @@ -1380,6 +1401,24 @@ int PipelineHandlerMaliC55::start(Camera *camera, [[maybe_unused]] const Control if (ret) return ret; + if (data->type() == MaliC55CameraData::CameraType::Memory) { + ret = data->cru()->start(); + if (ret) { + LOG(MaliC55, Error) + << "Failed to start CRU " << camera->id(); + freeBuffers(camera); + return ret; + } + + ret = input_->streamOn(); + if (ret) { + LOG(MaliC55, Error) + << "Failed to start IVC" << camera->id(); + freeBuffers(camera); + return ret; + } + } + if (data->ipa_) { ret = data->ipa_->start(); if (ret) { @@ -1469,6 +1508,12 @@ void PipelineHandlerMaliC55::stopDevice(Camera *camera) pipe.cap->releaseBuffers(); } + if (data->type() == MaliC55CameraData::CameraType::Memory) { + cancelPendingRequests(); + input_->streamOff(); + data->cru()->stop(); + } + stats_->streamOff(); params_->streamOff(); if (data->ipa_) @@ -1572,10 +1617,88 @@ void PipelineHandlerMaliC55::applyScalerCrop(Camera *camera, } } +void PipelineHandlerMaliC55::cancelPendingRequests() +{ + processingRequests_ = {}; + + while (!pendingRequests_.empty()) { + Request *request = pendingRequests_.front(); + + completeRequest(request); + pendingRequests_.pop(); + } +} + +int PipelineHandlerMaliC55::queuePendingRequests(MaliC55CameraData *data) +{ + ASSERT(data->type() == MaliC55CameraData::CameraType::Memory); + + while (!pendingRequests_.empty()) { + Request *request = pendingRequests_.front(); + + if (availableStatsBuffers_.empty()) { + LOG(MaliC55, Error) << "Stats buffer underrun"; + return -ENOENT; + } + + if (availableParamsBuffers_.empty()) { + LOG(MaliC55, Error) << "Params buffer underrun"; + return -ENOENT; + } + + MaliC55FrameInfo frameInfo; + frameInfo.request = request; + + frameInfo.rawBuffer = data->cru()->queueBuffer(request); + if (!frameInfo.rawBuffer) + return -ENOENT; + + frameInfo.statBuffer = availableStatsBuffers_.front(); + availableStatsBuffers_.pop(); + frameInfo.paramBuffer = availableParamsBuffers_.front(); + availableParamsBuffers_.pop(); + + frameInfo.paramsDone = false; + frameInfo.statsDone = false; + + frameInfoMap_[request->sequence()] = frameInfo; + + for (auto &[stream, buffer] : request->buffers()) { + MaliC55Pipe *pipe = pipeFromStream(data, stream); + + pipe->cap->queueBuffer(buffer); + } + + data->ipa_->queueRequest(request->sequence(), request->controls()); + + pendingRequests_.pop(); + processingRequests_.push(request); + } + + return 0; +} + int PipelineHandlerMaliC55::queueRequestDevice(Camera *camera, Request *request) { MaliC55CameraData *data = cameraData(camera); + /* + * If we're in memory input mode, we need to pop the requests onto the + * pending list until a CRU buffer is ready...otherwise we can just do + * everything immediately. + */ + if (data->type() == MaliC55CameraData::CameraType::Memory) { + pendingRequests_.push(request); + + int ret = queuePendingRequests(data); + if (ret) { + pendingRequests_.pop(); + return ret; + } + + return 0; + } + /* Do not run the IPA if the TPG is in use. */ if (!data->ipa_) { MaliC55FrameInfo frameInfo; @@ -1640,7 +1763,8 @@ MaliC55FrameInfo *PipelineHandlerMaliC55::findFrameInfo(FrameBuffer *buffer) { for (auto &[sequence, info] : frameInfoMap_) { if (info.paramBuffer == buffer || - info.statBuffer == buffer) + info.statBuffer == buffer || + info.rawBuffer == buffer) return &info; } @@ -1702,6 +1826,26 @@ void PipelineHandlerMaliC55::statsBufferReady(FrameBuffer *buffer) sensorControls); } +void PipelineHandlerMaliC55::cruBufferReady(FrameBuffer *buffer) +{ + MaliC55FrameInfo *info = findFrameInfo(buffer); + Request *request = info->request; + ASSERT(info); + + if (buffer->metadata().status == FrameMetadata::FrameCancelled) { + frameInfoMap_.erase(request->sequence()); + completeRequest(request); + return; + } + + request->metadata().set(controls::SensorTimestamp, + buffer->metadata().timestamp); + + /* Ought we do something with the sensor's controls here...? */ + MaliC55CameraData *data = cameraData(request->_d()->camera()); + data->ipa_->fillParams(request->sequence(), info->paramBuffer->cookie()); +} + void PipelineHandlerMaliC55::paramsComputed(unsigned int requestId, uint32_t bytesused) { MaliC55FrameInfo &frameInfo = frameInfoMap_[requestId]; @@ -1710,18 +1854,27 @@ void PipelineHandlerMaliC55::paramsComputed(unsigned int requestId, uint32_t byt /* * Queue buffers for stats and params, then queue buffers to the capture - * video devices. + * video devices if we're running in Inline mode or with the TPG. + * + * If we're running in M2M buffers have been queued to the capture + * devices at queuePendingRequests() time and here we only have to queue + * buffers to the IVC input to start a transfer. */ frameInfo.paramBuffer->_d()->metadata().planes()[0].bytesused = bytesused; params_->queueBuffer(frameInfo.paramBuffer); stats_->queueBuffer(frameInfo.statBuffer); - for (auto &[stream, buffer] : request->buffers()) { - MaliC55Pipe *pipe = pipeFromStream(data, stream); + if (data->type() != MaliC55CameraData::CameraType::Memory) { + for (auto &[stream, buffer] : request->buffers()) { + MaliC55Pipe *pipe = pipeFromStream(data, stream); - pipe->cap->queueBuffer(buffer); + pipe->cap->queueBuffer(buffer); + } } + + if (data->type() == MaliC55CameraData::CameraType::Memory) + input_->queueBuffer(frameInfo.rawBuffer); } void PipelineHandlerMaliC55::statsProcessed(unsigned int requestId, @@ -1863,6 +2016,9 @@ bool PipelineHandlerMaliC55::registerMemoryInputCamera() input_->bufferReady.connect(memoryData->cru(), &RZG2LCRU::cruReturnBuffer); + V4L2VideoDevice *cruOutput = memoryData->cru()->output(); + cruOutput->bufferReady.connect(this, &PipelineHandlerMaliC55::cruBufferReady); + std::unique_ptr data(memoryData); if (!registerMaliCamera(std::move(data), sensor->device()->entity()->name())) return false;