From patchwork Wed Apr 1 16:25:53 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jacopo Mondi X-Patchwork-Id: 26410 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 7B822C32F5 for ; Wed, 1 Apr 2026 16:26:10 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id DE31862D6F; Wed, 1 Apr 2026 18:26:07 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="QJuJYTAi"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id E76E562D5C for ; Wed, 1 Apr 2026 18:26:01 +0200 (CEST) Received: from [100.93.44.16] (net-93-65-100-155.cust.vodafonedsl.it [93.65.100.155]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id BC0511752; Wed, 1 Apr 2026 18:24:38 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1775060678; bh=8GJoWmvnnhVfjvW8kFKHhENpJe9siUMg4bcn9jHEZB8=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=QJuJYTAihH+K0TBWQhsHttBz123BDtQOYx57cAWoSh/hNJys8/m4R5VSlkttkMUmz HXMXf6lST09eZnEkX8FxkROQImBIEzUvPLNMOU4KBWD9+1jvYN9STm9N8q5Nu7HrIh 1Z2i5hIyVQ6FW7/Nti2GYMa89F3C7rBkCZ6+sxwM= From: Jacopo Mondi Date: Wed, 01 Apr 2026 18:25:53 +0200 Subject: [PATCH v8 4/8] libcamera: mali-c55: Register memory input camera MIME-Version: 1.0 Message-Id: <20260401-mali-cru-v8-4-44c48f990e28@ideasonboard.com> References: <20260401-mali-cru-v8-0-44c48f990e28@ideasonboard.com> In-Reply-To: <20260401-mali-cru-v8-0-44c48f990e28@ideasonboard.com> To: Daniel Scally , libcamera-devel@lists.libcamera.org Cc: =?utf-8?q?Barnab=C3=A1s_P=C5=91cze?= , Jacopo Mondi X-Mailer: b4 0.14.3 X-Developer-Signature: v=1; a=openpgp-sha256; l=9956; i=jacopo.mondi@ideasonboard.com; h=from:subject:message-id; bh=8GJoWmvnnhVfjvW8kFKHhENpJe9siUMg4bcn9jHEZB8=; b=owEBbQKS/ZANAwAKAXI0Bo8WoVY8AcsmYgBpzUcXEZvNpY1/FDChnfynSpRk9GRsSqJrNzNNK drOKlHeJRiJAjMEAAEKAB0WIQS1xD1IgJogio9YOMByNAaPFqFWPAUCac1HFwAKCRByNAaPFqFW POxEEAC/d5s2IFZhiYjOEyhQ/D63bPgs3XL+TyHJHsqPEJeLrWdPwWw4UpxK64COlzr1MHABmmS jJxHAuQ47Yr0C7HqQvyaS4vrd/vQui5ywNxjfioPrpkmx3IHvfx5ntAt71Vy4AhzTy+45yCrb7r /AhI5WeKODAkwgLwWeBTIVTVbbi1psdtLDX1ztHd5bupFpsmZK+jcEzvVIafqorQe3oLMusQ/Hs slqWA6anXCSupeAjto6p4fsfyiOuqsFX/UKN5LAF2EPMqLmlyEFXuSmNVuKipu8IjnCXczR3tc9 ijazg43K1YKcbBJ7bKAVXuQFlAQqEgIJMA5RnOsniohBHt9nEuB3B4/Eksl1uaYOjpgt88V0B0q ZmhB/EpduEmZc3sCDYVkuVfZR96VZAxJBf+AbmwSnKSFQO+PaLtqw8bZrZV/fhG5oDnNe3FczUJ zK60cfe/Q+9/LtZiGgwDuxLbi7+aqPMOdArSUvm+uBY49s3JQoEZF1G2hAtBqp5yvwSRN7bt/D4 X+nLr7N+LA7MmL/67+gWkpj9LlW8ILHx0LYkP4/qWJ6/yt8magDHXbELKLhHxjRFlwg3R15sm1F nnnnmBLn758MsowymXIg1szXz/n4Geqe+U0nRwErd6hh5D15eUmDpB6jUR5hv5K2mXhInhtskxi XgzRRy3pKqzEvQg== X-Developer-Key: i=jacopo.mondi@ideasonboard.com; a=openpgp; fpr=72392EDC88144A65C701EA9BA5826A2587AD026B X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Add support to Mali C55 pipeline handler for memory-to-memory camera mode. The Mali-C55, as integrated in the Renesas RZ/V2H(P) SoC, is fed with images saved to memory by the CRU unit and read on behalf of the ISP by the IVC. Update the pipeline handler's match() function to search for the CRU media entity necessary to register a memory input camera. Create and initialize a CameraData instance for the memory camera use case if a valid CRU is found and initialize the IVC components used to feed the ISP with images read from memory. Do not implement camera configuration for the time being to reuce the patch size. It will be provided in the following patches. Signed-off-by: Daniel Scally Signed-off-by: Jacopo Mondi --- src/libcamera/pipeline/mali-c55/mali-c55.cpp | 154 +++++++++++++++++++++++---- 1 file changed, 136 insertions(+), 18 deletions(-) diff --git a/src/libcamera/pipeline/mali-c55/mali-c55.cpp b/src/libcamera/pipeline/mali-c55/mali-c55.cpp index 26dfd0aef73c..d17b026db119 100644 --- a/src/libcamera/pipeline/mali-c55/mali-c55.cpp +++ b/src/libcamera/pipeline/mali-c55/mali-c55.cpp @@ -44,6 +44,8 @@ #include "libcamera/internal/v4l2_subdevice.h" #include "libcamera/internal/v4l2_videodevice.h" +#include "rzg2l-cru.h" + namespace { bool isFormatRaw(const libcamera::PixelFormat &pixFmt) @@ -105,7 +107,12 @@ public: std::unique_ptr csi2_; std::unique_ptr sensor_; }; - using CameraType = std::variant; + + struct Memory { + std::unique_ptr cru_; + }; + + using CameraType = std::variant; MaliC55CameraData(PipelineHandler *pipe) : Camera::Private(pipe) @@ -116,6 +123,7 @@ public: Tpg *initTpg(MediaEntity *entity); Inline *initInline(MediaEntity *entity); + Memory *initMemory(MediaDevice *cruMedia); std::vector sizes(unsigned int mbusCode) const { @@ -125,7 +133,10 @@ public: }, [&](const Inline &in) -> std::vector { return in.sensor_->sizes(mbusCode); - } + }, + [&](const Memory &mem) -> std::vector { + return mem.cru_->sizes(); + }, }, input_); } @@ -138,6 +149,9 @@ public: [&](const Inline &in) -> V4L2Subdevice * { return in.sensor_->device(); }, + [&](const Memory &mem) -> V4L2Subdevice * { + return mem.cru_->sensor()->device(); + }, }, input_); } @@ -151,6 +165,9 @@ public: [&](const Inline &in) -> CameraSensor * { return in.sensor_.get(); }, + [&](const Memory &mem) -> CameraSensor * { + return mem.cru_->sensor(); + }, }, input_); } @@ -163,6 +180,9 @@ public: [&](const Inline &in) -> Size { return in.sensor_->resolution(); }, + [&](const Memory &mem) -> Size { + return mem.cru_->resolution(); + }, }, input_); } @@ -241,6 +261,19 @@ MaliC55CameraData::Inline *MaliC55CameraData::initInline(MediaEntity *sensor) return &input_.emplace(std::move(in)); } +MaliC55CameraData::Memory *MaliC55CameraData::initMemory(MediaDevice *cruMedia) +{ + Memory mem; + + mem.cru_ = std::make_unique(); + + int ret = mem.cru_->init(cruMedia); + if (ret) + return nullptr; + + return &input_.emplace(std::move(mem)); +} + std::vector MaliC55CameraData::Tpg::sizes(unsigned int mbusCode) const { V4L2Subdevice::Formats formats = sd_->formats(0); @@ -721,11 +754,15 @@ private: const std::string &name); bool registerTPGCamera(MediaLink *link); bool registerSensorCamera(MediaLink *link); + bool registerMemoryInputCamera(MediaLink *link); std::shared_ptr media_; + std::shared_ptr cruMedia_; std::unique_ptr isp_; std::unique_ptr stats_; std::unique_ptr params_; + std::unique_ptr ivcSd_; + std::unique_ptr ivc_; std::vector> statsBuffers_; std::queue availableStatsBuffers_; @@ -976,6 +1013,9 @@ int PipelineHandlerMaliC55::configure(Camera *camera, const MediaEntity *csi2Entity = in.csi2_->entity(); return csi2Entity->getPadByIndex(1)->links()[0]->setEnabled(true); }, + [](MaliC55CameraData::Memory &) { + return 0; + }, }, data->input_); if (ret) return ret; @@ -1001,6 +1041,9 @@ int PipelineHandlerMaliC55::configure(Camera *camera, return in.csi2_->getFormat(1, &subdevFormat); }, + [](MaliC55CameraData::Memory &) { + return 0; + }, }, data->input_); if (ret) return ret; @@ -1673,6 +1716,66 @@ bool PipelineHandlerMaliC55::registerSensorCamera(MediaLink *ispLink) return true; } +bool PipelineHandlerMaliC55::registerMemoryInputCamera(MediaLink *link) +{ + /* + * Errors are not fatal at this point, return true and just skip + * registering a camera for this media link. + */ + const std::string &name = link->source()->entity()->name(); + if (name != "rzv2h ivc block") { + LOG(MaliC55, Warning) << "Unsupported direct connection to " + << link->source()->entity()->name(); + return true; + } + + if (!cruMedia_) { + LOG(MaliC55, Warning) << "Unable to find CRU for memory input"; + return true; + } + + /* + * From now on, errors should be reported up as we have the components + * we need to support memory-to-memory and if we fail something has + * to be fixed. + */ + + ivcSd_ = V4L2Subdevice::fromEntityName(media_.get(), + "rzv2h ivc block"); + if (!ivcSd_ || ivcSd_->open() < 0) + return false; + + ivc_ = V4L2VideoDevice::fromEntityName(media_.get(), + "rzv2h-ivc"); + if (!ivc_ || ivc_->open() < 0) + return false; + + std::unique_ptr data = + std::make_unique(this); + + auto *mem = data->initMemory(cruMedia_.get()); + if (!mem) + return false; + + CameraSensor *sensor = data->sensor(); + data->properties_ = sensor->properties(); + + const CameraSensorProperties::SensorDelays &delays = sensor->sensorDelays(); + std::unordered_map params = { + { V4L2_CID_ANALOGUE_GAIN, { delays.gainDelay, false } }, + { V4L2_CID_EXPOSURE, { delays.exposureDelay, false } }, + }; + + data->delayedCtrls_ = + std::make_unique(sensor->device(), params); + isp_->frameStart.connect(data->delayedCtrls_.get(), + &DelayedControls::applyControls); + + ivc_->bufferReady.connect(mem->cru_.get(), &RZG2LCRU::returnBuffer); + + return registerMaliCamera(std::move(data), sensor->device()->entity()->name()); +} + bool PipelineHandlerMaliC55::match(DeviceEnumerator *enumerator) { const MediaPad *ispSink; @@ -1682,14 +1785,14 @@ bool PipelineHandlerMaliC55::match(DeviceEnumerator *enumerator) * The TPG and the downscale pipe are both optional blocks and may not * be fitted. */ - DeviceMatch dm("mali-c55"); - dm.add("mali-c55 isp"); - dm.add("mali-c55 resizer fr"); - dm.add("mali-c55 fr"); - dm.add("mali-c55 3a stats"); - dm.add("mali-c55 3a params"); - - media_ = acquireMediaDevice(enumerator, dm); + DeviceMatch c55Dm("mali-c55"); + c55Dm.add("mali-c55 isp"); + c55Dm.add("mali-c55 resizer fr"); + c55Dm.add("mali-c55 fr"); + c55Dm.add("mali-c55 3a stats"); + c55Dm.add("mali-c55 3a params"); + + media_ = acquireMediaDevice(enumerator, c55Dm); if (!media_) return false; @@ -1749,6 +1852,25 @@ bool PipelineHandlerMaliC55::match(DeviceEnumerator *enumerator) stats_->bufferReady.connect(this, &PipelineHandlerMaliC55::statsBufferReady); params_->bufferReady.connect(this, &PipelineHandlerMaliC55::paramsBufferReady); + /* + * We also need to search for the rzg2l-cru CSI-2 receiver. If we find + * that then we need to work in memory input mode instead of the inline + * mode. The absence of this match is not necessarily a failure at this + * point...it depends on the media links that we investigate momentarily. + * + * This is a bit hacky, because there could be multiple of these media + * devices and we're just taking the first. We need modular pipelines to + * properly solve the issue. + */ + static const std::regex cruCsi2Regex("csi-[0-9a-f]{8}.csi2"); + static const std::regex cruIpRegex("cru-ip-[0-9a-f]{8}.cru[0-9]"); + + DeviceMatch cruDm("rzg2l_cru"); + cruDm.add(cruCsi2Regex); + cruDm.add(cruIpRegex); + cruDm.add("CRU output"); + cruMedia_ = acquireMediaDevice(enumerator, cruDm); + ispSink = isp_->entity()->getPadByIndex(0); if (!ispSink || ispSink->links().empty()) { LOG(MaliC55, Error) << "ISP sink pad error"; @@ -1762,13 +1884,6 @@ bool PipelineHandlerMaliC55::match(DeviceEnumerator *enumerator) * MEDIA_ENT_F_CAM_SENSOR - The test pattern generator * MEDIA_ENT_F_VID_IF_BRIDGE - A CSI-2 receiver * MEDIA_ENT_F_PROC_VIDEO_PIXEL_FORMATTER - An input device - * - * The last one will be unsupported for now. The TPG is relatively easy, - * we just register a Camera for it. If we have a CSI-2 receiver we need - * to check its sink pad and register Cameras for anything connected to - * it (probably...there are some complex situations in which that might - * not be true but let's pretend they don't exist until we come across - * them) */ bool registered; for (MediaLink *link : ispSink->links()) { @@ -1788,7 +1903,10 @@ bool PipelineHandlerMaliC55::match(DeviceEnumerator *enumerator) break; case MEDIA_ENT_F_PROC_VIDEO_PIXEL_FORMATTER: - LOG(MaliC55, Warning) << "Memory input not yet supported"; + registered = registerMemoryInputCamera(link); + if (!registered) + return registered; + break; default: LOG(MaliC55, Error) << "Unsupported entity function";