From patchwork Fri Mar 13 11:33:45 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jacopo Mondi X-Patchwork-Id: 26287 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id CB350BE086 for ; Fri, 13 Mar 2026 11:34:30 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 7F7166268F; Fri, 13 Mar 2026 12:34:30 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="alIezC1o"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 5CC0762695 for ; Fri, 13 Mar 2026 12:34:28 +0100 (CET) Received: from [192.168.224.131] (unknown [37.159.92.229]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id B96801783; Fri, 13 Mar 2026 12:33:17 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1773401599; bh=4IEqM7B3Z4FbQ2TizCoNOEr5YoUJQyY+Zj02Cdcr2LU=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=alIezC1oflvEB+qaI60YDxPVd8mLYGOHPJqnSB2sgJ+Y1bk0i6BaCV6PdY5dDcMiP x8+INS+enE1kLfVQphhV6wJE6vWZi8DyVhkR8L4t46Jp9Y4OpsnSyN7zHTWth0Syz1 H5dPhxnt15SKn+JMLYBoyVqzlFh5QBJ4pMURaTQI= From: Jacopo Mondi Date: Fri, 13 Mar 2026 12:33:45 +0100 Subject: [PATCH v4 4/7] libcamera: mali-c55: Register memory input camera MIME-Version: 1.0 Message-Id: <20260313-mali-cru-v4-4-c0d9bc8cd8fa@ideasonboard.com> References: <20260313-mali-cru-v4-0-c0d9bc8cd8fa@ideasonboard.com> In-Reply-To: <20260313-mali-cru-v4-0-c0d9bc8cd8fa@ideasonboard.com> To: Daniel Scally , libcamera-devel@lists.libcamera.org Cc: Jacopo Mondi X-Mailer: b4 0.14.3 X-Developer-Signature: v=1; a=openpgp-sha256; l=9060; i=jacopo.mondi@ideasonboard.com; h=from:subject:message-id; bh=v+nSg38rCLd41D6bC4lR0TADFlsu+yXwxFH+Di1xrDU=; b=owEBbQKS/ZANAwAKAXI0Bo8WoVY8AcsmYgBps/Y0dMX2GLRQAV6osQ9hcx6np1M1GdW8K+bgw 1/FN1/0LoSJAjMEAAEKAB0WIQS1xD1IgJogio9YOMByNAaPFqFWPAUCabP2NAAKCRByNAaPFqFW PGS3D/9q21tk2HREJqtSc209wipUMw2s6x68VrDtYNwK1CdBw/YoBKxWjfluG1ImJir5/XraNlV bTbTvX8iHV7V3hN2ceq3HA+tPY+Ifyk4G7P116K6iw2r0Osso0SdZIkiL4RVmcq3sWuwptIcMbu 8WcbhLJcNqKGUS1/za2KXZvWCLcUvA7nvvigSFdQAwEVldP2RltCzX2FJeSK3kcuuERupxIgvIx HMNcPbaSJo1i8Un1n1Kx0oFLMQRWwng50ycSHORGhlTj35HtH5e7bJ43Y0yY17ZeQ7AfD+S46Rf JzfQekpClh+f+wCX0apyloGAeaQ8C+JoyfygrNOZ6VKwLw8SD5q79ym8SkqKN3UPwONhXwZ7O/g Ex0sW9P6tvdV87EL7FgOqxNHfoOJ/7Ut7lfgk7CuAUkw2W3T4MrH90F74OmpekKX8J2kiJaxao5 dUodwRtAl6Yoe6c22/ncaxOa1uTBNYNYVAFjbJURE95v5/UxlW107yVfEZ3dcvXwRs2JmW6/Pp9 1bFDjg+PZzbuQcNPVcDMfAOEmoElIGskaYcvPV/mCZV6rI8JEMg80pTNrP6xN2vJYnfuRmNBSKP aEeJ+qo61aFnC5ImhI84dHgwORrtkJevXrYXUB/VNTML9tN7YW8oEylDQ+p0VrvD7IA1Bcnr0MW BVzogerugSA4JDw== X-Developer-Key: i=jacopo.mondi@ideasonboard.com; a=openpgp; fpr=72392EDC88144A65C701EA9BA5826A2587AD026B X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" From: Daniel Scally Add support to Mali C55 pipeline handler for memory-to-memory camera mode. The Mali-C55, as integrated in the Renesas RZ/V2H(P) SoC, is fed with images saved to memory by the CRU unit and read on behalf of the ISP by the IVC. Update the pipeline handler's match() function to search for the CRU media entity necessary to register a memory input camera. Create and initialize a CameraData instance for the memory camera use case if a valid CRU is found and initialize the IVC components used to feed the ISP with images read from memory. Signed-off-by: Daniel Scally Signed-off-by: Jacopo Mondi --- src/libcamera/pipeline/mali-c55/mali-c55.cpp | 145 +++++++++++++++++++++++---- 1 file changed, 128 insertions(+), 17 deletions(-) diff --git a/src/libcamera/pipeline/mali-c55/mali-c55.cpp b/src/libcamera/pipeline/mali-c55/mali-c55.cpp index 6581d13bbc52..3c8848d28707 100644 --- a/src/libcamera/pipeline/mali-c55/mali-c55.cpp +++ b/src/libcamera/pipeline/mali-c55/mali-c55.cpp @@ -44,6 +44,8 @@ #include "libcamera/internal/v4l2_subdevice.h" #include "libcamera/internal/v4l2_videodevice.h" +#include "rzg2l-cru.h" + namespace { bool isFormatRaw(const libcamera::PixelFormat &pixFmt) @@ -105,7 +107,12 @@ public: std::unique_ptr csi2_; std::unique_ptr sensor_; }; - using CameraType = std::variant; + + struct Memory { + std::unique_ptr cru_; + }; + + using CameraType = std::variant; MaliC55CameraData(PipelineHandler *pipe) : Camera::Private(pipe) @@ -116,6 +123,7 @@ public: int initTpg(MediaEntity *entity); int initInline(MediaEntity *entity); + int initMemory(MediaDevice *cruMedia); std::vector sizes(unsigned int mbusCode) const { @@ -125,6 +133,9 @@ public: }, [&](const Inline &in) -> std::vector { return in.sensor_->sizes(mbusCode); + }, + [&](const Memory &mem) -> std::vector { + return mem.cru_->sizes(); } }, input_); } @@ -137,6 +148,9 @@ public: }, [&](const Inline &in) -> V4L2Subdevice * { return in.sensor_->device(); + }, + [&](const Memory &mem) -> V4L2Subdevice * { + return mem.cru_->sensor()->device(); } }, input_); } @@ -150,6 +164,9 @@ public: }, [&](const Inline &in) -> CameraSensor * { return in.sensor_.get(); + }, + [&](const Memory &mem) -> CameraSensor * { + return mem.cru_->sensor(); } }, input_); } @@ -163,6 +180,9 @@ public: }, [&](const Inline &in) -> V4L2Subdevice * { return in.csi2_.get(); + }, + [&](const Memory &mem) -> V4L2Subdevice * { + return mem.cru_->csi2(); } }, input_); } @@ -175,6 +195,22 @@ public: }, [&](const Inline &in) -> Size { return in.sensor_->resolution(); + }, + [&](const Memory &mem) -> Size { + return mem.cru_->resolution(); + } }, + input_); + } + + RZG2LCRU *cru() const + { + return std::visit(utils::overloaded{ + [&](auto &) -> RZG2LCRU * { + ASSERT(false); + return nullptr; + }, + [&](const Memory &mem) -> RZG2LCRU * { + return mem.cru_.get(); } }, input_); } @@ -258,6 +294,21 @@ int MaliC55CameraData::initInline(MediaEntity *sensor) return 0; } +int MaliC55CameraData::initMemory(MediaDevice *cruMedia) +{ + Memory mem; + + mem.cru_ = std::make_unique(); + + int ret = mem.cru_->init(cruMedia); + if (ret) + return ret; + + input_.emplace(std::move(mem)); + + return 0; +} + std::vector MaliC55CameraData::Tpg::sizes(unsigned int mbusCode) const { V4L2Subdevice::Formats formats = sd_->formats(0); @@ -738,11 +789,15 @@ private: const std::string &name); bool registerTPGCamera(MediaLink *link); bool registerSensorCamera(MediaLink *link); + bool registerMemoryInputCamera(); std::shared_ptr media_; + std::shared_ptr cruMedia_; std::unique_ptr isp_; std::unique_ptr stats_; std::unique_ptr params_; + std::unique_ptr ivcSd_; + std::unique_ptr ivc_; std::vector> statsBuffers_; std::queue availableStatsBuffers_; @@ -1685,6 +1740,34 @@ bool PipelineHandlerMaliC55::registerSensorCamera(MediaLink *ispLink) return true; } +bool PipelineHandlerMaliC55::registerMemoryInputCamera() +{ + std::unique_ptr data = + std::make_unique(this); + + int ret = data->initMemory(cruMedia_.get()); + if (ret) + return false; + + CameraSensor *sensor = data->sensor(); + data->properties_ = sensor->properties(); + + const CameraSensorProperties::SensorDelays &delays = sensor->sensorDelays(); + std::unordered_map params = { + { V4L2_CID_ANALOGUE_GAIN, { delays.gainDelay, false } }, + { V4L2_CID_EXPOSURE, { delays.exposureDelay, false } }, + }; + + data->delayedCtrls_ = + std::make_unique(sensor->device(), params); + isp_->frameStart.connect(data->delayedCtrls_.get(), + &DelayedControls::applyControls); + + ivc_->bufferReady.connect(data->cru(), &RZG2LCRU::cruReturnBuffer); + + return registerMaliCamera(std::move(data), sensor->device()->entity()->name()); +} + bool PipelineHandlerMaliC55::match(DeviceEnumerator *enumerator) { const MediaPad *ispSink; @@ -1694,14 +1777,14 @@ bool PipelineHandlerMaliC55::match(DeviceEnumerator *enumerator) * The TPG and the downscale pipe are both optional blocks and may not * be fitted. */ - DeviceMatch dm("mali-c55"); - dm.add("mali-c55 isp"); - dm.add("mali-c55 resizer fr"); - dm.add("mali-c55 fr"); - dm.add("mali-c55 3a stats"); - dm.add("mali-c55 3a params"); - - media_ = acquireMediaDevice(enumerator, dm); + DeviceMatch c55_dm("mali-c55"); + c55_dm.add("mali-c55 isp"); + c55_dm.add("mali-c55 resizer fr"); + c55_dm.add("mali-c55 fr"); + c55_dm.add("mali-c55 3a stats"); + c55_dm.add("mali-c55 3a params"); + + media_ = acquireMediaDevice(enumerator, c55_dm); if (!media_) return false; @@ -1761,6 +1844,25 @@ bool PipelineHandlerMaliC55::match(DeviceEnumerator *enumerator) stats_->bufferReady.connect(this, &PipelineHandlerMaliC55::statsBufferReady); params_->bufferReady.connect(this, &PipelineHandlerMaliC55::paramsBufferReady); + /* + * We also need to search for the rzg2l-cru CSI-2 receiver. If we find + * that then we need to work in memory input mode instead of the inline + * mode. The absence of this match is not necessarily a failure at this + * point...it depends on the media links that we investigate momentarily. + * + * This is a bit hacky, because there could be multiple of these media + * devices and we're just taking the first. We need modular pipelines to + * properly solve the issue. + */ + static const std::regex cruCsi2Regex("csi-[0-9a-f]{8}.csi2"); + static const std::regex cruIpRegex("cru-ip-[0-9a-f]{8}.cru[0-9]"); + + DeviceMatch cruDm("rzg2l_cru"); + cruDm.add(cruCsi2Regex); + cruDm.add(cruIpRegex); + cruDm.add("CRU output"); + cruMedia_ = acquireMediaDevice(enumerator, cruDm); + ispSink = isp_->entity()->getPadByIndex(0); if (!ispSink || ispSink->links().empty()) { LOG(MaliC55, Error) << "ISP sink pad error"; @@ -1774,13 +1876,6 @@ bool PipelineHandlerMaliC55::match(DeviceEnumerator *enumerator) * MEDIA_ENT_F_CAM_SENSOR - The test pattern generator * MEDIA_ENT_F_VID_IF_BRIDGE - A CSI-2 receiver * MEDIA_ENT_F_PROC_VIDEO_PIXEL_FORMATTER - An input device - * - * The last one will be unsupported for now. The TPG is relatively easy, - * we just register a Camera for it. If we have a CSI-2 receiver we need - * to check its sink pad and register Cameras for anything connected to - * it (probably...there are some complex situations in which that might - * not be true but let's pretend they don't exist until we come across - * them) */ bool registered; for (MediaLink *link : ispSink->links()) { @@ -1800,7 +1895,23 @@ bool PipelineHandlerMaliC55::match(DeviceEnumerator *enumerator) break; case MEDIA_ENT_F_PROC_VIDEO_PIXEL_FORMATTER: - LOG(MaliC55, Warning) << "Memory input not yet supported"; + if (!cruMedia_) + return false; + + ivcSd_ = V4L2Subdevice::fromEntityName(media_.get(), + "rzv2h ivc block"); + if (ivcSd_->open() < 0) + return false; + + ivc_ = V4L2VideoDevice::fromEntityName(media_.get(), + "rzv2h-ivc"); + if (ivc_->open() < 0) + return false; + + registered = registerMemoryInputCamera(); + if (!registered) + return registered; + break; default: LOG(MaliC55, Error) << "Unsupported entity function";