From patchwork Wed Dec 10 14:39:19 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jacopo Mondi X-Patchwork-Id: 25460 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id EB8DEC326B for ; Wed, 10 Dec 2025 14:39:38 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id A78BC61491; Wed, 10 Dec 2025 15:39:37 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="dxFz8J9n"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 1E2E361489 for ; Wed, 10 Dec 2025 15:39:33 +0100 (CET) Received: from [192.168.1.106] (mob-5-90-55-146.net.vodafone.it [5.90.55.146]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id ACE9310C4; Wed, 10 Dec 2025 15:39:31 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1765377571; bh=t25rGKH/UyqsD/vjAyQ7tCb32tcmFGJ1n8U13MRNpkE=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=dxFz8J9ngkzaWHxa73Gyo5BhDaxBQwlNonkW35seVFo+5S1ehuUO0XXgg48QPerzr 5cvQm1u0cTLiTwBoWZw/q76VbahJqtb1zrKSf35YvRn0FPrcF9pdQINVSncWN1sdCa gvGQcHhXCE1QZedPcdcZMWBg26tbrCQmbG5Qxzzc= From: Jacopo Mondi Date: Wed, 10 Dec 2025 15:39:19 +0100 Subject: [PATCH v2 3/7] libcamera: mali-c55: Register memory input camera MIME-Version: 1.0 Message-Id: <20251210-mali-cru-v2-3-e26421de202b@ideasonboard.com> References: <20251210-mali-cru-v2-0-e26421de202b@ideasonboard.com> In-Reply-To: <20251210-mali-cru-v2-0-e26421de202b@ideasonboard.com> To: Daniel Scally , libcamera-devel@lists.libcamera.org Cc: Jacopo Mondi X-Mailer: b4 0.14.2 X-Developer-Signature: v=1; a=openpgp-sha256; l=14098; i=jacopo.mondi@ideasonboard.com; h=from:subject:message-id; bh=I2zhK9XmPmyRoJdqHwcIVL1RAhvX2ns7rhneW2H7+3I=; b=owEBbQKS/ZANAwAKAXI0Bo8WoVY8AcsmYgBpOYYixUPnbOrBTKZzBowmNMYhPP05PnqkSd9gt BScniYwQcyJAjMEAAEKAB0WIQS1xD1IgJogio9YOMByNAaPFqFWPAUCaTmGIgAKCRByNAaPFqFW PHQxD/0bK5yanMhXc/1+25HbB8tuwdH9daC34V5x1C8FuKAEV8G0WTdAWy30l8xpr7lSjTlROjB rTG7oCqzw5WqUZc7SHQAV8UPgUgFn5uZ+8ZGvEBIlPq8xS/i1I+5HOUK8fwBhsKb0OYtF5dvF2F C5foS1yJyeAJ/jPoJqdw1Y9xvbaKbMchgE5FrMzA71S489ds0YW9NdaRqSxnsQCdJ1eSIG9+4Xc 5463xmr1dOppYI+cy1b2rO30DDcwSVCVIUiOeYrKHzqszVKy5mxv1HGBDOdhuUPiRgldiQwzCoE qxl/02Yeqm4EZrCzxOTKrN6EE7/OA4R7u4g8KvgGEaulHZc/+P855+sJ6sU5Z4B+gFGfuXseMuG fJXWcTzkYXPRmdWFWWXZkqPL1KXhH/RUSxmP6U+UU3JBWUHG9ia6D7mCD8aPzTsPD2DriPwZw48 RWnX6EYoBvxveyqaqyQK7d1mWHkjHZS46/NvvZPtw8KjBTt+LIAGYRyKMuVH/G6e0FB6rUJxblT vav6y2kCXYMpMdNBR0knehEGI91jc/5MPK7RjF59Ix0ZV7JhNViexy7ffvNCPd85MIyIi3W0C5i kef5pdQopfYvPRafQiF1lieJ2CP40ZytY6M6+eB26wdzqYQkY1RW0H/BhozEsrHkU7qjJZ+phNp M0PHbXByDjDvreA== X-Developer-Key: i=jacopo.mondi@ideasonboard.com; a=openpgp; fpr=72392EDC88144A65C701EA9BA5826A2587AD026B X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" From: Daniel Scally Add support to Mali C55 pipeline handler for memory-to-memory camera mode. The Mali-C55, as integrated in the Renesas RZ/V2H(P) SoC, is fed with images saved to memory by the CRU unit and read on behalf of the ISP by the IVC. Update the pipeline handler's match() function to search for the CRU media entity necessary to register a memory input camera. Create and initialize a CameraData instance for the memory camera use case if a valid CRU is found and initialize the IVC components used to feed the ISP with images read from memory. The MaliC55CameraData class is now derived by a templated base class which allows, by using the CRTP pattern, to template the argument of the init() function which would have otherwise required to implement placeholders in all derived classes. Signed-off-by: Daniel Scally Signed-off-by: Jacopo Mondi --- src/libcamera/pipeline/mali-c55/mali-c55.cpp | 245 ++++++++++++++++++++++----- 1 file changed, 205 insertions(+), 40 deletions(-) diff --git a/src/libcamera/pipeline/mali-c55/mali-c55.cpp b/src/libcamera/pipeline/mali-c55/mali-c55.cpp index 552a258a6b849a2518fa6c83226cf9ab4e657717..c0c14338ddb3f1d0075df3b3d3d154bc1b5d678a 100644 --- a/src/libcamera/pipeline/mali-c55/mali-c55.cpp +++ b/src/libcamera/pipeline/mali-c55/mali-c55.cpp @@ -42,6 +42,8 @@ #include "libcamera/internal/v4l2_subdevice.h" #include "libcamera/internal/v4l2_videodevice.h" +#include "rzg2l-cru.h" + namespace { bool isFormatRaw(const libcamera::PixelFormat &pixFmt) @@ -95,6 +97,7 @@ public: enum CameraType { Tpg, Inline, + Memory, }; MaliC55CameraData(PipelineHandler *pipe) @@ -107,13 +110,12 @@ public: CameraType type() const { return type_; } /* Deflect these functionalities to either TPG or CameraSensor. */ - virtual int init(MediaEntity *entity) = 0; - virtual std::vector sizes(unsigned int mbusCode) const = 0; virtual V4L2Subdevice *subdev() const = 0; virtual CameraSensor *sensor() const = 0; virtual V4L2Subdevice *csi2() const = 0; virtual Size resolution() const = 0; + virtual RZG2LCRU *cru() const = 0; int pixfmtToMbusCode(const PixelFormat &pixFmt) const; const PixelFormat &bestRawFormat() const; @@ -140,12 +142,34 @@ private: std::string id_; }; -class MaliC55TpgCameraData : public MaliC55CameraData +template +class MaliC55TemplatedCameraData : public MaliC55CameraData +{ +public: + MaliC55TemplatedCameraData(PipelineHandler *pipe) + : MaliC55CameraData(pipe) + { + } + + /* + * The signature of the init() functions is different between the + * Tpg, Inline and Memory camera use cases. Use CRTP to deflect to the + * right implementation the templated argument and avoid polluting the + * interface of the base class. + */ + template + int init(MediaType *media) + { + return static_cast(this)->init(media); + } +}; + +class MaliC55TpgCameraData : public MaliC55TemplatedCameraData { public: MaliC55TpgCameraData(PipelineHandler *pipe); - int init(MediaEntity *entity) override; + int init(MediaEntity *entity); std::vector sizes(unsigned int mbusCode) const override; @@ -171,17 +195,23 @@ public: return nullptr; } + RZG2LCRU *cru() const override + { + ASSERT(false); + return nullptr; + } + private: Size resolution_; std::unique_ptr sd_; }; -class MaliC55InlineCameraData : public MaliC55CameraData +class MaliC55InlineCameraData : public MaliC55TemplatedCameraData { public: MaliC55InlineCameraData(PipelineHandler *pipe); - int init(MediaEntity *entity) override; + int init(MediaEntity *entity); std::vector sizes(unsigned int mbusCode) const override { @@ -208,13 +238,62 @@ public: return csi2_.get(); } + RZG2LCRU *cru() const override + { + ASSERT(false); + return nullptr; + } + private: std::unique_ptr csi2_; std::unique_ptr sensor_; }; +class MaliC55MemoryCameraData + : public MaliC55TemplatedCameraData +{ +public: + MaliC55MemoryCameraData(PipelineHandler *pipe); + + int init(MediaDevice *cruMedia); + + std::vector + sizes([[maybe_unused]] unsigned int mbusCode) const override + { + return cru_->sizes(); + } + + Size resolution() const override + { + return cru_->resolution(); + } + + V4L2Subdevice *subdev() const override + { + return cru_->sensor()->device(); + } + + CameraSensor *sensor() const override + { + return cru_->sensor(); + } + + V4L2Subdevice *csi2() const override + { + return cru_->csi2(); + } + + RZG2LCRU *cru() const override + { + return cru_.get(); + } + +private: + std::unique_ptr cru_; +}; + MaliC55TpgCameraData::MaliC55TpgCameraData(PipelineHandler *pipe) - : MaliC55CameraData(pipe) + : MaliC55TemplatedCameraData(pipe) { type_ = CameraType::Tpg; } @@ -267,7 +346,7 @@ std::vector MaliC55TpgCameraData::sizes(unsigned int mbusCode) const } MaliC55InlineCameraData::MaliC55InlineCameraData(PipelineHandler *pipe) - : MaliC55CameraData(pipe) + : MaliC55TemplatedCameraData(pipe) { type_ = CameraType::Inline; } @@ -292,6 +371,18 @@ int MaliC55InlineCameraData::init(MediaEntity *sensor) return ret; } +MaliC55MemoryCameraData::MaliC55MemoryCameraData(PipelineHandler *pipe) + : MaliC55TemplatedCameraData(pipe) +{ + type_ = CameraType::Memory; +} + +int MaliC55MemoryCameraData::init(MediaDevice *cruMedia) +{ + cru_ = std::make_unique(); + return cru_->init(cruMedia); +} + void MaliC55CameraData::setSensorControls(const ControlList &sensorControls) { delayedCtrls_->push(sensorControls); @@ -752,11 +843,15 @@ private: const std::string &name); bool registerTPGCamera(MediaLink *link); bool registerSensorCamera(MediaLink *link); + bool registerMemoryInputCamera(); std::shared_ptr media_; + std::shared_ptr cruMedia_; std::unique_ptr isp_; std::unique_ptr stats_; std::unique_ptr params_; + std::unique_ptr ivc_; + std::unique_ptr input_; std::vector> statsBuffers_; std::queue availableStatsBuffers_; @@ -1009,6 +1104,8 @@ int PipelineHandlerMaliC55::configure(Camera *camera, ret = csi2Entity->getPadByIndex(1)->links()[0]->setEnabled(true); break; } + case MaliC55CameraData::Memory: + break; } if (ret) return ret; @@ -1035,6 +1132,8 @@ int PipelineHandlerMaliC55::configure(Camera *camera, ret = data->csi2()->getFormat(1, &subdevFormat); + break; + case MaliC55CameraData::Memory: break; } } @@ -1110,6 +1209,7 @@ int PipelineHandlerMaliC55::configure(Camera *camera, pipe->stream = stream; } + /* TPG doesn't support the IPA, so stop here. */ if (!data->ipa_) return 0; @@ -1651,13 +1751,14 @@ bool PipelineHandlerMaliC55::registerTPGCamera(MediaLink *link) return true; } - std::unique_ptr data = - std::make_unique(this); - - int ret = data->init(link->source()->entity()); - if (ret) + MaliC55TpgCameraData *tpgData = new MaliC55TpgCameraData(this); + int ret = tpgData->init(link->source()->entity()); + if (ret) { + delete tpgData; return ret; + } + std::unique_ptr data(tpgData); return registerMaliCamera(std::move(data), name); } @@ -1679,30 +1780,33 @@ bool PipelineHandlerMaliC55::registerSensorCamera(MediaLink *ispLink) if (function != MEDIA_ENT_F_CAM_SENSOR) continue; - std::unique_ptr data = - std::make_unique(this); - - int ret = data->init(sensor); - if (ret) + MaliC55InlineCameraData *inlineData = + new MaliC55InlineCameraData(this); + int ret = inlineData->init(sensor); + if (ret) { + delete inlineData; return ret; + } - data->properties_ = data->sensor()->properties(); + inlineData->properties_ = inlineData->sensor()->properties(); const CameraSensorProperties::SensorDelays &delays = - data->sensor()->sensorDelays(); + inlineData->sensor()->sensorDelays(); std::unordered_map params = { { V4L2_CID_ANALOGUE_GAIN, { delays.gainDelay, false } }, { V4L2_CID_EXPOSURE, { delays.exposureDelay, false } }, }; - V4L2Subdevice *sensorSubdev = data->sensor()->device(); - data->delayedCtrls_ = std::make_unique(sensorSubdev, - params); - isp_->frameStart.connect(data->delayedCtrls_.get(), + V4L2Subdevice *sensorSubdev = inlineData->sensor()->device(); + inlineData->delayedCtrls_ = + std::make_unique(sensorSubdev, + params); + isp_->frameStart.connect(inlineData->delayedCtrls_.get(), &DelayedControls::applyControls); /* \todo Init properties. */ + std::unique_ptr data(inlineData); if (!registerMaliCamera(std::move(data), sensor->name())) return false; } @@ -1710,6 +1814,39 @@ bool PipelineHandlerMaliC55::registerSensorCamera(MediaLink *ispLink) return true; } +bool PipelineHandlerMaliC55::registerMemoryInputCamera() +{ + MaliC55MemoryCameraData *memoryData = new MaliC55MemoryCameraData(this); + + int ret = memoryData->init(cruMedia_.get()); + if (ret) { + delete memoryData; + return false; + } + + CameraSensor *sensor = memoryData->sensor(); + memoryData->properties_ = sensor->properties(); + + const CameraSensorProperties::SensorDelays &delays = sensor->sensorDelays(); + std::unordered_map params = { + { V4L2_CID_ANALOGUE_GAIN, { delays.gainDelay, false } }, + { V4L2_CID_EXPOSURE, { delays.exposureDelay, false } }, + }; + + memoryData->delayedCtrls_ = + std::make_unique(sensor->device(), params); + isp_->frameStart.connect(memoryData->delayedCtrls_.get(), + &DelayedControls::applyControls); + + input_->bufferReady.connect(memoryData->cru(), &RZG2LCRU::cruReturnBuffer); + + std::unique_ptr data(memoryData); + if (!registerMaliCamera(std::move(data), sensor->device()->entity()->name())) + return false; + + return true; +} + bool PipelineHandlerMaliC55::match(DeviceEnumerator *enumerator) { const MediaPad *ispSink; @@ -1719,14 +1856,14 @@ bool PipelineHandlerMaliC55::match(DeviceEnumerator *enumerator) * The TPG and the downscale pipe are both optional blocks and may not * be fitted. */ - DeviceMatch dm("mali-c55"); - dm.add("mali-c55 isp"); - dm.add("mali-c55 resizer fr"); - dm.add("mali-c55 fr"); - dm.add("mali-c55 3a stats"); - dm.add("mali-c55 3a params"); - - media_ = acquireMediaDevice(enumerator, dm); + DeviceMatch c55_dm("mali-c55"); + c55_dm.add("mali-c55 isp"); + c55_dm.add("mali-c55 resizer fr"); + c55_dm.add("mali-c55 fr"); + c55_dm.add("mali-c55 3a stats"); + c55_dm.add("mali-c55 3a params"); + + media_ = acquireMediaDevice(enumerator, c55_dm); if (!media_) return false; @@ -1786,6 +1923,25 @@ bool PipelineHandlerMaliC55::match(DeviceEnumerator *enumerator) stats_->bufferReady.connect(this, &PipelineHandlerMaliC55::statsBufferReady); params_->bufferReady.connect(this, &PipelineHandlerMaliC55::paramsBufferReady); + /* + * We also need to search for the rzg2l-cru CSI-2 receiver. If we find + * that then we need to work in memory input mode instead of the inline + * mode. The absence of this match is not necessarily a failure at this + * point...it depends on the media links that we investigate momentarily. + * + * This is a bit hacky, because there could be multiple of these media + * devices and we're just taking the first. We need modular pipelines to + * properly solve the issue. + */ + static const std::regex cruCsi2Regex("csi-[0-9a-f]{8}.csi2"); + static const std::regex cruIpRegex("cru-ip-[0-9a-f]{8}.cru[0-9]"); + + DeviceMatch cruDm("rzg2l_cru"); + cruDm.add(cruCsi2Regex); + cruDm.add(cruIpRegex); + cruDm.add("CRU output"); + cruMedia_ = acquireMediaDevice(enumerator, cruDm); + ispSink = isp_->entity()->getPadByIndex(0); if (!ispSink || ispSink->links().empty()) { LOG(MaliC55, Error) << "ISP sink pad error"; @@ -1799,13 +1955,6 @@ bool PipelineHandlerMaliC55::match(DeviceEnumerator *enumerator) * MEDIA_ENT_F_CAM_SENSOR - The test pattern generator * MEDIA_ENT_F_VID_IF_BRIDGE - A CSI-2 receiver * MEDIA_ENT_F_PROC_VIDEO_PIXEL_FORMATTER - An input device - * - * The last one will be unsupported for now. The TPG is relatively easy, - * we just register a Camera for it. If we have a CSI-2 receiver we need - * to check its sink pad and register Cameras for anything connected to - * it (probably...there are some complex situations in which that might - * not be true but let's pretend they don't exist until we come across - * them) */ bool registered; for (MediaLink *link : ispSink->links()) { @@ -1825,7 +1974,23 @@ bool PipelineHandlerMaliC55::match(DeviceEnumerator *enumerator) break; case MEDIA_ENT_F_PROC_VIDEO_PIXEL_FORMATTER: - LOG(MaliC55, Warning) << "Memory input not yet supported"; + if (!cruMedia_) + return false; + + ivc_ = V4L2Subdevice::fromEntityName(media_.get(), + "rzv2h ivc block"); + if (ivc_->open() < 0) + return false; + + input_ = V4L2VideoDevice::fromEntityName(media_.get(), + "rzv2h-ivc"); + if (input_->open() < 0) + return false; + + registered = registerMemoryInputCamera(); + if (!registered) + return registered; + break; default: LOG(MaliC55, Error) << "Unsupported entity function";