From patchwork Fri Dec 5 14:52:09 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jacopo Mondi X-Patchwork-Id: 25372 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 95C2CC3260 for ; Fri, 5 Dec 2025 14:52:38 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 39C36613DE; Fri, 5 Dec 2025 15:52:34 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="b5ffPrnJ"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 3166F6115A for ; Fri, 5 Dec 2025 15:52:31 +0100 (CET) Received: from [192.168.1.4] (net-93-65-100-155.cust.vodafonedsl.it [93.65.100.155]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id E42F7EAE; Fri, 5 Dec 2025 15:50:14 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1764946215; bh=SgrgQCL9TrEOmvw9ck0E9mB0BXthdIMrBljUU+foLoU=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=b5ffPrnJy416bqdo+D0sBeCjw/Nnjlx/NiAn4WgVaPvRNw0RwbREZWs/EpHamTjLI sF+2/I/Wk0vCFZUPxdgS0Ycgi9ncvXlXysKxDyEiuwDX4tTrdWPNqeQP6phGhVMaNW oQKwWMujTGOX1hMQHHHasm9uFIUfzm/wCXxRETBg= From: Jacopo Mondi Date: Fri, 05 Dec 2025 15:52:09 +0100 Subject: [PATCH 3/7] libcamera: mali-c55: Register memory input camera MIME-Version: 1.0 Message-Id: <20251205-mali-cru-v1-3-d81bb5ffe73a@ideasonboard.com> References: <20251205-mali-cru-v1-0-d81bb5ffe73a@ideasonboard.com> In-Reply-To: <20251205-mali-cru-v1-0-d81bb5ffe73a@ideasonboard.com> To: Daniel Scally , libcamera-devel@lists.libcamera.org Cc: Jacopo Mondi X-Mailer: b4 0.14.2 X-Developer-Signature: v=1; a=openpgp-sha256; l=13306; i=jacopo.mondi@ideasonboard.com; h=from:subject:message-id; bh=vl0sIQFlsoVvekcoNbHCxxhkB0wA+QaJwace1sYFD5E=; b=owEBbQKS/ZANAwAKAXI0Bo8WoVY8AcsmYgBpMvGsBynQHMqrh0gzwxb0kgjyxyr9sUzTelT7s JUBa0cnU7yJAjMEAAEKAB0WIQS1xD1IgJogio9YOMByNAaPFqFWPAUCaTLxrAAKCRByNAaPFqFW PHWMD/4+04KNA1lcHAnvlwWCjf8BGsptafXiPxhd4NcTrX+r2Se8bARU86vryklPARmffGfNHpR jcTGyX1zKjGBQYmsxTHHVzSYruVTtOTjLYVoUSckFLxlIO37TKxcoocqJCgteW7Fr1zsc36jsxI DivGJx6ejMQEd5SgKxz3Yrxujl1Ke+l3YTVsuQev49Pb8ENPOOaZeOIIPpm15ntmsDCX2URoyXe 1hOO83dYpUtydB8XxhBv46lGCEZryLZKMNPJra+Q4IWO4ZaREvUfUtgUUj18IuD2wJ4qtGboOAK fsa9B/m28uO/gZTwyww+WYtYwMGV0DxkuyMd5K85zks7jev9/XMeDGkGY5Ihl+S2+Wd5f/1MXB0 Whf5UaDcAsDPz1K84jBU6BsKR45moBNUcosjckRf8UMRnd5nz/XqzHloutZKdj8ytyXE7oljiic quB7sEtepfjDtpkuVhUgr09HhsHe7vJ2/hDgMP0B0ggwq2wx5re2+mb/9FCOOv670xhVV7tzCOk 6HgV1VTSXQnG8J2jCwsio8ZytYlJ3vko8KcDIoRSpVqV6dQRoxdSVWM0o0Q83+6vqM1zysACLSi Ic0XlRqkresi9rcnq5OElIQUotqA0oBeaJfxlb1VUU1quHeHCtYdPCKtGc1ZG+X5c44Q+XwioSS 7Lr1zBNQJWhb72w== X-Developer-Key: i=jacopo.mondi@ideasonboard.com; a=openpgp; fpr=72392EDC88144A65C701EA9BA5826A2587AD026B X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" From: Daniel Scally Add support to Mali C55 pipeline handler for memory-to-memory camera mode. The Mali-C55, as integrated in the Renesas RZ/V2H(P) SoC, is fed with images saved to memory by the CRU unit and read on behalf of the ISP by the IVC. Update the pipeline handler's match() function to search for the CRU media entity necessary to register a memory input camera. Create and initialize a CameraData instance for the memory camera use case if a valid CRU is found and initialize the IVC components used to feed the ISP with images read from memory. The CameraData functions that deflect the selection of the subdevice at the origin of the pipeline, the list of supported sizes and the maximum supported resolution have been updated to support memory-to-memory operations. Signed-off-by: Daniel Scally Signed-off-by: Jacopo Mondi --- src/libcamera/pipeline/mali-c55/mali-c55.cpp | 181 ++++++++++++++++++++++----- 1 file changed, 150 insertions(+), 31 deletions(-) diff --git a/src/libcamera/pipeline/mali-c55/mali-c55.cpp b/src/libcamera/pipeline/mali-c55/mali-c55.cpp index 801067ce00fe4e0fd6b81db699fcaed2ebb840b6..6816bf7179ae09434ec10c35ea296725d3ac0700 100644 --- a/src/libcamera/pipeline/mali-c55/mali-c55.cpp +++ b/src/libcamera/pipeline/mali-c55/mali-c55.cpp @@ -42,6 +42,8 @@ #include "libcamera/internal/v4l2_subdevice.h" #include "libcamera/internal/v4l2_videodevice.h" +#include "rzg2l-cru.h" + namespace { bool isFormatRaw(const libcamera::PixelFormat &pixFmt) @@ -95,6 +97,7 @@ public: enum CameraType { TPG, Inline, + Memory, }; MaliC55CameraData(PipelineHandler *pipe) @@ -104,10 +107,11 @@ public: int loadIPA(); - /* Deflect these functionalities to either TPG or CameraSensor. */ + /* Deflect these functionalities to either TPG, inline sensor or CRU. */ std::vector sizes(unsigned int mbusCode) const; Size resolution() const; V4L2Subdevice *subdev() const; + CameraSensor *sensor() const; int pixfmtToMbusCode(const PixelFormat &pixFmt) const; const PixelFormat &bestRawFormat() const; @@ -119,6 +123,7 @@ public: int initTPG(MediaEntity *tpg); int initInlineCamera(MediaEntity *sensor); + void initMemoryCamera(); Stream frStream_; Stream dsStream_; @@ -139,10 +144,15 @@ public: std::unique_ptr sensor_; } inlineInput; + struct { + std::unique_ptr cru_; + } memoryInput; + CameraType input_; private: void setSensorControls(const ControlList &sensorControls); + std::vector tpgSizes(unsigned int mbusCode) const; std::string id_; }; @@ -196,16 +206,18 @@ int MaliC55CameraData::initInlineCamera(MediaEntity *sensor) return 0; } +void MaliC55CameraData::initMemoryCamera() +{ + input_ = Memory; +} + void MaliC55CameraData::setSensorControls(const ControlList &sensorControls) { delayedCtrls_->push(sensorControls); } -std::vector MaliC55CameraData::sizes(unsigned int mbusCode) const +std::vector MaliC55CameraData::tpgSizes(unsigned int mbusCode) const { - if (input_ == Inline) - return inlineInput.sensor_->sizes(mbusCode); - V4L2Subdevice::Formats formats = tpgInput.sd_->formats(0); if (formats.empty()) return {}; @@ -224,6 +236,21 @@ std::vector MaliC55CameraData::sizes(unsigned int mbusCode) const return sizes; } +std::vector MaliC55CameraData::sizes(unsigned int mbusCode) const +{ + switch (input_) { + case TPG: + return tpgSizes(mbusCode); + case Inline: + return inlineInput.sensor_->sizes(mbusCode); + case Memory: + return memoryInput.cru_->sizes(); + } + + assert(false); + return {}; +} + Size MaliC55CameraData::resolution() const { switch (input_) { @@ -231,6 +258,8 @@ Size MaliC55CameraData::resolution() const return tpgInput.resolution_; case Inline: return inlineInput.sensor_->resolution(); + case Memory: + return memoryInput.cru_->resolution(); } assert(false); @@ -244,6 +273,27 @@ V4L2Subdevice *MaliC55CameraData::subdev() const return tpgInput.sd_.get(); case Inline: return inlineInput.sensor_->device(); + case Memory: + return memoryInput.cru_->sensor()->device(); + } + + assert(false); + return nullptr; +} + +CameraSensor *MaliC55CameraData::sensor() const +{ + switch (input_) { + case Inline: + return inlineInput.sensor_.get(); + case Memory: + return memoryInput.cru_->sensor(); + case TPG: + /* + * TPG has no camera sensor, we shall never call this + * function if TPG is in use. Exit the switch and assert(). + */ + break; } assert(false); @@ -332,7 +382,8 @@ void MaliC55CameraData::updateControls(const ControlInfoMap &ipaControls) return; IPACameraSensorInfo sensorInfo; - int ret = inlineInput.sensor_->sensorInfo(&sensorInfo); + CameraSensor *sensor = this->sensor(); + int ret = sensor->sensorInfo(&sensorInfo); if (ret) { LOG(MaliC55, Error) << "Failed to retrieve sensor info"; return; @@ -414,7 +465,7 @@ int MaliC55CameraData::loadIPA() ipa_->setSensorControls.connect(this, &MaliC55CameraData::setSensorControls); - CameraSensor *sensor = inlineInput.sensor_.get(); + CameraSensor *sensor = this->sensor(); std::string ipaTuningFile = ipa_->configurationFile(sensor->model() + ".yaml", "uncalibrated.yaml"); @@ -475,8 +526,8 @@ CameraConfiguration::Status MaliC55CameraConfiguration::validate() combinedTransform_ = Transform::Rot0; } else { Orientation requestedOrientation = orientation; - combinedTransform_ = - data_->inlineInput.sensor_->computeTransform(&orientation); + CameraSensor *sensor = data_->sensor(); + combinedTransform_ = sensor->computeTransform(&orientation); if (orientation != requestedOrientation) status = Adjusted; } @@ -709,11 +760,15 @@ private: const std::string &name); bool registerTPGCamera(MediaLink *link); bool registerSensorCamera(MediaLink *link); + bool registerMemoryInputCamera(); std::shared_ptr media_; + std::shared_ptr cruMedia_; std::unique_ptr isp_; std::unique_ptr stats_; std::unique_ptr params_; + std::unique_ptr ivc_; + std::unique_ptr input_; std::vector> statsBuffers_; std::queue availableStatsBuffers_; @@ -966,6 +1021,8 @@ int PipelineHandlerMaliC55::configure(Camera *camera, ret = csi2Entity->getPadByIndex(1)->links()[0]->setEnabled(true); break; } + case MaliC55CameraData::Memory: + break; } if (ret) return ret; @@ -991,6 +1048,8 @@ int PipelineHandlerMaliC55::configure(Camera *camera, ret = data->inlineInput.csi2_->getFormat(1, &subdevFormat); + break; + case MaliC55CameraData::Memory: break; } if (ret) @@ -1065,6 +1124,7 @@ int PipelineHandlerMaliC55::configure(Camera *camera, pipe->stream = stream; } + /* TPG doesn't support the IPA, so stop here. */ if (!data->ipa_) return 0; @@ -1091,20 +1151,20 @@ int PipelineHandlerMaliC55::configure(Camera *camera, /* We need to inform the IPA of the sensor configuration */ ipa::mali_c55::IPAConfigInfo ipaConfig{}; + CameraSensor *sensor = data->sensor(); - ret = data->inlineInput.sensor_->sensorInfo(&ipaConfig.sensorInfo); + ret = sensor->sensorInfo(&ipaConfig.sensorInfo); if (ret) return ret; - ipaConfig.sensorControls = data->inlineInput.sensor_->controls(); + ipaConfig.sensorControls = sensor->controls(); /* * And we also need to tell the IPA the bayerOrder of the data (as * affected by any flips that we've configured) */ const Transform &combinedTransform = maliConfig->combinedTransform(); - BayerFormat::Order bayerOrder = - data->inlineInput.sensor_->bayerOrder(combinedTransform); + BayerFormat::Order bayerOrder = sensor->bayerOrder(combinedTransform); ControlInfoMap ipaControls; ret = data->ipa_->configure(ipaConfig, utils::to_underlying(bayerOrder), @@ -1323,10 +1383,11 @@ void PipelineHandlerMaliC55::applyScalerCrop(Camera *camera, return; } + CameraSensor *sensor = data->sensor(); Rectangle nativeCrop = *scalerCrop; - IPACameraSensorInfo sensorInfo; - int ret = data->inlineInput.sensor_->sensorInfo(&sensorInfo); + + int ret = sensor->sensorInfo(&sensorInfo); if (ret) { LOG(MaliC55, Error) << "Failed to retrieve sensor info"; return; @@ -1661,6 +1722,39 @@ bool PipelineHandlerMaliC55::registerSensorCamera(MediaLink *ispLink) return true; } +bool PipelineHandlerMaliC55::registerMemoryInputCamera() +{ + std::unique_ptr data = std::make_unique(this); + + data->memoryInput.cru_ = std::make_unique(); + int ret = data->memoryInput.cru_->init(cruMedia_.get()); + if (ret) + return false; + + data->initMemoryCamera(); + + CameraSensor *sensor = data->memoryInput.cru_->sensor(); + data->properties_ = sensor->properties(); + + const CameraSensorProperties::SensorDelays &delays = sensor->sensorDelays(); + std::unordered_map params = { + { V4L2_CID_ANALOGUE_GAIN, { delays.gainDelay, false } }, + { V4L2_CID_EXPOSURE, { delays.exposureDelay, false } }, + }; + + data->delayedCtrls_ = std::make_unique(sensor->device(), params); + isp_->frameStart.connect(data->delayedCtrls_.get(), + &DelayedControls::applyControls); + + input_->bufferReady.connect(data->memoryInput.cru_.get(), + &RZG2LCRU::cruReturnBuffer); + + if (!registerMaliCamera(std::move(data), sensor->device()->entity()->name())) + return false; + + return true; +} + bool PipelineHandlerMaliC55::match(DeviceEnumerator *enumerator) { const MediaPad *ispSink; @@ -1670,14 +1764,14 @@ bool PipelineHandlerMaliC55::match(DeviceEnumerator *enumerator) * The TPG and the downscale pipe are both optional blocks and may not * be fitted. */ - DeviceMatch dm("mali-c55"); - dm.add("mali-c55 isp"); - dm.add("mali-c55 resizer fr"); - dm.add("mali-c55 fr"); - dm.add("mali-c55 3a stats"); - dm.add("mali-c55 3a params"); - - media_ = acquireMediaDevice(enumerator, dm); + DeviceMatch c55_dm("mali-c55"); + c55_dm.add("mali-c55 isp"); + c55_dm.add("mali-c55 resizer fr"); + c55_dm.add("mali-c55 fr"); + c55_dm.add("mali-c55 3a stats"); + c55_dm.add("mali-c55 3a params"); + + media_ = acquireMediaDevice(enumerator, c55_dm); if (!media_) return false; @@ -1737,6 +1831,22 @@ bool PipelineHandlerMaliC55::match(DeviceEnumerator *enumerator) stats_->bufferReady.connect(this, &PipelineHandlerMaliC55::statsBufferReady); params_->bufferReady.connect(this, &PipelineHandlerMaliC55::paramsBufferReady); + /* + * We also need to search for the rzg2l-cru CSI-2 receiver. If we find + * that then we need to work in memory input mode instead of the inline + * mode. The absence of this match is not necessarily a failure at this + * point...it depends on the media links that we investigate momentarily. + * + * This is a bit hacky, because there could be multiple of these media + * devices and we're just taking the first. We need modular pipelines to + * properly solve the issue. + */ + DeviceMatch cru_dm("rzg2l_cru"); + cru_dm.add(std::regex("csi-[0-9a-f]{8}.csi2")); + cru_dm.add(std::regex("cru-ip-[0-9a-f]{8}.cru[0-9]")); + cru_dm.add("CRU output"); + cruMedia_ = acquireMediaDevice(enumerator, cru_dm); + ispSink = isp_->entity()->getPadByIndex(0); if (!ispSink || ispSink->links().empty()) { LOG(MaliC55, Error) << "ISP sink pad error"; @@ -1750,13 +1860,6 @@ bool PipelineHandlerMaliC55::match(DeviceEnumerator *enumerator) * MEDIA_ENT_F_CAM_SENSOR - The test pattern generator * MEDIA_ENT_F_VID_IF_BRIDGE - A CSI-2 receiver * MEDIA_ENT_F_PROC_VIDEO_PIXEL_FORMATTER - An input device - * - * The last one will be unsupported for now. The TPG is relatively easy, - * we just register a Camera for it. If we have a CSI-2 receiver we need - * to check its sink pad and register Cameras for anything connected to - * it (probably...there are some complex situations in which that might - * not be true but let's pretend they don't exist until we come across - * them) */ bool registered; for (MediaLink *link : ispSink->links()) { @@ -1776,7 +1879,23 @@ bool PipelineHandlerMaliC55::match(DeviceEnumerator *enumerator) break; case MEDIA_ENT_F_PROC_VIDEO_PIXEL_FORMATTER: - LOG(MaliC55, Warning) << "Memory input not yet supported"; + if (!cruMedia_) + return false; + + ivc_ = V4L2Subdevice::fromEntityName(media_.get(), + "rzv2h ivc block"); + if (ivc_->open() < 0) + return false; + + input_ = V4L2VideoDevice::fromEntityName(media_.get(), + "rzv2h-ivc"); + if (input_->open() < 0) + return false; + + registered = registerMemoryInputCamera(); + if (!registered) + return registered; + break; default: LOG(MaliC55, Error) << "Unsupported entity function";