From patchwork Fri Dec 5 14:52:07 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jacopo Mondi X-Patchwork-Id: 25371 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 1B559BD80A for ; Fri, 5 Dec 2025 14:52:37 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 58BB4613E5; Fri, 5 Dec 2025 15:52:33 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="dGH2J4M7"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id EF08860D3C for ; Fri, 5 Dec 2025 15:52:30 +0100 (CET) Received: from [192.168.1.4] (net-93-65-100-155.cust.vodafonedsl.it [93.65.100.155]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 6134CE45; Fri, 5 Dec 2025 15:50:14 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1764946214; bh=2R6YTG/PmgNLESxZ5ZtioPem3otmk7MKdIu9eWdVb50=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=dGH2J4M7eLx7FzimoMaFi7hEU5H6y1DKdsTc2mSpcJ/sXta/tSG3Fduv7YdVC9QhY c0eklWBB6FJAuZ3ZgWPvaZ7QLOyt/vjZWHPyU+SBIYMB/RprjL3A9WORoLmqQbJfkw vILMDhQL05K9i8yW3NSteFGmo5VjfWQBjjg5hdds= From: Jacopo Mondi Date: Fri, 05 Dec 2025 15:52:07 +0100 Subject: [PATCH 1/7] libcamera: mali-c55: Add RZG2LCRU class MIME-Version: 1.0 Message-Id: <20251205-mali-cru-v1-1-d81bb5ffe73a@ideasonboard.com> References: <20251205-mali-cru-v1-0-d81bb5ffe73a@ideasonboard.com> In-Reply-To: <20251205-mali-cru-v1-0-d81bb5ffe73a@ideasonboard.com> To: Daniel Scally , libcamera-devel@lists.libcamera.org Cc: Jacopo Mondi X-Mailer: b4 0.14.2 X-Developer-Signature: v=1; a=openpgp-sha256; l=9898; i=jacopo.mondi@ideasonboard.com; h=from:subject:message-id; bh=nnBQQgOXmNBrvRiQgnI8kmH0kGY971LQmxXd+p2wgBQ=; b=owEBbQKS/ZANAwAKAXI0Bo8WoVY8AcsmYgBpMvGsd5UoqrZ4FJS688mf5K2IcY4DEskvLdQd7 vCtpRG9pOiJAjMEAAEKAB0WIQS1xD1IgJogio9YOMByNAaPFqFWPAUCaTLxrAAKCRByNAaPFqFW PEMxD/wJ7GcJ+rw91CHW5H0hPw2PIKJgnv4hY40BsUJ/rMzVtTgMO0GLxsnl3v81y6uBrTz3LK7 8zxe6DC+xhsNLqrh7OemvvA/Om0NauMR1DRPGB+Xjy8OW1kuszfQPMd7+U5befRLziUbY1G4g4E xyAvlUMWn4W/72TWawRy67B+/pGTQeK36pWjZQvfM+r30O/KPXLftNPRWBleDI/tpRyDde10Fko HrmeiDJp6EO7LBpYt9XWvWNBvRmUkeTYdDrb75rtyf6aqCqtFKpaUm58Rb1Pqv5ZCLMYH/AwDlm Tw7dv1NtEA38e/VFOZADtIEBgj2jaiNOwwMUN7yKPXN37xA8UcalDkRboy6zWx8GGWZhUNLBmZk 11/AH4EXM1FEeutYMraXVP9xo8COUUzN7crNAigiU3lVGVhTlXqP1+8H+amvpkRDvOc7lGGNjeZ 5ctum9WLxFz6/cXhwmcTCqRAS2PRMyBDQP5wwquKCKjxWKOb18uZTP51k6BibVJ08bCxTRUvQEv TLqJEbEfHD6vn0UqHet/U8Pl2Ic+PmlGWqBuz08krFlpkgLMngxRb7uDIUw5ABIWJWvHfdkML3k PrQ1/nk0aFvGiMcUxgrjXegbF7MwOkgm8J2g8cl02d47s2s7DVN4A2PYp46hxKoRz57QSaGykS2 yD29rMA1ozOINPA== X-Developer-Key: i=jacopo.mondi@ideasonboard.com; a=openpgp; fpr=72392EDC88144A65C701EA9BA5826A2587AD026B X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" From: Daniel Scally Add a class allowing us to control the RZ/G2L Camera Receiver Unit. The class is named after the media device it handles, which is called "rzg2l_cru" but the CRU is actually found in other SoCs than the RZ/G2L one, such as the RZ/V2H(P). The RZG2LCRU class models the CSI-2 receiver found on those SoCs and allows to add support for memory-to-memory operations on the RZ/V2H(P) SoC which integrates a Mali-C55 ISP and a specialized DMA engine named IVC that feeds images from memory, where the CRU has saved them. Signed-off-by: Daniel Scally Signed-off-by: Jacopo Mondi --- src/libcamera/pipeline/mali-c55/meson.build | 3 +- src/libcamera/pipeline/mali-c55/rzg2l-cru.cpp | 261 ++++++++++++++++++++++++++ src/libcamera/pipeline/mali-c55/rzg2l-cru.h | 72 +++++++ 3 files changed, 335 insertions(+), 1 deletion(-) diff --git a/src/libcamera/pipeline/mali-c55/meson.build b/src/libcamera/pipeline/mali-c55/meson.build index eba8e5a3905469297ade6ed5bf523473448cb9de..4e768242602e563612dca7ccbd0d11f5e5ab92a5 100644 --- a/src/libcamera/pipeline/mali-c55/meson.build +++ b/src/libcamera/pipeline/mali-c55/meson.build @@ -1,5 +1,6 @@ # SPDX-License-Identifier: CC0-1.0 libcamera_internal_sources += files([ - 'mali-c55.cpp' + 'mali-c55.cpp', + 'rzg2l-cru.cpp', ]) diff --git a/src/libcamera/pipeline/mali-c55/rzg2l-cru.cpp b/src/libcamera/pipeline/mali-c55/rzg2l-cru.cpp new file mode 100644 index 0000000000000000000000000000000000000000..ac540ba2e29d25191e87a6208d33094e4ef2b969 --- /dev/null +++ b/src/libcamera/pipeline/mali-c55/rzg2l-cru.cpp @@ -0,0 +1,261 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2025, Ideas on Board Oy + * + * Pipine handler element for the Renesas RZ/G2L Camera Receiver Unit + */ + +#include "rzg2l-cru.h" + +#include +#include + +#include + +#include +#include + +#include +#include + +#include "libcamera/internal/bayer_format.h" +#include "libcamera/internal/camera_sensor.h" +#include "libcamera/internal/framebuffer.h" +#include "libcamera/internal/media_device.h" +#include "libcamera/internal/request.h" + +namespace libcamera { + +static const std::map bitDepthToFmt{ + { 10, V4L2PixelFormat(V4L2_PIX_FMT_RAW_CRU10) }, + { 12, V4L2PixelFormat(V4L2_PIX_FMT_RAW_CRU12) }, + { 14, V4L2PixelFormat(V4L2_PIX_FMT_RAW_CRU14) }, +}; + +LOG_DEFINE_CATEGORY(RZG2LCRU) + +FrameBuffer *RZG2LCRU::queueBuffer(Request *request) +{ + FrameBuffer *buffer; + + if (availableBuffers_.empty()) { + LOG(RZG2LCRU, Debug) << "CRU buffer underrun"; + return nullptr; + } + + buffer = availableBuffers_.front(); + + int ret = output_->queueBuffer(buffer); + if (ret) { + LOG(RZG2LCRU, Error) << "Failed to queue buffer to CRU"; + return nullptr; + } + + availableBuffers_.pop(); + buffer->_d()->setRequest(request); + + return buffer; +} + +void RZG2LCRU::cruReturnBuffer(FrameBuffer *buffer) +{ + for (const std::unique_ptr &buf : buffers_) { + if (buf.get() == buffer) { + availableBuffers_.push(buffer); + break; + } + } +} + +int RZG2LCRU::start() +{ + int ret = output_->exportBuffers(kBufferCount, &buffers_); + if (ret < 0) + return ret; + + ret = output_->importBuffers(kBufferCount); + if (ret) + return ret; + + for (std::unique_ptr &buffer : buffers_) + availableBuffers_.push(buffer.get()); + + ret = output_->streamOn(); + if (ret) { + freeBuffers(); + return ret; + } + + return 0; +} + +int RZG2LCRU::stop() +{ + int ret; + + csi2_->setFrameStartEnabled(false); + + ret = output_->streamOff(); + + freeBuffers(); + + return ret; +} + +void RZG2LCRU::freeBuffers() +{ + availableBuffers_ = {}; + buffers_.clear(); + + if (output_->releaseBuffers()) + LOG(RZG2LCRU, Error) << "Failed to release CRU buffers"; +} + +int RZG2LCRU::configure(V4L2SubdeviceFormat *subdevFormat, V4L2DeviceFormat *inputFormat) +{ + /* + * Set format on the sensor and propagate it up to the CRU video + * device. + */ + + int ret = sensor_->setFormat(subdevFormat); + if (ret) + return ret; + + ret = csi2_->setFormat(0, subdevFormat); + if (ret) + return ret; + + ret = csi2_->getFormat(1, subdevFormat); + if (ret) + return ret; + + ret = cru_->setFormat(0, subdevFormat); + if (ret) + return ret; + + ret = cru_->getFormat(1, subdevFormat); + if (ret) + return ret; + + /* + * The capture device needs to be set with a format that can be produced + * from the mbus code of the subdevFormat. The CRU and IVC use bayer + * order agnostic pixel formats, so all we need to do is find the right + * bitdepth and select the appropriate format. + */ + BayerFormat bayerFormat = BayerFormat::fromMbusCode(subdevFormat->code); + if (!bayerFormat.isValid()) + return -EINVAL; + + V4L2DeviceFormat captureFormat; + captureFormat.fourcc = bitDepthToFmt.at(bayerFormat.bitDepth); + captureFormat.size = subdevFormat->size; + + ret = output_->setFormat(&captureFormat); + if (ret) + return ret; + + /* + * We return the format that we set against the output device, as the + * same format will also need to be set against the Input Video Control + * Block device. + */ + *inputFormat = captureFormat; + + return 0; +} + +void RZG2LCRU::initCRUSizes() +{ + Size maxCSI2Size; + + /* + * Get the maximum supported size on the CSI-2 receiver. We need to + * query the kernel interface as the size limits differ from RZ/G2L + * (2800x4095) and RZ/V2H (4096x4096). + */ + V4L2Subdevice::Formats csi2Formats = csi2_->formats(0); + if (csi2Formats.empty()) + return; + + for (const auto &format : csi2Formats) { + for (const auto &range : format.second) { + if (range.max > maxCSI2Size) + maxCSI2Size = range.max; + } + } + + /* + * Enumerate the sensor supported resolutiond and filter out the ones + * largest than the maximum supported CSI-2 receiver input size. + */ + V4L2Subdevice::Formats formats = sensor_->device()->formats(0); + if (formats.empty()) + return; + + for (const auto &format : formats) { + for (const auto &range : format.second) { + const Size &max = range.max; + + if (max.width > maxCSI2Size.width || + max.height > maxCSI2Size.height) + continue; + + csi2Sizes_.push_back(max); + } + } + + /* Sort in increasing order and remove duplicates. */ + std::sort(csi2Sizes_.begin(), csi2Sizes_.end()); + auto last = std::unique(csi2Sizes_.begin(), csi2Sizes_.end()); + csi2Sizes_.erase(last, csi2Sizes_.end()); + + csi2Resolution_ = csi2Sizes_.back(); +} + +int RZG2LCRU::init(const MediaDevice *media) +{ + int ret; + + csi2_ = V4L2Subdevice::fromEntityName(media, + std::regex("csi-[0-9a-f]{8}.csi2")); + if (!csi2_) + return -ENODEV; + + ret = csi2_->open(); + if (ret) + return ret; + + const std::vector &pads = csi2_->entity()->pads(); + if (pads.empty()) + return -ENODEV; + + /* The receiver has a single sink pad at index 0 */ + MediaPad *sink = pads[0]; + const std::vector &links = sink->links(); + if (links.empty()) + return -ENODEV; + + MediaLink *link = links[0]; + sensor_ = CameraSensorFactoryBase::create(link->source()->entity()); + if (!sensor_) + return -ENODEV; + + cru_ = V4L2Subdevice::fromEntityName(media, + std::regex("cru-ip-[0-9a-f]{8}.cru[0-9]")); + ret = cru_->open(); + if (ret) + return ret; + + output_ = V4L2VideoDevice::fromEntityName(media, "CRU output"); + ret = output_->open(); + if (ret) + return ret; + + initCRUSizes(); + + return 0; +} + +} /* namespace libcamera */ diff --git a/src/libcamera/pipeline/mali-c55/rzg2l-cru.h b/src/libcamera/pipeline/mali-c55/rzg2l-cru.h new file mode 100644 index 0000000000000000000000000000000000000000..9944e71cda82a494182619f01766f32f57a8b516 --- /dev/null +++ b/src/libcamera/pipeline/mali-c55/rzg2l-cru.h @@ -0,0 +1,72 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2025, Ideas on Board Oy + * + * Pipine handler element for the Renesas RZ/G2L Camera Receiver Unit + */ + +#pragma once + +#include +#include +#include + +#include + +#include "libcamera/internal/v4l2_subdevice.h" +#include "libcamera/internal/v4l2_videodevice.h" + +namespace libcamera { + +class CameraSensor; +class FrameBuffer; +class MediaDevice; +class Request; +class Size; + +class RZG2LCRU +{ +public: + static constexpr unsigned int kBufferCount = 4; + + RZG2LCRU() = default; + + const std::vector &sizes() const + { + return csi2Sizes_; + } + + int init(const MediaDevice *media); + const Size &resolution() const + { + return csi2Resolution_; + } + + CameraSensor *sensor() const { return sensor_.get(); } + + int configure(V4L2SubdeviceFormat *subdevFormat, V4L2DeviceFormat *inputFormat); + FrameBuffer *queueBuffer(Request *request); + void cruReturnBuffer(FrameBuffer *buffer); + V4L2VideoDevice *output() { return output_.get(); } + int start(); + int stop(); +private: + void freeBuffers(); + + void cruBufferReady(FrameBuffer *buffer); + + void initCRUSizes(); + + std::unique_ptr sensor_; + std::unique_ptr csi2_; + std::unique_ptr cru_; + std::unique_ptr output_; + + std::vector> buffers_; + std::queue availableBuffers_; + + std::vector csi2Sizes_; + Size csi2Resolution_; +}; + +} /* namespace libcamera */ From patchwork Fri Dec 5 14:52:08 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jacopo Mondi X-Patchwork-Id: 25373 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id B20B8BD80A for ; Fri, 5 Dec 2025 14:52:39 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id BD467613DA; Fri, 5 Dec 2025 15:52:34 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="k/m5f48R"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 3D9EB611BA for ; Fri, 5 Dec 2025 15:52:31 +0100 (CET) Received: from [192.168.1.4] (net-93-65-100-155.cust.vodafonedsl.it [93.65.100.155]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 9CE9AE7C; Fri, 5 Dec 2025 15:50:14 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1764946214; bh=/1F1iRZXbLowSjqdNb+I6FehS3zf86nbjM1hxdB7mRM=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=k/m5f48RIAGaCFAihc3D4SbCRwcyCSXaKr9mJTwYzz0J+nv2tt7fgE5GAPVw/mLfD wxJwyxh80wPIOa0nmIn5GotygEeb2Rm1NOT2PpwmKG81NcK2DtcZ84VuJX9iRk7fr2 9yNOwjyuvPPBTrwrjmfRq0dv+VKGq6ANdPt8o50c= From: Jacopo Mondi Date: Fri, 05 Dec 2025 15:52:08 +0100 Subject: [PATCH 2/7] libcamera: mali-c55: Split TPG and Inline camera handling MIME-Version: 1.0 Message-Id: <20251205-mali-cru-v1-2-d81bb5ffe73a@ideasonboard.com> References: <20251205-mali-cru-v1-0-d81bb5ffe73a@ideasonboard.com> In-Reply-To: <20251205-mali-cru-v1-0-d81bb5ffe73a@ideasonboard.com> To: Daniel Scally , libcamera-devel@lists.libcamera.org Cc: Jacopo Mondi X-Mailer: b4 0.14.2 X-Developer-Signature: v=1; a=openpgp-sha256; l=15652; i=jacopo.mondi@ideasonboard.com; h=from:subject:message-id; bh=/1F1iRZXbLowSjqdNb+I6FehS3zf86nbjM1hxdB7mRM=; b=owEBbQKS/ZANAwAKAXI0Bo8WoVY8AcsmYgBpMvGsvazwL5+i78/vL5P+m8gDxyGW+XQcELwRQ Eu6yyIgDzuJAjMEAAEKAB0WIQS1xD1IgJogio9YOMByNAaPFqFWPAUCaTLxrAAKCRByNAaPFqFW POwtD/9wVRlRMrBS4vBukOWDvgRVVgB/kdL+b+7pmkYDWsYVsN4naSV7aXoXk8FUN/ML+Z3FDME 00iG7FEcPyGruH5Jc7CqiulVG3I7FIbTN99DAzDsJeM/KeI7YjzthH4QjgeEheImLn0XyC4IcYJ jlmdMfvfy/vmN/wYaD9+jJUK4iZWlj01CQnENyEB0YTflLEuQb47rVZye/WfAq96fPC2jq7APfq zysvm+qSdG+MpDynAX3YYAF/7TnJd5sD+mr7rxAliDzn6XPE9KmoUIsTlxmChPn6aJ3XRHKFJqj bnXdqcyUdCCPPBjvfCwe5gbEiuKvWEvSe/cKC4JYI+vDfWrQHOVTRUkLnCmXCQW+CeMTiLYFMiV hLEAKGrYFMeIB8dHvRrANBrBMEgRlBzzJhOrVVQnxgYVLOJKA8/jG4GZzkssThyW2mUYAa8DN+o 6bDfWGLdqn7YhdLiiPjd3l7nSyAXBo/w9weAmWWJgWl58tEIJhX8HG2TtKvMNT9T1F97seaO8rK JbbgMwBIyYxrcBDk8bhbcbsguZ+aAY7GPrvMdqBJDbgTN091v2c+f68Jwgb6SY6gNOn/C22LEyo lT0g+h2SRMKQPgGyxZHfhXYKZ63f7WJ4I6VKImctbetiL5DOZ80Gnj3XQTaaxGhAZM5UZYY0ROg iGGSjS4HhoQSXxw== X-Developer-Key: i=jacopo.mondi@ideasonboard.com; a=openpgp; fpr=72392EDC88144A65C701EA9BA5826A2587AD026B X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" In order to prepare to support memory input cameras, split the handling of the TPG and Inline camera cases. The Mali C55 pipeline handler uses the entity and subdevice stored in the CameraData to support both the handling of the TPG and of the [CSI-2 + sensor] use cases. Adding support for memory cameras by using the CRU unit would add yet-another special case making the code harder to follow and more prone to errors. Split the handling of the TPG and inline cameras by introducing an enumeration for the camera operation modes, separating data into anonymous structures and handling the configuration and linking of the media graphs separately. Signed-off-by: Jacopo Mondi --- src/libcamera/pipeline/mali-c55/mali-c55.cpp | 236 +++++++++++++++------------ 1 file changed, 135 insertions(+), 101 deletions(-) diff --git a/src/libcamera/pipeline/mali-c55/mali-c55.cpp b/src/libcamera/pipeline/mali-c55/mali-c55.cpp index cf0cb15f8bb39143eea38aa8acb8d2b1268f5530..801067ce00fe4e0fd6b81db699fcaed2ebb840b6 100644 --- a/src/libcamera/pipeline/mali-c55/mali-c55.cpp +++ b/src/libcamera/pipeline/mali-c55/mali-c55.cpp @@ -92,17 +92,22 @@ struct MaliC55FrameInfo { class MaliC55CameraData : public Camera::Private { public: - MaliC55CameraData(PipelineHandler *pipe, MediaEntity *entity) - : Camera::Private(pipe), entity_(entity) + enum CameraType { + TPG, + Inline, + }; + + MaliC55CameraData(PipelineHandler *pipe) + : Camera::Private(pipe) { } - int init(); int loadIPA(); /* Deflect these functionalities to either TPG or CameraSensor. */ std::vector sizes(unsigned int mbusCode) const; Size resolution() const; + V4L2Subdevice *subdev() const; int pixfmtToMbusCode(const PixelFormat &pixFmt) const; const PixelFormat &bestRawFormat() const; @@ -112,11 +117,9 @@ public: PixelFormat adjustRawFormat(const PixelFormat &pixFmt) const; Size adjustRawSizes(const PixelFormat &pixFmt, const Size &rawSize) const; - std::unique_ptr sensor_; + int initTPG(MediaEntity *tpg); + int initInlineCamera(MediaEntity *sensor); - MediaEntity *entity_; - std::unique_ptr csi_; - std::unique_ptr sd_; Stream frStream_; Stream dsStream_; @@ -126,68 +129,71 @@ public: std::unique_ptr delayedCtrls_; + struct { + Size resolution_; + std::unique_ptr sd_; + } tpgInput; + + struct { + std::unique_ptr csi2_; + std::unique_ptr sensor_; + } inlineInput; + + CameraType input_; + private: - void initTPGData(); void setSensorControls(const ControlList &sensorControls); - std::string id_; - Size tpgResolution_; }; -int MaliC55CameraData::init() +int MaliC55CameraData::initTPG(MediaEntity *tpg) { - int ret; - - sd_ = std::make_unique(entity_); - ret = sd_->open(); + tpgInput.sd_ = std::make_unique(tpg); + int ret = tpgInput.sd_->open(); if (ret) { - LOG(MaliC55, Error) << "Failed to open sensor subdevice"; + LOG(MaliC55, Error) << "Failed to open TPG subdevice"; return ret; } - /* If this camera is created from TPG, we return here. */ - if (entity_->name() == "mali-c55 tpg") { - initTPGData(); - return 0; - } - - /* - * Register a CameraSensor if we connect to a sensor and create - * an entity for the connected CSI-2 receiver. - */ - sensor_ = CameraSensorFactoryBase::create(entity_); - if (!sensor_) - return -ENODEV; + /* Replicate the CameraSensor implementation for TPG. */ + V4L2Subdevice::Formats formats = tpgInput.sd_->formats(0); + if (formats.empty()) + return -EINVAL; - const MediaPad *sourcePad = entity_->getPadByIndex(0); - MediaEntity *csiEntity = sourcePad->links()[0]->sink()->entity(); + std::vector tpgSizes; - csi_ = std::make_unique(csiEntity); - ret = csi_->open(); - if (ret) { - LOG(MaliC55, Error) << "Failed to open CSI-2 subdevice"; - return ret; + for (const auto &format : formats) { + const std::vector &ranges = format.second; + std::transform(ranges.begin(), ranges.end(), std::back_inserter(tpgSizes), + [](const SizeRange &range) { return range.max; }); } + tpgInput.resolution_ = tpgSizes.back(); + input_ = TPG; + return 0; } -void MaliC55CameraData::initTPGData() +int MaliC55CameraData::initInlineCamera(MediaEntity *sensor) { - /* Replicate the CameraSensor implementation for TPG. */ - V4L2Subdevice::Formats formats = sd_->formats(0); - if (formats.empty()) - return; + /* Register a CameraSensor create an entity for the CSI-2 receiver. */ + inlineInput.sensor_ = CameraSensorFactoryBase::create(sensor); + if (!inlineInput.sensor_) + return -ENODEV; - std::vector tpgSizes; + const MediaPad *sourcePad = sensor->getPadByIndex(0); + MediaEntity *csiEntity = sourcePad->links()[0]->sink()->entity(); - for (const auto &format : formats) { - const std::vector &ranges = format.second; - std::transform(ranges.begin(), ranges.end(), std::back_inserter(tpgSizes), - [](const SizeRange &range) { return range.max; }); + inlineInput.csi2_ = std::make_unique(csiEntity); + int ret = inlineInput.csi2_->open(); + if (ret) { + LOG(MaliC55, Error) << "Failed to open CSI-2 subdevice"; + return ret; } - tpgResolution_ = tpgSizes.back(); + input_ = Inline; + + return 0; } void MaliC55CameraData::setSensorControls(const ControlList &sensorControls) @@ -197,10 +203,10 @@ void MaliC55CameraData::setSensorControls(const ControlList &sensorControls) std::vector MaliC55CameraData::sizes(unsigned int mbusCode) const { - if (sensor_) - return sensor_->sizes(mbusCode); + if (input_ == Inline) + return inlineInput.sensor_->sizes(mbusCode); - V4L2Subdevice::Formats formats = sd_->formats(0); + V4L2Subdevice::Formats formats = tpgInput.sd_->formats(0); if (formats.empty()) return {}; @@ -220,10 +226,28 @@ std::vector MaliC55CameraData::sizes(unsigned int mbusCode) const Size MaliC55CameraData::resolution() const { - if (sensor_) - return sensor_->resolution(); + switch (input_) { + case TPG: + return tpgInput.resolution_; + case Inline: + return inlineInput.sensor_->resolution(); + } + + assert(false); + return {}; +} + +V4L2Subdevice *MaliC55CameraData::subdev() const +{ + switch (input_) { + case TPG: + return tpgInput.sd_.get(); + case Inline: + return inlineInput.sensor_->device(); + } - return tpgResolution_; + assert(false); + return nullptr; } /* @@ -242,7 +266,8 @@ int MaliC55CameraData::pixfmtToMbusCode(const PixelFormat &pixFmt) const if (!bayerFormat.isValid()) return -EINVAL; - V4L2Subdevice::Formats formats = sd_->formats(0); + V4L2Subdevice *sd = subdev(); + V4L2Subdevice::Formats formats = sd->formats(0); unsigned int sensorMbusCode = 0; unsigned int bitDepth = 0; @@ -280,7 +305,8 @@ const PixelFormat &MaliC55CameraData::bestRawFormat() const { static const PixelFormat invalidPixFmt = {}; - for (const auto &fmt : sd_->formats(0)) { + V4L2Subdevice *sd = subdev(); + for (const auto &fmt : sd->formats(0)) { BayerFormat sensorBayer = BayerFormat::fromMbusCode(fmt.first); if (!sensorBayer.isValid()) @@ -302,11 +328,11 @@ const PixelFormat &MaliC55CameraData::bestRawFormat() const void MaliC55CameraData::updateControls(const ControlInfoMap &ipaControls) { - if (!sensor_) + if (input_ == TPG) return; IPACameraSensorInfo sensorInfo; - int ret = sensor_->sensorInfo(&sensorInfo); + int ret = inlineInput.sensor_->sensorInfo(&sensorInfo); if (ret) { LOG(MaliC55, Error) << "Failed to retrieve sensor info"; return; @@ -379,7 +405,7 @@ int MaliC55CameraData::loadIPA() int ret; /* Do not initialize IPA for TPG. */ - if (!sensor_) + if (input_ == TPG) return 0; ipa_ = IPAManager::createIPA(pipe(), 1, 1); @@ -388,20 +414,21 @@ int MaliC55CameraData::loadIPA() ipa_->setSensorControls.connect(this, &MaliC55CameraData::setSensorControls); - std::string ipaTuningFile = ipa_->configurationFile(sensor_->model() + ".yaml", + CameraSensor *sensor = inlineInput.sensor_.get(); + std::string ipaTuningFile = ipa_->configurationFile(sensor->model() + ".yaml", "uncalibrated.yaml"); /* We need to inform the IPA of the sensor configuration */ ipa::mali_c55::IPAConfigInfo ipaConfig{}; - ret = sensor_->sensorInfo(&ipaConfig.sensorInfo); + ret = sensor->sensorInfo(&ipaConfig.sensorInfo); if (ret) return ret; - ipaConfig.sensorControls = sensor_->controls(); + ipaConfig.sensorControls = sensor->controls(); ControlInfoMap ipaControls; - ret = ipa_->init({ ipaTuningFile, sensor_->model() }, ipaConfig, + ret = ipa_->init({ ipaTuningFile, sensor->model() }, ipaConfig, &ipaControls); if (ret) { LOG(MaliC55, Error) << "Failed to initialise the Mali-C55 IPA"; @@ -444,13 +471,14 @@ CameraConfiguration::Status MaliC55CameraConfiguration::validate() * The TPG doesn't support flips, so we only need to calculate a * transform if we have a sensor. */ - if (data_->sensor_) { + if (data_->input_ == MaliC55CameraData::TPG) { + combinedTransform_ = Transform::Rot0; + } else { Orientation requestedOrientation = orientation; - combinedTransform_ = data_->sensor_->computeTransform(&orientation); + combinedTransform_ = + data_->inlineInput.sensor_->computeTransform(&orientation); if (orientation != requestedOrientation) status = Adjusted; - } else { - combinedTransform_ = Transform::Rot0; } /* Only 2 streams available. */ @@ -927,11 +955,17 @@ int PipelineHandlerMaliC55::configure(Camera *camera, /* Link the graph depending if we are operating the TPG or a sensor. */ MaliC55CameraData *data = cameraData(camera); - if (data->csi_) { - const MediaEntity *csiEntity = data->csi_->entity(); - ret = csiEntity->getPadByIndex(1)->links()[0]->setEnabled(true); - } else { - ret = data->entity_->getPadByIndex(0)->links()[0]->setEnabled(true); + switch (data->input_) { + case MaliC55CameraData::TPG: { + const MediaEntity *tpgEntity = data->tpgInput.sd_->entity(); + ret = tpgEntity->getPadByIndex(0)->links()[0]->setEnabled(true); + break; + } + case MaliC55CameraData::Inline: { + const MediaEntity *csi2Entity = data->inlineInput.csi2_->entity(); + ret = csi2Entity->getPadByIndex(1)->links()[0]->setEnabled(true); + break; + } } if (ret) return ret; @@ -939,26 +973,28 @@ int PipelineHandlerMaliC55::configure(Camera *camera, MaliC55CameraConfiguration *maliConfig = static_cast(config); V4L2SubdeviceFormat subdevFormat = maliConfig->sensorFormat_; - ret = data->sd_->getFormat(0, &subdevFormat); - if (ret) - return ret; - if (data->sensor_) { - ret = data->sensor_->setFormat(&subdevFormat, - maliConfig->combinedTransform()); + /* Apply format to the origin of the pipeline and propagate it. */ + switch (data->input_) { + case MaliC55CameraData::TPG: + ret = data->tpgInput.sd_->setFormat(0, &subdevFormat); + break; + case MaliC55CameraData::Inline: + ret = data->inlineInput.sensor_->setFormat(&subdevFormat, + maliConfig->combinedTransform()); if (ret) return ret; - } - if (data->csi_) { - ret = data->csi_->setFormat(0, &subdevFormat); + ret = data->inlineInput.csi2_->setFormat(0, &subdevFormat); if (ret) return ret; - ret = data->csi_->getFormat(1, &subdevFormat); - if (ret) - return ret; + ret = data->inlineInput.csi2_->getFormat(1, &subdevFormat); + + break; } + if (ret) + return ret; V4L2DeviceFormat statsFormat; ret = stats_->getFormat(&statsFormat); @@ -973,8 +1009,6 @@ int PipelineHandlerMaliC55::configure(Camera *camera, /* * Propagate the format to the ISP sink pad and configure the input * crop rectangle (no crop at the moment). - * - * \todo Configure the CSI-2 receiver. */ ret = isp_->setFormat(0, &subdevFormat); if (ret) @@ -1058,18 +1092,19 @@ int PipelineHandlerMaliC55::configure(Camera *camera, /* We need to inform the IPA of the sensor configuration */ ipa::mali_c55::IPAConfigInfo ipaConfig{}; - ret = data->sensor_->sensorInfo(&ipaConfig.sensorInfo); + ret = data->inlineInput.sensor_->sensorInfo(&ipaConfig.sensorInfo); if (ret) return ret; - ipaConfig.sensorControls = data->sensor_->controls(); + ipaConfig.sensorControls = data->inlineInput.sensor_->controls(); /* * And we also need to tell the IPA the bayerOrder of the data (as * affected by any flips that we've configured) */ const Transform &combinedTransform = maliConfig->combinedTransform(); - BayerFormat::Order bayerOrder = data->sensor_->bayerOrder(combinedTransform); + BayerFormat::Order bayerOrder = + data->inlineInput.sensor_->bayerOrder(combinedTransform); ControlInfoMap ipaControls; ret = data->ipa_->configure(ipaConfig, utils::to_underlying(bayerOrder), @@ -1283,7 +1318,7 @@ void PipelineHandlerMaliC55::applyScalerCrop(Camera *camera, if (!scalerCrop) return; - if (!data->sensor_) { + if (data->input_ == MaliC55CameraData::TPG) { LOG(MaliC55, Error) << "ScalerCrop not supported for TPG"; return; } @@ -1291,7 +1326,7 @@ void PipelineHandlerMaliC55::applyScalerCrop(Camera *camera, Rectangle nativeCrop = *scalerCrop; IPACameraSensorInfo sensorInfo; - int ret = data->sensor_->sensorInfo(&sensorInfo); + int ret = data->inlineInput.sensor_->sensorInfo(&sensorInfo); if (ret) { LOG(MaliC55, Error) << "Failed to retrieve sensor info"; return; @@ -1572,10 +1607,8 @@ bool PipelineHandlerMaliC55::registerTPGCamera(MediaLink *link) return true; } - std::unique_ptr data = - std::make_unique(this, link->source()->entity()); - - if (data->init()) + std::unique_ptr data = std::make_unique(this); + if (data->initTPG(link->source()->entity())) return false; return registerMaliCamera(std::move(data), name); @@ -1600,21 +1633,22 @@ bool PipelineHandlerMaliC55::registerSensorCamera(MediaLink *ispLink) continue; std::unique_ptr data = - std::make_unique(this, sensor); - if (data->init()) + std::make_unique(this); + if (data->initInlineCamera(sensor)) return false; - data->properties_ = data->sensor_->properties(); + data->properties_ = data->inlineInput.sensor_->properties(); - const CameraSensorProperties::SensorDelays &delays = data->sensor_->sensorDelays(); + const CameraSensorProperties::SensorDelays &delays = + data->inlineInput.sensor_->sensorDelays(); std::unordered_map params = { { V4L2_CID_ANALOGUE_GAIN, { delays.gainDelay, false } }, { V4L2_CID_EXPOSURE, { delays.exposureDelay, false } }, }; - data->delayedCtrls_ = - std::make_unique(data->sensor_->device(), - params); + V4L2Subdevice *sensorSubdev = data->inlineInput.sensor_->device(); + data->delayedCtrls_ = std::make_unique(sensorSubdev, + params); isp_->frameStart.connect(data->delayedCtrls_.get(), &DelayedControls::applyControls); From patchwork Fri Dec 5 14:52:09 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jacopo Mondi X-Patchwork-Id: 25372 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 95C2CC3260 for ; Fri, 5 Dec 2025 14:52:38 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 39C36613DE; Fri, 5 Dec 2025 15:52:34 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="b5ffPrnJ"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 3166F6115A for ; Fri, 5 Dec 2025 15:52:31 +0100 (CET) Received: from [192.168.1.4] (net-93-65-100-155.cust.vodafonedsl.it [93.65.100.155]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id E42F7EAE; Fri, 5 Dec 2025 15:50:14 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1764946215; bh=SgrgQCL9TrEOmvw9ck0E9mB0BXthdIMrBljUU+foLoU=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=b5ffPrnJy416bqdo+D0sBeCjw/Nnjlx/NiAn4WgVaPvRNw0RwbREZWs/EpHamTjLI sF+2/I/Wk0vCFZUPxdgS0Ycgi9ncvXlXysKxDyEiuwDX4tTrdWPNqeQP6phGhVMaNW oQKwWMujTGOX1hMQHHHasm9uFIUfzm/wCXxRETBg= From: Jacopo Mondi Date: Fri, 05 Dec 2025 15:52:09 +0100 Subject: [PATCH 3/7] libcamera: mali-c55: Register memory input camera MIME-Version: 1.0 Message-Id: <20251205-mali-cru-v1-3-d81bb5ffe73a@ideasonboard.com> References: <20251205-mali-cru-v1-0-d81bb5ffe73a@ideasonboard.com> In-Reply-To: <20251205-mali-cru-v1-0-d81bb5ffe73a@ideasonboard.com> To: Daniel Scally , libcamera-devel@lists.libcamera.org Cc: Jacopo Mondi X-Mailer: b4 0.14.2 X-Developer-Signature: v=1; a=openpgp-sha256; l=13306; i=jacopo.mondi@ideasonboard.com; h=from:subject:message-id; bh=vl0sIQFlsoVvekcoNbHCxxhkB0wA+QaJwace1sYFD5E=; b=owEBbQKS/ZANAwAKAXI0Bo8WoVY8AcsmYgBpMvGsBynQHMqrh0gzwxb0kgjyxyr9sUzTelT7s JUBa0cnU7yJAjMEAAEKAB0WIQS1xD1IgJogio9YOMByNAaPFqFWPAUCaTLxrAAKCRByNAaPFqFW PHWMD/4+04KNA1lcHAnvlwWCjf8BGsptafXiPxhd4NcTrX+r2Se8bARU86vryklPARmffGfNHpR jcTGyX1zKjGBQYmsxTHHVzSYruVTtOTjLYVoUSckFLxlIO37TKxcoocqJCgteW7Fr1zsc36jsxI DivGJx6ejMQEd5SgKxz3Yrxujl1Ke+l3YTVsuQev49Pb8ENPOOaZeOIIPpm15ntmsDCX2URoyXe 1hOO83dYpUtydB8XxhBv46lGCEZryLZKMNPJra+Q4IWO4ZaREvUfUtgUUj18IuD2wJ4qtGboOAK fsa9B/m28uO/gZTwyww+WYtYwMGV0DxkuyMd5K85zks7jev9/XMeDGkGY5Ihl+S2+Wd5f/1MXB0 Whf5UaDcAsDPz1K84jBU6BsKR45moBNUcosjckRf8UMRnd5nz/XqzHloutZKdj8ytyXE7oljiic quB7sEtepfjDtpkuVhUgr09HhsHe7vJ2/hDgMP0B0ggwq2wx5re2+mb/9FCOOv670xhVV7tzCOk 6HgV1VTSXQnG8J2jCwsio8ZytYlJ3vko8KcDIoRSpVqV6dQRoxdSVWM0o0Q83+6vqM1zysACLSi Ic0XlRqkresi9rcnq5OElIQUotqA0oBeaJfxlb1VUU1quHeHCtYdPCKtGc1ZG+X5c44Q+XwioSS 7Lr1zBNQJWhb72w== X-Developer-Key: i=jacopo.mondi@ideasonboard.com; a=openpgp; fpr=72392EDC88144A65C701EA9BA5826A2587AD026B X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" From: Daniel Scally Add support to Mali C55 pipeline handler for memory-to-memory camera mode. The Mali-C55, as integrated in the Renesas RZ/V2H(P) SoC, is fed with images saved to memory by the CRU unit and read on behalf of the ISP by the IVC. Update the pipeline handler's match() function to search for the CRU media entity necessary to register a memory input camera. Create and initialize a CameraData instance for the memory camera use case if a valid CRU is found and initialize the IVC components used to feed the ISP with images read from memory. The CameraData functions that deflect the selection of the subdevice at the origin of the pipeline, the list of supported sizes and the maximum supported resolution have been updated to support memory-to-memory operations. Signed-off-by: Daniel Scally Signed-off-by: Jacopo Mondi --- src/libcamera/pipeline/mali-c55/mali-c55.cpp | 181 ++++++++++++++++++++++----- 1 file changed, 150 insertions(+), 31 deletions(-) diff --git a/src/libcamera/pipeline/mali-c55/mali-c55.cpp b/src/libcamera/pipeline/mali-c55/mali-c55.cpp index 801067ce00fe4e0fd6b81db699fcaed2ebb840b6..6816bf7179ae09434ec10c35ea296725d3ac0700 100644 --- a/src/libcamera/pipeline/mali-c55/mali-c55.cpp +++ b/src/libcamera/pipeline/mali-c55/mali-c55.cpp @@ -42,6 +42,8 @@ #include "libcamera/internal/v4l2_subdevice.h" #include "libcamera/internal/v4l2_videodevice.h" +#include "rzg2l-cru.h" + namespace { bool isFormatRaw(const libcamera::PixelFormat &pixFmt) @@ -95,6 +97,7 @@ public: enum CameraType { TPG, Inline, + Memory, }; MaliC55CameraData(PipelineHandler *pipe) @@ -104,10 +107,11 @@ public: int loadIPA(); - /* Deflect these functionalities to either TPG or CameraSensor. */ + /* Deflect these functionalities to either TPG, inline sensor or CRU. */ std::vector sizes(unsigned int mbusCode) const; Size resolution() const; V4L2Subdevice *subdev() const; + CameraSensor *sensor() const; int pixfmtToMbusCode(const PixelFormat &pixFmt) const; const PixelFormat &bestRawFormat() const; @@ -119,6 +123,7 @@ public: int initTPG(MediaEntity *tpg); int initInlineCamera(MediaEntity *sensor); + void initMemoryCamera(); Stream frStream_; Stream dsStream_; @@ -139,10 +144,15 @@ public: std::unique_ptr sensor_; } inlineInput; + struct { + std::unique_ptr cru_; + } memoryInput; + CameraType input_; private: void setSensorControls(const ControlList &sensorControls); + std::vector tpgSizes(unsigned int mbusCode) const; std::string id_; }; @@ -196,16 +206,18 @@ int MaliC55CameraData::initInlineCamera(MediaEntity *sensor) return 0; } +void MaliC55CameraData::initMemoryCamera() +{ + input_ = Memory; +} + void MaliC55CameraData::setSensorControls(const ControlList &sensorControls) { delayedCtrls_->push(sensorControls); } -std::vector MaliC55CameraData::sizes(unsigned int mbusCode) const +std::vector MaliC55CameraData::tpgSizes(unsigned int mbusCode) const { - if (input_ == Inline) - return inlineInput.sensor_->sizes(mbusCode); - V4L2Subdevice::Formats formats = tpgInput.sd_->formats(0); if (formats.empty()) return {}; @@ -224,6 +236,21 @@ std::vector MaliC55CameraData::sizes(unsigned int mbusCode) const return sizes; } +std::vector MaliC55CameraData::sizes(unsigned int mbusCode) const +{ + switch (input_) { + case TPG: + return tpgSizes(mbusCode); + case Inline: + return inlineInput.sensor_->sizes(mbusCode); + case Memory: + return memoryInput.cru_->sizes(); + } + + assert(false); + return {}; +} + Size MaliC55CameraData::resolution() const { switch (input_) { @@ -231,6 +258,8 @@ Size MaliC55CameraData::resolution() const return tpgInput.resolution_; case Inline: return inlineInput.sensor_->resolution(); + case Memory: + return memoryInput.cru_->resolution(); } assert(false); @@ -244,6 +273,27 @@ V4L2Subdevice *MaliC55CameraData::subdev() const return tpgInput.sd_.get(); case Inline: return inlineInput.sensor_->device(); + case Memory: + return memoryInput.cru_->sensor()->device(); + } + + assert(false); + return nullptr; +} + +CameraSensor *MaliC55CameraData::sensor() const +{ + switch (input_) { + case Inline: + return inlineInput.sensor_.get(); + case Memory: + return memoryInput.cru_->sensor(); + case TPG: + /* + * TPG has no camera sensor, we shall never call this + * function if TPG is in use. Exit the switch and assert(). + */ + break; } assert(false); @@ -332,7 +382,8 @@ void MaliC55CameraData::updateControls(const ControlInfoMap &ipaControls) return; IPACameraSensorInfo sensorInfo; - int ret = inlineInput.sensor_->sensorInfo(&sensorInfo); + CameraSensor *sensor = this->sensor(); + int ret = sensor->sensorInfo(&sensorInfo); if (ret) { LOG(MaliC55, Error) << "Failed to retrieve sensor info"; return; @@ -414,7 +465,7 @@ int MaliC55CameraData::loadIPA() ipa_->setSensorControls.connect(this, &MaliC55CameraData::setSensorControls); - CameraSensor *sensor = inlineInput.sensor_.get(); + CameraSensor *sensor = this->sensor(); std::string ipaTuningFile = ipa_->configurationFile(sensor->model() + ".yaml", "uncalibrated.yaml"); @@ -475,8 +526,8 @@ CameraConfiguration::Status MaliC55CameraConfiguration::validate() combinedTransform_ = Transform::Rot0; } else { Orientation requestedOrientation = orientation; - combinedTransform_ = - data_->inlineInput.sensor_->computeTransform(&orientation); + CameraSensor *sensor = data_->sensor(); + combinedTransform_ = sensor->computeTransform(&orientation); if (orientation != requestedOrientation) status = Adjusted; } @@ -709,11 +760,15 @@ private: const std::string &name); bool registerTPGCamera(MediaLink *link); bool registerSensorCamera(MediaLink *link); + bool registerMemoryInputCamera(); std::shared_ptr media_; + std::shared_ptr cruMedia_; std::unique_ptr isp_; std::unique_ptr stats_; std::unique_ptr params_; + std::unique_ptr ivc_; + std::unique_ptr input_; std::vector> statsBuffers_; std::queue availableStatsBuffers_; @@ -966,6 +1021,8 @@ int PipelineHandlerMaliC55::configure(Camera *camera, ret = csi2Entity->getPadByIndex(1)->links()[0]->setEnabled(true); break; } + case MaliC55CameraData::Memory: + break; } if (ret) return ret; @@ -991,6 +1048,8 @@ int PipelineHandlerMaliC55::configure(Camera *camera, ret = data->inlineInput.csi2_->getFormat(1, &subdevFormat); + break; + case MaliC55CameraData::Memory: break; } if (ret) @@ -1065,6 +1124,7 @@ int PipelineHandlerMaliC55::configure(Camera *camera, pipe->stream = stream; } + /* TPG doesn't support the IPA, so stop here. */ if (!data->ipa_) return 0; @@ -1091,20 +1151,20 @@ int PipelineHandlerMaliC55::configure(Camera *camera, /* We need to inform the IPA of the sensor configuration */ ipa::mali_c55::IPAConfigInfo ipaConfig{}; + CameraSensor *sensor = data->sensor(); - ret = data->inlineInput.sensor_->sensorInfo(&ipaConfig.sensorInfo); + ret = sensor->sensorInfo(&ipaConfig.sensorInfo); if (ret) return ret; - ipaConfig.sensorControls = data->inlineInput.sensor_->controls(); + ipaConfig.sensorControls = sensor->controls(); /* * And we also need to tell the IPA the bayerOrder of the data (as * affected by any flips that we've configured) */ const Transform &combinedTransform = maliConfig->combinedTransform(); - BayerFormat::Order bayerOrder = - data->inlineInput.sensor_->bayerOrder(combinedTransform); + BayerFormat::Order bayerOrder = sensor->bayerOrder(combinedTransform); ControlInfoMap ipaControls; ret = data->ipa_->configure(ipaConfig, utils::to_underlying(bayerOrder), @@ -1323,10 +1383,11 @@ void PipelineHandlerMaliC55::applyScalerCrop(Camera *camera, return; } + CameraSensor *sensor = data->sensor(); Rectangle nativeCrop = *scalerCrop; - IPACameraSensorInfo sensorInfo; - int ret = data->inlineInput.sensor_->sensorInfo(&sensorInfo); + + int ret = sensor->sensorInfo(&sensorInfo); if (ret) { LOG(MaliC55, Error) << "Failed to retrieve sensor info"; return; @@ -1661,6 +1722,39 @@ bool PipelineHandlerMaliC55::registerSensorCamera(MediaLink *ispLink) return true; } +bool PipelineHandlerMaliC55::registerMemoryInputCamera() +{ + std::unique_ptr data = std::make_unique(this); + + data->memoryInput.cru_ = std::make_unique(); + int ret = data->memoryInput.cru_->init(cruMedia_.get()); + if (ret) + return false; + + data->initMemoryCamera(); + + CameraSensor *sensor = data->memoryInput.cru_->sensor(); + data->properties_ = sensor->properties(); + + const CameraSensorProperties::SensorDelays &delays = sensor->sensorDelays(); + std::unordered_map params = { + { V4L2_CID_ANALOGUE_GAIN, { delays.gainDelay, false } }, + { V4L2_CID_EXPOSURE, { delays.exposureDelay, false } }, + }; + + data->delayedCtrls_ = std::make_unique(sensor->device(), params); + isp_->frameStart.connect(data->delayedCtrls_.get(), + &DelayedControls::applyControls); + + input_->bufferReady.connect(data->memoryInput.cru_.get(), + &RZG2LCRU::cruReturnBuffer); + + if (!registerMaliCamera(std::move(data), sensor->device()->entity()->name())) + return false; + + return true; +} + bool PipelineHandlerMaliC55::match(DeviceEnumerator *enumerator) { const MediaPad *ispSink; @@ -1670,14 +1764,14 @@ bool PipelineHandlerMaliC55::match(DeviceEnumerator *enumerator) * The TPG and the downscale pipe are both optional blocks and may not * be fitted. */ - DeviceMatch dm("mali-c55"); - dm.add("mali-c55 isp"); - dm.add("mali-c55 resizer fr"); - dm.add("mali-c55 fr"); - dm.add("mali-c55 3a stats"); - dm.add("mali-c55 3a params"); - - media_ = acquireMediaDevice(enumerator, dm); + DeviceMatch c55_dm("mali-c55"); + c55_dm.add("mali-c55 isp"); + c55_dm.add("mali-c55 resizer fr"); + c55_dm.add("mali-c55 fr"); + c55_dm.add("mali-c55 3a stats"); + c55_dm.add("mali-c55 3a params"); + + media_ = acquireMediaDevice(enumerator, c55_dm); if (!media_) return false; @@ -1737,6 +1831,22 @@ bool PipelineHandlerMaliC55::match(DeviceEnumerator *enumerator) stats_->bufferReady.connect(this, &PipelineHandlerMaliC55::statsBufferReady); params_->bufferReady.connect(this, &PipelineHandlerMaliC55::paramsBufferReady); + /* + * We also need to search for the rzg2l-cru CSI-2 receiver. If we find + * that then we need to work in memory input mode instead of the inline + * mode. The absence of this match is not necessarily a failure at this + * point...it depends on the media links that we investigate momentarily. + * + * This is a bit hacky, because there could be multiple of these media + * devices and we're just taking the first. We need modular pipelines to + * properly solve the issue. + */ + DeviceMatch cru_dm("rzg2l_cru"); + cru_dm.add(std::regex("csi-[0-9a-f]{8}.csi2")); + cru_dm.add(std::regex("cru-ip-[0-9a-f]{8}.cru[0-9]")); + cru_dm.add("CRU output"); + cruMedia_ = acquireMediaDevice(enumerator, cru_dm); + ispSink = isp_->entity()->getPadByIndex(0); if (!ispSink || ispSink->links().empty()) { LOG(MaliC55, Error) << "ISP sink pad error"; @@ -1750,13 +1860,6 @@ bool PipelineHandlerMaliC55::match(DeviceEnumerator *enumerator) * MEDIA_ENT_F_CAM_SENSOR - The test pattern generator * MEDIA_ENT_F_VID_IF_BRIDGE - A CSI-2 receiver * MEDIA_ENT_F_PROC_VIDEO_PIXEL_FORMATTER - An input device - * - * The last one will be unsupported for now. The TPG is relatively easy, - * we just register a Camera for it. If we have a CSI-2 receiver we need - * to check its sink pad and register Cameras for anything connected to - * it (probably...there are some complex situations in which that might - * not be true but let's pretend they don't exist until we come across - * them) */ bool registered; for (MediaLink *link : ispSink->links()) { @@ -1776,7 +1879,23 @@ bool PipelineHandlerMaliC55::match(DeviceEnumerator *enumerator) break; case MEDIA_ENT_F_PROC_VIDEO_PIXEL_FORMATTER: - LOG(MaliC55, Warning) << "Memory input not yet supported"; + if (!cruMedia_) + return false; + + ivc_ = V4L2Subdevice::fromEntityName(media_.get(), + "rzv2h ivc block"); + if (ivc_->open() < 0) + return false; + + input_ = V4L2VideoDevice::fromEntityName(media_.get(), + "rzv2h-ivc"); + if (input_->open() < 0) + return false; + + registered = registerMemoryInputCamera(); + if (!registered) + return registered; + break; default: LOG(MaliC55, Error) << "Unsupported entity function"; From patchwork Fri Dec 5 14:52:10 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jacopo Mondi X-Patchwork-Id: 25374 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 756C2C326B for ; Fri, 5 Dec 2025 14:52:40 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 1E311613F9; Fri, 5 Dec 2025 15:52:36 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="CYvNHnkM"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 6B023613A5 for ; Fri, 5 Dec 2025 15:52:31 +0100 (CET) Received: from [192.168.1.4] (net-93-65-100-155.cust.vodafonedsl.it [93.65.100.155]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 2C5E711EF; Fri, 5 Dec 2025 15:50:15 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1764946215; bh=YjJ9aXd41gmwwuZExWFJU4Q3KYuLVmrMwL87At7x/54=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=CYvNHnkMxtJjXQT6ZP5AcQ7O12+6zj2ocMaw3RQq9jSqyZg1vNUnJBM6/+WikvLBK kR5j5TBQYx/QaFNWUnzQ6vMgTzhw8qyIwKdAbwvG9htw3jS+X6CekGnQu/sbm/m+5p poOetMU4J5hgg5ogyyfh0uw4BpRYaj9Ui4beFZU4= From: Jacopo Mondi Date: Fri, 05 Dec 2025 15:52:10 +0100 Subject: [PATCH 4/7] libcamera: mali-c55: Configure camera in memory-to-memory MIME-Version: 1.0 Message-Id: <20251205-mali-cru-v1-4-d81bb5ffe73a@ideasonboard.com> References: <20251205-mali-cru-v1-0-d81bb5ffe73a@ideasonboard.com> In-Reply-To: <20251205-mali-cru-v1-0-d81bb5ffe73a@ideasonboard.com> To: Daniel Scally , libcamera-devel@lists.libcamera.org Cc: Jacopo Mondi X-Mailer: b4 0.14.2 X-Developer-Signature: v=1; a=openpgp-sha256; l=1828; i=jacopo.mondi@ideasonboard.com; h=from:subject:message-id; bh=YjJ9aXd41gmwwuZExWFJU4Q3KYuLVmrMwL87At7x/54=; b=owEBbQKS/ZANAwAKAXI0Bo8WoVY8AcsmYgBpMvGtxWQUjspbepm4kjtnEcqCa+phRAaKWBMOf yvXeein2dSJAjMEAAEKAB0WIQS1xD1IgJogio9YOMByNAaPFqFWPAUCaTLxrQAKCRByNAaPFqFW PId9EACkUuQAlk0xkIEun1SMk9vvebbjb3GGS9JFkfJeufSPJsO52v/yN29FIcKZ6OopIRpLZVx eCXoz15bMq+G9Si2oDQ1qd+RhaoeRdaSrflyKVkYkbZwuyD8abHi/4jHXFR46fuQiTd4eh76Ff7 Jjdetu15G4bxkNx6gwR52Q50jNql+Rosbf+CUyy6Ja7o120iwLX9lXepRjcXuQGqwMqEZGHiCWB 47BqLfAZWwyN0jv/uSHywiGdsV8mMmnJQ0yTX6slGIJsHOCCTfR4f0xI+1bBW/4MGBMwNt/GJ59 C2NY0PLrTSg9178EAWEm7cBkB0NJ2B2qtzXk39uk2Wism2jwwhDIKI6bKF6nK1GWS0Nk2XA2Lki s9CzRqdcS7EFTu0uy72XuFi7V3LGAJwkEqIdGJw74+TVbPWLLjMG2MQXO8dNK5iy8BUmZ+5hu1i 0ec95EiXyAqkd6ezerQaIMcVFkxVYO1FLBInlU4blhOxrZQ9ZTeFR8Bg+ufO7p03N7og/NbjMgc AqB+AmPUpPffzWhaZAJliXOGVxdcrfocnAMGOF3S9rFlCLxvgH3BCgZZZNXgsqt5/hT4b7yRgxI V/sOR16Pb1uaFlqxYJLGoVJD+NGHe7ALXvWjCBMz8z3uddg2L0+r31L942wV9ikKaPWpgqdqk7c +Qu+b3/bY7Hsbkw== X-Developer-Key: i=jacopo.mondi@ideasonboard.com; a=openpgp; fpr=72392EDC88144A65C701EA9BA5826A2587AD026B X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Add support for memory-to-memory Camera to the PipelineHandlerMaliC55::configure() function. Start by enabling the IVC links, then configure the CRU and propagate the format to the IVC. Signed-off-by: Jacopo Mondi --- src/libcamera/pipeline/mali-c55/mali-c55.cpp | 26 ++++++++++++++++++++++++-- 1 file changed, 24 insertions(+), 2 deletions(-) diff --git a/src/libcamera/pipeline/mali-c55/mali-c55.cpp b/src/libcamera/pipeline/mali-c55/mali-c55.cpp index 6816bf7179ae09434ec10c35ea296725d3ac0700..6dce315c1c82db3554e8c0eae727cba9d632ca82 100644 --- a/src/libcamera/pipeline/mali-c55/mali-c55.cpp +++ b/src/libcamera/pipeline/mali-c55/mali-c55.cpp @@ -1021,9 +1021,12 @@ int PipelineHandlerMaliC55::configure(Camera *camera, ret = csi2Entity->getPadByIndex(1)->links()[0]->setEnabled(true); break; } - case MaliC55CameraData::Memory: + case MaliC55CameraData::Memory: { + const MediaEntity *ivcEntity = ivc_->entity(); + ret = ivcEntity->getPadByIndex(1)->links()[0]->setEnabled(true); break; } + } if (ret) return ret; @@ -1049,9 +1052,28 @@ int PipelineHandlerMaliC55::configure(Camera *camera, ret = data->inlineInput.csi2_->getFormat(1, &subdevFormat); break; - case MaliC55CameraData::Memory: + case MaliC55CameraData::Memory: { + V4L2DeviceFormat inputFormat; + ret = data->memoryInput.cru_->configure(&subdevFormat, &inputFormat); + if (ret) + return ret; + + /* Propagate the CRU format to the IVC input. */ + ret = ivc_->setFormat(0, &subdevFormat); + if (ret) + return ret; + + ret = ivc_->getFormat(1, &subdevFormat); + if (ret) + return ret; + + ret = input_->setFormat(&inputFormat); + if (ret) + return ret; + break; } + } if (ret) return ret; From patchwork Fri Dec 5 14:52:11 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jacopo Mondi X-Patchwork-Id: 25375 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id DF2EFC32AF for ; Fri, 5 Dec 2025 14:52:40 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 314DF613EB; Fri, 5 Dec 2025 15:52:37 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="vJFfz/Es"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 941A8606A0 for ; Fri, 5 Dec 2025 15:52:31 +0100 (CET) Received: from [192.168.1.4] (net-93-65-100-155.cust.vodafonedsl.it [93.65.100.155]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 6C47D1340; Fri, 5 Dec 2025 15:50:15 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1764946215; bh=A/dnpEvqJ410WD7m1lspdx6fWRD2/0pPEEWUV8JhJ4g=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=vJFfz/EsYaCCrP8FlAgr7r7I19X0nKe3n6syEKGNepa5J2+8I5L+8/s3FhoQkFpJI HuzSu6ohYRSLSdzJuTqcfmlgLPWO4auYcaltfktvzRsHlcOoku/jKr+LsCsYlP1Bbg /XmPDaHxZ07COlY1Ed6vUADTHic1em7PHgiacDOU= From: Jacopo Mondi Date: Fri, 05 Dec 2025 15:52:11 +0100 Subject: [PATCH 5/7] libcamera: mali-c55: Implement capture for memory-to-memory MIME-Version: 1.0 Message-Id: <20251205-mali-cru-v1-5-d81bb5ffe73a@ideasonboard.com> References: <20251205-mali-cru-v1-0-d81bb5ffe73a@ideasonboard.com> In-Reply-To: <20251205-mali-cru-v1-0-d81bb5ffe73a@ideasonboard.com> To: Daniel Scally , libcamera-devel@lists.libcamera.org Cc: Jacopo Mondi X-Mailer: b4 0.14.2 X-Developer-Signature: v=1; a=openpgp-sha256; l=9311; i=jacopo.mondi@ideasonboard.com; h=from:subject:message-id; bh=4UcvgR4scTrxJYrijGo+f5eFnqOvmZWIz+luVNrEiCo=; b=owEBbQKS/ZANAwAKAXI0Bo8WoVY8AcsmYgBpMvGtixsOn0mJ0Y4bp6ty0X9Tc4GwwAdWkIxwZ Y0oiQgUhh+JAjMEAAEKAB0WIQS1xD1IgJogio9YOMByNAaPFqFWPAUCaTLxrQAKCRByNAaPFqFW PMx4D/0TYj/sO5NvolfcYCGsm+IJZ1PnM+Tz+HvGaPO3WC4hQtRhAUvWpwXu71PidXzqiOHXTe/ 0Md/DwMSzJ1wmjuFi9UNwF+hTGN2Eek3pjL8yV2n7HmgV1WP99WcFsZdksKSoeiqWPOTI54PBlr WXURmugqHHU1eEUNFpRhC3eZ1iZQsoLQOcfZDfyGsakr+9GUpT5WTidfsred9QMToItGJUO2LCX IfPv2skq2TTRTq7/nsWxl8XxhP3xgj5Et0SI1S+znE4pFYzsY6YU9/EwFQQ7MMLqxQTfNtVJUBh 1FGFPXRN1VF/FnGRyyTFnUmZ/bOZfI0S2ja2N5Qn67tKGYVxC+ctXe0ePRQEYGOyDg+DuVns7og ChAwh/+tuGyPxV28upf03XJfAaw6dP58AMTd7T+ChhuBV8+u6g70Em9QLJAIeNwt+4Hup/wZCQk jqXdvfK05CXojcS7GewSIEW03Z2rV9x9kUp/mBjIakLajrqlwcFuQuVWLxz6bco7HqcrYQkDMF4 e0N82BIiQIj2fBBTBFOarjt/3Qv3j3cnxWjmA/jA9K7dPoTzhr8vzB7cHRqNA4xIifFXzurkvJx a1PNlE0AEEGv49UIuZUy84cuoSf9yw3ZGmkY+wA4MxcHPmQ2eGnl1w7v4GCATeC55i2qCkhSxkn lfcZPHhYzgLbMeg== X-Developer-Key: i=jacopo.mondi@ideasonboard.com; a=openpgp; fpr=72392EDC88144A65C701EA9BA5826A2587AD026B X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" From: Daniel Scally Plumb in the MaliC55 pipeline handler support for capturing frames from memory using the CRU. Introduce a data flow which uses the CRU to feed the ISP through the IVC. In detail: - push incoming request to a pending queue until a buffer from the CRU is available - delay the call to ipa_->fillParams() to the CRU buffer ready even - once the IPA has computed parameters feed the ISP through the IVC with buffers from the CRU, params and statistics. Signed-off-by: Daniel Scally Signed-off-by: Jacopo Mondi --- src/libcamera/pipeline/mali-c55/mali-c55.cpp | 164 ++++++++++++++++++++++++++- 1 file changed, 159 insertions(+), 5 deletions(-) diff --git a/src/libcamera/pipeline/mali-c55/mali-c55.cpp b/src/libcamera/pipeline/mali-c55/mali-c55.cpp index 6dce315c1c82db3554e8c0eae727cba9d632ca82..46ef5e7735a30a0d4ae9bb6f8d671bbd2dff3f51 100644 --- a/src/libcamera/pipeline/mali-c55/mali-c55.cpp +++ b/src/libcamera/pipeline/mali-c55/mali-c55.cpp @@ -19,6 +19,7 @@ #include #include +#include #include #include #include @@ -86,6 +87,7 @@ struct MaliC55FrameInfo { FrameBuffer *paramBuffer; FrameBuffer *statBuffer; + FrameBuffer *rawBuffer; bool paramsDone; bool statsDone; @@ -692,11 +694,14 @@ public: int start(Camera *camera, const ControlList *controls) override; void stopDevice(Camera *camera) override; + int queuePendingRequests(); + void cancelPendingRequests(); int queueRequestDevice(Camera *camera, Request *request) override; void imageBufferReady(FrameBuffer *buffer); void paramsBufferReady(FrameBuffer *buffer); void statsBufferReady(FrameBuffer *buffer); + void cruBufferReady(FrameBuffer *buffer); void paramsComputed(unsigned int requestId, uint32_t bytesused); void statsProcessed(unsigned int requestId, const ControlList &metadata); @@ -778,6 +783,11 @@ private: std::map frameInfoMap_; + /* Requests for which no buffer has been queued to the CRU device yet. */ + std::queue pendingRequests_; + /* Requests queued to the CRU device but not yet processed by the ISP. */ + std::queue processingRequests_; + std::array pipes_; bool dsFitted_; @@ -1235,6 +1245,11 @@ void PipelineHandlerMaliC55::freeBuffers(Camera *camera) if (params_->releaseBuffers()) LOG(MaliC55, Error) << "Failed to release params buffers"; + if (data->input_ == MaliC55CameraData::Memory) { + if (input_->releaseBuffers()) + LOG(MaliC55, Error) << "Failed to release input buffers"; + } + return; } @@ -1264,6 +1279,12 @@ int PipelineHandlerMaliC55::allocateBuffers(Camera *camera) } }; + if (input_) { + ret = input_->importBuffers(RZG2LCRU::kBufferCount); + if (ret < 0) + return ret; + } + ret = stats_->allocateBuffers(bufferCount, &statsBuffers_); if (ret < 0) return ret; @@ -1295,6 +1316,24 @@ int PipelineHandlerMaliC55::start(Camera *camera, [[maybe_unused]] const Control if (ret) return ret; + if (data->input_ == MaliC55CameraData::Memory) { + ret = data->memoryInput.cru_->start(); + if (ret) { + LOG(MaliC55, Error) + << "Failed to start CRU " << camera->id(); + freeBuffers(camera); + return ret; + } + + ret = input_->streamOn(); + if (ret) { + LOG(MaliC55, Error) + << "Failed to start IVC" << camera->id(); + freeBuffers(camera); + return ret; + } + } + if (data->ipa_) { ret = data->ipa_->start(); if (ret) { @@ -1384,6 +1423,12 @@ void PipelineHandlerMaliC55::stopDevice(Camera *camera) pipe.cap->releaseBuffers(); } + if (data->input_ == MaliC55CameraData::Memory) { + cancelPendingRequests(); + input_->streamOff(); + data->memoryInput.cru_->stop(); + } + stats_->streamOff(); params_->streamOff(); if (data->ipa_) @@ -1488,10 +1533,87 @@ void PipelineHandlerMaliC55::applyScalerCrop(Camera *camera, } } +void PipelineHandlerMaliC55::cancelPendingRequests() +{ + processingRequests_ = {}; + + while (!pendingRequests_.empty()) { + Request *request = pendingRequests_.front(); + + completeRequest(request); + pendingRequests_.pop(); + } +} + +int PipelineHandlerMaliC55::queuePendingRequests() +{ + while (!pendingRequests_.empty()) { + Request *request = pendingRequests_.front(); + + if (availableStatsBuffers_.empty()) { + LOG(MaliC55, Error) << "Stats buffer underrun"; + return -ENOENT; + } + + if (availableParamsBuffers_.empty()) { + LOG(MaliC55, Error) << "Params buffer underrun"; + return -ENOENT; + } + + MaliC55FrameInfo frameInfo; + frameInfo.request = request; + + MaliC55CameraData *data = cameraData(request->_d()->camera()); + frameInfo.rawBuffer = data->memoryInput.cru_->queueBuffer(request); + if (!frameInfo.rawBuffer) + return -ENOENT; + + frameInfo.statBuffer = availableStatsBuffers_.front(); + availableStatsBuffers_.pop(); + frameInfo.paramBuffer = availableParamsBuffers_.front(); + availableParamsBuffers_.pop(); + + frameInfo.paramsDone = false; + frameInfo.statsDone = false; + + frameInfoMap_[request->sequence()] = frameInfo; + + for (auto &[stream, buffer] : request->buffers()) { + MaliC55Pipe *pipe = pipeFromStream(data, stream); + + pipe->cap->queueBuffer(buffer); + } + + data->ipa_->queueRequest(request->sequence(), request->controls()); + + pendingRequests_.pop(); + processingRequests_.push(request); + } + + return 0; +} + int PipelineHandlerMaliC55::queueRequestDevice(Camera *camera, Request *request) { MaliC55CameraData *data = cameraData(camera); + /* + * If we're in memory input mode, we need to pop the requests onto the + * pending list until a CRU buffer is ready...otherwise we can just do + * everything immediately. + */ + if (data->input_ == MaliC55CameraData::Memory) { + pendingRequests_.push(request); + + int ret = queuePendingRequests(); + if (ret) { + pendingRequests_.pop(); + return ret; + } + + return 0; + } + /* Do not run the IPA if the TPG is in use. */ if (!data->ipa_) { MaliC55FrameInfo frameInfo; @@ -1556,7 +1678,8 @@ MaliC55FrameInfo *PipelineHandlerMaliC55::findFrameInfo(FrameBuffer *buffer) { for (auto &[sequence, info] : frameInfoMap_) { if (info.paramBuffer == buffer || - info.statBuffer == buffer) + info.statBuffer == buffer || + info.rawBuffer == buffer) return &info; } @@ -1618,6 +1741,26 @@ void PipelineHandlerMaliC55::statsBufferReady(FrameBuffer *buffer) sensorControls); } +void PipelineHandlerMaliC55::cruBufferReady(FrameBuffer *buffer) +{ + MaliC55FrameInfo *info = findFrameInfo(buffer); + Request *request = info->request; + ASSERT(info); + + if (buffer->metadata().status == FrameMetadata::FrameCancelled) { + frameInfoMap_.erase(request->sequence()); + completeRequest(request); + return; + } + + request->metadata().set(controls::SensorTimestamp, + buffer->metadata().timestamp); + + /* Ought we do something with the sensor's controls here...? */ + MaliC55CameraData *data = cameraData(request->_d()->camera()); + data->ipa_->fillParams(request->sequence(), info->paramBuffer->cookie()); +} + void PipelineHandlerMaliC55::paramsComputed(unsigned int requestId, uint32_t bytesused) { MaliC55FrameInfo &frameInfo = frameInfoMap_[requestId]; @@ -1626,18 +1769,27 @@ void PipelineHandlerMaliC55::paramsComputed(unsigned int requestId, uint32_t byt /* * Queue buffers for stats and params, then queue buffers to the capture - * video devices. + * video devices if we're running in Inline mode or with the TPG. + * + * If we're running in M2M buffers have been queued to the capture + * devices at queuePendingRequests() time and here we only have to queue + * buffers to the IVC input to start a transfer. */ frameInfo.paramBuffer->_d()->metadata().planes()[0].bytesused = bytesused; params_->queueBuffer(frameInfo.paramBuffer); stats_->queueBuffer(frameInfo.statBuffer); - for (auto &[stream, buffer] : request->buffers()) { - MaliC55Pipe *pipe = pipeFromStream(data, stream); + if (data->input_ != MaliC55CameraData::Memory) { + for (auto &[stream, buffer] : request->buffers()) { + MaliC55Pipe *pipe = pipeFromStream(data, stream); - pipe->cap->queueBuffer(buffer); + pipe->cap->queueBuffer(buffer); + } } + + if (data->input_ == MaliC55CameraData::Memory) + input_->queueBuffer(frameInfo.rawBuffer); } void PipelineHandlerMaliC55::statsProcessed(unsigned int requestId, @@ -1768,6 +1920,8 @@ bool PipelineHandlerMaliC55::registerMemoryInputCamera() isp_->frameStart.connect(data->delayedCtrls_.get(), &DelayedControls::applyControls); + V4L2VideoDevice *cruOutput = data->memoryInput.cru_->output(); + cruOutput->bufferReady.connect(this, &PipelineHandlerMaliC55::cruBufferReady); input_->bufferReady.connect(data->memoryInput.cru_.get(), &RZG2LCRU::cruReturnBuffer); From patchwork Fri Dec 5 14:52:12 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jacopo Mondi X-Patchwork-Id: 25376 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 5F6B0C32DE for ; Fri, 5 Dec 2025 14:52:41 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 2766161400; Fri, 5 Dec 2025 15:52:38 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="umLkbG4T"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id D6853613A4 for ; Fri, 5 Dec 2025 15:52:31 +0100 (CET) Received: from [192.168.1.4] (net-93-65-100-155.cust.vodafonedsl.it [93.65.100.155]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id A7FCC558; Fri, 5 Dec 2025 15:50:15 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1764946215; bh=GaTpeR6Ev0sWnhJLRy/0ww3MNJ37XvE6rxX0o9HyIOY=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=umLkbG4TfEBJrxfYQUKMB9alVYJ2M8CWGpMJ6yT99ZVSndj+SJ7NIni5KxS/HxRY2 /Jpg1n+cITViSTH9otNViBOmOlzsvtxMtSJs3yNJHylr2duvxiQhWc+cvfONsBMh+b yXr1aWhX5DhEiZUjwgmkKwnM8Luq0W6geTnxi2J4= From: Jacopo Mondi Date: Fri, 05 Dec 2025 15:52:12 +0100 Subject: [PATCH 6/7] libcamera: mali-c55: Fix sensor size computation MIME-Version: 1.0 Message-Id: <20251205-mali-cru-v1-6-d81bb5ffe73a@ideasonboard.com> References: <20251205-mali-cru-v1-0-d81bb5ffe73a@ideasonboard.com> In-Reply-To: <20251205-mali-cru-v1-0-d81bb5ffe73a@ideasonboard.com> To: Daniel Scally , libcamera-devel@lists.libcamera.org Cc: Jacopo Mondi X-Mailer: b4 0.14.2 X-Developer-Signature: v=1; a=openpgp-sha256; l=1004; i=jacopo.mondi@ideasonboard.com; h=from:subject:message-id; bh=GaTpeR6Ev0sWnhJLRy/0ww3MNJ37XvE6rxX0o9HyIOY=; b=owEBbQKS/ZANAwAKAXI0Bo8WoVY8AcsmYgBpMvGt5rPmI8V3p0XhUrTKGPuJQkvN4bnBZg4GL 3mfcAN7jHKJAjMEAAEKAB0WIQS1xD1IgJogio9YOMByNAaPFqFWPAUCaTLxrQAKCRByNAaPFqFW PMNfEACMvcrhN0shAlAfxJ1CXvPVvW15Y2sfxOS7BOJjlzPWo1xDVgMhBYscO/Rtwwh2NkAHEsd xY6Z80Lfct0qCpMMfwqy+zl2o06AjpWnvKeNopXoaJi8iNEr+x7vgZ1Fgmh1evjyKIm6gpCrqab 09YcJdJbBnFQzwVuHmj62CktTKQS3dojqn3xjyBX41mEkInCCnGde9ejDEtGT3VoYAokubAFdPM EmJ1/cEjtE8Xtbrqx9ohTo1YWj4OPj3jRK6v3UhYTMPhAtIZ2sUt5CbZbefXKci8iVJ0F/qFSL/ 04zRQ/h/x9N/TUc2EEGPDqAjnQUL8qi4HMATtAtzGMXooptIi8zRUW+RxCkPD3e8Ib9QIhc6+sb MKrnozVGsGGbuo18RROVD3iy0ijPVkrfgt5BxkfGf5NsabVmqP55OcQ9lqx3vPRdRj+BSGwgopL 0HEr6thEUN+ugMLYV1uCx5mjPmoh5dckF8u+1mf93utknmixIaNaMQdbV4UsrXJW4u0IxorC0Af /Qqtfs7hBeVzQNEOwiJ4uSwbMzCotlV933BbzO0agdBBdYf04gNY/mIuBEIM0Ihc1bgy4Xt2Dft ix867cmGGck54KRxR9MzlvGtqg1nASvxhoqPElZkv8nQ50LHxMdr8PQI3h9mkhMA3B9D8oGBdnF vVn9Q1oZlkXQ2nA== X-Developer-Key: i=jacopo.mondi@ideasonboard.com; a=openpgp; fpr=72392EDC88144A65C701EA9BA5826A2587AD026B X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" The sensor size computation routine doesn't work well. Even if a more appriate size for a sensor is available, the largest one is always selected. Fix the size computation procedure which has an inverted assignment. Signed-off-by: Jacopo Mondi --- src/libcamera/pipeline/mali-c55/mali-c55.cpp | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/libcamera/pipeline/mali-c55/mali-c55.cpp b/src/libcamera/pipeline/mali-c55/mali-c55.cpp index 46ef5e7735a30a0d4ae9bb6f8d671bbd2dff3f51..a0a880f26281e5cd9ec176153970a2dd1d230496 100644 --- a/src/libcamera/pipeline/mali-c55/mali-c55.cpp +++ b/src/libcamera/pipeline/mali-c55/mali-c55.cpp @@ -666,7 +666,7 @@ CameraConfiguration::Status MaliC55CameraConfiguration::validate() std::abs(static_cast(minSensorSize.height) - static_cast(size.height)); if (dist < distance) { - dist = distance; + distance = dist; bestSize = size; } } From patchwork Fri Dec 5 14:52:13 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jacopo Mondi X-Patchwork-Id: 25377 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id AF62FC32E0 for ; Fri, 5 Dec 2025 14:52:41 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id E5476613FC; Fri, 5 Dec 2025 15:52:38 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="TMK5S3J3"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 16299613D8 for ; Fri, 5 Dec 2025 15:52:32 +0100 (CET) Received: from [192.168.1.4] (net-93-65-100-155.cust.vodafonedsl.it [93.65.100.155]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id E91481698; Fri, 5 Dec 2025 15:50:15 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1764946216; bh=d4+hyZoEoJcgjx/USn5RV1JkaAAFj++EIQfUN+n8wRQ=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=TMK5S3J3yp21KjGV89TbUs0brtSTRHlixka9amBMdJZkMGCeWim3o4aYlVaTw5Sxj NhL76WPxvM3SfZPxlpAoemhKNcIfkkffWflotGYUuq4x8cL3M2PL75B/cU+v86W44P 1QzGMzdy0x1Br+EZ8vUVWjz/7yKJz706EWMOqW7U= From: Jacopo Mondi Date: Fri, 05 Dec 2025 15:52:13 +0100 Subject: [PATCH DNI 7/7] Please the CI loop MIME-Version: 1.0 Message-Id: <20251205-mali-cru-v1-7-d81bb5ffe73a@ideasonboard.com> References: <20251205-mali-cru-v1-0-d81bb5ffe73a@ideasonboard.com> In-Reply-To: <20251205-mali-cru-v1-0-d81bb5ffe73a@ideasonboard.com> To: Daniel Scally , libcamera-devel@lists.libcamera.org Cc: Jacopo Mondi X-Mailer: b4 0.14.2 X-Developer-Signature: v=1; a=openpgp-sha256; l=1104; i=jacopo.mondi@ideasonboard.com; h=from:subject:message-id; bh=d4+hyZoEoJcgjx/USn5RV1JkaAAFj++EIQfUN+n8wRQ=; b=owEBbQKS/ZANAwAKAXI0Bo8WoVY8AcsmYgBpMvGtZeqUBW73q92XhaHi9TBwMRouf1iF/NeZQ vpczeyK632JAjMEAAEKAB0WIQS1xD1IgJogio9YOMByNAaPFqFWPAUCaTLxrQAKCRByNAaPFqFW PDhtD/9UNMtweO7o5xt1l6Lfd1VXg06F+4Q/RgzAUZE0Mh8CALLzQposyHlQSwo6omFUYGI5Ts+ dUWb+A0hibrQKTksaeU/2tIpt/mXVjsk/p3GEmSuDFXdJfyCrd87xMxY/j4n590nV6D/S7zUK1m u7/NKTa1GZ2T6m84+qT/h4n83L0FCMFZSsF67MwgSANIxz9FttLkUXxC7z0/zjjz1OOsxCbYRI4 +2/dsTw1lBWPgnMXuHe9tKO9CXRY2UI+Dz5f3J2pA2rugAuxFiiFKJjpQ8F4DxtJ7TJDTKdzgrg VFQlFelMOcuvnGfejgbaGtso0Ek7FZww9OIWF3JjfEmTVuJR8Ppg7ctwLSpcslLzRWNHWZbjend 6lOoVeHW6dOndkgW+axb5KybxWBt91+LJC/uSQRxr6FgAhas+P8x0TIlZ9LUKOqOdEvpO5J9UD/ ihTLwqcpz4V/uH19rsHjMewrq1pd3Ccy7awC3ciIyA7wnrpMicQE5PY3Jp6aE8/ae1ZyVAU3kf3 yQTHWSZw9w6l5adQtZ+3KMZtwQUPNyd42KJzlblK+Isl0KhRvxngQZN+8k7XW5YvNeqK9gJC7f2 jKrv2fcSeiG1MXee3QS1S6KHip5bHjuEUwedz98L68BHpZYkiCDi3FaFRnS+FDGd4iIYnig2CfP MfNH2pW1an3ZTaw== X-Developer-Key: i=jacopo.mondi@ideasonboard.com; a=openpgp; fpr=72392EDC88144A65C701EA9BA5826A2587AD026B X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" A non-sense commit just to make sure CI is run properly, as we can't specify dependencies on other series. Of course, not for inclusion. Signed-off-by: Jacopo Mondi --- src/libcamera/pipeline/mali-c55/rzg2l-cru.cpp | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/src/libcamera/pipeline/mali-c55/rzg2l-cru.cpp b/src/libcamera/pipeline/mali-c55/rzg2l-cru.cpp index ac540ba2e29d25191e87a6208d33094e4ef2b969..715fb5cdeae011855162beb97e3e2b39ce7fbe05 100644 --- a/src/libcamera/pipeline/mali-c55/rzg2l-cru.cpp +++ b/src/libcamera/pipeline/mali-c55/rzg2l-cru.cpp @@ -27,9 +27,9 @@ namespace libcamera { static const std::map bitDepthToFmt{ - { 10, V4L2PixelFormat(V4L2_PIX_FMT_RAW_CRU10) }, - { 12, V4L2PixelFormat(V4L2_PIX_FMT_RAW_CRU12) }, - { 14, V4L2PixelFormat(V4L2_PIX_FMT_RAW_CRU14) }, + { 10, V4L2PixelFormat(V4L2_PIX_FMT_SBGGR10) }, + { 12, V4L2PixelFormat(V4L2_PIX_FMT_SBGGR12) }, + { 14, V4L2PixelFormat(V4L2_PIX_FMT_SBGGR14) }, }; LOG_DEFINE_CATEGORY(RZG2LCRU)