From patchwork Fri Mar 13 11:33:42 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jacopo Mondi X-Patchwork-Id: 26284 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id D09ECBE086 for ; Fri, 13 Mar 2026 11:34:23 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 5D9F462681; Fri, 13 Mar 2026 12:34:23 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="GXz+a1bh"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id A6555622AE for ; Fri, 13 Mar 2026 12:34:20 +0100 (CET) Received: from [192.168.224.131] (unknown [37.159.92.229]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 47F91F52; Fri, 13 Mar 2026 12:33:10 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1773401591; bh=EaYAwOaNMgy9veLK7jWa8jDYsMPnnAIy+293/wcP5yk=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=GXz+a1bhyZoYOQiU9q6qGQTuHBQvrOVTxK5UVqkeaUWkN2Yh6soVsbOuE2ewejKkV UIIgCVuogXzgbXwQndmYccYZrS6vVaGjoRkaoDjS6DtOLblEgXhzfhjrW3RYUR9Wux qmTU4nXiURS8eGjqj+KoSUz+5G2Eq5X12oGtGiWI= From: Jacopo Mondi Date: Fri, 13 Mar 2026 12:33:42 +0100 Subject: [PATCH v4 1/7] libcamera: mali-c55: Add RZG2LCRU class MIME-Version: 1.0 Message-Id: <20260313-mali-cru-v4-1-c0d9bc8cd8fa@ideasonboard.com> References: <20260313-mali-cru-v4-0-c0d9bc8cd8fa@ideasonboard.com> In-Reply-To: <20260313-mali-cru-v4-0-c0d9bc8cd8fa@ideasonboard.com> To: Daniel Scally , libcamera-devel@lists.libcamera.org Cc: Jacopo Mondi X-Mailer: b4 0.14.3 X-Developer-Signature: v=1; a=openpgp-sha256; l=9931; i=jacopo.mondi@ideasonboard.com; h=from:subject:message-id; bh=ubisEqX/JdmGo8iOHOU9fEWbBd82jLtSOuScAx7m9A4=; b=owEBbQKS/ZANAwAKAXI0Bo8WoVY8AcsmYgBps/Y0he+Y+KwIXt1z87Sw+iYZpSg5pRE+kcN38 haViwWNSXOJAjMEAAEKAB0WIQS1xD1IgJogio9YOMByNAaPFqFWPAUCabP2NAAKCRByNAaPFqFW PAGUD/wLB3C+z2lfup4u+nmSRpBkry7aN+RTkhdbWv/kqnuz0BJpJV89bOHtC0k5YVQiMRZq/7/ vW2qBPCol140TTO5yIaRC7ayOO8TzjDg0h5BEOwHwAzIBV0paT7AOWMux745hyzxoIvqJ0Rp6e0 IAxsfae/zA6Akuwi909lnWU7O579j1M4B5mGQzUjAaFqxDenl38fecWAXH2ox/nIBlyiwfjYSeq ErhU2ESYfAPY1JJlb+3dtV90wbGGSUZl42VJ+v5JsNhlxIuI3QP009zrKfgOB+Jd432dxJDNGjV cbC4BwiJ0EnVmTBDdXL5cbISU4mZFSuLUMxArNv5k4nS43VBp4b5yzvTVda1bu65seF9z7bhoD5 gS0eb6T3fEALE51wN7KOOgxXAXnM965elxEY2zAzvZbVgh2TPCpvcJSXyTKEIt9zpiRomcUkyx3 Xj5spJFICIiHhaJT7pwg87Gbmg6anHlFlAvXOKjLNQwq4zkbJqREq7ucDFrcV+/RKWjnrgKDHDC pSt65PYfiZyuL1eH1cg4rTHZsnxIBuWANuOhyADwLQgursMjh5zf9xGMNCblSGWLn3/Ytj5AqgZ uX+MyXeDLsxYAdcuMeIYiwiE9VFV3tkS1Y/xbwJYgCNpDEatTGfYkz80OwpbotHhw/YB5hzuikF jglEHOKDPBOMAHA== X-Developer-Key: i=jacopo.mondi@ideasonboard.com; a=openpgp; fpr=72392EDC88144A65C701EA9BA5826A2587AD026B X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" From: Daniel Scally Add a class allowing us to control the RZ/G2L Camera Receiver Unit. The class is named after the media device it handles, which is called "rzg2l_cru" but the CRU is actually found in other SoCs than the RZ/G2L one, such as the RZ/V2H(P). The RZG2LCRU class models the CSI-2 receiver found on those SoCs and allows to add support for memory-to-memory operations on the RZ/V2H(P) SoC which integrates a Mali-C55 ISP and a specialized DMA engine named IVC that feeds images from memory, where the CRU has saved them. Signed-off-by: Daniel Scally Signed-off-by: Jacopo Mondi --- src/libcamera/pipeline/mali-c55/meson.build | 3 +- src/libcamera/pipeline/mali-c55/rzg2l-cru.cpp | 260 ++++++++++++++++++++++++++ src/libcamera/pipeline/mali-c55/rzg2l-cru.h | 73 ++++++++ 3 files changed, 335 insertions(+), 1 deletion(-) diff --git a/src/libcamera/pipeline/mali-c55/meson.build b/src/libcamera/pipeline/mali-c55/meson.build index eba8e5a39054..4e768242602e 100644 --- a/src/libcamera/pipeline/mali-c55/meson.build +++ b/src/libcamera/pipeline/mali-c55/meson.build @@ -1,5 +1,6 @@ # SPDX-License-Identifier: CC0-1.0 libcamera_internal_sources += files([ - 'mali-c55.cpp' + 'mali-c55.cpp', + 'rzg2l-cru.cpp', ]) diff --git a/src/libcamera/pipeline/mali-c55/rzg2l-cru.cpp b/src/libcamera/pipeline/mali-c55/rzg2l-cru.cpp new file mode 100644 index 000000000000..8f09d1c78058 --- /dev/null +++ b/src/libcamera/pipeline/mali-c55/rzg2l-cru.cpp @@ -0,0 +1,260 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2025, Ideas on Board Oy + * + * Pipine handler element for the Renesas RZ/G2L Camera Receiver Unit + */ + +#include "rzg2l-cru.h" + +#include +#include + +#include + +#include +#include +#include + +#include +#include + +#include "libcamera/internal/bayer_format.h" +#include "libcamera/internal/camera_sensor.h" +#include "libcamera/internal/framebuffer.h" +#include "libcamera/internal/media_device.h" +#include "libcamera/internal/request.h" + +namespace libcamera { + +static const std::map bitDepthToFmt{ + { 10, V4L2PixelFormat(V4L2_PIX_FMT_RAW_CRU10) }, + { 12, V4L2PixelFormat(V4L2_PIX_FMT_RAW_CRU12) }, + { 14, V4L2PixelFormat(V4L2_PIX_FMT_RAW_CRU14) }, +}; + +LOG_DEFINE_CATEGORY(RZG2LCRU) + +FrameBuffer *RZG2LCRU::queueBuffer(Request *request) +{ + FrameBuffer *buffer; + + if (availableBuffers_.empty()) { + LOG(RZG2LCRU, Debug) << "CRU buffer underrun"; + return nullptr; + } + + buffer = availableBuffers_.front(); + + int ret = output_->queueBuffer(buffer); + if (ret) { + LOG(RZG2LCRU, Error) << "Failed to queue buffer to CRU"; + return nullptr; + } + + availableBuffers_.pop(); + buffer->_d()->setRequest(request); + + return buffer; +} + +void RZG2LCRU::cruReturnBuffer(FrameBuffer *buffer) +{ + auto it = std::find_if(buffers_.begin(), buffers_.end(), + [&](const auto &b) { return b.get() == buffer; }); + ASSERT(it != buffers_.end()); + + availableBuffers_.push(buffer); +} + +int RZG2LCRU::start() +{ + int ret = output_->exportBuffers(kBufferCount, &buffers_); + if (ret < 0) + return ret; + + utils::scope_exit bufferGuard([&] { freeBuffers(); }); + + ret = output_->importBuffers(kBufferCount); + if (ret) + return ret; + + for (std::unique_ptr &buffer : buffers_) + availableBuffers_.push(buffer.get()); + + ret = output_->streamOn(); + if (ret) + return ret; + + bufferGuard.release(); + + return 0; +} + +int RZG2LCRU::stop() +{ + int ret; + + ret = output_->streamOff(); + + freeBuffers(); + + return ret; +} + +void RZG2LCRU::freeBuffers() +{ + availableBuffers_ = {}; + buffers_.clear(); + + if (output_->releaseBuffers()) + LOG(RZG2LCRU, Error) << "Failed to release CRU buffers"; +} + +int RZG2LCRU::configure(V4L2SubdeviceFormat *subdevFormat, V4L2DeviceFormat *inputFormat) +{ + /* + * Set format on the sensor and propagate it up to the CRU video + * device. + */ + + int ret = sensor_->setFormat(subdevFormat); + if (ret) + return ret; + + ret = csi2_->setFormat(0, subdevFormat); + if (ret) + return ret; + + ret = csi2_->getFormat(1, subdevFormat); + if (ret) + return ret; + + ret = cru_->setFormat(0, subdevFormat); + if (ret) + return ret; + + ret = cru_->getFormat(1, subdevFormat); + if (ret) + return ret; + + /* + * The capture device needs to be set with a format that can be produced + * from the mbus code of the subdevFormat. The CRU and IVC use bayer + * order agnostic pixel formats, so all we need to do is find the right + * bitdepth and select the appropriate format. + */ + BayerFormat bayerFormat = BayerFormat::fromMbusCode(subdevFormat->code); + if (!bayerFormat.isValid()) + return -EINVAL; + + V4L2DeviceFormat captureFormat; + captureFormat.fourcc = bitDepthToFmt.at(bayerFormat.bitDepth); + captureFormat.size = subdevFormat->size; + + ret = output_->setFormat(&captureFormat); + if (ret) + return ret; + + /* + * We return the format that we set against the output device, as the + * same format will also need to be set against the Input Video Control + * Block device. + */ + *inputFormat = captureFormat; + + return 0; +} + +void RZG2LCRU::initCRUSizes() +{ + Size maxCSI2Size; + + /* + * Get the maximum supported size on the CSI-2 receiver. We need to + * query the kernel interface as the size limits differ from RZ/G2L + * (2800x4095) and RZ/V2H (4096x4096). + */ + V4L2Subdevice::Formats csi2Formats = csi2_->formats(0); + if (csi2Formats.empty()) + return; + + for (const auto &format : csi2Formats) { + for (const auto &range : format.second) { + if (range.max > maxCSI2Size) + maxCSI2Size = range.max; + } + } + + /* + * Enumerate the sensor supported resolutions and filter out the ones + * larger than the maximum supported CSI-2 receiver input size. + */ + V4L2Subdevice::Formats formats = sensor_->device()->formats(0); + if (formats.empty()) + return; + + for (const auto &format : formats) { + for (const auto &range : format.second) { + const Size &max = range.max; + + if (max.width > maxCSI2Size.width || + max.height > maxCSI2Size.height) + continue; + + csi2Sizes_.push_back(max); + } + } + + /* Sort in increasing order and remove duplicates. */ + std::sort(csi2Sizes_.begin(), csi2Sizes_.end()); + auto last = std::unique(csi2Sizes_.begin(), csi2Sizes_.end()); + csi2Sizes_.erase(last, csi2Sizes_.end()); + + csi2Resolution_ = csi2Sizes_.back(); +} + +int RZG2LCRU::init(const MediaDevice *media) +{ + static const std::regex cruRegex("cru-ip-[0-9a-f]{8}.cru[0-9]"); + static const std::regex csi2Regex("csi-[0-9a-f]{8}.csi2"); + + csi2_ = V4L2Subdevice::fromEntityName(media, csi2Regex); + if (!csi2_) + return -ENODEV; + + int ret = csi2_->open(); + if (ret) + return ret; + + const std::vector &pads = csi2_->entity()->pads(); + if (pads.empty()) + return -ENODEV; + + /* The receiver has a single sink pad at index 0 */ + MediaPad *sink = pads[0]; + const std::vector &links = sink->links(); + if (links.empty()) + return -ENODEV; + + MediaLink *link = links[0]; + sensor_ = CameraSensorFactoryBase::create(link->source()->entity()); + if (!sensor_) + return -ENODEV; + + cru_ = V4L2Subdevice::fromEntityName(media, cruRegex); + ret = cru_->open(); + if (ret) + return ret; + + output_ = V4L2VideoDevice::fromEntityName(media, "CRU output"); + ret = output_->open(); + if (ret) + return ret; + + initCRUSizes(); + + return 0; +} + +} /* namespace libcamera */ diff --git a/src/libcamera/pipeline/mali-c55/rzg2l-cru.h b/src/libcamera/pipeline/mali-c55/rzg2l-cru.h new file mode 100644 index 000000000000..1e3cdd879cc5 --- /dev/null +++ b/src/libcamera/pipeline/mali-c55/rzg2l-cru.h @@ -0,0 +1,73 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2025, Ideas on Board Oy + * + * Pipine handler element for the Renesas RZ/G2L Camera Receiver Unit + */ + +#pragma once + +#include +#include +#include + +#include + +#include "libcamera/internal/v4l2_subdevice.h" +#include "libcamera/internal/v4l2_videodevice.h" + +namespace libcamera { + +class CameraSensor; +class FrameBuffer; +class MediaDevice; +class Request; +class Size; + +class RZG2LCRU +{ +public: + static constexpr unsigned int kBufferCount = 4; + + RZG2LCRU() = default; + + const std::vector &sizes() const + { + return csi2Sizes_; + } + + int init(const MediaDevice *media); + const Size &resolution() const + { + return csi2Resolution_; + } + + CameraSensor *sensor() const { return sensor_.get(); } + V4L2Subdevice *csi2() const { return csi2_.get(); } + V4L2Subdevice *cru() const { return cru_.get(); } + + int configure(V4L2SubdeviceFormat *subdevFormat, V4L2DeviceFormat *inputFormat); + FrameBuffer *queueBuffer(Request *request); + void cruReturnBuffer(FrameBuffer *buffer); + V4L2VideoDevice *output() { return output_.get(); } + int start(); + int stop(); + +private: + void freeBuffers(); + + void initCRUSizes(); + + std::unique_ptr sensor_; + std::unique_ptr csi2_; + std::unique_ptr cru_; + std::unique_ptr output_; + + std::vector> buffers_; + std::queue availableBuffers_; + + std::vector csi2Sizes_; + Size csi2Resolution_; +}; + +} /* namespace libcamera */ From patchwork Fri Mar 13 11:33:43 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Jacopo Mondi X-Patchwork-Id: 26285 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 96ABBBE086 for ; Fri, 13 Mar 2026 11:34:25 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 3B3CC62657; Fri, 13 Mar 2026 12:34:25 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="RCMyNpcb"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 37D7A6263D for ; Fri, 13 Mar 2026 12:34:23 +0100 (CET) Received: from [192.168.224.131] (unknown [37.159.92.229]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 844FA103D; Fri, 13 Mar 2026 12:33:12 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1773401594; bh=zprjRPHJf6qrBYQuTrxKQDqXGcDi5fNentWk7weQpOI=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=RCMyNpcbj45DanT51KNshYzuEN7YFT/U6CXcEBoKw8Ai9paG3OcLx5bn+CSA/7vRd rSg4PRNKzGonz1FZoNAGCTMl+bzZCwhOwZrXzWwYmT3vIVaTNzG//zjLjPb3Gej4AF ++2Zhgu+XWHCihFXe7VL9/ftOqNGCjXCXXZDCrTM= From: Jacopo Mondi Date: Fri, 13 Mar 2026 12:33:43 +0100 Subject: [PATCH v4 2/7] libcamera: utils: Add overloaded visitor helpers MIME-Version: 1.0 Message-Id: <20260313-mali-cru-v4-2-c0d9bc8cd8fa@ideasonboard.com> References: <20260313-mali-cru-v4-0-c0d9bc8cd8fa@ideasonboard.com> In-Reply-To: <20260313-mali-cru-v4-0-c0d9bc8cd8fa@ideasonboard.com> To: Daniel Scally , libcamera-devel@lists.libcamera.org Cc: Jacopo Mondi X-Mailer: b4 0.14.3 X-Developer-Signature: v=1; a=openpgp-sha256; l=4117; i=jacopo.mondi@ideasonboard.com; h=from:subject:message-id; bh=zprjRPHJf6qrBYQuTrxKQDqXGcDi5fNentWk7weQpOI=; b=owEBbQKS/ZANAwAKAXI0Bo8WoVY8AcsmYgBps/Y06wo47K0QwTTk2FCH3z+v5bSgXx/ukxo3P RC64Bd9iCyJAjMEAAEKAB0WIQS1xD1IgJogio9YOMByNAaPFqFWPAUCabP2NAAKCRByNAaPFqFW PLvqEADHmdaMo+P2tCCqeM1fK7GDIcQhWzLFbxD1wUPl3bhGofXGgXqwJB1gTv+wd6SZup3Ovf3 8quBiQiE3PefeOHPAlD2+fy9xtDIUFhmav4UU6vuLSI7slMwlK5Pyj3f0kvWv7b1e5irQ3IGsyB +lkefw47H7HPESviBBcTvMyiuWCYhRgZYgLoi+SDxvS9oKCjvFV53WIAc5fc90yRnU4/NFurC+W UrJkUWmAmRc25qF8hEnoGHUrZLE00+vsJwNWoVEfA+/ZEZsztt6VatD1uThD6ers3nfrhlAvF8H cMYeJTftO5yHdUKfytBmW9W9MRk9XMa2FARj6J3vB9icxiWloZg36Iy7obe6wiNHcEMX6JFt5DA hoTnBP9sqy/6c7QfUWg8b0LW6R32hSp8ovFHd5k+RzzCgz+RVShLTGzOo5KmJ2yg5eV+a57ev5q Na17jz2EIIs1zjN91JKvQ6p8MoqN0ZfKbiMlqUNUSJ6LGGyhacBDpLv/hms9ZeqresVyBa6zdfD D3R999C1G8OICXd/mABL1SpOQtpB54PbqFBi5KbcrGL+T4m07TECINtuXw6CWjckNj2bhlPh9xY XzROWCH6xzDeLQPcBXDGuthLJuB/EF9ARYQmyKgnZ/6zLRn6yzkyifbFXJ86SUMIJoJE/8ielYN jDKSPGmp0/s9hqw== X-Developer-Key: i=jacopo.mondi@ideasonboard.com; a=openpgp; fpr=72392EDC88144A65C701EA9BA5826A2587AD026B X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" std::visit() allows quite elegant type-matching implementation of the visitor pattern. The overaloded type helpers allow to define a hierarchy of overloaded operator() implementations which can be used by std::visit(). Currently only the Virtual pipeline handler uses this type-matching implementation of std::visit. To prepare to add another user in the Mali C55 pipeline handler move the 'overloaded' helper types to libcamera::utils for easier re-use. Signed-off-by: Jacopo Mondi Reviewed-by: Barnabás Pőcze --- include/libcamera/base/utils.h | 7 ++++++ src/libcamera/base/utils.cpp | 35 ++++++++++++++++++++++++++++++ src/libcamera/pipeline/virtual/virtual.cpp | 10 ++------- 3 files changed, 44 insertions(+), 8 deletions(-) diff --git a/include/libcamera/base/utils.h b/include/libcamera/base/utils.h index 7083b7ce9ce9..62c7f89c25a1 100644 --- a/include/libcamera/base/utils.h +++ b/include/libcamera/base/utils.h @@ -37,6 +37,13 @@ namespace libcamera { namespace utils { +template +struct overloaded : Ts... { + using Ts::operator()...; +}; +template +overloaded(Ts...) -> overloaded; + const char *basename(const char *path); char *secure_getenv(const char *name); diff --git a/src/libcamera/base/utils.cpp b/src/libcamera/base/utils.cpp index 42a516097be2..a18ca82e8d07 100644 --- a/src/libcamera/base/utils.cpp +++ b/src/libcamera/base/utils.cpp @@ -23,6 +23,41 @@ namespace libcamera { namespace utils { +/** + * \struct overloaded + * \brief Helper type for type-matching std::visit implementations + * \tparam Ts... Template arguments pack of visitors + * + * Expand the template argument pack \a Ts... to provide overloaded + * implementations of \a operator(). + */ + +/** + * \var overloaded(Ts...) -> overloaded + * \brief Overloaded visitor class for type-matching std::visit implementations + * \tparam Ts... Template arguments pack of visitor functions + * + * std::visit allows quite elegant type-matching implementation of the visitor + * pattern. An example is provided by the STL documentation in the form of: + * + * \code{.cpp} + * template struct overloaded : Ts... { using Ts::operator()...; }; + * template overloaded(Ts...) -> overloaded; + * + * using var_t = std::variant; + * std::vector vec = {10, 15l, 1.5, "hello"}; + * + * for (auto& v: vec) { + * std::visit(overloaded { + * [](auto arg) { std::cout << arg << ' '; }, + * [](double arg) { std::cout << std::fixed << arg << ' '; }, + * [](const std::string& arg) { std::cout << std::quoted(arg) << ' '; }, + * }, v); + * \endcode + * + * Use this helper to implement type-matching visitors using std::visit(). + */ + /** * \brief Strip the directory prefix from the path * \param[in] path The path to process diff --git a/src/libcamera/pipeline/virtual/virtual.cpp b/src/libcamera/pipeline/virtual/virtual.cpp index efd800ebe3d6..e8ef7e524ccf 100644 --- a/src/libcamera/pipeline/virtual/virtual.cpp +++ b/src/libcamera/pipeline/virtual/virtual.cpp @@ -23,6 +23,7 @@ #include #include +#include #include #include @@ -57,13 +58,6 @@ uint64_t currentTimestamp() } /* namespace */ -template -struct overloaded : Ts... { - using Ts::operator()...; -}; -template -overloaded(Ts...) -> overloaded; - class VirtualCameraConfiguration : public CameraConfiguration { public: @@ -428,7 +422,7 @@ bool PipelineHandlerVirtual::initFrameGenerator(Camera *camera) { auto data = cameraData(camera); auto &frame = data->config_.frame; - std::visit(overloaded{ + std::visit(utils::overloaded{ [&](TestPattern &testPattern) { for (auto &streamConfig : data->streamConfigs_) { if (testPattern == TestPattern::DiagonalLines) From patchwork Fri Mar 13 11:33:44 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jacopo Mondi X-Patchwork-Id: 26286 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 396ABBE086 for ; Fri, 13 Mar 2026 11:34:28 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id E3EEC626D0; Fri, 13 Mar 2026 12:34:27 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="kO4cnwWI"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 3F9F56267D for ; Fri, 13 Mar 2026 12:34:26 +0100 (CET) Received: from [192.168.224.131] (unknown [37.159.92.229]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 96AF91356; Fri, 13 Mar 2026 12:33:14 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1773401597; bh=GwgINAI7Dhe7Z22i9TeaDxYUq0FMK+wvgqU0KEsWxAQ=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=kO4cnwWICs2zWNfsEn2FB83T1Juokfj1foLtMtBg/iyv2Flod5zE8v4Ug7rhPV6d4 YG6C3WTIbkw9tFfP/FBQ9aKSm3SbSJArjpyV6qocT6ZjADRITc0OOMU+mpRxc9ZLUg vPWkGK1UYrXahDLKfQfz7ysICeU5s1iFr3W1U/NQ= From: Jacopo Mondi Date: Fri, 13 Mar 2026 12:33:44 +0100 Subject: [PATCH v4 3/7] libcamera: mali-c55: Split TPG and Inline camera handling MIME-Version: 1.0 Message-Id: <20260313-mali-cru-v4-3-c0d9bc8cd8fa@ideasonboard.com> References: <20260313-mali-cru-v4-0-c0d9bc8cd8fa@ideasonboard.com> In-Reply-To: <20260313-mali-cru-v4-0-c0d9bc8cd8fa@ideasonboard.com> To: Daniel Scally , libcamera-devel@lists.libcamera.org Cc: Jacopo Mondi X-Mailer: b4 0.14.3 X-Developer-Signature: v=1; a=openpgp-sha256; l=16323; i=jacopo.mondi@ideasonboard.com; h=from:subject:message-id; bh=GwgINAI7Dhe7Z22i9TeaDxYUq0FMK+wvgqU0KEsWxAQ=; b=owEBbQKS/ZANAwAKAXI0Bo8WoVY8AcsmYgBps/Y0iIVJrCVFp0YDPTtD7QEAhjrfnUa7m85sV yzHq1Ro0NuJAjMEAAEKAB0WIQS1xD1IgJogio9YOMByNAaPFqFWPAUCabP2NAAKCRByNAaPFqFW PDgqD/98ak76lb0E+GmC5+4cXMVGszWzRCk5AqqnjEO/BYhDfidfWLAV3kO/SHAD+DYpCzTe85L i/3TOxo7ntRAHi2Am61ILJiDoCCY70a10DxJdPF7I6oH5RA/g9skw3zT3r9rdO2zNA89jNJ1faf GpbeXxwJbIJ4JE3fzaHWkljqnJ+XS99pQEBXMzOcETFu4utQZc/lti/uWsWhicfXLMTWI8hD8By +2L7GiAB2G13xoInHl7z0bNH4l48ee0H3GzBQt0hK8Ub7LTFbbmu652UUyvSl0kUkh/gSsXlTtS cDjS6UuS1rdr/L4sKHE3/p01gR5FMVFSPX95kgEruF+PXIUHc5MpnVM3RHcTFsh+Lzs+vCirKdh 4taGcf66PAyRYcVOEM3mt6C6JlZD5HwuOwef8qepeYOZM3rQE6xux4bVUeSzCn2LBquLFU8Tiih R8+PBUjFYS+NvotT8Z+p5hU56igtUTeiVTUPbqsmT2Erj/bxsdcD4zfGk3a07eTOcw/yU4Qr5a4 hZxC49v8Lk5XVixt/Umm1o1m5MqId7mZ9Fvqqw6hcxwtaJuxtYugeHS8TzH36qsOaAB/vUJ8P3f lHgapjvHYoIXizVJRPAb/whcJDlwhxLuVTIhuHi/cQC8o7jS3YbXVWyTLtPNPRQdEZTBNeJ9vm4 psjVXVtVqj/x9Dg== X-Developer-Key: i=jacopo.mondi@ideasonboard.com; a=openpgp; fpr=72392EDC88144A65C701EA9BA5826A2587AD026B X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" In order to prepare to support memory input cameras, split the handling of the TPG and Inline camera cases. The Mali C55 pipeline handler uses the entity and subdevice stored in the CameraData to support both the handling of the TPG and of the Inline (CSI-2 + sensor) use cases. Adding support for memory cameras by using the CRU unit would add yet-another special case making the code harder to follow and more prone to errors. Split the handling of the TPG and Inline cameras by introducing an std::variant<> variable and to deflect the functions called on the camera data to the correct type by using overloaded std::visit<>(). Signed-off-by: Jacopo Mondi --- src/libcamera/pipeline/mali-c55/mali-c55.cpp | 276 ++++++++++++++++----------- 1 file changed, 167 insertions(+), 109 deletions(-) diff --git a/src/libcamera/pipeline/mali-c55/mali-c55.cpp b/src/libcamera/pipeline/mali-c55/mali-c55.cpp index c209b0b070b1..6581d13bbc52 100644 --- a/src/libcamera/pipeline/mali-c55/mali-c55.cpp +++ b/src/libcamera/pipeline/mali-c55/mali-c55.cpp @@ -11,12 +11,14 @@ #include #include #include +#include #include #include #include #include +#include #include #include @@ -92,17 +94,90 @@ struct MaliC55FrameInfo { class MaliC55CameraData : public Camera::Private { public: - MaliC55CameraData(PipelineHandler *pipe, MediaEntity *entity) - : Camera::Private(pipe), entity_(entity) + struct Tpg { + std::vector sizes(unsigned int mbusCode) const; + + Size resolution_; + std::unique_ptr sd_; + }; + + struct Inline { + std::unique_ptr csi2_; + std::unique_ptr sensor_; + }; + using CameraType = std::variant; + + MaliC55CameraData(PipelineHandler *pipe) + : Camera::Private(pipe) { } - int init(); int loadIPA(); - /* Deflect these functionalities to either TPG or CameraSensor. */ - std::vector sizes(unsigned int mbusCode) const; - Size resolution() const; + int initTpg(MediaEntity *entity); + int initInline(MediaEntity *entity); + + std::vector sizes(unsigned int mbusCode) const + { + return std::visit(utils::overloaded{ + [&](const Tpg &tpg) -> std::vector { + return tpg.sizes(mbusCode); + }, + [&](const Inline &in) -> std::vector { + return in.sensor_->sizes(mbusCode); + } }, + input_); + } + + V4L2Subdevice *subdev() const + { + return std::visit(utils::overloaded{ + [&](const Tpg &tpg) -> V4L2Subdevice * { + return tpg.sd_.get(); + }, + [&](const Inline &in) -> V4L2Subdevice * { + return in.sensor_->device(); + } }, + input_); + } + + CameraSensor *sensor() const + { + return std::visit(utils::overloaded{ + [&](auto &) -> CameraSensor * { + ASSERT(false); + return nullptr; + }, + [&](const Inline &in) -> CameraSensor * { + return in.sensor_.get(); + } }, + input_); + } + + V4L2Subdevice *csi2() const + { + return std::visit(utils::overloaded{ + [&](auto &) -> V4L2Subdevice * { + ASSERT(false); + return nullptr; + }, + [&](const Inline &in) -> V4L2Subdevice * { + return in.csi2_.get(); + } }, + input_); + } + + Size resolution() const + { + return std::visit(utils::overloaded{ + [&](const Tpg &tpg) -> Size { + return tpg.resolution_; + }, + [&](const Inline &in) -> Size { + return in.sensor_->resolution(); + } }, + input_); + } int pixfmtToMbusCode(const PixelFormat &pixFmt) const; const PixelFormat &bestRawFormat() const; @@ -112,11 +187,6 @@ public: PixelFormat adjustRawFormat(const PixelFormat &pixFmt) const; Size adjustRawSizes(const PixelFormat &pixFmt, const Size &rawSize) const; - std::unique_ptr sensor_; - - MediaEntity *entity_; - std::unique_ptr csi_; - std::unique_ptr sd_; Stream frStream_; Stream dsStream_; @@ -126,58 +196,28 @@ public: std::unique_ptr delayedCtrls_; + CameraType input_; + private: - void initTPGData(); void setSensorControls(const ControlList &sensorControls); - std::string id_; - Size tpgResolution_; }; -int MaliC55CameraData::init() +int MaliC55CameraData::initTpg(MediaEntity *entity) { - int ret; + Tpg tpg; - sd_ = std::make_unique(entity_); - ret = sd_->open(); + tpg.sd_ = std::make_unique(entity); + int ret = tpg.sd_->open(); if (ret) { - LOG(MaliC55, Error) << "Failed to open sensor subdevice"; + LOG(MaliC55, Error) << "Failed to open TPG subdevice"; return ret; } - /* If this camera is created from TPG, we return here. */ - if (entity_->name() == "mali-c55 tpg") { - initTPGData(); - return 0; - } - - /* - * Register a CameraSensor if we connect to a sensor and create - * an entity for the connected CSI-2 receiver. - */ - sensor_ = CameraSensorFactoryBase::create(entity_); - if (!sensor_) - return -ENODEV; - - const MediaPad *sourcePad = entity_->getPadByIndex(0); - MediaEntity *csiEntity = sourcePad->links()[0]->sink()->entity(); - - csi_ = std::make_unique(csiEntity); - ret = csi_->open(); - if (ret) { - LOG(MaliC55, Error) << "Failed to open CSI-2 subdevice"; - return ret; - } - - return 0; -} - -void MaliC55CameraData::initTPGData() -{ /* Replicate the CameraSensor implementation for TPG. */ - V4L2Subdevice::Formats formats = sd_->formats(0); + V4L2Subdevice::Formats formats = tpg.sd_->formats(0); if (formats.empty()) - return; + return -EINVAL; std::vector tpgSizes; @@ -187,19 +227,39 @@ void MaliC55CameraData::initTPGData() [](const SizeRange &range) { return range.max; }); } - tpgResolution_ = tpgSizes.back(); + tpg.resolution_ = tpgSizes.back(); + + input_.emplace(std::move(tpg)); + + return 0; } -void MaliC55CameraData::setSensorControls(const ControlList &sensorControls) +int MaliC55CameraData::initInline(MediaEntity *sensor) { - delayedCtrls_->push(sensorControls); + Inline in; + + /* Register a CameraSensor and create an entity for the CSI-2 receiver. */ + in.sensor_ = CameraSensorFactoryBase::create(sensor); + if (!in.sensor_) + return -EINVAL; + + const MediaPad *sourcePad = sensor->getPadByIndex(0); + MediaEntity *csiEntity = sourcePad->links()[0]->sink()->entity(); + + in.csi2_ = std::make_unique(csiEntity); + int ret = in.csi2_->open(); + if (ret) { + LOG(MaliC55, Error) << "Failed to open CSI-2 subdevice"; + return ret; + } + + input_.emplace(std::move(in)); + + return 0; } -std::vector MaliC55CameraData::sizes(unsigned int mbusCode) const +std::vector MaliC55CameraData::Tpg::sizes(unsigned int mbusCode) const { - if (sensor_) - return sensor_->sizes(mbusCode); - V4L2Subdevice::Formats formats = sd_->formats(0); if (formats.empty()) return {}; @@ -218,12 +278,9 @@ std::vector MaliC55CameraData::sizes(unsigned int mbusCode) const return sizes; } -Size MaliC55CameraData::resolution() const +void MaliC55CameraData::setSensorControls(const ControlList &sensorControls) { - if (sensor_) - return sensor_->resolution(); - - return tpgResolution_; + delayedCtrls_->push(sensorControls); } /* @@ -242,7 +299,7 @@ int MaliC55CameraData::pixfmtToMbusCode(const PixelFormat &pixFmt) const if (!bayerFormat.isValid()) return -EINVAL; - V4L2Subdevice::Formats formats = sd_->formats(0); + V4L2Subdevice::Formats formats = subdev()->formats(0); unsigned int sensorMbusCode = 0; unsigned int bitDepth = 0; @@ -280,7 +337,7 @@ const PixelFormat &MaliC55CameraData::bestRawFormat() const { static const PixelFormat invalidPixFmt = {}; - for (const auto &fmt : sd_->formats(0)) { + for (const auto &fmt : subdev()->formats(0)) { BayerFormat sensorBayer = BayerFormat::fromMbusCode(fmt.first); if (!sensorBayer.isValid()) @@ -302,11 +359,11 @@ const PixelFormat &MaliC55CameraData::bestRawFormat() const void MaliC55CameraData::updateControls(const ControlInfoMap &ipaControls) { - if (!sensor_) + if (std::holds_alternative(input_)) return; IPACameraSensorInfo sensorInfo; - int ret = sensor_->sensorInfo(&sensorInfo); + int ret = sensor()->sensorInfo(&sensorInfo); if (ret) { LOG(MaliC55, Error) << "Failed to retrieve sensor info"; return; @@ -379,7 +436,7 @@ int MaliC55CameraData::loadIPA() int ret; /* Do not initialize IPA for TPG. */ - if (!sensor_) + if (std::holds_alternative(input_)) return 0; ipa_ = IPAManager::createIPA(pipe(), 1, 1); @@ -388,20 +445,20 @@ int MaliC55CameraData::loadIPA() ipa_->setSensorControls.connect(this, &MaliC55CameraData::setSensorControls); - std::string ipaTuningFile = ipa_->configurationFile(sensor_->model() + ".yaml", + std::string ipaTuningFile = ipa_->configurationFile(sensor()->model() + ".yaml", "uncalibrated.yaml"); /* We need to inform the IPA of the sensor configuration */ ipa::mali_c55::IPAConfigInfo ipaConfig{}; - ret = sensor_->sensorInfo(&ipaConfig.sensorInfo); + ret = sensor()->sensorInfo(&ipaConfig.sensorInfo); if (ret) return ret; - ipaConfig.sensorControls = sensor_->controls(); + ipaConfig.sensorControls = sensor()->controls(); ControlInfoMap ipaControls; - ret = ipa_->init({ ipaTuningFile, sensor_->model() }, ipaConfig, + ret = ipa_->init({ ipaTuningFile, sensor()->model() }, ipaConfig, &ipaControls); if (ret) { LOG(MaliC55, Error) << "Failed to initialise the Mali-C55 IPA"; @@ -444,13 +501,13 @@ CameraConfiguration::Status MaliC55CameraConfiguration::validate() * The TPG doesn't support flips, so we only need to calculate a * transform if we have a sensor. */ - if (data_->sensor_) { + if (std::holds_alternative(data_->input_)) { + combinedTransform_ = Transform::Rot0; + } else { Orientation requestedOrientation = orientation; - combinedTransform_ = data_->sensor_->computeTransform(&orientation); + combinedTransform_ = data_->sensor()->computeTransform(&orientation); if (orientation != requestedOrientation) status = Adjusted; - } else { - combinedTransform_ = Transform::Rot0; } /* Only 2 streams available. */ @@ -927,11 +984,12 @@ int PipelineHandlerMaliC55::configure(Camera *camera, /* Link the graph depending if we are operating the TPG or a sensor. */ MaliC55CameraData *data = cameraData(camera); - if (data->csi_) { - const MediaEntity *csiEntity = data->csi_->entity(); - ret = csiEntity->getPadByIndex(1)->links()[0]->setEnabled(true); + if (std::holds_alternative(data->input_)) { + const MediaEntity *tpgEntity = data->subdev()->entity(); + ret = tpgEntity->getPadByIndex(0)->links()[0]->setEnabled(true); } else { - ret = data->entity_->getPadByIndex(0)->links()[0]->setEnabled(true); + const MediaEntity *csi2Entity = data->csi2()->entity(); + ret = csi2Entity->getPadByIndex(1)->links()[0]->setEnabled(true); } if (ret) return ret; @@ -939,26 +997,24 @@ int PipelineHandlerMaliC55::configure(Camera *camera, MaliC55CameraConfiguration *maliConfig = static_cast(config); V4L2SubdeviceFormat subdevFormat = maliConfig->sensorFormat_; - ret = data->sd_->getFormat(0, &subdevFormat); - if (ret) - return ret; - if (data->sensor_) { - ret = data->sensor_->setFormat(&subdevFormat, - maliConfig->combinedTransform()); + /* Apply format to the origin of the pipeline and propagate it. */ + if (std::holds_alternative(data->input_)) { + ret = data->subdev()->setFormat(0, &subdevFormat); + } else { + ret = data->sensor()->setFormat(&subdevFormat, + maliConfig->combinedTransform()); if (ret) return ret; - } - if (data->csi_) { - ret = data->csi_->setFormat(0, &subdevFormat); + ret = data->csi2()->setFormat(0, &subdevFormat); if (ret) return ret; - ret = data->csi_->getFormat(1, &subdevFormat); - if (ret) - return ret; + ret = data->csi2()->getFormat(1, &subdevFormat); } + if (ret) + return ret; V4L2DeviceFormat statsFormat; ret = stats_->getFormat(&statsFormat); @@ -973,8 +1029,6 @@ int PipelineHandlerMaliC55::configure(Camera *camera, /* * Propagate the format to the ISP sink pad and configure the input * crop rectangle (no crop at the moment). - * - * \todo Configure the CSI-2 receiver. */ ret = isp_->setFormat(0, &subdevFormat); if (ret) @@ -1058,18 +1112,18 @@ int PipelineHandlerMaliC55::configure(Camera *camera, /* We need to inform the IPA of the sensor configuration */ ipa::mali_c55::IPAConfigInfo ipaConfig{}; - ret = data->sensor_->sensorInfo(&ipaConfig.sensorInfo); + ret = data->sensor()->sensorInfo(&ipaConfig.sensorInfo); if (ret) return ret; - ipaConfig.sensorControls = data->sensor_->controls(); + ipaConfig.sensorControls = data->sensor()->controls(); /* * And we also need to tell the IPA the bayerOrder of the data (as * affected by any flips that we've configured) */ const Transform &combinedTransform = maliConfig->combinedTransform(); - BayerFormat::Order bayerOrder = data->sensor_->bayerOrder(combinedTransform); + BayerFormat::Order bayerOrder = data->sensor()->bayerOrder(combinedTransform); ControlInfoMap ipaControls; ret = data->ipa_->configure(ipaConfig, utils::to_underlying(bayerOrder), @@ -1283,7 +1337,7 @@ void PipelineHandlerMaliC55::applyScalerCrop(Camera *camera, if (!scalerCrop) return; - if (!data->sensor_) { + if (std::holds_alternative(data->input_)) { LOG(MaliC55, Error) << "ScalerCrop not supported for TPG"; return; } @@ -1291,7 +1345,7 @@ void PipelineHandlerMaliC55::applyScalerCrop(Camera *camera, Rectangle nativeCrop = *scalerCrop; IPACameraSensorInfo sensorInfo; - int ret = data->sensor_->sensorInfo(&sensorInfo); + int ret = data->sensor()->sensorInfo(&sensorInfo); if (ret) { LOG(MaliC55, Error) << "Failed to retrieve sensor info"; return; @@ -1573,10 +1627,11 @@ bool PipelineHandlerMaliC55::registerTPGCamera(MediaLink *link) } std::unique_ptr data = - std::make_unique(this, link->source()->entity()); + std::make_unique(this); - if (data->init()) - return false; + int ret = data->initTpg(link->source()->entity()); + if (ret) + return ret; return registerMaliCamera(std::move(data), name); } @@ -1600,21 +1655,24 @@ bool PipelineHandlerMaliC55::registerSensorCamera(MediaLink *ispLink) continue; std::unique_ptr data = - std::make_unique(this, sensor); - if (data->init()) - return false; + std::make_unique(this); + + int ret = data->initInline(sensor); + if (ret) + return ret; - data->properties_ = data->sensor_->properties(); + data->properties_ = data->sensor()->properties(); - const CameraSensorProperties::SensorDelays &delays = data->sensor_->sensorDelays(); + const CameraSensorProperties::SensorDelays &delays = + data->sensor()->sensorDelays(); std::unordered_map params = { { V4L2_CID_ANALOGUE_GAIN, { delays.gainDelay, false } }, { V4L2_CID_EXPOSURE, { delays.exposureDelay, false } }, }; - data->delayedCtrls_ = - std::make_unique(data->sensor_->device(), - params); + V4L2Subdevice *sensorSubdev = data->sensor()->device(); + data->delayedCtrls_ = std::make_unique(sensorSubdev, + params); isp_->frameStart.connect(data->delayedCtrls_.get(), &DelayedControls::applyControls); From patchwork Fri Mar 13 11:33:45 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jacopo Mondi X-Patchwork-Id: 26287 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id CB350BE086 for ; Fri, 13 Mar 2026 11:34:30 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 7F7166268F; Fri, 13 Mar 2026 12:34:30 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="alIezC1o"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 5CC0762695 for ; Fri, 13 Mar 2026 12:34:28 +0100 (CET) Received: from [192.168.224.131] (unknown [37.159.92.229]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id B96801783; Fri, 13 Mar 2026 12:33:17 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1773401599; bh=4IEqM7B3Z4FbQ2TizCoNOEr5YoUJQyY+Zj02Cdcr2LU=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=alIezC1oflvEB+qaI60YDxPVd8mLYGOHPJqnSB2sgJ+Y1bk0i6BaCV6PdY5dDcMiP x8+INS+enE1kLfVQphhV6wJE6vWZi8DyVhkR8L4t46Jp9Y4OpsnSyN7zHTWth0Syz1 H5dPhxnt15SKn+JMLYBoyVqzlFh5QBJ4pMURaTQI= From: Jacopo Mondi Date: Fri, 13 Mar 2026 12:33:45 +0100 Subject: [PATCH v4 4/7] libcamera: mali-c55: Register memory input camera MIME-Version: 1.0 Message-Id: <20260313-mali-cru-v4-4-c0d9bc8cd8fa@ideasonboard.com> References: <20260313-mali-cru-v4-0-c0d9bc8cd8fa@ideasonboard.com> In-Reply-To: <20260313-mali-cru-v4-0-c0d9bc8cd8fa@ideasonboard.com> To: Daniel Scally , libcamera-devel@lists.libcamera.org Cc: Jacopo Mondi X-Mailer: b4 0.14.3 X-Developer-Signature: v=1; a=openpgp-sha256; l=9060; i=jacopo.mondi@ideasonboard.com; h=from:subject:message-id; bh=v+nSg38rCLd41D6bC4lR0TADFlsu+yXwxFH+Di1xrDU=; b=owEBbQKS/ZANAwAKAXI0Bo8WoVY8AcsmYgBps/Y0dMX2GLRQAV6osQ9hcx6np1M1GdW8K+bgw 1/FN1/0LoSJAjMEAAEKAB0WIQS1xD1IgJogio9YOMByNAaPFqFWPAUCabP2NAAKCRByNAaPFqFW PGS3D/9q21tk2HREJqtSc209wipUMw2s6x68VrDtYNwK1CdBw/YoBKxWjfluG1ImJir5/XraNlV bTbTvX8iHV7V3hN2ceq3HA+tPY+Ifyk4G7P116K6iw2r0Osso0SdZIkiL4RVmcq3sWuwptIcMbu 8WcbhLJcNqKGUS1/za2KXZvWCLcUvA7nvvigSFdQAwEVldP2RltCzX2FJeSK3kcuuERupxIgvIx HMNcPbaSJo1i8Un1n1Kx0oFLMQRWwng50ycSHORGhlTj35HtH5e7bJ43Y0yY17ZeQ7AfD+S46Rf JzfQekpClh+f+wCX0apyloGAeaQ8C+JoyfygrNOZ6VKwLw8SD5q79ym8SkqKN3UPwONhXwZ7O/g Ex0sW9P6tvdV87EL7FgOqxNHfoOJ/7Ut7lfgk7CuAUkw2W3T4MrH90F74OmpekKX8J2kiJaxao5 dUodwRtAl6Yoe6c22/ncaxOa1uTBNYNYVAFjbJURE95v5/UxlW107yVfEZ3dcvXwRs2JmW6/Pp9 1bFDjg+PZzbuQcNPVcDMfAOEmoElIGskaYcvPV/mCZV6rI8JEMg80pTNrP6xN2vJYnfuRmNBSKP aEeJ+qo61aFnC5ImhI84dHgwORrtkJevXrYXUB/VNTML9tN7YW8oEylDQ+p0VrvD7IA1Bcnr0MW BVzogerugSA4JDw== X-Developer-Key: i=jacopo.mondi@ideasonboard.com; a=openpgp; fpr=72392EDC88144A65C701EA9BA5826A2587AD026B X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" From: Daniel Scally Add support to Mali C55 pipeline handler for memory-to-memory camera mode. The Mali-C55, as integrated in the Renesas RZ/V2H(P) SoC, is fed with images saved to memory by the CRU unit and read on behalf of the ISP by the IVC. Update the pipeline handler's match() function to search for the CRU media entity necessary to register a memory input camera. Create and initialize a CameraData instance for the memory camera use case if a valid CRU is found and initialize the IVC components used to feed the ISP with images read from memory. Signed-off-by: Daniel Scally Signed-off-by: Jacopo Mondi --- src/libcamera/pipeline/mali-c55/mali-c55.cpp | 145 +++++++++++++++++++++++---- 1 file changed, 128 insertions(+), 17 deletions(-) diff --git a/src/libcamera/pipeline/mali-c55/mali-c55.cpp b/src/libcamera/pipeline/mali-c55/mali-c55.cpp index 6581d13bbc52..3c8848d28707 100644 --- a/src/libcamera/pipeline/mali-c55/mali-c55.cpp +++ b/src/libcamera/pipeline/mali-c55/mali-c55.cpp @@ -44,6 +44,8 @@ #include "libcamera/internal/v4l2_subdevice.h" #include "libcamera/internal/v4l2_videodevice.h" +#include "rzg2l-cru.h" + namespace { bool isFormatRaw(const libcamera::PixelFormat &pixFmt) @@ -105,7 +107,12 @@ public: std::unique_ptr csi2_; std::unique_ptr sensor_; }; - using CameraType = std::variant; + + struct Memory { + std::unique_ptr cru_; + }; + + using CameraType = std::variant; MaliC55CameraData(PipelineHandler *pipe) : Camera::Private(pipe) @@ -116,6 +123,7 @@ public: int initTpg(MediaEntity *entity); int initInline(MediaEntity *entity); + int initMemory(MediaDevice *cruMedia); std::vector sizes(unsigned int mbusCode) const { @@ -125,6 +133,9 @@ public: }, [&](const Inline &in) -> std::vector { return in.sensor_->sizes(mbusCode); + }, + [&](const Memory &mem) -> std::vector { + return mem.cru_->sizes(); } }, input_); } @@ -137,6 +148,9 @@ public: }, [&](const Inline &in) -> V4L2Subdevice * { return in.sensor_->device(); + }, + [&](const Memory &mem) -> V4L2Subdevice * { + return mem.cru_->sensor()->device(); } }, input_); } @@ -150,6 +164,9 @@ public: }, [&](const Inline &in) -> CameraSensor * { return in.sensor_.get(); + }, + [&](const Memory &mem) -> CameraSensor * { + return mem.cru_->sensor(); } }, input_); } @@ -163,6 +180,9 @@ public: }, [&](const Inline &in) -> V4L2Subdevice * { return in.csi2_.get(); + }, + [&](const Memory &mem) -> V4L2Subdevice * { + return mem.cru_->csi2(); } }, input_); } @@ -175,6 +195,22 @@ public: }, [&](const Inline &in) -> Size { return in.sensor_->resolution(); + }, + [&](const Memory &mem) -> Size { + return mem.cru_->resolution(); + } }, + input_); + } + + RZG2LCRU *cru() const + { + return std::visit(utils::overloaded{ + [&](auto &) -> RZG2LCRU * { + ASSERT(false); + return nullptr; + }, + [&](const Memory &mem) -> RZG2LCRU * { + return mem.cru_.get(); } }, input_); } @@ -258,6 +294,21 @@ int MaliC55CameraData::initInline(MediaEntity *sensor) return 0; } +int MaliC55CameraData::initMemory(MediaDevice *cruMedia) +{ + Memory mem; + + mem.cru_ = std::make_unique(); + + int ret = mem.cru_->init(cruMedia); + if (ret) + return ret; + + input_.emplace(std::move(mem)); + + return 0; +} + std::vector MaliC55CameraData::Tpg::sizes(unsigned int mbusCode) const { V4L2Subdevice::Formats formats = sd_->formats(0); @@ -738,11 +789,15 @@ private: const std::string &name); bool registerTPGCamera(MediaLink *link); bool registerSensorCamera(MediaLink *link); + bool registerMemoryInputCamera(); std::shared_ptr media_; + std::shared_ptr cruMedia_; std::unique_ptr isp_; std::unique_ptr stats_; std::unique_ptr params_; + std::unique_ptr ivcSd_; + std::unique_ptr ivc_; std::vector> statsBuffers_; std::queue availableStatsBuffers_; @@ -1685,6 +1740,34 @@ bool PipelineHandlerMaliC55::registerSensorCamera(MediaLink *ispLink) return true; } +bool PipelineHandlerMaliC55::registerMemoryInputCamera() +{ + std::unique_ptr data = + std::make_unique(this); + + int ret = data->initMemory(cruMedia_.get()); + if (ret) + return false; + + CameraSensor *sensor = data->sensor(); + data->properties_ = sensor->properties(); + + const CameraSensorProperties::SensorDelays &delays = sensor->sensorDelays(); + std::unordered_map params = { + { V4L2_CID_ANALOGUE_GAIN, { delays.gainDelay, false } }, + { V4L2_CID_EXPOSURE, { delays.exposureDelay, false } }, + }; + + data->delayedCtrls_ = + std::make_unique(sensor->device(), params); + isp_->frameStart.connect(data->delayedCtrls_.get(), + &DelayedControls::applyControls); + + ivc_->bufferReady.connect(data->cru(), &RZG2LCRU::cruReturnBuffer); + + return registerMaliCamera(std::move(data), sensor->device()->entity()->name()); +} + bool PipelineHandlerMaliC55::match(DeviceEnumerator *enumerator) { const MediaPad *ispSink; @@ -1694,14 +1777,14 @@ bool PipelineHandlerMaliC55::match(DeviceEnumerator *enumerator) * The TPG and the downscale pipe are both optional blocks and may not * be fitted. */ - DeviceMatch dm("mali-c55"); - dm.add("mali-c55 isp"); - dm.add("mali-c55 resizer fr"); - dm.add("mali-c55 fr"); - dm.add("mali-c55 3a stats"); - dm.add("mali-c55 3a params"); - - media_ = acquireMediaDevice(enumerator, dm); + DeviceMatch c55_dm("mali-c55"); + c55_dm.add("mali-c55 isp"); + c55_dm.add("mali-c55 resizer fr"); + c55_dm.add("mali-c55 fr"); + c55_dm.add("mali-c55 3a stats"); + c55_dm.add("mali-c55 3a params"); + + media_ = acquireMediaDevice(enumerator, c55_dm); if (!media_) return false; @@ -1761,6 +1844,25 @@ bool PipelineHandlerMaliC55::match(DeviceEnumerator *enumerator) stats_->bufferReady.connect(this, &PipelineHandlerMaliC55::statsBufferReady); params_->bufferReady.connect(this, &PipelineHandlerMaliC55::paramsBufferReady); + /* + * We also need to search for the rzg2l-cru CSI-2 receiver. If we find + * that then we need to work in memory input mode instead of the inline + * mode. The absence of this match is not necessarily a failure at this + * point...it depends on the media links that we investigate momentarily. + * + * This is a bit hacky, because there could be multiple of these media + * devices and we're just taking the first. We need modular pipelines to + * properly solve the issue. + */ + static const std::regex cruCsi2Regex("csi-[0-9a-f]{8}.csi2"); + static const std::regex cruIpRegex("cru-ip-[0-9a-f]{8}.cru[0-9]"); + + DeviceMatch cruDm("rzg2l_cru"); + cruDm.add(cruCsi2Regex); + cruDm.add(cruIpRegex); + cruDm.add("CRU output"); + cruMedia_ = acquireMediaDevice(enumerator, cruDm); + ispSink = isp_->entity()->getPadByIndex(0); if (!ispSink || ispSink->links().empty()) { LOG(MaliC55, Error) << "ISP sink pad error"; @@ -1774,13 +1876,6 @@ bool PipelineHandlerMaliC55::match(DeviceEnumerator *enumerator) * MEDIA_ENT_F_CAM_SENSOR - The test pattern generator * MEDIA_ENT_F_VID_IF_BRIDGE - A CSI-2 receiver * MEDIA_ENT_F_PROC_VIDEO_PIXEL_FORMATTER - An input device - * - * The last one will be unsupported for now. The TPG is relatively easy, - * we just register a Camera for it. If we have a CSI-2 receiver we need - * to check its sink pad and register Cameras for anything connected to - * it (probably...there are some complex situations in which that might - * not be true but let's pretend they don't exist until we come across - * them) */ bool registered; for (MediaLink *link : ispSink->links()) { @@ -1800,7 +1895,23 @@ bool PipelineHandlerMaliC55::match(DeviceEnumerator *enumerator) break; case MEDIA_ENT_F_PROC_VIDEO_PIXEL_FORMATTER: - LOG(MaliC55, Warning) << "Memory input not yet supported"; + if (!cruMedia_) + return false; + + ivcSd_ = V4L2Subdevice::fromEntityName(media_.get(), + "rzv2h ivc block"); + if (ivcSd_->open() < 0) + return false; + + ivc_ = V4L2VideoDevice::fromEntityName(media_.get(), + "rzv2h-ivc"); + if (ivc_->open() < 0) + return false; + + registered = registerMemoryInputCamera(); + if (!registered) + return registered; + break; default: LOG(MaliC55, Error) << "Unsupported entity function"; From patchwork Fri Mar 13 11:33:46 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jacopo Mondi X-Patchwork-Id: 26288 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 508F7C32B5 for ; Fri, 13 Mar 2026 11:34:31 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 0DD45626E8; Fri, 13 Mar 2026 12:34:31 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="m0WrLKrv"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id BC1F36268F for ; Fri, 13 Mar 2026 12:34:29 +0100 (CET) Received: from [192.168.224.131] (unknown [37.159.92.229]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 1FF531356; Fri, 13 Mar 2026 12:33:20 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1773401600; bh=8ul1n4i5T6cYcm/npmTY7Y82IwwGSy83YUy+ayD6KtU=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=m0WrLKrvOFeS7TlvSBCFx34XdqzwSiWrQo3B3sYHRPhbpfCb4EASq056itZTTn84v u7gzD/AqJCwVe7wQsYpuNmv5cl+vbjJ6P7Vjnkn9EPr+X5RZlx+8+7Ak/IXJo9epUP YDgbXPh8/lP9f2Sk9aBWew6c+EchjLnRq1qY3w18= From: Jacopo Mondi Date: Fri, 13 Mar 2026 12:33:46 +0100 Subject: [PATCH v4 5/7] libcamera: mali-c55: Configure camera in memory-to-memory MIME-Version: 1.0 Message-Id: <20260313-mali-cru-v4-5-c0d9bc8cd8fa@ideasonboard.com> References: <20260313-mali-cru-v4-0-c0d9bc8cd8fa@ideasonboard.com> In-Reply-To: <20260313-mali-cru-v4-0-c0d9bc8cd8fa@ideasonboard.com> To: Daniel Scally , libcamera-devel@lists.libcamera.org Cc: Jacopo Mondi X-Mailer: b4 0.14.3 X-Developer-Signature: v=1; a=openpgp-sha256; l=2412; i=jacopo.mondi@ideasonboard.com; h=from:subject:message-id; bh=8ul1n4i5T6cYcm/npmTY7Y82IwwGSy83YUy+ayD6KtU=; b=owEBbQKS/ZANAwAKAXI0Bo8WoVY8AcsmYgBps/Y1Nq5xHluLiAcnOzGo8TIY8pZx8w0aWwxY/ 9OkgryGQEuJAjMEAAEKAB0WIQS1xD1IgJogio9YOMByNAaPFqFWPAUCabP2NQAKCRByNAaPFqFW PLmDD/4gUrsvl6nCwQcPinOXiJN6OVELclzx7F4gkAyO4N6DPv1+Ycz8+GfFdXDUPhPuP1Wy8Yr 9XdZKHp/jAU5CnAZqoFqUeN5rVoGazZ1/cxWFa7zttYu+BuS+FEudr16hk3HuRd3Cs5dHaBge7p 88VMe+zFGmJoHWIRQP4QNmInzbBUdotntZUFvzic2eU2az9nnsQQVNh9DkmjrdjPi5L1TIXLo// eFg1Q0M1IMXpyQ2uIC1OlQGlltpV5JlgcEkx+CPhglq02m/GRI2yytp4NwaQUH1b+B8C2dKDhlm Sj6hR/X7BYt0WG2fzqQ1BObReBawwg1x04y72d0sqQ5iM9uDqXnmB6+bU04xQKwk6aVuoCBuAJ1 7ovzyYSwl7vxhr+t9oeL7u47sYPwWDnunlF7DsZtSoZCXcmqrHKsAuNC3PPrDNW7Kv9oaNJNu51 P/WtE4Cr5+c3SQAKAWkdkfyNx/gMbUZDNJmSBt8sEKLHr06ADKL3ExdrLwySE6tKXVKsPTYYZUk KpmvrM1/9L7eLJKQEDFsEWuLKzVmFoy4pNR3uib3U9hper4Q7gHS+DbuRrbcW8dFthdu34/HKHb khO5nhLP/8czCR0FOGcU4G8aaf5ru4zTyrqVAlvZadc1axHg3iS7qFH0jGM9SLdXupW4vkuHAVi SV8ORXqb/dnMZuQ== X-Developer-Key: i=jacopo.mondi@ideasonboard.com; a=openpgp; fpr=72392EDC88144A65C701EA9BA5826A2587AD026B X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Add support for memory-to-memory Camera to the PipelineHandlerMaliC55::configure() function. Start by enabling the IVC links, then configure the CRU and propagate the format to the IVC. Signed-off-by: Jacopo Mondi --- src/libcamera/pipeline/mali-c55/mali-c55.cpp | 26 ++++++++++++++++++++++++-- 1 file changed, 24 insertions(+), 2 deletions(-) diff --git a/src/libcamera/pipeline/mali-c55/mali-c55.cpp b/src/libcamera/pipeline/mali-c55/mali-c55.cpp index 3c8848d28707..575c8451fc4f 100644 --- a/src/libcamera/pipeline/mali-c55/mali-c55.cpp +++ b/src/libcamera/pipeline/mali-c55/mali-c55.cpp @@ -1042,9 +1042,12 @@ int PipelineHandlerMaliC55::configure(Camera *camera, if (std::holds_alternative(data->input_)) { const MediaEntity *tpgEntity = data->subdev()->entity(); ret = tpgEntity->getPadByIndex(0)->links()[0]->setEnabled(true); - } else { + } else if (std::holds_alternative(data->input_)) { const MediaEntity *csi2Entity = data->csi2()->entity(); ret = csi2Entity->getPadByIndex(1)->links()[0]->setEnabled(true); + } else { + const MediaEntity *ivcEntity = ivcSd_->entity(); + ret = ivcEntity->getPadByIndex(1)->links()[0]->setEnabled(true); } if (ret) return ret; @@ -1056,7 +1059,7 @@ int PipelineHandlerMaliC55::configure(Camera *camera, /* Apply format to the origin of the pipeline and propagate it. */ if (std::holds_alternative(data->input_)) { ret = data->subdev()->setFormat(0, &subdevFormat); - } else { + } else if (std::holds_alternative(data->input_)) { ret = data->sensor()->setFormat(&subdevFormat, maliConfig->combinedTransform()); if (ret) @@ -1067,6 +1070,25 @@ int PipelineHandlerMaliC55::configure(Camera *camera, return ret; ret = data->csi2()->getFormat(1, &subdevFormat); + } else { + V4L2DeviceFormat inputFormat; + + ret = data->cru()->configure(&subdevFormat, &inputFormat); + if (ret) + return ret; + + /* Propagate the CRU format to the IVC input. */ + ret = ivcSd_->setFormat(0, &subdevFormat); + if (ret) + return ret; + + ret = ivcSd_->getFormat(1, &subdevFormat); + if (ret) + return ret; + + ret = ivc_->setFormat(&inputFormat); + if (ret) + return ret; } if (ret) return ret; From patchwork Fri Mar 13 11:33:47 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jacopo Mondi X-Patchwork-Id: 26289 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 1AB1ABE086 for ; Fri, 13 Mar 2026 11:34:34 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id C22D7626CC; Fri, 13 Mar 2026 12:34:33 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="kjeNef8R"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 9982B626BE for ; Fri, 13 Mar 2026 12:34:31 +0100 (CET) Received: from [192.168.224.131] (unknown [37.159.92.229]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 30F85F52; Fri, 13 Mar 2026 12:33:21 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1773401602; bh=Pu5vGFdppL5hYCM49zYblomenvaxGbY4rr9u/hv8H2k=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=kjeNef8RqafOwt1SwHg6fvWcYkVas8PpoUPYwKqPOR4Nbw3LgQRrL6z0VO7XnVcqH A8mbWyGb2M26/N6lkvK2LSnB5XEfZl9xkIDXm/gi2YqnOoQu8ihBMO9+WmggoNZpnG wGN2DL72+Ncv0M5tfz/a0M3Rsq/TjXLsIIPbfzMc= From: Jacopo Mondi Date: Fri, 13 Mar 2026 12:33:47 +0100 Subject: [PATCH v4 6/7] libcamera: mali-c55: Implement capture for memory-to-memory MIME-Version: 1.0 Message-Id: <20260313-mali-cru-v4-6-c0d9bc8cd8fa@ideasonboard.com> References: <20260313-mali-cru-v4-0-c0d9bc8cd8fa@ideasonboard.com> In-Reply-To: <20260313-mali-cru-v4-0-c0d9bc8cd8fa@ideasonboard.com> To: Daniel Scally , libcamera-devel@lists.libcamera.org Cc: Jacopo Mondi X-Mailer: b4 0.14.3 X-Developer-Signature: v=1; a=openpgp-sha256; l=9435; i=jacopo.mondi@ideasonboard.com; h=from:subject:message-id; bh=9BNJlLRoxEiMcYg3pm6FhwrkFt9rU7wwZfV5nbZEg4c=; b=owEBbQKS/ZANAwAKAXI0Bo8WoVY8AcsmYgBps/Y1X3Nwyt4uEDhacT4yBW3AKraGSnwdzMAz0 I9wras/uZKJAjMEAAEKAB0WIQS1xD1IgJogio9YOMByNAaPFqFWPAUCabP2NQAKCRByNAaPFqFW PM/xD/9l8Pycyu+DjzLAfvl2jQWoL46yRRHTcHRaquFuvQ/cJBlOHYJl4lCcNCJadrBa1okPSAc tb8XYn0KkL/vvKxKmGebddERc7ywWrUHiZCPJeNYyhZ4nCLFKGYtBU3nsCbCHdRXqAdce/qsKwa NwUzil9ghRQZiaPW2VR+loSfwejJx/QXkuvCw/14FYw/rJv9LDxFGKqf1faYdzHIggYh/phDIiB 7jE11eDLQBgD+G3LNs96wDDVWsVfU6dqZdBmtIztVc7/XX5KwAaC0khQ5qUXXWEGwsQNcTrp8g+ O2FNW0qKskVdWe7m5nopfX5X9jHHdQefpkRPCOUFhLnbVtQ0Eh800LFHZlsiZ48ON0T1BPei1rq IeBc8GtbLElSs3KIKAyV3tFXrIUZoWBjw8N6MNLyXSgI1+VxM7CihPYakvPHkCHDKM9cNbJmSSa o6o6D4/tyq8+ln5P8+yxXXDg8FyKiidXQO0uUxlEo13+NKhgem1mssUzg9MSz0R0CrG8slYxNG9 mkwhWuReA3L0EEgL3u7ayhNvNNrpMjeor7GtmSMse95wZenLcvu6dMjvNhLCilIecAPsnSgRyk4 cvX0Rrsoy3BWN6Xgs4iRLYhzW2S435xV56vqEby46t33UuMfBqACCcRJ0XcHUVH1CK1rm8euxet rd0lU5g86CouA7Q== X-Developer-Key: i=jacopo.mondi@ideasonboard.com; a=openpgp; fpr=72392EDC88144A65C701EA9BA5826A2587AD026B X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" From: Daniel Scally Plumb in the MaliC55 pipeline handler support for capturing frames from memory using the CRU. Introduce a data flow which uses the CRU to feed the ISP through the IVC. In detail: - push incoming request to a pending queue until a buffer from the CRU is available - delay the call to ipa_->fillParams() to the CRU buffer ready even - once the IPA has computed parameters feed the ISP through the IVC with buffers from the CRU, params and statistics. Signed-off-by: Daniel Scally Signed-off-by: Jacopo Mondi --- src/libcamera/pipeline/mali-c55/mali-c55.cpp | 166 ++++++++++++++++++++++++++- 1 file changed, 161 insertions(+), 5 deletions(-) diff --git a/src/libcamera/pipeline/mali-c55/mali-c55.cpp b/src/libcamera/pipeline/mali-c55/mali-c55.cpp index 575c8451fc4f..a2f8f4b8e881 100644 --- a/src/libcamera/pipeline/mali-c55/mali-c55.cpp +++ b/src/libcamera/pipeline/mali-c55/mali-c55.cpp @@ -21,6 +21,7 @@ #include #include +#include #include #include #include @@ -88,6 +89,7 @@ struct MaliC55FrameInfo { FrameBuffer *paramBuffer; FrameBuffer *statBuffer; + FrameBuffer *rawBuffer; bool paramsDone; bool statsDone; @@ -721,11 +723,14 @@ public: int start(Camera *camera, const ControlList *controls) override; void stopDevice(Camera *camera) override; + int queuePendingRequests(MaliC55CameraData *data); + void cancelPendingRequests(); int queueRequestDevice(Camera *camera, Request *request) override; void imageBufferReady(FrameBuffer *buffer); void paramsBufferReady(FrameBuffer *buffer); void statsBufferReady(FrameBuffer *buffer); + void cruBufferReady(FrameBuffer *buffer); void paramsComputed(unsigned int requestId, uint32_t bytesused); void statsProcessed(unsigned int requestId, const ControlList &metadata); @@ -807,6 +812,11 @@ private: std::map frameInfoMap_; + /* Requests for which no buffer has been queued to the CRU device yet. */ + std::queue pendingRequests_; + /* Requests queued to the CRU device but not yet processed by the ISP. */ + std::queue processingRequests_; + std::array pipes_; bool dsFitted_; @@ -1249,6 +1259,11 @@ void PipelineHandlerMaliC55::freeBuffers(Camera *camera) if (params_->releaseBuffers()) LOG(MaliC55, Error) << "Failed to release params buffers"; + if (std::holds_alternative(data->input_)) { + if (ivc_->releaseBuffers()) + LOG(MaliC55, Error) << "Failed to release input buffers"; + } + return; } @@ -1278,6 +1293,12 @@ int PipelineHandlerMaliC55::allocateBuffers(Camera *camera) } }; + if (std::holds_alternative(data->input_)) { + ret = ivc_->importBuffers(RZG2LCRU::kBufferCount); + if (ret < 0) + return ret; + } + ret = stats_->allocateBuffers(bufferCount, &statsBuffers_); if (ret < 0) return ret; @@ -1309,6 +1330,24 @@ int PipelineHandlerMaliC55::start(Camera *camera, [[maybe_unused]] const Control if (ret) return ret; + if (std::holds_alternative(data->input_)) { + ret = data->cru()->start(); + if (ret) { + LOG(MaliC55, Error) + << "Failed to start CRU " << camera->id(); + freeBuffers(camera); + return ret; + } + + ret = ivc_->streamOn(); + if (ret) { + LOG(MaliC55, Error) + << "Failed to start IVC" << camera->id(); + freeBuffers(camera); + return ret; + } + } + if (data->ipa_) { ret = data->ipa_->start(); if (ret) { @@ -1398,6 +1437,12 @@ void PipelineHandlerMaliC55::stopDevice(Camera *camera) pipe.cap->releaseBuffers(); } + if (std::holds_alternative(data->input_)) { + cancelPendingRequests(); + ivc_->streamOff(); + data->cru()->stop(); + } + stats_->streamOff(); params_->streamOff(); if (data->ipa_) @@ -1501,10 +1546,88 @@ void PipelineHandlerMaliC55::applyScalerCrop(Camera *camera, } } +void PipelineHandlerMaliC55::cancelPendingRequests() +{ + processingRequests_ = {}; + + while (!pendingRequests_.empty()) { + Request *request = pendingRequests_.front(); + + completeRequest(request); + pendingRequests_.pop(); + } +} + +int PipelineHandlerMaliC55::queuePendingRequests(MaliC55CameraData *data) +{ + ASSERT(std::holds_alternative(data->input_)); + + while (!pendingRequests_.empty()) { + Request *request = pendingRequests_.front(); + + if (availableStatsBuffers_.empty()) { + LOG(MaliC55, Error) << "Stats buffer underrun"; + return -ENOENT; + } + + if (availableParamsBuffers_.empty()) { + LOG(MaliC55, Error) << "Params buffer underrun"; + return -ENOENT; + } + + MaliC55FrameInfo frameInfo; + frameInfo.request = request; + + frameInfo.rawBuffer = data->cru()->queueBuffer(request); + if (!frameInfo.rawBuffer) + return -ENOENT; + + frameInfo.statBuffer = availableStatsBuffers_.front(); + availableStatsBuffers_.pop(); + frameInfo.paramBuffer = availableParamsBuffers_.front(); + availableParamsBuffers_.pop(); + + frameInfo.paramsDone = false; + frameInfo.statsDone = false; + + frameInfoMap_[request->sequence()] = frameInfo; + + for (auto &[stream, buffer] : request->buffers()) { + MaliC55Pipe *pipe = pipeFromStream(data, stream); + + pipe->cap->queueBuffer(buffer); + } + + data->ipa_->queueRequest(request->sequence(), request->controls()); + + pendingRequests_.pop(); + processingRequests_.push(request); + } + + return 0; +} + int PipelineHandlerMaliC55::queueRequestDevice(Camera *camera, Request *request) { MaliC55CameraData *data = cameraData(camera); + /* + * If we're in memory input mode, we need to pop the requests onto the + * pending list until a CRU buffer is ready...otherwise we can just do + * everything immediately. + */ + if (std::holds_alternative(data->input_)) { + pendingRequests_.push(request); + + int ret = queuePendingRequests(data); + if (ret) { + pendingRequests_.pop(); + return ret; + } + + return 0; + } + /* Do not run the IPA if the TPG is in use. */ if (!data->ipa_) { MaliC55FrameInfo frameInfo; @@ -1569,7 +1692,8 @@ MaliC55FrameInfo *PipelineHandlerMaliC55::findFrameInfo(FrameBuffer *buffer) { for (auto &[sequence, info] : frameInfoMap_) { if (info.paramBuffer == buffer || - info.statBuffer == buffer) + info.statBuffer == buffer || + info.rawBuffer == buffer) return &info; } @@ -1631,6 +1755,26 @@ void PipelineHandlerMaliC55::statsBufferReady(FrameBuffer *buffer) sensorControls); } +void PipelineHandlerMaliC55::cruBufferReady(FrameBuffer *buffer) +{ + MaliC55FrameInfo *info = findFrameInfo(buffer); + Request *request = info->request; + ASSERT(info); + + if (buffer->metadata().status == FrameMetadata::FrameCancelled) { + frameInfoMap_.erase(request->sequence()); + completeRequest(request); + return; + } + + request->_d()->metadata().set(controls::SensorTimestamp, + buffer->metadata().timestamp); + + /* Ought we do something with the sensor's controls here...? */ + MaliC55CameraData *data = cameraData(request->_d()->camera()); + data->ipa_->fillParams(request->sequence(), info->paramBuffer->cookie()); +} + void PipelineHandlerMaliC55::paramsComputed(unsigned int requestId, uint32_t bytesused) { MaliC55FrameInfo &frameInfo = frameInfoMap_[requestId]; @@ -1639,18 +1783,27 @@ void PipelineHandlerMaliC55::paramsComputed(unsigned int requestId, uint32_t byt /* * Queue buffers for stats and params, then queue buffers to the capture - * video devices. + * video devices if we're running in Inline mode or with the TPG. + * + * If we're running in M2M buffers have been queued to the capture + * devices at queuePendingRequests() time and here we only have to queue + * buffers to the IVC input to start a transfer. */ frameInfo.paramBuffer->_d()->metadata().planes()[0].bytesused = bytesused; params_->queueBuffer(frameInfo.paramBuffer); stats_->queueBuffer(frameInfo.statBuffer); - for (auto &[stream, buffer] : request->buffers()) { - MaliC55Pipe *pipe = pipeFromStream(data, stream); + if (!std::holds_alternative(data->input_)) { + for (auto &[stream, buffer] : request->buffers()) { + MaliC55Pipe *pipe = pipeFromStream(data, stream); - pipe->cap->queueBuffer(buffer); + pipe->cap->queueBuffer(buffer); + } } + + if (std::holds_alternative(data->input_)) + ivc_->queueBuffer(frameInfo.rawBuffer); } void PipelineHandlerMaliC55::statsProcessed(unsigned int requestId, @@ -1787,6 +1940,9 @@ bool PipelineHandlerMaliC55::registerMemoryInputCamera() ivc_->bufferReady.connect(data->cru(), &RZG2LCRU::cruReturnBuffer); + V4L2VideoDevice *cruOutput = data->cru()->output(); + cruOutput->bufferReady.connect(this, &PipelineHandlerMaliC55::cruBufferReady); + return registerMaliCamera(std::move(data), sensor->device()->entity()->name()); } From patchwork Fri Mar 13 11:33:48 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Jacopo Mondi X-Patchwork-Id: 26290 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 9F7ADC32B5 for ; Fri, 13 Mar 2026 11:34:34 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 587A562714; Fri, 13 Mar 2026 12:34:34 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="X/nHvBf5"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 88AB16267F for ; Fri, 13 Mar 2026 12:34:32 +0100 (CET) Received: from [192.168.224.131] (unknown [37.159.92.229]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id EE9C01783; Fri, 13 Mar 2026 12:33:22 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1773401603; bh=RODaCUPDemFMAoR1fwRD2HIo2c+Jvu0vwlMja1pXbkQ=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=X/nHvBf5b8L35y4N9siLe1i/GkJGewI5AxgBShrp2XY1bEHPQlwQmMsaAn5eCESkx kJiJHdpH4y06lB1go6ILOb27+PEFjKuzyiO/nHgwscfnxD46Bx2fHAhEb1fi7NrztX /N/70f+VgNbOnOipKddPR4eRpUppkQXV4fZegMn0= From: Jacopo Mondi Date: Fri, 13 Mar 2026 12:33:48 +0100 Subject: [PATCH v4 7/7] libcamera: mali-c55: Fix sensor size computation MIME-Version: 1.0 Message-Id: <20260313-mali-cru-v4-7-c0d9bc8cd8fa@ideasonboard.com> References: <20260313-mali-cru-v4-0-c0d9bc8cd8fa@ideasonboard.com> In-Reply-To: <20260313-mali-cru-v4-0-c0d9bc8cd8fa@ideasonboard.com> To: Daniel Scally , libcamera-devel@lists.libcamera.org Cc: Jacopo Mondi , =?utf-8?b?QmFybmFiw6Fz?= =?utf-8?q?_P=C5=91cze?= X-Mailer: b4 0.14.3 X-Developer-Signature: v=1; a=openpgp-sha256; l=1117; i=jacopo.mondi@ideasonboard.com; h=from:subject:message-id; bh=RODaCUPDemFMAoR1fwRD2HIo2c+Jvu0vwlMja1pXbkQ=; b=owEBbQKS/ZANAwAKAXI0Bo8WoVY8AcsmYgBps/Y1t3gBkcWJ+5x/OVZdwStVpeeQTrrLdZtCG oT6iLs8m76JAjMEAAEKAB0WIQS1xD1IgJogio9YOMByNAaPFqFWPAUCabP2NQAKCRByNAaPFqFW PMgIEACORgRZYa67DVYJpZbWbq3KucAaPgxOJkNCMRV1/VZ9E/kf5/maYhKFczLX0QhAK3248ui uq91orfZ/JLteo6mileTwJqDwxdZRh0IffXNheuY0xUm47U4Qrcn8Ei5z+vhaQegw5W+ShuhE9+ WVgnQbrlF1TrSk0P9KB/O+zOVSapMwwmdLuWoqj4tvSuI460aSvSidXj1+Cm5Gj9qUtTNANSjkf /sMB8cX1mJaIsgCFm+BeqdcXtqgk7i/30RrFgn7SwHOWGrTS9PcyEtOwwg+nhL0qfkR/1QXSLKV PqxS0NLOqTLnWnXvWja4xDm9F3AmaCbv4GxiXvvJQ3daIOx3XqoAcpX9vrtwy/irngurqvYfJo6 njdIuu59QRVAEQLj9nCEGDGTchX0D7S4XUnVvfLXoi7bwqDJO7cyLExMvY4z+cl53HreRSX1d+Y h/mmbNckTz+Z9eNNwO7uWmnApEXJFzuo3KEkel8y6MdPrweDubjXiJlpZo4KvkTvFE2dfIIggbv oSGTRCeQBRj7y/Pfj1JMOYhVcOodaRPcFsKOx9llPRtyKpRQ3C7cZ+JJtkWvj1QuL6dpBuBVJtJ N6fTzkhcyZb9rmggdw6wVE7WOnF3dq7ihNplOVsbpiV0DylDFw63hWIJu10gwL0ELP/UvR8k4pr QwXMMa4DYMAR24A== X-Developer-Key: i=jacopo.mondi@ideasonboard.com; a=openpgp; fpr=72392EDC88144A65C701EA9BA5826A2587AD026B X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" The last size in the list is always selected unconditionally because the current best distance is not updated properly during the search. Fix the size computation procedure which has an inverted assignment. Signed-off-by: Jacopo Mondi Reviewed-by: Daniel Scally Tested-by: Daniel Scally Reviewed-by: Barnabás Pőcze --- src/libcamera/pipeline/mali-c55/mali-c55.cpp | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/libcamera/pipeline/mali-c55/mali-c55.cpp b/src/libcamera/pipeline/mali-c55/mali-c55.cpp index a2f8f4b8e881..71bd249a00e9 100644 --- a/src/libcamera/pipeline/mali-c55/mali-c55.cpp +++ b/src/libcamera/pipeline/mali-c55/mali-c55.cpp @@ -695,7 +695,7 @@ CameraConfiguration::Status MaliC55CameraConfiguration::validate() std::abs(static_cast(minSensorSize.height) - static_cast(size.height)); if (dist < distance) { - dist = distance; + distance = dist; bestSize = size; } }