From patchwork Wed Mar 25 14:44:08 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jacopo Mondi X-Patchwork-Id: 26335 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id D5ACABD87C for ; Wed, 25 Mar 2026 14:44:24 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 6FE7562837; Wed, 25 Mar 2026 15:44:22 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="uEfGL8rZ"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 134206275A for ; Wed, 25 Mar 2026 15:44:19 +0100 (CET) Received: from [192.168.1.104] (net-93-65-100-155.cust.vodafonedsl.it [93.65.100.155]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 0EFF712BB; Wed, 25 Mar 2026 15:43:01 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1774449781; bh=LzE7K2ZT92lC+0OgQ7GC7AW/OnLVL9s58QCkNi4oU1Q=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=uEfGL8rZZxjDEH6g0XhC2JcgPBAqQJIBF5o8wsoIJfm7VJgn5szpLn05GvEAD2QLQ pgPlZ8llnNrS/WwdMJuAI1KZk79q7iQS5PBQOWylMCza449I8fAWPouiRQKFF5s9Mv xwldWYXhPNzOj8EcUkmrv/KtFNbpIiS3nAmu5fY8= From: Jacopo Mondi Date: Wed, 25 Mar 2026 15:44:08 +0100 Subject: [PATCH v6 1/7] libcamera: mali-c55: Add RZG2LCRU class MIME-Version: 1.0 Message-Id: <20260325-mali-cru-v6-1-b16b0c49819a@ideasonboard.com> References: <20260325-mali-cru-v6-0-b16b0c49819a@ideasonboard.com> In-Reply-To: <20260325-mali-cru-v6-0-b16b0c49819a@ideasonboard.com> To: Daniel Scally , libcamera-devel@lists.libcamera.org Cc: =?utf-8?q?Barnab=C3=A1s_P=C5=91cze?= , Jacopo Mondi X-Mailer: b4 0.14.3 X-Developer-Signature: v=1; a=openpgp-sha256; l=9811; i=jacopo.mondi@ideasonboard.com; h=from:subject:message-id; bh=HRUXzB5lV5U5cG/O4+CYUjUzCClh4TlH2bSyiJVsx3c=; b=owEBbQKS/ZANAwAKAXI0Bo8WoVY8AcsmYgBpw/TBlz9AIloXt7WpQEwapvYFoFE9CC3RiA3CG EZC23OC/9aJAjMEAAEKAB0WIQS1xD1IgJogio9YOMByNAaPFqFWPAUCacP0wQAKCRByNAaPFqFW PN9RD/4tyZowA83Qiehvu2hSg9ypDXTTuFSeGUQzbrBFysM0cxr7ls9sa6fLT9HOx95ZOiXxwSa guP2XpgRJ+ffKpM6s1YFpO6DAmgMaUXipBESwpy019HY2lEINrAut9iVR6vADErUT/aBylvRXqg YlFfS0TvNPUOdP09oIxpw56NfRIpfwJDLOt6rlQ6lVeKpEkhL41DLTp3vZQ5qX+DLzo0SBTRWIg 8Q2FSIchfNL2g7dIEran3ah5XQNNy8J75+0wDmYsx8kxnqf45SixiIi/gNg36ladKGYLl0bE8q+ 0Ph/U0nVEW/sgmxMubx9qWdZG+ypfeQ4hWFJZe/I79Ynvl8mVe3d1ZR+NtjqjEGWlli50mRbXBT yHA+EvT4Ke3cgA+g7Nw+qlaKCZfOaBDUQU39RnSRIRHrX4QpSKF0iGuQ4Rjxr4oy7FyVmdaZwAu vfTeyPVXEbV/toD3pWBjPxJVmdRdfRBpS6esyGuiWgQevSM1z2NbDd9UFGMSvOGyePx5geTOe/T P/80d4NIdcmK/8VffX/y8K0XD+5Vf3e3sqqYDUe52a+rwa0orZujN30pKY+P4tepm6gv6hDueDH 9HXp1kcljxud9GtN7umQLeOI2LbdT6Jg1X6kYchdQDEsZZ9rywHmAXqWmxtVJhZfdf5H+nt4BbT 6smZnanoMDLTf2w== X-Developer-Key: i=jacopo.mondi@ideasonboard.com; a=openpgp; fpr=72392EDC88144A65C701EA9BA5826A2587AD026B X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" From: Daniel Scally Add a class allowing us to control the RZ/G2L Camera Receiver Unit. The class is named after the media device it handles, which is called "rzg2l_cru" but the CRU is actually found in other SoCs than the RZ/G2L one, such as the RZ/V2H(P). The RZG2LCRU class models the CSI-2 receiver found on those SoCs and allows to add support for memory-to-memory operations on the RZ/V2H(P) SoC which integrates a Mali-C55 ISP and a specialized DMA engine named IVC that feeds images from memory, where the CRU has saved them. Signed-off-by: Daniel Scally Signed-off-by: Jacopo Mondi --- src/libcamera/pipeline/mali-c55/meson.build | 3 +- src/libcamera/pipeline/mali-c55/rzg2l-cru.cpp | 253 ++++++++++++++++++++++++++ src/libcamera/pipeline/mali-c55/rzg2l-cru.h | 72 ++++++++ 3 files changed, 327 insertions(+), 1 deletion(-) diff --git a/src/libcamera/pipeline/mali-c55/meson.build b/src/libcamera/pipeline/mali-c55/meson.build index eba8e5a39054..4e768242602e 100644 --- a/src/libcamera/pipeline/mali-c55/meson.build +++ b/src/libcamera/pipeline/mali-c55/meson.build @@ -1,5 +1,6 @@ # SPDX-License-Identifier: CC0-1.0 libcamera_internal_sources += files([ - 'mali-c55.cpp' + 'mali-c55.cpp', + 'rzg2l-cru.cpp', ]) diff --git a/src/libcamera/pipeline/mali-c55/rzg2l-cru.cpp b/src/libcamera/pipeline/mali-c55/rzg2l-cru.cpp new file mode 100644 index 000000000000..cfb188db5ae4 --- /dev/null +++ b/src/libcamera/pipeline/mali-c55/rzg2l-cru.cpp @@ -0,0 +1,253 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2025, Ideas on Board Oy + * + * Pipine handler element for the Renesas RZ/G2L Camera Receiver Unit + */ + +#include "rzg2l-cru.h" + +#include +#include + +#include + +#include +#include +#include + +#include +#include + +#include "libcamera/internal/bayer_format.h" +#include "libcamera/internal/camera_sensor.h" +#include "libcamera/internal/framebuffer.h" +#include "libcamera/internal/media_device.h" +#include "libcamera/internal/request.h" + +namespace libcamera { + +static const std::map bitDepthToFmt{ + { 10, V4L2PixelFormat(V4L2_PIX_FMT_RAW_CRU10) }, + { 12, V4L2PixelFormat(V4L2_PIX_FMT_RAW_CRU12) }, + { 14, V4L2PixelFormat(V4L2_PIX_FMT_RAW_CRU14) }, +}; + +LOG_DEFINE_CATEGORY(RZG2LCRU) + +FrameBuffer *RZG2LCRU::queueBuffer(Request *request) +{ + FrameBuffer *buffer; + + if (availableBuffers_.empty()) { + LOG(RZG2LCRU, Debug) << "CRU buffer underrun"; + return nullptr; + } + + buffer = availableBuffers_.back(); + + int ret = output_->queueBuffer(buffer); + if (ret) { + LOG(RZG2LCRU, Error) << "Failed to queue buffer to CRU"; + return nullptr; + } + + availableBuffers_.pop_back(); + buffer->_d()->setRequest(request); + + return buffer; +} + +void RZG2LCRU::returnBuffer(FrameBuffer *buffer) +{ + auto it = std::find_if(buffers_.begin(), buffers_.end(), + [&](const auto &b) { return b.get() == buffer; }); + ASSERT(it != buffers_.end()); + + availableBuffers_.push_back(buffer); +} + +int RZG2LCRU::start() +{ + int ret = output_->exportBuffers(kBufferCount, &buffers_); + if (ret < 0) + return ret; + + utils::scope_exit bufferGuard([&] { freeBuffers(); }); + + ret = output_->importBuffers(kBufferCount); + if (ret) + return ret; + + for (std::unique_ptr &buffer : buffers_) + availableBuffers_.push_back(buffer.get()); + + ret = output_->streamOn(); + if (ret) + return ret; + + bufferGuard.release(); + + return 0; +} + +int RZG2LCRU::stop() +{ + return output_->streamOff(); +} + +int RZG2LCRU::freeBuffers() +{ + availableBuffers_.clear(); + buffers_.clear(); + + return output_->releaseBuffers(); +} + +int RZG2LCRU::configure(V4L2SubdeviceFormat *subdevFormat, V4L2DeviceFormat *inputFormat) +{ + /* + * Set format on the sensor and propagate it up to the CRU video + * device. + */ + + int ret = sensor_->setFormat(subdevFormat); + if (ret) + return ret; + + ret = csi2_->setFormat(0, subdevFormat); + if (ret) + return ret; + + ret = csi2_->getFormat(1, subdevFormat); + if (ret) + return ret; + + ret = cru_->setFormat(0, subdevFormat); + if (ret) + return ret; + + ret = cru_->getFormat(1, subdevFormat); + if (ret) + return ret; + + /* + * The capture device needs to be set with a format that can be produced + * from the mbus code of the subdevFormat. The CRU and IVC use bayer + * order agnostic pixel formats, so all we need to do is find the right + * bitdepth and select the appropriate format. + */ + BayerFormat bayerFormat = BayerFormat::fromMbusCode(subdevFormat->code); + if (!bayerFormat.isValid()) + return -EINVAL; + + V4L2DeviceFormat captureFormat; + captureFormat.fourcc = bitDepthToFmt.at(bayerFormat.bitDepth); + captureFormat.size = subdevFormat->size; + + ret = output_->setFormat(&captureFormat); + if (ret) + return ret; + + /* + * We return the format that we set against the output device, as the + * same format will also need to be set against the Input Video Control + * Block device. + */ + *inputFormat = captureFormat; + + return 0; +} + +void RZG2LCRU::initCRUSizes() +{ + Size maxCSI2Size; + + /* + * Get the maximum supported size on the CSI-2 receiver. We need to + * query the kernel interface as the size limits differ from RZ/G2L + * (2800x4095) and RZ/V2H (4096x4096). + */ + V4L2Subdevice::Formats csi2Formats = csi2_->formats(0); + if (csi2Formats.empty()) + return; + + for (const auto &format : csi2Formats) { + for (const auto &range : format.second) { + if (range.max > maxCSI2Size) + maxCSI2Size = range.max; + } + } + + /* + * Enumerate the sensor supported resolutions and filter out the ones + * larger than the maximum supported CSI-2 receiver input size. + */ + V4L2Subdevice::Formats formats = sensor_->device()->formats(0); + if (formats.empty()) + return; + + for (const auto &format : formats) { + for (const auto &range : format.second) { + const Size &max = range.max; + + if (max.width > maxCSI2Size.width || + max.height > maxCSI2Size.height) + continue; + + csi2Sizes_.push_back(max); + } + } + + /* Sort in increasing order and remove duplicates. */ + std::sort(csi2Sizes_.begin(), csi2Sizes_.end()); + auto last = std::unique(csi2Sizes_.begin(), csi2Sizes_.end()); + csi2Sizes_.erase(last, csi2Sizes_.end()); + + csi2Resolution_ = csi2Sizes_.back(); +} + +int RZG2LCRU::init(const MediaDevice *media) +{ + static const std::regex cruRegex("cru-ip-[0-9a-f]{8}.cru[0-9]"); + static const std::regex csi2Regex("csi-[0-9a-f]{8}.csi2"); + + csi2_ = V4L2Subdevice::fromEntityName(media, csi2Regex); + if (!csi2_) + return -ENODEV; + + int ret = csi2_->open(); + if (ret) + return ret; + + const std::vector &pads = csi2_->entity()->pads(); + if (pads.empty()) + return -ENODEV; + + /* The receiver has a single sink pad at index 0 */ + MediaPad *sink = pads[0]; + const std::vector &links = sink->links(); + if (links.empty()) + return -ENODEV; + + MediaLink *link = links[0]; + sensor_ = CameraSensorFactoryBase::create(link->source()->entity()); + if (!sensor_) + return -ENODEV; + + cru_ = V4L2Subdevice::fromEntityName(media, cruRegex); + ret = cru_->open(); + if (ret) + return ret; + + output_ = V4L2VideoDevice::fromEntityName(media, "CRU output"); + ret = output_->open(); + if (ret) + return ret; + + initCRUSizes(); + + return 0; +} + +} /* namespace libcamera */ diff --git a/src/libcamera/pipeline/mali-c55/rzg2l-cru.h b/src/libcamera/pipeline/mali-c55/rzg2l-cru.h new file mode 100644 index 000000000000..235763d66733 --- /dev/null +++ b/src/libcamera/pipeline/mali-c55/rzg2l-cru.h @@ -0,0 +1,72 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2025, Ideas on Board Oy + * + * Pipine handler element for the Renesas RZ/G2L Camera Receiver Unit + */ + +#pragma once + +#include +#include + +#include + +#include "libcamera/internal/v4l2_subdevice.h" +#include "libcamera/internal/v4l2_videodevice.h" + +namespace libcamera { + +class CameraSensor; +class FrameBuffer; +class MediaDevice; +class Request; +class Size; + +class RZG2LCRU +{ +public: + static constexpr unsigned int kBufferCount = 4; + + RZG2LCRU() = default; + + const std::vector &sizes() const + { + return csi2Sizes_; + } + + int init(const MediaDevice *media); + const Size &resolution() const + { + return csi2Resolution_; + } + + CameraSensor *sensor() const { return sensor_.get(); } + V4L2Subdevice *csi2() const { return csi2_.get(); } + V4L2Subdevice *cru() const { return cru_.get(); } + V4L2VideoDevice *output() { return output_.get(); } + + int configure(V4L2SubdeviceFormat *subdevFormat, V4L2DeviceFormat *inputFormat); + FrameBuffer *queueBuffer(Request *request); + void returnBuffer(FrameBuffer *buffer); + int freeBuffers(); + + int start(); + int stop(); + +private: + void initCRUSizes(); + + std::unique_ptr sensor_; + std::unique_ptr csi2_; + std::unique_ptr cru_; + std::unique_ptr output_; + + std::vector> buffers_; + std::vector availableBuffers_; + + std::vector csi2Sizes_; + Size csi2Resolution_; +}; + +} /* namespace libcamera */ From patchwork Wed Mar 25 14:44:09 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Jacopo Mondi X-Patchwork-Id: 26336 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id D9DE7C32DE for ; Wed, 25 Mar 2026 14:44:26 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id EDA6A6274D; Wed, 25 Mar 2026 15:44:22 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="c1lJI5tQ"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 5DD046274D for ; Wed, 25 Mar 2026 15:44:19 +0100 (CET) Received: from [192.168.1.104] (net-93-65-100-155.cust.vodafonedsl.it [93.65.100.155]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 5DF011943; Wed, 25 Mar 2026 15:43:01 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1774449781; bh=FegylNAS/ZE8S1oZUyAdmGxPOxc4ElLR/QAsnKcxhNE=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=c1lJI5tQfvmoapB3LC2CgETRqulLpb2O2rig4M7SI/9/nH5WeZUSwq69kVMo9yXEt DcPvR0LIlMykC4JCkEQqYEyzNANLSJ1qY4UfiK6ChNEkrFhM2wlKldQ7Io917SLfAW jlwaNAg+LvP2jtEl8JahihC4XEU8RX8RVuJ2Wiao= From: Jacopo Mondi Date: Wed, 25 Mar 2026 15:44:09 +0100 Subject: [PATCH v6 2/7] libcamera: utils: Add overloaded visitor helpers MIME-Version: 1.0 Message-Id: <20260325-mali-cru-v6-2-b16b0c49819a@ideasonboard.com> References: <20260325-mali-cru-v6-0-b16b0c49819a@ideasonboard.com> In-Reply-To: <20260325-mali-cru-v6-0-b16b0c49819a@ideasonboard.com> To: Daniel Scally , libcamera-devel@lists.libcamera.org Cc: =?utf-8?q?Barnab=C3=A1s_P=C5=91cze?= , Jacopo Mondi X-Mailer: b4 0.14.3 X-Developer-Signature: v=1; a=openpgp-sha256; l=4147; i=jacopo.mondi@ideasonboard.com; h=from:subject:message-id; bh=FegylNAS/ZE8S1oZUyAdmGxPOxc4ElLR/QAsnKcxhNE=; b=kA0DAAoBcjQGjxahVjwByyZiAGnD9MGjgZHy5H2qMyHMYnglPixqilbRwVyt8+W3tWKVmsXDF okCMwQAAQoAHRYhBLXEPUiAmiCKj1g4wHI0Bo8WoVY8BQJpw/TBAAoJEHI0Bo8WoVY84B0P/j7j ZFXkLrMkFZxfvIL9DzuHjfmSN+ACjnl68A+tr8PYBjYnUNPbwNKs4K+YNzTHL4csNQlRculBG5B Cyk0Zo37cBUMz7VUb1DwLqe2n2TM+TmSksSS0hreyGioz4Elma8zPGnXiRxW+muVh+kdeOf70SU ihnfQ0P6YPeV/fIpw8RvDr6l+fO49aCS+b0Cqu1+37K+ks8+Br4Q1RwLjcp/4ht2Uq+xPg2XRaI RmIWORs3kc6NSwqNeNTkQpS84wYAdp5zPL72Umm5N5s5GxheDfFE/xs1T9BNKPLwB0EAXSN50Ob p22i12A5JnPH75CcaXV9W6QAAQNjY11TCqwnE5QMzTcttu5Tciqh1ha8brBuHhviQR8byOBlkLH sqXBf2uOsTUx2DukF+p1gctkTxy2NhAOlXMKFtCLs7KnmQX22rnGOrAhvgtENW26FXU3SDaQ/fV nxUxa1UBM7U+dFCMImgzRO3Vy9CX446+pqGgN6L7kZpDttBG3gJ346iZtYOrGcTb7AHT6mnUIpb rFbrwCaQ/my8rXsRupwKlQDZMvlGTUa147GwPizeETO4MCTj1mSxkcLwX+3D6hwO8I682GRN5oZ J7K8/uiXi4o3wxYELRlMsnxHNoFlrB+oLC/5H1XcGrngg6yrTccTbWb2qmjJ3BdWGbIda5JltNq ZziZ6 X-Developer-Key: i=jacopo.mondi@ideasonboard.com; a=openpgp; fpr=72392EDC88144A65C701EA9BA5826A2587AD026B X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" std::visit() allows quite elegant type-matching implementation of the visitor pattern. The 'overloaded' type helpers allow to define a hierarchy of overloaded operator() implementations which can be used by std::visit(). Currently only the Virtual pipeline handler uses this type-matching implementation of std::visit. To prepare to add another user in the Mali C55 pipeline handler move the 'overloaded' helper type to libcamera::utils for easier re-use. Reviewed-by: Barnabás Pőcze Signed-off-by: Jacopo Mondi --- include/libcamera/base/utils.h | 7 ++++++ src/libcamera/base/utils.cpp | 35 ++++++++++++++++++++++++++++++ src/libcamera/pipeline/virtual/virtual.cpp | 10 ++------- 3 files changed, 44 insertions(+), 8 deletions(-) diff --git a/include/libcamera/base/utils.h b/include/libcamera/base/utils.h index 7083b7ce9ce9..62c7f89c25a1 100644 --- a/include/libcamera/base/utils.h +++ b/include/libcamera/base/utils.h @@ -37,6 +37,13 @@ namespace libcamera { namespace utils { +template +struct overloaded : Ts... { + using Ts::operator()...; +}; +template +overloaded(Ts...) -> overloaded; + const char *basename(const char *path); char *secure_getenv(const char *name); diff --git a/src/libcamera/base/utils.cpp b/src/libcamera/base/utils.cpp index 42a516097be2..836aec87c539 100644 --- a/src/libcamera/base/utils.cpp +++ b/src/libcamera/base/utils.cpp @@ -23,6 +23,41 @@ namespace libcamera { namespace utils { +/** + * \struct overloaded + * \brief Helper type for type-matching std::visit implementations + * \tparam Ts... Template arguments pack of visitors + * + * Expand the template argument pack \a Ts... to provide overloaded \a + * operator() to support type-matching implementations of the visitor design + * pattern using std::visit. + * + * An example is provided by the STL documentation in the form of: + * + * \code{.cpp} + * template struct overloaded : Ts... { using Ts::operator()...; }; + * template overloaded(Ts...) -> overloaded; + * + * using var_t = std::variant; + * std::vector vec = {10, 15l, 1.5, "hello"}; + * + * for (auto& v: vec) { + * std::visit(overloaded { + * [](auto arg) { std::cout << arg << ' '; }, + * [](double arg) { std::cout << std::fixed << arg << ' '; }, + * [](const std::string& arg) { std::cout << std::quoted(arg) << ' '; }, + * }, v); + * \endcode + * + * Use this helper to implement type-matching visitors using std::visit(). + */ + +/** + * \var overloaded(Ts...) -> overloaded + * \brief Deduction guide necessary for C++17 compatibility + * \tparam Ts... Template arguments pack of visitor functions + */ + /** * \brief Strip the directory prefix from the path * \param[in] path The path to process diff --git a/src/libcamera/pipeline/virtual/virtual.cpp b/src/libcamera/pipeline/virtual/virtual.cpp index efd800ebe3d6..e8ef7e524ccf 100644 --- a/src/libcamera/pipeline/virtual/virtual.cpp +++ b/src/libcamera/pipeline/virtual/virtual.cpp @@ -23,6 +23,7 @@ #include #include +#include #include #include @@ -57,13 +58,6 @@ uint64_t currentTimestamp() } /* namespace */ -template -struct overloaded : Ts... { - using Ts::operator()...; -}; -template -overloaded(Ts...) -> overloaded; - class VirtualCameraConfiguration : public CameraConfiguration { public: @@ -428,7 +422,7 @@ bool PipelineHandlerVirtual::initFrameGenerator(Camera *camera) { auto data = cameraData(camera); auto &frame = data->config_.frame; - std::visit(overloaded{ + std::visit(utils::overloaded{ [&](TestPattern &testPattern) { for (auto &streamConfig : data->streamConfigs_) { if (testPattern == TestPattern::DiagonalLines) From patchwork Wed Mar 25 14:44:10 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jacopo Mondi X-Patchwork-Id: 26337 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id AE323C32F5 for ; Wed, 25 Mar 2026 14:44:27 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id B5EB562848; Wed, 25 Mar 2026 15:44:23 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="ad4XUgvj"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id C2CA96274D for ; Wed, 25 Mar 2026 15:44:19 +0100 (CET) Received: from [192.168.1.104] (net-93-65-100-155.cust.vodafonedsl.it [93.65.100.155]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id AA0E51992; Wed, 25 Mar 2026 15:43:01 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1774449781; bh=XZteA98BJvRZqHQDIZ+YmPArJOla5ZwfJaXTmvhkFrA=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=ad4XUgvjDb1lGCD/TETKXu0lTbNqSvkTJnVFyvAMiA4ang6r4YZB3kzGvuinHCGXK vF7RHyitgjJsQJPu9bDVxpl13bNzvmaRCMXoK5nZs+/iSrZiJ3/qOkvf4QAnWsX161 7gjHRiD1sTUEBVvh+kKtV2yTPcza3A9IMM3X+GDg= From: Jacopo Mondi Date: Wed, 25 Mar 2026 15:44:10 +0100 Subject: [PATCH v6 3/7] libcamera: mali-c55: Split TPG and Inline camera handling MIME-Version: 1.0 Message-Id: <20260325-mali-cru-v6-3-b16b0c49819a@ideasonboard.com> References: <20260325-mali-cru-v6-0-b16b0c49819a@ideasonboard.com> In-Reply-To: <20260325-mali-cru-v6-0-b16b0c49819a@ideasonboard.com> To: Daniel Scally , libcamera-devel@lists.libcamera.org Cc: =?utf-8?q?Barnab=C3=A1s_P=C5=91cze?= , Jacopo Mondi X-Mailer: b4 0.14.3 X-Developer-Signature: v=1; a=openpgp-sha256; l=16121; i=jacopo.mondi@ideasonboard.com; h=from:subject:message-id; bh=XZteA98BJvRZqHQDIZ+YmPArJOla5ZwfJaXTmvhkFrA=; b=owEBbQKS/ZANAwAKAXI0Bo8WoVY8AcsmYgBpw/TBRmiY4GgcjbAXxoaU27SVVoj+mp/WIJDWO YCZoaUwMOSJAjMEAAEKAB0WIQS1xD1IgJogio9YOMByNAaPFqFWPAUCacP0wQAKCRByNAaPFqFW PNLOD/41wN6k0AA/M3JauoZl1JKHCeSuxy42hZsCkwH7loaOIGkQG0hpvuA/xokDvpMNdFzT6CF J8b12F1MIpn7iht5So9j7fS75Iu9P83khvvpbyo24NyByVP/h42PFYVJ2t0mzQrrMcWgNLxkFE2 LVy1I42ZhpC2KM4L1niF1AePYz25IbSAbXy8p8wOrfWeZ5av1Ky57Z6Fnu1gEfM2w3upaMzc9dG X7/Uxr03YnmCSujgOc1rTW3Y7LH65yYN6bwtTdrqqC1H5A1pj7nsFMrFuZ96f6AGARzVulBamkT Fl0KB64eXnBPtXQH+JPJ8rM+Fy8NcwbUXgtipNSgiD2c6EI1gzs8f1jfq9zZYtb3FdV6ygaNyKN y8JLquoeNJX0tHt/agzcYcMm48T/G87+bt9d24WK/E+47Ku76K6eu3yYvCMqxYUuCzq0IABhGoZ Fv64C7EwHxj/beb0+qnIIZSeFHmbGw9mloiJTLsQuqmDGmmkzh6YkwZbWJlrcGogUef6zQfg8IJ nPCi9pp4g3NXftENUonT2IYwJsLKyNLP1fIOv8sfhe9CdcTWNQCUu65zqErsCFMZmf1CaSRukQM XVvddRiV83TjJgt2KDhooJFosETq5dG3Wz4S9gJxB6v8kOgCwKEhhH/h4Af+43bgUkAhlWjvy7W 5p3hMJhBPr0CZcw== X-Developer-Key: i=jacopo.mondi@ideasonboard.com; a=openpgp; fpr=72392EDC88144A65C701EA9BA5826A2587AD026B X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" In order to prepare to support memory input cameras, split the handling of the TPG and Inline camera cases. The Mali C55 pipeline handler uses the entity and subdevice stored in the CameraData to support both the handling of the TPG and of the Inline (CSI-2 + sensor) use cases. Adding support for memory cameras by using the CRU unit would add yet-another special case making the code harder to follow and more prone to errors. Split the handling of the TPG and Inline cameras by introducing an std::variant<> variable and to deflect the functions called on the camera data to the correct type by using overloaded std::visit<>(). Signed-off-by: Jacopo Mondi --- src/libcamera/pipeline/mali-c55/mali-c55.cpp | 278 ++++++++++++++++----------- 1 file changed, 162 insertions(+), 116 deletions(-) diff --git a/src/libcamera/pipeline/mali-c55/mali-c55.cpp b/src/libcamera/pipeline/mali-c55/mali-c55.cpp index c209b0b070b1..dca82564e2eb 100644 --- a/src/libcamera/pipeline/mali-c55/mali-c55.cpp +++ b/src/libcamera/pipeline/mali-c55/mali-c55.cpp @@ -11,12 +11,14 @@ #include #include #include +#include #include #include #include #include +#include #include #include @@ -92,17 +94,77 @@ struct MaliC55FrameInfo { class MaliC55CameraData : public Camera::Private { public: - MaliC55CameraData(PipelineHandler *pipe, MediaEntity *entity) - : Camera::Private(pipe), entity_(entity) + struct Tpg { + std::vector sizes(unsigned int mbusCode) const; + + Size resolution_; + std::unique_ptr sd_; + }; + + struct Inline { + std::unique_ptr csi2_; + std::unique_ptr sensor_; + }; + using CameraType = std::variant; + + MaliC55CameraData(PipelineHandler *pipe) + : Camera::Private(pipe) { } - int init(); int loadIPA(); - /* Deflect these functionalities to either TPG or CameraSensor. */ - std::vector sizes(unsigned int mbusCode) const; - Size resolution() const; + Tpg *initTpg(MediaEntity *entity); + Inline *initInline(MediaEntity *entity); + + std::vector sizes(unsigned int mbusCode) const + { + return std::visit(utils::overloaded{ + [&](const Tpg &tpg) -> std::vector { + return tpg.sizes(mbusCode); + }, + [&](const Inline &in) -> std::vector { + return in.sensor_->sizes(mbusCode); + } + }, input_); + } + + V4L2Subdevice *subdev() const + { + return std::visit(utils::overloaded{ + [&](const Tpg &tpg) -> V4L2Subdevice * { + return tpg.sd_.get(); + }, + [&](const Inline &in) -> V4L2Subdevice * { + return in.sensor_->device(); + }, + }, input_); + } + + CameraSensor *sensor() const + { + return std::visit(utils::overloaded{ + [&](auto &) -> CameraSensor * { + ASSERT(false); + return nullptr; + }, + [&](const Inline &in) -> CameraSensor * { + return in.sensor_.get(); + }, + }, input_); + } + + Size resolution() const + { + return std::visit(utils::overloaded{ + [&](const Tpg &tpg) -> Size { + return tpg.resolution_; + }, + [&](const Inline &in) -> Size { + return in.sensor_->resolution(); + }, + }, input_); + } int pixfmtToMbusCode(const PixelFormat &pixFmt) const; const PixelFormat &bestRawFormat() const; @@ -112,11 +174,6 @@ public: PixelFormat adjustRawFormat(const PixelFormat &pixFmt) const; Size adjustRawSizes(const PixelFormat &pixFmt, const Size &rawSize) const; - std::unique_ptr sensor_; - - MediaEntity *entity_; - std::unique_ptr csi_; - std::unique_ptr sd_; Stream frStream_; Stream dsStream_; @@ -126,58 +183,28 @@ public: std::unique_ptr delayedCtrls_; + CameraType input_; + private: - void initTPGData(); void setSensorControls(const ControlList &sensorControls); - std::string id_; - Size tpgResolution_; }; -int MaliC55CameraData::init() +MaliC55CameraData::Tpg *MaliC55CameraData::initTpg(MediaEntity *entity) { - int ret; + Tpg tpg; - sd_ = std::make_unique(entity_); - ret = sd_->open(); + tpg.sd_ = std::make_unique(entity); + int ret = tpg.sd_->open(); if (ret) { - LOG(MaliC55, Error) << "Failed to open sensor subdevice"; - return ret; - } - - /* If this camera is created from TPG, we return here. */ - if (entity_->name() == "mali-c55 tpg") { - initTPGData(); - return 0; - } - - /* - * Register a CameraSensor if we connect to a sensor and create - * an entity for the connected CSI-2 receiver. - */ - sensor_ = CameraSensorFactoryBase::create(entity_); - if (!sensor_) - return -ENODEV; - - const MediaPad *sourcePad = entity_->getPadByIndex(0); - MediaEntity *csiEntity = sourcePad->links()[0]->sink()->entity(); - - csi_ = std::make_unique(csiEntity); - ret = csi_->open(); - if (ret) { - LOG(MaliC55, Error) << "Failed to open CSI-2 subdevice"; - return ret; + LOG(MaliC55, Error) << "Failed to open TPG subdevice"; + return nullptr; } - return 0; -} - -void MaliC55CameraData::initTPGData() -{ /* Replicate the CameraSensor implementation for TPG. */ - V4L2Subdevice::Formats formats = sd_->formats(0); + V4L2Subdevice::Formats formats = tpg.sd_->formats(0); if (formats.empty()) - return; + return nullptr; std::vector tpgSizes; @@ -187,19 +214,35 @@ void MaliC55CameraData::initTPGData() [](const SizeRange &range) { return range.max; }); } - tpgResolution_ = tpgSizes.back(); + tpg.resolution_ = tpgSizes.back(); + + return &input_.emplace(std::move(tpg)); } -void MaliC55CameraData::setSensorControls(const ControlList &sensorControls) +MaliC55CameraData::Inline *MaliC55CameraData::initInline(MediaEntity *sensor) { - delayedCtrls_->push(sensorControls); + Inline in; + + /* Register a CameraSensor and create an entity for the CSI-2 receiver. */ + in.sensor_ = CameraSensorFactoryBase::create(sensor); + if (!in.sensor_) + return nullptr; + + const MediaPad *sourcePad = sensor->getPadByIndex(0); + MediaEntity *csiEntity = sourcePad->links()[0]->sink()->entity(); + + in.csi2_ = std::make_unique(csiEntity); + int ret = in.csi2_->open(); + if (ret) { + LOG(MaliC55, Error) << "Failed to open CSI-2 subdevice"; + return nullptr; + } + + return &input_.emplace(std::move(in)); } -std::vector MaliC55CameraData::sizes(unsigned int mbusCode) const +std::vector MaliC55CameraData::Tpg::sizes(unsigned int mbusCode) const { - if (sensor_) - return sensor_->sizes(mbusCode); - V4L2Subdevice::Formats formats = sd_->formats(0); if (formats.empty()) return {}; @@ -218,12 +261,9 @@ std::vector MaliC55CameraData::sizes(unsigned int mbusCode) const return sizes; } -Size MaliC55CameraData::resolution() const +void MaliC55CameraData::setSensorControls(const ControlList &sensorControls) { - if (sensor_) - return sensor_->resolution(); - - return tpgResolution_; + delayedCtrls_->push(sensorControls); } /* @@ -242,7 +282,7 @@ int MaliC55CameraData::pixfmtToMbusCode(const PixelFormat &pixFmt) const if (!bayerFormat.isValid()) return -EINVAL; - V4L2Subdevice::Formats formats = sd_->formats(0); + V4L2Subdevice::Formats formats = subdev()->formats(0); unsigned int sensorMbusCode = 0; unsigned int bitDepth = 0; @@ -280,7 +320,7 @@ const PixelFormat &MaliC55CameraData::bestRawFormat() const { static const PixelFormat invalidPixFmt = {}; - for (const auto &fmt : sd_->formats(0)) { + for (const auto &fmt : subdev()->formats(0)) { BayerFormat sensorBayer = BayerFormat::fromMbusCode(fmt.first); if (!sensorBayer.isValid()) @@ -302,11 +342,11 @@ const PixelFormat &MaliC55CameraData::bestRawFormat() const void MaliC55CameraData::updateControls(const ControlInfoMap &ipaControls) { - if (!sensor_) + if (std::holds_alternative(input_)) return; IPACameraSensorInfo sensorInfo; - int ret = sensor_->sensorInfo(&sensorInfo); + int ret = sensor()->sensorInfo(&sensorInfo); if (ret) { LOG(MaliC55, Error) << "Failed to retrieve sensor info"; return; @@ -379,7 +419,7 @@ int MaliC55CameraData::loadIPA() int ret; /* Do not initialize IPA for TPG. */ - if (!sensor_) + if (std::holds_alternative(input_)) return 0; ipa_ = IPAManager::createIPA(pipe(), 1, 1); @@ -388,20 +428,20 @@ int MaliC55CameraData::loadIPA() ipa_->setSensorControls.connect(this, &MaliC55CameraData::setSensorControls); - std::string ipaTuningFile = ipa_->configurationFile(sensor_->model() + ".yaml", + std::string ipaTuningFile = ipa_->configurationFile(sensor()->model() + ".yaml", "uncalibrated.yaml"); /* We need to inform the IPA of the sensor configuration */ ipa::mali_c55::IPAConfigInfo ipaConfig{}; - ret = sensor_->sensorInfo(&ipaConfig.sensorInfo); + ret = sensor()->sensorInfo(&ipaConfig.sensorInfo); if (ret) return ret; - ipaConfig.sensorControls = sensor_->controls(); + ipaConfig.sensorControls = sensor()->controls(); ControlInfoMap ipaControls; - ret = ipa_->init({ ipaTuningFile, sensor_->model() }, ipaConfig, + ret = ipa_->init({ ipaTuningFile, sensor()->model() }, ipaConfig, &ipaControls); if (ret) { LOG(MaliC55, Error) << "Failed to initialise the Mali-C55 IPA"; @@ -444,13 +484,13 @@ CameraConfiguration::Status MaliC55CameraConfiguration::validate() * The TPG doesn't support flips, so we only need to calculate a * transform if we have a sensor. */ - if (data_->sensor_) { + if (std::holds_alternative(data_->input_)) { + combinedTransform_ = Transform::Rot0; + } else { Orientation requestedOrientation = orientation; - combinedTransform_ = data_->sensor_->computeTransform(&orientation); + combinedTransform_ = data_->sensor()->computeTransform(&orientation); if (orientation != requestedOrientation) status = Adjusted; - } else { - combinedTransform_ = Transform::Rot0; } /* Only 2 streams available. */ @@ -927,39 +967,44 @@ int PipelineHandlerMaliC55::configure(Camera *camera, /* Link the graph depending if we are operating the TPG or a sensor. */ MaliC55CameraData *data = cameraData(camera); - if (data->csi_) { - const MediaEntity *csiEntity = data->csi_->entity(); - ret = csiEntity->getPadByIndex(1)->links()[0]->setEnabled(true); - } else { - ret = data->entity_->getPadByIndex(0)->links()[0]->setEnabled(true); - } + ret = std::visit(utils::overloaded{ + [](MaliC55CameraData::Tpg &tpg) { + const MediaEntity *tpgEntity = tpg.sd_->entity(); + return tpgEntity->getPadByIndex(0)->links()[0]->setEnabled(true); + }, + [](MaliC55CameraData::Inline &in) { + const MediaEntity *csi2Entity = in.csi2_->entity(); + return csi2Entity->getPadByIndex(1)->links()[0]->setEnabled(true); + }, + }, data->input_); if (ret) return ret; MaliC55CameraConfiguration *maliConfig = static_cast(config); V4L2SubdeviceFormat subdevFormat = maliConfig->sensorFormat_; - ret = data->sd_->getFormat(0, &subdevFormat); + + /* Apply format to the origin of the pipeline and propagate it. */ + ret = std::visit(utils::overloaded{ + [&](MaliC55CameraData::Tpg &) { + return data->subdev()->setFormat(0, &subdevFormat); + }, + [&](MaliC55CameraData::Inline &in) { + int r = in.sensor_->setFormat(&subdevFormat, + maliConfig->combinedTransform()); + if (r) + return r; + + r = in.csi2_->setFormat(0, &subdevFormat); + if (r) + return r; + + return in.csi2_->getFormat(1, &subdevFormat); + }, + }, data->input_); if (ret) return ret; - if (data->sensor_) { - ret = data->sensor_->setFormat(&subdevFormat, - maliConfig->combinedTransform()); - if (ret) - return ret; - } - - if (data->csi_) { - ret = data->csi_->setFormat(0, &subdevFormat); - if (ret) - return ret; - - ret = data->csi_->getFormat(1, &subdevFormat); - if (ret) - return ret; - } - V4L2DeviceFormat statsFormat; ret = stats_->getFormat(&statsFormat); if (ret) @@ -973,8 +1018,6 @@ int PipelineHandlerMaliC55::configure(Camera *camera, /* * Propagate the format to the ISP sink pad and configure the input * crop rectangle (no crop at the moment). - * - * \todo Configure the CSI-2 receiver. */ ret = isp_->setFormat(0, &subdevFormat); if (ret) @@ -1058,18 +1101,18 @@ int PipelineHandlerMaliC55::configure(Camera *camera, /* We need to inform the IPA of the sensor configuration */ ipa::mali_c55::IPAConfigInfo ipaConfig{}; - ret = data->sensor_->sensorInfo(&ipaConfig.sensorInfo); + ret = data->sensor()->sensorInfo(&ipaConfig.sensorInfo); if (ret) return ret; - ipaConfig.sensorControls = data->sensor_->controls(); + ipaConfig.sensorControls = data->sensor()->controls(); /* * And we also need to tell the IPA the bayerOrder of the data (as * affected by any flips that we've configured) */ const Transform &combinedTransform = maliConfig->combinedTransform(); - BayerFormat::Order bayerOrder = data->sensor_->bayerOrder(combinedTransform); + BayerFormat::Order bayerOrder = data->sensor()->bayerOrder(combinedTransform); ControlInfoMap ipaControls; ret = data->ipa_->configure(ipaConfig, utils::to_underlying(bayerOrder), @@ -1283,7 +1326,7 @@ void PipelineHandlerMaliC55::applyScalerCrop(Camera *camera, if (!scalerCrop) return; - if (!data->sensor_) { + if (std::holds_alternative(data->input_)) { LOG(MaliC55, Error) << "ScalerCrop not supported for TPG"; return; } @@ -1291,7 +1334,7 @@ void PipelineHandlerMaliC55::applyScalerCrop(Camera *camera, Rectangle nativeCrop = *scalerCrop; IPACameraSensorInfo sensorInfo; - int ret = data->sensor_->sensorInfo(&sensorInfo); + int ret = data->sensor()->sensorInfo(&sensorInfo); if (ret) { LOG(MaliC55, Error) << "Failed to retrieve sensor info"; return; @@ -1573,9 +1616,9 @@ bool PipelineHandlerMaliC55::registerTPGCamera(MediaLink *link) } std::unique_ptr data = - std::make_unique(this, link->source()->entity()); + std::make_unique(this); - if (data->init()) + if (!data->initTpg(link->source()->entity())) return false; return registerMaliCamera(std::move(data), name); @@ -1600,21 +1643,24 @@ bool PipelineHandlerMaliC55::registerSensorCamera(MediaLink *ispLink) continue; std::unique_ptr data = - std::make_unique(this, sensor); - if (data->init()) + std::make_unique(this); + + auto *in = data->initInline(sensor); + if (!in) return false; - data->properties_ = data->sensor_->properties(); + data->properties_ = in->sensor_->properties(); - const CameraSensorProperties::SensorDelays &delays = data->sensor_->sensorDelays(); + const CameraSensorProperties::SensorDelays &delays = + in->sensor_->sensorDelays(); std::unordered_map params = { { V4L2_CID_ANALOGUE_GAIN, { delays.gainDelay, false } }, { V4L2_CID_EXPOSURE, { delays.exposureDelay, false } }, }; - data->delayedCtrls_ = - std::make_unique(data->sensor_->device(), - params); + V4L2Subdevice *sensorSubdev = in->sensor_->device(); + data->delayedCtrls_ = std::make_unique(sensorSubdev, + params); isp_->frameStart.connect(data->delayedCtrls_.get(), &DelayedControls::applyControls); From patchwork Wed Mar 25 14:44:11 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jacopo Mondi X-Patchwork-Id: 26338 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 7FECDC32F6 for ; Wed, 25 Mar 2026 14:44:28 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 2732C627D7; Wed, 25 Mar 2026 15:44:25 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="aMq3hSwF"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 1046B627A8 for ; Wed, 25 Mar 2026 15:44:20 +0100 (CET) Received: from [192.168.1.104] (net-93-65-100-155.cust.vodafonedsl.it [93.65.100.155]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 16737121A; Wed, 25 Mar 2026 15:43:02 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1774449782; bh=Ojnmz9x+DXQFBfVhMAnLOJ9jhNzcj66e63HUZdvKwDE=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=aMq3hSwF67pR9nUtnBIuUz9AG0kYjfLVP3jL2wYX1B4Z6+Bj+tN1/twThILMGKslp eF4yZELIG3a6Cd33reVLAjFauvi22NzFBn1mYI98qVXsaSiWWKDRWheCKsQIYrywtO bo2fau20dwVwdAL+0/Bh3pEmdWJxfB73ZeYPpnHE= From: Jacopo Mondi Date: Wed, 25 Mar 2026 15:44:11 +0100 Subject: [PATCH v6 4/7] libcamera: mali-c55: Register memory input camera MIME-Version: 1.0 Message-Id: <20260325-mali-cru-v6-4-b16b0c49819a@ideasonboard.com> References: <20260325-mali-cru-v6-0-b16b0c49819a@ideasonboard.com> In-Reply-To: <20260325-mali-cru-v6-0-b16b0c49819a@ideasonboard.com> To: Daniel Scally , libcamera-devel@lists.libcamera.org Cc: =?utf-8?q?Barnab=C3=A1s_P=C5=91cze?= , Jacopo Mondi X-Mailer: b4 0.14.3 X-Developer-Signature: v=1; a=openpgp-sha256; l=9308; i=jacopo.mondi@ideasonboard.com; h=from:subject:message-id; bh=Ojnmz9x+DXQFBfVhMAnLOJ9jhNzcj66e63HUZdvKwDE=; b=owEBbQKS/ZANAwAKAXI0Bo8WoVY8AcsmYgBpw/TB7WTWMWTC/9Y4tc/+gf+xzA6XEuWdyMERD Bk+cLYd6WOJAjMEAAEKAB0WIQS1xD1IgJogio9YOMByNAaPFqFWPAUCacP0wQAKCRByNAaPFqFW PFASD/4qi6WEF6a4y9mkSoJssTRGtxsifPPwTXr/CYIVcRD4f4CXZNDgqbx0IPy6I4pU1P2lvj7 VF0C3CpvR/qCi2Pj6dyL3/Q61PIMWfO9OUGBVfelcPX4EnfBa97nhyy2Qu9KN2QiYcxquD1cKGA XSgWTgdfQd94BhBSmW9hiFG5eQF/gHveAjMSNFICblUToD7nr3Mf6GPjE3xrXGQO29qxkHK7jq1 3jfgDxpb/Dud0tzpqH1FWsQw6dkq6LtcxoZmMHAyIW1Nx5+KRZO667h/EtrcBR5NgYDr+0kMPX8 KHiYtKpmkp+zAaOb/ulv5x53O2qY6PmpyvgYkiHaK8Y4gISj9tMB2LDjzZTa1n65K7UkqRJt8H4 gGYkimduLihhp3uZMOX5TpzLeee6z97If8tjInDpLCjFffwz1j5xqk6XfJ61RUijCHV2W1EuM1a oqvpaOu1K4YpVWNNKRvytQ8cHaeS8DFl4fCRe3bY2dk6feIWel41nYJL9hbI5xxwh5s5DvIy1/Z LqkLgHO5eB44CS/IIhZ3Na3wAtOVe66bUjmNvxEJrN9zxLLxWhFfWDYHHzdwhiBScO+X9K8SyU1 UJbIQIasFATz6gyzuxNival98NWvMnFPxfWZhIKEJFClUtmCJkI7b4zOqqywp2nTNv297taL5aJ 9gNwjJPxKF8fRnQ== X-Developer-Key: i=jacopo.mondi@ideasonboard.com; a=openpgp; fpr=72392EDC88144A65C701EA9BA5826A2587AD026B X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Add support to Mali C55 pipeline handler for memory-to-memory camera mode. The Mali-C55, as integrated in the Renesas RZ/V2H(P) SoC, is fed with images saved to memory by the CRU unit and read on behalf of the ISP by the IVC. Update the pipeline handler's match() function to search for the CRU media entity necessary to register a memory input camera. Create and initialize a CameraData instance for the memory camera use case if a valid CRU is found and initialize the IVC components used to feed the ISP with images read from memory. Do not implement camera configuration for the time being to reuce the patch size. It will be provided in the following patches. Signed-off-by: Daniel Scally Signed-off-by: Jacopo Mondi --- src/libcamera/pipeline/mali-c55/mali-c55.cpp | 135 +++++++++++++++++++++++---- 1 file changed, 117 insertions(+), 18 deletions(-) diff --git a/src/libcamera/pipeline/mali-c55/mali-c55.cpp b/src/libcamera/pipeline/mali-c55/mali-c55.cpp index dca82564e2eb..26d368bb9132 100644 --- a/src/libcamera/pipeline/mali-c55/mali-c55.cpp +++ b/src/libcamera/pipeline/mali-c55/mali-c55.cpp @@ -44,6 +44,8 @@ #include "libcamera/internal/v4l2_subdevice.h" #include "libcamera/internal/v4l2_videodevice.h" +#include "rzg2l-cru.h" + namespace { bool isFormatRaw(const libcamera::PixelFormat &pixFmt) @@ -105,7 +107,12 @@ public: std::unique_ptr csi2_; std::unique_ptr sensor_; }; - using CameraType = std::variant; + + struct Memory { + std::unique_ptr cru_; + }; + + using CameraType = std::variant; MaliC55CameraData(PipelineHandler *pipe) : Camera::Private(pipe) @@ -116,6 +123,7 @@ public: Tpg *initTpg(MediaEntity *entity); Inline *initInline(MediaEntity *entity); + Memory *initMemory(MediaDevice *cruMedia); std::vector sizes(unsigned int mbusCode) const { @@ -125,7 +133,10 @@ public: }, [&](const Inline &in) -> std::vector { return in.sensor_->sizes(mbusCode); - } + }, + [&](const Memory &mem) -> std::vector { + return mem.cru_->sizes(); + }, }, input_); } @@ -138,6 +149,9 @@ public: [&](const Inline &in) -> V4L2Subdevice * { return in.sensor_->device(); }, + [&](const Memory &mem) -> V4L2Subdevice * { + return mem.cru_->sensor()->device(); + }, }, input_); } @@ -151,6 +165,9 @@ public: [&](const Inline &in) -> CameraSensor * { return in.sensor_.get(); }, + [&](const Memory &mem) -> CameraSensor * { + return mem.cru_->sensor(); + }, }, input_); } @@ -163,6 +180,9 @@ public: [&](const Inline &in) -> Size { return in.sensor_->resolution(); }, + [&](const Memory &mem) -> Size { + return mem.cru_->resolution(); + }, }, input_); } @@ -241,6 +261,19 @@ MaliC55CameraData::Inline *MaliC55CameraData::initInline(MediaEntity *sensor) return &input_.emplace(std::move(in)); } +MaliC55CameraData::Memory *MaliC55CameraData::initMemory(MediaDevice *cruMedia) +{ + Memory mem; + + mem.cru_ = std::make_unique(); + + int ret = mem.cru_->init(cruMedia); + if (ret) + return nullptr; + + return &input_.emplace(std::move(mem)); +} + std::vector MaliC55CameraData::Tpg::sizes(unsigned int mbusCode) const { V4L2Subdevice::Formats formats = sd_->formats(0); @@ -721,11 +754,15 @@ private: const std::string &name); bool registerTPGCamera(MediaLink *link); bool registerSensorCamera(MediaLink *link); + bool registerMemoryInputCamera(); std::shared_ptr media_; + std::shared_ptr cruMedia_; std::unique_ptr isp_; std::unique_ptr stats_; std::unique_ptr params_; + std::unique_ptr ivcSd_; + std::unique_ptr ivc_; std::vector> statsBuffers_; std::queue availableStatsBuffers_; @@ -976,6 +1013,9 @@ int PipelineHandlerMaliC55::configure(Camera *camera, const MediaEntity *csi2Entity = in.csi2_->entity(); return csi2Entity->getPadByIndex(1)->links()[0]->setEnabled(true); }, + [](MaliC55CameraData::Memory &) { + return 0; + }, }, data->input_); if (ret) return ret; @@ -1001,6 +1041,9 @@ int PipelineHandlerMaliC55::configure(Camera *camera, return in.csi2_->getFormat(1, &subdevFormat); }, + [](MaliC55CameraData::Memory &) { + return 0; + }, }, data->input_); if (ret) return ret; @@ -1673,6 +1716,34 @@ bool PipelineHandlerMaliC55::registerSensorCamera(MediaLink *ispLink) return true; } +bool PipelineHandlerMaliC55::registerMemoryInputCamera() +{ + std::unique_ptr data = + std::make_unique(this); + + auto *mem = data->initMemory(cruMedia_.get()); + if (!mem) + return false; + + CameraSensor *sensor = data->sensor(); + data->properties_ = sensor->properties(); + + const CameraSensorProperties::SensorDelays &delays = sensor->sensorDelays(); + std::unordered_map params = { + { V4L2_CID_ANALOGUE_GAIN, { delays.gainDelay, false } }, + { V4L2_CID_EXPOSURE, { delays.exposureDelay, false } }, + }; + + data->delayedCtrls_ = + std::make_unique(sensor->device(), params); + isp_->frameStart.connect(data->delayedCtrls_.get(), + &DelayedControls::applyControls); + + ivc_->bufferReady.connect(mem->cru_.get(), &RZG2LCRU::returnBuffer); + + return registerMaliCamera(std::move(data), sensor->device()->entity()->name()); +} + bool PipelineHandlerMaliC55::match(DeviceEnumerator *enumerator) { const MediaPad *ispSink; @@ -1682,14 +1753,14 @@ bool PipelineHandlerMaliC55::match(DeviceEnumerator *enumerator) * The TPG and the downscale pipe are both optional blocks and may not * be fitted. */ - DeviceMatch dm("mali-c55"); - dm.add("mali-c55 isp"); - dm.add("mali-c55 resizer fr"); - dm.add("mali-c55 fr"); - dm.add("mali-c55 3a stats"); - dm.add("mali-c55 3a params"); - - media_ = acquireMediaDevice(enumerator, dm); + DeviceMatch c55_dm("mali-c55"); + c55_dm.add("mali-c55 isp"); + c55_dm.add("mali-c55 resizer fr"); + c55_dm.add("mali-c55 fr"); + c55_dm.add("mali-c55 3a stats"); + c55_dm.add("mali-c55 3a params"); + + media_ = acquireMediaDevice(enumerator, c55_dm); if (!media_) return false; @@ -1749,6 +1820,25 @@ bool PipelineHandlerMaliC55::match(DeviceEnumerator *enumerator) stats_->bufferReady.connect(this, &PipelineHandlerMaliC55::statsBufferReady); params_->bufferReady.connect(this, &PipelineHandlerMaliC55::paramsBufferReady); + /* + * We also need to search for the rzg2l-cru CSI-2 receiver. If we find + * that then we need to work in memory input mode instead of the inline + * mode. The absence of this match is not necessarily a failure at this + * point...it depends on the media links that we investigate momentarily. + * + * This is a bit hacky, because there could be multiple of these media + * devices and we're just taking the first. We need modular pipelines to + * properly solve the issue. + */ + static const std::regex cruCsi2Regex("csi-[0-9a-f]{8}.csi2"); + static const std::regex cruIpRegex("cru-ip-[0-9a-f]{8}.cru[0-9]"); + + DeviceMatch cruDm("rzg2l_cru"); + cruDm.add(cruCsi2Regex); + cruDm.add(cruIpRegex); + cruDm.add("CRU output"); + cruMedia_ = acquireMediaDevice(enumerator, cruDm); + ispSink = isp_->entity()->getPadByIndex(0); if (!ispSink || ispSink->links().empty()) { LOG(MaliC55, Error) << "ISP sink pad error"; @@ -1762,13 +1852,6 @@ bool PipelineHandlerMaliC55::match(DeviceEnumerator *enumerator) * MEDIA_ENT_F_CAM_SENSOR - The test pattern generator * MEDIA_ENT_F_VID_IF_BRIDGE - A CSI-2 receiver * MEDIA_ENT_F_PROC_VIDEO_PIXEL_FORMATTER - An input device - * - * The last one will be unsupported for now. The TPG is relatively easy, - * we just register a Camera for it. If we have a CSI-2 receiver we need - * to check its sink pad and register Cameras for anything connected to - * it (probably...there are some complex situations in which that might - * not be true but let's pretend they don't exist until we come across - * them) */ bool registered; for (MediaLink *link : ispSink->links()) { @@ -1788,7 +1871,23 @@ bool PipelineHandlerMaliC55::match(DeviceEnumerator *enumerator) break; case MEDIA_ENT_F_PROC_VIDEO_PIXEL_FORMATTER: - LOG(MaliC55, Warning) << "Memory input not yet supported"; + if (!cruMedia_) + return false; + + ivcSd_ = V4L2Subdevice::fromEntityName(media_.get(), + "rzv2h ivc block"); + if (ivcSd_->open() < 0) + return false; + + ivc_ = V4L2VideoDevice::fromEntityName(media_.get(), + "rzv2h-ivc"); + if (ivc_->open() < 0) + return false; + + registered = registerMemoryInputCamera(); + if (!registered) + return registered; + break; default: LOG(MaliC55, Error) << "Unsupported entity function"; From patchwork Wed Mar 25 14:44:12 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jacopo Mondi X-Patchwork-Id: 26339 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id F25AEC32F7 for ; Wed, 25 Mar 2026 14:44:28 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id C1C536283B; Wed, 25 Mar 2026 15:44:25 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="Hce/ANx4"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 61B186274D for ; Wed, 25 Mar 2026 15:44:20 +0100 (CET) Received: from [192.168.1.104] (net-93-65-100-155.cust.vodafonedsl.it [93.65.100.155]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 5F5FC12BB; Wed, 25 Mar 2026 15:43:02 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1774449782; bh=sP0IWdRDNIIoDrW3dfKUPHYNyxEAn42AOQ7h2wJnwIs=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=Hce/ANx41uhdH+xPBlQCiWl/7jq838ZpPJ/uWi76g1qPhgL5opoblSYI+odv/mYSD pQcOnmUw4EO3QXUaCbBsdRAGJCq4z3ih50PMZQWYyPh/zPSHwrryTbhPCs1V2ipsD3 bsIOEYbxqBj5szTxPIIwUUvNNoRggGQ0bf2F0Zx0= From: Jacopo Mondi Date: Wed, 25 Mar 2026 15:44:12 +0100 Subject: [PATCH v6 5/7] libcamera: mali-c55: Configure camera in memory-to-memory MIME-Version: 1.0 Message-Id: <20260325-mali-cru-v6-5-b16b0c49819a@ideasonboard.com> References: <20260325-mali-cru-v6-0-b16b0c49819a@ideasonboard.com> In-Reply-To: <20260325-mali-cru-v6-0-b16b0c49819a@ideasonboard.com> To: Daniel Scally , libcamera-devel@lists.libcamera.org Cc: =?utf-8?q?Barnab=C3=A1s_P=C5=91cze?= , Jacopo Mondi X-Mailer: b4 0.14.3 X-Developer-Signature: v=1; a=openpgp-sha256; l=1783; i=jacopo.mondi@ideasonboard.com; h=from:subject:message-id; bh=sP0IWdRDNIIoDrW3dfKUPHYNyxEAn42AOQ7h2wJnwIs=; b=owEBbQKS/ZANAwAKAXI0Bo8WoVY8AcsmYgBpw/TBFOcfItju3wI/6K0agWpPfOVINColHeEFx IauDWpntk+JAjMEAAEKAB0WIQS1xD1IgJogio9YOMByNAaPFqFWPAUCacP0wQAKCRByNAaPFqFW PJoyD/9k/UmLYEncMNbIEINmNyDPHYNEC/k7kijP+2r2IZKEgoVGt5oFFsiOCTkt2/4zGN+m1TN POYywo64l8WPCvWpZpIhNPUIUMnbEz987EkpVdNL0BK85C5eZkQQHNJdeDhX7I0Pv2x6pfmCY9v GygTtqtbcdqG3VstHy1ZV1iiOk1BOxgaroH8DhtBM4FehFkGqEyBboiAcFgNBdU/aQgu2KSmi74 KY5L/R95ki9dFbaInfzTuto2chglsL41dPNldpozCNhptq38in967KGfDisK3Ydve3dX/Fh+WHr lIGizyQoEHGBYA9/gYeaaF2HEYFBhbxwzKp3a6rA+1ec1GNQKN9lGKivurzdeebQHVEX1TjBnkl TysNJiRBQ5HaZsfXicPYNzaoAJZtobdaTV/QnAFxrmT3dJ/aapFuz+WUmL/J8kulqu8xPhEZHrI b3htcWfefKuKtqZv1Q7svuUX/UDxtPfggn6shG7ARQWdbBlBCvIGaCqBMZ2xYbLrJoYQAOEPy49 M5gt1+FVEpaZ7W3um3hKWbSjsJmSqeCEM+HNIkt8CKqF77I5uoezhxPTu17TsAmvlqrT5eWgMG3 vQ19LCKCC8WVQ8JoYz7aBoPU6oH6+8Jf0gyVjKYuctBFb50k2PD3L331Rqv1Yt061JqRM9C2gkG Orcpep4uh0aU2Kw== X-Developer-Key: i=jacopo.mondi@ideasonboard.com; a=openpgp; fpr=72392EDC88144A65C701EA9BA5826A2587AD026B X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Add support for memory-to-memory Camera to the PipelineHandlerMaliC55::configure() function. Start by enabling the IVC links, then configure the CRU and propagate the format to the IVC. Signed-off-by: Jacopo Mondi --- src/libcamera/pipeline/mali-c55/mali-c55.cpp | 24 ++++++++++++++++++++---- 1 file changed, 20 insertions(+), 4 deletions(-) diff --git a/src/libcamera/pipeline/mali-c55/mali-c55.cpp b/src/libcamera/pipeline/mali-c55/mali-c55.cpp index 26d368bb9132..be3e38da95ab 100644 --- a/src/libcamera/pipeline/mali-c55/mali-c55.cpp +++ b/src/libcamera/pipeline/mali-c55/mali-c55.cpp @@ -1013,8 +1013,9 @@ int PipelineHandlerMaliC55::configure(Camera *camera, const MediaEntity *csi2Entity = in.csi2_->entity(); return csi2Entity->getPadByIndex(1)->links()[0]->setEnabled(true); }, - [](MaliC55CameraData::Memory &) { - return 0; + [&](MaliC55CameraData::Memory &) { + const MediaEntity *ivcEntity = ivcSd_->entity(); + return ivcEntity->getPadByIndex(1)->links()[0]->setEnabled(true); }, }, data->input_); if (ret) @@ -1041,8 +1042,23 @@ int PipelineHandlerMaliC55::configure(Camera *camera, return in.csi2_->getFormat(1, &subdevFormat); }, - [](MaliC55CameraData::Memory &) { - return 0; + [&](MaliC55CameraData::Memory &mem) { + V4L2DeviceFormat inputFormat; + + int r = mem.cru_->configure(&subdevFormat, &inputFormat); + if (r) + return r; + + /* Propagate the CRU format to the IVC input. */ + r = ivcSd_->setFormat(0, &subdevFormat); + if (r) + return r; + + r = ivcSd_->getFormat(1, &subdevFormat); + if (r) + return r; + + return ivc_->setFormat(&inputFormat); }, }, data->input_); if (ret) From patchwork Wed Mar 25 14:44:13 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jacopo Mondi X-Patchwork-Id: 26340 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 748C5C32F8 for ; Wed, 25 Mar 2026 14:44:29 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id B03376283C; Wed, 25 Mar 2026 15:44:26 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="l7HOhznP"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id B2A246274D for ; Wed, 25 Mar 2026 15:44:20 +0100 (CET) Received: from [192.168.1.104] (net-93-65-100-155.cust.vodafonedsl.it [93.65.100.155]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id AE9211ADE; Wed, 25 Mar 2026 15:43:02 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1774449782; bh=flq4ge5QIzfQeoUke8dLji5yEiMp+FGTl/TsXLIuW3M=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=l7HOhznPjmwC4thbZvDloByz3yiPN675TqSavFDcBAJz8aKvHgI7ikNonwE0cX8Ef eBlBsws59UrNd4uZYwxOPY5IIoaoVcEKao38PYupNV3hJuYIXdXw8EZUHSmPZbYsJ2 V5OA5Lh5jPWHkSKOnZuGWZgycVwskzNq1bcyz21o= From: Jacopo Mondi Date: Wed, 25 Mar 2026 15:44:13 +0100 Subject: [PATCH v6 6/7] libcamera: mali-c55: Implement capture for memory-to-memory MIME-Version: 1.0 Message-Id: <20260325-mali-cru-v6-6-b16b0c49819a@ideasonboard.com> References: <20260325-mali-cru-v6-0-b16b0c49819a@ideasonboard.com> In-Reply-To: <20260325-mali-cru-v6-0-b16b0c49819a@ideasonboard.com> To: Daniel Scally , libcamera-devel@lists.libcamera.org Cc: =?utf-8?q?Barnab=C3=A1s_P=C5=91cze?= , Jacopo Mondi X-Mailer: b4 0.14.3 X-Developer-Signature: v=1; a=openpgp-sha256; l=13937; i=jacopo.mondi@ideasonboard.com; h=from:subject:message-id; bh=pHJmjnkudjHJjFFuc+UJG4KlbidDyD0DgZrmZ5fOwbQ=; b=owEBbQKS/ZANAwAKAXI0Bo8WoVY8AcsmYgBpw/TCUWHsn7psmT4Uig1WfvGxmoqC6265FkuOF iKZBUFJjRuJAjMEAAEKAB0WIQS1xD1IgJogio9YOMByNAaPFqFWPAUCacP0wgAKCRByNAaPFqFW PE6hD/9yhdTQuOhs4+8fRgqddfkoSm2bOVccSAa/a9yPNvXgI40QdWCbe+v+GIarlKSEhixnmZJ xXVanvKkj7ofWsGHuSXycEgv0f6Z6fxhdV6wEfnOejN8eZV88nTH67l+JfTqhGEB4meB13iiUeG /0UFrfu7jTsMRwMjtDQ73JvJcbQRqJzTox6Ykooy82ipPxNtaKOdZuoO2ZO1pXJIkfg+4ijci2u BvXFVSG0LZQJGmqorrU1d7IT9yR1IKgC0NRShDo407vVUBoJg0krnK6KPgfn5n/xQRul88aExWE r4UZub6DH1Z7Sx5v1U1wjoX46hb3tv4QVzHDblopsuAh7ld3YJKKYTr/x5vUzQ64B0ubKbzTOFb tl6Kp4HZjoV5pf+bGZOB2eFbB7f0SbFNg6xWwnNgdXiYB26xAMLhX7WDGJfhMmQhPW6aGf8QkjL Yl9dFsQ30EzC9IIF52vU3REFUfUTEE7Djp1dfqFwMboev7G8rH1UcKmtpm1GQtXy2NCgstOhZf2 B9rowozmWz2M++Y9zCMUil1SxXtt63906EvYXBB8T0zcM6Fo2vrTK19dd/dBk7qVZYwXauo2c+B rGf+UPoYJWRUH6c1bSU9r1gzgZJ9Rus7C8RwYgRRd7Llc9xxEIxU/A+hpUg5zChtZKbKkNo2btP /WGkuiF7PDUBf2A== X-Developer-Key: i=jacopo.mondi@ideasonboard.com; a=openpgp; fpr=72392EDC88144A65C701EA9BA5826A2587AD026B X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" From: Daniel Scally Plumb in the MaliC55 pipeline handler support for capturing frames from memory using the CRU. Introduce a data flow which uses the CRU to feed the ISP through the IVC. In detail: - push incoming request to a pending queue until a buffer from the CRU is available - delay the call to ipa_->fillParams() to the CRU buffer ready even - once the IPA has computed parameters feed the ISP through the IVC with buffers from the CRU, params and statistics. - Handle completion of in-flight requests: in m2m mode buffers are queued earlier to the ISP capture devices. If the camera is stopped these buffers are returned earlier in error state: complete the Request bypassing the IPA operations. - As cancelled Requests might now be completed earlier then IPA events, modify the IPA event handler to ignore Requests that have been completed already Signed-off-by: Daniel Scally Signed-off-by: Jacopo Mondi --- src/libcamera/pipeline/mali-c55/mali-c55.cpp | 244 +++++++++++++++++++++++---- 1 file changed, 207 insertions(+), 37 deletions(-) diff --git a/src/libcamera/pipeline/mali-c55/mali-c55.cpp b/src/libcamera/pipeline/mali-c55/mali-c55.cpp index be3e38da95ab..10848c412849 100644 --- a/src/libcamera/pipeline/mali-c55/mali-c55.cpp +++ b/src/libcamera/pipeline/mali-c55/mali-c55.cpp @@ -21,6 +21,7 @@ #include #include +#include #include #include #include @@ -88,6 +89,7 @@ struct MaliC55FrameInfo { FrameBuffer *paramBuffer; FrameBuffer *statBuffer; + FrameBuffer *rawBuffer; bool paramsDone; bool statsDone; @@ -691,6 +693,7 @@ public: void imageBufferReady(FrameBuffer *buffer); void paramsBufferReady(FrameBuffer *buffer); void statsBufferReady(FrameBuffer *buffer); + void cruBufferReady(FrameBuffer *buffer); void paramsComputed(unsigned int requestId, uint32_t bytesused); void statsProcessed(unsigned int requestId, const ControlList &metadata); @@ -739,7 +742,7 @@ private: MaliC55FrameInfo *findFrameInfo(FrameBuffer *buffer); MaliC55FrameInfo *findFrameInfo(Request *request); - void tryComplete(MaliC55FrameInfo *info); + void tryComplete(MaliC55FrameInfo *info, bool cancelled = false); int configureRawStream(MaliC55CameraData *data, const StreamConfiguration &config, @@ -747,6 +750,11 @@ private: int configureProcessedStream(MaliC55CameraData *data, const StreamConfiguration &config, V4L2SubdeviceFormat &subdevFormat); + void cancelPendingRequests(); + + MaliC55FrameInfo * + prepareFrameInfo(Request *request, RZG2LCRU *cru = nullptr); + int queuePendingRequests(MaliC55CameraData *data); void applyScalerCrop(Camera *camera, const ControlList &controls); @@ -772,6 +780,9 @@ private: std::map frameInfoMap_; + /* Requests for which no buffer has been queued to the CRU device yet. */ + std::queue pendingRequests_; + std::array pipes_; bool dsFitted_; @@ -1220,6 +1231,13 @@ void PipelineHandlerMaliC55::freeBuffers(Camera *camera) if (params_->releaseBuffers()) LOG(MaliC55, Error) << "Failed to release params buffers"; + if (auto *mem = std::get_if(&data->input_)) { + if (ivc_->releaseBuffers()) + LOG(MaliC55, Error) << "Failed to release input buffers"; + if (mem->cru_->freeBuffers()) + LOG(MaliC55, Error) << "Failed to release CRU buffers"; + } + return; } @@ -1249,6 +1267,12 @@ int PipelineHandlerMaliC55::allocateBuffers(Camera *camera) } }; + if (std::holds_alternative(data->input_)) { + ret = ivc_->importBuffers(RZG2LCRU::kBufferCount); + if (ret < 0) + return ret; + } + ret = stats_->allocateBuffers(bufferCount, &statsBuffers_); if (ret < 0) return ret; @@ -1280,6 +1304,24 @@ int PipelineHandlerMaliC55::start(Camera *camera, [[maybe_unused]] const Control if (ret) return ret; + if (auto *mem = std::get_if(&data->input_)) { + ret = mem->cru_->start(); + if (ret) { + LOG(MaliC55, Error) + << "Failed to start CRU " << camera->id(); + freeBuffers(camera); + return ret; + } + + ret = ivc_->streamOn(); + if (ret) { + LOG(MaliC55, Error) + << "Failed to start IVC" << camera->id(); + freeBuffers(camera); + return ret; + } + } + if (data->ipa_) { ret = data->ipa_->start(); if (ret) { @@ -1355,12 +1397,25 @@ int PipelineHandlerMaliC55::start(Camera *camera, [[maybe_unused]] const Control return 0; } +void PipelineHandlerMaliC55::cancelPendingRequests() +{ + while (!pendingRequests_.empty()) { + cancelRequest(pendingRequests_.front()); + pendingRequests_.pop(); + } +} + void PipelineHandlerMaliC55::stopDevice(Camera *camera) { MaliC55CameraData *data = cameraData(camera); isp_->setFrameStartEnabled(false); + if (auto *mem = std::get_if(&data->input_)) { + ivc_->streamOff(); + mem->cru_->stop(); + } + for (MaliC55Pipe &pipe : pipes_) { if (!pipe.stream) continue; @@ -1374,6 +1429,8 @@ void PipelineHandlerMaliC55::stopDevice(Camera *camera) if (data->ipa_) data->ipa_->stop(); freeBuffers(camera); + + cancelPendingRequests(); } void PipelineHandlerMaliC55::applyScalerCrop(Camera *camera, @@ -1472,10 +1529,85 @@ void PipelineHandlerMaliC55::applyScalerCrop(Camera *camera, } } +MaliC55FrameInfo * +PipelineHandlerMaliC55::prepareFrameInfo(Request *request, RZG2LCRU *cru) +{ + if (availableStatsBuffers_.empty()) { + LOG(MaliC55, Error) << "Stats buffer underrun"; + return nullptr; + } + + if (availableParamsBuffers_.empty()) { + LOG(MaliC55, Error) << "Params buffer underrun"; + return nullptr; + } + + MaliC55FrameInfo &frameInfo = frameInfoMap_[request->sequence()]; + frameInfo.request = request; + + if (cru) { + frameInfo.rawBuffer = cru->queueBuffer(request); + if (!frameInfo.rawBuffer) + return nullptr; + } + + frameInfo.statBuffer = availableStatsBuffers_.front(); + availableStatsBuffers_.pop(); + frameInfo.paramBuffer = availableParamsBuffers_.front(); + availableParamsBuffers_.pop(); + + frameInfo.paramsDone = false; + frameInfo.statsDone = false; + + return &frameInfo; +} + +int PipelineHandlerMaliC55::queuePendingRequests(MaliC55CameraData *data) +{ + auto *mem = std::get_if(&data->input_); + ASSERT(mem); + + while (!pendingRequests_.empty()) { + Request *request = pendingRequests_.front(); + + if (!prepareFrameInfo(request, mem->cru_.get())) + return -ENOENT; + + for (auto &[stream, buffer] : request->buffers()) { + MaliC55Pipe *pipe = pipeFromStream(data, stream); + + pipe->cap->queueBuffer(buffer); + } + + data->ipa_->queueRequest(request->sequence(), request->controls()); + + pendingRequests_.pop(); + } + + return 0; +} + int PipelineHandlerMaliC55::queueRequestDevice(Camera *camera, Request *request) { MaliC55CameraData *data = cameraData(camera); + /* + * If we're in memory input mode, we need to pop the requests onto the + * pending list until a CRU buffer is ready...otherwise we can just do + * everything immediately. + */ + if (std::holds_alternative(data->input_)) { + pendingRequests_.push(request); + + int ret = queuePendingRequests(data); + if (ret) { + pendingRequests_.pop(); + return ret; + } + + return 0; + } + /* Do not run the IPA if the TPG is in use. */ if (!data->ipa_) { MaliC55FrameInfo frameInfo; @@ -1496,32 +1628,13 @@ int PipelineHandlerMaliC55::queueRequestDevice(Camera *camera, Request *request) return 0; } - if (availableStatsBuffers_.empty()) { - LOG(MaliC55, Error) << "Stats buffer underrun"; - return -ENOENT; - } - - if (availableParamsBuffers_.empty()) { - LOG(MaliC55, Error) << "Params buffer underrun"; + auto frameInfo = prepareFrameInfo(request); + if (!frameInfo) return -ENOENT; - } - - MaliC55FrameInfo frameInfo; - frameInfo.request = request; - - frameInfo.statBuffer = availableStatsBuffers_.front(); - availableStatsBuffers_.pop(); - frameInfo.paramBuffer = availableParamsBuffers_.front(); - availableParamsBuffers_.pop(); - - frameInfo.paramsDone = false; - frameInfo.statsDone = false; - - frameInfoMap_[request->sequence()] = frameInfo; data->ipa_->queueRequest(request->sequence(), request->controls()); data->ipa_->fillParams(request->sequence(), - frameInfo.paramBuffer->cookie()); + frameInfo->paramBuffer->cookie()); return 0; } @@ -1540,18 +1653,21 @@ MaliC55FrameInfo *PipelineHandlerMaliC55::findFrameInfo(FrameBuffer *buffer) { for (auto &[sequence, info] : frameInfoMap_) { if (info.paramBuffer == buffer || - info.statBuffer == buffer) + info.statBuffer == buffer || + info.rawBuffer == buffer) return &info; } return nullptr; } -void PipelineHandlerMaliC55::tryComplete(MaliC55FrameInfo *info) +void PipelineHandlerMaliC55::tryComplete(MaliC55FrameInfo *info, bool cancelled) { - if (!info->paramsDone) - return; - if (!info->statsDone) + /* + * If the buffer has been cancelled, we complete the request without + * waiting for the IPA. + */ + if (!cancelled && (!info->paramsDone || !info->statsDone)) return; Request *request = info->request; @@ -1575,13 +1691,15 @@ void PipelineHandlerMaliC55::imageBufferReady(FrameBuffer *buffer) ASSERT(info); if (completeBuffer(request, buffer)) - tryComplete(info); + tryComplete(info, + buffer->metadata().status == FrameMetadata::FrameCancelled); } void PipelineHandlerMaliC55::paramsBufferReady(FrameBuffer *buffer) { MaliC55FrameInfo *info = findFrameInfo(buffer); - ASSERT(info); + if (!info) + return; info->paramsDone = true; @@ -1591,7 +1709,8 @@ void PipelineHandlerMaliC55::paramsBufferReady(FrameBuffer *buffer) void PipelineHandlerMaliC55::statsBufferReady(FrameBuffer *buffer) { MaliC55FrameInfo *info = findFrameInfo(buffer); - ASSERT(info); + if (!info) + return; Request *request = info->request; MaliC55CameraData *data = cameraData(request->_d()->camera()); @@ -1602,32 +1721,80 @@ void PipelineHandlerMaliC55::statsBufferReady(FrameBuffer *buffer) sensorControls); } +void PipelineHandlerMaliC55::cruBufferReady(FrameBuffer *buffer) +{ + /* + * If the buffer has been cancelled, do not ask the IPA to prepare + * parameters. + * + * The Request this cancelled buffer belongs to will be handled by + * imageBufferReady() as we have queued buffers to the ISP capture + * devices at the same time we have queued this buffer to the CRU + * when running in m2m mode. + */ + if (buffer->metadata().status == FrameMetadata::FrameCancelled) + return; + + MaliC55FrameInfo *info = findFrameInfo(buffer); + ASSERT(info); + + Request *request = info->request; + request->_d()->metadata().set(controls::SensorTimestamp, + buffer->metadata().timestamp); + + /* Ought we do something with the sensor's controls here...? */ + MaliC55CameraData *data = cameraData(request->_d()->camera()); + data->ipa_->fillParams(request->sequence(), info->paramBuffer->cookie()); +} + void PipelineHandlerMaliC55::paramsComputed(unsigned int requestId, uint32_t bytesused) { - MaliC55FrameInfo &frameInfo = frameInfoMap_[requestId]; + auto it = frameInfoMap_.find(requestId); + if (it == frameInfoMap_.end()) + return; + + MaliC55FrameInfo &frameInfo = it->second; Request *request = frameInfo.request; + if (!request) + return; + MaliC55CameraData *data = cameraData(request->_d()->camera()); /* * Queue buffers for stats and params, then queue buffers to the capture - * video devices. + * video devices if we're running in Inline mode or with the TPG. + * + * If we're running in M2M buffers have been queued to the capture + * devices at queuePendingRequests() time and here we only have to queue + * buffers to the IVC input to start a transfer. */ frameInfo.paramBuffer->_d()->metadata().planes()[0].bytesused = bytesused; params_->queueBuffer(frameInfo.paramBuffer); stats_->queueBuffer(frameInfo.statBuffer); - for (auto &[stream, buffer] : request->buffers()) { - MaliC55Pipe *pipe = pipeFromStream(data, stream); + if (!std::holds_alternative(data->input_)) { + for (auto &[stream, buffer] : request->buffers()) { + MaliC55Pipe *pipe = pipeFromStream(data, stream); - pipe->cap->queueBuffer(buffer); + pipe->cap->queueBuffer(buffer); + } + } else { + ivc_->queueBuffer(frameInfo.rawBuffer); + frameInfo.rawBuffer = nullptr; } } void PipelineHandlerMaliC55::statsProcessed(unsigned int requestId, const ControlList &metadata) { - MaliC55FrameInfo &frameInfo = frameInfoMap_[requestId]; + auto it = frameInfoMap_.find(requestId); + if (it == frameInfoMap_.end()) + return; + + MaliC55FrameInfo &frameInfo = it->second; + if (!frameInfo.request) + return; frameInfo.statsDone = true; frameInfo.request->_d()->metadata().merge(metadata); @@ -1757,6 +1924,9 @@ bool PipelineHandlerMaliC55::registerMemoryInputCamera() ivc_->bufferReady.connect(mem->cru_.get(), &RZG2LCRU::returnBuffer); + V4L2VideoDevice *cruOutput = mem->cru_->output(); + cruOutput->bufferReady.connect(this, &PipelineHandlerMaliC55::cruBufferReady); + return registerMaliCamera(std::move(data), sensor->device()->entity()->name()); } From patchwork Wed Mar 25 14:44:14 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Jacopo Mondi X-Patchwork-Id: 26341 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id D3C0EC32F9 for ; Wed, 25 Mar 2026 14:44:29 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 684F46283D; Wed, 25 Mar 2026 15:44:27 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="JyPWibXy"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 13B606274D for ; Wed, 25 Mar 2026 15:44:21 +0100 (CET) Received: from [192.168.1.104] (net-93-65-100-155.cust.vodafonedsl.it [93.65.100.155]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 0A0071943; Wed, 25 Mar 2026 15:43:03 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1774449783; bh=LATrMea+GPrXRekBlXe2H8bkzQeumpp9rKZdniQ4tDo=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=JyPWibXyptsIg76YU346NCcHErRlKfKzbUWNBqyLqbK9iWFQtqjVDRaa/szOrZ9xs dqQk83eWgj7HytIP7vfSYEo73t/LCwoKkNcGE/v5bwfkxhaRqh46IaYNtIgSK2O8vX 7XGSu92Ml47uySKQKNp8jw+zQDBdD0vcz2ysHO5w= From: Jacopo Mondi Date: Wed, 25 Mar 2026 15:44:14 +0100 Subject: [PATCH v6 7/7] libcamera: mali-c55: Fix sensor size computation MIME-Version: 1.0 Message-Id: <20260325-mali-cru-v6-7-b16b0c49819a@ideasonboard.com> References: <20260325-mali-cru-v6-0-b16b0c49819a@ideasonboard.com> In-Reply-To: <20260325-mali-cru-v6-0-b16b0c49819a@ideasonboard.com> To: Daniel Scally , libcamera-devel@lists.libcamera.org Cc: =?utf-8?q?Barnab=C3=A1s_P=C5=91cze?= , Jacopo Mondi X-Mailer: b4 0.14.3 X-Developer-Signature: v=1; a=openpgp-sha256; l=1117; i=jacopo.mondi@ideasonboard.com; h=from:subject:message-id; bh=LATrMea+GPrXRekBlXe2H8bkzQeumpp9rKZdniQ4tDo=; b=owEBbQKS/ZANAwAKAXI0Bo8WoVY8AcsmYgBpw/TCqNkulhKYAxGira/OXUZWmhy0laqZUlUoD /lvo4odNuiJAjMEAAEKAB0WIQS1xD1IgJogio9YOMByNAaPFqFWPAUCacP0wgAKCRByNAaPFqFW PDl2D/9s4IbZnonhJ4WOei8PpsDewT0uUESLdGz3wv7gyfkUPcWLswzGwVtGkufzph0N0oTCCoQ corKaqLHBUnJjmtPoizKZkDkKNfgCzwRCu7jAkhehlC6SRhbusGtvnWTKDoEF4mWOqm4IGTi/1/ hr9mlOu7Kbm4Yr8frEKwsKGvC55PlNz2FNYkZlYGuvUQ0mRzGDqapYDuYT0zt8Mtg8gR0YXv3EI 9k4FCXNL34S+FYwZPXUeAm516OSEI121pRnYnuCWWQeYq3ACnn+aDnxhtwA4g1hf5u9Q8pNvEXv nDkh/8IfV5j/F0chAm9uCKgt6mCiW/CGyzfuuraFoDPnHAe5BywY5QLIehBrwGGUt/cP8RJaFMj 2WNm7J9fb/s1ff30eaTNA7YHoBrsvCclcmzje6ugRrDau3edogEE3Tud6kvAP4vshBI/uL7PIPU 99hhbH9/L+IUGmal58QEak1CoK7jK+YMTE5YWOR9tWMQSr2V8mH7/sV1B6g7Mt2iIdqeIEwGs66 Q9FL7mtAkNp4pIdPKwOBxJB3sezDoFhvnEnVXU9bnpPIxg7Dc8Tn4gGBxcTwpyANHFsrusrI0GY rBFzsliwsyIEYuL96mQyqYFV5KK0fL8D0EXGnhXq/RtD47k+trS/48tVN5CRGrbrZeiPGU/Oq1y CEQptqZPWqhue2Q== X-Developer-Key: i=jacopo.mondi@ideasonboard.com; a=openpgp; fpr=72392EDC88144A65C701EA9BA5826A2587AD026B X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" The last size in the list is always selected unconditionally because the current best distance is not updated properly during the search. Fix the size computation procedure which has an inverted assignment. Signed-off-by: Jacopo Mondi Reviewed-by: Daniel Scally Tested-by: Daniel Scally Reviewed-by: Barnabás Pőcze --- src/libcamera/pipeline/mali-c55/mali-c55.cpp | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/libcamera/pipeline/mali-c55/mali-c55.cpp b/src/libcamera/pipeline/mali-c55/mali-c55.cpp index 10848c412849..2ba403fb348e 100644 --- a/src/libcamera/pipeline/mali-c55/mali-c55.cpp +++ b/src/libcamera/pipeline/mali-c55/mali-c55.cpp @@ -660,7 +660,7 @@ CameraConfiguration::Status MaliC55CameraConfiguration::validate() std::abs(static_cast(minSensorSize.height) - static_cast(size.height)); if (dist < distance) { - dist = distance; + distance = dist; bestSize = size; } }