From patchwork Fri Mar 13 16:14:35 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jacopo Mondi X-Patchwork-Id: 26292 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 49A22C32B5 for ; Fri, 13 Mar 2026 16:14:49 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 51E1C626CC; Fri, 13 Mar 2026 17:14:47 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="nHDMBbNY"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 583D46261B for ; Fri, 13 Mar 2026 17:14:45 +0100 (CET) Received: from [192.168.224.131] (unknown [37.159.122.93]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id CE7B8103D; Fri, 13 Mar 2026 17:13:35 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1773418416; bh=EaYAwOaNMgy9veLK7jWa8jDYsMPnnAIy+293/wcP5yk=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=nHDMBbNYvzAvzd8AXUkqkbMyn0PgCcr5hdn3xVWw83ESkQKysNj1+qQjCG6Qz0vVz yg0cbvWqmvnb3k1VVxAN9DkEK5h76vvQHW5VgpaL802JoL62LtcE8vAbMs82rWtU8s gYE3k/X5+5epOlhAsOmexxqoY9Vu/www5GspwtmY= From: Jacopo Mondi Date: Fri, 13 Mar 2026 17:14:35 +0100 Subject: [PATCH v5 1/7] libcamera: mali-c55: Add RZG2LCRU class MIME-Version: 1.0 Message-Id: <20260313-mali-cru-v5-1-48f93e431294@ideasonboard.com> References: <20260313-mali-cru-v5-0-48f93e431294@ideasonboard.com> In-Reply-To: <20260313-mali-cru-v5-0-48f93e431294@ideasonboard.com> To: Daniel Scally , libcamera-devel@lists.libcamera.org Cc: Jacopo Mondi X-Mailer: b4 0.14.3 X-Developer-Signature: v=1; a=openpgp-sha256; l=9931; i=jacopo.mondi@ideasonboard.com; h=from:subject:message-id; bh=ubisEqX/JdmGo8iOHOU9fEWbBd82jLtSOuScAx7m9A4=; b=owEBbQKS/ZANAwAKAXI0Bo8WoVY8AcsmYgBptDfy+kb7CtebyrXvNOFFw/qfdwetvi/HiE/N2 Bb3GZCVysCJAjMEAAEKAB0WIQS1xD1IgJogio9YOMByNAaPFqFWPAUCabQ38gAKCRByNAaPFqFW PLQND/4vzJneNuqbF3XYqcBROwdKAI0SB4dnMB9je3A+Haah3GyvJjB393YDbQGoju6+YXDSezO hPcuqjAaHJqKG24YUkGZWvale8EgFIE7RV1Nj+q9T62EsoUQxgAqdo/miIXeDAcvLIE4/LqAcUB cVE4hVNH/ycjzwr+Ii8+0BwvSm2KgtrIGJgZG/YayHjwc9o9t/pe1z3UMtIjCDjVCpePImFoqTr KRbUu2dsEYNLhdR278jo5FPTGWZsjaIke/vVsG2N39r6yJgEnl51iiXeRv3JfsOu3oq9XYe7RGP hrnGZkNxoRJc6F+THp+x74xbWq7QU1XENQ/BoAJLZVvW/PvS9g6pnyfTtPRnXCNwTXNTzJk1Hh9 6QdxsbckbGVadeq0SoF70IHFE3FquaE/sucU/Viz2E8P/4KWI49GCg2fiUr+91Vg5p3BxCAzcLh UmffO6Ei4REPmurEPeMk8rIuzlLQLcHY9h9WSUgJNL50XpXnYXabzrPWFvO2pVd/fmgSxcG+wZj vfIvSKwyg7HG+riYGQXbyqPbGTvm/be1HXVInfegqaaPfyO4DXO9OWcJHpM+nkVh/iIUOQ6wSO9 6e8j7kGQlgNeDnAGpf34lH3UcplHUf7uZqOUN6nsqCn3Yj/oun0L0Wl8lXdhsPn8zCkebZ+GL7a /PzJe3JMsITUy/g== X-Developer-Key: i=jacopo.mondi@ideasonboard.com; a=openpgp; fpr=72392EDC88144A65C701EA9BA5826A2587AD026B X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" From: Daniel Scally Add a class allowing us to control the RZ/G2L Camera Receiver Unit. The class is named after the media device it handles, which is called "rzg2l_cru" but the CRU is actually found in other SoCs than the RZ/G2L one, such as the RZ/V2H(P). The RZG2LCRU class models the CSI-2 receiver found on those SoCs and allows to add support for memory-to-memory operations on the RZ/V2H(P) SoC which integrates a Mali-C55 ISP and a specialized DMA engine named IVC that feeds images from memory, where the CRU has saved them. Signed-off-by: Daniel Scally Signed-off-by: Jacopo Mondi --- src/libcamera/pipeline/mali-c55/meson.build | 3 +- src/libcamera/pipeline/mali-c55/rzg2l-cru.cpp | 260 ++++++++++++++++++++++++++ src/libcamera/pipeline/mali-c55/rzg2l-cru.h | 73 ++++++++ 3 files changed, 335 insertions(+), 1 deletion(-) diff --git a/src/libcamera/pipeline/mali-c55/meson.build b/src/libcamera/pipeline/mali-c55/meson.build index eba8e5a39054..4e768242602e 100644 --- a/src/libcamera/pipeline/mali-c55/meson.build +++ b/src/libcamera/pipeline/mali-c55/meson.build @@ -1,5 +1,6 @@ # SPDX-License-Identifier: CC0-1.0 libcamera_internal_sources += files([ - 'mali-c55.cpp' + 'mali-c55.cpp', + 'rzg2l-cru.cpp', ]) diff --git a/src/libcamera/pipeline/mali-c55/rzg2l-cru.cpp b/src/libcamera/pipeline/mali-c55/rzg2l-cru.cpp new file mode 100644 index 000000000000..8f09d1c78058 --- /dev/null +++ b/src/libcamera/pipeline/mali-c55/rzg2l-cru.cpp @@ -0,0 +1,260 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2025, Ideas on Board Oy + * + * Pipine handler element for the Renesas RZ/G2L Camera Receiver Unit + */ + +#include "rzg2l-cru.h" + +#include +#include + +#include + +#include +#include +#include + +#include +#include + +#include "libcamera/internal/bayer_format.h" +#include "libcamera/internal/camera_sensor.h" +#include "libcamera/internal/framebuffer.h" +#include "libcamera/internal/media_device.h" +#include "libcamera/internal/request.h" + +namespace libcamera { + +static const std::map bitDepthToFmt{ + { 10, V4L2PixelFormat(V4L2_PIX_FMT_RAW_CRU10) }, + { 12, V4L2PixelFormat(V4L2_PIX_FMT_RAW_CRU12) }, + { 14, V4L2PixelFormat(V4L2_PIX_FMT_RAW_CRU14) }, +}; + +LOG_DEFINE_CATEGORY(RZG2LCRU) + +FrameBuffer *RZG2LCRU::queueBuffer(Request *request) +{ + FrameBuffer *buffer; + + if (availableBuffers_.empty()) { + LOG(RZG2LCRU, Debug) << "CRU buffer underrun"; + return nullptr; + } + + buffer = availableBuffers_.front(); + + int ret = output_->queueBuffer(buffer); + if (ret) { + LOG(RZG2LCRU, Error) << "Failed to queue buffer to CRU"; + return nullptr; + } + + availableBuffers_.pop(); + buffer->_d()->setRequest(request); + + return buffer; +} + +void RZG2LCRU::cruReturnBuffer(FrameBuffer *buffer) +{ + auto it = std::find_if(buffers_.begin(), buffers_.end(), + [&](const auto &b) { return b.get() == buffer; }); + ASSERT(it != buffers_.end()); + + availableBuffers_.push(buffer); +} + +int RZG2LCRU::start() +{ + int ret = output_->exportBuffers(kBufferCount, &buffers_); + if (ret < 0) + return ret; + + utils::scope_exit bufferGuard([&] { freeBuffers(); }); + + ret = output_->importBuffers(kBufferCount); + if (ret) + return ret; + + for (std::unique_ptr &buffer : buffers_) + availableBuffers_.push(buffer.get()); + + ret = output_->streamOn(); + if (ret) + return ret; + + bufferGuard.release(); + + return 0; +} + +int RZG2LCRU::stop() +{ + int ret; + + ret = output_->streamOff(); + + freeBuffers(); + + return ret; +} + +void RZG2LCRU::freeBuffers() +{ + availableBuffers_ = {}; + buffers_.clear(); + + if (output_->releaseBuffers()) + LOG(RZG2LCRU, Error) << "Failed to release CRU buffers"; +} + +int RZG2LCRU::configure(V4L2SubdeviceFormat *subdevFormat, V4L2DeviceFormat *inputFormat) +{ + /* + * Set format on the sensor and propagate it up to the CRU video + * device. + */ + + int ret = sensor_->setFormat(subdevFormat); + if (ret) + return ret; + + ret = csi2_->setFormat(0, subdevFormat); + if (ret) + return ret; + + ret = csi2_->getFormat(1, subdevFormat); + if (ret) + return ret; + + ret = cru_->setFormat(0, subdevFormat); + if (ret) + return ret; + + ret = cru_->getFormat(1, subdevFormat); + if (ret) + return ret; + + /* + * The capture device needs to be set with a format that can be produced + * from the mbus code of the subdevFormat. The CRU and IVC use bayer + * order agnostic pixel formats, so all we need to do is find the right + * bitdepth and select the appropriate format. + */ + BayerFormat bayerFormat = BayerFormat::fromMbusCode(subdevFormat->code); + if (!bayerFormat.isValid()) + return -EINVAL; + + V4L2DeviceFormat captureFormat; + captureFormat.fourcc = bitDepthToFmt.at(bayerFormat.bitDepth); + captureFormat.size = subdevFormat->size; + + ret = output_->setFormat(&captureFormat); + if (ret) + return ret; + + /* + * We return the format that we set against the output device, as the + * same format will also need to be set against the Input Video Control + * Block device. + */ + *inputFormat = captureFormat; + + return 0; +} + +void RZG2LCRU::initCRUSizes() +{ + Size maxCSI2Size; + + /* + * Get the maximum supported size on the CSI-2 receiver. We need to + * query the kernel interface as the size limits differ from RZ/G2L + * (2800x4095) and RZ/V2H (4096x4096). + */ + V4L2Subdevice::Formats csi2Formats = csi2_->formats(0); + if (csi2Formats.empty()) + return; + + for (const auto &format : csi2Formats) { + for (const auto &range : format.second) { + if (range.max > maxCSI2Size) + maxCSI2Size = range.max; + } + } + + /* + * Enumerate the sensor supported resolutions and filter out the ones + * larger than the maximum supported CSI-2 receiver input size. + */ + V4L2Subdevice::Formats formats = sensor_->device()->formats(0); + if (formats.empty()) + return; + + for (const auto &format : formats) { + for (const auto &range : format.second) { + const Size &max = range.max; + + if (max.width > maxCSI2Size.width || + max.height > maxCSI2Size.height) + continue; + + csi2Sizes_.push_back(max); + } + } + + /* Sort in increasing order and remove duplicates. */ + std::sort(csi2Sizes_.begin(), csi2Sizes_.end()); + auto last = std::unique(csi2Sizes_.begin(), csi2Sizes_.end()); + csi2Sizes_.erase(last, csi2Sizes_.end()); + + csi2Resolution_ = csi2Sizes_.back(); +} + +int RZG2LCRU::init(const MediaDevice *media) +{ + static const std::regex cruRegex("cru-ip-[0-9a-f]{8}.cru[0-9]"); + static const std::regex csi2Regex("csi-[0-9a-f]{8}.csi2"); + + csi2_ = V4L2Subdevice::fromEntityName(media, csi2Regex); + if (!csi2_) + return -ENODEV; + + int ret = csi2_->open(); + if (ret) + return ret; + + const std::vector &pads = csi2_->entity()->pads(); + if (pads.empty()) + return -ENODEV; + + /* The receiver has a single sink pad at index 0 */ + MediaPad *sink = pads[0]; + const std::vector &links = sink->links(); + if (links.empty()) + return -ENODEV; + + MediaLink *link = links[0]; + sensor_ = CameraSensorFactoryBase::create(link->source()->entity()); + if (!sensor_) + return -ENODEV; + + cru_ = V4L2Subdevice::fromEntityName(media, cruRegex); + ret = cru_->open(); + if (ret) + return ret; + + output_ = V4L2VideoDevice::fromEntityName(media, "CRU output"); + ret = output_->open(); + if (ret) + return ret; + + initCRUSizes(); + + return 0; +} + +} /* namespace libcamera */ diff --git a/src/libcamera/pipeline/mali-c55/rzg2l-cru.h b/src/libcamera/pipeline/mali-c55/rzg2l-cru.h new file mode 100644 index 000000000000..1e3cdd879cc5 --- /dev/null +++ b/src/libcamera/pipeline/mali-c55/rzg2l-cru.h @@ -0,0 +1,73 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2025, Ideas on Board Oy + * + * Pipine handler element for the Renesas RZ/G2L Camera Receiver Unit + */ + +#pragma once + +#include +#include +#include + +#include + +#include "libcamera/internal/v4l2_subdevice.h" +#include "libcamera/internal/v4l2_videodevice.h" + +namespace libcamera { + +class CameraSensor; +class FrameBuffer; +class MediaDevice; +class Request; +class Size; + +class RZG2LCRU +{ +public: + static constexpr unsigned int kBufferCount = 4; + + RZG2LCRU() = default; + + const std::vector &sizes() const + { + return csi2Sizes_; + } + + int init(const MediaDevice *media); + const Size &resolution() const + { + return csi2Resolution_; + } + + CameraSensor *sensor() const { return sensor_.get(); } + V4L2Subdevice *csi2() const { return csi2_.get(); } + V4L2Subdevice *cru() const { return cru_.get(); } + + int configure(V4L2SubdeviceFormat *subdevFormat, V4L2DeviceFormat *inputFormat); + FrameBuffer *queueBuffer(Request *request); + void cruReturnBuffer(FrameBuffer *buffer); + V4L2VideoDevice *output() { return output_.get(); } + int start(); + int stop(); + +private: + void freeBuffers(); + + void initCRUSizes(); + + std::unique_ptr sensor_; + std::unique_ptr csi2_; + std::unique_ptr cru_; + std::unique_ptr output_; + + std::vector> buffers_; + std::queue availableBuffers_; + + std::vector csi2Sizes_; + Size csi2Resolution_; +}; + +} /* namespace libcamera */ From patchwork Fri Mar 13 16:14:36 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Jacopo Mondi X-Patchwork-Id: 26293 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 1EA2BC32EF for ; Fri, 13 Mar 2026 16:14:50 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id D9A7262703; Fri, 13 Mar 2026 17:14:47 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="hJJH6skD"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 40C6E62695 for ; Fri, 13 Mar 2026 17:14:46 +0100 (CET) Received: from [192.168.224.131] (unknown [37.159.122.93]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 84A32E70; Fri, 13 Mar 2026 17:13:36 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1773418417; bh=FegylNAS/ZE8S1oZUyAdmGxPOxc4ElLR/QAsnKcxhNE=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=hJJH6skDqHReuyo0FSM+y6SKefdwe3jelp0UB5pcjtkJXPjg30R8Uyh85ECEYyUxV gmv1q5+93gZhjeJAkhPrWCKT6aYFR8V4ElGV35PZY9Kd1EiUaG7wtV5iX02SvxRbwn BhIDjTXKz2y8KC0V5xgFUSB36yyXApr3KOpGqbiY= From: Jacopo Mondi Date: Fri, 13 Mar 2026 17:14:36 +0100 Subject: [PATCH v5 2/7] libcamera: utils: Add overloaded visitor helpers MIME-Version: 1.0 Message-Id: <20260313-mali-cru-v5-2-48f93e431294@ideasonboard.com> References: <20260313-mali-cru-v5-0-48f93e431294@ideasonboard.com> In-Reply-To: <20260313-mali-cru-v5-0-48f93e431294@ideasonboard.com> To: Daniel Scally , libcamera-devel@lists.libcamera.org Cc: Jacopo Mondi , =?utf-8?b?QmFybmFiw6Fz?= =?utf-8?q?_P=C5=91cze?= X-Mailer: b4 0.14.3 X-Developer-Signature: v=1; a=openpgp-sha256; l=4147; i=jacopo.mondi@ideasonboard.com; h=from:subject:message-id; bh=FegylNAS/ZE8S1oZUyAdmGxPOxc4ElLR/QAsnKcxhNE=; b=owEBbQKS/ZANAwAKAXI0Bo8WoVY8AcsmYgBptDfyxMGFcVvnzbZWAtJsA0zxYr5YnAHMwzpAc XRcjLPr4jKJAjMEAAEKAB0WIQS1xD1IgJogio9YOMByNAaPFqFWPAUCabQ38gAKCRByNAaPFqFW PBwnEACXOfwUsFkCcGkNTTosPRvJsmU6YsOFAR68NOgoAKprBKNsRXVavn8D/XdSTXtwUHWpxBX BdyQnI4Re0ATaLfV/vqqA+IDLVXootjCvi3o+q6Sr+PuCVbVelNJUkCXpfMDgA7TIpBO3VK5rUE NobCIWdH/83Z2niRwZv3+dN6otjMiOm+AGgeNTutVfxjwpnDA1xndYYPskWG1qOKvpW5Y+ebJl8 d7PEnF1tQ8wLGT7fg1u8RK0sShSdYoQ09Xyfng8EvRFyO4EWlYrcxrfB8YW2uWY+5gdWhGT6AOs u38jLV1wnkWHwFLNJSwc7OTWTSwTRfvabmlI2eJxMJ3jlsNGPuyvzlDHKNERrzZ81MtkvanuClH 1QjW4rjgc0dQfpcF29hMGfq8Z9TP+LIFADzj1kgwzB6iyTYbMRVwvdJjdDNDGbt7jJB4zUjdHMu /6Y2h25SRsS4WcaZxxyU/B713+z9zwDKgakuXYb23pGvRkkhv6G2bwY3OOsIA/9FOSaDOtYhCuw wfuCyP5L8UARNb+8NTblEPRnDbxkNJ5z8ndrlbxd6u+Vod0AciqZn/6c3UMhbD7ua1ksBHapkgr 5fq1PuN/eOCs38RRKD3iTf+tf3pTnkfpo8KMvBPKGb582gww2mD1BeFCHtt3EBUXZNnJqoPfP1d HQEHbKyb5moQy4g== X-Developer-Key: i=jacopo.mondi@ideasonboard.com; a=openpgp; fpr=72392EDC88144A65C701EA9BA5826A2587AD026B X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" std::visit() allows quite elegant type-matching implementation of the visitor pattern. The 'overloaded' type helpers allow to define a hierarchy of overloaded operator() implementations which can be used by std::visit(). Currently only the Virtual pipeline handler uses this type-matching implementation of std::visit. To prepare to add another user in the Mali C55 pipeline handler move the 'overloaded' helper type to libcamera::utils for easier re-use. Reviewed-by: Barnabás Pőcze Signed-off-by: Jacopo Mondi --- include/libcamera/base/utils.h | 7 ++++++ src/libcamera/base/utils.cpp | 35 ++++++++++++++++++++++++++++++ src/libcamera/pipeline/virtual/virtual.cpp | 10 ++------- 3 files changed, 44 insertions(+), 8 deletions(-) diff --git a/include/libcamera/base/utils.h b/include/libcamera/base/utils.h index 7083b7ce9ce9..62c7f89c25a1 100644 --- a/include/libcamera/base/utils.h +++ b/include/libcamera/base/utils.h @@ -37,6 +37,13 @@ namespace libcamera { namespace utils { +template +struct overloaded : Ts... { + using Ts::operator()...; +}; +template +overloaded(Ts...) -> overloaded; + const char *basename(const char *path); char *secure_getenv(const char *name); diff --git a/src/libcamera/base/utils.cpp b/src/libcamera/base/utils.cpp index 42a516097be2..836aec87c539 100644 --- a/src/libcamera/base/utils.cpp +++ b/src/libcamera/base/utils.cpp @@ -23,6 +23,41 @@ namespace libcamera { namespace utils { +/** + * \struct overloaded + * \brief Helper type for type-matching std::visit implementations + * \tparam Ts... Template arguments pack of visitors + * + * Expand the template argument pack \a Ts... to provide overloaded \a + * operator() to support type-matching implementations of the visitor design + * pattern using std::visit. + * + * An example is provided by the STL documentation in the form of: + * + * \code{.cpp} + * template struct overloaded : Ts... { using Ts::operator()...; }; + * template overloaded(Ts...) -> overloaded; + * + * using var_t = std::variant; + * std::vector vec = {10, 15l, 1.5, "hello"}; + * + * for (auto& v: vec) { + * std::visit(overloaded { + * [](auto arg) { std::cout << arg << ' '; }, + * [](double arg) { std::cout << std::fixed << arg << ' '; }, + * [](const std::string& arg) { std::cout << std::quoted(arg) << ' '; }, + * }, v); + * \endcode + * + * Use this helper to implement type-matching visitors using std::visit(). + */ + +/** + * \var overloaded(Ts...) -> overloaded + * \brief Deduction guide necessary for C++17 compatibility + * \tparam Ts... Template arguments pack of visitor functions + */ + /** * \brief Strip the directory prefix from the path * \param[in] path The path to process diff --git a/src/libcamera/pipeline/virtual/virtual.cpp b/src/libcamera/pipeline/virtual/virtual.cpp index efd800ebe3d6..e8ef7e524ccf 100644 --- a/src/libcamera/pipeline/virtual/virtual.cpp +++ b/src/libcamera/pipeline/virtual/virtual.cpp @@ -23,6 +23,7 @@ #include #include +#include #include #include @@ -57,13 +58,6 @@ uint64_t currentTimestamp() } /* namespace */ -template -struct overloaded : Ts... { - using Ts::operator()...; -}; -template -overloaded(Ts...) -> overloaded; - class VirtualCameraConfiguration : public CameraConfiguration { public: @@ -428,7 +422,7 @@ bool PipelineHandlerVirtual::initFrameGenerator(Camera *camera) { auto data = cameraData(camera); auto &frame = data->config_.frame; - std::visit(overloaded{ + std::visit(utils::overloaded{ [&](TestPattern &testPattern) { for (auto &streamConfig : data->streamConfigs_) { if (testPattern == TestPattern::DiagonalLines) From patchwork Fri Mar 13 16:14:37 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jacopo Mondi X-Patchwork-Id: 26294 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id DCCC7BE086 for ; Fri, 13 Mar 2026 16:14:51 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 8B645626B7; Fri, 13 Mar 2026 17:14:51 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="o9voGf8B"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 937B7626EE for ; Fri, 13 Mar 2026 17:14:49 +0100 (CET) Received: from [192.168.224.131] (unknown [37.159.122.93]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id B6CB6E70; Fri, 13 Mar 2026 17:13:37 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1773418420; bh=VMML+UEQKCrQcKwjRg7VzpyZuxK6btGaRZfOylECzsQ=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=o9voGf8Bi4CpTIFfF2b8Dpq65WsVE/HHw2vdUbVXHgGOAWdQ242eaP7NOsucTk2Gw aTItqc9SNU/oA2RkcfXVT6ekGu9mIkBwVeHY5qfWc/8U1wXwSgfB0WjmCtAZcTPFMP xbpcDbee1tYaRHfrQvHuRMi0Sq/uwP2G3i4WcaL4= From: Jacopo Mondi Date: Fri, 13 Mar 2026 17:14:37 +0100 Subject: [PATCH v5 3/7] libcamera: mali-c55: Split TPG and Inline camera handling MIME-Version: 1.0 Message-Id: <20260313-mali-cru-v5-3-48f93e431294@ideasonboard.com> References: <20260313-mali-cru-v5-0-48f93e431294@ideasonboard.com> In-Reply-To: <20260313-mali-cru-v5-0-48f93e431294@ideasonboard.com> To: Daniel Scally , libcamera-devel@lists.libcamera.org Cc: Jacopo Mondi X-Mailer: b4 0.14.3 X-Developer-Signature: v=1; a=openpgp-sha256; l=16415; i=jacopo.mondi@ideasonboard.com; h=from:subject:message-id; bh=VMML+UEQKCrQcKwjRg7VzpyZuxK6btGaRZfOylECzsQ=; b=owEBbQKS/ZANAwAKAXI0Bo8WoVY8AcsmYgBptDfz/1TClF3UgzWLIq2X1Lb2dFVfvysCOFDpJ S2/4hNhlbmJAjMEAAEKAB0WIQS1xD1IgJogio9YOMByNAaPFqFWPAUCabQ38wAKCRByNAaPFqFW PCnlEAChukTNobAMAoj/a4kmXOOIsjGBSJl+oyL4vFxgM5fNUsY7bkJTokx+g6grVij1LQ8CeIo KMGj1Vikmlt7Mc8MZdC6wzRWMqJ8tqP/OZjIbyLCUsmv3VNbV422obO6OGMipmke9mfZ9nFoSe4 t9BzzWFYttkAVc6XAdgYYAtsMLJ141hAb+iUYwjY99DXsJQMWUBhmp4QeEKfvdLzAMIe9zXGeKr AI/eJZNAfb+fvF7aDfkef07hE2sFUR3/rNsCpj4fE4xP9ICeMoIh/yjsZe1tehUK45PIzOkLv/G +NjHO7Tixg6p+dzU+F141Wl+jyK3tcOVrdB+NnPTcSRR9iRl+MlzOymWseV2vjkTK4Oz+ecvwIi SHJ/rmRy7dr8wdVF+61WBLPYWjT7Uct3hQaJnQg58BQxk/QRfZ5V/4F0sk+yDsZ12OUvXt6MXZF NRt4PcjNfwNUIhdSId0KsmlXc+4F2e2DONPQJxM4pE17O55yDcIM9eUfRE+u24+HENEKrkqE44m S7nnivvwis0Tbsf9h3xspBCGSWRa6yWIWqjTmDOnxLwT9JeJThrnIoGFXdGWKP/6E57DziRUO6C V9jeg3BclLYj+t5VNuyvd5PcPXrdf1f43cG8UAxpDTCj7/bAfwHZGXz4hgnF9lQ0hESHJMHbQFG POI8tr6QDU9mrHA== X-Developer-Key: i=jacopo.mondi@ideasonboard.com; a=openpgp; fpr=72392EDC88144A65C701EA9BA5826A2587AD026B X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" In order to prepare to support memory input cameras, split the handling of the TPG and Inline camera cases. The Mali C55 pipeline handler uses the entity and subdevice stored in the CameraData to support both the handling of the TPG and of the Inline (CSI-2 + sensor) use cases. Adding support for memory cameras by using the CRU unit would add yet-another special case making the code harder to follow and more prone to errors. Split the handling of the TPG and Inline cameras by introducing an std::variant<> variable and to deflect the functions called on the camera data to the correct type by using overloaded std::visit<>(). Signed-off-by: Jacopo Mondi --- src/libcamera/pipeline/mali-c55/mali-c55.cpp | 298 ++++++++++++++++----------- 1 file changed, 181 insertions(+), 117 deletions(-) diff --git a/src/libcamera/pipeline/mali-c55/mali-c55.cpp b/src/libcamera/pipeline/mali-c55/mali-c55.cpp index c209b0b070b1..45e23ffd3841 100644 --- a/src/libcamera/pipeline/mali-c55/mali-c55.cpp +++ b/src/libcamera/pipeline/mali-c55/mali-c55.cpp @@ -11,12 +11,14 @@ #include #include #include +#include #include #include #include #include +#include #include #include @@ -92,17 +94,90 @@ struct MaliC55FrameInfo { class MaliC55CameraData : public Camera::Private { public: - MaliC55CameraData(PipelineHandler *pipe, MediaEntity *entity) - : Camera::Private(pipe), entity_(entity) + struct Tpg { + std::vector sizes(unsigned int mbusCode) const; + + Size resolution_; + std::unique_ptr sd_; + }; + + struct Inline { + std::unique_ptr csi2_; + std::unique_ptr sensor_; + }; + using CameraType = std::variant; + + MaliC55CameraData(PipelineHandler *pipe) + : Camera::Private(pipe) { } - int init(); int loadIPA(); - /* Deflect these functionalities to either TPG or CameraSensor. */ - std::vector sizes(unsigned int mbusCode) const; - Size resolution() const; + int initTpg(MediaEntity *entity); + int initInline(MediaEntity *entity); + + std::vector sizes(unsigned int mbusCode) const + { + return std::visit(utils::overloaded{ + [&](const Tpg &tpg) -> std::vector { + return tpg.sizes(mbusCode); + }, + [&](const Inline &in) -> std::vector { + return in.sensor_->sizes(mbusCode); + } + }, input_); + } + + V4L2Subdevice *subdev() const + { + return std::visit(utils::overloaded{ + [&](const Tpg &tpg) -> V4L2Subdevice * { + return tpg.sd_.get(); + }, + [&](const Inline &in) -> V4L2Subdevice * { + return in.sensor_->device(); + } + }, input_); + } + + CameraSensor *sensor() const + { + return std::visit(utils::overloaded{ + [&](auto &) -> CameraSensor * { + ASSERT(false); + return nullptr; + }, + [&](const Inline &in) -> CameraSensor * { + return in.sensor_.get(); + } + }, input_); + } + + V4L2Subdevice *csi2() const + { + return std::visit(utils::overloaded{ + [&](auto &) -> V4L2Subdevice * { + ASSERT(false); + return nullptr; + }, + [&](const Inline &in) -> V4L2Subdevice * { + return in.csi2_.get(); + } + }, input_); + } + + Size resolution() const + { + return std::visit(utils::overloaded{ + [&](const Tpg &tpg) -> Size { + return tpg.resolution_; + }, + [&](const Inline &in) -> Size { + return in.sensor_->resolution(); + } + }, input_); + } int pixfmtToMbusCode(const PixelFormat &pixFmt) const; const PixelFormat &bestRawFormat() const; @@ -112,11 +187,6 @@ public: PixelFormat adjustRawFormat(const PixelFormat &pixFmt) const; Size adjustRawSizes(const PixelFormat &pixFmt, const Size &rawSize) const; - std::unique_ptr sensor_; - - MediaEntity *entity_; - std::unique_ptr csi_; - std::unique_ptr sd_; Stream frStream_; Stream dsStream_; @@ -126,58 +196,28 @@ public: std::unique_ptr delayedCtrls_; + CameraType input_; + private: - void initTPGData(); void setSensorControls(const ControlList &sensorControls); - std::string id_; - Size tpgResolution_; }; -int MaliC55CameraData::init() +int MaliC55CameraData::initTpg(MediaEntity *entity) { - int ret; + Tpg tpg; - sd_ = std::make_unique(entity_); - ret = sd_->open(); + tpg.sd_ = std::make_unique(entity); + int ret = tpg.sd_->open(); if (ret) { - LOG(MaliC55, Error) << "Failed to open sensor subdevice"; + LOG(MaliC55, Error) << "Failed to open TPG subdevice"; return ret; } - /* If this camera is created from TPG, we return here. */ - if (entity_->name() == "mali-c55 tpg") { - initTPGData(); - return 0; - } - - /* - * Register a CameraSensor if we connect to a sensor and create - * an entity for the connected CSI-2 receiver. - */ - sensor_ = CameraSensorFactoryBase::create(entity_); - if (!sensor_) - return -ENODEV; - - const MediaPad *sourcePad = entity_->getPadByIndex(0); - MediaEntity *csiEntity = sourcePad->links()[0]->sink()->entity(); - - csi_ = std::make_unique(csiEntity); - ret = csi_->open(); - if (ret) { - LOG(MaliC55, Error) << "Failed to open CSI-2 subdevice"; - return ret; - } - - return 0; -} - -void MaliC55CameraData::initTPGData() -{ /* Replicate the CameraSensor implementation for TPG. */ - V4L2Subdevice::Formats formats = sd_->formats(0); + V4L2Subdevice::Formats formats = tpg.sd_->formats(0); if (formats.empty()) - return; + return -EINVAL; std::vector tpgSizes; @@ -187,19 +227,39 @@ void MaliC55CameraData::initTPGData() [](const SizeRange &range) { return range.max; }); } - tpgResolution_ = tpgSizes.back(); + tpg.resolution_ = tpgSizes.back(); + + input_.emplace(std::move(tpg)); + + return 0; } -void MaliC55CameraData::setSensorControls(const ControlList &sensorControls) +int MaliC55CameraData::initInline(MediaEntity *sensor) { - delayedCtrls_->push(sensorControls); + Inline in; + + /* Register a CameraSensor and create an entity for the CSI-2 receiver. */ + in.sensor_ = CameraSensorFactoryBase::create(sensor); + if (!in.sensor_) + return -EINVAL; + + const MediaPad *sourcePad = sensor->getPadByIndex(0); + MediaEntity *csiEntity = sourcePad->links()[0]->sink()->entity(); + + in.csi2_ = std::make_unique(csiEntity); + int ret = in.csi2_->open(); + if (ret) { + LOG(MaliC55, Error) << "Failed to open CSI-2 subdevice"; + return ret; + } + + input_.emplace(std::move(in)); + + return 0; } -std::vector MaliC55CameraData::sizes(unsigned int mbusCode) const +std::vector MaliC55CameraData::Tpg::sizes(unsigned int mbusCode) const { - if (sensor_) - return sensor_->sizes(mbusCode); - V4L2Subdevice::Formats formats = sd_->formats(0); if (formats.empty()) return {}; @@ -218,12 +278,9 @@ std::vector MaliC55CameraData::sizes(unsigned int mbusCode) const return sizes; } -Size MaliC55CameraData::resolution() const +void MaliC55CameraData::setSensorControls(const ControlList &sensorControls) { - if (sensor_) - return sensor_->resolution(); - - return tpgResolution_; + delayedCtrls_->push(sensorControls); } /* @@ -242,7 +299,7 @@ int MaliC55CameraData::pixfmtToMbusCode(const PixelFormat &pixFmt) const if (!bayerFormat.isValid()) return -EINVAL; - V4L2Subdevice::Formats formats = sd_->formats(0); + V4L2Subdevice::Formats formats = subdev()->formats(0); unsigned int sensorMbusCode = 0; unsigned int bitDepth = 0; @@ -280,7 +337,7 @@ const PixelFormat &MaliC55CameraData::bestRawFormat() const { static const PixelFormat invalidPixFmt = {}; - for (const auto &fmt : sd_->formats(0)) { + for (const auto &fmt : subdev()->formats(0)) { BayerFormat sensorBayer = BayerFormat::fromMbusCode(fmt.first); if (!sensorBayer.isValid()) @@ -302,11 +359,11 @@ const PixelFormat &MaliC55CameraData::bestRawFormat() const void MaliC55CameraData::updateControls(const ControlInfoMap &ipaControls) { - if (!sensor_) + if (std::holds_alternative(input_)) return; IPACameraSensorInfo sensorInfo; - int ret = sensor_->sensorInfo(&sensorInfo); + int ret = sensor()->sensorInfo(&sensorInfo); if (ret) { LOG(MaliC55, Error) << "Failed to retrieve sensor info"; return; @@ -379,7 +436,7 @@ int MaliC55CameraData::loadIPA() int ret; /* Do not initialize IPA for TPG. */ - if (!sensor_) + if (std::holds_alternative(input_)) return 0; ipa_ = IPAManager::createIPA(pipe(), 1, 1); @@ -388,20 +445,20 @@ int MaliC55CameraData::loadIPA() ipa_->setSensorControls.connect(this, &MaliC55CameraData::setSensorControls); - std::string ipaTuningFile = ipa_->configurationFile(sensor_->model() + ".yaml", + std::string ipaTuningFile = ipa_->configurationFile(sensor()->model() + ".yaml", "uncalibrated.yaml"); /* We need to inform the IPA of the sensor configuration */ ipa::mali_c55::IPAConfigInfo ipaConfig{}; - ret = sensor_->sensorInfo(&ipaConfig.sensorInfo); + ret = sensor()->sensorInfo(&ipaConfig.sensorInfo); if (ret) return ret; - ipaConfig.sensorControls = sensor_->controls(); + ipaConfig.sensorControls = sensor()->controls(); ControlInfoMap ipaControls; - ret = ipa_->init({ ipaTuningFile, sensor_->model() }, ipaConfig, + ret = ipa_->init({ ipaTuningFile, sensor()->model() }, ipaConfig, &ipaControls); if (ret) { LOG(MaliC55, Error) << "Failed to initialise the Mali-C55 IPA"; @@ -444,13 +501,13 @@ CameraConfiguration::Status MaliC55CameraConfiguration::validate() * The TPG doesn't support flips, so we only need to calculate a * transform if we have a sensor. */ - if (data_->sensor_) { + if (std::holds_alternative(data_->input_)) { + combinedTransform_ = Transform::Rot0; + } else { Orientation requestedOrientation = orientation; - combinedTransform_ = data_->sensor_->computeTransform(&orientation); + combinedTransform_ = data_->sensor()->computeTransform(&orientation); if (orientation != requestedOrientation) status = Adjusted; - } else { - combinedTransform_ = Transform::Rot0; } /* Only 2 streams available. */ @@ -927,39 +984,44 @@ int PipelineHandlerMaliC55::configure(Camera *camera, /* Link the graph depending if we are operating the TPG or a sensor. */ MaliC55CameraData *data = cameraData(camera); - if (data->csi_) { - const MediaEntity *csiEntity = data->csi_->entity(); - ret = csiEntity->getPadByIndex(1)->links()[0]->setEnabled(true); - } else { - ret = data->entity_->getPadByIndex(0)->links()[0]->setEnabled(true); - } + ret = std::visit(utils::overloaded { + [](MaliC55CameraData::Tpg &tpg) { + const MediaEntity *tpgEntity = tpg.sd_->entity(); + return tpgEntity->getPadByIndex(0)->links()[0]->setEnabled(true); + }, + [](MaliC55CameraData::Inline &in) { + const MediaEntity *csi2Entity = in.csi2_->entity(); + return csi2Entity->getPadByIndex(1)->links()[0]->setEnabled(true); + }, + }, data->input_); if (ret) return ret; MaliC55CameraConfiguration *maliConfig = static_cast(config); V4L2SubdeviceFormat subdevFormat = maliConfig->sensorFormat_; - ret = data->sd_->getFormat(0, &subdevFormat); + + /* Apply format to the origin of the pipeline and propagate it. */ + ret = std::visit(utils::overloaded { + [&](MaliC55CameraData::Tpg &) { + return data->subdev()->setFormat(0, &subdevFormat); + }, + [&](MaliC55CameraData::Inline &) { + ret = data->sensor()->setFormat(&subdevFormat, + maliConfig->combinedTransform()); + if (ret) + return ret; + + ret = data->csi2()->setFormat(0, &subdevFormat); + if (ret) + return ret; + + return data->csi2()->getFormat(1, &subdevFormat); + }, + }, data->input_); if (ret) return ret; - if (data->sensor_) { - ret = data->sensor_->setFormat(&subdevFormat, - maliConfig->combinedTransform()); - if (ret) - return ret; - } - - if (data->csi_) { - ret = data->csi_->setFormat(0, &subdevFormat); - if (ret) - return ret; - - ret = data->csi_->getFormat(1, &subdevFormat); - if (ret) - return ret; - } - V4L2DeviceFormat statsFormat; ret = stats_->getFormat(&statsFormat); if (ret) @@ -973,8 +1035,6 @@ int PipelineHandlerMaliC55::configure(Camera *camera, /* * Propagate the format to the ISP sink pad and configure the input * crop rectangle (no crop at the moment). - * - * \todo Configure the CSI-2 receiver. */ ret = isp_->setFormat(0, &subdevFormat); if (ret) @@ -1058,18 +1118,18 @@ int PipelineHandlerMaliC55::configure(Camera *camera, /* We need to inform the IPA of the sensor configuration */ ipa::mali_c55::IPAConfigInfo ipaConfig{}; - ret = data->sensor_->sensorInfo(&ipaConfig.sensorInfo); + ret = data->sensor()->sensorInfo(&ipaConfig.sensorInfo); if (ret) return ret; - ipaConfig.sensorControls = data->sensor_->controls(); + ipaConfig.sensorControls = data->sensor()->controls(); /* * And we also need to tell the IPA the bayerOrder of the data (as * affected by any flips that we've configured) */ const Transform &combinedTransform = maliConfig->combinedTransform(); - BayerFormat::Order bayerOrder = data->sensor_->bayerOrder(combinedTransform); + BayerFormat::Order bayerOrder = data->sensor()->bayerOrder(combinedTransform); ControlInfoMap ipaControls; ret = data->ipa_->configure(ipaConfig, utils::to_underlying(bayerOrder), @@ -1283,7 +1343,7 @@ void PipelineHandlerMaliC55::applyScalerCrop(Camera *camera, if (!scalerCrop) return; - if (!data->sensor_) { + if (std::holds_alternative(data->input_)) { LOG(MaliC55, Error) << "ScalerCrop not supported for TPG"; return; } @@ -1291,7 +1351,7 @@ void PipelineHandlerMaliC55::applyScalerCrop(Camera *camera, Rectangle nativeCrop = *scalerCrop; IPACameraSensorInfo sensorInfo; - int ret = data->sensor_->sensorInfo(&sensorInfo); + int ret = data->sensor()->sensorInfo(&sensorInfo); if (ret) { LOG(MaliC55, Error) << "Failed to retrieve sensor info"; return; @@ -1573,10 +1633,11 @@ bool PipelineHandlerMaliC55::registerTPGCamera(MediaLink *link) } std::unique_ptr data = - std::make_unique(this, link->source()->entity()); + std::make_unique(this); - if (data->init()) - return false; + int ret = data->initTpg(link->source()->entity()); + if (ret) + return ret; return registerMaliCamera(std::move(data), name); } @@ -1600,21 +1661,24 @@ bool PipelineHandlerMaliC55::registerSensorCamera(MediaLink *ispLink) continue; std::unique_ptr data = - std::make_unique(this, sensor); - if (data->init()) - return false; + std::make_unique(this); + + int ret = data->initInline(sensor); + if (ret) + return ret; - data->properties_ = data->sensor_->properties(); + data->properties_ = data->sensor()->properties(); - const CameraSensorProperties::SensorDelays &delays = data->sensor_->sensorDelays(); + const CameraSensorProperties::SensorDelays &delays = + data->sensor()->sensorDelays(); std::unordered_map params = { { V4L2_CID_ANALOGUE_GAIN, { delays.gainDelay, false } }, { V4L2_CID_EXPOSURE, { delays.exposureDelay, false } }, }; - data->delayedCtrls_ = - std::make_unique(data->sensor_->device(), - params); + V4L2Subdevice *sensorSubdev = data->sensor()->device(); + data->delayedCtrls_ = std::make_unique(sensorSubdev, + params); isp_->frameStart.connect(data->delayedCtrls_.get(), &DelayedControls::applyControls); From patchwork Fri Mar 13 16:14:38 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jacopo Mondi X-Patchwork-Id: 26295 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 8E17DC32F0 for ; Fri, 13 Mar 2026 16:14:52 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 29AF762727; Fri, 13 Mar 2026 17:14:52 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="DY0md4I7"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 59E3B6270C for ; Fri, 13 Mar 2026 17:14:50 +0100 (CET) Received: from [192.168.224.131] (unknown [37.159.122.93]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id B462A1934; Fri, 13 Mar 2026 17:13:40 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1773418421; bh=APUyktNx0MVd4SoGkw/VTrDTc+XxoVQwTfbNIXCEEy4=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=DY0md4I73mYOqBKzEo77h/dqB6TW7C6j8kgtjl3RFnwdetJljIe7u+yYN81OW6D6A 7Vjg9mhUHMkCn4Zt+xE/Zdc0Cg1bUBXpmJl63ZeEDB+wlwK7qqQ+bqYCU2VO76P3Vn 9vbTMwevK5lDhz3puIlEvxToNPkAPqLzgkL+U1QI= From: Jacopo Mondi Date: Fri, 13 Mar 2026 17:14:38 +0100 Subject: [PATCH v5 4/7] libcamera: mali-c55: Register memory input camera MIME-Version: 1.0 Message-Id: <20260313-mali-cru-v5-4-48f93e431294@ideasonboard.com> References: <20260313-mali-cru-v5-0-48f93e431294@ideasonboard.com> In-Reply-To: <20260313-mali-cru-v5-0-48f93e431294@ideasonboard.com> To: Daniel Scally , libcamera-devel@lists.libcamera.org Cc: Jacopo Mondi X-Mailer: b4 0.14.3 X-Developer-Signature: v=1; a=openpgp-sha256; l=9819; i=jacopo.mondi@ideasonboard.com; h=from:subject:message-id; bh=KOQwFbPgOiDmtCmZ7//v9C+mAaBkiNylRVHBHbip3zQ=; b=owEBbQKS/ZANAwAKAXI0Bo8WoVY8AcsmYgBptDfzCFn5VtPPGHQWM4Qc1RAbIvaiuhf6Ef7bx EmEgvuQAdOJAjMEAAEKAB0WIQS1xD1IgJogio9YOMByNAaPFqFWPAUCabQ38wAKCRByNAaPFqFW PNFID/9ewLFuZPpkbErgjcIYRW2xX9zb0xylp1hmWomI6q9i2RR5oR0jobBQiIe7W8OsDBQy+hM DrP3s7vzRmiC2JtoSiGx0tPvlrLdB+CvouQpFcS0pe6VwIh4BWU389lVHygf4usc5Z3m9BCteVc 0WwliyLDAMkkcOMbJN1lJMqJOocnEIeCxp5em/IEnP/hFjxnQsNvQ4rwkCAZUUPK6+llFrjLvMZ rUjOv53FmtQWmTyVdoj5UZ5OgP1J20FKSst3duwFrkmWbe+Kbq5UUjyMxKF23k2Xrrqxtxp1Mob 1tkrV1hq7WRcWps3Jem2L/cs40nGIZkPYQLrDQTpgFT7oppPDbH0lMf1es62yqdZENGyW3WnwNS 3aXNw9HbMNOXCa8wprPM7nr45spkryiVuxTJw20B0Fknyzz9ddSizGXzQx20NuS11TpR/XIy6Rm IPWeiTAANMp9nkW4EdQJRBfeniReRXl5+iChHfVALdDO867vPk3BVRjw+wPpVib+k1EMW7P7o3e Qp0Z7xP9E5KyrmzpKanY8L1DIHGIgQVs7Aq70AERrVXL7xGlXphz58sM66JkmFEi7pIjD4kAz1e gytFfhyQfO1U7CQElkX8VqjyHYzKg5trvGdV42gDLZujHB2QbT6yzb/6Pje4A2p5nFv46lwKU5E tTmC2uHvfAFFbSw== X-Developer-Key: i=jacopo.mondi@ideasonboard.com; a=openpgp; fpr=72392EDC88144A65C701EA9BA5826A2587AD026B X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" From: Daniel Scally Add support to Mali C55 pipeline handler for memory-to-memory camera mode. The Mali-C55, as integrated in the Renesas RZ/V2H(P) SoC, is fed with images saved to memory by the CRU unit and read on behalf of the ISP by the IVC. Update the pipeline handler's match() function to search for the CRU media entity necessary to register a memory input camera. Create and initialize a CameraData instance for the memory camera use case if a valid CRU is found and initialize the IVC components used to feed the ISP with images read from memory. Do not implement camera configuration for the time being to reuce the patch size. It will be provided in the following patches. Signed-off-by: Daniel Scally Signed-off-by: Jacopo Mondi --- src/libcamera/pipeline/mali-c55/mali-c55.cpp | 151 ++++++++++++++++++++++++--- 1 file changed, 134 insertions(+), 17 deletions(-) diff --git a/src/libcamera/pipeline/mali-c55/mali-c55.cpp b/src/libcamera/pipeline/mali-c55/mali-c55.cpp index 45e23ffd3841..5e93d66b01cd 100644 --- a/src/libcamera/pipeline/mali-c55/mali-c55.cpp +++ b/src/libcamera/pipeline/mali-c55/mali-c55.cpp @@ -44,6 +44,8 @@ #include "libcamera/internal/v4l2_subdevice.h" #include "libcamera/internal/v4l2_videodevice.h" +#include "rzg2l-cru.h" + namespace { bool isFormatRaw(const libcamera::PixelFormat &pixFmt) @@ -105,7 +107,12 @@ public: std::unique_ptr csi2_; std::unique_ptr sensor_; }; - using CameraType = std::variant; + + struct Memory { + std::unique_ptr cru_; + }; + + using CameraType = std::variant; MaliC55CameraData(PipelineHandler *pipe) : Camera::Private(pipe) @@ -116,6 +123,7 @@ public: int initTpg(MediaEntity *entity); int initInline(MediaEntity *entity); + int initMemory(MediaDevice *cruMedia); std::vector sizes(unsigned int mbusCode) const { @@ -125,6 +133,9 @@ public: }, [&](const Inline &in) -> std::vector { return in.sensor_->sizes(mbusCode); + }, + [&](const Memory &mem) -> std::vector { + return mem.cru_->sizes(); } }, input_); } @@ -137,6 +148,9 @@ public: }, [&](const Inline &in) -> V4L2Subdevice * { return in.sensor_->device(); + }, + [&](const Memory &mem) -> V4L2Subdevice * { + return mem.cru_->sensor()->device(); } }, input_); } @@ -150,6 +164,9 @@ public: }, [&](const Inline &in) -> CameraSensor * { return in.sensor_.get(); + }, + [&](const Memory &mem) -> CameraSensor * { + return mem.cru_->sensor(); } }, input_); } @@ -163,6 +180,9 @@ public: }, [&](const Inline &in) -> V4L2Subdevice * { return in.csi2_.get(); + }, + [&](const Memory &mem) -> V4L2Subdevice * { + return mem.cru_->csi2(); } }, input_); } @@ -175,10 +195,26 @@ public: }, [&](const Inline &in) -> Size { return in.sensor_->resolution(); + }, + [&](const Memory &mem) -> Size { + return mem.cru_->resolution(); } }, input_); } + RZG2LCRU *cru() const + { + return std::visit(utils::overloaded{ + [&](auto &) -> RZG2LCRU * { + ASSERT(false); + return nullptr; + }, + [&](const Memory &mem) -> RZG2LCRU * { + return mem.cru_.get(); + }, + }, input_); + } + int pixfmtToMbusCode(const PixelFormat &pixFmt) const; const PixelFormat &bestRawFormat() const; @@ -258,6 +294,21 @@ int MaliC55CameraData::initInline(MediaEntity *sensor) return 0; } +int MaliC55CameraData::initMemory(MediaDevice *cruMedia) +{ + Memory mem; + + mem.cru_ = std::make_unique(); + + int ret = mem.cru_->init(cruMedia); + if (ret) + return ret; + + input_.emplace(std::move(mem)); + + return 0; +} + std::vector MaliC55CameraData::Tpg::sizes(unsigned int mbusCode) const { V4L2Subdevice::Formats formats = sd_->formats(0); @@ -738,11 +789,15 @@ private: const std::string &name); bool registerTPGCamera(MediaLink *link); bool registerSensorCamera(MediaLink *link); + bool registerMemoryInputCamera(); std::shared_ptr media_; + std::shared_ptr cruMedia_; std::unique_ptr isp_; std::unique_ptr stats_; std::unique_ptr params_; + std::unique_ptr ivcSd_; + std::unique_ptr ivc_; std::vector> statsBuffers_; std::queue availableStatsBuffers_; @@ -993,6 +1048,9 @@ int PipelineHandlerMaliC55::configure(Camera *camera, const MediaEntity *csi2Entity = in.csi2_->entity(); return csi2Entity->getPadByIndex(1)->links()[0]->setEnabled(true); }, + [](MaliC55CameraData::Memory &) { + return 0; + } }, data->input_); if (ret) return ret; @@ -1018,6 +1076,9 @@ int PipelineHandlerMaliC55::configure(Camera *camera, return data->csi2()->getFormat(1, &subdevFormat); }, + [](MaliC55CameraData::Memory &) { + return 0; + } }, data->input_); if (ret) return ret; @@ -1691,6 +1752,34 @@ bool PipelineHandlerMaliC55::registerSensorCamera(MediaLink *ispLink) return true; } +bool PipelineHandlerMaliC55::registerMemoryInputCamera() +{ + std::unique_ptr data = + std::make_unique(this); + + int ret = data->initMemory(cruMedia_.get()); + if (ret) + return false; + + CameraSensor *sensor = data->sensor(); + data->properties_ = sensor->properties(); + + const CameraSensorProperties::SensorDelays &delays = sensor->sensorDelays(); + std::unordered_map params = { + { V4L2_CID_ANALOGUE_GAIN, { delays.gainDelay, false } }, + { V4L2_CID_EXPOSURE, { delays.exposureDelay, false } }, + }; + + data->delayedCtrls_ = + std::make_unique(sensor->device(), params); + isp_->frameStart.connect(data->delayedCtrls_.get(), + &DelayedControls::applyControls); + + ivc_->bufferReady.connect(data->cru(), &RZG2LCRU::cruReturnBuffer); + + return registerMaliCamera(std::move(data), sensor->device()->entity()->name()); +} + bool PipelineHandlerMaliC55::match(DeviceEnumerator *enumerator) { const MediaPad *ispSink; @@ -1700,14 +1789,14 @@ bool PipelineHandlerMaliC55::match(DeviceEnumerator *enumerator) * The TPG and the downscale pipe are both optional blocks and may not * be fitted. */ - DeviceMatch dm("mali-c55"); - dm.add("mali-c55 isp"); - dm.add("mali-c55 resizer fr"); - dm.add("mali-c55 fr"); - dm.add("mali-c55 3a stats"); - dm.add("mali-c55 3a params"); - - media_ = acquireMediaDevice(enumerator, dm); + DeviceMatch c55_dm("mali-c55"); + c55_dm.add("mali-c55 isp"); + c55_dm.add("mali-c55 resizer fr"); + c55_dm.add("mali-c55 fr"); + c55_dm.add("mali-c55 3a stats"); + c55_dm.add("mali-c55 3a params"); + + media_ = acquireMediaDevice(enumerator, c55_dm); if (!media_) return false; @@ -1767,6 +1856,25 @@ bool PipelineHandlerMaliC55::match(DeviceEnumerator *enumerator) stats_->bufferReady.connect(this, &PipelineHandlerMaliC55::statsBufferReady); params_->bufferReady.connect(this, &PipelineHandlerMaliC55::paramsBufferReady); + /* + * We also need to search for the rzg2l-cru CSI-2 receiver. If we find + * that then we need to work in memory input mode instead of the inline + * mode. The absence of this match is not necessarily a failure at this + * point...it depends on the media links that we investigate momentarily. + * + * This is a bit hacky, because there could be multiple of these media + * devices and we're just taking the first. We need modular pipelines to + * properly solve the issue. + */ + static const std::regex cruCsi2Regex("csi-[0-9a-f]{8}.csi2"); + static const std::regex cruIpRegex("cru-ip-[0-9a-f]{8}.cru[0-9]"); + + DeviceMatch cruDm("rzg2l_cru"); + cruDm.add(cruCsi2Regex); + cruDm.add(cruIpRegex); + cruDm.add("CRU output"); + cruMedia_ = acquireMediaDevice(enumerator, cruDm); + ispSink = isp_->entity()->getPadByIndex(0); if (!ispSink || ispSink->links().empty()) { LOG(MaliC55, Error) << "ISP sink pad error"; @@ -1780,13 +1888,6 @@ bool PipelineHandlerMaliC55::match(DeviceEnumerator *enumerator) * MEDIA_ENT_F_CAM_SENSOR - The test pattern generator * MEDIA_ENT_F_VID_IF_BRIDGE - A CSI-2 receiver * MEDIA_ENT_F_PROC_VIDEO_PIXEL_FORMATTER - An input device - * - * The last one will be unsupported for now. The TPG is relatively easy, - * we just register a Camera for it. If we have a CSI-2 receiver we need - * to check its sink pad and register Cameras for anything connected to - * it (probably...there are some complex situations in which that might - * not be true but let's pretend they don't exist until we come across - * them) */ bool registered; for (MediaLink *link : ispSink->links()) { @@ -1806,7 +1907,23 @@ bool PipelineHandlerMaliC55::match(DeviceEnumerator *enumerator) break; case MEDIA_ENT_F_PROC_VIDEO_PIXEL_FORMATTER: - LOG(MaliC55, Warning) << "Memory input not yet supported"; + if (!cruMedia_) + return false; + + ivcSd_ = V4L2Subdevice::fromEntityName(media_.get(), + "rzv2h ivc block"); + if (ivcSd_->open() < 0) + return false; + + ivc_ = V4L2VideoDevice::fromEntityName(media_.get(), + "rzv2h-ivc"); + if (ivc_->open() < 0) + return false; + + registered = registerMemoryInputCamera(); + if (!registered) + return registered; + break; default: LOG(MaliC55, Error) << "Unsupported entity function"; From patchwork Fri Mar 13 16:14:39 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jacopo Mondi X-Patchwork-Id: 26296 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 88F39C32B5 for ; Fri, 13 Mar 2026 16:14:54 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 379506271D; Fri, 13 Mar 2026 17:14:54 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="p5p0UKhz"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id D72056271D for ; Fri, 13 Mar 2026 17:14:51 +0100 (CET) Received: from [192.168.224.131] (unknown [37.159.122.93]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 8CBAB1D98; Fri, 13 Mar 2026 17:13:41 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1773418422; bh=AsomUuJdFnFq0ZzK2S3TIpdAp6bgAC4XuKCZHuC6+HE=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=p5p0UKhzIsDk7rsGGTzjN87VRU74q7usoizsZ3jWvKKies0gnBvXKAjvSYGXw4n8Y dONuI/Jg7d4LXACt9TzM08dXleZJaIEcmuMgq9bYchh4CM3jQ8OT1NJwYSQ0LSDGu1 MHfKu3zucVPNZq2CYWClrlh2MoPWiCPwHtBIoSLs= From: Jacopo Mondi Date: Fri, 13 Mar 2026 17:14:39 +0100 Subject: [PATCH v5 5/7] libcamera: mali-c55: Configure camera in memory-to-memory MIME-Version: 1.0 Message-Id: <20260313-mali-cru-v5-5-48f93e431294@ideasonboard.com> References: <20260313-mali-cru-v5-0-48f93e431294@ideasonboard.com> In-Reply-To: <20260313-mali-cru-v5-0-48f93e431294@ideasonboard.com> To: Daniel Scally , libcamera-devel@lists.libcamera.org Cc: Jacopo Mondi X-Mailer: b4 0.14.3 X-Developer-Signature: v=1; a=openpgp-sha256; l=1799; i=jacopo.mondi@ideasonboard.com; h=from:subject:message-id; bh=AsomUuJdFnFq0ZzK2S3TIpdAp6bgAC4XuKCZHuC6+HE=; b=owEBbQKS/ZANAwAKAXI0Bo8WoVY8AcsmYgBptDfzTXbzskx/VLtLyd4AGfMT2l4vM7NUHAchh Aa9+/v10hyJAjMEAAEKAB0WIQS1xD1IgJogio9YOMByNAaPFqFWPAUCabQ38wAKCRByNAaPFqFW PGNiD/wOXIvzaj1+u+TMEXhKvfhGGqKd7Huu3WWanMMk99zXKJIFvLiVoqTrNs9/4izEZiAhFVb tVuP8q3nCb0WBrROGqGx9e2OW+XWqmmbkpds8Pm66QQZk2c68N1BdaMruwRTX708UCSGLEWL/Fa zURdx3cO5jWB9kNcTJLaHZ/SzJ2+7+Hs+tj2z2GvYDLLOCn0vP4BSOOX+RQgIXzpiZJrhE6oOk7 IDRrX6uzgP4XqKhsTTroXPKQ7Dqsaut7CLzP/DM1Hj+IqAsc9CUAznUfsAZYrpmGCAmyncmwueQ N8wohkZBGEic8nojmqjjY+DA34MNwk7YP+U+FdpeGZdupBgcVczvve/XzWDH6s/gTkQcr/tZG2B ojmZpV/KXX6eyYaLgZuq1zo6DnTFEITGmt7gQKCOrHFqS1rgnV/N11L3BUTaCofNoIT3IbLIqdK 0yTaOPfM8eAdppJN0gZnNfpnNsYcWf0TALS3B5MmyUxP+7/pcFCoIULtfWdNhqUTQUNl1C9PPbJ efhHfTZ/IAoevGG+P17irn1XgD8k+RIic8zyRg/nP8AVmuZ5DjIb1jjE+zZoZBqpKW7AwZFEHqr KyxT6rRogMcf/jAQPq8HZb9uuwJseKrmzCeup1UjoGHtf+LpFoQoahx7VVAberD+KZUe+fZwQAq KcIcE5oMCxkLQTg== X-Developer-Key: i=jacopo.mondi@ideasonboard.com; a=openpgp; fpr=72392EDC88144A65C701EA9BA5826A2587AD026B X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Add support for memory-to-memory Camera to the PipelineHandlerMaliC55::configure() function. Start by enabling the IVC links, then configure the CRU and propagate the format to the IVC. Signed-off-by: Jacopo Mondi --- src/libcamera/pipeline/mali-c55/mali-c55.cpp | 24 ++++++++++++++++++++---- 1 file changed, 20 insertions(+), 4 deletions(-) diff --git a/src/libcamera/pipeline/mali-c55/mali-c55.cpp b/src/libcamera/pipeline/mali-c55/mali-c55.cpp index 5e93d66b01cd..9b83927d8441 100644 --- a/src/libcamera/pipeline/mali-c55/mali-c55.cpp +++ b/src/libcamera/pipeline/mali-c55/mali-c55.cpp @@ -1048,8 +1048,9 @@ int PipelineHandlerMaliC55::configure(Camera *camera, const MediaEntity *csi2Entity = in.csi2_->entity(); return csi2Entity->getPadByIndex(1)->links()[0]->setEnabled(true); }, - [](MaliC55CameraData::Memory &) { - return 0; + [&](MaliC55CameraData::Memory &) { + const MediaEntity *ivcEntity = ivcSd_->entity(); + return ivcEntity->getPadByIndex(1)->links()[0]->setEnabled(true); } }, data->input_); if (ret) @@ -1076,8 +1077,23 @@ int PipelineHandlerMaliC55::configure(Camera *camera, return data->csi2()->getFormat(1, &subdevFormat); }, - [](MaliC55CameraData::Memory &) { - return 0; + [&](MaliC55CameraData::Memory &) { + V4L2DeviceFormat inputFormat; + + ret = data->cru()->configure(&subdevFormat, &inputFormat); + if (ret) + return ret; + + /* Propagate the CRU format to the IVC input. */ + ret = ivcSd_->setFormat(0, &subdevFormat); + if (ret) + return ret; + + ret = ivcSd_->getFormat(1, &subdevFormat); + if (ret) + return ret; + + return ivc_->setFormat(&inputFormat); } }, data->input_); if (ret) From patchwork Fri Mar 13 16:14:40 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jacopo Mondi X-Patchwork-Id: 26297 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 27B80C32EF for ; Fri, 13 Mar 2026 16:14:58 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id B015562730; Fri, 13 Mar 2026 17:14:57 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="pvnevIlt"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 0E54B626BE for ; Fri, 13 Mar 2026 17:14:56 +0100 (CET) Received: from [192.168.224.131] (unknown [37.159.122.93]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 0ACD21E20; Fri, 13 Mar 2026 17:13:42 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1773418426; bh=EwM0lPDrHo3KScdzd1O4arRS97/pjfMTXAzsxqxebOo=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=pvnevIltH3xz313fOyTI/BmByAWayOIbNiGHTkGnhtwmU9DJl7DcQ+juemYuVtCdx xLrs86SL+rpCfmvDU0K21H5xdgk3hQI1f6IPVyna93kUSFpBHefH73V5TxHzVSPfcm 6B4GvjUNIUSqgS8PpBWwfTb5D9rnoZfQIkL8fKDM= From: Jacopo Mondi Date: Fri, 13 Mar 2026 17:14:40 +0100 Subject: [PATCH v5 6/7] libcamera: mali-c55: Implement capture for memory-to-memory MIME-Version: 1.0 Message-Id: <20260313-mali-cru-v5-6-48f93e431294@ideasonboard.com> References: <20260313-mali-cru-v5-0-48f93e431294@ideasonboard.com> In-Reply-To: <20260313-mali-cru-v5-0-48f93e431294@ideasonboard.com> To: Daniel Scally , libcamera-devel@lists.libcamera.org Cc: Jacopo Mondi X-Mailer: b4 0.14.3 X-Developer-Signature: v=1; a=openpgp-sha256; l=9435; i=jacopo.mondi@ideasonboard.com; h=from:subject:message-id; bh=mGB/JK1uaiNUsm5JpSdV97aWgFIcj2ZI7a1IZpjhjX0=; b=owEBbQKS/ZANAwAKAXI0Bo8WoVY8AcsmYgBptDfz9LLvNbB0g/OtUm8noa2XBVB3a1FwGc3V2 ERsSe6O/Z6JAjMEAAEKAB0WIQS1xD1IgJogio9YOMByNAaPFqFWPAUCabQ38wAKCRByNAaPFqFW POWuD/0c+imOFOXwWwQ7agIZ3EiK4Q+TGqZGsJMlV7SMf7l6SLsm9Cm9FmWE9DBUx6+fcEm7HmQ V6nkSxS2pzO3tKQLOS57iGS+o5zEc2yML2OyVFTVBHC6pWyiUaPqqoVQkZdCNjFDIU8ijq9X0HJ Q5OaQBAYnFhQiTkly0PNxK++sYLaLAqzpc91Uq+0H7mRPzdgJjjy7CZVrH8jGm4KQyBTao3HG/5 NyR/aKItMS06cw6hL7plYb95jpS1A8Elrs/EZ+2NT6LsxZnp8xh2t2I700I4r8GAukVItr63zSY Jzrn2W+t50329AIPnVrRT8cjMulIx2ZnFocj9gkZahAb+ahrX/JiN69/tjRZZOXpQRBXb15rEpE uMMVoG2nVygudI5Hrph6MEsMMeFV4znRWo2sEmpRnW41atCuRPhZHdzpgXtqOFbJeEOBE+JrDBe nIHnFYwQF9PURoYw5sfvAiPtvNCB7SUkgzI3JQWkBBa4OD/7x2Sf2Q4Yi2Yg26U4tRB7ljWkHyb U8gsxvPxucpvRbt9xJShsT2XbpOx1r+SvPUBjrhS5bNxKDqHB66Qe+82Z3IUBd/cWuoO4Wi/Wq7 tXazUiWSBfpXfOcG1xkHS6IqarJOk5ec6wTAFw0r1AgAgPigWQO+C2K2do0hzJvVJ5wkqDnsQkZ qyfEt3qcCf4ZomQ== X-Developer-Key: i=jacopo.mondi@ideasonboard.com; a=openpgp; fpr=72392EDC88144A65C701EA9BA5826A2587AD026B X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" From: Daniel Scally Plumb in the MaliC55 pipeline handler support for capturing frames from memory using the CRU. Introduce a data flow which uses the CRU to feed the ISP through the IVC. In detail: - push incoming request to a pending queue until a buffer from the CRU is available - delay the call to ipa_->fillParams() to the CRU buffer ready even - once the IPA has computed parameters feed the ISP through the IVC with buffers from the CRU, params and statistics. Signed-off-by: Daniel Scally Signed-off-by: Jacopo Mondi --- src/libcamera/pipeline/mali-c55/mali-c55.cpp | 166 ++++++++++++++++++++++++++- 1 file changed, 161 insertions(+), 5 deletions(-) diff --git a/src/libcamera/pipeline/mali-c55/mali-c55.cpp b/src/libcamera/pipeline/mali-c55/mali-c55.cpp index 9b83927d8441..84c2a030b470 100644 --- a/src/libcamera/pipeline/mali-c55/mali-c55.cpp +++ b/src/libcamera/pipeline/mali-c55/mali-c55.cpp @@ -21,6 +21,7 @@ #include #include +#include #include #include #include @@ -88,6 +89,7 @@ struct MaliC55FrameInfo { FrameBuffer *paramBuffer; FrameBuffer *statBuffer; + FrameBuffer *rawBuffer; bool paramsDone; bool statsDone; @@ -721,11 +723,14 @@ public: int start(Camera *camera, const ControlList *controls) override; void stopDevice(Camera *camera) override; + int queuePendingRequests(MaliC55CameraData *data); + void cancelPendingRequests(); int queueRequestDevice(Camera *camera, Request *request) override; void imageBufferReady(FrameBuffer *buffer); void paramsBufferReady(FrameBuffer *buffer); void statsBufferReady(FrameBuffer *buffer); + void cruBufferReady(FrameBuffer *buffer); void paramsComputed(unsigned int requestId, uint32_t bytesused); void statsProcessed(unsigned int requestId, const ControlList &metadata); @@ -807,6 +812,11 @@ private: std::map frameInfoMap_; + /* Requests for which no buffer has been queued to the CRU device yet. */ + std::queue pendingRequests_; + /* Requests queued to the CRU device but not yet processed by the ISP. */ + std::queue processingRequests_; + std::array pipes_; bool dsFitted_; @@ -1255,6 +1265,11 @@ void PipelineHandlerMaliC55::freeBuffers(Camera *camera) if (params_->releaseBuffers()) LOG(MaliC55, Error) << "Failed to release params buffers"; + if (std::holds_alternative(data->input_)) { + if (ivc_->releaseBuffers()) + LOG(MaliC55, Error) << "Failed to release input buffers"; + } + return; } @@ -1284,6 +1299,12 @@ int PipelineHandlerMaliC55::allocateBuffers(Camera *camera) } }; + if (std::holds_alternative(data->input_)) { + ret = ivc_->importBuffers(RZG2LCRU::kBufferCount); + if (ret < 0) + return ret; + } + ret = stats_->allocateBuffers(bufferCount, &statsBuffers_); if (ret < 0) return ret; @@ -1315,6 +1336,24 @@ int PipelineHandlerMaliC55::start(Camera *camera, [[maybe_unused]] const Control if (ret) return ret; + if (std::holds_alternative(data->input_)) { + ret = data->cru()->start(); + if (ret) { + LOG(MaliC55, Error) + << "Failed to start CRU " << camera->id(); + freeBuffers(camera); + return ret; + } + + ret = ivc_->streamOn(); + if (ret) { + LOG(MaliC55, Error) + << "Failed to start IVC" << camera->id(); + freeBuffers(camera); + return ret; + } + } + if (data->ipa_) { ret = data->ipa_->start(); if (ret) { @@ -1404,6 +1443,12 @@ void PipelineHandlerMaliC55::stopDevice(Camera *camera) pipe.cap->releaseBuffers(); } + if (std::holds_alternative(data->input_)) { + cancelPendingRequests(); + ivc_->streamOff(); + data->cru()->stop(); + } + stats_->streamOff(); params_->streamOff(); if (data->ipa_) @@ -1507,10 +1552,88 @@ void PipelineHandlerMaliC55::applyScalerCrop(Camera *camera, } } +void PipelineHandlerMaliC55::cancelPendingRequests() +{ + processingRequests_ = {}; + + while (!pendingRequests_.empty()) { + Request *request = pendingRequests_.front(); + + completeRequest(request); + pendingRequests_.pop(); + } +} + +int PipelineHandlerMaliC55::queuePendingRequests(MaliC55CameraData *data) +{ + ASSERT(std::holds_alternative(data->input_)); + + while (!pendingRequests_.empty()) { + Request *request = pendingRequests_.front(); + + if (availableStatsBuffers_.empty()) { + LOG(MaliC55, Error) << "Stats buffer underrun"; + return -ENOENT; + } + + if (availableParamsBuffers_.empty()) { + LOG(MaliC55, Error) << "Params buffer underrun"; + return -ENOENT; + } + + MaliC55FrameInfo frameInfo; + frameInfo.request = request; + + frameInfo.rawBuffer = data->cru()->queueBuffer(request); + if (!frameInfo.rawBuffer) + return -ENOENT; + + frameInfo.statBuffer = availableStatsBuffers_.front(); + availableStatsBuffers_.pop(); + frameInfo.paramBuffer = availableParamsBuffers_.front(); + availableParamsBuffers_.pop(); + + frameInfo.paramsDone = false; + frameInfo.statsDone = false; + + frameInfoMap_[request->sequence()] = frameInfo; + + for (auto &[stream, buffer] : request->buffers()) { + MaliC55Pipe *pipe = pipeFromStream(data, stream); + + pipe->cap->queueBuffer(buffer); + } + + data->ipa_->queueRequest(request->sequence(), request->controls()); + + pendingRequests_.pop(); + processingRequests_.push(request); + } + + return 0; +} + int PipelineHandlerMaliC55::queueRequestDevice(Camera *camera, Request *request) { MaliC55CameraData *data = cameraData(camera); + /* + * If we're in memory input mode, we need to pop the requests onto the + * pending list until a CRU buffer is ready...otherwise we can just do + * everything immediately. + */ + if (std::holds_alternative(data->input_)) { + pendingRequests_.push(request); + + int ret = queuePendingRequests(data); + if (ret) { + pendingRequests_.pop(); + return ret; + } + + return 0; + } + /* Do not run the IPA if the TPG is in use. */ if (!data->ipa_) { MaliC55FrameInfo frameInfo; @@ -1575,7 +1698,8 @@ MaliC55FrameInfo *PipelineHandlerMaliC55::findFrameInfo(FrameBuffer *buffer) { for (auto &[sequence, info] : frameInfoMap_) { if (info.paramBuffer == buffer || - info.statBuffer == buffer) + info.statBuffer == buffer || + info.rawBuffer == buffer) return &info; } @@ -1637,6 +1761,26 @@ void PipelineHandlerMaliC55::statsBufferReady(FrameBuffer *buffer) sensorControls); } +void PipelineHandlerMaliC55::cruBufferReady(FrameBuffer *buffer) +{ + MaliC55FrameInfo *info = findFrameInfo(buffer); + Request *request = info->request; + ASSERT(info); + + if (buffer->metadata().status == FrameMetadata::FrameCancelled) { + frameInfoMap_.erase(request->sequence()); + completeRequest(request); + return; + } + + request->_d()->metadata().set(controls::SensorTimestamp, + buffer->metadata().timestamp); + + /* Ought we do something with the sensor's controls here...? */ + MaliC55CameraData *data = cameraData(request->_d()->camera()); + data->ipa_->fillParams(request->sequence(), info->paramBuffer->cookie()); +} + void PipelineHandlerMaliC55::paramsComputed(unsigned int requestId, uint32_t bytesused) { MaliC55FrameInfo &frameInfo = frameInfoMap_[requestId]; @@ -1645,18 +1789,27 @@ void PipelineHandlerMaliC55::paramsComputed(unsigned int requestId, uint32_t byt /* * Queue buffers for stats and params, then queue buffers to the capture - * video devices. + * video devices if we're running in Inline mode or with the TPG. + * + * If we're running in M2M buffers have been queued to the capture + * devices at queuePendingRequests() time and here we only have to queue + * buffers to the IVC input to start a transfer. */ frameInfo.paramBuffer->_d()->metadata().planes()[0].bytesused = bytesused; params_->queueBuffer(frameInfo.paramBuffer); stats_->queueBuffer(frameInfo.statBuffer); - for (auto &[stream, buffer] : request->buffers()) { - MaliC55Pipe *pipe = pipeFromStream(data, stream); + if (!std::holds_alternative(data->input_)) { + for (auto &[stream, buffer] : request->buffers()) { + MaliC55Pipe *pipe = pipeFromStream(data, stream); - pipe->cap->queueBuffer(buffer); + pipe->cap->queueBuffer(buffer); + } } + + if (std::holds_alternative(data->input_)) + ivc_->queueBuffer(frameInfo.rawBuffer); } void PipelineHandlerMaliC55::statsProcessed(unsigned int requestId, @@ -1793,6 +1946,9 @@ bool PipelineHandlerMaliC55::registerMemoryInputCamera() ivc_->bufferReady.connect(data->cru(), &RZG2LCRU::cruReturnBuffer); + V4L2VideoDevice *cruOutput = data->cru()->output(); + cruOutput->bufferReady.connect(this, &PipelineHandlerMaliC55::cruBufferReady); + return registerMaliCamera(std::move(data), sensor->device()->entity()->name()); } From patchwork Fri Mar 13 16:14:41 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Jacopo Mondi X-Patchwork-Id: 26298 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id C7517BE086 for ; Fri, 13 Mar 2026 16:14:59 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 7917F626EE; Fri, 13 Mar 2026 17:14:59 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="DXO+xN24"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 2F25F626BE for ; Fri, 13 Mar 2026 17:14:57 +0100 (CET) Received: from [192.168.224.131] (unknown [37.159.122.93]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 58DEF103D; Fri, 13 Mar 2026 17:13:47 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1773418428; bh=deapVA1Shx6XLGOt2/xUg2Rv4+jLetI/mAuFJF4dFRw=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=DXO+xN24+oeVOj5lI2w0iF1e7eVJdJXY6x12M5qujvpXIPdMeYGO1gl4Nhb6a+KD1 w8F7OF24FjSQRFIeYtD47OWYGze8TdCvt31oQbV7vp+/72lQT/U1BC3YUWAi9DeyiC Q3pNqaQvvz3toBInJiBq8fkNO+OuYfSDiiVogmCc= From: Jacopo Mondi Date: Fri, 13 Mar 2026 17:14:41 +0100 Subject: [PATCH v5 7/7] libcamera: mali-c55: Fix sensor size computation MIME-Version: 1.0 Message-Id: <20260313-mali-cru-v5-7-48f93e431294@ideasonboard.com> References: <20260313-mali-cru-v5-0-48f93e431294@ideasonboard.com> In-Reply-To: <20260313-mali-cru-v5-0-48f93e431294@ideasonboard.com> To: Daniel Scally , libcamera-devel@lists.libcamera.org Cc: Jacopo Mondi , =?utf-8?b?QmFybmFiw6Fz?= =?utf-8?q?_P=C5=91cze?= X-Mailer: b4 0.14.3 X-Developer-Signature: v=1; a=openpgp-sha256; l=1117; i=jacopo.mondi@ideasonboard.com; h=from:subject:message-id; bh=deapVA1Shx6XLGOt2/xUg2Rv4+jLetI/mAuFJF4dFRw=; b=owEBbQKS/ZANAwAKAXI0Bo8WoVY8AcsmYgBptDfzhgBv1razLzojfnR6Gbt04ATDTm87Gl/wy IQOsS1aOIqJAjMEAAEKAB0WIQS1xD1IgJogio9YOMByNAaPFqFWPAUCabQ38wAKCRByNAaPFqFW PDHtD/9j7CjbV8bOjiK1hr3dfK36LG3zpahgLmtvpVnr1hIXM40P+Sk2eUpv2R1k68NwG3IVdEi l4Q5tnmuOSpnCMlXjcsQsekWaCdcvxPVWN3jc4vU3qnTFcseBDqo9jOkxCoTS6ctE9TzYVbN5ID Fe34EWjxJHNH+eQdIIZYL0m1HEFL39olXTvPcS95xxp7OTWvwwLfpDrmMqsZi+ipeM06ZduWr5u oOKZ2Bp3Bl1YUSaN+CjtcbMtsBpMbjW284fDqIxp7y0+Y2uwJsRLx27d7D8sR8Cf5aDU92GBlaa 7Bzi4dpWwjg0MgIVm38LrnXdaW6RP+1Sk9Gjd76PhylbYZ1iFIwndoeI9oCBzIVYr5hMAtB8x1F vt57y2ByA2nFsFDWfqXEI781WnrJKjDPFDgWLe8wN+rWMI3mL3CiHA5YESSFIO76ba1m30mI7z0 2DEhyespQjlBxuUyFW0WQw4p5PjjU44e8kqBmomng9W9M9H7SiE/mwuj+jWZczAaJvRjOmSxEEO E0WGCDKns59qy4xy00d1OzGrDbLTSkEevZI8FFv1uQgwUaIF7NSRbh8Jjhy7Gbs+YMelniiD8bd ijQlGbienFDi9l19frtNjdTj+pYdPZILYNw42H02xI7+Z1yAEXvebKyTqlaHrr30svrvDYot9ok eQDA4CgPCfb+SIg== X-Developer-Key: i=jacopo.mondi@ideasonboard.com; a=openpgp; fpr=72392EDC88144A65C701EA9BA5826A2587AD026B X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" The last size in the list is always selected unconditionally because the current best distance is not updated properly during the search. Fix the size computation procedure which has an inverted assignment. Signed-off-by: Jacopo Mondi Reviewed-by: Daniel Scally Tested-by: Daniel Scally Reviewed-by: Barnabás Pőcze --- src/libcamera/pipeline/mali-c55/mali-c55.cpp | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/libcamera/pipeline/mali-c55/mali-c55.cpp b/src/libcamera/pipeline/mali-c55/mali-c55.cpp index 84c2a030b470..33c69342cd49 100644 --- a/src/libcamera/pipeline/mali-c55/mali-c55.cpp +++ b/src/libcamera/pipeline/mali-c55/mali-c55.cpp @@ -695,7 +695,7 @@ CameraConfiguration::Status MaliC55CameraConfiguration::validate() std::abs(static_cast(minSensorSize.height) - static_cast(size.height)); if (dist < distance) { - dist = distance; + distance = dist; bestSize = size; } }