From patchwork Wed Dec 10 16:15:16 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Plowman X-Patchwork-Id: 25466 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 4B211C3257 for ; Wed, 10 Dec 2025 16:41:03 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id D296E614B3; Wed, 10 Dec 2025 17:41:02 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="sjRZjNZU"; dkim-atps=neutral Received: from mail-wm1-x332.google.com (mail-wm1-x332.google.com [IPv6:2a00:1450:4864:20::332]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 2AA556149B for ; Wed, 10 Dec 2025 17:41:00 +0100 (CET) Received: by mail-wm1-x332.google.com with SMTP id 5b1f17b1804b1-477563e28a3so77485e9.1 for ; Wed, 10 Dec 2025 08:41:00 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; t=1765384859; x=1765989659; darn=lists.libcamera.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=Wtgl8b6OKI9BVILRPi19ZaWqlmqtJLVOkWTLBjKeLR0=; b=sjRZjNZU2oOh6HeNGCCIYqgu1URx9YQdn7XZX+TDndYNZrdkd6Ro2vgvBPZPhz3nYs oAEYH7V/OfWnzqKHXbMP7krLoLSRr18ScJPX7SXB01VEATN0/Fc/v721X68soSfrl2pJ 28ZiHVEMHxvtRbq2nJpC1QNxKkAT767U8zsC8bjc5XA/Q6IxUwr6UYzw++Mzg94Lp7EK MjIDbbvxLZXp2Ypz2zvtd8stTaJiZlIpWUFcz1aldeOnVPUffQTeNFQoeqa90Tm8DQM6 GyjQxM70kaRgB7lvFQAAxUyK0YfCWkCNFZd9PQ/RmUMI8ysC+ux8bzsTq9ifO6wGMeXS wTOw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1765384859; x=1765989659; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=Wtgl8b6OKI9BVILRPi19ZaWqlmqtJLVOkWTLBjKeLR0=; b=hdDNOFuY3XbE4HU+BnjnXcg8PDPYxenzcJisR/NCafP9LOnvH4aNtD8K9yDgt0Yb2M uYtyj9GsqcPYcqOCWMVpylyZWT9P3v51eLZ1a7uT4wP7XCY7Uj938aZHKVAV+WAvOImm 71HXY7Ccb/vjsNlGo/ReF/NhRxec4dgK9Ud4qWLsgDcAlPj1tc5Pop72ZGQl2Et5jyzg ZGtAjhNep/Pqx1cMUySSjGGgRuVaPf5sMx9b1VuEQYYWs15cKeKmBkUwJCPbg4G2Szjy 3Vd0jSVdgIaatMBm/XLDXV1DXIDxiTJA2g2Yw5Pej2NcSV/KKH2kVhXBETLQh2JXA5Vz dwBw== X-Gm-Message-State: AOJu0Yyfs6JsP6mNRQlagpP2O6IRgySHZQKPK7c8LuwNue98SbpAJdqV LXq8QKp3z9sGV4J5o7l8sZlrIFUpERsfvcJTflUNNChTh3EIXzQQAbNJ3yqzU/Cr2zYO+Girpnq +AGxh X-Gm-Gg: ASbGncvFn8Xk+LKrGkQQF41Yw0lkx93jDe0DCTEbImvKySQhpK9o0VU02XhAfZe2uMU bnHRz1qi5m5rumC6CvBR3iC4+G8JHemK721hSXFv2iJK5Y/wbO0Gv9iojbmtugK5xR2fMQ4X7Pd Uew29gK3JWKMgIaFJ0k+Zj+OR4JxrbhDzmIFORFhrpQJV33XAJEIgaMPlbTctJfXqvoN1dOgGAY P1jTN6PhEkM15IKAN+uzsw1S7ePDy1VUZkWmYKjo2GPc0cIsl6qXFo/fKmn1jYAAeLgRlddHh83 V6Pa8TOk/RvrtD60qBjWov+EosBuAsygWO6RFNfBvcqSxJTpE6ZNSWmN+OtbbMqHoxPY3BBqDbf fsfJUNL0VgmxQC8V+lE1pncYVaQ9ZBD1fTbmT3vuZQ8JMML+ICqJ+763/y73s7CFOav++7bTFTi 5s5fLOnKXzQdV6XXF0AsWCCx/IA56LCGO8a4GmLUAvdXoLDjpSa1rKxEeN0cpGcxKoeosF7sPAw eeRIrhN2M4VmxLCivgW0eAn3ftpSg== X-Google-Smtp-Source: AGHT+IHPLrzKdRqu5yH4i1clWtwiIPDmVcjFEO46jOyHRWquJrJB76KskKEf2j/PqMGQtQ18yQtToQ== X-Received: by 2002:a05:600c:4818:b0:479:3a8e:c85c with SMTP id 5b1f17b1804b1-47a888ea47amr48775e9.18.1765384859249; Wed, 10 Dec 2025 08:40:59 -0800 (PST) Received: from davidp-pi5.pitowers.org ([2a00:1098:3142:1f:88ea:c658:5b20:5e46]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-47a88371e13sm1270415e9.11.2025.12.10.08.40.57 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 10 Dec 2025 08:40:58 -0800 (PST) From: David Plowman To: libcamera-devel@lists.libcamera.org Cc: David Plowman Subject: [PATCH 01/11] libcamera: Infrastructure to ask for "memory" cameras Date: Wed, 10 Dec 2025 16:15:16 +0000 Message-ID: <20251210164055.17856-2-david.plowman@raspberrypi.com> X-Mailer: git-send-email 2.47.3 In-Reply-To: <20251210164055.17856-1-david.plowman@raspberrypi.com> References: <20251210164055.17856-1-david.plowman@raspberrypi.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" "Memory" cameras are used to make it easier to process raw camera images that are stored in memory buffers. There may be no actual camera attached to the system at all. Memory cameras are not created when the system starts. Depending on the pipeline handler, it may be possible to open many memory cameras simultaneously, or none at all. So any time an application requests a memory camera, we check with the pipeline handler whether it can provide one. Signed-off-by: David Plowman --- include/libcamera/camera_manager.h | 4 ++ include/libcamera/internal/camera_manager.h | 5 ++ include/libcamera/internal/pipeline_handler.h | 8 +++ src/libcamera/camera_manager.cpp | 64 +++++++++++++++++++ src/libcamera/pipeline_handler.cpp | 23 +++++++ 5 files changed, 104 insertions(+) diff --git a/include/libcamera/camera_manager.h b/include/libcamera/camera_manager.h index 27835500..436db409 100644 --- a/include/libcamera/camera_manager.h +++ b/include/libcamera/camera_manager.h @@ -7,6 +7,7 @@ #pragma once +#include #include #include #include @@ -20,6 +21,7 @@ namespace libcamera { class Camera; +class PipelineHandler; class CameraManager : public Object, public Extensible { @@ -33,6 +35,8 @@ public: std::vector> cameras() const; std::shared_ptr get(std::string_view id); + std::vector memoryCameras() const; + std::shared_ptr getMemoryCamera(std::string_view id, std::string_view settings); static const std::string &version() { return version_; } diff --git a/include/libcamera/internal/camera_manager.h b/include/libcamera/internal/camera_manager.h index 755928ce..4dbc1dc8 100644 --- a/include/libcamera/internal/camera_manager.h +++ b/include/libcamera/internal/camera_manager.h @@ -55,6 +55,9 @@ private: void pipelineFactoryMatch(const PipelineHandlerFactoryBase *factory); void cleanup() LIBCAMERA_TSA_EXCLUDES(mutex_); + std::shared_ptr getMemoryCamera(const PipelineHandlerFactoryBase *factory, + std::string_view settings); + /* * This mutex protects * @@ -70,6 +73,8 @@ private: std::unique_ptr enumerator_; + std::vector memoryCameras_; + std::unique_ptr ipaManager_; const GlobalConfiguration configuration_; diff --git a/include/libcamera/internal/pipeline_handler.h b/include/libcamera/internal/pipeline_handler.h index b4f97477..cb631529 100644 --- a/include/libcamera/internal/pipeline_handler.h +++ b/include/libcamera/internal/pipeline_handler.h @@ -70,6 +70,14 @@ public: CameraManager *cameraManager() const { return manager_; } + virtual bool supportsMemoryCamera() { return false; } + + virtual std::shared_ptr createMemoryCamera([[maybe_unused]] DeviceEnumerator *enumerator, + [[maybe_unused]] std::string_view settings) + { + return nullptr; + } + protected: void registerCamera(std::shared_ptr camera); void hotplugMediaDevice(std::shared_ptr media); diff --git a/src/libcamera/camera_manager.cpp b/src/libcamera/camera_manager.cpp index 83510e06..734f5836 100644 --- a/src/libcamera/camera_manager.cpp +++ b/src/libcamera/camera_manager.cpp @@ -155,6 +155,13 @@ void CameraManager::Private::pipelineFactoryMatch(const PipelineHandlerFactoryBa { CameraManager *const o = LIBCAMERA_O_PTR(); + /* First check for any memory-to-memory camera pipelines. */ + { + std::shared_ptr pipe = factory->create(o); + if (pipe->supportsMemoryCamera()) + memoryCameras_.push_back(std::string(pipe->name())); + } + /* Provide as many matching pipelines as possible. */ while (1) { std::shared_ptr pipe = factory->create(o); @@ -167,6 +174,19 @@ void CameraManager::Private::pipelineFactoryMatch(const PipelineHandlerFactoryBa } } +std::shared_ptr CameraManager::Private::getMemoryCamera(const PipelineHandlerFactoryBase *factory, + std::string_view settings) +{ + CameraManager *const o = LIBCAMERA_O_PTR(); + + std::shared_ptr pipe = factory->create(o); + + pipe->moveToThread(this); + + return pipe->invokeMethod(&PipelineHandler::createMemoryCamera, + ConnectionTypeBlocking, enumerator_.get(), settings); +} + void CameraManager::Private::cleanup() { enumerator_->devicesAdded.disconnect(this); @@ -371,6 +391,9 @@ void CameraManager::stop() * Before calling this function the caller is responsible for ensuring that * the camera manager is running. * + * This function does not list any memory cameras (for processing raw images + * held in memory buffers) that may be available. + * * \context This function is \threadsafe. * * \return List of all available cameras @@ -409,6 +432,47 @@ std::shared_ptr CameraManager::get(std::string_view id) return nullptr; } +/** + * \brief List the pipeline handlers that support memory cameras + * + * Lists the pipeline handlers in the system that have indicated they + * support memory cameras. Memory cameras are used for processing raw + * camera images stored in memory buffers, where no corresponding + * camera is available. + * + * \return Vector of pipeline handler ids (strings) + */ +std::vector CameraManager::memoryCameras() const +{ + return _d()->memoryCameras_; +} + +/** + * \brief Get a memory camera from the named pipeline handler + * \param[in] id ID of the pipeline handler that should create the memory camera + * + * This function causes the named pipeline handler to create and return a + * memory camera to the application. Pipeline handlers may support multiple + * simultaneous memory cameras, or none at all. + * + * \return Shared pointer to Camera object or nullptr if no memory camera + * could be provided + */ +std::shared_ptr CameraManager::getMemoryCamera(std::string_view id, + std::string_view settings) +{ + for (const auto &name : _d()->memoryCameras_) { + if (name == id) { + const PipelineHandlerFactoryBase *factory; + factory = PipelineHandlerFactoryBase::getFactoryByName(name); + + return _d()->getMemoryCamera(factory, settings); + } + } + + return nullptr; +} + /** * \var CameraManager::cameraAdded * \brief Notify of a new camera added to the system diff --git a/src/libcamera/pipeline_handler.cpp b/src/libcamera/pipeline_handler.cpp index 5c469e5b..c0f3c84e 100644 --- a/src/libcamera/pipeline_handler.cpp +++ b/src/libcamera/pipeline_handler.cpp @@ -835,6 +835,29 @@ void PipelineHandler::disconnect() * \return The CameraManager for this pipeline handler */ +/** + * \fn PipelineHandler::supportsMemoryCamera() + * \brief Indicate whether this pipeline handler supports memory cameras. + * \return True if the pipeline handler supports memory cameras, otherwise false. + */ + +/** + * \fn PipelineHandler::createMemoryCamera() + * \brief Create a memory camera to process raw camera images from a memory buffer + * \param[in] enumerator The enumerator providing all media devices found in the + * system + * \param[in] settings A string of settings that the pipeline handler should use + * + * The pipeline handler should create a memory camera, which an application can + * use to process raw camera images that are stored in memory buffers (rather + * than being delivered from a live camera). + * + * The settings should indicate exactly what processing the application requires + * to be performed on the raw image. + * + * \return Shared pointer to a camera, or a nullptr if none could be created + */ + /** * \class PipelineHandlerFactoryBase * \brief Base class for pipeline handler factories From patchwork Wed Dec 10 16:15:17 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Plowman X-Patchwork-Id: 25467 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 29809C3257 for ; Wed, 10 Dec 2025 16:41:07 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id C9607614C0; Wed, 10 Dec 2025 17:41:06 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="b0qhvtxE"; dkim-atps=neutral Received: from mail-wm1-x330.google.com (mail-wm1-x330.google.com [IPv6:2a00:1450:4864:20::330]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 7879A6149B for ; Wed, 10 Dec 2025 17:41:01 +0100 (CET) Received: by mail-wm1-x330.google.com with SMTP id 5b1f17b1804b1-477b198f4bcso61194715e9.3 for ; Wed, 10 Dec 2025 08:41:01 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; t=1765384860; x=1765989660; darn=lists.libcamera.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=uWau+eh2Ecj16as7FhmAPsEXKhyf4sxG9O0J3vJHad8=; b=b0qhvtxEwV3tEPHOuaGqc4JCeg4COoT0mettg3lqiALvL4ntKLYoeyLPYgGBNWnzZl rBN8rHxgns66WbUd7l0hgnSfdQAMJ4DpvFJuPkmeMx8/aKaJrqPMtgk4ogsnv7whV/hX Kmxk/isFlF/jmGdluUj6qxbDG8QRo0FTEvBcfQk23TSJHL/Xn59JyPjPHDYik9lcUXIG MkIvTOkW5nODciBxHIhkaWAmWcOHxkD9dkilqMLJs7TryEUa80/HHMsVvvfJWZt5Xh7m 2clRzpyLiSy+AM53aAPdqh1w+UsPh0EJCAMHGMHJl2n/tdWtnjBArUlgxsOAN1q7QSEA IbdQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1765384860; x=1765989660; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=uWau+eh2Ecj16as7FhmAPsEXKhyf4sxG9O0J3vJHad8=; b=u4YXEetuHW0Hnbjs/TKOp6t0A+cQuDWufxBnPBmig10Pk8taxWbXld5ZRZ/wZg4YFi irCIdAElsbjoW4uZ0hXxqQxSc9OwDQPxZEbLuuMCfNYL94Uhmty0bXY98OFXUnqSIwZJ jaQl5rOGz0ck1cgOuUTB3mbbLuZv1dzoKOrMMN8WsfeVY2hF+hgtx/vym3M1J874jg1m j4aLSJ2Pjw7lAZuigw9X46y+Pw3IFhALBSUTMwNAzDbTakRCe0npwSLTZKT1/nXa4IcP eATLKvjzydVZMv9rwB6/5OPXiPHd9YV38HWI0UyVqnKDquh/0vPWhKP1irKFI0Ljyw/r Me4w== X-Gm-Message-State: AOJu0YyuaxDdOcsPJ0uGlv9E2fDwY0oQkvYC0YUiFxQdgQfA15WMCdzx IM9mwTtJpOiFVk1kfenOaU8XxeWq8AT4thLmstMu/6PXJJLfAlAxzDDnX8PG2QXbhYbFPSzIqyx i9b8n X-Gm-Gg: ASbGnctEa4+/J6oRjDV6ny6U4MMA8ZrxV4k/dQRb6JdiGXrZzGNDWqpprNp4EPspx1i 1jIIK+PjqZs9fZhr09HMMZ1/JB8rHKFFEebzDKQmlTdGtV5fRbB5q2gO0M40lILKm0g8pxNMO4Q 8ZDxBHQ01NIPIwJAUF25OGHQUGAx4II5zjnk7xYbnwhd4R48JS5VGrav4pMQP3vB6I+fRk1mKat 0KzOJhuZcX0AHhCIWkNq8jSH8hwFs8lx+WR2+9YKcEo9LW+K8xqDEw6w858Vyusp0uLJTU2oBkq X0aLKMofzUuhbm78HICR5jHipcpexJnT94XgMOXhvmW0wQKG6yJd06C10OB1sR6WkZC1t8f8XPL jToHqWBLyAOiMC5B+wHThKWRcyo+KkgxjoRL0XE0+mIU0hiIhclM/M/nIGUVU2tDZR6Pv+1CxdW bvb+pWv5VMxrlE3VD1zryCu0MukwGqK9QXxEixm7O+6aONnjcWSNjfkt5/FLaAx9n5H8j+QfbBN qcaZc1xjtQRTk5Yrgla+euaFSsxfg== X-Google-Smtp-Source: AGHT+IHIQwgYoQ13cFTStu76MMedA6pusDKtUegtSqJOxV5s0/BmYFX554HU5Xy5HuCFW3WYbuwmqQ== X-Received: by 2002:a05:600c:45cd:b0:471:672:3486 with SMTP id 5b1f17b1804b1-47a83759eb3mr29591035e9.15.1765384860432; Wed, 10 Dec 2025 08:41:00 -0800 (PST) Received: from davidp-pi5.pitowers.org ([2a00:1098:3142:1f:88ea:c658:5b20:5e46]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-47a88371e13sm1270415e9.11.2025.12.10.08.40.59 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 10 Dec 2025 08:40:59 -0800 (PST) From: David Plowman To: libcamera-devel@lists.libcamera.org Cc: David Plowman Subject: [PATCH 02/11] libcamera: Add a direction (input or output) to the stream configuration Date: Wed, 10 Dec 2025 16:15:17 +0000 Message-ID: <20251210164055.17856-3-david.plowman@raspberrypi.com> X-Mailer: git-send-email 2.47.3 In-Reply-To: <20251210164055.17856-1-david.plowman@raspberrypi.com> References: <20251210164055.17856-1-david.plowman@raspberrypi.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" We are now going to support input streams as well as output streams, for example for passing in a Bayer image for raw re-processing. Signed-off-by: David Plowman --- include/libcamera/stream.h | 10 ++++++++++ src/libcamera/stream.cpp | 6 ++++-- 2 files changed, 14 insertions(+), 2 deletions(-) diff --git a/include/libcamera/stream.h b/include/libcamera/stream.h index b5e8f0a9..39fed302 100644 --- a/include/libcamera/stream.h +++ b/include/libcamera/stream.h @@ -37,6 +37,11 @@ private: std::map> formats_; }; +enum class StreamDirection { + Input, + Output +}; + struct StreamConfiguration { StreamConfiguration(); StreamConfiguration(const StreamFormats &formats); @@ -46,6 +51,8 @@ struct StreamConfiguration { unsigned int stride; unsigned int frameSize; + StreamDirection direction; + unsigned int bufferCount; std::optional colorSpace; @@ -53,6 +60,8 @@ struct StreamConfiguration { Stream *stream() const { return stream_; } void setStream(Stream *stream) { stream_ = stream; } const StreamFormats &formats() const { return formats_; } + bool isOutput() const { return direction == StreamDirection::Output; } + bool isInput() const { return direction == StreamDirection::Input; } std::string toString() const; @@ -68,6 +77,7 @@ enum class StreamRole { StillCapture, VideoRecording, Viewfinder, + RawInput }; std::ostream &operator<<(std::ostream &out, StreamRole role); diff --git a/src/libcamera/stream.cpp b/src/libcamera/stream.cpp index f091487c..439c61c4 100644 --- a/src/libcamera/stream.cpp +++ b/src/libcamera/stream.cpp @@ -280,8 +280,8 @@ SizeRange StreamFormats::range(const PixelFormat &pixelformat) const * handlers provide StreamFormats. */ StreamConfiguration::StreamConfiguration() - : pixelFormat(0), stride(0), frameSize(0), bufferCount(0), - stream_(nullptr) + : pixelFormat(0), stride(0), frameSize(0), + direction(StreamDirection::Output), bufferCount(0), stream_(nullptr) { } @@ -409,6 +409,8 @@ std::ostream &operator<<(std::ostream &out, const StreamConfiguration &cfg) { out << cfg.size << "-" << cfg.pixelFormat << "/" << ColorSpace::toString(cfg.colorSpace); + if (cfg.direction == StreamDirection::Input) + out << "/in"; return out; } From patchwork Wed Dec 10 16:15:18 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Plowman X-Patchwork-Id: 25468 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 77DF2C326B for ; Wed, 10 Dec 2025 16:41:08 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 05288614A5; Wed, 10 Dec 2025 17:41:08 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="lZDyan4T"; dkim-atps=neutral Received: from mail-wm1-x332.google.com (mail-wm1-x332.google.com [IPv6:2a00:1450:4864:20::332]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 65062614B1 for ; Wed, 10 Dec 2025 17:41:02 +0100 (CET) Received: by mail-wm1-x332.google.com with SMTP id 5b1f17b1804b1-47795f6f5c0so46405385e9.1 for ; Wed, 10 Dec 2025 08:41:02 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; t=1765384862; x=1765989662; darn=lists.libcamera.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=8qIf10Gl0HoH/74okVhYV2SwichdzIgd8dv13WEYdyk=; b=lZDyan4TJAaNVdYcuruMwbUErwkwwd5imu1voPpGpJ5Q+bSjHVhpStIid4GaVjcgMJ VoP38tg9CZUr/AsG+9S2VgREm4XBhcsrNq9MMjLD60C+wN0pGqK9W5NX69uNRUgyIPJQ 1SE+OteBQR8iXSb8r7K8IWqBuQBr8CkokXka43W1JQ9TslHdm+MloEvCJrNhmdmaiEzY iPvHiMraIt31L7LA/03TTYjuiVW7GmoPh9Uyal2z0Il6ydKXUBoLCZChXhglMEHHcna3 u/whhl8kNu/rmxVWZXgrQ+BuZ61CAwyI67z9R0BzlMWRMCK/mA5VoDSv2rNtwyKJvXqv ADzw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1765384862; x=1765989662; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=8qIf10Gl0HoH/74okVhYV2SwichdzIgd8dv13WEYdyk=; b=eWDto0oluwmv+WmGI7v9glYif/RmCSyQNuMLhwMdhnDyAppL3wbvtOwOUPBKQWJQfC u39Oo+INpIwZVhopLT7IfsMSWuCHDWniIuGtYFs1BwtfaN+hgFq1np7HUrLLKCGFn8zy WHjljACH5R3Y6gOp0FSzrzNGgGN81sZ7LCHkI0k4D0MeUnl2R3AcO2JizVey0VflEpVG QSOZPXOU9vWanP9cjHGF3MdL46ixkm6F9uK3iUx3DBJu14ivom4ZvDeGrAi0JeZo2wDf pFtVZARhf4cSZnFR7Mr0QvwaAfTmuOpQm0lejY/8pmSXWv3xy7KL4xCv0cPY9GDQ1TQL pPJQ== X-Gm-Message-State: AOJu0YxAuQ+kL7U3Of+r4drbgkb9PwQedbfhkqajCbs0ZvUbLtvr9dxT xFnPVyWWw2TmgOvJ9U2P0Xf8DtVFCC+wiyrCahk7EYLHz7TGL8lCRnvWq7daKtu1KDjBXQf0fAf OnYpz X-Gm-Gg: ASbGncvdHHi9Mza+ekI/c5uIA+E6XGuIyxFez/U1aCOz+ZZCsP1wmiaBEy41denbO3Z UHg7uGeZceDNyqzvThF6PU+Qp382Ce424ZZr8m7FI4ZcQYlQgqw2LyGhJ4f9arixsE2yWtvUmj3 Rcs7YP8jqpo2XWTAhgLhDu3lqG/YdvyH91rsYuyEo+E849Y1FxbvKZkn76HAXyFe/+06w9aIjkQ dR/P/15FD6UE/8ven3tpW7lEf2iDRhpzT8WzQXmWpC/5ZA56Fnh4Of4thQ0oJpWWbjpAQdfR2Lu wcC6XbRIcCwh9LuF/LAQdtuUHcZRYJ9rNINmLtCx+Av7JseMAbx6VM5rAEdSRugjLvg4xZEQMu+ Ekq28/md+VbWe4Wrsni9aGQXUcm7q2gB94Sf227AKQ6+IjJd84OZO/5iM88Sb3RFfg/LeHtH6R4 cy7vmjGZkI+b8XzmDRPXsB0saSiyVCXQj+ln00/uBANibVxyNRAAXu+jXUnEQBrwbQawCU7nib8 Psp11jT5VZ6gMjkCFSawe4zG3iGEm5vPo2WSQoa X-Google-Smtp-Source: AGHT+IEgIn/mqhl99WKqU5WnIkNOcPnWJnVMAl7iGF2NU8xIgvlSLpiliP51wDTxEEFROhuUNXF7ZQ== X-Received: by 2002:a05:600c:3513:b0:479:3a89:121e with SMTP id 5b1f17b1804b1-47a8381e5a5mr24587125e9.37.1765384861397; Wed, 10 Dec 2025 08:41:01 -0800 (PST) Received: from davidp-pi5.pitowers.org ([2a00:1098:3142:1f:88ea:c658:5b20:5e46]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-47a88371e13sm1270415e9.11.2025.12.10.08.41.00 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 10 Dec 2025 08:41:00 -0800 (PST) From: David Plowman To: libcamera-devel@lists.libcamera.org Cc: David Plowman Subject: [PATCH 03/11] libcamera: sensor: Add CameraSensorMemory class Date: Wed, 10 Dec 2025 16:15:18 +0000 Message-ID: <20251210164055.17856-4-david.plowman@raspberrypi.com> X-Mailer: git-send-email 2.47.3 In-Reply-To: <20251210164055.17856-1-david.plowman@raspberrypi.com> References: <20251210164055.17856-1-david.plowman@raspberrypi.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Representation of a "camera" that actually takes its input from a memory buffer. libcamera is, unsurprisingly, very dependent on having a camera connected to the system. But sometimes we may want to process raw camera images from elsewhere, and we may not have a camera connected to the system at all. In such cases, the path of a "memory buffer" through the code is eased considerably by introducing the CameraSensorMemory class, which allows the memory buffer to behave, to an extent at least, like a real camera. Signed-off-by: David Plowman --- include/libcamera/internal/camera_sensor.h | 2 + .../libcamera/internal/camera_sensor_memory.h | 110 ++++++++ include/libcamera/internal/meson.build | 1 + src/libcamera/sensor/camera_sensor_memory.cpp | 241 ++++++++++++++++++ src/libcamera/sensor/meson.build | 1 + 5 files changed, 355 insertions(+) create mode 100644 include/libcamera/internal/camera_sensor_memory.h create mode 100644 src/libcamera/sensor/camera_sensor_memory.cpp diff --git a/include/libcamera/internal/camera_sensor.h b/include/libcamera/internal/camera_sensor.h index e6b72d22..5580d6ec 100644 --- a/include/libcamera/internal/camera_sensor.h +++ b/include/libcamera/internal/camera_sensor.h @@ -49,6 +49,8 @@ public: virtual CameraLens *focusLens() = 0; + virtual bool isMemory() const { return false; } + virtual const std::vector &mbusCodes() const = 0; virtual std::vector sizes(unsigned int mbusCode) const = 0; virtual Size resolution() const = 0; diff --git a/include/libcamera/internal/camera_sensor_memory.h b/include/libcamera/internal/camera_sensor_memory.h new file mode 100644 index 00000000..944d4c96 --- /dev/null +++ b/include/libcamera/internal/camera_sensor_memory.h @@ -0,0 +1,110 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2025, Raspberry Pi plc + * + * camera_sensor_memory.h - A fake camera sensor for reading raw data from memory + */ + +#pragma once + +#include +#include +#include + +#include + +#include "libcamera/internal/camera_sensor.h" + +namespace libcamera { + +class BayerFormat; +class Camera; +class CameraLens; +class MediaEntity; +class SensorConfiguration; + +struct CameraSensorProperties; + +enum class Orientation; + +LOG_DECLARE_CATEGORY(CameraSensor) + +class CameraSensorMemory : public CameraSensor, protected Loggable +{ +public: + CameraSensorMemory(const StreamConfiguration &rawInput, unsigned int mbusCode); + ~CameraSensorMemory(); + + static std::variant, int> + match(MediaEntity *entity); + + const std::string &model() const override { return model_; } + const std::string &id() const override { return id_; } + + const MediaEntity *entity() const override { return nullptr; } + V4L2Subdevice *device() override { return nullptr; } + + CameraLens *focusLens() override { return nullptr; } + + virtual bool isMemory() const override { return true; } + + const std::vector &mbusCodes() const override; + std::vector sizes(unsigned int mbusCode) const override; + Size resolution() const override; + + V4L2SubdeviceFormat getFormat(Span mbusCodes, + const Size &size, + const Size maxSize) const override; + int setFormat(V4L2SubdeviceFormat *format, + Transform transform = Transform::Identity) override; + int tryFormat(V4L2SubdeviceFormat *format) const override; + + int applyConfiguration(const SensorConfiguration &config, + Transform transform = Transform::Identity, + V4L2SubdeviceFormat *sensorFormat = nullptr) override; + + V4L2Subdevice::Stream imageStream() const override; + std::optional embeddedDataStream() const override; + V4L2SubdeviceFormat embeddedDataFormat() const override; + int setEmbeddedDataEnabled(bool enable) override; + + const ControlList &properties() const override; + int sensorInfo(IPACameraSensorInfo *info) const override; + Transform computeTransform(Orientation *orientation) const override; + BayerFormat::Order bayerOrder(Transform t) const override; + Orientation mountingOrientation() const override; + + const ControlInfoMap &controls() const override; + ControlList getControls(Span ids) override; + int setControls(ControlList *ctrls) override; + + const std::vector & + testPatternModes() const override { return testPatternModes_; } + int setTestPatternMode(controls::draft::TestPatternModeEnum mode) override; + const CameraSensorProperties::SensorDelays &sensorDelays() override; + +protected: + std::string logPrefix() const override; + +private: + LIBCAMERA_DISABLE_COPY(CameraSensorMemory) + + StreamConfiguration rawInput_; + + std::string model_; + std::string id_; + + BayerFormat bayerFormat_; + std::vector mbusCodes_; + + V4L2SubdeviceFormat v4l2SubdeviceFormat_; + + ControlInfoMap propertiesInfoMap_; + ControlInfoMap controlsInfoMap_; + ControlList properties_; + ControlList controls_; + + std::vector testPatternModes_; +}; + +} /* namespace libcamera */ diff --git a/include/libcamera/internal/meson.build b/include/libcamera/internal/meson.build index e9540a2f..9994baae 100644 --- a/include/libcamera/internal/meson.build +++ b/include/libcamera/internal/meson.build @@ -10,6 +10,7 @@ libcamera_internal_headers = files([ 'camera_lens.h', 'camera_manager.h', 'camera_sensor.h', + 'camera_sensor_memory.h', 'camera_sensor_properties.h', 'clock_recovery.h', 'control_serializer.h', diff --git a/src/libcamera/sensor/camera_sensor_memory.cpp b/src/libcamera/sensor/camera_sensor_memory.cpp new file mode 100644 index 00000000..8e344016 --- /dev/null +++ b/src/libcamera/sensor/camera_sensor_memory.cpp @@ -0,0 +1,241 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2025, Raspberry Pi plc + * + * camera_sensor_memory.cpp - A fake camera sensor for reading raw data from memory + */ + +#include "libcamera/internal/camera_sensor_memory.h" + +#include +#include +#include + +#include +#include + +#include +#include +#include +#include +#include +#include + +#include + +#include "libcamera/internal/bayer_format.h" +#include "libcamera/internal/formats.h" +#include "libcamera/internal/v4l2_subdevice.h" + +namespace libcamera { + +LOG_DECLARE_CATEGORY(CameraSensor) + +static bool v4l2SubdeviceFormatEqual(const V4L2SubdeviceFormat &lhs, const V4L2SubdeviceFormat &rhs) +{ + return lhs.code == rhs.code && lhs.size == rhs.size && lhs.colorSpace == rhs.colorSpace; +} + +CameraSensorMemory::CameraSensorMemory(const StreamConfiguration &rawInput, unsigned int mbusCode) + : rawInput_(rawInput), properties_(propertiesInfoMap_), controls_(controlsInfoMap_) +{ + model_ = "memory"; + + std::ostringstream oss; + oss << &rawInput; + id_ = oss.str(); + + /* The "camera" must appear to return the format the raw input wants. */ + bayerFormat_ = BayerFormat::fromPixelFormat(rawInput.pixelFormat); + mbusCodes_ = { mbusCode }; + + v4l2SubdeviceFormat_ = V4L2SubdeviceFormat{ + .code = mbusCode, + .size = rawInput.size, + .colorSpace = ColorSpace::Raw, + }; +} + +CameraSensorMemory::~CameraSensorMemory() = default; + +std::variant, int> +CameraSensorMemory::match([[maybe_unused]] MediaEntity *entity) +{ + return {}; +} + +const std::vector &CameraSensorMemory::mbusCodes() const +{ + return mbusCodes_; +} + +std::vector CameraSensorMemory::sizes(unsigned int mbusCode) const +{ + if (mbusCode == mbusCodes_[0]) + return { rawInput_.size }; + else + return {}; +} + +Size CameraSensorMemory::resolution() const +{ + return rawInput_.size; +} + +V4L2SubdeviceFormat CameraSensorMemory::getFormat(Span mbusCodes, + [[maybe_unused]] const Size &size, + const Size maxSize) const +{ + if (std::find(mbusCodes.begin(), mbusCodes.end(), mbusCodes_[0]) == mbusCodes.end()) + return {}; + + if (maxSize.width < rawInput_.size.width || maxSize.height < rawInput_.size.height) + return {}; + + return v4l2SubdeviceFormat_; +} + +int CameraSensorMemory::setFormat(V4L2SubdeviceFormat *format, + Transform transform) +{ + if (v4l2SubdeviceFormatEqual(*format, v4l2SubdeviceFormat_) && + transform == Transform::Identity) + return 0; + + return -EPERM; +} + +int CameraSensorMemory::tryFormat(V4L2SubdeviceFormat *format) const +{ + if (v4l2SubdeviceFormatEqual(*format, v4l2SubdeviceFormat_)) + return 0; + + return -EPERM; +} + +int CameraSensorMemory::applyConfiguration(const SensorConfiguration &config, + Transform transform, + V4L2SubdeviceFormat *sensorFormat) +{ + if (config.bitDepth != bayerFormat_.bitDepth || + config.outputSize != rawInput_.size || + config.binning.binX != 1 || config.binning.binY != 1 || + config.skipping.xOddInc != 1 || config.skipping.xEvenInc != 1 || + config.skipping.yOddInc != 1 || config.skipping.yEvenInc != 1 || + transform != Transform::Identity) + return -EPERM; + + if (sensorFormat) + *sensorFormat = v4l2SubdeviceFormat_; + + return 0; +} + +V4L2Subdevice::Stream CameraSensorMemory::imageStream() const +{ + return V4L2Subdevice::Stream(); +} + +std::optional CameraSensorMemory::embeddedDataStream() const +{ + return {}; +} + +V4L2SubdeviceFormat CameraSensorMemory::embeddedDataFormat() const +{ + return {}; +} + +int CameraSensorMemory::setEmbeddedDataEnabled(bool enable) +{ + return enable ? -ENOSTR : 0; +} + +const ControlList &CameraSensorMemory::properties() const +{ + return properties_; +} + +int CameraSensorMemory::sensorInfo([[maybe_unused]] IPACameraSensorInfo *info) const +{ + info->model = model(); + + info->bitsPerPixel = bayerFormat_.bitDepth; + info->cfaPattern = properties::draft::RGB; + + info->activeAreaSize = rawInput_.size; + info->analogCrop = Rectangle(rawInput_.size); + info->outputSize = rawInput_.size; + + /* + * These are meaningless for us, fill with ones rather than zeros because the + * code will divide by some of these numbers. + */ + info->pixelRate = 1; + info->minLineLength = 1; + info->maxLineLength = 1; + info->minFrameLength = 1; + info->maxFrameLength = 1; + + return 0; +} + +Transform CameraSensorMemory::computeTransform(Orientation *orientation) const +{ + *orientation = Orientation::Rotate0; + return Transform::Identity; +} + +BayerFormat::Order CameraSensorMemory::bayerOrder([[maybe_unused]] Transform t) const +{ + return bayerFormat_.order; +} + +Orientation CameraSensorMemory::mountingOrientation() const +{ + return Orientation::Rotate0; +} + +const ControlInfoMap &CameraSensorMemory::controls() const +{ + return *controls_.infoMap(); +} + +ControlList CameraSensorMemory::getControls([[maybe_unused]] Span ids) +{ + return ControlList(); +} + +int CameraSensorMemory::setControls([[maybe_unused]] ControlList *ctrls) +{ + return -EPERM; +} + +int CameraSensorMemory::setTestPatternMode([[maybe_unused]] controls::draft::TestPatternModeEnum mode) +{ + return -EPERM; +} + +const CameraSensorProperties::SensorDelays &CameraSensorMemory::sensorDelays() +{ + static constexpr CameraSensorProperties::SensorDelays defaultSensorDelays = { + .exposureDelay = 2, + .gainDelay = 1, + .vblankDelay = 2, + .hblankDelay = 2, + }; + + return defaultSensorDelays; /* but doesn't mean anything */ +} + +std::string CameraSensorMemory::logPrefix() const +{ + return "'memory'"; +} + +/* + * We're not going to register this camera sensor as it doesn't match media entities + * like other sensors. Pipeline handlers will have to call it explicitly. + */ + +} /* namespace libcamera */ diff --git a/src/libcamera/sensor/meson.build b/src/libcamera/sensor/meson.build index dce74ed6..b9b87612 100644 --- a/src/libcamera/sensor/meson.build +++ b/src/libcamera/sensor/meson.build @@ -3,6 +3,7 @@ libcamera_internal_sources += files([ 'camera_sensor.cpp', 'camera_sensor_legacy.cpp', + 'camera_sensor_memory.cpp', 'camera_sensor_properties.cpp', 'camera_sensor_raw.cpp', ]) From patchwork Wed Dec 10 16:15:19 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Plowman X-Patchwork-Id: 25469 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id BDD6BC3257 for ; Wed, 10 Dec 2025 16:41:09 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 93AD0614BD; Wed, 10 Dec 2025 17:41:08 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="aSayuMXD"; dkim-atps=neutral Received: from mail-wm1-x333.google.com (mail-wm1-x333.google.com [IPv6:2a00:1450:4864:20::333]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 170FA6149F for ; Wed, 10 Dec 2025 17:41:03 +0100 (CET) Received: by mail-wm1-x333.google.com with SMTP id 5b1f17b1804b1-477b5e0323bso6251245e9.0 for ; Wed, 10 Dec 2025 08:41:03 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; t=1765384862; x=1765989662; darn=lists.libcamera.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=iHr6Y5W5ilP5vo9pl+dVLtmHzFbYz74H+sbzpnc0Cuc=; b=aSayuMXDnAuBeukuMs2NAWUhRHHeXANUTApMgtFfI+U5ooqZndtP/znJ/ZNAjXeUkR /gcz/1yp1djGKvOGYjN5dz85lFGAzY8v4AEWh0edybFhISxc9tKI3eYwiSWYoV6LFDRe BDP0PM+rPnBPsiOwixKanRo5U6B5yCHT0Hc7kZDG/pRlpUGjRuOrsTSvr7WQODVEtaaN NXPZXG794FqhyFZwkpZvuRTstyWPMLtG8rDBAC3iam3y3G9EHaaX4/kbcpV363f55Taz 5njU78xT+1WzHqe9Z6W+ZNP/GwjHA4nVgpWayt68PcQpLQR2FdWN7XeEUBoExb+ylbo2 JoVQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1765384862; x=1765989662; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=iHr6Y5W5ilP5vo9pl+dVLtmHzFbYz74H+sbzpnc0Cuc=; b=iqPtbYx5q0CoBOoQ1Xoe5V4rTm3/f+BkMPVVgDD+LyezbbuMXydIQ+jG9qzfYec+cB ofC8NJyhaqD++bHgjrBsIo3qXIrXEk0Rv+W1M8Q+qglwAtoczkiCyPO8kmGY1+8WW9bz q/MXcB00e7c/Py9jRmGyG4vJVKJT2YS6MK3QxcvMuwuXpI166GtdjVpvhn2T/w1XY/cd 82vg4a+dOouRQIHQztniATWdj1m/M3kcAQq6BOZ4P8es2E0XOMfa87nZFYFQri5SoIoS o2JV2UweIRvbpctuiOqFHsF/7av7/uiNMnGqErG3nFvu9PIfIdKxUeN3AxsUJi0dwaxo NvEg== X-Gm-Message-State: AOJu0YwP+EDgMJ7ORrhYQQgKGgpOLDtj+F6m3w/5IS5b87/3RZhup+qg 6YgagjXK+ouIOdr7HMR7oEXBlK4sJjez8vFPlh4pyGDfBp3Tb7SPAV5ggt+ichxHTrtPwuCGSHQ daWnn X-Gm-Gg: ASbGncujz6RvEaCUxObk0wvyuqGVX0dWAgBNJff0EtQEPzHkjeCyH67+9uRuumZCrTs +xwIAck+sDX1MXXXEX8YnAi4wZrTOcc35HAg4TRFtvdR2DNiTvOwuujKW6cdrFcygYZ8dRSoMbH Afg4c5BbSWBqXO2S6U/4GXAPHoiZFe4/2o9y4z3tz12hzXuAwRKcjkzd5OxzWDnwWkHesq1lcmG v09YBzSyFL7z+N12AsBYSxPXAzdjc1XYoYRv2EU74Wm9oXGfGS3PJctDoZonakXjjUN5FLZzoOu wSsTfj1DvYoCFK3x13fOLcmY+QdnzzYSxqPAXstCzUhu/xVhrJ/20DJqNibNPI4gVgFYASicrau g/DMmiK9Ew1HDFndhqR0yMRS850YN1l4xIFKU2aIY3GU9pLQ0OLFtRJIZwgcSGs5Ht6goodnAIR WrsrHg2g8+JaJrFYIE+aSv65muV6r3f1zoFLKQBd9R8J3CrEgDjjxd8IuRarddEWVZU8NguK2RV iay/u03LgHZq/MQcZdH5stcq4W0kA== X-Google-Smtp-Source: AGHT+IG7V2NDW7TOkVpf8lUl9bntreePeaV3CZsoNZVnWp9JlpUAutSYPlaslmpQPaasKvufRq29CA== X-Received: by 2002:a05:600c:3505:b0:479:1348:c63e with SMTP id 5b1f17b1804b1-47a886d92b4mr333015e9.9.1765384862158; Wed, 10 Dec 2025 08:41:02 -0800 (PST) Received: from davidp-pi5.pitowers.org ([2a00:1098:3142:1f:88ea:c658:5b20:5e46]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-47a88371e13sm1270415e9.11.2025.12.10.08.41.01 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 10 Dec 2025 08:41:01 -0800 (PST) From: David Plowman To: libcamera-devel@lists.libcamera.org Cc: David Plowman Subject: [PATCH 04/11] pipeline: rpi: Add PipelineHandlerBase::bayerToMbusCode Date: Wed, 10 Dec 2025 16:15:19 +0000 Message-ID: <20251210164055.17856-5-david.plowman@raspberrypi.com> X-Mailer: git-send-email 2.47.3 In-Reply-To: <20251210164055.17856-1-david.plowman@raspberrypi.com> References: <20251210164055.17856-1-david.plowman@raspberrypi.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" In future we are going to need this function outside pisp.cpp, so move it to PipelineHandlerBase and amend any existing callers to use the new version. It is functionally identical. Signed-off-by: David Plowman --- .../pipeline/rpi/common/pipeline_base.cpp | 43 +++++++++++++++++ .../pipeline/rpi/common/pipeline_base.h | 1 + src/libcamera/pipeline/rpi/pisp/pisp.cpp | 47 +------------------ 3 files changed, 46 insertions(+), 45 deletions(-) diff --git a/src/libcamera/pipeline/rpi/common/pipeline_base.cpp b/src/libcamera/pipeline/rpi/common/pipeline_base.cpp index 9d65dc83..00b088fc 100644 --- a/src/libcamera/pipeline/rpi/common/pipeline_base.cpp +++ b/src/libcamera/pipeline/rpi/common/pipeline_base.cpp @@ -381,6 +381,49 @@ V4L2DeviceFormat PipelineHandlerBase::toV4L2DeviceFormat(const V4L2VideoDevice * return deviceFormat; } +const std::vector> BayerToMbusCodeMap{ + { { BayerFormat::BGGR, 8, BayerFormat::Packing::None }, MEDIA_BUS_FMT_SBGGR8_1X8, }, + { { BayerFormat::GBRG, 8, BayerFormat::Packing::None }, MEDIA_BUS_FMT_SGBRG8_1X8, }, + { { BayerFormat::GRBG, 8, BayerFormat::Packing::None }, MEDIA_BUS_FMT_SGRBG8_1X8, }, + { { BayerFormat::RGGB, 8, BayerFormat::Packing::None }, MEDIA_BUS_FMT_SRGGB8_1X8, }, + { { BayerFormat::BGGR, 10, BayerFormat::Packing::None }, MEDIA_BUS_FMT_SBGGR10_1X10, }, + { { BayerFormat::GBRG, 10, BayerFormat::Packing::None }, MEDIA_BUS_FMT_SGBRG10_1X10, }, + { { BayerFormat::GRBG, 10, BayerFormat::Packing::None }, MEDIA_BUS_FMT_SGRBG10_1X10, }, + { { BayerFormat::RGGB, 10, BayerFormat::Packing::None }, MEDIA_BUS_FMT_SRGGB10_1X10, }, + { { BayerFormat::BGGR, 12, BayerFormat::Packing::None }, MEDIA_BUS_FMT_SBGGR12_1X12, }, + { { BayerFormat::GBRG, 12, BayerFormat::Packing::None }, MEDIA_BUS_FMT_SGBRG12_1X12, }, + { { BayerFormat::GRBG, 12, BayerFormat::Packing::None }, MEDIA_BUS_FMT_SGRBG12_1X12, }, + { { BayerFormat::RGGB, 12, BayerFormat::Packing::None }, MEDIA_BUS_FMT_SRGGB12_1X12, }, + { { BayerFormat::BGGR, 14, BayerFormat::Packing::None }, MEDIA_BUS_FMT_SBGGR14_1X14, }, + { { BayerFormat::GBRG, 14, BayerFormat::Packing::None }, MEDIA_BUS_FMT_SGBRG14_1X14, }, + { { BayerFormat::GRBG, 14, BayerFormat::Packing::None }, MEDIA_BUS_FMT_SGRBG14_1X14, }, + { { BayerFormat::RGGB, 14, BayerFormat::Packing::None }, MEDIA_BUS_FMT_SRGGB14_1X14, }, + { { BayerFormat::BGGR, 16, BayerFormat::Packing::None }, MEDIA_BUS_FMT_SBGGR16_1X16, }, + { { BayerFormat::GBRG, 16, BayerFormat::Packing::None }, MEDIA_BUS_FMT_SGBRG16_1X16, }, + { { BayerFormat::GRBG, 16, BayerFormat::Packing::None }, MEDIA_BUS_FMT_SGRBG16_1X16, }, + { { BayerFormat::RGGB, 16, BayerFormat::Packing::None }, MEDIA_BUS_FMT_SRGGB16_1X16, }, + { { BayerFormat::BGGR, 16, BayerFormat::Packing::PISP1 }, MEDIA_BUS_FMT_SBGGR16_1X16, }, + { { BayerFormat::GBRG, 16, BayerFormat::Packing::PISP1 }, MEDIA_BUS_FMT_SGBRG16_1X16, }, + { { BayerFormat::GRBG, 16, BayerFormat::Packing::PISP1 }, MEDIA_BUS_FMT_SGRBG16_1X16, }, + { { BayerFormat::RGGB, 16, BayerFormat::Packing::PISP1 }, MEDIA_BUS_FMT_SRGGB16_1X16, }, + { { BayerFormat::RGGB, 16, BayerFormat::Packing::PISP1 }, MEDIA_BUS_FMT_SRGGB16_1X16, }, + { { BayerFormat::MONO, 16, BayerFormat::Packing::None }, MEDIA_BUS_FMT_Y16_1X16, }, + { { BayerFormat::MONO, 16, BayerFormat::Packing::PISP1 }, MEDIA_BUS_FMT_Y16_1X16, }, +}; + +unsigned int PipelineHandlerBase::bayerToMbusCode(const BayerFormat &bayer) +{ + const auto it = std::find_if(BayerToMbusCodeMap.begin(), BayerToMbusCodeMap.end(), + [bayer](const std::pair &match) { + return bayer == match.first; + }); + + if (it != BayerToMbusCodeMap.end()) + return it->second; + + return 0; +} + std::unique_ptr PipelineHandlerBase::generateConfiguration(Camera *camera, Span roles) { diff --git a/src/libcamera/pipeline/rpi/common/pipeline_base.h b/src/libcamera/pipeline/rpi/common/pipeline_base.h index 15628259..1f174473 100644 --- a/src/libcamera/pipeline/rpi/common/pipeline_base.h +++ b/src/libcamera/pipeline/rpi/common/pipeline_base.h @@ -208,6 +208,7 @@ public: static V4L2DeviceFormat toV4L2DeviceFormat(const V4L2VideoDevice *dev, const V4L2SubdeviceFormat &format, BayerFormat::Packing packingReq); + static unsigned int bayerToMbusCode(const BayerFormat &bayer); std::unique_ptr generateConfiguration(Camera *camera, Span roles) override; diff --git a/src/libcamera/pipeline/rpi/pisp/pisp.cpp b/src/libcamera/pipeline/rpi/pisp/pisp.cpp index ade31de1..c08210b4 100644 --- a/src/libcamera/pipeline/rpi/pisp/pisp.cpp +++ b/src/libcamera/pipeline/rpi/pisp/pisp.cpp @@ -55,55 +55,12 @@ enum class Isp : unsigned int { Input, Output0, Output1, TdnInput, TdnOutput, constexpr unsigned int DefaultCompressionOffset = 2048; constexpr unsigned int DefaultCompressionMode = 1; -const std::vector> BayerToMbusCodeMap{ - { { BayerFormat::BGGR, 8, BayerFormat::Packing::None }, MEDIA_BUS_FMT_SBGGR8_1X8, }, - { { BayerFormat::GBRG, 8, BayerFormat::Packing::None }, MEDIA_BUS_FMT_SGBRG8_1X8, }, - { { BayerFormat::GRBG, 8, BayerFormat::Packing::None }, MEDIA_BUS_FMT_SGRBG8_1X8, }, - { { BayerFormat::RGGB, 8, BayerFormat::Packing::None }, MEDIA_BUS_FMT_SRGGB8_1X8, }, - { { BayerFormat::BGGR, 10, BayerFormat::Packing::None }, MEDIA_BUS_FMT_SBGGR10_1X10, }, - { { BayerFormat::GBRG, 10, BayerFormat::Packing::None }, MEDIA_BUS_FMT_SGBRG10_1X10, }, - { { BayerFormat::GRBG, 10, BayerFormat::Packing::None }, MEDIA_BUS_FMT_SGRBG10_1X10, }, - { { BayerFormat::RGGB, 10, BayerFormat::Packing::None }, MEDIA_BUS_FMT_SRGGB10_1X10, }, - { { BayerFormat::BGGR, 12, BayerFormat::Packing::None }, MEDIA_BUS_FMT_SBGGR12_1X12, }, - { { BayerFormat::GBRG, 12, BayerFormat::Packing::None }, MEDIA_BUS_FMT_SGBRG12_1X12, }, - { { BayerFormat::GRBG, 12, BayerFormat::Packing::None }, MEDIA_BUS_FMT_SGRBG12_1X12, }, - { { BayerFormat::RGGB, 12, BayerFormat::Packing::None }, MEDIA_BUS_FMT_SRGGB12_1X12, }, - { { BayerFormat::BGGR, 14, BayerFormat::Packing::None }, MEDIA_BUS_FMT_SBGGR14_1X14, }, - { { BayerFormat::GBRG, 14, BayerFormat::Packing::None }, MEDIA_BUS_FMT_SGBRG14_1X14, }, - { { BayerFormat::GRBG, 14, BayerFormat::Packing::None }, MEDIA_BUS_FMT_SGRBG14_1X14, }, - { { BayerFormat::RGGB, 14, BayerFormat::Packing::None }, MEDIA_BUS_FMT_SRGGB14_1X14, }, - { { BayerFormat::BGGR, 16, BayerFormat::Packing::None }, MEDIA_BUS_FMT_SBGGR16_1X16, }, - { { BayerFormat::GBRG, 16, BayerFormat::Packing::None }, MEDIA_BUS_FMT_SGBRG16_1X16, }, - { { BayerFormat::GRBG, 16, BayerFormat::Packing::None }, MEDIA_BUS_FMT_SGRBG16_1X16, }, - { { BayerFormat::RGGB, 16, BayerFormat::Packing::None }, MEDIA_BUS_FMT_SRGGB16_1X16, }, - { { BayerFormat::BGGR, 16, BayerFormat::Packing::PISP1 }, MEDIA_BUS_FMT_SBGGR16_1X16, }, - { { BayerFormat::GBRG, 16, BayerFormat::Packing::PISP1 }, MEDIA_BUS_FMT_SGBRG16_1X16, }, - { { BayerFormat::GRBG, 16, BayerFormat::Packing::PISP1 }, MEDIA_BUS_FMT_SGRBG16_1X16, }, - { { BayerFormat::RGGB, 16, BayerFormat::Packing::PISP1 }, MEDIA_BUS_FMT_SRGGB16_1X16, }, - { { BayerFormat::RGGB, 16, BayerFormat::Packing::PISP1 }, MEDIA_BUS_FMT_SRGGB16_1X16, }, - { { BayerFormat::MONO, 16, BayerFormat::Packing::None }, MEDIA_BUS_FMT_Y16_1X16, }, - { { BayerFormat::MONO, 16, BayerFormat::Packing::PISP1 }, MEDIA_BUS_FMT_Y16_1X16, }, -}; - -unsigned int bayerToMbusCode(const BayerFormat &bayer) -{ - const auto it = std::find_if(BayerToMbusCodeMap.begin(), BayerToMbusCodeMap.end(), - [bayer](const std::pair &match) { - return bayer == match.first; - }); - - if (it != BayerToMbusCodeMap.end()) - return it->second; - - return 0; -} - uint32_t mbusCodeUnpacked16(unsigned int code) { BayerFormat bayer = BayerFormat::fromMbusCode(code); BayerFormat bayer16(bayer.order, 16, BayerFormat::Packing::None); - return bayerToMbusCode(bayer16); + return RPi::PipelineHandlerBase::bayerToMbusCode(bayer16); } uint8_t toPiSPBayerOrder(V4L2PixelFormat format) @@ -2219,7 +2176,7 @@ int PiSPCameraData::configureEntities(V4L2SubdeviceFormat sensorFormat, cfe_[Cfe::Output0].dev()->getFormat(&feOutputFormat); BayerFormat feOutputBayer = BayerFormat::fromV4L2PixelFormat(feOutputFormat.fourcc); - feFormat.code = bayerToMbusCode(feOutputBayer); + feFormat.code = RPi::PipelineHandlerBase::bayerToMbusCode(feOutputBayer); ret = feSubdev_->setFormat(feVideo0SourcePad, &feFormat); return ret; From patchwork Wed Dec 10 16:15:20 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Plowman X-Patchwork-Id: 25470 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id A9B00C326B for ; Wed, 10 Dec 2025 16:41:10 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 53682614B4; Wed, 10 Dec 2025 17:41:09 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="RUA5rFfl"; dkim-atps=neutral Received: from mail-wm1-x336.google.com (mail-wm1-x336.google.com [IPv6:2a00:1450:4864:20::336]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id BE945614A5 for ; Wed, 10 Dec 2025 17:41:03 +0100 (CET) Received: by mail-wm1-x336.google.com with SMTP id 5b1f17b1804b1-477ba2c1ca2so92553875e9.2 for ; Wed, 10 Dec 2025 08:41:03 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; t=1765384863; x=1765989663; darn=lists.libcamera.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=QLlvmGgcOMsYAJYOTv4cJBu9eq16xZuXXiVWbWJ1GqE=; b=RUA5rFflb2OdJLz8uy8UzkDOkx9H+GsJ87xzbYHxMyTouuspxy4I+479fSUQL8ozGA +X9n5WuhAGkVmsPRZUYHZko3nrim71uS1iAx8hrf/oIypeUtkyrm7Xpxo0TruttFjovc qpWXvyMOKkJMt2pLrLYywu1xoDIaYAkWcwfAwciyXwkF2IIIm6uvTPENYkCbZ4IqGpSV Ey5dOAwUdJj7nvmrGs7EbjfkqDLuNZdIeTl82Nyn7kJNl1GgLTX5aATJsKe06a5/tlZG JjrJWEazjbm1SMs35mwmaT0M/rIlMOr8h5Kf1fNPavhZXmA+z8N1/5pZ6Smsshypp+II 7pGg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1765384863; x=1765989663; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=QLlvmGgcOMsYAJYOTv4cJBu9eq16xZuXXiVWbWJ1GqE=; b=OjoPgkI2XveTdKZN5PGF/sUHR5d0NrBRTyAGqnn2FDQRP5mId9Eib04Ob0aghFk2F2 5UzsIyTJtK6/q0QLBpieg+51A5+Wg/9SkGJSCsBEEqBBXrOJWNKGVaFDhmkT/JACC+ry nDeCwkZwD6TrQmfgEeqtbbFBcCGq53uKHGSOmgEbDIdTKwePOq3Ta8OVT03ImXUvj1EZ JNl6/NcXY5jNQhD/WvJ3QkMn7t7lZVcJauSt4KYH+0Dd91IMkMiWzpPgNjIQmJXb6J6e TYV5tOTd1a3+LnYbQd9PdJrgqdMPEGGMykAwxDqTLddBc4fKKH5bzgIdOayBZeTBXlHi E4Zg== X-Gm-Message-State: AOJu0YwgCnjSfINWAIC9WtpinlC09W52DW9XTSkjQtPn9NzGuCllzAvM bFxLyvGa6oVZKluHpiJvAw5vaVLJ0toxWl2v9NBU5QBThkaqyQheI/O2yg4wPubCssPwW1Fmngj DK4ah X-Gm-Gg: ASbGnctCGphQD1GKwd8UKfW+nt5PKRq9eWvJ3JQ1Iax0+xX+8GnLNYxx2HeY0aouU2M 5gvEAxYGXzjPOq4e3e9zjog+xL/pZxmYm7X6IllEAHVAOpYd0YUI25KJjBDKV9IQpUtVjTo925y p+TQVXt5O9rUKOX4SWrilXlQqyJskWcIYKSvTy64G5T8TrgpEkv+QlUuy0wPKKA6unQyB1IhrJG kJaTqJAzUZvqNu2V33N2/RWfyQ+foqioB+K724EWJ+k5sHOAAHidbL4JCuZCi9AzpFPpPguPEWK 1D1TMBoBz4Q6MNQLFH/K+eMRyYemvmlozVRqgMaZi6f94oPa4DvsMINM9I7N3V9hvOMh23XTJuY a40XhND7y/vnRY8mZGE3WX+RA/98wKQu0NTO+Ihdg6YanFiC+OQJzk+yXdfsFVb3MlgAAn0XHUx DNcOB5MenG3q41Gh3uK0M/B9D0i0nU2or2xsExDonezU9W2R/3MktVSXPTgTqEHW7Ci9L+iFQl6 Z+W6iHMY9VBD7HEhPOzxzVkyKKUoA== X-Google-Smtp-Source: AGHT+IGbWLCI6qNynvO0NNDwj5j0b70UVlp4oHGjDa1pjXb0k76VKYw8AB/cMjVGs3gHab4XtlWfeQ== X-Received: by 2002:a05:600c:1909:b0:471:115e:87bd with SMTP id 5b1f17b1804b1-47a8384cfa1mr30526515e9.26.1765384862859; Wed, 10 Dec 2025 08:41:02 -0800 (PST) Received: from davidp-pi5.pitowers.org ([2a00:1098:3142:1f:88ea:c658:5b20:5e46]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-47a88371e13sm1270415e9.11.2025.12.10.08.41.02 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 10 Dec 2025 08:41:02 -0800 (PST) From: David Plowman To: libcamera-devel@lists.libcamera.org Cc: David Plowman Subject: [PATCH 05/11] ipa: rpi: cam_helper: Add CamHelperDefault class Date: Wed, 10 Dec 2025 16:15:20 +0000 Message-ID: <20251210164055.17856-6-david.plowman@raspberrypi.com> X-Mailer: git-send-email 2.47.3 In-Reply-To: <20251210164055.17856-1-david.plowman@raspberrypi.com> References: <20251210164055.17856-1-david.plowman@raspberrypi.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" This will help make "memory cameras" work where there is no real sensor. This implementation does not really have to do anything except provide a few default values to satisfy any calling code. We use "memory cameras" to process raw camera images obtained from other sources, when in fact no actual camera may be present at all. Signed-off-by: David Plowman --- src/ipa/rpi/cam_helper/cam_helper_default.cpp | 45 +++++++++++++++++++ src/ipa/rpi/cam_helper/meson.build | 1 + 2 files changed, 46 insertions(+) create mode 100644 src/ipa/rpi/cam_helper/cam_helper_default.cpp diff --git a/src/ipa/rpi/cam_helper/cam_helper_default.cpp b/src/ipa/rpi/cam_helper/cam_helper_default.cpp new file mode 100644 index 00000000..99801cc3 --- /dev/null +++ b/src/ipa/rpi/cam_helper/cam_helper_default.cpp @@ -0,0 +1,45 @@ +/* SPDX-License-Identifier: BSD-2-Clause */ +/* + * Copyright (C) 2025, Raspberry Pi Ltd + * + * default camera helper for unknown sensors + */ + +#include +#include +#include +#include + +#include "cam_helper.h" + +using namespace RPiController; + +class CamHelperDefault : public CamHelper +{ +public: + CamHelperDefault(); + uint32_t gainCode(double gain) const override; + double gain(uint32_t gainCode) const override; +}; + +CamHelperDefault::CamHelperDefault() + : CamHelper({}, 0) +{ +} + +uint32_t CamHelperDefault::gainCode([[maybe_unused]] double gain) const +{ + return 0; +} + +double CamHelperDefault::gain([[maybe_unused]] uint32_t gainCode) const +{ + return 1.0; +} + +static CamHelper *create() +{ + return new CamHelperDefault(); +} + +static RegisterCamHelper reg("default", &create); diff --git a/src/ipa/rpi/cam_helper/meson.build b/src/ipa/rpi/cam_helper/meson.build index c1eac689..0c968033 100644 --- a/src/ipa/rpi/cam_helper/meson.build +++ b/src/ipa/rpi/cam_helper/meson.build @@ -2,6 +2,7 @@ rpi_ipa_cam_helper_sources = files([ 'cam_helper.cpp', + 'cam_helper_default.cpp', 'cam_helper_ov5647.cpp', 'cam_helper_imx219.cpp', 'cam_helper_imx283.cpp', From patchwork Wed Dec 10 16:15:21 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Plowman X-Patchwork-Id: 25471 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 9AEBEC326C for ; Wed, 10 Dec 2025 16:41:11 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 65EEF614BA; Wed, 10 Dec 2025 17:41:10 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="d+RxSkA+"; dkim-atps=neutral Received: from mail-wm1-x334.google.com (mail-wm1-x334.google.com [IPv6:2a00:1450:4864:20::334]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 8CCB8614AB for ; Wed, 10 Dec 2025 17:41:04 +0100 (CET) Received: by mail-wm1-x334.google.com with SMTP id 5b1f17b1804b1-477632d9326so47941455e9.1 for ; Wed, 10 Dec 2025 08:41:04 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; t=1765384864; x=1765989664; darn=lists.libcamera.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=xHq6lLERxDdLmM67XaICA48L1mdiYZYJFWVPoXLhIsw=; b=d+RxSkA+wLq68Uc2Rq3rqDU3u28gw+ehaGXVnjpTbmnYDBvCYqME96d8D4wHG2aC7F Gu2MjbHblpYEi7HVzfdtqIlFUz+Xv7kyGT4VdxU9lOdV397DXvAH0cUwDSHKQCvLlUR/ KtAZJ/DpiOHXZyyc3sf489ZWiGMXZT+asWL8hl78RjcsCFguRyD47sfy0tk9P4LZFA0w Wy8T4fBoFUJKA4QGSOCnihpj+Q1kdIEPK3Y5CI0yN9yXObmo2+O2HKuCOwpWF32qljLQ z2I5SvXsxNRQTCbk/m3LqAEQXZJ3rrSfLPsyqqdf9wmb3uAlLNLDMdyrg4VE3DMCDLlj rkog== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1765384864; x=1765989664; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=xHq6lLERxDdLmM67XaICA48L1mdiYZYJFWVPoXLhIsw=; b=M08qjaDDg3ThUhmoehEoD6XXQLhnw58oknZ0491QtgdvE2ZkW8v9Cxb+GBeXI7OrX7 g0N7QAEwo4NqiX1S5DLwugpne6LUjL6XU3YMzAxznZoI6xPiaxcugYLX/Rn4c1an7cbE q/kzn9StaL98Ztme8Zfs7Zc+H6b6BEsXerw5jRDXSznyXD4SSVNb4c4qjx6vSGhrMiQ7 cgdlVziwSbJHw1NlFk1AT0TEhclnoRVXaWGVn3rnkyG2WjPnYANKFe1AOQI+VQ5aj+m7 ceuZ2Jea+tnhBOMXPjovhh5v5l7AksAgaifpaMmBkXtZLB3OgBQs6yeeoE50QnvWRQyD 9ZUQ== X-Gm-Message-State: AOJu0YzqbymgBslI06S6y9u/NRCksqZh3f7UtDW4kvEBS9EDHgKmHyQy ZdRpXfA8Yz9v6BG/mkj64SBHgnDAUMdhsdi4Gw2tqnyeabNd0sC2GYFuDdLxINcB95uCLaepxMJ UTztp X-Gm-Gg: ASbGncubbbXkzF4L5MojJ/+NQUtRFdJGbaj3D4sfSQQvX9YeJtI+2bkw9uUV3toy+iZ LfswFoyA4lLr6hq1e9HOfDSPjRLHuw9Wiq4tfwPTp8mBRIFw1pJ+s+Duw/9yzpBIi+JVrgYHEq4 5kIByBL1j63N4m+v4042AvRL/xQJsNxa5Z8Jq6EdtHHQMbDU1pk3CDLafzte1MSDOZIPBO4Qiv0 ojT/O/FqS2TFjLWOTKp+ihpUnLkc+MClm8YYdMgj4NGJz6kVLhGjBKlpIEM2VrO4umL40yHDVgP Bc2Z658U/CfQY7pFumKa6N2SuBEcRFCceqn/RfqgvATC1rRV3qQa9xUe0Vp/HmfZJEi0hZLq/pR P36gpnf6gaMkMyWlY1aKus9E+2DOpiIwSmmD2mjgMn+upHA+8jtiymhRtrrS4a2gVj15v2eJsxs V5vf3lUaVLa8COUMNiO7gfk5c6Fp+oRtX0amjFkCvNRz8AGNmAJeOBGeGRLKNRHuO3VB6neXi2t Ovl+fYLgkwGwze2+4Mxg5FA6YRReQ== X-Google-Smtp-Source: AGHT+IGWCUCABkuOVZ1Ydh+rnYbbB5LdkrfVDglFf9NQJ1dKY6iVjW18ECqHtnKtVWgR/E66qfLXhw== X-Received: by 2002:a05:600c:820c:b0:477:a54a:acba with SMTP id 5b1f17b1804b1-47a8384c38bmr34671815e9.17.1765384863555; Wed, 10 Dec 2025 08:41:03 -0800 (PST) Received: from davidp-pi5.pitowers.org ([2a00:1098:3142:1f:88ea:c658:5b20:5e46]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-47a88371e13sm1270415e9.11.2025.12.10.08.41.02 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 10 Dec 2025 08:41:03 -0800 (PST) From: David Plowman To: libcamera-devel@lists.libcamera.org Cc: David Plowman Subject: [PATCH 06/11] libcamera: pipeline: rpi: Allow creation of the first "memory" camera Date: Wed, 10 Dec 2025 16:15:21 +0000 Message-ID: <20251210164055.17856-7-david.plowman@raspberrypi.com> X-Mailer: git-send-email 2.47.3 In-Reply-To: <20251210164055.17856-1-david.plowman@raspberrypi.com> References: <20251210164055.17856-1-david.plowman@raspberrypi.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" The Raspberry Pi PiSP pipeline handler is updated to indicate that it supports the creation of "memory" cameras, and extended actually to do so. Memory cameras are created on demand when applications request them. Subsequent commits will support the correct handling of camera configurations with "input" streams, that will require the use of these memory cameras. Signed-off-by: David Plowman --- .../pipeline/rpi/common/pipeline_base.cpp | 19 ++ .../pipeline/rpi/common/pipeline_base.h | 1 + src/libcamera/pipeline/rpi/pisp/pisp.cpp | 220 +++++++++++++----- 3 files changed, 188 insertions(+), 52 deletions(-) diff --git a/src/libcamera/pipeline/rpi/common/pipeline_base.cpp b/src/libcamera/pipeline/rpi/common/pipeline_base.cpp index 00b088fc..6878bdee 100644 --- a/src/libcamera/pipeline/rpi/common/pipeline_base.cpp +++ b/src/libcamera/pipeline/rpi/common/pipeline_base.cpp @@ -1217,6 +1217,25 @@ int CameraData::loadIPA(ipa::RPi::InitResult *result) return ipa_->init(settings, params, result); } +int CameraData::loadNamedIPA(std::string const &tuningFile, ipa::RPi::InitResult *result) +{ + int ret; + + ipa_ = IPAManager::createIPA(pipe(), 1, 1); + + if (!ipa_) + return -ENOENT; + + IPASettings settings(tuningFile, "default"); + ipa::RPi::InitParams params; + + ret = platformInitIpa(params); + if (ret) + return ret; + + return ipa_->init(settings, params, result); +} + int CameraData::configureIPA(const CameraConfiguration *config, ipa::RPi::ConfigResult *result) { ipa::RPi::ConfigParams params; diff --git a/src/libcamera/pipeline/rpi/common/pipeline_base.h b/src/libcamera/pipeline/rpi/common/pipeline_base.h index 1f174473..9453ae7e 100644 --- a/src/libcamera/pipeline/rpi/common/pipeline_base.h +++ b/src/libcamera/pipeline/rpi/common/pipeline_base.h @@ -72,6 +72,7 @@ public: int loadPipelineConfiguration(); int loadIPA(ipa::RPi::InitResult *result); + int loadNamedIPA(std::string const &settings, ipa::RPi::InitResult *result); int configureIPA(const CameraConfiguration *config, ipa::RPi::ConfigResult *result); virtual int platformInitIpa(ipa::RPi::InitParams ¶ms) = 0; virtual int platformConfigureIpa(ipa::RPi::ConfigParams ¶ms) = 0; diff --git a/src/libcamera/pipeline/rpi/pisp/pisp.cpp b/src/libcamera/pipeline/rpi/pisp/pisp.cpp index c08210b4..726ab063 100644 --- a/src/libcamera/pipeline/rpi/pisp/pisp.cpp +++ b/src/libcamera/pipeline/rpi/pisp/pisp.cpp @@ -815,6 +815,11 @@ public: bool match(DeviceEnumerator *enumerator) override; + bool supportsMemoryCamera() override; + + std::shared_ptr createMemoryCamera(DeviceEnumerator *enumerator, + std::string_view settings) override; + private: PiSPCameraData *cameraData(Camera *camera) { @@ -822,6 +827,8 @@ private: } int prepareBuffers(Camera *camera) override; + std::shared_ptr platformCreateCamera(std::unique_ptr &cameraData, + MediaDevice *cfe, MediaDevice *isp); int platformRegister(std::unique_ptr &cameraData, std::shared_ptr cfe, std::shared_ptr isp) override; @@ -889,10 +896,8 @@ bool PipelineHandlerPiSP::match(DeviceEnumerator *enumerator) PiSPCameraData *pisp = static_cast(cameraData.get()); - pisp->fe_ = SharedMemObject - ("pisp_frontend", true, pisp->pispVariant_); - pisp->be_ = SharedMemObject - ("pisp_backend", BackEnd::Config({}), pisp->pispVariant_); + pisp->fe_ = SharedMemObject("pisp_frontend", true, pisp->pispVariant_); + pisp->be_ = SharedMemObject("pisp_backend", BackEnd::Config({}), pisp->pispVariant_); if (!pisp->fe_.fd().isValid() || !pisp->be_.fd().isValid()) { LOG(RPI, Error) << "Failed to create ISP shared objects"; @@ -915,6 +920,84 @@ bool PipelineHandlerPiSP::match(DeviceEnumerator *enumerator) return false; } +bool PipelineHandlerPiSP::supportsMemoryCamera() +{ + return true; +} + +static bool get_variant(unsigned int be_version, const libpisp::PiSPVariant **variant) +{ + for (const auto &hw : libpisp::get_variants()) { + if (hw.BackEndVersion() == be_version) { + *variant = &hw; + return true; + } + } + + return false; +} + +std::shared_ptr +PipelineHandlerPiSP::createMemoryCamera(DeviceEnumerator *enumerator, + std::string_view settings) +{ + DeviceMatch isp("pispbe"); + isp.add("pispbe-input"); + isp.add("pispbe-config"); + isp.add("pispbe-output0"); + isp.add("pispbe-output1"); + isp.add("pispbe-tdn_output"); + isp.add("pispbe-tdn_input"); + isp.add("pispbe-stitch_output"); + isp.add("pispbe-stitch_input"); + std::shared_ptr ispDevice = acquireMediaDevice(enumerator, isp); + + if (!ispDevice) { + LOG(RPI, Warning) << "Unable to acquire ISP instance"; + return nullptr; + } + + const libpisp::PiSPVariant *variant = nullptr; + if (!get_variant(ispDevice->hwRevision(), &variant)) { + LOG(RPI, Warning) << "Failed to find variant"; + return nullptr; + } + + std::unique_ptr cameraData = + std::make_unique(this, *variant); + PiSPCameraData *pisp = + static_cast(cameraData.get()); + + /* Only need the back end, but creating the front end makes things easier */ + pisp->fe_ = SharedMemObject("pisp_frontend", true, pisp->pispVariant_); + pisp->be_ = SharedMemObject("pisp_backend", BackEnd::Config({}), pisp->pispVariant_); + + if (!pisp->fe_.fd().isValid() || !pisp->be_.fd().isValid()) { + LOG(RPI, Error) << "Failed to create ISP shared objects"; + return nullptr; + } + + /* + * Next, we want to do PipelineHandlerBase::registerCamera, including: + * loadIPA + * platformReigster + * loadPipelineConfiguration + */ + + /* loadIPA is a bit different for us as we have a filename */ + ipa::RPi::InitResult result; + std::string configFile = std::string(settings); + if (pisp->loadNamedIPA(configFile, &result)) { + LOG(RPI, Error) << "Failed to load a suitable IPA library"; + return nullptr; + } + LOG(RPI, Info) << "Loaded IPA library " << configFile << " for memory camera"; + + std::shared_ptr camera = platformCreateCamera(cameraData, nullptr, ispDevice.get()); + + return camera; +} + int PipelineHandlerPiSP::prepareBuffers(Camera *camera) { PiSPCameraData *data = cameraData(camera); @@ -1022,17 +1105,51 @@ int PipelineHandlerPiSP::prepareBuffers(Camera *camera) return 0; } -int PipelineHandlerPiSP::platformRegister(std::unique_ptr &cameraData, - std::shared_ptr cfe, - std::shared_ptr isp) +static int createCameraCfe(PiSPCameraData *data, MediaDevice *cfe, std::set &streams) { - PiSPCameraData *data = static_cast(cameraData.get()); - int ret; - MediaEntity *cfeImage = cfe->getEntityByName("rp1-cfe-fe_image0"); MediaEntity *cfeEmbedded = cfe->getEntityByName("rp1-cfe-embedded"); MediaEntity *cfeStats = cfe->getEntityByName("rp1-cfe-fe_stats"); MediaEntity *cfeConfig = cfe->getEntityByName("rp1-cfe-fe_config"); + + /* Locate and open the cfe video streams. */ + data->cfe_[Cfe::Output0] = RPi::Stream("CFE Image", cfeImage, StreamFlag::RequiresMmap); + data->cfe_[Cfe::Embedded] = RPi::Stream("CFE Embedded", cfeEmbedded); + data->cfe_[Cfe::Stats] = RPi::Stream("CFE Stats", cfeStats); + data->cfe_[Cfe::Config] = RPi::Stream("CFE Config", cfeConfig, + StreamFlag::Recurrent | StreamFlag::RequiresMmap); + + /* Wire up all the buffer connections. */ + data->cfe_[Cfe::Output0].dev()->bufferReady.connect(data, &PiSPCameraData::cfeBufferDequeue); + data->cfe_[Cfe::Stats].dev()->bufferReady.connect(data, &PiSPCameraData::cfeBufferDequeue); + data->cfe_[Cfe::Config].dev()->bufferReady.connect(data, &PiSPCameraData::cfeBufferDequeue); + data->cfe_[Cfe::Embedded].dev()->bufferReady.connect(data, &PiSPCameraData::cfeBufferDequeue); + + data->csi2Subdev_ = std::make_unique(cfe->getEntityByName("csi2")); + data->feSubdev_ = std::make_unique(cfe->getEntityByName("pisp-fe")); + data->csi2Subdev_->open(); + data->feSubdev_->open(); + + /* + * The below grouping is just for convenience so that we can easily + * iterate over all streams in one go. + */ + data->streams_.push_back(&data->cfe_[Cfe::Output0]); + data->streams_.push_back(&data->cfe_[Cfe::Config]); + data->streams_.push_back(&data->cfe_[Cfe::Stats]); + if (data->sensorMetadata_) + data->streams_.push_back(&data->cfe_[Cfe::Embedded]); + + data->ipa_->setCameraTimeout.connect(data, &PiSPCameraData::setCameraTimeout); + + /* Applications may ask for this stream. */ + streams.insert(&data->cfe_[Cfe::Output0]); + + return 0; +} + +static int createCameraBe(PiSPCameraData *data, MediaDevice *isp, std::set &streams) +{ MediaEntity *ispInput = isp->getEntityByName("pispbe-input"); MediaEntity *IpaPrepare = isp->getEntityByName("pispbe-config"); MediaEntity *ispOutput0 = isp->getEntityByName("pispbe-output0"); @@ -1042,13 +1159,6 @@ int PipelineHandlerPiSP::platformRegister(std::unique_ptr &came MediaEntity *ispStitchOutput = isp->getEntityByName("pispbe-stitch_output"); MediaEntity *ispStitchInput = isp->getEntityByName("pispbe-stitch_input"); - /* Locate and open the cfe video streams. */ - data->cfe_[Cfe::Output0] = RPi::Stream("CFE Image", cfeImage, StreamFlag::RequiresMmap); - data->cfe_[Cfe::Embedded] = RPi::Stream("CFE Embedded", cfeEmbedded); - data->cfe_[Cfe::Stats] = RPi::Stream("CFE Stats", cfeStats); - data->cfe_[Cfe::Config] = RPi::Stream("CFE Config", cfeConfig, - StreamFlag::Recurrent | StreamFlag::RequiresMmap); - /* Tag the ISP input stream as an import stream. */ data->isp_[Isp::Input] = RPi::Stream("ISP Input", ispInput, StreamFlag::ImportOnly); @@ -1071,34 +1181,15 @@ int PipelineHandlerPiSP::platformRegister(std::unique_ptr &came StreamFlag::Recurrent); /* Wire up all the buffer connections. */ - data->cfe_[Cfe::Output0].dev()->bufferReady.connect(data, &PiSPCameraData::cfeBufferDequeue); - data->cfe_[Cfe::Stats].dev()->bufferReady.connect(data, &PiSPCameraData::cfeBufferDequeue); - data->cfe_[Cfe::Config].dev()->bufferReady.connect(data, &PiSPCameraData::cfeBufferDequeue); data->isp_[Isp::Input].dev()->bufferReady.connect(data, &PiSPCameraData::beInputDequeue); data->isp_[Isp::Config].dev()->bufferReady.connect(data, &PiSPCameraData::beOutputDequeue); data->isp_[Isp::Output0].dev()->bufferReady.connect(data, &PiSPCameraData::beOutputDequeue); data->isp_[Isp::Output1].dev()->bufferReady.connect(data, &PiSPCameraData::beOutputDequeue); - data->cfe_[Cfe::Embedded].dev()->bufferReady.connect(data, &PiSPCameraData::cfeBufferDequeue); - - data->csi2Subdev_ = std::make_unique(cfe->getEntityByName("csi2")); - data->feSubdev_ = std::make_unique(cfe->getEntityByName("pisp-fe")); - data->csi2Subdev_->open(); - data->feSubdev_->open(); /* - * Open all CFE and ISP streams. The exception is the embedded data - * stream, which only gets opened below if the IPA reports that the sensor - * supports embedded data. - * * The below grouping is just for convenience so that we can easily * iterate over all streams in one go. */ - data->streams_.push_back(&data->cfe_[Cfe::Output0]); - data->streams_.push_back(&data->cfe_[Cfe::Config]); - data->streams_.push_back(&data->cfe_[Cfe::Stats]); - if (data->sensorMetadata_) - data->streams_.push_back(&data->cfe_[Cfe::Embedded]); - data->streams_.push_back(&data->isp_[Isp::Input]); data->streams_.push_back(&data->isp_[Isp::Output0]); data->streams_.push_back(&data->isp_[Isp::Output1]); @@ -1108,38 +1199,63 @@ int PipelineHandlerPiSP::platformRegister(std::unique_ptr &came data->streams_.push_back(&data->isp_[Isp::StitchInput]); data->streams_.push_back(&data->isp_[Isp::StitchOutput]); + /* Applications may ask for these stream. */ + streams.insert(&data->isp_[Isp::Output0]); + streams.insert(&data->isp_[Isp::Output1]); + + return 0; +} + +std::shared_ptr +PipelineHandlerPiSP::platformCreateCamera(std::unique_ptr &cameraData, + MediaDevice *cfe, MediaDevice *isp) +{ + PiSPCameraData *data = static_cast(cameraData.get()); + int ret; + std::set streams; + + /* cfe is a null pointer when making a memory camera that doesn't use the front end. */ + if (cfe) { + ret = createCameraCfe(data, cfe, streams); + if (ret) + return nullptr; + } + + ret = createCameraBe(data, isp, streams); + if (ret) + return nullptr; + + /* All the streams must open successfully. */ for (auto stream : data->streams_) { ret = stream->dev()->open(); if (ret) - return ret; + return nullptr; } - /* Write up all the IPA connections. */ + /* Wire up all the IPA connections. */ data->ipa_->prepareIspComplete.connect(data, &PiSPCameraData::prepareIspComplete); data->ipa_->processStatsComplete.connect(data, &PiSPCameraData::processStatsComplete); - data->ipa_->setCameraTimeout.connect(data, &PiSPCameraData::setCameraTimeout); - - /* - * List the available streams an application may request. At present, we - * do not advertise CFE Embedded and ISP Statistics streams, as there - * is no mechanism for the application to request non-image buffer formats. - */ - std::set streams; - streams.insert(&data->cfe_[Cfe::Output0]); - streams.insert(&data->isp_[Isp::Output0]); - streams.insert(&data->isp_[Isp::Output1]); /* Create and register the camera. */ - const std::string &id = data->sensor_->id(); + const std::string &id = cfe ? data->sensor_->id() : "memory"; std::shared_ptr camera = Camera::create(std::move(cameraData), id, streams); - PipelineHandler::registerCamera(std::move(camera)); LOG(RPI, Info) << "Registered camera " << id - << " to CFE device " << cfe->deviceNode() + << (cfe ? " to CFE device " + cfe->deviceNode() : "") << " and ISP device " << isp->deviceNode() << " using PiSP variant " << data->pispVariant_.Name(); + return camera; +} + +int PipelineHandlerPiSP::platformRegister(std::unique_ptr &cameraData, + std::shared_ptr cfe, + std::shared_ptr isp) +{ + std::shared_ptr camera = platformCreateCamera(cameraData, cfe.get(), isp.get()); + PipelineHandler::registerCamera(std::move(camera)); + return 0; } From patchwork Wed Dec 10 16:15:22 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Plowman X-Patchwork-Id: 25472 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 32CBDC3257 for ; Wed, 10 Dec 2025 16:41:12 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 0F1CA61509; Wed, 10 Dec 2025 17:41:11 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="Swvy/CW7"; dkim-atps=neutral Received: from mail-wm1-x329.google.com (mail-wm1-x329.google.com [IPv6:2a00:1450:4864:20::329]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 054226149B for ; Wed, 10 Dec 2025 17:41:05 +0100 (CET) Received: by mail-wm1-x329.google.com with SMTP id 5b1f17b1804b1-47796a837c7so68495e9.0 for ; Wed, 10 Dec 2025 08:41:05 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; t=1765384864; x=1765989664; darn=lists.libcamera.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=WP1VupwtxAszKVcn9Ser24SVpYpOcJsqWi7KlZMCpE8=; b=Swvy/CW7gin8LXWHuvplE/JbYo/SqApIXS7WXyr0hp0D/omA/PEHAyhmqaDqJiLV2G C3kGkbS+j6JtNOraMe2/8rv7GlIgIL4TMh/O7aIdJjap3cxFx0RyMJ7rjvPDQEXGqy6f zCHfnxUDjAd8ZobS674++A9f9Cq9+YZub+CokS4zoN4T6VFgqAPwFmfEGpzXeE0k1HkS tSZaFPmf1vU9XeI56XaLHxvYJQJOigaeU1MFFB3R1GN/pfcgudO91aQ10cs8imIZ5mta s+4vlKIrowFwE9M6IHNHIh0RSugTeefU0LyukLzwQsv1PVfIy2VyWabDcLCBtEXux5b+ TrDA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1765384864; x=1765989664; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=WP1VupwtxAszKVcn9Ser24SVpYpOcJsqWi7KlZMCpE8=; b=w/5ZIc4ARcUhQUTJUjm223Wkio5UMeN6oI6SBXY9xWg9Zk9srMfps9Sn/yM3mWF9LD FRiESg4piUhV6JLhH09kfOyEtoWjV5Ozx8Fuh2QgCxIJXWYgTwIOm49xgeHBd39+6xwj Zz0Kc59vhR73u9xy5tdNpN/B57Y2iBCWiimvn5qtoIa34o6qrguGjw3hsFsXC2xV9zs9 0VMAu+Et4mfJn70mq3TsFxO6ijHovmSAYnQAKu42ivapm3KaV8u7wZJqd3zF5MZCIB2Q Is3ImMQ99bDHfKfXe5IQPYzdOCX3ZflpxAARfGBk3r1PgDoDQ+ESe2zvroz4s/AffSt7 esXw== X-Gm-Message-State: AOJu0YyRAeEtfCNQnbmkbjHFCCKb4ZelF4bSqXn2NZM5zBWG4KcfZ4YA L4vm14x3tUWhzMbAq8VCu4LA4C7lV+mFNeN3qJefU36/s7fHIBcgVepaFfWRyAhpRC6KHmI1mBp e2CCR X-Gm-Gg: ASbGncvorlZW/hWfF0eRugQFlmwxdebFv6K6h7Zbb8GAkUkYTTwHkLtUiUZCuLGgmR3 rkYWoiaivKgzoftvlNDqJXYUbQBbH7VJwpYjXc32iKABcr3rJ9WA/xm5tzBjpwPPZBDgERt0adS qpGADoT2KoW6QqmUdQBbvOmXuxjgxcq9B8yjEw2srEhNDOTYnjG0nChsDj4Yih4MjrvvM+UCLIh ogaELhIKNx90hFLS0emURux+vp76CVZb5E+ZmLsudlwVdZ9pDKtuqfvdyPzo7d6VgJfX4Ki25Wz g2BGMABoHBsY4HJHa5WJ+MWlYoqPXsTqBcJGh+ioWcEtJx6UZPoqeJfaZ32GwxLHOuI9up81gE6 MyWOMjc/o8/eMaqm37M3V3fYPUvtvkl/3Dz80PI9IyWHUtubzEqWl1SO74x6LZNIRSekh5fPn5x hu92hxYl1xLGOx2196U8/23hgegwnn8QRMNiG674mjOy2UsnQgI7kfepR0OMhAuaRyd4wo4NWIq 0UMuuTZNr+nCMs0IrrBfzkY+sKVAA== X-Google-Smtp-Source: AGHT+IFJUQ+YnA3RrO7tWmKeEtcYivHfEGyWN+CNNgsv7zAkdeb0cv8U2QVPDYs2z+JtiN+rFJK6MA== X-Received: by 2002:a05:600c:190f:b0:47a:8154:33e3 with SMTP id 5b1f17b1804b1-47a83810f33mr30398065e9.28.1765384864172; Wed, 10 Dec 2025 08:41:04 -0800 (PST) Received: from davidp-pi5.pitowers.org ([2a00:1098:3142:1f:88ea:c658:5b20:5e46]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-47a88371e13sm1270415e9.11.2025.12.10.08.41.03 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 10 Dec 2025 08:41:03 -0800 (PST) From: David Plowman To: libcamera-devel@lists.libcamera.org Cc: David Plowman Subject: [PATCH 07/11] pipeline: rpi: Allow generation of raw input configurations Date: Wed, 10 Dec 2025 16:15:22 +0000 Message-ID: <20251210164055.17856-8-david.plowman@raspberrypi.com> X-Mailer: git-send-email 2.47.3 In-Reply-To: <20251210164055.17856-1-david.plowman@raspberrypi.com> References: <20251210164055.17856-1-david.plowman@raspberrypi.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Support the RawInput stream role, which makes a raw stream that is marked as an input (for example, for Bayer re-processing). Signed-off-by: David Plowman --- .../pipeline/rpi/common/pipeline_base.cpp | 23 ++++++++++++++++++- 1 file changed, 22 insertions(+), 1 deletion(-) diff --git a/src/libcamera/pipeline/rpi/common/pipeline_base.cpp b/src/libcamera/pipeline/rpi/common/pipeline_base.cpp index 6878bdee..76bcb2a4 100644 --- a/src/libcamera/pipeline/rpi/common/pipeline_base.cpp +++ b/src/libcamera/pipeline/rpi/common/pipeline_base.cpp @@ -436,12 +436,23 @@ PipelineHandlerBase::generateConfiguration(Camera *camera, Span colorSpace; + StreamDirection direction; if (roles.empty()) return config; - Size sensorSize = data->sensor_->resolution(); + /* + * When running memory to memory, without an actual sensor, we can't ask for its size. + * We'll make a CameraSensorMemory, to match the raw input stream configuration in due + * course, but for now use a placeholder size in this case. + */ + Size sensorSize = Size(640, 480); + if (data->sensor_) + sensorSize = data->sensor_->resolution(); + for (const StreamRole role : roles) { + direction = StreamDirection::Output; + switch (role) { case StreamRole::Raw: size = sensorSize; @@ -453,6 +464,15 @@ PipelineHandlerBase::generateConfiguration(Camera *camera, SpanispFormats(); pixelFormat = formats::YUV420; @@ -538,6 +558,7 @@ PipelineHandlerBase::generateConfiguration(Camera *camera, SpanaddConfiguration(cfg); } From patchwork Wed Dec 10 16:15:23 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Plowman X-Patchwork-Id: 25473 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id D69DFC32AF for ; Wed, 10 Dec 2025 16:41:12 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id BC016614B6; Wed, 10 Dec 2025 17:41:11 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="O6c3/Cks"; dkim-atps=neutral Received: from mail-wm1-x336.google.com (mail-wm1-x336.google.com [IPv6:2a00:1450:4864:20::336]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id A96A86149B for ; Wed, 10 Dec 2025 17:41:05 +0100 (CET) Received: by mail-wm1-x336.google.com with SMTP id 5b1f17b1804b1-477b1cc8fb4so49470485e9.1 for ; Wed, 10 Dec 2025 08:41:05 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; t=1765384865; x=1765989665; darn=lists.libcamera.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=agjDQf+CjyYZrdgafhj0qeb/BZpvrZ5zUAV15xS2+/M=; b=O6c3/CksFqloV4FNJ8UOXpxt1g4znewLGkT99JAwohWyZQQIcAtaXYjZV0gvQD8m/Q D4FWi9BCg4vnV3qxYg4neSgJZYLwcmgYhqKXmHPyksDhNbtaaxp5PP7EtMqWpOKYU3Py SylGEKGzvk/MW3IY4IhoaxjiOaVaiGcfBxi/1xNhBIMrFV8E+2vpCtDmOIHAeYY48mMn xAUW5D4xj+us5g76AxuCfilqmLC/9asr8x2xR8JG0elDQskjz6SpYayitF79/K0DfENG PlcvBlLe8fwkJs0DGlcVr7NKWInCGCg+YKoFV+6LXPAz9zp66XUC65NnSykho9jeQfO9 Qqhw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1765384865; x=1765989665; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=agjDQf+CjyYZrdgafhj0qeb/BZpvrZ5zUAV15xS2+/M=; b=GOPu/8fiJvgGVidDu8362Lco1MwSf6EX4AU1dqhblW6AL23DpnAZMvE9g8mZB6QaSj d4izsGGr+yZ5WkC6yxcgqamt00j7mvvSo/W9kfbTJ47U371cFwoeIxby+5wdPmvDntvs Bx4fGWMZPn/esHE9zV+9G1ToyWafHo/VLbTMwyCS8xPtMrg/us55H5kTdWK9//ngJ9Qo ZdjdBRUeUaJz2BAzSS5/0fN0rizGLwOMYExPQkohuI8qV1PGfunkxY4dBZt9Yueule2p CJfA2S67oCO509nIJ+pQq0yK5K77HpupFNSPqA0CFwEi9bZsIywRJ58ZM45/rN7M20Cd gYHA== X-Gm-Message-State: AOJu0Yz01Uc9NTnxFz8qtY7mv+xAPkBOTyfuIlG1+FWIo3c0t1b6oDHc DahZK39iqW7Po1PudwzRkhuUN7Zn69lAooMwFYZXRc1WTbTtnDMKafhm4SPbA4dLhFPz2mWX9NA bkb3r X-Gm-Gg: ASbGncv9+f9OJS6QHS8rq4Qe7vUMUH+sEUMyqKOlASvY4+axCd6A6YBTGQsI5uaxMGj 9cQU6whmYgjg70ri+OKEzMLLEFGbpafcIielPKTeR1zKqR14Q5UrxxwsnMCMPg+smjAqG04P+Dx tKts6H9sMAIrfSBIyI55vOPrBWRIrOPuITKEvC6fUxXefIoy0Bu1IWx1BpdmiSMlJ0h8/HR7PAn DG/bjEGj3obmi/69YTPrOEpficQFyjwIZnhNxzKhNr/FEArUuPvgZ5gU7JdUr4UPaFQ1REBC73j GH+cndLPuiAT94wovcOZtK7ZTAsxHNE4bjIAA0OVNa9OJWPKpqBgxkTJjZE9EMUaFjvXPrvqmOr zjZ0Zc3Nomnmv+6C03POQHRQDo04AIfoaDSic2Lnan+ss/8Bj/C9VDiSrBfvxs1IlaMBnwBArH9 fYD9aV6yRZZryB31/mgsxfQFuD/3TM5WvNuac6CmcBtcDTd+Gb21y/KQWvMelqaXXqvu9qQHKYh /0i3dYZynw4Od+meIdO/qa0mUbePA== X-Google-Smtp-Source: AGHT+IFdSRfIGBalCd1rdXJsa07qMN353a3ywIHtezOogRedeGao4EWdB0dV6aNQQyW6NJ3+kE0U3w== X-Received: by 2002:a05:600c:1c93:b0:471:d2f:7987 with SMTP id 5b1f17b1804b1-47a837b9a4cmr32309655e9.26.1765384864887; Wed, 10 Dec 2025 08:41:04 -0800 (PST) Received: from davidp-pi5.pitowers.org ([2a00:1098:3142:1f:88ea:c658:5b20:5e46]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-47a88371e13sm1270415e9.11.2025.12.10.08.41.04 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 10 Dec 2025 08:41:04 -0800 (PST) From: David Plowman To: libcamera-devel@lists.libcamera.org Cc: David Plowman Subject: [PATCH 08/11] libcamera: Don't wait for input streams to complete Date: Wed, 10 Dec 2025 16:15:23 +0000 Message-ID: <20251210164055.17856-9-david.plowman@raspberrypi.com> X-Mailer: git-send-email 2.47.3 In-Reply-To: <20251210164055.17856-1-david.plowman@raspberrypi.com> References: <20251210164055.17856-1-david.plowman@raspberrypi.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Don't add buffers corresponding to input streams to the "pending" list. These buffers don't "complete" in the normal way, which would stop a request from ever completing. Signed-off-by: David Plowman --- src/libcamera/request.cpp | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/src/libcamera/request.cpp b/src/libcamera/request.cpp index 60565f59..1be5e6e5 100644 --- a/src/libcamera/request.cpp +++ b/src/libcamera/request.cpp @@ -493,7 +493,9 @@ int Request::addBuffer(const Stream *stream, FrameBuffer *buffer, } buffer->_d()->setRequest(this); - _d()->pending_.insert(buffer); + /* Don't want to wait for input buffers as they don't "complete". */ + if (!stream->configuration().isInput()) + _d()->pending_.insert(buffer); if (fence && fence->isValid()) buffer->_d()->setFence(std::move(fence)); From patchwork Wed Dec 10 16:15:24 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Plowman X-Patchwork-Id: 25474 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 64454C326B for ; Wed, 10 Dec 2025 16:41:13 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 7F7C9614BB; Wed, 10 Dec 2025 17:41:12 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="GW77hlHX"; dkim-atps=neutral Received: from mail-wm1-x333.google.com (mail-wm1-x333.google.com [IPv6:2a00:1450:4864:20::333]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id CCACD614CD for ; Wed, 10 Dec 2025 17:41:06 +0100 (CET) Received: by mail-wm1-x333.google.com with SMTP id 5b1f17b1804b1-47796a837c7so68795e9.0 for ; Wed, 10 Dec 2025 08:41:06 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; t=1765384866; x=1765989666; darn=lists.libcamera.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=blvqyUtkLPoAGDVhsamTfHUttwbtDGPkwxR46PSscrY=; b=GW77hlHXXzbYBO/sGZCby0w8ecSfCPqUg1MSjhn315fDuo5jvZbtyolPVZLu2X3aj5 IagC4g+bnvhHhJbYaG8EGn7ukqmjWb7CBwdwkxY2AioY4CkAyXdnDjKsZY427SX4o3Qa bdy/0k/6g8XY/azQQMm3tF0wOOhrE7w4m1rgqExBRt9PBdKDWaaS5Ncd3BRSC1k8FZ2D rmCCJ1h90gOB5d5LLHUPp9YuChXy+s3XC1Kc7voZ007ZO7772cSuD0ryxteAnnCBSL9j QfGOPc8mBo4rJRshIzO34VcQ25jxt7WKV/bPtW5EKQjUiYnpy3nbmOIcUu+dEj/Bli6q qAlQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1765384866; x=1765989666; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=blvqyUtkLPoAGDVhsamTfHUttwbtDGPkwxR46PSscrY=; b=QfCGaJGz4W2Q8hHMhhOC13iNmzry4+5nHz05c9veuenxHzOhMuQrcXg2coe99uCmzN c9NuI5sij4AIBztyEl64GvEI5k8MauZ+5mErkOFVv0Dk0QNfGlrLUGAjXRVN96kTRPFI VXfuxKnlmUY4Q70iolvoqTP+qCfMpyXHPRcYJvTpxbtlAD1YliXQfex9a21zn6Zln4OF cROTSTbruAVPUX0MFTjjjtlWPKYKvlZZXa8TlNIp8EmOI5psVFopbeMFRpgfVcrDx/ZX Amrm9kmVQhxbJVyNuteeB/J0QwZ7ejAFEbJQejRdiK5BQEbhOx3VzuJLJyawmojQyjp+ qgPg== X-Gm-Message-State: AOJu0Ywnb3ZhOPvWAlvniLa8IhneMFPR2vaTmZ4WFVR9k1JCSjrErIOS hM5y0mo6F+2+MpmUyrTYhqjJMDuEZ+rvDhIVHi8Y6HP+AgejwwkEdRUwHc7Ri4tTa14zmJERxPE usZSY X-Gm-Gg: ASbGncu8mvm5UpHU7yq1VJZXoIkIRqdpHN3EbkiuGBDauBWjQS5af6YpE+ZCdP2T85c YH+tlWgJn/ZlsyzgonLf8ZcAzHhxmYTks2MeCnEdwUlgMdIeh6iP+EP5lJsVcvJr37v+jDCaDke 4T8t8zS0dN0Q3hL60o61Xab/EgA+zPgwLQ8gwJc0oH6I2l4k2tsm0H98aeaA84QM1x+VvagSR2z ZQJ7naYSe2zxaR0PB12CP2DWtVOICddJeDY6kc54hzN79PP+8UWCZHYlFrOWMsz9vqR2ABcRgN+ lgeob/oq0enAYuRhTdvW8ixAdmqSUiVockME8x4yVYJU2V9dIISe2Z8jkTSk8vQOKrkgzNm4xnN 2mhRTGNbvtMrpw6iPo7IOHstIrYWOmLYNN4giCdRwFq6W31Jm/T5MVZzNOviLL6qSKwH1ga3hlh +v0t2idRYximM3wh/6e/1Ore5XSY3dmr0bnHqb+/kA1vE1CzWsDm/qQELljHN2jsfrt8jl2wRNm puCapprR7RQNwAMx8J8E+GnEk0lX8sbtE87bNxz X-Google-Smtp-Source: AGHT+IGuo/hbgCB3jxXHWsxACNjKjgMny7K6/XQAAIvwKeKqpvoQNNfYUwx3/r0IKpA5dUb6UyfXnQ== X-Received: by 2002:a05:600c:3421:b0:479:1348:c61e with SMTP id 5b1f17b1804b1-47a83cadf28mr19865365e9.20.1765384865950; Wed, 10 Dec 2025 08:41:05 -0800 (PST) Received: from davidp-pi5.pitowers.org ([2a00:1098:3142:1f:88ea:c658:5b20:5e46]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-47a88371e13sm1270415e9.11.2025.12.10.08.41.05 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 10 Dec 2025 08:41:05 -0800 (PST) From: David Plowman To: libcamera-devel@lists.libcamera.org Cc: David Plowman Subject: [PATCH 09/11] ipa: rpi: Support memory cameras Date: Wed, 10 Dec 2025 16:15:24 +0000 Message-ID: <20251210164055.17856-10-david.plowman@raspberrypi.com> X-Mailer: git-send-email 2.47.3 In-Reply-To: <20251210164055.17856-1-david.plowman@raspberrypi.com> References: <20251210164055.17856-1-david.plowman@raspberrypi.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Add support for "memory cameras" (for Bayer reprocessing). Mostly very minor changes, but the way we fill in certain bits of metadata before running the IPAs changes significantly (see prepareIspFillMetadata). For example, regular sensors get device status information from the the sensor (or possibly from values we wrote to the sensor a few frames back), whereas "memory cameras" assume that the exposure/gain values passed in with the request controls will be the ones for setting up the tuning algorithms. Signed-off-by: David Plowman --- src/ipa/rpi/common/ipa_base.cpp | 135 +++++++++++++++++++++++++------- src/ipa/rpi/common/ipa_base.h | 5 ++ 2 files changed, 111 insertions(+), 29 deletions(-) diff --git a/src/ipa/rpi/common/ipa_base.cpp b/src/ipa/rpi/common/ipa_base.cpp index 04e737f6..7413c086 100644 --- a/src/ipa/rpi/common/ipa_base.cpp +++ b/src/ipa/rpi/common/ipa_base.cpp @@ -195,6 +195,8 @@ int32_t IpaBase::init(const IPASettings &settings, const InitParams ¶ms, Ini int32_t IpaBase::configure(const IPACameraSensorInfo &sensorInfo, const ConfigParams ¶ms, ConfigResult *result) { + isMemoryCamera_ = sensorInfo.model == "memory"; + sensorCtrls_ = params.sensorControls; if (!validateSensorControls()) { @@ -310,10 +312,13 @@ void IpaBase::start(const ControlList &controls, StartResult *result) /* * SwitchMode may supply updated exposure/gain values to use. * agcStatus_ will store these values for us to use until delayed_status values - * start to appear. + * start to appear. Initialise digital gain to 1, as we'll otherwise end up with + * completely black output if AGC isn't running (which is unusual, but does + * happen with memory cameras). */ agcStatus_.exposureTime = 0.0s; agcStatus_.analogueGain = 0.0; + agcStatus_.digitalGain = 1.0; metadata.get("agc.status", agcStatus_); if (agcStatus_.exposureTime && agcStatus_.analogueGain) { @@ -417,6 +422,75 @@ void IpaBase::unmapBuffers(const std::vector &ids) } } +void IpaBase::prepareIspFillMetadata(const PrepareParams ¶ms, unsigned int ipaContext, + Span &embeddedBuffer, bool &hdrChange) +{ + RPiController::Metadata &rpiMetadata = rpiMetadata_[ipaContext]; + + rpiMetadata.clear(); + hdrChange = false; + + if (!isMemoryCamera_) { + /* + * Regular cameras need to get the device status from the sensor, find the + * appropriate embedded data buffers (if any), and so forth. + */ + fillDeviceStatus(params.sensorControls, ipaContext); + fillSyncParams(params, ipaContext); + + if (params.buffers.embedded) { + /* + * Pipeline handler has supplied us with an embedded data buffer, + * we must pass it to the CamHelper for parsing. + */ + auto it = buffers_.find(params.buffers.embedded); + ASSERT(it != buffers_.end()); + embeddedBuffer = it->second.planes()[0]; + } + + /* + * AGC wants to know the algorithm status from the time it actioned the + * sensor exposure/gain changes. So fetch it from the metadata list + * indexed by the IPA cookie returned, and put it in the current frame + * metadata. + * + * Note if the HDR mode has changed, as things like tonemaps may need updating. + */ + AgcStatus agcStatus; + RPiController::Metadata &delayedMetadata = rpiMetadata_[params.delayContext]; + if (!delayedMetadata.get("agc.status", agcStatus)) { + rpiMetadata.set("agc.delayed_status", agcStatus); + hdrChange = agcStatus.hdr.mode != hdrStatus_.mode; + hdrStatus_ = agcStatus.hdr; + } + } else { + /* + * Memory cameras make the device status from exposure/gain information passed + * in the request controls. Making a "fake" device status like this will cause + * the correct values to be used to set up all the tuning algorithms for this + * frame. + */ + DeviceStatus deviceStatus = {}; + + const auto exposureTime = params.requestControls.get(controls::ExposureTime); + if (exposureTime) + deviceStatus.exposureTime = Duration(*exposureTime * 1000); + else + deviceStatus.exposureTime = Duration(1ms); + + const auto analogueGain = params.requestControls.get(controls::AnalogueGain); + if (analogueGain) + deviceStatus.analogueGain = *analogueGain; + else + deviceStatus.analogueGain = 1.0; + + LOG(IPARPI, Debug) << "Exposure time " << deviceStatus.exposureTime + << " AG " << deviceStatus.analogueGain; + + rpiMetadata.set("device.status", deviceStatus); + } +} + void IpaBase::prepareIsp(const PrepareParams ¶ms) { applyControls(params.requestControls); @@ -430,37 +504,14 @@ void IpaBase::prepareIsp(const PrepareParams ¶ms) unsigned int ipaContext = params.ipaContext % rpiMetadata_.size(); RPiController::Metadata &rpiMetadata = rpiMetadata_[ipaContext]; Span embeddedBuffer; - - rpiMetadata.clear(); - fillDeviceStatus(params.sensorControls, ipaContext); - fillSyncParams(params, ipaContext); - - if (params.buffers.embedded) { - /* - * Pipeline handler has supplied us with an embedded data buffer, - * we must pass it to the CamHelper for parsing. - */ - auto it = buffers_.find(params.buffers.embedded); - ASSERT(it != buffers_.end()); - embeddedBuffer = it->second.planes()[0]; - } + bool hdrChange; /* - * AGC wants to know the algorithm status from the time it actioned the - * sensor exposure/gain changes. So fetch it from the metadata list - * indexed by the IPA cookie returned, and put it in the current frame - * metadata. - * - * Note if the HDR mode has changed, as things like tonemaps may need updating. + * This call fills in certain metadata items that subsequent calls to prepare + * will expect to find. Note that different things happen here for regular + * sensors and "memory cameras". */ - AgcStatus agcStatus; - bool hdrChange = false; - RPiController::Metadata &delayedMetadata = rpiMetadata_[params.delayContext]; - if (!delayedMetadata.get("agc.status", agcStatus)) { - rpiMetadata.set("agc.delayed_status", agcStatus); - hdrChange = agcStatus.hdr.mode != hdrStatus_.mode; - hdrStatus_ = agcStatus.hdr; - } + prepareIspFillMetadata(params, ipaContext, embeddedBuffer, hdrChange); /* * This may overwrite the DeviceStatus using values from the sensor @@ -650,6 +701,18 @@ void IpaBase::setMode(const IPACameraSensorInfo &sensorInfo) */ mode_.sensitivity = helper_->getModeSensitivity(mode_); + if (isMemoryCamera_) { + /* + * For memory cameras, the min/max gain/exposure doesn't make much + * sense (and our "sensor" won't have any V4L2 controls), but we do + * need to set the camera mode information that we have so far, so + * that the control algorithms will see it. + */ + helper_->setCameraMode(mode_); + + return; + } + const ControlInfo &gainCtrl = sensorCtrls_.at(V4L2_CID_ANALOGUE_GAIN); const ControlInfo &exposureTimeCtrl = sensorCtrls_.at(V4L2_CID_EXPOSURE); @@ -690,6 +753,10 @@ void IpaBase::setCameraTimeoutValue() bool IpaBase::validateSensorControls() { + /* Don't need any of these controls with memory cameras. */ + if (isMemoryCamera_) + return true; + static const uint32_t ctrls[] = { V4L2_CID_ANALOGUE_GAIN, V4L2_CID_EXPOSURE, @@ -955,6 +1022,11 @@ void IpaBase::applyControls(const ControlList &controls) case controls::EXPOSURE_TIME: { RPiController::AgcAlgorithm *agc = dynamic_cast( controller_.getAlgorithm("agc")); + + /* Don't print a "no AGC" warning if this is a memory camera. */ + if (!agc && isMemoryCamera_) + break; + if (!agc) { LOG(IPARPI, Warning) << "Could not set EXPOSURE_TIME - no AGC algorithm"; @@ -981,6 +1053,11 @@ void IpaBase::applyControls(const ControlList &controls) case controls::ANALOGUE_GAIN: { RPiController::AgcAlgorithm *agc = dynamic_cast( controller_.getAlgorithm("agc")); + + /* Don't print a "no AGC" warning if this is a memory camera. */ + if (!agc && isMemoryCamera_) + break; + if (!agc) { LOG(IPARPI, Warning) << "Could not set ANALOGUE_GAIN - no AGC algorithm"; diff --git a/src/ipa/rpi/common/ipa_base.h b/src/ipa/rpi/common/ipa_base.h index 0ebd80db..94efd9a9 100644 --- a/src/ipa/rpi/common/ipa_base.h +++ b/src/ipa/rpi/common/ipa_base.h @@ -45,6 +45,9 @@ public: void mapBuffers(const std::vector &buffers) override; void unmapBuffers(const std::vector &ids) override; + void prepareIspFillMetadata(const PrepareParams ¶ms, unsigned int ipaContext, + Span &embeddedBuffer, bool &hdrChange); + void prepareIsp(const PrepareParams ¶ms) override; void processStats(const ProcessParams ¶ms) override; @@ -143,6 +146,8 @@ private: utils::Duration manualPeriod; } flickerState_; + bool isMemoryCamera_; + bool cnnEnableInputTensor_; bool awbEnabled_; }; From patchwork Wed Dec 10 16:15:25 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Plowman X-Patchwork-Id: 25475 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 80693C32DE for ; Wed, 10 Dec 2025 16:41:14 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 3F42A61595; Wed, 10 Dec 2025 17:41:14 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="iMgkf0wR"; dkim-atps=neutral Received: from mail-wm1-x332.google.com (mail-wm1-x332.google.com [IPv6:2a00:1450:4864:20::332]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 0C2E0614B1 for ; Wed, 10 Dec 2025 17:41:08 +0100 (CET) Received: by mail-wm1-x332.google.com with SMTP id 5b1f17b1804b1-47795f6f5c0so46406035e9.1 for ; Wed, 10 Dec 2025 08:41:08 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; t=1765384867; x=1765989667; darn=lists.libcamera.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=4ltnT0SLnKxShoSiZLAZpMr3VNz5MfJiqnPTdurewV0=; b=iMgkf0wRD2/UvybjARqfItqdvLOqFBVuAnYV/qe6wmlXa/SP6By8PeJqdAxlOdfq4j JOdAzR8GFe7wyHhEOZNNZ5A3bhlx040sjiSixxpIoeDbimyq1CVX77WTG7Frmn4lf8mg crOWhbwAstPf/lN5DZPw0gtALBxqebBGA6Pxep8EA/yKGUFh9FKgq+35y8IlbW4DsubC dLOUzsd0P50lktO2lSvzLkV82qLSH4WHZ3zZ+PzfTV6JOEaMajJVWvCzOKMghmpoiLWt Is/xVmpo4VYTGQWtbK/EFjibPXlvZvrxh8VpRc7gyWxXlzm2EAHSabb0DiP87riPiH4L tSrw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1765384867; x=1765989667; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=4ltnT0SLnKxShoSiZLAZpMr3VNz5MfJiqnPTdurewV0=; b=ubrOZmdELWCZ5jRbDpagBCpeu9Sckx10r5gYHiAIlNskYRe9AEYMfxYKO4J2LiXRsl nxpST7tbWU7jGFEru9mn0XwyFT6A6Z/6tV2Vayafpo1WpBH9yRZU92EhKK2E4Nn61UHj hdAbfKwgI6uUUCuW5wDWANd4d8de55tp4tdlTFDrWhlaMgEk3tAZLsmBzxNqCkA8CKGk heakEhXBcX5G5+e80Uuq9u8BSStLf6v2SsMj8b5695QeaTf9BPS/sqW0KthSL7uhu/Dy UEJUpQnqXoAxBBOjY5JdmDTkSeEfU2jk0wXwnUOHyPlQ1TSTzy/EoWaRNCiWd9Xd+gXE RJTQ== X-Gm-Message-State: AOJu0YxMV3s4UGoeVpx+TJl+r8e/EfZHhGeJOeJdW4MMxsA/ylN+3uYT SR27gq6ZRbjyel1JZooQBrBxOuu/aUyBcm7dlnTcDqlLyPudOUz+9ckS9tMsduYLB5izSkuoCV7 iK8+/ X-Gm-Gg: ASbGncv9+dTw6T18tOmq6r7NJHVZqc9GfIKGN7jgnBzt/i/CMLdy72cYHL7u/wks1CY A+8pGhW08ulO7VonL47CcbNBidC2bt0Fxl1/V/KzDzUMySKLwQBIgIHQWyyUuBxV2HMhaf/7Ac2 Un6oje5+o4b/Duzh5wsdg6En3AUmKb9PYHjYexcEb4Ns7Dh6kluPf9GNPMjljeCHArRxFcpVvYZ cmX3S2VxVGc/zUKe+ks33TXaDGFg51hbuuMzfAo0SkLAO6uY+SAZRaVM4YxHB/VT1BnxbRJOpap 3crsRh0PdNetp2yuahgoPJXLwZ8122fsuVgEIGO8fQst8LcPLmZ128PGDkB4xrk2ncvLD6F5SY1 zNrYqXmAgSQckOFFdzMftHmWdhsfpFbeVsFIpRFE0ftAKScZssyNxQEhDBOIoX0Zp8F6mQJ/PJ9 rUUKxnn7HcidXPcC5cLkxkY56zvqCUdZQaeSXif/CHPcQDKCp6yldHbiw67fAFIN7JJsyBPg6kb 6eQUq3H4pXGIxarPe4z5dylVkOrFS4g2WthM/qm X-Google-Smtp-Source: AGHT+IF1maZPO5gPAzBl4nUdrtf6FV1kCXUNsvAQCQP2+YJKs5CAypjFEHy4aW0NyeVnwfOPGPHFFA== X-Received: by 2002:a05:600c:3587:b0:477:7b9a:bb0a with SMTP id 5b1f17b1804b1-47a85ea3ff7mr14888085e9.21.1765384867116; Wed, 10 Dec 2025 08:41:07 -0800 (PST) Received: from davidp-pi5.pitowers.org ([2a00:1098:3142:1f:88ea:c658:5b20:5e46]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-47a88371e13sm1270415e9.11.2025.12.10.08.41.06 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 10 Dec 2025 08:41:06 -0800 (PST) From: David Plowman To: libcamera-devel@lists.libcamera.org Cc: David Plowman Subject: [PATCH 10/11] pipeline: rpi: Support memory cameras Date: Wed, 10 Dec 2025 16:15:25 +0000 Message-ID: <20251210164055.17856-11-david.plowman@raspberrypi.com> X-Mailer: git-send-email 2.47.3 In-Reply-To: <20251210164055.17856-1-david.plowman@raspberrypi.com> References: <20251210164055.17856-1-david.plowman@raspberrypi.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Mostly small changes to support memory cameras (for Bayer reprocessing) in the pipeline handler. Firstly, we make a "CameraSensorMemory" as the sensor as soon as we have the raw input buffer configuration, which helps the rest of the code to work with fewer changes. The PiSP platformConfigure method is refactored to split the Cfe and Isp parts into separate functions. Memory cameras then omit platformConfigureCfe but run platformConfigureIsp as before. Signed-off-by: David Plowman --- .../pipeline/rpi/common/pipeline_base.cpp | 36 ++++- .../pipeline/rpi/common/pipeline_base.h | 4 +- src/libcamera/pipeline/rpi/pisp/pisp.cpp | 141 ++++++++++++------ 3 files changed, 124 insertions(+), 57 deletions(-) diff --git a/src/libcamera/pipeline/rpi/common/pipeline_base.cpp b/src/libcamera/pipeline/rpi/common/pipeline_base.cpp index 76bcb2a4..2f02ec4c 100644 --- a/src/libcamera/pipeline/rpi/common/pipeline_base.cpp +++ b/src/libcamera/pipeline/rpi/common/pipeline_base.cpp @@ -20,6 +20,7 @@ #include #include "libcamera/internal/camera_lens.h" +#include "libcamera/internal/camera_sensor_memory.h" #include "libcamera/internal/ipa_manager.h" #include "libcamera/internal/v4l2_subdevice.h" @@ -170,15 +171,9 @@ CameraConfiguration::Status RPiCameraConfiguration::validate() status = validateColorSpaces(ColorSpaceFlag::StreamsShareColorSpace); /* - * Validate the requested transform against the sensor capabilities and - * rotation and store the final combined transform that configure() will - * need to apply to the sensor to save us working it out again. + * Separate the raw and output streams first, to make it easier to + * detect the raw reprocessing use case. */ - Orientation requestedOrientation = orientation; - combinedTransform_ = data_->sensor_->computeTransform(&orientation); - if (orientation != requestedOrientation) - status = Adjusted; - rawStreams_.clear(); outStreams_.clear(); unsigned int rawStreamIndex = 0; @@ -191,6 +186,31 @@ CameraConfiguration::Status RPiCameraConfiguration::validate() outStreams_.emplace_back(outStreamIndex++, &cfg); } + /* + * For the reprocessing use case, make a "memory camera" to match the + * raw input buffer. This will make all the subsequent code run more like + * the regular sensor case. + */ + if (!rawStreams_.empty() && rawStreams_[0].cfg->isInput()) { + LOG(RPI, Debug) << "Raw reprocessing use case for " << *rawStreams_[0].cfg; + const StreamConfiguration &rawInput = *rawStreams_[0].cfg; + BayerFormat bayerFormat = BayerFormat::fromPixelFormat(rawInput.pixelFormat); + unsigned int mbusCode = PipelineHandlerBase::bayerToMbusCode(bayerFormat); + data_->sensor_ = std::make_unique(rawInput, mbusCode); + /* We can fill in the only sensor format we support! */ + data_->sensorFormats_.emplace(mbusCode, data_->sensor_->sizes(mbusCode)); + } + + /* + * Validate the requested transform against the sensor capabilities and + * rotation and store the final combined transform that configure() will + * need to apply to the sensor to save us working it out again. + */ + Orientation requestedOrientation = orientation; + combinedTransform_ = data_->sensor_->computeTransform(&orientation); + if (orientation != requestedOrientation) + status = Adjusted; + /* Sort the streams so the highest resolution is first. */ std::sort(rawStreams_.begin(), rawStreams_.end(), [](auto &l, auto &r) { return l.cfg->size > r.cfg->size; }); diff --git a/src/libcamera/pipeline/rpi/common/pipeline_base.h b/src/libcamera/pipeline/rpi/common/pipeline_base.h index 9453ae7e..76a269bd 100644 --- a/src/libcamera/pipeline/rpi/common/pipeline_base.h +++ b/src/libcamera/pipeline/rpi/common/pipeline_base.h @@ -250,7 +250,7 @@ private: class RPiCameraConfiguration final : public CameraConfiguration { public: - RPiCameraConfiguration(const CameraData *data) + RPiCameraConfiguration(CameraData *data) : CameraConfiguration(), data_(data) { } @@ -292,7 +292,7 @@ public: std::optional rgbColorSpace_; private: - const CameraData *data_; + CameraData *data_; }; } /* namespace RPi */ diff --git a/src/libcamera/pipeline/rpi/pisp/pisp.cpp b/src/libcamera/pipeline/rpi/pisp/pisp.cpp index 726ab063..da3f5f40 100644 --- a/src/libcamera/pipeline/rpi/pisp/pisp.cpp +++ b/src/libcamera/pipeline/rpi/pisp/pisp.cpp @@ -756,6 +756,10 @@ public: bool adjustDeviceFormat(V4L2DeviceFormat &format) const; private: + int platformConfigureCfe(const RPi::RPiCameraConfiguration *rpiConfig, + V4L2DeviceFormat &cfeFormat); + int platformConfigureIsp(const RPi::RPiCameraConfiguration *rpiConfig, + V4L2DeviceFormat cfeFormat); int platformConfigure(const RPi::RPiCameraConfiguration *rpiConfig) override; int platformConfigureIpa([[maybe_unused]] ipa::RPi::ConfigParams ¶ms) override @@ -995,6 +999,12 @@ PipelineHandlerPiSP::createMemoryCamera(DeviceEnumerator *enumerator, std::shared_ptr camera = platformCreateCamera(cameraData, nullptr, ispDevice.get()); + int ret = pisp->loadPipelineConfiguration(); + if (ret) { + LOG(RPI, Error) << "Unable to load pipeline configuration"; + return nullptr; + } + return camera; } @@ -1274,10 +1284,14 @@ PiSPCameraData::platformValidate(RPi::RPiCameraConfiguration *rpiConfig) const } if (!rawStreams.empty()) { - rawStreams[0].dev = cfe_[Cfe::Output0].dev(); - StreamConfiguration *rawStream = rawStreams[0].cfg; BayerFormat bayer = BayerFormat::fromPixelFormat(rawStream->pixelFormat); + + if (rawStream->isInput()) + rawStreams[0].dev = isp_[Isp::Input].dev(); + else + rawStreams[0].dev = cfe_[Cfe::Output0].dev(); + /* * We cannot output CSI2 packed or non 16-bit output from the frontend, * so signal the output as unpacked 16-bits in these cases. @@ -1297,7 +1311,7 @@ PiSPCameraData::platformValidate(RPi::RPiCameraConfiguration *rpiConfig) const } rawStreams[0].format = - RPi::PipelineHandlerBase::toV4L2DeviceFormat(cfe_[Cfe::Output0].dev(), rawStream); + RPi::PipelineHandlerBase::toV4L2DeviceFormat(rawStreams[0].dev, rawStream); computeOptimalStride(rawStreams[0].format); } @@ -1491,13 +1505,13 @@ bool PiSPCameraData::adjustDeviceFormat(V4L2DeviceFormat &format) const return false; } -int PiSPCameraData::platformConfigure(const RPi::RPiCameraConfiguration *rpiConfig) +int PiSPCameraData::platformConfigureCfe(const RPi::RPiCameraConfiguration *rpiConfig, + V4L2DeviceFormat &cfeFormat) { const std::vector &rawStreams = rpiConfig->rawStreams_; - const std::vector &outStreams = rpiConfig->outStreams_; int ret; V4L2VideoDevice *cfe = cfe_[Cfe::Output0].dev(); - V4L2DeviceFormat cfeFormat; + V4L2DeviceFormat format; /* * See which streams are requested, and route the user @@ -1542,8 +1556,59 @@ int PiSPCameraData::platformConfigure(const RPi::RPiCameraConfiguration *rpiConf } ret = cfe->setFormat(&cfeFormat); - if (ret) + + /* CFE statistics output format. */ + format = {}; + format.fourcc = V4L2PixelFormat(V4L2_META_FMT_RPI_FE_STATS); + ret = cfe_[Cfe::Stats].dev()->setFormat(&format); + if (ret) { + LOG(RPI, Error) << "Failed to set format on CFE stats stream: " + << format.toString(); + return ret; + } + + /* CFE config format. */ + format = {}; + format.fourcc = V4L2PixelFormat(V4L2_META_FMT_RPI_FE_CFG); + ret = cfe_[Cfe::Config].dev()->setFormat(&format); + if (ret) { + LOG(RPI, Error) << "Failed to set format on CFE config stream: " + << format.toString(); return ret; + } + + /* + * Configure the CFE embedded data output format only if the sensor + * supports it. + */ + V4L2SubdeviceFormat embeddedFormat; + if (sensorMetadata_) { + sensor_->device()->getFormat(1, &embeddedFormat); + format = {}; + format.fourcc = V4L2PixelFormat(V4L2_META_FMT_SENSOR_DATA); + format.planes[0].size = embeddedFormat.size.width * embeddedFormat.size.height; + + LOG(RPI, Debug) << "Setting embedded data format " << format.toString(); + ret = cfe_[Cfe::Embedded].dev()->setFormat(&format); + if (ret) { + LOG(RPI, Error) << "Failed to set format on CFE embedded: " + << format; + return ret; + } + } + + configureEntities(rpiConfig->sensorFormat_, embeddedFormat); + configureCfe(); + + return 0; +} + +int PiSPCameraData::platformConfigureIsp(const RPi::RPiCameraConfiguration *rpiConfig, + V4L2DeviceFormat cfeFormat) +{ + int ret; + + const std::vector &outStreams = rpiConfig->outStreams_; /* Set the TDN and Stitch node formats in case they are turned on. */ isp_[Isp::TdnOutput].dev()->setFormat(&cfeFormat); @@ -1679,53 +1744,35 @@ int PiSPCameraData::platformConfigure(const RPi::RPiCameraConfiguration *rpiConf beEnabled_ = beEnables & (PISP_BE_RGB_ENABLE_OUTPUT0 | PISP_BE_RGB_ENABLE_OUTPUT1); - /* CFE statistics output format. */ - format = {}; - format.fourcc = V4L2PixelFormat(V4L2_META_FMT_RPI_FE_STATS); - ret = cfe_[Cfe::Stats].dev()->setFormat(&format); - if (ret) { - LOG(RPI, Error) << "Failed to set format on CFE stats stream: " - << format.toString(); - return ret; - } + if (beEnabled_) + configureBe(rpiConfig->yuvColorSpace_); - /* CFE config format. */ - format = {}; - format.fourcc = V4L2PixelFormat(V4L2_META_FMT_RPI_FE_CFG); - ret = cfe_[Cfe::Config].dev()->setFormat(&format); - if (ret) { - LOG(RPI, Error) << "Failed to set format on CFE config stream: " - << format.toString(); - return ret; - } + return 0; +} + +int PiSPCameraData::platformConfigure(const RPi::RPiCameraConfiguration *rpiConfig) +{ + /* What we call the cfeFormat here is also the input format for the ISP (Back End). */ + V4L2DeviceFormat cfeFormat; /* - * Configure the CFE embedded data output format only if the sensor - * supports it. + * First configure the CFE (if it's being used). In the case of memory cameras + * (Bayer reprocessing), there's no CFE but the raw input stream should already + * contain the correct ISP input format. */ - V4L2SubdeviceFormat embeddedFormat; - if (sensorMetadata_) { - sensor_->device()->getFormat(1, &embeddedFormat); - format = {}; - format.fourcc = V4L2PixelFormat(V4L2_META_FMT_SENSOR_DATA); - format.planes[0].size = embeddedFormat.size.width * embeddedFormat.size.height; - - LOG(RPI, Debug) << "Setting embedded data format " << format.toString(); - ret = cfe_[Cfe::Embedded].dev()->setFormat(&format); - if (ret) { - LOG(RPI, Error) << "Failed to set format on CFE embedded: " - << format; + if (cfe_[Cfe::Output0].dev()) { + /* Regular sensor. */ + int ret = platformConfigureCfe(rpiConfig, cfeFormat); + if (ret) return ret; - } + } else { + /* Memory camera. */ + cfeFormat = rpiConfig->rawStreams_[0].format; + rpiConfig->rawStreams_[0].cfg->setStream(&isp_[Isp::Input]); } - configureEntities(rpiConfig->sensorFormat_, embeddedFormat); - configureCfe(); - - if (beEnabled_) - configureBe(rpiConfig->yuvColorSpace_); - - return 0; + /* Finally configure the back end ISP. */ + return platformConfigureIsp(rpiConfig, cfeFormat); } void PiSPCameraData::platformStart() From patchwork Wed Dec 10 16:15:26 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: David Plowman X-Patchwork-Id: 25476 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 96AEDC326C for ; Wed, 10 Dec 2025 16:41:15 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 45967614EE; Wed, 10 Dec 2025 17:41:15 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="QICpIUTS"; dkim-atps=neutral Received: from mail-wm1-x329.google.com (mail-wm1-x329.google.com [IPv6:2a00:1450:4864:20::329]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id B9A8D614B2 for ; Wed, 10 Dec 2025 17:41:08 +0100 (CET) Received: by mail-wm1-x329.google.com with SMTP id 5b1f17b1804b1-47796a837c7so69085e9.0 for ; Wed, 10 Dec 2025 08:41:08 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; t=1765384868; x=1765989668; darn=lists.libcamera.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=Z2u6QWJXLLkERKnj/8SmMWdGzZUbKd7GPA9TOeKwsPo=; b=QICpIUTSVpG58ybyWpey7plogzGABP9dgjoL2sKfBwIYOeCvI2+Y8UyFfYVl6DSI6P It9YxRHGEmau1oUNkJa6c4Rb8nbEE8LHSExSDdib5r+gJdLybzT+ALNG9lnY49F+4Ici KDxHT7UlcVYHw8dtvE8U/go9fiyyeALdiMYuGoQ9jQkxyLapMXHgbQ1xakukxnu5vum8 J5T6ByRKnMN5IEkZbWNqPMvyKf9PsNlSfLQAi2vzBfvsIx2RJPMGtZJnGSGjXkPgUhD8 fjf3oGWdDL1TaaYcWTNwU0tK0g+bvnJRmgtKw0AoBFzBejoTB2V+gtpJKWkz5KACll8B Z6JQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1765384868; x=1765989668; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=Z2u6QWJXLLkERKnj/8SmMWdGzZUbKd7GPA9TOeKwsPo=; b=BFxkte2pamTLPIQe4/vAUyyLAgvmYz56KqMFDokKxQ/AguD0mQv5hEdQmXCRvC81oY UyL2wbw8j/oLsH+RDohIMlioqBjDVNvFeTd+KEzUagF+k4LinCSH0IiHg5lsIQt1hqoU VdTOGqR5dLdpi2mVNEKo7DDcCw8Kim9kSrYoR8m8je3M5CM8Ch3KZ0T1VPDHs1HOo2Iv cjar1FH+DAHxl2oggvxyorDc1rCf7z/xe7YG6WBGpfUpI1ZLAaASu+82FtO1holB3mqk 09VFcDlh5n6EoLzeLJAKzVUnDT4qomMh3UodiiNwXLbuhhO3FTf4nPcVoONbStZl83I5 p2+A== X-Gm-Message-State: AOJu0YxJvuZquxJ6FKFhb9IlPKyPVnzs2OmRQjJJnfxMMmk4efyt0qWN MP66NmwJ5U60OY+/pqFNBoyDysbP/Mq71WlB42ihPx7Ip/sBb79kxmWZf+9oX5uOAKgx4mej+XJ cWvJB X-Gm-Gg: ASbGncvUZcFiKqysy86qrKeVj3JNpZpyMUlZrn9GQT3WaEsD+HT5o332PMeEw5qTuYm VCotfmEjMRWKsuL8eIEa7WlR1wCwhaNC2Fmt6DSZwJD8wtUHv0Hyg0yG0YVZIHH755YUpw1pRwx /WJMEB0QCocWfAnpbl2+SS+4F1daqLF5bi0Etfll9MFlZaBfwyJ2HOCuF6HplZ8K4WIiE2jvZgG cA6t0b8mj2KPIyVmfDx1aCYL2ejKjrAxHgYaxRr+trA1uTjfzG8VtL+GAd7gu6WWbWRoI2/qEGw nsSslQcAFAN9KHAscpiSgwVq/q0IQDUwFRYCKh113T1Xfoss3fMOWh/9xnoTO9KfVFsCI4Qz8Bb +ta3TN+lITCOnGkxJFE/jTbi8a5CPHyXiZFIjaexpzxtxnWuKPIe4WgrAa1R/iuMULmbbtOHzu6 7Y5flyQihe05KPOODTZwCk7HBtzUERmM/89Gv2fCy3X71gv8k9ssAId/HVLUko/of1nnzQAagzK eCRm0uNt1RoVBO7OWT1+mZRK1e4mQ== X-Google-Smtp-Source: AGHT+IEBIlga7joPo2AUOCNwHCI7eLccqhlH1/s15HBJdBzNhBAkHnrsTbLp9eFhTv4xbM08/GHnZA== X-Received: by 2002:a05:600c:3487:b0:46f:b32e:5094 with SMTP id 5b1f17b1804b1-47a83811c87mr28065745e9.32.1765384867858; Wed, 10 Dec 2025 08:41:07 -0800 (PST) Received: from davidp-pi5.pitowers.org ([2a00:1098:3142:1f:88ea:c658:5b20:5e46]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-47a88371e13sm1270415e9.11.2025.12.10.08.41.07 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 10 Dec 2025 08:41:07 -0800 (PST) From: David Plowman To: libcamera-devel@lists.libcamera.org Cc: David Plowman Subject: [PATCH 11/11] pipline: rpi: Support memory cameras processing requests Date: Wed, 10 Dec 2025 16:15:26 +0000 Message-ID: <20251210164055.17856-12-david.plowman@raspberrypi.com> X-Mailer: git-send-email 2.47.3 In-Reply-To: <20251210164055.17856-1-david.plowman@raspberrypi.com> References: <20251210164055.17856-1-david.plowman@raspberrypi.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" This is the final commit in the set to support memory cameras (Bayer reprocessing). There are some small fix-ups to allow the camera system to start without the CFE being involved, and we add runMemoryCamera() which pushes a buffer from the request into the Back End ISP. Signed-off-by: David Plowman --- .../pipeline/rpi/common/pipeline_base.cpp | 23 ++++--- src/libcamera/pipeline/rpi/pisp/pisp.cpp | 63 +++++++++++++++++-- 2 files changed, 71 insertions(+), 15 deletions(-) diff --git a/src/libcamera/pipeline/rpi/common/pipeline_base.cpp b/src/libcamera/pipeline/rpi/common/pipeline_base.cpp index 2f02ec4c..48dbfaa5 100644 --- a/src/libcamera/pipeline/rpi/common/pipeline_base.cpp +++ b/src/libcamera/pipeline/rpi/common/pipeline_base.cpp @@ -772,15 +772,19 @@ int PipelineHandlerBase::start(Camera *camera, const ControlList *controls) /* A good moment to add an initial clock sample. */ data->wallClockRecovery_.addSample(); - /* - * Reset the delayed controls with the gain and exposure values set by - * the IPA. - */ - data->delayedCtrls_->reset(0); - data->state_ = CameraData::State::Idle; + /* A memory camera wouldn't have a front end device, so you would skip this. */ + if (data->frontendDevice()) { + /* + * Reset the delayed controls with the gain and exposure values set by + * the IPA. + */ + data->delayedCtrls_->reset(0); + + /* Enable SOF event generation. */ + data->frontendDevice()->setFrameStartEnabled(true); + } - /* Enable SOF event generation. */ - data->frontendDevice()->setFrameStartEnabled(true); + data->state_ = CameraData::State::Idle; data->platformStart(); @@ -807,7 +811,8 @@ void PipelineHandlerBase::stopDevice(Camera *camera) stream->dev()->streamOff(); /* Disable SOF event generation. */ - data->frontendDevice()->setFrameStartEnabled(false); + if (data->frontendDevice()) + data->frontendDevice()->setFrameStartEnabled(false); data->clearIncompleteRequests(); diff --git a/src/libcamera/pipeline/rpi/pisp/pisp.cpp b/src/libcamera/pipeline/rpi/pisp/pisp.cpp index da3f5f40..ce350378 100644 --- a/src/libcamera/pipeline/rpi/pisp/pisp.cpp +++ b/src/libcamera/pipeline/rpi/pisp/pisp.cpp @@ -780,6 +780,7 @@ private: void prepareCfe(); void prepareBe(uint32_t bufferId, bool stitchSwapBuffers); + void runMemoryCamera(); void tryRunPipeline() override; struct CfeJob { @@ -1244,7 +1245,9 @@ PipelineHandlerPiSP::platformCreateCamera(std::unique_ptr &came /* Wire up all the IPA connections. */ data->ipa_->prepareIspComplete.connect(data, &PiSPCameraData::prepareIspComplete); - data->ipa_->processStatsComplete.connect(data, &PiSPCameraData::processStatsComplete); + /* Connect IPA stats for hardware cameras; memory cams don't need it. */ + if (cfe) + data->ipa_->processStatsComplete.connect(data, &PiSPCameraData::processStatsComplete); /* Create and register the camera. */ const std::string &id = cfe ? data->sensor_->id() : "memory"; @@ -1787,8 +1790,11 @@ void PiSPCameraData::platformStart() cfeJobQueue_ = {}; - for (unsigned int i = 0; i < config_.numCfeConfigQueue; i++) - prepareCfe(); + /* A memory camera wouldn't be using the CFE. */ + if (cfe_[Cfe::Output0].dev()) { + for (unsigned int i = 0; i < config_.numCfeConfigQueue; i++) + prepareCfe(); + } /* Clear the debug dump file history. */ last_dump_file_.clear(); @@ -1892,8 +1898,12 @@ void PiSPCameraData::beInputDequeue(FrameBuffer *buffer) << ", buffer id " << cfe_[Cfe::Output0].getBufferId(buffer) << ", timestamp: " << buffer->metadata().timestamp; - /* The ISP input buffer gets re-queued into CFE. */ - handleStreamBuffer(buffer, &cfe_[Cfe::Output0]); + /* + * The ISP input buffer gets re-queued into CFE (but not for memory cameras + * that aren't using the CFE). + */ + if (cfe_[Cfe::Output0].dev()) + handleStreamBuffer(buffer, &cfe_[Cfe::Output0]); handleState(); } @@ -2363,7 +2373,23 @@ void PiSPCameraData::prepareCfe() void PiSPCameraData::prepareBe(uint32_t bufferId, bool stitchSwapBuffers) { - FrameBuffer *buffer = cfe_[Cfe::Output0].getBuffers().at(bufferId).buffer; + FrameBuffer *buffer; + if (!sensor_->isMemory()) + buffer = cfe_[Cfe::Output0].getBuffers().at(bufferId).buffer; + else { + /* Memory camera. Back end input buffer is in the request. */ + Request *request = requestQueue_.front(); + const libcamera::Stream *rawStream = &isp_[Isp::Input]; + buffer = request->findBuffer(rawStream); + /* If there's no buffer to process, there's no recovering from that. */ + if (!buffer) + LOG(RPI, Fatal) << "No input buffer supplied for re-processing"; + /* + * Can't see how an application would set bytesused if it were different + * from the allocated size (length), so assume that's the number we need. + */ + buffer->_d()->metadata().planes()[0].bytesused = buffer->planes()[0].length; + } LOG(RPI, Debug) << "Input re-queue to ISP, buffer id " << bufferId << ", timestamp: " << buffer->metadata().timestamp; @@ -2414,8 +2440,33 @@ void PiSPCameraData::prepareBe(uint32_t bufferId, bool stitchSwapBuffers) isp_[Isp::Config].queueBuffer(config.buffer); } +void PiSPCameraData::runMemoryCamera() +{ + Request *request = requestQueue_.front(); + + applyScalerCrop(request->controls()); + + request->metadata().clear(); + + state_ = State::Busy; + + ipa::RPi::PrepareParams params; + params.buffers.embedded = 0; + params.ipaContext = requestQueue_.front()->sequence(); + params.requestControls = request->controls(); + + LOG(RPI, Debug) << "runMemoryCamera: call prepareIsp"; + ipa_->prepareIsp(params); +} + void PiSPCameraData::tryRunPipeline() { + if (state_ == State::Idle && !requestQueue_.empty() && sensor_->isMemory()) { + runMemoryCamera(); + + return; + } + /* If any of our request or buffer queues are empty, we cannot proceed. */ if (state_ != State::Idle || requestQueue_.empty() || !cfeJobComplete()) return;