From patchwork Wed Nov 11 20:13:48 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Milan Zamazal X-Patchwork-Id: 28867 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id BE808C3304 for ; Wed, 11 Nov 2026 20:14:10 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 7A10A6917C; Wed, 11 Nov 2026 21:14:10 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.b="F/bX91nJ"; dkim-atps=neutral Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 6295368F9F for ; Wed, 11 Nov 2026 21:14:08 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1794428047; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=1n5T+TXRzWYNZC7txybOoecX5qXShCcjspxNvcFyJpk=; b=F/bX91nJe8aPDlZEWXmF1iqXC75RAcUa2x8WCfxL2cGpHkfOgnwF6S8dIKPTSAKpPYfdCQ D4VJG7gLwFAWJWkmEv6gqgLF1PpfCSsFhNN36xSf8FU3qW7QoaihKQV7jNUPyiQh8Jzr5o MfZrThNnNZo+84f86Wf7gbB+oG6tLDU= Received: from mx-prod-mc-06.mail-002.prod.us-west-2.aws.redhat.com (ec2-35-165-154-97.us-west-2.compute.amazonaws.com [35.165.154.97]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-610-htccj3GANficKLHA9FZOfg-1; Wed, 11 Nov 2026 15:14:02 -0500 X-MC-Unique: htccj3GANficKLHA9FZOfg-1 X-Mimecast-MFC-AGG-ID: htccj3GANficKLHA9FZOfg_1794428041 Received: from mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-06.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 1A01F1800547; Wed, 11 Nov 2026 20:14:01 +0000 (UTC) Received: from mzamazal-thinkpadp1gen7.redhat.corp (headnet04.pony-001.prod.iad2.dc.redhat.com [10.2.32.116]) by mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id 74C4530000E5; Wed, 11 Nov 2026 20:13:59 +0000 (UTC) From: Milan Zamazal To: libcamera-devel@lists.libcamera.org Cc: Milan Zamazal , Kieran Bingham , Jacopo Mondi Subject: [RFC PATCH v2 1/2] libcamera: software_isp: Use shared stats Date: Wed, 11 Nov 2026 21:13:48 +0100 Message-ID: <20261111201351.39373-2-mzamazal@redhat.com> In-Reply-To: <20261111201351.39373-1-mzamazal@redhat.com> References: <20261111201351.39373-1-mzamazal@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.4.1 on 10.30.177.4 X-Mimecast-Spam-Score: 0 X-Mimecast-MFC-PROC-ID: 9MgXREWtRZQemGkdEbk9PP3PmH9ggsOQu_mXbcgIlFU_1794428041 X-Mimecast-Originator: redhat.com content-type: text/plain; charset="US-ASCII"; x-default=true X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" The SwStatsCpu instance created in SoftwareIsp is currently moved to the corresponding debayering object. This binds the stats to debayering, but we'd like to be able to use stats with raw output as well. Let's keep the created stats in SoftwareIsp as a shared pointer that is still passed to the debayering objects but remains accessible outside them. This also allows us to get rid of Debayer::getStatsFD method. Signed-off-by: Milan Zamazal Reviewed-by: Kieran Bingham Reviewed-by: Jacopo Mondi --- .../libcamera/internal/software_isp/software_isp.h | 2 ++ src/libcamera/software_isp/debayer.cpp | 10 ---------- src/libcamera/software_isp/debayer.h | 2 -- src/libcamera/software_isp/debayer_cpu.cpp | 2 +- src/libcamera/software_isp/debayer_cpu.h | 5 ++--- src/libcamera/software_isp/debayer_egl.cpp | 2 +- src/libcamera/software_isp/debayer_egl.h | 6 ++---- src/libcamera/software_isp/software_isp.cpp | 12 ++++++------ 8 files changed, 14 insertions(+), 27 deletions(-) diff --git a/include/libcamera/internal/software_isp/software_isp.h b/include/libcamera/internal/software_isp/software_isp.h index b64910637..2f3cc5b6b 100644 --- a/include/libcamera/internal/software_isp/software_isp.h +++ b/include/libcamera/internal/software_isp/software_isp.h @@ -33,6 +33,7 @@ #include "libcamera/internal/pipeline_handler.h" #include "libcamera/internal/shared_mem_object.h" #include "libcamera/internal/software_isp/debayer_params.h" +#include "libcamera/internal/software_isp/swstats_cpu.h" namespace libcamera { @@ -102,6 +103,7 @@ private: DmaBufAllocator dmaHeap_; bool ccmEnabled_; bool lscEnabled_; + std::shared_ptr stats_; std::unique_ptr ipa_; std::deque queuedInputBuffers_; diff --git a/src/libcamera/software_isp/debayer.cpp b/src/libcamera/software_isp/debayer.cpp index 4f6659000..547533263 100644 --- a/src/libcamera/software_isp/debayer.cpp +++ b/src/libcamera/software_isp/debayer.cpp @@ -174,16 +174,6 @@ Debayer::~Debayer() * \return The valid size ranges or an empty range if there are none */ -/** - * \fn const SharedFD &Debayer::getStatsFD() - * \brief Get the file descriptor for the statistics - * - * This file descriptor provides access to the output statistics buffer - * associated with the current debayering process. - * - * \return The file descriptor pointing to the statistics data - */ - /** * \fn unsigned int Debayer::frameSize() * \brief Get the output frame size diff --git a/src/libcamera/software_isp/debayer.h b/src/libcamera/software_isp/debayer.h index e30a05279..98d27457b 100644 --- a/src/libcamera/software_isp/debayer.h +++ b/src/libcamera/software_isp/debayer.h @@ -55,8 +55,6 @@ public: virtual SizeRange sizes(PixelFormat inputFormat, const Size &inputSize) = 0; - virtual const SharedFD &getStatsFD() = 0; - unsigned int frameSize() { return outputConfig_.frameSize; } Signal inputBufferReady; diff --git a/src/libcamera/software_isp/debayer_cpu.cpp b/src/libcamera/software_isp/debayer_cpu.cpp index 91867baa6..b69d59f70 100644 --- a/src/libcamera/software_isp/debayer_cpu.cpp +++ b/src/libcamera/software_isp/debayer_cpu.cpp @@ -93,7 +93,7 @@ DebayerCpuThread::DebayerCpuThread(DebayerCpu *debayer, unsigned int threadIndex * \param[in] stats Pointer to the stats object to use * \param[in] cm The camera manager */ -DebayerCpu::DebayerCpu(std::unique_ptr stats, const CameraManager &cm) +DebayerCpu::DebayerCpu(std::shared_ptr stats, const CameraManager &cm) : Debayer(cm), stats_(std::move(stats)) { /* diff --git a/src/libcamera/software_isp/debayer_cpu.h b/src/libcamera/software_isp/debayer_cpu.h index ae570893b..a3addac64 100644 --- a/src/libcamera/software_isp/debayer_cpu.h +++ b/src/libcamera/software_isp/debayer_cpu.h @@ -32,7 +32,7 @@ class DebayerCpuThread; class DebayerCpu : public Debayer { public: - DebayerCpu(std::unique_ptr stats, const CameraManager &cm); + DebayerCpu(std::shared_ptr stats, const CameraManager &cm); ~DebayerCpu(); int configure(const StreamConfiguration &inputCfg, @@ -47,7 +47,6 @@ public: int start() override; void stop() override; SizeRange sizes(PixelFormat inputFormat, const Size &inputSize) override; - const SharedFD &getStatsFD() override { return stats_->getStatsFD(); } private: friend class DebayerCpuThread; @@ -154,7 +153,7 @@ private: debayerFn debayer2_; debayerFn debayer3_; Rectangle window_; - std::unique_ptr stats_; + std::shared_ptr stats_; unsigned int xShift_; /* Offset of 0/1 applied to window_.x */ static constexpr unsigned int kMinThreads = 1; diff --git a/src/libcamera/software_isp/debayer_egl.cpp b/src/libcamera/software_isp/debayer_egl.cpp index 33933b11d..5f1e5ea2d 100644 --- a/src/libcamera/software_isp/debayer_egl.cpp +++ b/src/libcamera/software_isp/debayer_egl.cpp @@ -42,7 +42,7 @@ namespace libcamera { * \param[in] cm The camera manager * \param[in] display The EGL display to use */ -DebayerEGL::DebayerEGL(std::unique_ptr stats, const CameraManager &cm, EGLDisplay display) +DebayerEGL::DebayerEGL(std::shared_ptr stats, const CameraManager &cm, EGLDisplay display) : Debayer(cm), stats_(std::move(stats)), egl_(display) { } diff --git a/src/libcamera/software_isp/debayer_egl.h b/src/libcamera/software_isp/debayer_egl.h index cbbc6a450..6957452d0 100644 --- a/src/libcamera/software_isp/debayer_egl.h +++ b/src/libcamera/software_isp/debayer_egl.h @@ -40,7 +40,7 @@ class CameraManager; class DebayerEGL : public Debayer { public: - DebayerEGL(std::unique_ptr stats, const CameraManager &cm, EGLDisplay display); + DebayerEGL(std::shared_ptr stats, const CameraManager &cm, EGLDisplay display); ~DebayerEGL(); int configure(const StreamConfiguration &inputCfg, @@ -58,8 +58,6 @@ public: int start() override; void stop() override; - const SharedFD &getStatsFD() override { return stats_->getStatsFD(); } - SizeRange sizes(PixelFormat inputFormat, const Size &inputSize) override; private: @@ -121,7 +119,7 @@ private: Size nativeOutputSize_; Rectangle window_; - std::unique_ptr stats_; + std::shared_ptr stats_; eGL egl_; uint32_t width_; uint32_t height_; diff --git a/src/libcamera/software_isp/software_isp.cpp b/src/libcamera/software_isp/software_isp.cpp index 627625b4f..3b9cf3710 100644 --- a/src/libcamera/software_isp/software_isp.cpp +++ b/src/libcamera/software_isp/software_isp.cpp @@ -98,12 +98,12 @@ SoftwareIsp::SoftwareIsp(PipelineHandler *pipe, const CameraSensor *sensor, const CameraManager &cm = *pipe->cameraManager(); - auto stats = std::make_unique(cm); - if (!stats->isValid()) { + stats_ = std::make_shared(cm); + if (!stats_->isValid()) { LOG(SoftwareIsp, Error) << "Failed to create SwStatsCpu object"; return; } - stats->statsReady.connect(this, &SoftwareIsp::statsReady); + stats_->statsReady.connect(this, &SoftwareIsp::statsReady); #if HAVE_DEBAYER_EGL const GlobalConfiguration &configuration = cm._d()->configuration(); @@ -121,7 +121,7 @@ SoftwareIsp::SoftwareIsp(PipelineHandler *pipe, const CameraSensor *sensor, if (!softISPMode || softISPMode == "gpu") { auto display = eGL::probeDisplay(); if (display != EGL_NO_DISPLAY) { - debayer_ = std::make_unique(std::move(stats), cm, display); + debayer_ = std::make_unique(stats_, cm, display); } else { LOG(SoftwareIsp, Info) << "EGL not available, falling back to CPU debayer"; @@ -130,7 +130,7 @@ SoftwareIsp::SoftwareIsp(PipelineHandler *pipe, const CameraSensor *sensor, #endif if (!debayer_) - debayer_ = std::make_unique(std::move(stats), cm); + debayer_ = std::make_unique(stats_, cm); debayer_->inputBufferReady.connect(this, &SoftwareIsp::inputReady); debayer_->outputBufferReady.connect(this, &SoftwareIsp::outputReady); @@ -158,7 +158,7 @@ SoftwareIsp::SoftwareIsp(PipelineHandler *pipe, const CameraSensor *sensor, } ret = ipa_->init(IPASettings{ ipaTuningFile, sensor->model() }, - debayer_->getStatsFD(), + stats_->getStatsFD(), sharedParams_.fd(), sensorInfo, sensor->controls(), From patchwork Wed Nov 11 20:13:49 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Milan Zamazal X-Patchwork-Id: 28868 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 3DC14C3304 for ; Wed, 11 Nov 2026 20:14:12 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 3CB4869146; Wed, 11 Nov 2026 21:14:11 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.b="KcmMlOWC"; dkim-atps=neutral Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id DAE34690F6 for ; Wed, 11 Nov 2026 21:14:08 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1794428047; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=I1VyrRwyy7R4o0O4lzx8pbp1+wglak2xvvJ8GCdeuns=; b=KcmMlOWCZY8P1mu4ejMdgOwzzvCRtONXi5MXHL40chR0f3jyM+7CYsfit9biA56FCRMCPa Z9PYoU23UaX8NmkO2oHnaG06Gqy1o9aItEVonKw55Z64eS2GhwGq1fk3/7yYtfxMlXLWkT HsNjtQM7Rf94ifFaC+QymU4Mbly2QDE= Received: from mx-prod-mc-05.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-684-StK2YXoRNuSpn7AH4eawng-1; Wed, 11 Nov 2026 15:14:04 -0500 X-MC-Unique: StK2YXoRNuSpn7AH4eawng-1 X-Mimecast-MFC-AGG-ID: StK2YXoRNuSpn7AH4eawng_1794428043 Received: from mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-05.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id DDFF419560AF; Wed, 11 Nov 2026 20:14:02 +0000 (UTC) Received: from mzamazal-thinkpadp1gen7.redhat.corp (headnet04.pony-001.prod.iad2.dc.redhat.com [10.2.32.116]) by mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id 76D7430001A2; Wed, 11 Nov 2026 20:14:01 +0000 (UTC) From: Milan Zamazal To: libcamera-devel@lists.libcamera.org Cc: Milan Zamazal , Kieran Bingham , Jacopo Mondi Subject: [RFC PATCH v2 2/2] libcamera: software_isp: Allow running IPA and stats on raw Date: Wed, 11 Nov 2026 21:13:49 +0100 Message-ID: <20261111201351.39373-3-mzamazal@redhat.com> In-Reply-To: <20261111201351.39373-1-mzamazal@redhat.com> References: <20261111201351.39373-1-mzamazal@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.4.1 on 10.30.177.4 X-Mimecast-Spam-Score: 0 X-Mimecast-MFC-PROC-ID: Ag9rjqgUiOwB9Ywv7ripqFbIVVS5topkYOxrDUurASg_1794428043 X-Mimecast-Originator: redhat.com content-type: text/plain; charset="US-ASCII"; x-default=true X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" `simple' pipeline can basically pass raw buffers from V4L2 to the output. While the raw data is untouched, we may still want to: - Gather statistics from it. - Run IPA on the images (without modifying the images). - Provide metadata from the IPA. - Adjust exposure/gain. This patch implements all of that. Which introduces sort of processing to raw streams, making them similar to debayered streams. In order to not complicate `simple' pipeline more than necessary, SoftwareIsp class is split into a base class that serves just the raw streams, and a subclass with software ISP processing including debayering. Signed-off-by: Milan Zamazal --- .../internal/software_isp/software_isp.h | 102 ++-- src/libcamera/pipeline/simple/simple.cpp | 63 +- src/libcamera/software_isp/software_isp.cpp | 540 ++++++++++++------ 3 files changed, 485 insertions(+), 220 deletions(-) diff --git a/include/libcamera/internal/software_isp/software_isp.h b/include/libcamera/internal/software_isp/software_isp.h index 2f3cc5b6b..21f8c9b5e 100644 --- a/include/libcamera/internal/software_isp/software_isp.h +++ b/include/libcamera/internal/software_isp/software_isp.h @@ -12,7 +12,6 @@ #include #include #include -#include #include #include @@ -45,44 +44,40 @@ struct StreamConfiguration; LOG_DECLARE_CATEGORY(SoftwareIsp) -class SoftwareIsp : public Object +class RawIsp : public Object { public: - SoftwareIsp(PipelineHandler *pipe, const CameraSensor *sensor, + RawIsp(PipelineHandler *pipe, const CameraSensor *sensor, ControlInfoMap *ipaControls); - ~SoftwareIsp(); + virtual ~RawIsp(); - int loadConfiguration([[maybe_unused]] const std::string &filename) { return 0; } + virtual bool isValid() const; - bool isValid() const; - - std::vector formats(PixelFormat input); - - SizeRange sizes(PixelFormat inputFormat, const Size &inputSize); - - std::tuple + virtual std::vector formats(PixelFormat input); + virtual SizeRange sizes(PixelFormat inputFormat, const Size &inputSize); + virtual std::tuple strideAndFrameSize(const PixelFormat &outputFormat, const Size &size); - uint32_t preferredInputStride(const PixelFormat &inputFormat, const Size &size); + virtual uint32_t preferredInputStride(const PixelFormat &inputFormat, const Size &size); - int configure(const StreamConfiguration &inputCfg, - const std::vector> &outputCfgs, - const ipa::softisp::IPAConfigInfo &configInfo, - ControlInfoMap *ipaControls); + virtual int configure(const StreamConfiguration &inputCfg, + const std::vector> &outputCfgs, + const ipa::softisp::IPAConfigInfo &configInfo, + ControlInfoMap *ipaControls); - int exportBuffers(const Stream *stream, unsigned int count, - std::vector> *buffers); + virtual int exportBuffers(const Stream *stream, unsigned int count, + std::vector> *buffers); - void processStats(const uint32_t frame, const uint32_t bufferId, - const ControlList &sensorControls); + virtual void processStats(const uint32_t frame, const uint32_t bufferId, + const ControlList &sensorControls); - int start(); - void stop(); + virtual int start(); + virtual void stop(); - void queueRequest(const uint32_t frame, const ControlList &controls); - int queueBuffers(uint32_t frame, FrameBuffer *input, - const std::map &outputs); + virtual void queueRequest(const uint32_t frame, const ControlList &controls); + virtual int queueBuffers(uint32_t frame, FrameBuffer *input, + const std::map &outputs); - void process(uint32_t frame, FrameBuffer *input, FrameBuffer *output); + virtual void process(uint32_t frame, FrameBuffer *input, FrameBuffer *output); Signal inputBufferReady; Signal outputBufferReady; @@ -90,14 +85,16 @@ public: Signal metadataReady; Signal setSensorControls; -private: - void saveIspParams(uint32_t frame); - void setSensorCtrls(const ControlList &sensorControls); - void statsReady(uint32_t frame, uint32_t bufferId); +protected: + virtual void saveIspParams(uint32_t frame); + virtual void setSensorCtrls(const ControlList &sensorControls); + virtual void statsReady(uint32_t frame, uint32_t bufferId); void inputReady(FrameBuffer *input); void outputReady(FrameBuffer *output); - std::unique_ptr debayer_; + Thread ispWorkerThread_; + bool initialized_ = false; + unsigned int inputStride_ = 0; SharedMemObject sharedParams_; DebayerParams debayerParams_; DmaBufAllocator dmaHeap_; @@ -108,6 +105,47 @@ private: std::unique_ptr ipa_; std::deque queuedInputBuffers_; std::deque queuedOutputBuffers_; + +private: + int init(PipelineHandler *pipe, const CameraSensor *sensor, + ControlInfoMap *ipaControls); +}; + +class DebayerIsp : public RawIsp +{ +public: + DebayerIsp(PipelineHandler *pipe, const CameraSensor *sensor, + ControlInfoMap *ipaControls); + ~DebayerIsp(); + + bool isValid() const override; + + std::vector formats(PixelFormat input) override; + SizeRange sizes(PixelFormat inputFormat, const Size &inputSize) override; + std::tuple + strideAndFrameSize(const PixelFormat &outputFormat, const Size &size) override; + uint32_t preferredInputStride(const PixelFormat &inputFormat, const Size &size) override; + + int configure(const StreamConfiguration &inputCfg, + const std::vector> &outputCfgs, + const ipa::softisp::IPAConfigInfo &configInfo, + ControlInfoMap *ipaControls) override; + + int exportBuffers(const Stream *stream, unsigned int count, + std::vector> *buffers) override; + + virtual int start() override; + virtual void stop() override; + + int queueBuffers(uint32_t frame, FrameBuffer *input, + const std::map &outputs) override; + + void process(uint32_t frame, FrameBuffer *input, FrameBuffer *output) override; + +private: + int init(PipelineHandler *pipe); + + std::unique_ptr debayer_; }; } /* namespace libcamera */ diff --git a/src/libcamera/pipeline/simple/simple.cpp b/src/libcamera/pipeline/simple/simple.cpp index 35e066499..d2da63db8 100644 --- a/src/libcamera/pipeline/simple/simple.cpp +++ b/src/libcamera/pipeline/simple/simple.cpp @@ -359,7 +359,9 @@ public: bool useConversion_; std::unique_ptr converter_; - std::unique_ptr swIsp_; + std::shared_ptr rawIsp_; + std::shared_ptr debayerIsp_; + std::shared_ptr swIsp_; SimpleFrames frameInfo_; Rectangle scalerCrop_; @@ -618,17 +620,30 @@ int SimpleCameraData::init() * Instantiate Soft ISP if this is enabled for the given driver and no converter is used. */ if (!converter_ && pipe->swIspEnabled()) { - swIsp_ = std::make_unique(pipe, sensor_.get(), &controlInfo_); - if (!swIsp_->isValid()) { + rawIsp_ = std::make_unique(pipe, sensor_.get(), &controlInfo_); + if (!rawIsp_->isValid()) { LOG(SimplePipeline, Warning) << "Failed to create software ISP, disabling software debayering"; - swIsp_.reset(); + rawIsp_.reset(); } else { - swIsp_->inputBufferReady.connect(this, &SimpleCameraData::conversionInputDone); - swIsp_->outputBufferReady.connect(this, &SimpleCameraData::conversionOutputDone); - swIsp_->ispStatsReady.connect(this, &SimpleCameraData::ispStatsReady); - swIsp_->metadataReady.connect(this, &SimpleCameraData::metadataReady); - swIsp_->setSensorControls.connect(this, &SimpleCameraData::setSensorControls); + rawIsp_->inputBufferReady.connect(this, &SimpleCameraData::conversionInputDone); + rawIsp_->outputBufferReady.connect(this, &SimpleCameraData::conversionOutputDone); + rawIsp_->ispStatsReady.connect(this, &SimpleCameraData::ispStatsReady); + rawIsp_->metadataReady.connect(this, &SimpleCameraData::metadataReady); + rawIsp_->setSensorControls.connect(this, &SimpleCameraData::setSensorControls); + } + + debayerIsp_ = std::make_unique(pipe, sensor_.get(), &controlInfo_); + if (!debayerIsp_->isValid()) { + LOG(SimplePipeline, Warning) + << "Failed to create software ISP, disabling software debayering"; + debayerIsp_.reset(); + } else { + debayerIsp_->inputBufferReady.connect(this, &SimpleCameraData::conversionInputDone); + debayerIsp_->outputBufferReady.connect(this, &SimpleCameraData::conversionOutputDone); + debayerIsp_->ispStatsReady.connect(this, &SimpleCameraData::ispStatsReady); + debayerIsp_->metadataReady.connect(this, &SimpleCameraData::metadataReady); + debayerIsp_->setSensorControls.connect(this, &SimpleCameraData::setSensorControls); } } @@ -744,9 +759,9 @@ void SimpleCameraData::tryPipeline(unsigned int code, const Size &size) if (converter_) { config.outputFormats = converter_->formats(pixelFormat); config.outputSizes = converter_->sizes(format.size); - } else if (swIsp_) { - config.outputFormats = swIsp_->formats(pixelFormat); - config.outputSizes = swIsp_->sizes(pixelFormat, format.size); + } else if (debayerIsp_) { + config.outputFormats = debayerIsp_->formats(pixelFormat); + config.outputSizes = debayerIsp_->sizes(pixelFormat, format.size); if (config.outputFormats.empty()) { /* Do not use swIsp for unsupported pixelFormat's. */ config.outputFormats = { pixelFormat }; @@ -888,7 +903,7 @@ void SimpleCameraData::imageBufferReady(FrameBuffer *buffer) * point converting an erroneous buffer. */ if (buffer->metadata().status != FrameMetadata::FrameSuccess) { - if (!useConversion_ || rawStream_) { + if (!useConversion_) { /* No conversion, just complete the request. */ Request *request = buffer->request(); pipe->completeBuffer(request, buffer); @@ -904,13 +919,16 @@ void SimpleCameraData::imageBufferReady(FrameBuffer *buffer) * buffer for capture (unless the stream is being stopped), and * complete the request with all the user-facing buffers. */ - if (buffer->metadata().status != FrameMetadata::FrameCancelled) + if (!rawStream_ && + buffer->metadata().status != FrameMetadata::FrameCancelled) video_->queueBuffer(buffer); if (conversionQueue_.empty()) return; const RequestOutputs &outputs = conversionQueue_.front(); + if (rawStream_) + pipe->completeBuffer(outputs.request, buffer); for (auto &[stream, buf] : outputs.outputs) pipe->completeBuffer(outputs.request, buf); SimpleFrameInfo *info = frameInfo_.find(outputs.request->sequence()); @@ -1295,7 +1313,14 @@ CameraConfiguration::Status SimpleCameraConfiguration::validate() * require any conversion, similar to raw capture use cases). This is * left as a future improvement. */ - needConversion_ = config_.size() > 1 + rawCount; + needConversion_ = (config_.size() > 1 + rawCount) || data_->rawIsp_; + + data_->swIsp_ = data_->rawIsp_; + for (unsigned int i = 0; i < config_.size(); ++i) + if (!isRaw(config_[i])) { + data_->swIsp_ = data_->debayerIsp_; + break; + } for (unsigned int i = 0; i < config_.size(); ++i) { StreamConfiguration &cfg = config_[i]; @@ -1637,8 +1662,8 @@ int SimplePipelineHandler::configure(Camera *camera, CameraConfiguration *c) if (rawStream) data->rawStream_ = &data->streams_[i]; } - - if (outputCfgs.empty()) + if (outputCfgs.empty() && + (data->converter_ || !data->rawStream_)) return 0; StreamConfiguration inputCfg; @@ -1787,7 +1812,7 @@ int SimplePipelineHandler::queueRequestDevice(Camera *camera, Request *request) int ret; std::map buffers; - bool metadataRequired = false; + bool metadataRequired = data->useConversion_ && !!data->swIsp_; for (auto &[stream, buffer] : request->buffers()) { /* @@ -1797,7 +1822,6 @@ int SimplePipelineHandler::queueRequestDevice(Camera *camera, Request *request) */ if (data->useConversion_ && stream != data->rawStream_) { buffers.emplace(stream, buffer); - metadataRequired = !!data->swIsp_; } else { ret = data->video_->queueBuffer(buffer); if (ret < 0) @@ -1811,7 +1835,6 @@ int SimplePipelineHandler::queueRequestDevice(Camera *camera, Request *request) if (data->swIsp_) data->swIsp_->queueRequest(request->sequence(), request->controls()); } - return 0; } diff --git a/src/libcamera/software_isp/software_isp.cpp b/src/libcamera/software_isp/software_isp.cpp index 3b9cf3710..a64ab32ed 100644 --- a/src/libcamera/software_isp/software_isp.cpp +++ b/src/libcamera/software_isp/software_isp.cpp @@ -25,6 +25,7 @@ #include "libcamera/internal/bayer_format.h" #include "libcamera/internal/framebuffer.h" +#include "libcamera/internal/mapped_framebuffer.h" #include "libcamera/internal/software_isp/debayer_params.h" #include "debayer_cpu.h" @@ -42,58 +43,42 @@ namespace libcamera { LOG_DEFINE_CATEGORY(SoftwareIsp) /** - * \class SoftwareIsp - * \brief Class for the Software ISP + * \class RawIsp + * \brief Base class for software image processing */ /** - * \var SoftwareIsp::inputBufferReady - * \brief A signal emitted when the input frame buffer completes - */ - -/** - * \var SoftwareIsp::outputBufferReady - * \brief A signal emitted when the output frame buffer completes - */ - -/** - * \var SoftwareIsp::ispStatsReady - * \brief A signal emitted when the statistics for IPA are ready - */ - -/** - * \var SoftwareIsp::metadataReady - * \brief A signal emitted when the metadata for IPA is ready - */ - -/** - * \var SoftwareIsp::setSensorControls - * \brief A signal emitted when the values to write to the sensor controls are - * ready - */ - -/** - * \brief Constructs SoftwareIsp object + * \brief Constructs RawIsp object * \param[in] pipe The pipeline handler in use * \param[in] sensor Pointer to the CameraSensor instance owned by the pipeline handler * \param[out] ipaControls The IPA controls to update */ -SoftwareIsp::SoftwareIsp(PipelineHandler *pipe, const CameraSensor *sensor, +RawIsp::RawIsp(PipelineHandler *pipe, const CameraSensor *sensor, ControlInfoMap *ipaControls) : ispWorkerThread_("SWIspWorker"), dmaHeap_(DmaBufAllocator::DmaBufAllocatorFlag::CmaHeap | DmaBufAllocator::DmaBufAllocatorFlag::SystemHeap | DmaBufAllocator::DmaBufAllocatorFlag::UDmaBuf) +{ + initialized_ = init(pipe, sensor, ipaControls) == 0; +} + +RawIsp::~RawIsp() +{ +} + +int RawIsp::init(PipelineHandler *pipe, const CameraSensor *sensor, + ControlInfoMap *ipaControls) { if (!dmaHeap_.isValid()) { LOG(SoftwareIsp, Error) << "Failed to create DmaBufAllocator object"; - return; + return -1; } sharedParams_ = SharedMemObject("softIsp_params"); if (!sharedParams_) { LOG(SoftwareIsp, Error) << "Failed to create shared memory for parameters"; - return; + return -1; } const CameraManager &cm = *pipe->cameraManager(); @@ -101,46 +86,15 @@ SoftwareIsp::SoftwareIsp(PipelineHandler *pipe, const CameraSensor *sensor, stats_ = std::make_shared(cm); if (!stats_->isValid()) { LOG(SoftwareIsp, Error) << "Failed to create SwStatsCpu object"; - return; - } - stats_->statsReady.connect(this, &SoftwareIsp::statsReady); - -#if HAVE_DEBAYER_EGL - const GlobalConfiguration &configuration = cm._d()->configuration(); - std::optional softISPMode = configuration.option({ "software_isp", "mode" }); - if (softISPMode) { - if (softISPMode != "gpu" && softISPMode != "cpu") { - LOG(SoftwareIsp, Error) - << "Invalid software ISP mode \"" - << softISPMode.value() - << "\", must be \"cpu\" or \"gpu\""; - return; - } - } - - if (!softISPMode || softISPMode == "gpu") { - auto display = eGL::probeDisplay(); - if (display != EGL_NO_DISPLAY) { - debayer_ = std::make_unique(stats_, cm, display); - } else { - LOG(SoftwareIsp, Info) - << "EGL not available, falling back to CPU debayer"; - } + return -1; } - -#endif - if (!debayer_) - debayer_ = std::make_unique(stats_, cm); - - debayer_->inputBufferReady.connect(this, &SoftwareIsp::inputReady); - debayer_->outputBufferReady.connect(this, &SoftwareIsp::outputReady); + stats_->statsReady.connect(this, &RawIsp::statsReady); ipa_ = pipe->createIPA(0, 0); if (!ipa_) { LOG(SoftwareIsp, Error) << "Creating IPA for software ISP failed"; - debayer_.reset(); - return; + return -1; } /* @@ -154,7 +108,7 @@ SoftwareIsp::SoftwareIsp(PipelineHandler *pipe, const CameraSensor *sensor, int ret = sensor->sensorInfo(&sensorInfo); if (ret) { LOG(SoftwareIsp, Error) << "Camera sensor information not available"; - return; + return -1; } ret = ipa_->init(IPASettings{ ipaTuningFile, sensor->model() }, @@ -167,46 +121,29 @@ SoftwareIsp::SoftwareIsp(PipelineHandler *pipe, const CameraSensor *sensor, &lscEnabled_); if (ret) { LOG(SoftwareIsp, Error) << "IPA init failed"; - debayer_.reset(); - return; + return -1; } - ipa_->paramsComputed.connect(this, &SoftwareIsp::saveIspParams); + ipa_->paramsComputed.connect(this, &RawIsp::saveIspParams); ipa_->metadataReady.connect(this, [this](uint32_t frame, const ControlList &metadata) { metadataReady.emit(frame, metadata); }); - ipa_->setSensorControls.connect(this, &SoftwareIsp::setSensorCtrls); + ipa_->setSensorControls.connect(this, &RawIsp::setSensorCtrls); - debayer_->moveToThread(&ispWorkerThread_); -} - -SoftwareIsp::~SoftwareIsp() -{ - /* make sure to destroy the DebayerCpu before the ispWorkerThread_ is gone */ - debayer_.reset(); + return 0; } -/** - * \fn int SoftwareIsp::loadConfiguration([[maybe_unused]] const std::string &filename) - * \brief Load a configuration from a file - * \param[in] filename The file to load the configuration data from - * - * Currently is a stub doing nothing and always returning "success". - * - * \return 0 on success - */ - /** * \brief Process the statistics gathered * \param[in] frame The frame number * \param[in] bufferId ID of the statistics buffer * \param[in] sensorControls The sensor controls * - * Requests the IPA to calculate new parameters for ISP and new control + * Requests the IPA to calculate new parameters for software ISP and new control * values for the sensor. */ -void SoftwareIsp::processStats(const uint32_t frame, const uint32_t bufferId, +void RawIsp::processStats(const uint32_t frame, const uint32_t bufferId, const ControlList &sensorControls) { ASSERT(ipa_); @@ -214,12 +151,12 @@ void SoftwareIsp::processStats(const uint32_t frame, const uint32_t bufferId, } /** - * \brief Check the validity of Software Isp object - * \return True if Software Isp is valid, false otherwise + * \brief Check the validity of the Isp object + * \return True if the Isp is valid, false otherwise */ -bool SoftwareIsp::isValid() const +bool RawIsp::isValid() const { - return !!debayer_; + return initialized_; } /** @@ -227,11 +164,9 @@ bool SoftwareIsp::isValid() const * \param[in] inputFormat The input format * \return All the supported output formats or an empty vector if there are none */ -std::vector SoftwareIsp::formats(PixelFormat inputFormat) +std::vector RawIsp::formats(PixelFormat inputFormat) { - ASSERT(debayer_); - - return debayer_->formats(inputFormat); + return std::vector({ inputFormat }); } /** @@ -240,11 +175,9 @@ std::vector SoftwareIsp::formats(PixelFormat inputFormat) * \param[in] inputSize The input frame size * \return The valid size range or an empty range if there are none */ -SizeRange SoftwareIsp::sizes(PixelFormat inputFormat, const Size &inputSize) +SizeRange RawIsp::sizes([[maybe_unused]] PixelFormat inputFormat, const Size &inputSize) { - ASSERT(debayer_); - - return debayer_->sizes(inputFormat, inputSize); + return SizeRange(inputSize); } /** @@ -255,11 +188,9 @@ SizeRange SoftwareIsp::sizes(PixelFormat inputFormat, const Size &inputSize) * if there is no valid output config */ std::tuple -SoftwareIsp::strideAndFrameSize(const PixelFormat &outputFormat, const Size &size) +RawIsp::strideAndFrameSize([[maybe_unused]] const PixelFormat &outputFormat, const Size &size) { - ASSERT(debayer_); - - return debayer_->strideAndFrameSize(outputFormat, size); + return std::make_tuple(inputStride_, inputStride_ * size.height); } /** @@ -268,33 +199,34 @@ SoftwareIsp::strideAndFrameSize(const PixelFormat &outputFormat, const Size &siz * \param[in] size The input size (width and height in pixels) * \return The preferred input stride in bytes or 0 if there is no preference */ -uint32_t SoftwareIsp::preferredInputStride(const PixelFormat &inputFormat, const Size &size) +uint32_t RawIsp::preferredInputStride([[maybe_unused]] const PixelFormat &inputFormat, [[maybe_unused]] const Size &size) { - ASSERT(debayer_); - - return debayer_->preferredInputStride(inputFormat, size); + return 0; } /** - * \brief Configure the SoftwareIsp object according to the passed in parameters + * \brief Configure the RawIsp object according to the passed in parameters * \param[in] inputCfg The input configuration * \param[in] outputCfgs The output configurations * \param[in] configInfo The IPA configuration data, received from the pipeline handler * \param[out] ipaControls The IPA controls to update * \return 0 on success, a negative errno on failure */ -int SoftwareIsp::configure(const StreamConfiguration &inputCfg, - const std::vector> &outputCfgs, - const ipa::softisp::IPAConfigInfo &configInfo, - ControlInfoMap *ipaControls) +int RawIsp::configure(const StreamConfiguration &inputCfg, + [[maybe_unused]] const std::vector> &outputCfgs, + const ipa::softisp::IPAConfigInfo &configInfo, + ControlInfoMap *ipaControls) { - ASSERT(ipa_ && debayer_); + ASSERT(ipa_); - int ret = ipa_->configure(configInfo, ipaControls); + inputStride_ = inputCfg.stride; + + int ret = stats_->configure(inputCfg); if (ret < 0) return ret; + stats_->setWindow(Rectangle(inputCfg.size)); - ret = debayer_->configure(inputCfg, outputCfgs, ccmEnabled_, lscEnabled_); + ret = ipa_->configure(configInfo, ipaControls); if (ret < 0) return ret; @@ -307,23 +239,18 @@ int SoftwareIsp::configure(const StreamConfiguration &inputCfg, } /** - * \brief Export the buffers from the Software ISP + * \brief Export the buffers from the software ISP * \param[in] stream Output stream exporting the buffers * \param[in] count Number of buffers to allocate * \param[out] buffers Vector to store the allocated buffers * \return The number of allocated buffers on success or a negative error code * otherwise */ -int SoftwareIsp::exportBuffers(const Stream *stream, unsigned int count, - std::vector> *buffers) +int RawIsp::exportBuffers([[maybe_unused]] const Stream *stream, [[maybe_unused]] unsigned int count, + [[maybe_unused]] std::vector> *buffers) { - ASSERT(debayer_ != nullptr); - - /* single output for now */ - if (stream == nullptr) - return -EINVAL; - - return dmaHeap_.exportBuffers(count, { debayer_->frameSize() }, buffers); + /* There are no separate output buffers for raw streams. */ + return -1; } /** @@ -331,53 +258,37 @@ int SoftwareIsp::exportBuffers(const Stream *stream, unsigned int count, * \param[in] frame The number of the frame which will be processed next * \param[in] controls The controls for the \a frame */ -void SoftwareIsp::queueRequest(const uint32_t frame, const ControlList &controls) +void RawIsp::queueRequest(const uint32_t frame, const ControlList &controls) { ipa_->queueRequest(frame, controls); } /** - * \brief Queue buffers to Software ISP + * \brief Queue buffers to the software ISP * \param[in] frame The frame number * \param[in] input The input framebuffer * \param[in] outputs The container holding the output stream pointers and * their respective frame buffer outputs * \return 0 on success, a negative errno on failure */ -int SoftwareIsp::queueBuffers(uint32_t frame, FrameBuffer *input, - const std::map &outputs) +int RawIsp::queueBuffers(uint32_t frame, FrameBuffer *input, + const std::map &outputs) { - /* - * Validate the outputs as a sanity check: at least one output is - * required, all outputs must reference a valid stream. - */ - if (outputs.empty()) - return -EINVAL; - - /* We only support a single stream for now. */ - if (outputs.size() != 1) + if (outputs.size() > 0) return -EINVAL; - for (auto [stream, buffer] : outputs) { - if (!buffer) - return -EINVAL; - } - queuedInputBuffers_.push_back(input); - for (const auto &[stream, buffer] : outputs) { - queuedOutputBuffers_.push_back(buffer); - process(frame, input, buffer); - } + process(frame, input, nullptr); return 0; } /** - * \brief Starts the Software ISP streaming operation + * \brief Starts the software ISP streaming operation * \return 0 on success, any other value indicates an error */ -int SoftwareIsp::start() +int RawIsp::start() { int ret = ipa_->start(); if (ret) @@ -385,8 +296,7 @@ int SoftwareIsp::start() ispWorkerThread_.start(); - return debayer_->invokeMethod(&Debayer::start, - ConnectionTypeBlocking); + return 0; } /** @@ -397,14 +307,11 @@ int SoftwareIsp::start() * * This function will not take any action when the ISP is not already running. */ -void SoftwareIsp::stop() +void RawIsp::stop() { if (!ispWorkerThread_.isRunning()) return; - debayer_->invokeMethod(&Debayer::stop, - ConnectionTypeBlocking); - ispWorkerThread_.exit(); ispWorkerThread_.wait(); @@ -426,45 +333,342 @@ void SoftwareIsp::stop() } /** - * \brief Passes the input framebuffer to the ISP worker to process + * \brief Passes the input framebuffer to the software ISP worker to process * \param[in] frame The frame number * \param[in] input The input framebuffer - * \param[out] output The framebuffer to write the processed frame to + * \param[out] output The framebuffer to write the processed frame to; if + * nullptr then debayering is skipped and stats are processed synchronously */ -void SoftwareIsp::process(uint32_t frame, FrameBuffer *input, FrameBuffer *output) +void RawIsp::process(uint32_t frame, FrameBuffer *input, [[maybe_unused]] FrameBuffer *output) { ipa_->computeParams(frame); - debayer_->invokeMethod(&Debayer::process, - ConnectionTypeQueued, frame, input, output, debayerParams_); + + /* Compute stats, which are otherwise computed in debayering */ + if (frame % SwStatsCpu::kStatPerNumFrames) { + stats_->finishFrame(frame, 0); + } else { + MappedFrameBuffer inputMapped(input, MappedFrameBuffer::MapFlag::Read); + stats_->processFrame(frame, 0, inputMapped); + } + + outputReady(output); + inputReady(input); } -void SoftwareIsp::saveIspParams([[maybe_unused]] uint32_t frame) +/** + * \brief TODO + */ +void RawIsp::saveIspParams([[maybe_unused]] uint32_t frame) { debayerParams_ = *sharedParams_; } -void SoftwareIsp::setSensorCtrls(const ControlList &sensorControls) +/** + * \brief TODO + */ +void RawIsp::setSensorCtrls(const ControlList &sensorControls) { setSensorControls.emit(sensorControls); } -void SoftwareIsp::statsReady(uint32_t frame, uint32_t bufferId) +/** + * \brief TODO + */ +void RawIsp::statsReady(uint32_t frame, uint32_t bufferId) { ispStatsReady.emit(frame, bufferId); } -void SoftwareIsp::inputReady(FrameBuffer *input) +/** + * \brief TODO + */ +void RawIsp::inputReady(FrameBuffer *input) { ASSERT(queuedInputBuffers_.front() == input); queuedInputBuffers_.pop_front(); inputBufferReady.emit(input); } -void SoftwareIsp::outputReady(FrameBuffer *output) +/** + * \brief TODO + */ +void RawIsp::outputReady(FrameBuffer *output) { + if (output == nullptr) + return; + ASSERT(queuedOutputBuffers_.front() == output); queuedOutputBuffers_.pop_front(); outputBufferReady.emit(output); } +/** + * \var RawIsp::inputBufferReady + * \brief A signal emitted when the input frame buffer completes + */ + +/** + * \var RawIsp::outputBufferReady + * \brief A signal emitted when the output frame buffer completes + */ + +/** + * \var RawIsp::ispStatsReady + * \brief A signal emitted when the statistics for IPA are ready + */ + +/** + * \var RawIsp::metadataReady + * \brief A signal emitted when the metadata for IPA is ready + */ + +/** + * \var RawIsp::setSensorControls + * \brief A signal emitted when the values to write to the sensor controls are + * ready + */ + +/** + * \class DebayerIsp + * \brief Debayering software ISP class + */ + +/** + * \var RawIsp::ispWorkerThread_ + * \brief TODO + */ + +/** + * \var RawIsp::initialized_ + * \brief TODO + */ + +/** + * \var RawIsp::inputStride_ + * \brief TODO + */ + +/** + * \var RawIsp::sharedParams_ + * \brief TODO + */ + +/** + * \var RawIsp::debayerParams_ + * \brief TODO + */ + +/** + * \var RawIsp::dmaHeap_ + * \brief TODO + */ + +/** + * \var RawIsp::ccmEnabled_ + * \brief TODO + */ + +/** + * \var RawIsp::lscEnabled_ + * \brief TODO + */ + +/** + * \var RawIsp::stats_ + * \brief TODO + */ + +/** + * \var RawIsp::ipa_ + * \brief TODO + */ + +/** + * \var RawIsp::queuedInputBuffers_ + * \brief TODO + */ + +/** + * \var RawIsp::queuedOutputBuffers_ + * \brief TODO + */ + +/** + * \brief Constructs DebayerIsp object + * \param[in] pipe The pipeline handler in use + * \param[in] sensor Pointer to the CameraSensor instance owned by the pipeline handler + * \param[out] ipaControls The IPA controls to update + */ +DebayerIsp::DebayerIsp(PipelineHandler *pipe, const CameraSensor *sensor, + ControlInfoMap *ipaControls) + : RawIsp(pipe, sensor, ipaControls) +{ + if (RawIsp::isValid()) + init(pipe); +} + +DebayerIsp::~DebayerIsp() +{ + /* make sure to destroy the DebayerCpu before the ispWorkerThread_ is gone */ + debayer_.reset(); +} + +int DebayerIsp::init(PipelineHandler *pipe) +{ + const CameraManager &cm = *pipe->cameraManager(); + +#if HAVE_DEBAYER_EGL + const GlobalConfiguration &configuration = cm._d()->configuration(); + std::optional softISPMode = configuration.option({ "software_isp", "mode" }); + if (softISPMode) { + if (softISPMode != "gpu" && softISPMode != "cpu") { + LOG(SoftwareIsp, Error) + << "Invalid software ISP mode \"" + << softISPMode.value() + << "\", must be \"cpu\" or \"gpu\""; + return -1; + } + } + + if (!softISPMode || softISPMode == "gpu") { + auto display = eGL::probeDisplay(); + if (display != EGL_NO_DISPLAY) { + debayer_ = std::make_unique(stats_, cm, display); + } else { + LOG(SoftwareIsp, Info) + << "EGL not available, falling back to CPU debayer"; + } + } + +#endif + if (!debayer_) + debayer_ = std::make_unique(stats_, cm); + + debayer_->inputBufferReady.connect(this, &DebayerIsp::inputReady); + debayer_->outputBufferReady.connect(this, &DebayerIsp::outputReady); + debayer_->moveToThread(&ispWorkerThread_); + + return 0; +} + +bool DebayerIsp::isValid() const +{ + return RawIsp::isValid() && !!debayer_; +} + +std::vector DebayerIsp::formats(PixelFormat inputFormat) +{ + ASSERT(debayer_); + + return debayer_->formats(inputFormat); +} + +SizeRange DebayerIsp::sizes(PixelFormat inputFormat, const Size &inputSize) +{ + ASSERT(debayer_); + + return debayer_->sizes(inputFormat, inputSize); +} + +std::tuple +DebayerIsp::strideAndFrameSize(const PixelFormat &outputFormat, const Size &size) +{ + ASSERT(debayer_); + + return debayer_->strideAndFrameSize(outputFormat, size); +} + +uint32_t DebayerIsp::preferredInputStride(const PixelFormat &inputFormat, const Size &size) +{ + ASSERT(debayer_); + + return debayer_->preferredInputStride(inputFormat, size); +} + +int DebayerIsp::configure(const StreamConfiguration &inputCfg, + const std::vector> &outputCfgs, + const ipa::softisp::IPAConfigInfo &configInfo, + ControlInfoMap *ipaControls) +{ + ASSERT(ipa_ && debayer_); + + int ret = ipa_->configure(configInfo, ipaControls); + if (ret < 0) + return ret; + + ret = debayer_->configure(inputCfg, outputCfgs, ccmEnabled_, lscEnabled_); + if (ret < 0) + return ret; + + LOG(SoftwareIsp, Info) + << "Input " << inputCfg.size + << "-" << BayerFormat::fromPixelFormat(inputCfg.pixelFormat) + << " stride " << inputCfg.stride; + + return 0; +} + +int DebayerIsp::exportBuffers(const Stream *stream, unsigned int count, + std::vector> *buffers) +{ + ASSERT(debayer_ != nullptr); + + /* single output for now */ + if (stream == nullptr) + return -EINVAL; + + return dmaHeap_.exportBuffers(count, { debayer_->frameSize() }, buffers); +} + +int DebayerIsp::queueBuffers(uint32_t frame, FrameBuffer *input, + const std::map &outputs) +{ + /* We only support a single stream for now. */ + if (outputs.size() != 1) + return -EINVAL; + + for (auto [stream, buffer] : outputs) { + if (!buffer) + return -EINVAL; + } + + queuedInputBuffers_.push_back(input); + + for (const auto &[stream, buffer] : outputs) { + queuedOutputBuffers_.push_back(buffer); + process(frame, input, buffer); + } + + return 0; +} + +int DebayerIsp::start() +{ + int ret = RawIsp::start(); + if (ret) + return ret; + + return debayer_->invokeMethod(&Debayer::start, + ConnectionTypeBlocking); +} + +void DebayerIsp::stop() +{ + if (!ispWorkerThread_.isRunning()) + return; + + debayer_->invokeMethod(&Debayer::stop, + ConnectionTypeBlocking); + + RawIsp::stop(); +} + +void DebayerIsp::process(uint32_t frame, FrameBuffer *input, FrameBuffer *output) +{ + ipa_->computeParams(frame); + + debayer_->invokeMethod(&Debayer::process, + ConnectionTypeQueued, frame, input, output, debayerParams_); +} + } /* namespace libcamera */