From patchwork Fri Sep 27 13:46:11 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Milan Zamazal X-Patchwork-Id: 21398 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 76ECDC3257 for ; Fri, 27 Sep 2024 13:46:54 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 748AD6351E; Fri, 27 Sep 2024 15:46:53 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.b="R1ba15Lx"; dkim-atps=neutral Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id E110163519 for ; Fri, 27 Sep 2024 15:46:47 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1727444806; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=zggw+12luMwqe+pkcziq9ChnBTV62x8urgGB28XWzmA=; b=R1ba15Lx2JFdj6HIGccSQjBI+WWc83M2AWEHra4Plb8Fy1biIugLrZuiKrCvtOUf2nF0+O j/DpbYtY7Za762H/CnWDkPQ8d0xNO66lOmnbJJ+2aBugzjCrLzPKRrNk0z0iFDvXuYyMT7 WbObB0cb8DAVgB1m97ps9KLfrsmRwgk= Received: from mx-prod-mc-03.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-505-Hrv9KAT3MByy6bYu7X-9UA-1; Fri, 27 Sep 2024 09:46:45 -0400 X-MC-Unique: Hrv9KAT3MByy6bYu7X-9UA-1 Received: from mx-prod-int-04.mail-002.prod.us-west-2.aws.redhat.com (unknown [10.30.177.40]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-03.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 7AE7F18F4988; Fri, 27 Sep 2024 13:46:44 +0000 (UTC) Received: from nuthatch.redhat.com (unknown [10.45.224.123]) by mx-prod-int-04.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id A74D01944E45; Fri, 27 Sep 2024 13:46:42 +0000 (UTC) From: Milan Zamazal To: libcamera-devel@lists.libcamera.org Cc: Milan Zamazal , Umang Jain , Laurent Pinchart , Daniel Scally , Kieran Bingham Subject: [PATCH v8 05/18] libcamera: software_isp: Make stats frame and buffer aware Date: Fri, 27 Sep 2024 15:46:11 +0200 Message-ID: <20240927134624.801004-6-mzamazal@redhat.com> In-Reply-To: <20240927134624.801004-1-mzamazal@redhat.com> References: <20240927134624.801004-1-mzamazal@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.0 on 10.30.177.40 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" This patch adds frame and bufferId arguments to stats related calls. Although the parameters are currently unused, because frame ids are not tracked and used and the stats buffer is passed around directly rather than being referred by its id, they bring the internal APIs closer to their counterparts in hardware pipelines. It serves as a preparation for followup patches that will introduce: - Frame number tracking in order to switch to DelayedControls (software ISP TODO #11 + #12). - A ring buffer for stats in order to improve passing the stats (software ISP TODO #2). Frame and buffer ids are unrelated for the given purposes but since they are passed together at the same places, the change is implemented as a single patch rather than two, basically the same, patches. Signed-off-by: Milan Zamazal Reviewed-by: Laurent Pinchart Reviewed-by: Daniel Scally --- .../libcamera/internal/software_isp/software_isp.h | 8 +++++--- include/libcamera/ipa/soft.mojom | 4 +++- src/ipa/simple/soft_simple.cpp | 7 +++++-- src/libcamera/pipeline/simple/simple.cpp | 8 +++++--- src/libcamera/software_isp/debayer_cpu.cpp | 8 +++++++- src/libcamera/software_isp/software_isp.cpp | 11 +++++++---- src/libcamera/software_isp/swstats_cpu.cpp | 6 ++++-- src/libcamera/software_isp/swstats_cpu.h | 4 ++-- 8 files changed, 38 insertions(+), 18 deletions(-) diff --git a/include/libcamera/internal/software_isp/software_isp.h b/include/libcamera/internal/software_isp/software_isp.h index f8e000036..3602bce85 100644 --- a/include/libcamera/internal/software_isp/software_isp.h +++ b/include/libcamera/internal/software_isp/software_isp.h @@ -11,6 +11,7 @@ #include #include #include +#include #include #include #include @@ -66,7 +67,8 @@ public: int exportBuffers(const Stream *stream, unsigned int count, std::vector> *buffers); - void processStats(const ControlList &sensorControls); + void processStats(const uint32_t frame, const uint32_t bufferId, + const ControlList &sensorControls); int start(); void stop(); @@ -78,13 +80,13 @@ public: Signal inputBufferReady; Signal outputBufferReady; - Signal<> ispStatsReady; + Signal ispStatsReady; Signal setSensorControls; private: void saveIspParams(); void setSensorCtrls(const ControlList &sensorControls); - void statsReady(); + void statsReady(uint32_t frame, uint32_t bufferId); void inputReady(FrameBuffer *input); void outputReady(FrameBuffer *output); diff --git a/include/libcamera/ipa/soft.mojom b/include/libcamera/ipa/soft.mojom index 0fd47bb0f..f65b6231d 100644 --- a/include/libcamera/ipa/soft.mojom +++ b/include/libcamera/ipa/soft.mojom @@ -23,7 +23,9 @@ interface IPASoftInterface { configure(libcamera.ControlInfoMap sensorCtrlInfoMap) => (int32 ret); - [async] processStats(libcamera.ControlList sensorControls); + [async] processStats(uint32 frame, + uint32 bufferId, + libcamera.ControlList sensorControls); }; interface IPASoftEventInterface { diff --git a/src/ipa/simple/soft_simple.cpp b/src/ipa/simple/soft_simple.cpp index 72321f443..12b5245ed 100644 --- a/src/ipa/simple/soft_simple.cpp +++ b/src/ipa/simple/soft_simple.cpp @@ -75,7 +75,8 @@ public: int start() override; void stop() override; - void processStats(const ControlList &sensorControls) override; + void processStats(const uint32_t frame, const uint32_t bufferId, + const ControlList &sensorControls) override; protected: std::string logPrefix() const override; @@ -249,7 +250,9 @@ void IPASoftSimple::stop() { } -void IPASoftSimple::processStats(const ControlList &sensorControls) +void IPASoftSimple::processStats([[maybe_unused]] const uint32_t frame, + [[maybe_unused]] const uint32_t bufferId, + const ControlList &sensorControls) { SwIspStats::Histogram histogram = stats_->yHistogram; if (ignoreUpdates_ > 0) diff --git a/src/libcamera/pipeline/simple/simple.cpp b/src/libcamera/pipeline/simple/simple.cpp index 81915573b..5aa52789f 100644 --- a/src/libcamera/pipeline/simple/simple.cpp +++ b/src/libcamera/pipeline/simple/simple.cpp @@ -13,6 +13,7 @@ #include #include #include +#include #include #include #include @@ -291,7 +292,7 @@ private: void conversionInputDone(FrameBuffer *buffer); void conversionOutputDone(FrameBuffer *buffer); - void ispStatsReady(); + void ispStatsReady(uint32_t frame, uint32_t bufferId); void setSensorControls(const ControlList &sensorControls); }; @@ -887,10 +888,11 @@ void SimpleCameraData::conversionOutputDone(FrameBuffer *buffer) pipe->completeRequest(request); } -void SimpleCameraData::ispStatsReady() +void SimpleCameraData::ispStatsReady(uint32_t frame, uint32_t bufferId) { /* \todo Use the DelayedControls class */ - swIsp_->processStats(sensor_->getControls({ V4L2_CID_ANALOGUE_GAIN, + swIsp_->processStats(frame, bufferId, + sensor_->getControls({ V4L2_CID_ANALOGUE_GAIN, V4L2_CID_EXPOSURE })); } diff --git a/src/libcamera/software_isp/debayer_cpu.cpp b/src/libcamera/software_isp/debayer_cpu.cpp index 8a757fe9e..6cc6946c7 100644 --- a/src/libcamera/software_isp/debayer_cpu.cpp +++ b/src/libcamera/software_isp/debayer_cpu.cpp @@ -807,7 +807,13 @@ void DebayerCpu::process(FrameBuffer *input, FrameBuffer *output, DebayerParams } } - stats_->finishFrame(); + /* + * Frame and buffer ids are currently not used, so pass zeros as parameters. + * + * \todo Pass real values once frame is passed here and stats buffer passing + * is changed. + */ + stats_->finishFrame(0, 0); outputBufferReady.emit(output); inputBufferReady.emit(input); } diff --git a/src/libcamera/software_isp/software_isp.cpp b/src/libcamera/software_isp/software_isp.cpp index a3ae3e220..a38555684 100644 --- a/src/libcamera/software_isp/software_isp.cpp +++ b/src/libcamera/software_isp/software_isp.cpp @@ -155,15 +155,18 @@ SoftwareIsp::~SoftwareIsp() /** * \brief Process the statistics gathered + * \param[in] frame The frame number + * \param[in] bufferId ID of the statistics buffer * \param[in] sensorControls The sensor controls * * Requests the IPA to calculate new parameters for ISP and new control * values for the sensor. */ -void SoftwareIsp::processStats(const ControlList &sensorControls) +void SoftwareIsp::processStats(const uint32_t frame, const uint32_t bufferId, + const ControlList &sensorControls) { ASSERT(ipa_); - ipa_->processStats(sensorControls); + ipa_->processStats(frame, bufferId, sensorControls); } /** @@ -349,9 +352,9 @@ void SoftwareIsp::setSensorCtrls(const ControlList &sensorControls) setSensorControls.emit(sensorControls); } -void SoftwareIsp::statsReady() +void SoftwareIsp::statsReady(uint32_t frame, uint32_t bufferId) { - ispStatsReady.emit(); + ispStatsReady.emit(frame, bufferId); } void SoftwareIsp::inputReady(FrameBuffer *input) diff --git a/src/libcamera/software_isp/swstats_cpu.cpp b/src/libcamera/software_isp/swstats_cpu.cpp index 815c4d4fa..c520c806e 100644 --- a/src/libcamera/software_isp/swstats_cpu.cpp +++ b/src/libcamera/software_isp/swstats_cpu.cpp @@ -311,13 +311,15 @@ void SwStatsCpu::startFrame(void) /** * \brief Finish statistics calculation for the current frame + * \param[in] frame The frame number + * \param[in] bufferId ID of the statistics buffer * * This may only be called after a successful setWindow() call. */ -void SwStatsCpu::finishFrame(void) +void SwStatsCpu::finishFrame(uint32_t frame, uint32_t bufferId) { *sharedStats_ = stats_; - statsReady.emit(); + statsReady.emit(frame, bufferId); } /** diff --git a/src/libcamera/software_isp/swstats_cpu.h b/src/libcamera/software_isp/swstats_cpu.h index 363e326ff..26a2f462e 100644 --- a/src/libcamera/software_isp/swstats_cpu.h +++ b/src/libcamera/software_isp/swstats_cpu.h @@ -41,7 +41,7 @@ public: int configure(const StreamConfiguration &inputCfg); void setWindow(const Rectangle &window); void startFrame(); - void finishFrame(); + void finishFrame(uint32_t frame, uint32_t bufferId); void processLine0(unsigned int y, const uint8_t *src[]) { @@ -61,7 +61,7 @@ public: (this->*stats2_)(src); } - Signal<> statsReady; + Signal statsReady; private: using statsProcessFn = void (SwStatsCpu::*)(const uint8_t *src[]);