From patchwork Mon Aug 12 11:49:50 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Milan Zamazal X-Patchwork-Id: 20875 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 82FF0C323E for ; Mon, 12 Aug 2024 11:50:26 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 31077633B5; Mon, 12 Aug 2024 13:50:26 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.b="AFLqUvnI"; dkim-atps=neutral Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 4E58D63398 for ; Mon, 12 Aug 2024 13:50:24 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1723463423; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=ckm9IYEVs79Eo9BZ1sbpfcyqt4ruM+3AhHlz5g92kFw=; b=AFLqUvnIv7g5ylBZvtkQE7jI2ezR96e674IeU8nriD7h0fyw+IGTDiUODd/amOhsg76atx g9PxG06bIH04kdUhzNkUJbQ1ACgUSKbIP4IbRjfsXp0KASeA/kgkOnMJdu9h1cqkpl9J2t VDZUbrFD0UX/rsUHloHJHC7ZVA3tANY= Received: from mx-prod-mc-01.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-20-JTzt_XAuMv-qWrOMV9dttg-1; Mon, 12 Aug 2024 07:50:21 -0400 X-MC-Unique: JTzt_XAuMv-qWrOMV9dttg-1 Received: from mx-prod-int-02.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-02.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.15]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-01.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id D6EC819541A8 for ; Mon, 12 Aug 2024 11:50:20 +0000 (UTC) Received: from nuthatch.redhat.com (unknown [10.45.225.57]) by mx-prod-int-02.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id 996ED19772C7; Mon, 12 Aug 2024 11:50:19 +0000 (UTC) From: Milan Zamazal To: libcamera-devel@lists.libcamera.org Cc: Milan Zamazal Subject: [PATCH 01/16] libcamera: software_isp: Remove initializer_list include Date: Mon, 12 Aug 2024 13:49:50 +0200 Message-ID: <20240812115009.946036-2-mzamazal@redhat.com> In-Reply-To: <20240812115009.946036-1-mzamazal@redhat.com> References: <20240812115009.946036-1-mzamazal@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.0 on 10.30.177.15 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" It is not used anywhere in software_isp.h. Signed-off-by: Milan Zamazal --- include/libcamera/internal/software_isp/software_isp.h | 1 - 1 file changed, 1 deletion(-) diff --git a/include/libcamera/internal/software_isp/software_isp.h b/include/libcamera/internal/software_isp/software_isp.h index a3e3a9da..4bb30f07 100644 --- a/include/libcamera/internal/software_isp/software_isp.h +++ b/include/libcamera/internal/software_isp/software_isp.h @@ -8,7 +8,6 @@ #pragma once #include -#include #include #include #include From patchwork Mon Aug 12 11:49:51 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Milan Zamazal X-Patchwork-Id: 20876 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 2B22BC323E for ; Mon, 12 Aug 2024 11:50:29 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id C891B633B5; Mon, 12 Aug 2024 13:50:28 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.b="fBrFvzgP"; dkim-atps=neutral Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id AB02163398 for ; Mon, 12 Aug 2024 13:50:26 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1723463425; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=A/ibfddvq3q51VE8O+dwfQ+15VTVPOwYcKCoeHE0qCs=; b=fBrFvzgPPjo05z8O4+84iFOVDPdvfqrgEViyo3zYMiAFkRtHTpps0z0/A4Feffg6tkKKRC 77m3Pg9AlxdYc4eZ9nRVDHMDDwIAphPgbUrbLHDPEeEDv33QHSuaNuhHbSUjyF0rRTYubS 7QFF0hmEfej+IPpy1YkFS3YZITsvPd0= Received: from mx-prod-mc-04.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-76-ZjpKTn-tMXC6ZdM3p_vwkg-1; Mon, 12 Aug 2024 07:50:23 -0400 X-MC-Unique: ZjpKTn-tMXC6ZdM3p_vwkg-1 Received: from mx-prod-int-02.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-02.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.15]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-04.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id C9838192538A for ; Mon, 12 Aug 2024 11:50:22 +0000 (UTC) Received: from nuthatch.redhat.com (unknown [10.45.225.57]) by mx-prod-int-02.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id 743D419772C7; Mon, 12 Aug 2024 11:50:21 +0000 (UTC) From: Milan Zamazal To: libcamera-devel@lists.libcamera.org Cc: Milan Zamazal Subject: [PATCH 02/16] libcamera: software_isp: Introduce arguments for parameters buffers Date: Mon, 12 Aug 2024 13:49:51 +0200 Message-ID: <20240812115009.946036-3-mzamazal@redhat.com> In-Reply-To: <20240812115009.946036-1-mzamazal@redhat.com> References: <20240812115009.946036-1-mzamazal@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.0 on 10.30.177.15 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Processing parameters in software ISP are currently passed by value. This is unlike hardware pipelines, which use a ring of buffers for the purpose and pass references to the buffers currently selected from the ring rather than passing the whole parameters buffers. This is a preparatory patch to introduce a similar mechanism in software ISP, in order to resolve TODO #5. It adds a new argument paramsBufferId for the future parameters buffer ids passed to the calls. The buffer ids must be passed to the following groups of methods: - IPASoftSimple::prepare, in order to create the parameters in the given buffer for the corresponding frame processing. - SoftwareIsp::saveParams, in order to signal that the parameters are set by the IPA. - Debayer::process, in order to get the buffer with the processing parameters. - SoftwareIsp::releaseIspParams, in order to signal that the selected parameters buffer from the ring is no longer needed for the given frame and can be reused. This is a newly introduced signal. The type of the buffer id parameter is set to uint32_t because: - It can be used in mojom. - It is consistent with the similar types in the hardware pipelines. - It covers file descriptor number range, which will be used as buffer ids (more on this in followup patches). This patch doesn't do more than adding the arguments, to keep the patch simple. The buffer handling will be implemented in the followup patches. Signed-off-by: Milan Zamazal --- .../libcamera/internal/software_isp/software_isp.h | 3 ++- include/libcamera/ipa/soft.mojom | 4 ++-- src/ipa/simple/soft_simple.cpp | 7 ++++--- src/libcamera/software_isp/debayer.cpp | 8 +++++++- src/libcamera/software_isp/debayer.h | 6 +++++- src/libcamera/software_isp/debayer_cpu.cpp | 6 +++++- src/libcamera/software_isp/debayer_cpu.h | 4 +++- src/libcamera/software_isp/software_isp.cpp | 13 ++++++++++--- 8 files changed, 38 insertions(+), 13 deletions(-) diff --git a/include/libcamera/internal/software_isp/software_isp.h b/include/libcamera/internal/software_isp/software_isp.h index 4bb30f07..d2b47ecc 100644 --- a/include/libcamera/internal/software_isp/software_isp.h +++ b/include/libcamera/internal/software_isp/software_isp.h @@ -84,7 +84,8 @@ public: Signal setSensorControls; private: - void saveIspParams(); + void saveIspParams(uint32_t paramsBufferId); + void releaseIspParams(uint32_t paramsBufferId); void setSensorCtrls(const ControlList &sensorControls); void statsReady(uint32_t frame, uint32_t bufferId); void inputReady(FrameBuffer *input); diff --git a/include/libcamera/ipa/soft.mojom b/include/libcamera/ipa/soft.mojom index 85c61824..1498aedf 100644 --- a/include/libcamera/ipa/soft.mojom +++ b/include/libcamera/ipa/soft.mojom @@ -24,11 +24,11 @@ interface IPASoftInterface { => (int32 ret); [async] queueRequest(uint32 frame, libcamera.ControlList sensorControls); - prepare(uint32 frame); + prepare(uint32 frame, uint32 paramsBufferId); [async] processStats(uint32 frame, uint32 bufferId, libcamera.ControlList sensorControls); }; interface IPASoftEventInterface { setSensorControls(libcamera.ControlList sensorControls); - setIspParams(); + setIspParams(uint32 paramsBufferId); }; diff --git a/src/ipa/simple/soft_simple.cpp b/src/ipa/simple/soft_simple.cpp index 547d0fcf..9d5fe923 100644 --- a/src/ipa/simple/soft_simple.cpp +++ b/src/ipa/simple/soft_simple.cpp @@ -56,7 +56,7 @@ public: void stop() override; void queueRequest(const uint32_t frame, const ControlList &controls) override; - void prepare(const uint32_t frame) override; + void prepare(const uint32_t frame, const uint32_t paramsBufferId) override; void processStats(const uint32_t frame, const uint32_t bufferId, const ControlList &sensorControls) override; @@ -254,12 +254,13 @@ void IPASoftSimple::queueRequest(const uint32_t frame, const ControlList &contro algo->queueRequest(context_, frame, frameContext, controls); } -void IPASoftSimple::prepare(const uint32_t frame) +void IPASoftSimple::prepare(const uint32_t frame, + const uint32_t paramsBufferId) { IPAFrameContext &frameContext = context_.frameContexts.get(frame); for (auto const &algo : algorithms()) algo->prepare(context_, frame, frameContext, params_); - setIspParams.emit(); + setIspParams.emit(paramsBufferId); } void IPASoftSimple::processStats( diff --git a/src/libcamera/software_isp/debayer.cpp b/src/libcamera/software_isp/debayer.cpp index e2295f35..8bd58c3d 100644 --- a/src/libcamera/software_isp/debayer.cpp +++ b/src/libcamera/software_isp/debayer.cpp @@ -94,9 +94,10 @@ Debayer::~Debayer() */ /** - * \fn void Debayer::process(uint32_t frame, FrameBuffer *input, FrameBuffer *output, DebayerParams params) + * \fn void Debayer::process(uint32_t frame, const uint32_t paramsBufferId, FrameBuffer *input, FrameBuffer *output, DebayerParams params) * \brief Process the bayer data into the requested format * \param[in] frame The frame number + * \param[in] paramsBufferId The id of the params buffer in use * \param[in] input The input buffer * \param[in] output The output buffer * \param[in] params The parameters to be used in debayering @@ -124,4 +125,9 @@ Debayer::~Debayer() * \brief Signals when the output buffer is ready */ +/** + * \var Signal Debayer::releaseIspParams + * \brief Signals when the processing params are no longer needed + */ + } /* namespace libcamera */ diff --git a/src/libcamera/software_isp/debayer.h b/src/libcamera/software_isp/debayer.h index d7ca060d..93020735 100644 --- a/src/libcamera/software_isp/debayer.h +++ b/src/libcamera/software_isp/debayer.h @@ -40,12 +40,16 @@ public: virtual std::tuple strideAndFrameSize(const PixelFormat &outputFormat, const Size &size) = 0; - virtual void process(uint32_t frame, FrameBuffer *input, FrameBuffer *output, DebayerParams params) = 0; + virtual void process(uint32_t frame, + const uint32_t paramsBufferId, + FrameBuffer *input, FrameBuffer *output, + DebayerParams params) = 0; virtual SizeRange sizes(PixelFormat inputFormat, const Size &inputSize) = 0; Signal inputBufferReady; Signal outputBufferReady; + Signal releaseIspParams; private: virtual Size patternSize(PixelFormat inputFormat) = 0; diff --git a/src/libcamera/software_isp/debayer_cpu.cpp b/src/libcamera/software_isp/debayer_cpu.cpp index f7b3a7d1..20a15ae0 100644 --- a/src/libcamera/software_isp/debayer_cpu.cpp +++ b/src/libcamera/software_isp/debayer_cpu.cpp @@ -724,7 +724,9 @@ static inline int64_t timeDiff(timespec &after, timespec &before) (int64_t)after.tv_nsec - (int64_t)before.tv_nsec; } -void DebayerCpu::process(uint32_t frame, FrameBuffer *input, FrameBuffer *output, DebayerParams params) +void DebayerCpu::process(uint32_t frame, + const uint32_t paramsBufferId, + FrameBuffer *input, FrameBuffer *output, DebayerParams params) { timespec frameStartTime; @@ -737,6 +739,8 @@ void DebayerCpu::process(uint32_t frame, FrameBuffer *input, FrameBuffer *output red_ = swapRedBlueGains_ ? params.blue : params.red; blue_ = swapRedBlueGains_ ? params.red : params.blue; + releaseIspParams.emit(paramsBufferId); + /* Copy metadata from the input buffer */ FrameMetadata &metadata = output->_d()->metadata(); metadata.status = input->metadata().status; diff --git a/src/libcamera/software_isp/debayer_cpu.h b/src/libcamera/software_isp/debayer_cpu.h index 2c47e7c6..0b5a5258 100644 --- a/src/libcamera/software_isp/debayer_cpu.h +++ b/src/libcamera/software_isp/debayer_cpu.h @@ -36,7 +36,9 @@ public: std::vector formats(PixelFormat input); std::tuple strideAndFrameSize(const PixelFormat &outputFormat, const Size &size); - void process(uint32_t frame, FrameBuffer *input, FrameBuffer *output, DebayerParams params); + void process(uint32_t frame, + const uint32_t paramsBufferId, + FrameBuffer *input, FrameBuffer *output, DebayerParams params); SizeRange sizes(PixelFormat inputFormat, const Size &inputSize); /** diff --git a/src/libcamera/software_isp/software_isp.cpp b/src/libcamera/software_isp/software_isp.cpp index dbf27f31..0412b401 100644 --- a/src/libcamera/software_isp/software_isp.cpp +++ b/src/libcamera/software_isp/software_isp.cpp @@ -106,6 +106,7 @@ SoftwareIsp::SoftwareIsp(PipelineHandler *pipe, const CameraSensor *sensor) debayer_ = std::make_unique(std::move(stats)); debayer_->inputBufferReady.connect(this, &SoftwareIsp::inputReady); debayer_->outputBufferReady.connect(this, &SoftwareIsp::outputReady); + debayer_->releaseIspParams.connect(this, &SoftwareIsp::releaseIspParams); ipa_ = IPAManager::createIPA(pipe, 0, 0); if (!ipa_) { @@ -352,16 +353,22 @@ void SoftwareIsp::stop() */ void SoftwareIsp::process(uint32_t frame, FrameBuffer *input, FrameBuffer *output) { - ipa_->prepare(frame); + /* \todo Provide a real value */ + constexpr uint32_t paramsBufferId = 0; + ipa_->prepare(frame, paramsBufferId); debayer_->invokeMethod(&DebayerCpu::process, - ConnectionTypeQueued, frame, input, output, debayerParams_); + ConnectionTypeQueued, frame, paramsBufferId, input, output, debayerParams_); } -void SoftwareIsp::saveIspParams() +void SoftwareIsp::saveIspParams([[maybe_unused]] uint32_t paramsBufferId) { debayerParams_ = *sharedParams_; } +void SoftwareIsp::releaseIspParams([[maybe_unused]] uint32_t paramsBufferId) +{ +} + void SoftwareIsp::setSensorCtrls(const ControlList &sensorControls) { setSensorControls.emit(sensorControls); From patchwork Mon Aug 12 11:49:52 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Milan Zamazal X-Patchwork-Id: 20877 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 45233C323E for ; Mon, 12 Aug 2024 11:50:31 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id D4462633B5; Mon, 12 Aug 2024 13:50:30 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.b="F3wdGEDp"; dkim-atps=neutral Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 96BF763398 for ; Mon, 12 Aug 2024 13:50:27 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1723463426; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=JOfNXZhsJOwjlwrExudQJC5YRNJ5y6fUPZY+0WyjHOM=; b=F3wdGEDpMhHAj/Kzm9aCli3zklt5YloxhRDyIfPIItLwNjjpeIxQX/ix9OAJ5lm/GnR/tJ l0h0o61n54jqOZ6eCR6moEYezzyCuo2n3TpfnNgzPtpOPwlcAw0P5vAfIDno9OoloOaM2z nRq0xPhSVG2pD+2Zhmd8o3oLkJr/sDE= Received: from mx-prod-mc-04.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-486-AKJ3hmxbOse8QyuKb_ExaQ-1; Mon, 12 Aug 2024 07:50:25 -0400 X-MC-Unique: AKJ3hmxbOse8QyuKb_ExaQ-1 Received: from mx-prod-int-02.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-02.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.15]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-04.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 985FE1918166 for ; Mon, 12 Aug 2024 11:50:24 +0000 (UTC) Received: from nuthatch.redhat.com (unknown [10.45.225.57]) by mx-prod-int-02.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id 4CB7E19772C4; Mon, 12 Aug 2024 11:50:22 +0000 (UTC) From: Milan Zamazal To: libcamera-devel@lists.libcamera.org Cc: Milan Zamazal Subject: [PATCH 03/16] libcamera: software_isp: Separate allocation of the parameters buffer Date: Mon, 12 Aug 2024 13:49:52 +0200 Message-ID: <20240812115009.946036-4-mzamazal@redhat.com> In-Reply-To: <20240812115009.946036-1-mzamazal@redhat.com> References: <20240812115009.946036-1-mzamazal@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.0 on 10.30.177.15 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" There will be some additions to that code in followup patches and let's not blow the code in SoftwareIsp constructor even more. Signed-off-by: Milan Zamazal --- .../internal/software_isp/software_isp.h | 1 + src/libcamera/software_isp/software_isp.cpp | 51 +++++++++++-------- 2 files changed, 30 insertions(+), 22 deletions(-) diff --git a/include/libcamera/internal/software_isp/software_isp.h b/include/libcamera/internal/software_isp/software_isp.h index d2b47ecc..fc9200dd 100644 --- a/include/libcamera/internal/software_isp/software_isp.h +++ b/include/libcamera/internal/software_isp/software_isp.h @@ -95,6 +95,7 @@ private: Thread ispWorkerThread_; SharedMemObject sharedParams_; DebayerParams debayerParams_; + bool allocateParamsBuffers(); DmaBufAllocator dmaHeap_; std::unique_ptr ipa_; diff --git a/src/libcamera/software_isp/software_isp.cpp b/src/libcamera/software_isp/software_isp.cpp index 0412b401..3f41b71d 100644 --- a/src/libcamera/software_isp/software_isp.cpp +++ b/src/libcamera/software_isp/software_isp.cpp @@ -68,33 +68,12 @@ SoftwareIsp::SoftwareIsp(PipelineHandler *pipe, const CameraSensor *sensor) DmaBufAllocator::DmaBufAllocatorFlag::SystemHeap | DmaBufAllocator::DmaBufAllocatorFlag::UDmaBuf) { - /* - * debayerParams_ must be initialized because the initial value is used for - * the first two frames, i.e. until stats processing starts providing its - * own parameters. - * - * \todo This should be handled in the same place as the related - * operations, in the IPA module. - */ - std::array gammaTable; - for (unsigned int i = 0; i < 256; i++) - gammaTable[i] = UINT8_MAX * std::pow(i / 256.0, 0.5); - for (unsigned int i = 0; i < DebayerParams::kRGBLookupSize; i++) { - debayerParams_.red[i] = gammaTable[i]; - debayerParams_.green[i] = gammaTable[i]; - debayerParams_.blue[i] = gammaTable[i]; - } - if (!dmaHeap_.isValid()) { LOG(SoftwareIsp, Error) << "Failed to create DmaBufAllocator object"; return; } - - sharedParams_ = SharedMemObject("softIsp_params"); - if (!sharedParams_) { - LOG(SoftwareIsp, Error) << "Failed to create shared memory for parameters"; + if (!allocateParamsBuffers()) return; - } auto stats = std::make_unique(); if (!stats->isValid()) { @@ -145,6 +124,34 @@ SoftwareIsp::~SoftwareIsp() debayer_.reset(); } +bool SoftwareIsp::allocateParamsBuffers() +{ + /* + * DebayerParams must be initialized because the initial value is used for + * the first two frames, i.e. until stats processing starts providing its + * own parameters. + * + * \todo This should be handled in the same place as the related + * operations, in the IPA module. + */ + std::array gammaTable; + for (unsigned int i = 0; i < 256; i++) + gammaTable[i] = UINT8_MAX * std::pow(i / 256.0, 0.5); + for (unsigned int j = 0; j < DebayerParams::kRGBLookupSize; j++) { + debayerParams_.red[j] = gammaTable[j]; + debayerParams_.green[j] = gammaTable[j]; + debayerParams_.blue[j] = gammaTable[j]; + } + + sharedParams_ = SharedMemObject("softIsp_params"); + if (!sharedParams_) { + LOG(SoftwareIsp, Error) << "Failed to create shared memory for parameters"; + return false; + } + + return true; +} + /** * \fn int SoftwareIsp::loadConfiguration([[maybe_unused]] const std::string &filename) * \brief Load a configuration from a file From patchwork Mon Aug 12 11:49:53 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Milan Zamazal X-Patchwork-Id: 20878 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 984A7C323E for ; Mon, 12 Aug 2024 11:50:33 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 19B53633C8; Mon, 12 Aug 2024 13:50:33 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.b="FXfaqRP1"; dkim-atps=neutral Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 455A9633C3 for ; Mon, 12 Aug 2024 13:50:29 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1723463428; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=CPqZkoFnoMEU874j+kZXP8wCNZx/M3NBdrBgOYAhvVY=; b=FXfaqRP1f59yHhasvaV9m7hUOj1saFIzUpPC/MZRwZXxFYvay6jOF6rdvPn3tzlRhWDESc 8vB6nz6mk/oZVwvQNDZJmb4t246VGEuiA7Ztva5qmtmect7fZMUPAjtqbCxfcGA91tTB6+ Y11c5ztDKfNTSGagmk79kR+/8M2kmmo= Received: from mx-prod-mc-05.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-357-lTtgKFUMMMm7NSo5x9L-jw-1; Mon, 12 Aug 2024 07:50:26 -0400 X-MC-Unique: lTtgKFUMMMm7NSo5x9L-jw-1 Received: from mx-prod-int-02.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-02.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.15]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-05.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 05A421955D5A for ; Mon, 12 Aug 2024 11:50:26 +0000 (UTC) Received: from nuthatch.redhat.com (unknown [10.45.225.57]) by mx-prod-int-02.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id D022219772C4; Mon, 12 Aug 2024 11:50:24 +0000 (UTC) From: Milan Zamazal To: libcamera-devel@lists.libcamera.org Cc: Milan Zamazal Subject: [PATCH 04/16] libcamera: software_isp: Track unused parameters buffers Date: Mon, 12 Aug 2024 13:49:53 +0200 Message-ID: <20240812115009.946036-5-mzamazal@redhat.com> In-Reply-To: <20240812115009.946036-1-mzamazal@redhat.com> References: <20240812115009.946036-1-mzamazal@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.0 on 10.30.177.15 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" As a preparation for introducing a ring of parameters buffers, this patch introduces tracking of parameters buffers available for use. SofwareIsp::availableParams_ is a queue of ids of the buffers available for use. When a fresh parameter buffer is needed, its id is retrieved from there and once the buffer is no longer needed, it's put back there, in a method invoked by a signal. This is similar to what the hardware pipelines do. If a parameters buffer is requested but there is no free available, it is an erroneous situation. To recover from it, we have currently no better mechanism than to wait for an available buffer. Simply returning from the processing would mean the finishing signals are not called, resulting in e.g. the input buffer not being returned. This is different from the hardware pipelines, which are structured somewhat differently. We use buffers' file descriptors as buffer ids. The parameters buffers will be shared with the IPA and debayering and we need their common identification everywhere. The buffer file descriptors are shared unchanged so they can be used for the purpose, avoiding a need to pass a special id->buffer mapping to the IPA and debayering on initialization. Note that the parameters buffer id is still actually unused and the only buffer currently used is still copied when passed to debayering. This patch is just preparation for followup patches that will introduce multiple buffers. Signed-off-by: Milan Zamazal --- .../internal/software_isp/software_isp.h | 2 ++ src/libcamera/software_isp/software_isp.cpp | 18 +++++++++++++++--- 2 files changed, 17 insertions(+), 3 deletions(-) diff --git a/include/libcamera/internal/software_isp/software_isp.h b/include/libcamera/internal/software_isp/software_isp.h index fc9200dd..74f5b952 100644 --- a/include/libcamera/internal/software_isp/software_isp.h +++ b/include/libcamera/internal/software_isp/software_isp.h @@ -10,6 +10,7 @@ #include #include #include +#include #include #include #include @@ -95,6 +96,7 @@ private: Thread ispWorkerThread_; SharedMemObject sharedParams_; DebayerParams debayerParams_; + std::queue availableParams_; bool allocateParamsBuffers(); DmaBufAllocator dmaHeap_; diff --git a/src/libcamera/software_isp/software_isp.cpp b/src/libcamera/software_isp/software_isp.cpp index 3f41b71d..a9865398 100644 --- a/src/libcamera/software_isp/software_isp.cpp +++ b/src/libcamera/software_isp/software_isp.cpp @@ -149,6 +149,10 @@ bool SoftwareIsp::allocateParamsBuffers() return false; } + ASSERT(sharedParams_.fd().get() >= 0); + const uint32_t bufferId = sharedParams_.fd().get(); + availableParams_.push(bufferId); + return true; } @@ -360,8 +364,15 @@ void SoftwareIsp::stop() */ void SoftwareIsp::process(uint32_t frame, FrameBuffer *input, FrameBuffer *output) { - /* \todo Provide a real value */ - constexpr uint32_t paramsBufferId = 0; + if (availableParams_.empty()) { + LOG(SoftwareIsp, Error) << "Parameters buffer underrun"; + /* Well, busy loop, but this situation shouldn't normally happen. */ + while (availableParams_.empty()) + ; + } + + const uint32_t paramsBufferId = availableParams_.front(); + availableParams_.pop(); ipa_->prepare(frame, paramsBufferId); debayer_->invokeMethod(&DebayerCpu::process, ConnectionTypeQueued, frame, paramsBufferId, input, output, debayerParams_); @@ -372,8 +383,9 @@ void SoftwareIsp::saveIspParams([[maybe_unused]] uint32_t paramsBufferId) debayerParams_ = *sharedParams_; } -void SoftwareIsp::releaseIspParams([[maybe_unused]] uint32_t paramsBufferId) +void SoftwareIsp::releaseIspParams(uint32_t paramsBufferId) { + availableParams_.push(paramsBufferId); } void SoftwareIsp::setSensorCtrls(const ControlList &sensorControls) From patchwork Mon Aug 12 11:49:54 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Milan Zamazal X-Patchwork-Id: 20879 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id E7F53C324E for ; Mon, 12 Aug 2024 11:50:35 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 86DA5633C9; Mon, 12 Aug 2024 13:50:35 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.b="IDx/sXCB"; dkim-atps=neutral Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id C0B48633C5 for ; Mon, 12 Aug 2024 13:50:30 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1723463429; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=k2P6O0UU/Tz73OWa50/59GqEoB8O6J8pGlqrVXwKshk=; b=IDx/sXCBxySKoWiaXXzZrxaEseV2RlumutuWNF52ix7VE5V3G2JCvplrUQqWVFyAOkQUNn Q1GCA7O+UZjl8dVv2PGauJ+u+hixFzRYag871ZMPt9YYDdmh/g3BskZ3LjpG1lfUi5U5fF IfT2BW8HsrywhJxTUOwei6igpjhDNi0= Received: from mx-prod-mc-04.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-427-sYjgxDlEO1qbokU1HxNqKA-1; Mon, 12 Aug 2024 07:50:28 -0400 X-MC-Unique: sYjgxDlEO1qbokU1HxNqKA-1 Received: from mx-prod-int-02.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-02.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.15]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-04.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 605691925383 for ; Mon, 12 Aug 2024 11:50:27 +0000 (UTC) Received: from nuthatch.redhat.com (unknown [10.45.225.57]) by mx-prod-int-02.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id 5D9F719772C4; Mon, 12 Aug 2024 11:50:26 +0000 (UTC) From: Milan Zamazal To: libcamera-devel@lists.libcamera.org Cc: Milan Zamazal Subject: [PATCH 05/16] libcamera: software_isp: Allocation of multiple params buffers Date: Mon, 12 Aug 2024 13:49:54 +0200 Message-ID: <20240812115009.946036-6-mzamazal@redhat.com> In-Reply-To: <20240812115009.946036-1-mzamazal@redhat.com> References: <20240812115009.946036-1-mzamazal@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.0 on 10.30.177.15 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" SoftwareIsp::sharedParams_ is changed from a single buffer to a map of multiple buffer ids and the corresponding buffers. This requires some code rearranging within SoftwareIsp::allocateBuffers to keep the right sequence of actions. Currently, only a single buffer is allocated, this will be changed in a followup patch. Signed-off-by: Milan Zamazal --- .../internal/software_isp/software_isp.h | 3 +- src/libcamera/software_isp/software_isp.cpp | 55 +++++++++++-------- 2 files changed, 34 insertions(+), 24 deletions(-) diff --git a/include/libcamera/internal/software_isp/software_isp.h b/include/libcamera/internal/software_isp/software_isp.h index 74f5b952..c2fcc307 100644 --- a/include/libcamera/internal/software_isp/software_isp.h +++ b/include/libcamera/internal/software_isp/software_isp.h @@ -94,7 +94,8 @@ private: std::unique_ptr debayer_; Thread ispWorkerThread_; - SharedMemObject sharedParams_; + static constexpr unsigned int kParamStatBufferCount = 1; + std::map> sharedParams_; DebayerParams debayerParams_; std::queue availableParams_; bool allocateParamsBuffers(); diff --git a/src/libcamera/software_isp/software_isp.cpp b/src/libcamera/software_isp/software_isp.cpp index a9865398..d78537fc 100644 --- a/src/libcamera/software_isp/software_isp.cpp +++ b/src/libcamera/software_isp/software_isp.cpp @@ -57,6 +57,11 @@ LOG_DEFINE_CATEGORY(SoftwareIsp) * ready */ +/** + * \var SoftwareIsp::kParamStatBufferCount + * \brief The number of stats and params buffers (each of them) + */ + /** * \brief Constructs SoftwareIsp object * \param[in] pipe The pipeline handler in use @@ -104,7 +109,7 @@ SoftwareIsp::SoftwareIsp(PipelineHandler *pipe, const CameraSensor *sensor) int ret = ipa_->init(IPASettings{ ipaTuningFile, sensor->model() }, debayer_->getStatsFD(), - sharedParams_.fd(), + sharedParams_.begin()->second.fd(), sensor->controls()); if (ret) { LOG(SoftwareIsp, Error) << "IPA init failed"; @@ -126,32 +131,36 @@ SoftwareIsp::~SoftwareIsp() bool SoftwareIsp::allocateParamsBuffers() { - /* - * DebayerParams must be initialized because the initial value is used for - * the first two frames, i.e. until stats processing starts providing its - * own parameters. - * - * \todo This should be handled in the same place as the related - * operations, in the IPA module. - */ std::array gammaTable; for (unsigned int i = 0; i < 256; i++) gammaTable[i] = UINT8_MAX * std::pow(i / 256.0, 0.5); - for (unsigned int j = 0; j < DebayerParams::kRGBLookupSize; j++) { - debayerParams_.red[j] = gammaTable[j]; - debayerParams_.green[j] = gammaTable[j]; - debayerParams_.blue[j] = gammaTable[j]; - } - sharedParams_ = SharedMemObject("softIsp_params"); - if (!sharedParams_) { - LOG(SoftwareIsp, Error) << "Failed to create shared memory for parameters"; - return false; - } + for (unsigned int i = 0; i < kParamStatBufferCount; i++) { + auto params = SharedMemObject("softIsp_params"); + if (!params) { + LOG(SoftwareIsp, Error) << "Failed to create shared memory for parameters"; + return false; + } - ASSERT(sharedParams_.fd().get() >= 0); - const uint32_t bufferId = sharedParams_.fd().get(); - availableParams_.push(bufferId); + /* + * DebayerParams must be initialized because the initial value is used for + * the first two frames, i.e. until stats processing starts providing its + * own parameters. + * + * \todo This should be handled in the same place as the related + * operations, in the IPA module. + */ + for (unsigned int j = 0; j < DebayerParams::kRGBLookupSize; j++) { + params->red[j] = gammaTable[j]; + params->green[j] = gammaTable[j]; + params->blue[j] = gammaTable[j]; + } + + ASSERT(params.fd().get() >= 0); + const uint32_t bufferId = params.fd().get(); + availableParams_.push(bufferId); + sharedParams_.emplace(bufferId, std::move(params)); + } return true; } @@ -380,7 +389,7 @@ void SoftwareIsp::process(uint32_t frame, FrameBuffer *input, FrameBuffer *outpu void SoftwareIsp::saveIspParams([[maybe_unused]] uint32_t paramsBufferId) { - debayerParams_ = *sharedParams_; + debayerParams_ = *sharedParams_.begin()->second; } void SoftwareIsp::releaseIspParams(uint32_t paramsBufferId) From patchwork Mon Aug 12 11:49:55 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Milan Zamazal X-Patchwork-Id: 20880 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 82AC7C323E for ; Mon, 12 Aug 2024 11:50:37 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id D44D7633CB; Mon, 12 Aug 2024 13:50:36 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.b="KlyqG98r"; dkim-atps=neutral Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 4AD33633BF for ; Mon, 12 Aug 2024 13:50:32 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1723463431; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=OF9/rB2Yo1/21SBoGbiRvv4x8gx7d18lkjE7VG1i1OQ=; b=KlyqG98rbvVUF34lfzihapCvYiBVGWzV7eVNFsVEpPWzoGn8lzAuzMtt/hCw3xSZeRQ25/ Q8TCScAVu9Rbz+54L83VjVEjOtDQTUJuofvqBI//vByamnLa5TVeGZvP34oyHGUN+XFzZB gkwkKkLwcRjDnfU4Z7d/8aHziyfapq0= Received: from mx-prod-mc-01.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-449-gpeJGR9kOxG7vOOvz4-bKg-1; Mon, 12 Aug 2024 07:50:29 -0400 X-MC-Unique: gpeJGR9kOxG7vOOvz4-bKg-1 Received: from mx-prod-int-02.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-02.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.15]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-01.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id CCAC61953942 for ; Mon, 12 Aug 2024 11:50:28 +0000 (UTC) Received: from nuthatch.redhat.com (unknown [10.45.225.57]) by mx-prod-int-02.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id CBD9519772C4; Mon, 12 Aug 2024 11:50:27 +0000 (UTC) From: Milan Zamazal To: libcamera-devel@lists.libcamera.org Cc: Milan Zamazal Subject: [PATCH 06/16] libcamera: software_isp: Allocate multiple parameters buffers Date: Mon, 12 Aug 2024 13:49:55 +0200 Message-ID: <20240812115009.946036-7-mzamazal@redhat.com> In-Reply-To: <20240812115009.946036-1-mzamazal@redhat.com> References: <20240812115009.946036-1-mzamazal@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.0 on 10.30.177.15 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" We want to use more than one parameters buffer. Processing statistics (and setting the corresponding parameters) is asynchronous and may work with a parameters buffer while the last parameters buffer may be in or wait for use in debayering. The question is how many buffer we need. A starting idea might be to use the same number as for input and output buffers. This may not be necessarily the best or even correct idea, but if the number of those buffers is sufficient now then perhaps the same number of the parameters buffers (and later statistics buffers) would work. Unless multiple streams are used... Nevertheless, the number of the input and output buffers is currently fixed, which makes it possible to use in init, before configure, where the current single buffer is allocated. It just requires to make the given constant public in SimplePipelineHandler so that it can be passed to the SoftwareIsp constructor. Another alternative would be to determine the right number of buffers in SoftwareIsp::configure. This would be more complicated as it would require some more refactoring, allocating the buffers in configure and passing their file descriptors to the IPA and debayering on demand. Only the first allocated buffer is actually used at the moment. This will be changed in the followup patches once support for multiple buffers is added to the IPA and debayering. Signed-off-by: Milan Zamazal --- .../internal/software_isp/software_isp.h | 6 +++--- src/libcamera/pipeline/simple/simple.cpp | 6 +++--- src/libcamera/software_isp/software_isp.cpp | 15 ++++++--------- 3 files changed, 12 insertions(+), 15 deletions(-) diff --git a/include/libcamera/internal/software_isp/software_isp.h b/include/libcamera/internal/software_isp/software_isp.h index c2fcc307..8956978c 100644 --- a/include/libcamera/internal/software_isp/software_isp.h +++ b/include/libcamera/internal/software_isp/software_isp.h @@ -46,7 +46,8 @@ LOG_DECLARE_CATEGORY(SoftwareIsp) class SoftwareIsp { public: - SoftwareIsp(PipelineHandler *pipe, const CameraSensor *sensor); + SoftwareIsp(PipelineHandler *pipe, const CameraSensor *sensor, + const unsigned int bufferCount); ~SoftwareIsp(); int loadConfiguration([[maybe_unused]] const std::string &filename) { return 0; } @@ -94,11 +95,10 @@ private: std::unique_ptr debayer_; Thread ispWorkerThread_; - static constexpr unsigned int kParamStatBufferCount = 1; std::map> sharedParams_; DebayerParams debayerParams_; std::queue availableParams_; - bool allocateParamsBuffers(); + bool allocateParamsBuffers(const unsigned int bufferCount); DmaBufAllocator dmaHeap_; std::unique_ptr ipa_; diff --git a/src/libcamera/pipeline/simple/simple.cpp b/src/libcamera/pipeline/simple/simple.cpp index f0bcafba..459f617b 100644 --- a/src/libcamera/pipeline/simple/simple.cpp +++ b/src/libcamera/pipeline/simple/simple.cpp @@ -349,13 +349,12 @@ public: V4L2Subdevice *subdev(const MediaEntity *entity); MediaDevice *converter() { return converter_; } bool swIspEnabled() const { return swIspEnabled_; } + static constexpr unsigned int kNumInternalBuffers = 3; protected: int queueRequestDevice(Camera *camera, Request *request) override; private: - static constexpr unsigned int kNumInternalBuffers = 3; - struct EntityData { std::unique_ptr video; std::unique_ptr subdev; @@ -531,7 +530,8 @@ int SimpleCameraData::init() * Instantiate Soft ISP if this is enabled for the given driver and no converter is used. */ if (!converter_ && pipe->swIspEnabled()) { - swIsp_ = std::make_unique(pipe, sensor_.get()); + swIsp_ = std::make_unique(pipe, sensor_.get(), + pipe->kNumInternalBuffers); if (!swIsp_->isValid()) { LOG(SimplePipeline, Warning) << "Failed to create software ISP, disabling software debayering"; diff --git a/src/libcamera/software_isp/software_isp.cpp b/src/libcamera/software_isp/software_isp.cpp index d78537fc..d48be2d2 100644 --- a/src/libcamera/software_isp/software_isp.cpp +++ b/src/libcamera/software_isp/software_isp.cpp @@ -57,18 +57,15 @@ LOG_DEFINE_CATEGORY(SoftwareIsp) * ready */ -/** - * \var SoftwareIsp::kParamStatBufferCount - * \brief The number of stats and params buffers (each of them) - */ - /** * \brief Constructs SoftwareIsp object * \param[in] pipe The pipeline handler in use * \param[in] sensor Pointer to the CameraSensor instance owned by the pipeline + * \param[in] bufferCount Number of parameters buffers and stats buffers to allocate * handler */ -SoftwareIsp::SoftwareIsp(PipelineHandler *pipe, const CameraSensor *sensor) +SoftwareIsp::SoftwareIsp(PipelineHandler *pipe, const CameraSensor *sensor, + const unsigned bufferCount) : dmaHeap_(DmaBufAllocator::DmaBufAllocatorFlag::CmaHeap | DmaBufAllocator::DmaBufAllocatorFlag::SystemHeap | DmaBufAllocator::DmaBufAllocatorFlag::UDmaBuf) @@ -77,7 +74,7 @@ SoftwareIsp::SoftwareIsp(PipelineHandler *pipe, const CameraSensor *sensor) LOG(SoftwareIsp, Error) << "Failed to create DmaBufAllocator object"; return; } - if (!allocateParamsBuffers()) + if (!allocateParamsBuffers(bufferCount)) return; auto stats = std::make_unique(); @@ -129,13 +126,13 @@ SoftwareIsp::~SoftwareIsp() debayer_.reset(); } -bool SoftwareIsp::allocateParamsBuffers() +bool SoftwareIsp::allocateParamsBuffers(const unsigned int bufferCount) { std::array gammaTable; for (unsigned int i = 0; i < 256; i++) gammaTable[i] = UINT8_MAX * std::pow(i / 256.0, 0.5); - for (unsigned int i = 0; i < kParamStatBufferCount; i++) { + for (unsigned int i = 0; i < bufferCount; i++) { auto params = SharedMemObject("softIsp_params"); if (!params) { LOG(SoftwareIsp, Error) << "Failed to create shared memory for parameters"; From patchwork Mon Aug 12 11:49:56 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Milan Zamazal X-Patchwork-Id: 20881 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 84722C323E for ; Mon, 12 Aug 2024 11:50:39 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 34E2F633D0; Mon, 12 Aug 2024 13:50:39 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.b="aWl/8jIm"; dkim-atps=neutral Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 76387633C9 for ; Mon, 12 Aug 2024 13:50:33 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1723463432; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=0VH6w5H5H/h2H5c+QEbIYYLm9AFq2VBsC+KuYmpCxZ0=; b=aWl/8jImTNFAHNwnSr2paIAr420P5mLl/xFuOa7Sw9ADOL6r9Zb7ZyGP/S55Wd3e8yAdce lzLiToDIn3qMgP+BV+k6SS/vV1Zoe4tKyjR3g0DACfuzUKoqR7O++iTo9ulE+yaQbOyDKW xppflaLtDN45ghFpn0AZd+f+S0puBtg= Received: from mx-prod-mc-02.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-66-Pr4eGMrwNKWFgBcavFyrcw-1; Mon, 12 Aug 2024 07:50:31 -0400 X-MC-Unique: Pr4eGMrwNKWFgBcavFyrcw-1 Received: from mx-prod-int-02.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-02.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.15]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-02.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 6881F18EB226 for ; Mon, 12 Aug 2024 11:50:30 +0000 (UTC) Received: from nuthatch.redhat.com (unknown [10.45.225.57]) by mx-prod-int-02.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id 4CE0819772C7; Mon, 12 Aug 2024 11:50:29 +0000 (UTC) From: Milan Zamazal To: libcamera-devel@lists.libcamera.org Cc: Milan Zamazal Subject: [PATCH 07/16] libcamera: software_isp: Use multiple parameters buffers in IPA Date: Mon, 12 Aug 2024 13:49:56 +0200 Message-ID: <20240812115009.946036-8-mzamazal@redhat.com> In-Reply-To: <20240812115009.946036-1-mzamazal@redhat.com> References: <20240812115009.946036-1-mzamazal@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.0 on 10.30.177.15 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Instead of the single parameters buffer file descriptor, pass the file descriptors of all the parameters buffers to the IPA and let it use them according to the passed buffer id. This means: - The file descriptors of all the buffers must be passed to the IPA. - The IPA must store mappings of the buffer ids to the corresponding buffers. - The current buffer id is passed to/from the IPA. The parameters buffers are still copied for debayering, this will be addressed in the next patch. Signed-off-by: Milan Zamazal --- include/libcamera/ipa/soft.mojom | 2 +- src/ipa/simple/soft_simple.cpp | 29 +++++++++++---------- src/libcamera/software_isp/software_isp.cpp | 12 ++++++--- 3 files changed, 25 insertions(+), 18 deletions(-) diff --git a/include/libcamera/ipa/soft.mojom b/include/libcamera/ipa/soft.mojom index 1498aedf..86dd988f 100644 --- a/include/libcamera/ipa/soft.mojom +++ b/include/libcamera/ipa/soft.mojom @@ -15,7 +15,7 @@ struct IPAConfigInfo { interface IPASoftInterface { init(libcamera.IPASettings settings, libcamera.SharedFD fdStats, - libcamera.SharedFD fdParams, + array fdParams, libcamera.ControlInfoMap sensorCtrlInfoMap) => (int32 ret); start() => (int32 ret); diff --git a/src/ipa/simple/soft_simple.cpp b/src/ipa/simple/soft_simple.cpp index 9d5fe923..50f8f194 100644 --- a/src/ipa/simple/soft_simple.cpp +++ b/src/ipa/simple/soft_simple.cpp @@ -48,7 +48,7 @@ public: int init(const IPASettings &settings, const SharedFD &fdStats, - const SharedFD &fdParams, + const std::vector &fdParams, const ControlInfoMap &sensorInfoMap) override; int configure(const IPAConfigInfo &configInfo) override; @@ -66,7 +66,7 @@ protected: private: void updateExposure(double exposureMSV); - DebayerParams *params_; + std::map paramsBuffers_; SwIspStats *stats_; std::unique_ptr camHelper_; ControlInfoMap sensorInfoMap_; @@ -79,13 +79,13 @@ IPASoftSimple::~IPASoftSimple() { if (stats_) munmap(stats_, sizeof(SwIspStats)); - if (params_) - munmap(params_, sizeof(DebayerParams)); + for (auto &item : paramsBuffers_) + munmap(item.second, sizeof(DebayerParams)); } int IPASoftSimple::init(const IPASettings &settings, const SharedFD &fdStats, - const SharedFD &fdParams, + const std::vector &fdParams, const ControlInfoMap &sensorInfoMap) { camHelper_ = CameraSensorHelperFactoryBase::create(settings.sensorModel); @@ -122,7 +122,6 @@ int IPASoftSimple::init(const IPASettings &settings, if (ret) return ret; - params_ = nullptr; stats_ = nullptr; if (!fdStats.isValid()) { @@ -130,20 +129,21 @@ int IPASoftSimple::init(const IPASettings &settings, return -ENODEV; } - if (!fdParams.isValid()) { - LOG(IPASoft, Error) << "Invalid Parameters handle"; - return -ENODEV; - } + for (auto &sharedFd : fdParams) { + if (!sharedFd.isValid()) { + LOG(IPASoft, Error) << "Invalid Parameters handle"; + return -ENODEV; + } - { void *mem = mmap(nullptr, sizeof(DebayerParams), PROT_WRITE, - MAP_SHARED, fdParams.get(), 0); + MAP_SHARED, sharedFd.get(), 0); if (mem == MAP_FAILED) { LOG(IPASoft, Error) << "Unable to map Parameters"; return -errno; } - params_ = static_cast(mem); + ASSERT(sharedFd.get() >= 0); + paramsBuffers_[sharedFd.get()] = static_cast(mem); } { @@ -258,8 +258,9 @@ void IPASoftSimple::prepare(const uint32_t frame, const uint32_t paramsBufferId) { IPAFrameContext &frameContext = context_.frameContexts.get(frame); + DebayerParams *params = paramsBuffers_.at(paramsBufferId); for (auto const &algo : algorithms()) - algo->prepare(context_, frame, frameContext, params_); + algo->prepare(context_, frame, frameContext, params); setIspParams.emit(paramsBufferId); } diff --git a/src/libcamera/software_isp/software_isp.cpp b/src/libcamera/software_isp/software_isp.cpp index d48be2d2..480ecf6e 100644 --- a/src/libcamera/software_isp/software_isp.cpp +++ b/src/libcamera/software_isp/software_isp.cpp @@ -13,6 +13,8 @@ #include #include +#include + #include #include #include @@ -85,6 +87,9 @@ SoftwareIsp::SoftwareIsp(PipelineHandler *pipe, const CameraSensor *sensor, stats->statsReady.connect(this, &SoftwareIsp::statsReady); debayer_ = std::make_unique(std::move(stats)); + std::vector fdParams; + for (auto &item : sharedParams_) + fdParams.emplace_back(item.second.fd()); debayer_->inputBufferReady.connect(this, &SoftwareIsp::inputReady); debayer_->outputBufferReady.connect(this, &SoftwareIsp::outputReady); debayer_->releaseIspParams.connect(this, &SoftwareIsp::releaseIspParams); @@ -106,7 +111,7 @@ SoftwareIsp::SoftwareIsp(PipelineHandler *pipe, const CameraSensor *sensor, int ret = ipa_->init(IPASettings{ ipaTuningFile, sensor->model() }, debayer_->getStatsFD(), - sharedParams_.begin()->second.fd(), + fdParams, sensor->controls()); if (ret) { LOG(SoftwareIsp, Error) << "IPA init failed"; @@ -380,13 +385,14 @@ void SoftwareIsp::process(uint32_t frame, FrameBuffer *input, FrameBuffer *outpu const uint32_t paramsBufferId = availableParams_.front(); availableParams_.pop(); ipa_->prepare(frame, paramsBufferId); + debayerParams_ = *sharedParams_.at(paramsBufferId); debayer_->invokeMethod(&DebayerCpu::process, ConnectionTypeQueued, frame, paramsBufferId, input, output, debayerParams_); } -void SoftwareIsp::saveIspParams([[maybe_unused]] uint32_t paramsBufferId) +void SoftwareIsp::saveIspParams(uint32_t paramsBufferId) { - debayerParams_ = *sharedParams_.begin()->second; + debayerParams_ = *sharedParams_.at(paramsBufferId); } void SoftwareIsp::releaseIspParams(uint32_t paramsBufferId) From patchwork Mon Aug 12 11:49:57 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Milan Zamazal X-Patchwork-Id: 20882 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 4E457C323E for ; Mon, 12 Aug 2024 11:50:41 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 830D5633D3; Mon, 12 Aug 2024 13:50:40 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.b="C8uoMFJj"; dkim-atps=neutral Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 3AE89633C0 for ; Mon, 12 Aug 2024 13:50:35 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1723463434; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=t/AeldxtUm1Ot/bNwPa6PVHmDctwluBTF6dzD6k2h+E=; b=C8uoMFJjlaBH60UIYHQSfRazK4YCsotzdEFNUxbf/QGBlCWmoMJRUVUG7fccxbPvZRbA2g hpGylGyPhl2HpmIey79Y0NvjhYa7LOc4RIZcF9bI3H1V3w+kS2Uys4xAwtKp49Z60hji99 xGibdz0w6ZqoCOv9Z609jFAKbWPs1l4= Received: from mx-prod-mc-04.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-379-ZxrDM5SsPFyxaCRQR9hprw-1; Mon, 12 Aug 2024 07:50:32 -0400 X-MC-Unique: ZxrDM5SsPFyxaCRQR9hprw-1 Received: from mx-prod-int-02.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-02.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.15]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-04.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 121C9191815B for ; Mon, 12 Aug 2024 11:50:32 +0000 (UTC) Received: from nuthatch.redhat.com (unknown [10.45.225.57]) by mx-prod-int-02.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id D164B19772C4; Mon, 12 Aug 2024 11:50:30 +0000 (UTC) From: Milan Zamazal To: libcamera-devel@lists.libcamera.org Cc: Milan Zamazal Subject: [PATCH 08/16] libcamera: software_isp: Share parameters buffers with debayering Date: Mon, 12 Aug 2024 13:49:57 +0200 Message-ID: <20240812115009.946036-9-mzamazal@redhat.com> In-Reply-To: <20240812115009.946036-1-mzamazal@redhat.com> References: <20240812115009.946036-1-mzamazal@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.0 on 10.30.177.15 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Share the multiple parameters buffers with debayering rather than copying the current parameters buffer there. This is done in a similar way as sharing the buffers with IPA in the preceding patch. The last missing step about parameters sharing is to avoid copying the color lookup arrays in debayering, which will be addressed in the next patch. Signed-off-by: Milan Zamazal --- src/libcamera/software_isp/debayer.cpp | 3 +- src/libcamera/software_isp/debayer.h | 3 +- src/libcamera/software_isp/debayer_cpu.cpp | 36 +++++++++++++++------ src/libcamera/software_isp/debayer_cpu.h | 7 ++-- src/libcamera/software_isp/software_isp.cpp | 6 ++-- 5 files changed, 37 insertions(+), 18 deletions(-) diff --git a/src/libcamera/software_isp/debayer.cpp b/src/libcamera/software_isp/debayer.cpp index 8bd58c3d..52df8c23 100644 --- a/src/libcamera/software_isp/debayer.cpp +++ b/src/libcamera/software_isp/debayer.cpp @@ -94,13 +94,12 @@ Debayer::~Debayer() */ /** - * \fn void Debayer::process(uint32_t frame, const uint32_t paramsBufferId, FrameBuffer *input, FrameBuffer *output, DebayerParams params) + * \fn void Debayer::process(uint32_t frame, const uint32_t paramsBufferId, FrameBuffer *input, FrameBuffer *output) * \brief Process the bayer data into the requested format * \param[in] frame The frame number * \param[in] paramsBufferId The id of the params buffer in use * \param[in] input The input buffer * \param[in] output The output buffer - * \param[in] params The parameters to be used in debayering * * \note DebayerParams is passed by value deliberately so that a copy is passed * when this is run in another thread by invokeMethod(). diff --git a/src/libcamera/software_isp/debayer.h b/src/libcamera/software_isp/debayer.h index 93020735..251b14fd 100644 --- a/src/libcamera/software_isp/debayer.h +++ b/src/libcamera/software_isp/debayer.h @@ -42,8 +42,7 @@ public: virtual void process(uint32_t frame, const uint32_t paramsBufferId, - FrameBuffer *input, FrameBuffer *output, - DebayerParams params) = 0; + FrameBuffer *input, FrameBuffer *output) = 0; virtual SizeRange sizes(PixelFormat inputFormat, const Size &inputSize) = 0; diff --git a/src/libcamera/software_isp/debayer_cpu.cpp b/src/libcamera/software_isp/debayer_cpu.cpp index 20a15ae0..af08ab10 100644 --- a/src/libcamera/software_isp/debayer_cpu.cpp +++ b/src/libcamera/software_isp/debayer_cpu.cpp @@ -11,9 +11,10 @@ #include "debayer_cpu.h" -#include #include +#include + #include #include "libcamera/internal/bayer_format.h" @@ -32,8 +33,10 @@ namespace libcamera { /** * \brief Constructs a DebayerCpu object * \param[in] stats Pointer to the stats object to use + * \param[in] paramBuffers SharedFDs of parameter buffers */ -DebayerCpu::DebayerCpu(std::unique_ptr stats) +DebayerCpu::DebayerCpu(std::unique_ptr stats, + const std::vector ¶mBuffers) : stats_(std::move(stats)) { /* @@ -49,6 +52,20 @@ DebayerCpu::DebayerCpu(std::unique_ptr stats) /* Initialize color lookup tables */ for (unsigned int i = 0; i < DebayerParams::kRGBLookupSize; i++) red_[i] = green_[i] = blue_[i] = i; + + paramsBuffers_ = std::map(); + + for (auto &sharedFd : paramBuffers) { + void *mem = mmap(nullptr, sizeof(DebayerParams), PROT_WRITE, + MAP_SHARED, sharedFd.get(), 0); + if (mem == MAP_FAILED) { + LOG(Debayer, Error) << "Unable to map Parameters"; + return; + } + + ASSERT(sharedFd.get() >= 0); + paramsBuffers_[sharedFd.get()] = static_cast(mem); + } } DebayerCpu::~DebayerCpu() = default; @@ -608,8 +625,7 @@ void DebayerCpu::memcpyNextLine(const uint8_t *linePointers[]) memcpy(lineBuffers_[lineBufferIndex_].data(), linePointers[patternHeight] - lineBufferPadding_, lineBufferLength_); - linePointers[patternHeight] = lineBuffers_[lineBufferIndex_].data() - + lineBufferPadding_; + linePointers[patternHeight] = lineBuffers_[lineBufferIndex_].data() + lineBufferPadding_; lineBufferIndex_ = (lineBufferIndex_ + 1) % (patternHeight + 1); } @@ -726,7 +742,7 @@ static inline int64_t timeDiff(timespec &after, timespec &before) void DebayerCpu::process(uint32_t frame, const uint32_t paramsBufferId, - FrameBuffer *input, FrameBuffer *output, DebayerParams params) + FrameBuffer *input, FrameBuffer *output) { timespec frameStartTime; @@ -735,12 +751,14 @@ void DebayerCpu::process(uint32_t frame, clock_gettime(CLOCK_MONOTONIC_RAW, &frameStartTime); } - green_ = params.green; - red_ = swapRedBlueGains_ ? params.blue : params.red; - blue_ = swapRedBlueGains_ ? params.red : params.blue; - + DebayerParams *params = paramsBuffers_.at(paramsBufferId); releaseIspParams.emit(paramsBufferId); + /* \todo Avoid copying here. */ + green_ = params->green; + red_ = swapRedBlueGains_ ? params->blue : params->red; + blue_ = swapRedBlueGains_ ? params->red : params->blue; + /* Copy metadata from the input buffer */ FrameMetadata &metadata = output->_d()->metadata(); metadata.status = input->metadata().status; diff --git a/src/libcamera/software_isp/debayer_cpu.h b/src/libcamera/software_isp/debayer_cpu.h index 0b5a5258..324fc6c0 100644 --- a/src/libcamera/software_isp/debayer_cpu.h +++ b/src/libcamera/software_isp/debayer_cpu.h @@ -16,6 +16,7 @@ #include #include +#include #include "libcamera/internal/bayer_format.h" @@ -27,7 +28,8 @@ namespace libcamera { class DebayerCpu : public Debayer, public Object { public: - DebayerCpu(std::unique_ptr stats); + DebayerCpu(std::unique_ptr stats, + const std::vector ¶mBuffers); ~DebayerCpu(); int configure(const StreamConfiguration &inputCfg, @@ -38,7 +40,7 @@ public: strideAndFrameSize(const PixelFormat &outputFormat, const Size &size); void process(uint32_t frame, const uint32_t paramsBufferId, - FrameBuffer *input, FrameBuffer *output, DebayerParams params); + FrameBuffer *input, FrameBuffer *output); SizeRange sizes(PixelFormat inputFormat, const Size &inputSize); /** @@ -160,6 +162,7 @@ private: /* Skip 30 frames for things to stabilize then measure 30 frames */ static constexpr unsigned int kFramesToSkip = 30; static constexpr unsigned int kLastFrameToMeasure = 60; + std::map paramsBuffers_; }; } /* namespace libcamera */ diff --git a/src/libcamera/software_isp/software_isp.cpp b/src/libcamera/software_isp/software_isp.cpp index 480ecf6e..db77f6f9 100644 --- a/src/libcamera/software_isp/software_isp.cpp +++ b/src/libcamera/software_isp/software_isp.cpp @@ -86,10 +86,11 @@ SoftwareIsp::SoftwareIsp(PipelineHandler *pipe, const CameraSensor *sensor, } stats->statsReady.connect(this, &SoftwareIsp::statsReady); - debayer_ = std::make_unique(std::move(stats)); std::vector fdParams; for (auto &item : sharedParams_) fdParams.emplace_back(item.second.fd()); + + debayer_ = std::make_unique(std::move(stats), fdParams); debayer_->inputBufferReady.connect(this, &SoftwareIsp::inputReady); debayer_->outputBufferReady.connect(this, &SoftwareIsp::outputReady); debayer_->releaseIspParams.connect(this, &SoftwareIsp::releaseIspParams); @@ -385,9 +386,8 @@ void SoftwareIsp::process(uint32_t frame, FrameBuffer *input, FrameBuffer *outpu const uint32_t paramsBufferId = availableParams_.front(); availableParams_.pop(); ipa_->prepare(frame, paramsBufferId); - debayerParams_ = *sharedParams_.at(paramsBufferId); debayer_->invokeMethod(&DebayerCpu::process, - ConnectionTypeQueued, frame, paramsBufferId, input, output, debayerParams_); + ConnectionTypeQueued, frame, paramsBufferId, input, output); } void SoftwareIsp::saveIspParams(uint32_t paramsBufferId) From patchwork Mon Aug 12 11:49:58 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Milan Zamazal X-Patchwork-Id: 20883 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 3844CC323E for ; Mon, 12 Aug 2024 11:50:44 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id D562B633D3; Mon, 12 Aug 2024 13:50:43 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.b="axPFnCBr"; dkim-atps=neutral Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 2FA9F633CC for ; Mon, 12 Aug 2024 13:50:36 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1723463435; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=LGTinOMIdh/wLBfcPnfMvqOVU/mkZ44ix0MPiPqx12o=; b=axPFnCBr4DuxGAi3CA/TeuBfIxXTKBwKDjDFu40dw8XE54dd8x42vFZGs9rpYbVjEsm686 FhlInMqYoEDmznBzrjeeuJtRN+0/FBEXziF2DYCYmnIK8BrPqtNPI+ee6LRYcySxSAPNox Cybxaxz2p1QGEv7la83g/f1IKulTE/w= Received: from mx-prod-mc-05.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-616-Lo5Ug71uOtyF-Wrxvk8EEg-1; Mon, 12 Aug 2024 07:50:34 -0400 X-MC-Unique: Lo5Ug71uOtyF-Wrxvk8EEg-1 Received: from mx-prod-int-02.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-02.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.15]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-05.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 786141954211 for ; Mon, 12 Aug 2024 11:50:33 +0000 (UTC) Received: from nuthatch.redhat.com (unknown [10.45.225.57]) by mx-prod-int-02.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id 600E019772C4; Mon, 12 Aug 2024 11:50:32 +0000 (UTC) From: Milan Zamazal To: libcamera-devel@lists.libcamera.org Cc: Milan Zamazal Subject: [PATCH 09/16] libcamera: software_isp: Avoid copying particular color tables Date: Mon, 12 Aug 2024 13:49:58 +0200 Message-ID: <20240812115009.946036-10-mzamazal@redhat.com> In-Reply-To: <20240812115009.946036-1-mzamazal@redhat.com> References: <20240812115009.946036-1-mzamazal@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.0 on 10.30.177.15 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" The color lookup tables in software CPU debayering are currently copied from the provided parameters. The preceding work avoided copying the parameters when passed to debayering, this patch avoids the copying inside debayering. Signed-off-by: Milan Zamazal --- src/libcamera/software_isp/debayer_cpu.cpp | 60 +++++++++++----------- src/libcamera/software_isp/debayer_cpu.h | 7 +-- 2 files changed, 34 insertions(+), 33 deletions(-) diff --git a/src/libcamera/software_isp/debayer_cpu.cpp b/src/libcamera/software_isp/debayer_cpu.cpp index af08ab10..d3831474 100644 --- a/src/libcamera/software_isp/debayer_cpu.cpp +++ b/src/libcamera/software_isp/debayer_cpu.cpp @@ -51,7 +51,8 @@ DebayerCpu::DebayerCpu(std::unique_ptr stats, /* Initialize color lookup tables */ for (unsigned int i = 0; i < DebayerParams::kRGBLookupSize; i++) - red_[i] = green_[i] = blue_[i] = i; + initColorLookupTable_[i] = i; + red_ = green_ = blue_ = &initColorLookupTable_; paramsBuffers_ = std::map(); @@ -80,12 +81,12 @@ DebayerCpu::~DebayerCpu() = default; * GBG * RGR */ -#define BGGR_BGR888(p, n, div) \ - *dst++ = blue_[curr[x] / (div)]; \ - *dst++ = green_[(prev[x] + curr[x - p] + curr[x + n] + next[x]) / (4 * (div))]; \ - *dst++ = red_[(prev[x - p] + prev[x + n] + next[x - p] + next[x + n]) / (4 * (div))]; \ - if constexpr (addAlphaByte) \ - *dst++ = 255; \ +#define BGGR_BGR888(p, n, div) \ + *dst++ = (*blue_)[curr[x] / (div)]; \ + *dst++ = (*green_)[(prev[x] + curr[x - p] + curr[x + n] + next[x]) / (4 * (div))]; \ + *dst++ = (*red_)[(prev[x - p] + prev[x + n] + next[x - p] + next[x + n]) / (4 * (div))]; \ + if constexpr (addAlphaByte) \ + *dst++ = 255; \ x++; /* @@ -93,12 +94,12 @@ DebayerCpu::~DebayerCpu() = default; * RGR * GBG */ -#define GRBG_BGR888(p, n, div) \ - *dst++ = blue_[(prev[x] + next[x]) / (2 * (div))]; \ - *dst++ = green_[curr[x] / (div)]; \ - *dst++ = red_[(curr[x - p] + curr[x + n]) / (2 * (div))]; \ - if constexpr (addAlphaByte) \ - *dst++ = 255; \ +#define GRBG_BGR888(p, n, div) \ + *dst++ = (*blue_)[(prev[x] + next[x]) / (2 * (div))]; \ + *dst++ = (*green_)[curr[x] / (div)]; \ + *dst++ = (*red_)[(curr[x - p] + curr[x + n]) / (2 * (div))]; \ + if constexpr (addAlphaByte) \ + *dst++ = 255; \ x++; /* @@ -106,12 +107,12 @@ DebayerCpu::~DebayerCpu() = default; * BGB * GRG */ -#define GBRG_BGR888(p, n, div) \ - *dst++ = blue_[(curr[x - p] + curr[x + n]) / (2 * (div))]; \ - *dst++ = green_[curr[x] / (div)]; \ - *dst++ = red_[(prev[x] + next[x]) / (2 * (div))]; \ - if constexpr (addAlphaByte) \ - *dst++ = 255; \ +#define GBRG_BGR888(p, n, div) \ + *dst++ = (*blue_)[(curr[x - p] + curr[x + n]) / (2 * (div))]; \ + *dst++ = (*green_)[curr[x] / (div)]; \ + *dst++ = (*red_)[(prev[x] + next[x]) / (2 * (div))]; \ + if constexpr (addAlphaByte) \ + *dst++ = 255; \ x++; /* @@ -119,12 +120,12 @@ DebayerCpu::~DebayerCpu() = default; * GRG * BGB */ -#define RGGB_BGR888(p, n, div) \ - *dst++ = blue_[(prev[x - p] + prev[x + n] + next[x - p] + next[x + n]) / (4 * (div))]; \ - *dst++ = green_[(prev[x] + curr[x - p] + curr[x + n] + next[x]) / (4 * (div))]; \ - *dst++ = red_[curr[x] / (div)]; \ - if constexpr (addAlphaByte) \ - *dst++ = 255; \ +#define RGGB_BGR888(p, n, div) \ + *dst++ = (*blue_)[(prev[x - p] + prev[x + n] + next[x - p] + next[x + n]) / (4 * (div))]; \ + *dst++ = (*green_)[(prev[x] + curr[x - p] + curr[x + n] + next[x]) / (4 * (div))]; \ + *dst++ = (*red_)[curr[x] / (div)]; \ + if constexpr (addAlphaByte) \ + *dst++ = 255; \ x++; template @@ -752,12 +753,10 @@ void DebayerCpu::process(uint32_t frame, } DebayerParams *params = paramsBuffers_.at(paramsBufferId); - releaseIspParams.emit(paramsBufferId); - /* \todo Avoid copying here. */ - green_ = params->green; - red_ = swapRedBlueGains_ ? params->blue : params->red; - blue_ = swapRedBlueGains_ ? params->red : params->blue; + green_ = ¶ms->green; + red_ = swapRedBlueGains_ ? ¶ms->blue : ¶ms->red; + blue_ = swapRedBlueGains_ ? ¶ms->red : ¶ms->blue; /* Copy metadata from the input buffer */ FrameMetadata &metadata = output->_d()->metadata(); @@ -779,6 +778,7 @@ void DebayerCpu::process(uint32_t frame, process2(in.planes()[0].data(), out.planes()[0].data()); else process4(in.planes()[0].data(), out.planes()[0].data()); + releaseIspParams.emit(paramsBufferId); metadata.planes()[0].bytesused = out.planes()[0].size(); diff --git a/src/libcamera/software_isp/debayer_cpu.h b/src/libcamera/software_isp/debayer_cpu.h index 324fc6c0..f25520be 100644 --- a/src/libcamera/software_isp/debayer_cpu.h +++ b/src/libcamera/software_isp/debayer_cpu.h @@ -139,9 +139,10 @@ private: /* Max. supported Bayer pattern height is 4, debayering this requires 5 lines */ static constexpr unsigned int kMaxLineBuffers = 5; - DebayerParams::ColorLookupTable red_; - DebayerParams::ColorLookupTable green_; - DebayerParams::ColorLookupTable blue_; + DebayerParams::ColorLookupTable initColorLookupTable_; + DebayerParams::ColorLookupTable *red_; + DebayerParams::ColorLookupTable *green_; + DebayerParams::ColorLookupTable *blue_; debayerFn debayer0_; debayerFn debayer1_; debayerFn debayer2_; From patchwork Mon Aug 12 11:49:59 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Milan Zamazal X-Patchwork-Id: 20884 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id C43E9C323E for ; Mon, 12 Aug 2024 11:50:46 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 55F0A633C9; Mon, 12 Aug 2024 13:50:46 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.b="Few57IY7"; dkim-atps=neutral Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 60987633C8 for ; Mon, 12 Aug 2024 13:50:38 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1723463437; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=6fhFLPfLnFUhbehU4Z4FuXIeVjmixZwW2IeYtY9/7PA=; b=Few57IY7KsiAGznVMFUEESWBhPrE9wJTzSTn/lAIzsnnzgEmwRUIj4/3EJoQZk5Rah2Gzf lmQ8OH9WTLQaCkGxfGo/fdamRhpYUwCrfd3ocgRuQowTaNL3jWmLBVeIHT6HFCGgfkZlVi PGhE1OkW34Tw25zyOJteHKCFK77zw9w= Received: from mx-prod-mc-05.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-626-k1XUCoR8NMGJfZcBtsFKrg-1; Mon, 12 Aug 2024 07:50:35 -0400 X-MC-Unique: k1XUCoR8NMGJfZcBtsFKrg-1 Received: from mx-prod-int-02.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-02.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.15]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-05.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 1B7851955F3B for ; Mon, 12 Aug 2024 11:50:35 +0000 (UTC) Received: from nuthatch.redhat.com (unknown [10.45.225.57]) by mx-prod-int-02.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id E2C2A19772C4; Mon, 12 Aug 2024 11:50:33 +0000 (UTC) From: Milan Zamazal To: libcamera-devel@lists.libcamera.org Cc: Milan Zamazal Subject: [PATCH 10/16] libcamera: software_isp: Remove per-frame params buffers TODO item Date: Mon, 12 Aug 2024 13:49:59 +0200 Message-ID: <20240812115009.946036-11-mzamazal@redhat.com> In-Reply-To: <20240812115009.946036-1-mzamazal@redhat.com> References: <20240812115009.946036-1-mzamazal@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.0 on 10.30.177.15 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Parameters are stored in per-frame buffers and shared properly now. Signed-off-by: Milan Zamazal --- src/libcamera/software_isp/TODO | 18 ------------------ 1 file changed, 18 deletions(-) diff --git a/src/libcamera/software_isp/TODO b/src/libcamera/software_isp/TODO index a50db668..db1115dd 100644 --- a/src/libcamera/software_isp/TODO +++ b/src/libcamera/software_isp/TODO @@ -53,24 +53,6 @@ stats in hardware, such as the i.MX7), but please keep it on your radar. --- -5. Store ISP parameters in per-frame buffers - -> /** -> * \fn void Debayer::process(FrameBuffer *input, FrameBuffer *output, DebayerParams params) -> * \brief Process the bayer data into the requested format. -> * \param[in] input The input buffer. -> * \param[in] output The output buffer. -> * \param[in] params The parameters to be used in debayering. -> * -> * \note DebayerParams is passed by value deliberately so that a copy is passed -> * when this is run in another thread by invokeMethod(). -> */ - -Possibly something to address later, by storing ISP parameters in -per-frame buffers like we do for hardware ISPs. - ---- - 6. Input buffer copying configuration > DebayerCpu::DebayerCpu(std::unique_ptr stats) From patchwork Mon Aug 12 11:50:00 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Milan Zamazal X-Patchwork-Id: 20885 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 95EA8C323E for ; Mon, 12 Aug 2024 11:50:48 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 1BE24633D0; Mon, 12 Aug 2024 13:50:48 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.b="P8GKzGex"; dkim-atps=neutral Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id BF376633CB for ; Mon, 12 Aug 2024 13:50:39 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1723463438; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=nXpdYdYDgTLzmjsGrOK6UMJvZOH4lJe3pR2X6EUIf64=; b=P8GKzGexd9wdN91k3zDkwzs0jMR+OD5BgyxNk3r1hsKoD25zIriMiFoyB27DnW8DQ18vOy vURutTes0l74+bT3D9cDi2DQN3lJP0xlvVvErv+rnXUNFVQP3JhdOwVSoTGGPnAwgNFzlg jx2smwhejLtrL+FqeO4/X+SPtkK4Hd4= Received: from mx-prod-mc-01.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-217-u_nnMEdzMQ6C13bxiyuhPA-1; Mon, 12 Aug 2024 07:50:37 -0400 X-MC-Unique: u_nnMEdzMQ6C13bxiyuhPA-1 Received: from mx-prod-int-02.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-02.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.15]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-01.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 94BBB19541A9 for ; Mon, 12 Aug 2024 11:50:36 +0000 (UTC) Received: from nuthatch.redhat.com (unknown [10.45.225.57]) by mx-prod-int-02.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id 7AAE819772C4; Mon, 12 Aug 2024 11:50:35 +0000 (UTC) From: Milan Zamazal To: libcamera-devel@lists.libcamera.org Cc: Milan Zamazal Subject: [PATCH 11/16] libcamera: software_isp: Drop a comment about DelayedControls Date: Mon, 12 Aug 2024 13:50:00 +0200 Message-ID: <20240812115009.946036-12-mzamazal@redhat.com> In-Reply-To: <20240812115009.946036-1-mzamazal@redhat.com> References: <20240812115009.946036-1-mzamazal@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.0 on 10.30.177.15 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" The comment is obsolete, that has already been done. Signed-off-by: Milan Zamazal --- src/libcamera/pipeline/simple/simple.cpp | 1 - 1 file changed, 1 deletion(-) diff --git a/src/libcamera/pipeline/simple/simple.cpp b/src/libcamera/pipeline/simple/simple.cpp index 459f617b..ebead8f0 100644 --- a/src/libcamera/pipeline/simple/simple.cpp +++ b/src/libcamera/pipeline/simple/simple.cpp @@ -903,7 +903,6 @@ void SimpleCameraData::conversionOutputDone(FrameBuffer *buffer) void SimpleCameraData::ispStatsReady(uint32_t frame, uint32_t bufferId) { - /* \todo Use the DelayedControls class */ swIsp_->processStats( frame, bufferId, delayedCtrls_->get(frame)); From patchwork Mon Aug 12 11:50:01 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Milan Zamazal X-Patchwork-Id: 20886 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 77184C323E for ; Mon, 12 Aug 2024 11:50:50 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id D7DF6633C9; Mon, 12 Aug 2024 13:50:49 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.b="DesCpda1"; dkim-atps=neutral Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id D718C633C9 for ; Mon, 12 Aug 2024 13:50:41 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1723463440; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Jinnvnu8XCLnfQ5GOxGmcrGvFsqD0Ia5DcXJl/WHwzA=; b=DesCpda1wOFxP32gfFL6/j13iggmpd12allpaY3TTrbejwlXSNeyU3rfpAE5touGp0OGT2 MTablHBJ+unZWO3YDkR3BzotRZtJ058psbMUcfS9XtWrd39It4WywowGLjFsLMZBHv2qAV xDurpISEPEwcpCIEHwJ7tAIuSpKZAn0= Received: from mx-prod-mc-01.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-241-iPoelU1yMNqJKMOUGTwD9A-1; Mon, 12 Aug 2024 07:50:38 -0400 X-MC-Unique: iPoelU1yMNqJKMOUGTwD9A-1 Received: from mx-prod-int-02.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-02.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.15]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-01.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 352BD195420E for ; Mon, 12 Aug 2024 11:50:38 +0000 (UTC) Received: from nuthatch.redhat.com (unknown [10.45.225.57]) by mx-prod-int-02.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id 0D4AB19772C4; Mon, 12 Aug 2024 11:50:36 +0000 (UTC) From: Milan Zamazal To: libcamera-devel@lists.libcamera.org Cc: Milan Zamazal Subject: [PATCH 12/16] libcamera: software_isp: Introduce arguments for statistics buffers Date: Mon, 12 Aug 2024 13:50:01 +0200 Message-ID: <20240812115009.946036-13-mzamazal@redhat.com> In-Reply-To: <20240812115009.946036-1-mzamazal@redhat.com> References: <20240812115009.946036-1-mzamazal@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.0 on 10.30.177.15 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Statistics in software ISP is held in two fixed buffers, copying the statistics from the working one to the shared one in SwIspStats::finishFrame. This patch introduces support for a ring of statistics buffers that are never copied, similarly to parameters buffers and to how hardware pipelines operate. This is to address software ISP TODO #2. The patch adds a new argument statsBufferId for the future parameters buffer ids passed to the calls. The buffer ids must be passed to the following groups of methods: - SwStatsCpu::startFrame, to reset the statistics. - Debayer::process, to signal what statistics buffer is started and stopped being used to gather the statistics. - SoftwareIsp::statsProcessed, a new signal to indicate that the given statistics buffer contents is no longer needed. The type of the buffer id statistics is set to uint32_t because: - It can be used in mojom. - The same type is used for parameters buffers. - It is consistent with the similar types in the hardware pipelines. - It covers file descriptor number range, which will be used as buffer ids. Already provided statistics buffer arguments are renamed from bufferId to statsBufferId for consistency and to avoid confusion between statistics and parameters buffer id arguments. This patch doesn't do more than adding the arguments, to keep the patch simple. The buffer handling will be implemented in the followup patches. Statistics and parameters buffers use is coupled to much extent. It can be tempting to perhaps use the same buffer ids and perhaps some common signals. But it's better to keep them separate to avoid contingent future refactoring in case this coupling becomes less tight in future. Signed-off-by: Milan Zamazal --- .../libcamera/internal/software_isp/software_isp.h | 3 ++- include/libcamera/ipa/soft.mojom | 3 ++- src/ipa/simple/soft_simple.cpp | 3 ++- src/libcamera/pipeline/simple/simple.cpp | 7 ++++--- src/libcamera/software_isp/debayer.cpp | 3 ++- src/libcamera/software_isp/debayer.h | 2 +- src/libcamera/software_isp/debayer_cpu.cpp | 11 +++-------- src/libcamera/software_isp/debayer_cpu.h | 2 +- src/libcamera/software_isp/software_isp.cpp | 13 ++++++++++--- src/libcamera/software_isp/swstats_cpu.cpp | 6 +++--- src/libcamera/software_isp/swstats_cpu.h | 4 ++-- 11 files changed, 32 insertions(+), 25 deletions(-) diff --git a/include/libcamera/internal/software_isp/software_isp.h b/include/libcamera/internal/software_isp/software_isp.h index 8956978c..ae64b247 100644 --- a/include/libcamera/internal/software_isp/software_isp.h +++ b/include/libcamera/internal/software_isp/software_isp.h @@ -89,7 +89,8 @@ private: void saveIspParams(uint32_t paramsBufferId); void releaseIspParams(uint32_t paramsBufferId); void setSensorCtrls(const ControlList &sensorControls); - void statsReady(uint32_t frame, uint32_t bufferId); + void statsReady(uint32_t frame, const uint32_t statsBufferId); + void statsProcessed(const uint32_t statsBufferId); void inputReady(FrameBuffer *input); void outputReady(FrameBuffer *output); diff --git a/include/libcamera/ipa/soft.mojom b/include/libcamera/ipa/soft.mojom index 86dd988f..7b85c454 100644 --- a/include/libcamera/ipa/soft.mojom +++ b/include/libcamera/ipa/soft.mojom @@ -25,10 +25,11 @@ interface IPASoftInterface { [async] queueRequest(uint32 frame, libcamera.ControlList sensorControls); prepare(uint32 frame, uint32 paramsBufferId); - [async] processStats(uint32 frame, uint32 bufferId, libcamera.ControlList sensorControls); + [async] processStats(uint32 frame, uint32 statsBufferId, libcamera.ControlList sensorControls); }; interface IPASoftEventInterface { setSensorControls(libcamera.ControlList sensorControls); + statsProcessed(uint32 statsBufferId); setIspParams(uint32 paramsBufferId); }; diff --git a/src/ipa/simple/soft_simple.cpp b/src/ipa/simple/soft_simple.cpp index 50f8f194..3c95c4d9 100644 --- a/src/ipa/simple/soft_simple.cpp +++ b/src/ipa/simple/soft_simple.cpp @@ -266,7 +266,7 @@ void IPASoftSimple::prepare(const uint32_t frame, void IPASoftSimple::processStats( const uint32_t frame, - [[maybe_unused]] const uint32_t bufferId, + const uint32_t statsBufferId, const ControlList &sensorControls) { IPAFrameContext &frameContext = context_.frameContexts.get(frame); @@ -285,6 +285,7 @@ void IPASoftSimple::processStats( ControlList metadata(controls::controls); for (auto const &algo : algorithms()) algo->process(context_, frame, frameContext, stats_, metadata); + statsProcessed.emit(statsBufferId); /* Sanity check */ if (!sensorControls.contains(V4L2_CID_EXPOSURE) || diff --git a/src/libcamera/pipeline/simple/simple.cpp b/src/libcamera/pipeline/simple/simple.cpp index ebead8f0..d9791ab1 100644 --- a/src/libcamera/pipeline/simple/simple.cpp +++ b/src/libcamera/pipeline/simple/simple.cpp @@ -295,7 +295,7 @@ private: void conversionInputDone(FrameBuffer *buffer); void conversionOutputDone(FrameBuffer *buffer); - void ispStatsReady(uint32_t frame, uint32_t bufferId); + void ispStatsReady(uint32_t frame, const uint32_t statsBufferId); void setSensorControls(const ControlList &sensorControls); }; @@ -901,10 +901,11 @@ void SimpleCameraData::conversionOutputDone(FrameBuffer *buffer) pipe->completeRequest(request); } -void SimpleCameraData::ispStatsReady(uint32_t frame, uint32_t bufferId) +void SimpleCameraData::ispStatsReady(uint32_t frame, + const uint32_t statsBufferId) { swIsp_->processStats( - frame, bufferId, + frame, statsBufferId, delayedCtrls_->get(frame)); } diff --git a/src/libcamera/software_isp/debayer.cpp b/src/libcamera/software_isp/debayer.cpp index 52df8c23..af54bb18 100644 --- a/src/libcamera/software_isp/debayer.cpp +++ b/src/libcamera/software_isp/debayer.cpp @@ -94,9 +94,10 @@ Debayer::~Debayer() */ /** - * \fn void Debayer::process(uint32_t frame, const uint32_t paramsBufferId, FrameBuffer *input, FrameBuffer *output) + * \fn void Debayer::process(uint32_t frame, const uint32_t statsBufferId, const uint32_t paramsBufferId, FrameBuffer *input, FrameBuffer *output) * \brief Process the bayer data into the requested format * \param[in] frame The frame number + * \param[in] statsBufferId The id of the stats buffer to use * \param[in] paramsBufferId The id of the params buffer in use * \param[in] input The input buffer * \param[in] output The output buffer diff --git a/src/libcamera/software_isp/debayer.h b/src/libcamera/software_isp/debayer.h index 251b14fd..0be7c59a 100644 --- a/src/libcamera/software_isp/debayer.h +++ b/src/libcamera/software_isp/debayer.h @@ -41,7 +41,7 @@ public: strideAndFrameSize(const PixelFormat &outputFormat, const Size &size) = 0; virtual void process(uint32_t frame, - const uint32_t paramsBufferId, + const uint32_t statsBufferId, const uint32_t paramsBufferId, FrameBuffer *input, FrameBuffer *output) = 0; virtual SizeRange sizes(PixelFormat inputFormat, const Size &inputSize) = 0; diff --git a/src/libcamera/software_isp/debayer_cpu.cpp b/src/libcamera/software_isp/debayer_cpu.cpp index d3831474..14bd58b5 100644 --- a/src/libcamera/software_isp/debayer_cpu.cpp +++ b/src/libcamera/software_isp/debayer_cpu.cpp @@ -742,7 +742,7 @@ static inline int64_t timeDiff(timespec &after, timespec &before) } void DebayerCpu::process(uint32_t frame, - const uint32_t paramsBufferId, + const uint32_t statsBufferId, const uint32_t paramsBufferId, FrameBuffer *input, FrameBuffer *output) { timespec frameStartTime; @@ -772,7 +772,7 @@ void DebayerCpu::process(uint32_t frame, return; } - stats_->startFrame(); + stats_->startFrame(statsBufferId); if (inputConfig_.patternSize.height == 2) process2(in.planes()[0].data(), out.planes()[0].data()); @@ -799,12 +799,7 @@ void DebayerCpu::process(uint32_t frame, } } - /* - * Buffer ids are currently not used, so pass zeros as its parameter. - * - * \todo Pass real bufferId once stats buffer passing is changed. - */ - stats_->finishFrame(frame, 0); + stats_->finishFrame(frame, statsBufferId); outputBufferReady.emit(output); inputBufferReady.emit(input); } diff --git a/src/libcamera/software_isp/debayer_cpu.h b/src/libcamera/software_isp/debayer_cpu.h index f25520be..7fb399b0 100644 --- a/src/libcamera/software_isp/debayer_cpu.h +++ b/src/libcamera/software_isp/debayer_cpu.h @@ -39,7 +39,7 @@ public: std::tuple strideAndFrameSize(const PixelFormat &outputFormat, const Size &size); void process(uint32_t frame, - const uint32_t paramsBufferId, + const uint32_t statsBufferId, const uint32_t paramsBufferId, FrameBuffer *input, FrameBuffer *output); SizeRange sizes(PixelFormat inputFormat, const Size &inputSize); diff --git a/src/libcamera/software_isp/software_isp.cpp b/src/libcamera/software_isp/software_isp.cpp index db77f6f9..5d9f5008 100644 --- a/src/libcamera/software_isp/software_isp.cpp +++ b/src/libcamera/software_isp/software_isp.cpp @@ -120,6 +120,7 @@ SoftwareIsp::SoftwareIsp(PipelineHandler *pipe, const CameraSensor *sensor, return; } + ipa_->statsProcessed.connect(this, &SoftwareIsp::statsProcessed); ipa_->setIspParams.connect(this, &SoftwareIsp::saveIspParams); ipa_->setSensorControls.connect(this, &SoftwareIsp::setSensorCtrls); @@ -386,8 +387,10 @@ void SoftwareIsp::process(uint32_t frame, FrameBuffer *input, FrameBuffer *outpu const uint32_t paramsBufferId = availableParams_.front(); availableParams_.pop(); ipa_->prepare(frame, paramsBufferId); + const uint32_t statsBufferId = 0; debayer_->invokeMethod(&DebayerCpu::process, - ConnectionTypeQueued, frame, paramsBufferId, input, output); + ConnectionTypeQueued, frame, + statsBufferId, paramsBufferId, input, output); } void SoftwareIsp::saveIspParams(uint32_t paramsBufferId) @@ -405,9 +408,13 @@ void SoftwareIsp::setSensorCtrls(const ControlList &sensorControls) setSensorControls.emit(sensorControls); } -void SoftwareIsp::statsReady(uint32_t frame, uint32_t bufferId) +void SoftwareIsp::statsReady(uint32_t frame, const uint32_t statsBufferId) +{ + ispStatsReady.emit(frame, statsBufferId); +} + +void SoftwareIsp::statsProcessed([[maybe_unused]] const uint32_t statsBufferId) { - ispStatsReady.emit(frame, bufferId); } void SoftwareIsp::inputReady(FrameBuffer *input) diff --git a/src/libcamera/software_isp/swstats_cpu.cpp b/src/libcamera/software_isp/swstats_cpu.cpp index a9b1f35a..b8ee06a0 100644 --- a/src/libcamera/software_isp/swstats_cpu.cpp +++ b/src/libcamera/software_isp/swstats_cpu.cpp @@ -298,7 +298,7 @@ void SwStatsCpu::statsGBRG10PLine0(const uint8_t *src[]) * * This may only be called after a successful setWindow() call. */ -void SwStatsCpu::startFrame(void) +void SwStatsCpu::startFrame([[maybe_unused]] const uint32_t statsBufferId) { if (window_.width == 0) LOG(SwStatsCpu, Error) << "Calling startFrame() without setWindow()"; @@ -314,10 +314,10 @@ void SwStatsCpu::startFrame(void) * * This may only be called after a successful setWindow() call. */ -void SwStatsCpu::finishFrame(uint32_t frame, uint32_t bufferId) +void SwStatsCpu::finishFrame(uint32_t frame, const uint32_t statsBufferId) { *sharedStats_ = stats_; - statsReady.emit(frame, bufferId); + statsReady.emit(frame, statsBufferId); } /** diff --git a/src/libcamera/software_isp/swstats_cpu.h b/src/libcamera/software_isp/swstats_cpu.h index 26a2f462..402eef2d 100644 --- a/src/libcamera/software_isp/swstats_cpu.h +++ b/src/libcamera/software_isp/swstats_cpu.h @@ -40,8 +40,8 @@ public: int configure(const StreamConfiguration &inputCfg); void setWindow(const Rectangle &window); - void startFrame(); - void finishFrame(uint32_t frame, uint32_t bufferId); + void startFrame(const uint32_t statsBufferId); + void finishFrame(uint32_t frame, const uint32_t statsBufferId); void processLine0(unsigned int y, const uint8_t *src[]) { From patchwork Mon Aug 12 11:50:02 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Milan Zamazal X-Patchwork-Id: 20887 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 0AACBC324E for ; Mon, 12 Aug 2024 11:50:52 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 7E6B8633D9; Mon, 12 Aug 2024 13:50:51 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.b="bA/RPFGE"; dkim-atps=neutral Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 9C786633C7 for ; Mon, 12 Aug 2024 13:50:42 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1723463441; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=RYkEQqU4OQMZuH1F9JMEKFSiqPXzGv1mheEG5il1IjA=; b=bA/RPFGEW4kEj1hnjWZ7tcLKJv848N0Nra7hYIv6T0DrSN1DAnEK7ksKKmKareILZ/CUD+ BY1vmbjIFxGCmCA7aGt52Ss686Zrg3NTIAdoM4zcipNagZ1PBgTv7gWztLiU9fVPSJ1Y0E 5dqhifovCbqTKC0DmHwwBLkgcwf7eTg= Received: from mx-prod-mc-05.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-657-sN3QjctLOk6o-JQBfG5abA-1; Mon, 12 Aug 2024 07:50:40 -0400 X-MC-Unique: sN3QjctLOk6o-JQBfG5abA-1 Received: from mx-prod-int-02.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-02.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.15]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-05.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 9749B1955F3D for ; Mon, 12 Aug 2024 11:50:39 +0000 (UTC) Received: from nuthatch.redhat.com (unknown [10.45.225.57]) by mx-prod-int-02.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id 8B9C419772C4; Mon, 12 Aug 2024 11:50:38 +0000 (UTC) From: Milan Zamazal To: libcamera-devel@lists.libcamera.org Cc: Milan Zamazal Subject: [PATCH 13/16] libcamera: software_isp: Allocate statistics buffers Date: Mon, 12 Aug 2024 13:50:02 +0200 Message-ID: <20240812115009.946036-14-mzamazal@redhat.com> In-Reply-To: <20240812115009.946036-1-mzamazal@redhat.com> References: <20240812115009.946036-1-mzamazal@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.0 on 10.30.177.15 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" In order to be able to use multiple shared statistics buffers, we must allocate them. They are allocated in SoftwareIsp and passed to SwIspStats. This changes the previous behavior when the (single) shared statistics buffer was created in SwIspStats. Centralizing it to SoftwareIsp makes sharing multiple buffers with IPA easier. Currently only one of the allocated buffers is used. This will be changed once the buffers are shared with IPA in a followup patch. Signed-off-by: Milan Zamazal --- .../internal/software_isp/software_isp.h | 5 +++ src/libcamera/software_isp/software_isp.cpp | 44 ++++++++++++++++--- src/libcamera/software_isp/swstats_cpu.cpp | 16 ++++--- src/libcamera/software_isp/swstats_cpu.h | 9 ++-- 4 files changed, 57 insertions(+), 17 deletions(-) diff --git a/include/libcamera/internal/software_isp/software_isp.h b/include/libcamera/internal/software_isp/software_isp.h index ae64b247..a6ba4a07 100644 --- a/include/libcamera/internal/software_isp/software_isp.h +++ b/include/libcamera/internal/software_isp/software_isp.h @@ -33,6 +33,8 @@ #include "libcamera/internal/shared_mem_object.h" #include "libcamera/internal/software_isp/debayer_params.h" +#include "software_isp/swstats_cpu.h" + namespace libcamera { class DebayerCpu; @@ -100,6 +102,9 @@ private: DebayerParams debayerParams_; std::queue availableParams_; bool allocateParamsBuffers(const unsigned int bufferCount); + std::unique_ptr allocateStatsBuffers( + std::vector &fdStats, + const unsigned int bufferCount); DmaBufAllocator dmaHeap_; std::unique_ptr ipa_; diff --git a/src/libcamera/software_isp/software_isp.cpp b/src/libcamera/software_isp/software_isp.cpp index 5d9f5008..1e224b6f 100644 --- a/src/libcamera/software_isp/software_isp.cpp +++ b/src/libcamera/software_isp/software_isp.cpp @@ -12,6 +12,8 @@ #include #include #include +#include +#include #include @@ -20,7 +22,9 @@ #include #include "libcamera/internal/ipa_manager.h" +#include "libcamera/internal/shared_mem_object.h" #include "libcamera/internal/software_isp/debayer_params.h" +#include "libcamera/internal/software_isp/swisp_stats.h" #include "debayer_cpu.h" @@ -79,17 +83,15 @@ SoftwareIsp::SoftwareIsp(PipelineHandler *pipe, const CameraSensor *sensor, if (!allocateParamsBuffers(bufferCount)) return; - auto stats = std::make_unique(); - if (!stats->isValid()) { - LOG(SoftwareIsp, Error) << "Failed to create SwStatsCpu object"; - return; - } - stats->statsReady.connect(this, &SoftwareIsp::statsReady); - std::vector fdParams; for (auto &item : sharedParams_) fdParams.emplace_back(item.second.fd()); + std::vector fdStats; + auto stats = allocateStatsBuffers(fdStats, bufferCount); + if (!stats) + return; + debayer_ = std::make_unique(std::move(stats), fdParams); debayer_->inputBufferReady.connect(this, &SoftwareIsp::inputReady); debayer_->outputBufferReady.connect(this, &SoftwareIsp::outputReady); @@ -169,6 +171,34 @@ bool SoftwareIsp::allocateParamsBuffers(const unsigned int bufferCount) return true; } +std::unique_ptr SoftwareIsp::allocateStatsBuffers( + std::vector &fdStats, + const unsigned int bufferCount) +{ + auto sharedStats = std::make_unique>>(); + for (unsigned int i = 0; i < bufferCount; i++) { + auto shared = SharedMemObject("softIsp_stats"); + if (!shared) { + LOG(SoftwareIsp, Error) << "Failed to create shared memory for statistics"; + return std::unique_ptr(); + } + if (!shared.fd().isValid()) { + LOG(SoftwareIsp, Error) << "Invalid fd of shared statistics"; + return std::unique_ptr(); + } + + ASSERT(shared.fd().get() >= 0); + unsigned int bufferId = shared.fd().get(); + fdStats.emplace_back(shared.fd()); + sharedStats->emplace(bufferId, std::move(shared)); + } + + auto stats = std::make_unique(std::move(sharedStats)); + stats->statsReady.connect(this, &SoftwareIsp::statsReady); + + return stats; +} + /** * \fn int SoftwareIsp::loadConfiguration([[maybe_unused]] const std::string &filename) * \brief Load a configuration from a file diff --git a/src/libcamera/software_isp/swstats_cpu.cpp b/src/libcamera/software_isp/swstats_cpu.cpp index b8ee06a0..cb411a18 100644 --- a/src/libcamera/software_isp/swstats_cpu.cpp +++ b/src/libcamera/software_isp/swstats_cpu.cpp @@ -11,6 +11,8 @@ #include "swstats_cpu.h" +#include + #include #include @@ -129,12 +131,14 @@ namespace libcamera { LOG_DEFINE_CATEGORY(SwStatsCpu) -SwStatsCpu::SwStatsCpu() - : sharedStats_("softIsp_stats") +/** + * \brief The constructor of SwStatsCpu + * \param [in] sharedStats Mapping of statistics buffer ids to statistics + * instances that are shared with the IPA + */ +SwStatsCpu::SwStatsCpu(std::unique_ptr>> sharedStats) + : sharedStats_(std::move(sharedStats)) { - if (!sharedStats_) - LOG(SwStatsCpu, Error) - << "Failed to create shared memory for statistics"; } static constexpr unsigned int kRedYMul = 77; /* 0.299 * 256 */ @@ -316,7 +320,7 @@ void SwStatsCpu::startFrame([[maybe_unused]] const uint32_t statsBufferId) */ void SwStatsCpu::finishFrame(uint32_t frame, const uint32_t statsBufferId) { - *sharedStats_ = stats_; + *(sharedStats_->at(statsBufferId)) = stats_; statsReady.emit(frame, statsBufferId); } diff --git a/src/libcamera/software_isp/swstats_cpu.h b/src/libcamera/software_isp/swstats_cpu.h index 402eef2d..fbdea4a3 100644 --- a/src/libcamera/software_isp/swstats_cpu.h +++ b/src/libcamera/software_isp/swstats_cpu.h @@ -11,6 +11,7 @@ #pragma once +#include #include #include @@ -29,12 +30,12 @@ struct StreamConfiguration; class SwStatsCpu { public: - SwStatsCpu(); + SwStatsCpu(std::unique_ptr>> sharedStats); ~SwStatsCpu() = default; - bool isValid() const { return sharedStats_.fd().isValid(); } + bool isValid() const { return sharedStats_->begin()->second.fd().isValid(); } - const SharedFD &getStatsFD() { return sharedStats_.fd(); } + const SharedFD &getStatsFD() { return sharedStats_->begin()->second.fd(); } const Size &patternSize() { return patternSize_; } @@ -90,7 +91,7 @@ private: unsigned int xShift_; - SharedMemObject sharedStats_; + std::unique_ptr>> sharedStats_; SwIspStats stats_; }; From patchwork Mon Aug 12 11:50:03 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Milan Zamazal X-Patchwork-Id: 20888 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id D1FF2C323E for ; Mon, 12 Aug 2024 11:50:54 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 83FFE633D3; Mon, 12 Aug 2024 13:50:54 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.b="f+/+g8ML"; dkim-atps=neutral Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 94B58633D6 for ; Mon, 12 Aug 2024 13:50:44 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1723463443; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=BhDmjC8q+iup+NUxl+/6VmxG1PqcdwEFDvjvinyQbmE=; b=f+/+g8MLZzxh0jrs4ZT2UZaZzQ3fxf38MSlZessAOXeP4VZ3DKafuZiY311G+GF+f27jq2 uQcj5tF71oB+y/uPxqaE/HlU3u7CfWXqOHqvzA20/ezv/qICcJXIZLUPQjyaeBbmgAbFZ5 W3UcHiyu7Pre7QQKYYSaFrZMLUHaBmY= Received: from mx-prod-mc-03.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-562-OZXEfsYwPEqFVZ0GbrEA5w-1; Mon, 12 Aug 2024 07:50:42 -0400 X-MC-Unique: OZXEfsYwPEqFVZ0GbrEA5w-1 Received: from mx-prod-int-02.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-02.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.15]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-03.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 2619A1954225 for ; Mon, 12 Aug 2024 11:50:41 +0000 (UTC) Received: from nuthatch.redhat.com (unknown [10.45.225.57]) by mx-prod-int-02.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id 2AD2D19772C4; Mon, 12 Aug 2024 11:50:39 +0000 (UTC) From: Milan Zamazal To: libcamera-devel@lists.libcamera.org Cc: Milan Zamazal Subject: [PATCH 14/16] libcamera: software_isp: Track statistics buffers Date: Mon, 12 Aug 2024 13:50:03 +0200 Message-ID: <20240812115009.946036-15-mzamazal@redhat.com> In-Reply-To: <20240812115009.946036-1-mzamazal@redhat.com> References: <20240812115009.946036-1-mzamazal@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.0 on 10.30.177.15 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" As a preparation for passing the shared statistics buffers to IPA, this patch introduces tracking statistics buffer ids. SofwareIsp::availableStats_ is a queue of ids of the buffers available for use. When a fresh statistics buffer is needed, its id is retrieved from there and once the buffer is no longer needed, it's put back there, in a method invoked by a signal. This is similar to what the hardware pipelines do. We use buffers' file descriptors as buffer ids. The statistics buffers will be shared with the IPA, SwIspStats and debayering and we need their common identification everywhere. The buffer file descriptors are shared unchanged so they can be used for the purpose, avoiding a need to pass a special mapping to the IPA and debayering on initialization. Note that the statistics buffer id is still actually unused, this will be changed once the buffers are shared with the IPA. Signed-off-by: Milan Zamazal --- .../libcamera/internal/software_isp/software_isp.h | 1 + src/libcamera/software_isp/software_isp.cpp | 13 +++++++++++-- 2 files changed, 12 insertions(+), 2 deletions(-) diff --git a/include/libcamera/internal/software_isp/software_isp.h b/include/libcamera/internal/software_isp/software_isp.h index a6ba4a07..5959e86e 100644 --- a/include/libcamera/internal/software_isp/software_isp.h +++ b/include/libcamera/internal/software_isp/software_isp.h @@ -102,6 +102,7 @@ private: DebayerParams debayerParams_; std::queue availableParams_; bool allocateParamsBuffers(const unsigned int bufferCount); + std::queue availableStats_; std::unique_ptr allocateStatsBuffers( std::vector &fdStats, const unsigned int bufferCount); diff --git a/src/libcamera/software_isp/software_isp.cpp b/src/libcamera/software_isp/software_isp.cpp index 1e224b6f..c4ad22be 100644 --- a/src/libcamera/software_isp/software_isp.cpp +++ b/src/libcamera/software_isp/software_isp.cpp @@ -191,6 +191,7 @@ std::unique_ptr SoftwareIsp::allocateStatsBuffers( unsigned int bufferId = shared.fd().get(); fdStats.emplace_back(shared.fd()); sharedStats->emplace(bufferId, std::move(shared)); + availableStats_.push(bufferId); } auto stats = std::make_unique(std::move(sharedStats)); @@ -413,11 +414,18 @@ void SoftwareIsp::process(uint32_t frame, FrameBuffer *input, FrameBuffer *outpu while (availableParams_.empty()) ; } + if (availableStats_.empty()) { + LOG(SoftwareIsp, Error) << "Statistics buffer underrun"; + /* Well, busy loop, but this situation shouldn't normally happen. */ + while (availableStats_.empty()) + ; + } const uint32_t paramsBufferId = availableParams_.front(); availableParams_.pop(); + const uint32_t statsBufferId = availableStats_.front(); + availableStats_.pop(); ipa_->prepare(frame, paramsBufferId); - const uint32_t statsBufferId = 0; debayer_->invokeMethod(&DebayerCpu::process, ConnectionTypeQueued, frame, statsBufferId, paramsBufferId, input, output); @@ -443,8 +451,9 @@ void SoftwareIsp::statsReady(uint32_t frame, const uint32_t statsBufferId) ispStatsReady.emit(frame, statsBufferId); } -void SoftwareIsp::statsProcessed([[maybe_unused]] const uint32_t statsBufferId) +void SoftwareIsp::statsProcessed(const uint32_t statsBufferId) { + availableStats_.push(statsBufferId); } void SoftwareIsp::inputReady(FrameBuffer *input) From patchwork Mon Aug 12 11:50:04 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Milan Zamazal X-Patchwork-Id: 20889 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 26491C324E for ; Mon, 12 Aug 2024 11:50:56 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 84476633D2; Mon, 12 Aug 2024 13:50:55 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.b="JP09QSAF"; dkim-atps=neutral Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 3A7D0633D3 for ; Mon, 12 Aug 2024 13:50:46 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1723463445; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=e+8dUtgJJp1QhBqBjIxq7uQCwmxP1iH+rGvy3/Mu6ps=; b=JP09QSAFhhmtDXz5oP38/j5/GccpQqVKoKJ5+2JyrpFW43RQPPzz18r7MWRKtcvcQOyXF0 RLM2EgeeCZtCQaEO0MOVGTZrCBoMMMV3CW3I9y+WblC8kxQ41o3Mkq50T+etL95MBqKTN+ h1jInauEPindvB6Qzddq6a0oDeFIt/8= Received: from mx-prod-mc-04.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-203-Co5p2UpFPfOhYX5gNpuHUg-1; Mon, 12 Aug 2024 07:50:43 -0400 X-MC-Unique: Co5p2UpFPfOhYX5gNpuHUg-1 Received: from mx-prod-int-02.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-02.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.15]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-04.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id D91A6192538C for ; Mon, 12 Aug 2024 11:50:42 +0000 (UTC) Received: from nuthatch.redhat.com (unknown [10.45.225.57]) by mx-prod-int-02.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id A3F3319772C4; Mon, 12 Aug 2024 11:50:41 +0000 (UTC) From: Milan Zamazal To: libcamera-devel@lists.libcamera.org Cc: Milan Zamazal Subject: [PATCH 15/16] libcamera: software_isp: Share statistics buffers with IPA Date: Mon, 12 Aug 2024 13:50:04 +0200 Message-ID: <20240812115009.946036-16-mzamazal@redhat.com> In-Reply-To: <20240812115009.946036-1-mzamazal@redhat.com> References: <20240812115009.946036-1-mzamazal@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.0 on 10.30.177.15 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" The last step to complete statistics buffer sharing is to pass all the allocated statistics buffers to the IPA and refer to them using their ids. This allows to remove the buffer copying in SwStatsCpu::finishFrame. In order to track the current statistics buffer in SwStatsCpu instead, we change SwStatsCpu::stats_ to a wrapper structure. This is because we need a reference to a shared mem object but a class attribute cannot be a dynamically assigned reference. This hack works around the problem. We can also remove now the methods that served for handling the former single buffer shared between debayering and IPA. Signed-off-by: Milan Zamazal --- include/libcamera/ipa/soft.mojom | 2 +- src/ipa/simple/soft_simple.cpp | 43 +++++++++++---------- src/libcamera/software_isp/debayer_cpu.h | 7 ---- src/libcamera/software_isp/software_isp.cpp | 2 +- src/libcamera/software_isp/swstats_cpu.cpp | 37 ++++++------------ src/libcamera/software_isp/swstats_cpu.h | 12 +++--- 6 files changed, 43 insertions(+), 60 deletions(-) diff --git a/include/libcamera/ipa/soft.mojom b/include/libcamera/ipa/soft.mojom index 7b85c454..e5af50d6 100644 --- a/include/libcamera/ipa/soft.mojom +++ b/include/libcamera/ipa/soft.mojom @@ -14,7 +14,7 @@ struct IPAConfigInfo { interface IPASoftInterface { init(libcamera.IPASettings settings, - libcamera.SharedFD fdStats, + array fdStats, array fdParams, libcamera.ControlInfoMap sensorCtrlInfoMap) => (int32 ret); diff --git a/src/ipa/simple/soft_simple.cpp b/src/ipa/simple/soft_simple.cpp index 3c95c4d9..2769bce2 100644 --- a/src/ipa/simple/soft_simple.cpp +++ b/src/ipa/simple/soft_simple.cpp @@ -47,7 +47,7 @@ public: ~IPASoftSimple(); int init(const IPASettings &settings, - const SharedFD &fdStats, + const std::vector &fdStats, const std::vector &fdParams, const ControlInfoMap &sensorInfoMap) override; int configure(const IPAConfigInfo &configInfo) override; @@ -67,7 +67,7 @@ private: void updateExposure(double exposureMSV); std::map paramsBuffers_; - SwIspStats *stats_; + std::map statsBuffers_; std::unique_ptr camHelper_; ControlInfoMap sensorInfoMap_; @@ -77,14 +77,14 @@ private: IPASoftSimple::~IPASoftSimple() { - if (stats_) - munmap(stats_, sizeof(SwIspStats)); + for (auto &item : statsBuffers_) + munmap(item.second, sizeof(SwIspStats)); for (auto &item : paramsBuffers_) munmap(item.second, sizeof(DebayerParams)); } int IPASoftSimple::init(const IPASettings &settings, - const SharedFD &fdStats, + const std::vector &fdStats, const std::vector &fdParams, const ControlInfoMap &sensorInfoMap) { @@ -122,11 +122,21 @@ int IPASoftSimple::init(const IPASettings &settings, if (ret) return ret; - stats_ = nullptr; + for (auto &sharedFd : fdStats) { + if (!sharedFd.isValid()) { + LOG(IPASoft, Error) << "Invalid Statistics handle"; + return -ENODEV; + } - if (!fdStats.isValid()) { - LOG(IPASoft, Error) << "Invalid Statistics handle"; - return -ENODEV; + void *mem = mmap(nullptr, sizeof(SwIspStats), PROT_READ, + MAP_SHARED, sharedFd.get(), 0); + if (mem == MAP_FAILED) { + LOG(IPASoft, Error) << "Unable to map Statistics"; + return -errno; + } + + ASSERT(sharedFd.get() >= 0); + statsBuffers_[sharedFd.get()] = static_cast(mem); } for (auto &sharedFd : fdParams) { @@ -146,17 +156,6 @@ int IPASoftSimple::init(const IPASettings &settings, paramsBuffers_[sharedFd.get()] = static_cast(mem); } - { - void *mem = mmap(nullptr, sizeof(SwIspStats), PROT_READ, - MAP_SHARED, fdStats.get(), 0); - if (mem == MAP_FAILED) { - LOG(IPASoft, Error) << "Unable to map Statistics"; - return -errno; - } - - stats_ = static_cast(mem); - } - /* * Check if the sensor driver supports the controls required by the * Soft IPA. @@ -271,6 +270,8 @@ void IPASoftSimple::processStats( { IPAFrameContext &frameContext = context_.frameContexts.get(frame); + const SwIspStats *stats = statsBuffers_.at(statsBufferId); + frameContext.sensor.exposure = sensorControls.get(V4L2_CID_EXPOSURE).get(); int32_t again = sensorControls.get(V4L2_CID_ANALOGUE_GAIN).get(); @@ -284,7 +285,7 @@ void IPASoftSimple::processStats( */ ControlList metadata(controls::controls); for (auto const &algo : algorithms()) - algo->process(context_, frame, frameContext, stats_, metadata); + algo->process(context_, frame, frameContext, stats, metadata); statsProcessed.emit(statsBufferId); /* Sanity check */ diff --git a/src/libcamera/software_isp/debayer_cpu.h b/src/libcamera/software_isp/debayer_cpu.h index 7fb399b0..36300534 100644 --- a/src/libcamera/software_isp/debayer_cpu.h +++ b/src/libcamera/software_isp/debayer_cpu.h @@ -43,13 +43,6 @@ public: FrameBuffer *input, FrameBuffer *output); SizeRange sizes(PixelFormat inputFormat, const Size &inputSize); - /** - * \brief Get the file descriptor for the statistics - * - * \return the file descriptor pointing to the statistics - */ - const SharedFD &getStatsFD() { return stats_->getStatsFD(); } - /** * \brief Get the output frame size * diff --git a/src/libcamera/software_isp/software_isp.cpp b/src/libcamera/software_isp/software_isp.cpp index c4ad22be..025093a9 100644 --- a/src/libcamera/software_isp/software_isp.cpp +++ b/src/libcamera/software_isp/software_isp.cpp @@ -113,7 +113,7 @@ SoftwareIsp::SoftwareIsp(PipelineHandler *pipe, const CameraSensor *sensor, ipa_->configurationFile(sensor->model() + ".yaml", "uncalibrated.yaml"); int ret = ipa_->init(IPASettings{ ipaTuningFile, sensor->model() }, - debayer_->getStatsFD(), + fdStats, fdParams, sensor->controls()); if (ret) { diff --git a/src/libcamera/software_isp/swstats_cpu.cpp b/src/libcamera/software_isp/swstats_cpu.cpp index cb411a18..2c7acf47 100644 --- a/src/libcamera/software_isp/swstats_cpu.cpp +++ b/src/libcamera/software_isp/swstats_cpu.cpp @@ -36,20 +36,6 @@ namespace libcamera { * instead of processing the whole frame. */ -/** - * \fn bool SwStatsCpu::isValid() const - * \brief Gets whether the statistics object is valid - * - * \return True if it's valid, false otherwise - */ - -/** - * \fn const SharedFD &SwStatsCpu::getStatsFD() - * \brief Get the file descriptor for the statistics - * - * \return The file descriptor - */ - /** * \fn const Size &SwStatsCpu::patternSize() * \brief Get the pattern size @@ -161,12 +147,12 @@ static constexpr unsigned int kBlueYMul = 29; /* 0.114 * 256 */ yVal = r * kRedYMul; \ yVal += g * kGreenYMul; \ yVal += b * kBlueYMul; \ - stats_.yHistogram[yVal * SwIspStats::kYHistogramSize / (256 * 256 * (div))]++; + stats_->stats->yHistogram[yVal * SwIspStats::kYHistogramSize / (256 * 256 * (div))]++; -#define SWSTATS_FINISH_LINE_STATS() \ - stats_.sumR_ += sumR; \ - stats_.sumG_ += sumG; \ - stats_.sumB_ += sumB; +#define SWSTATS_FINISH_LINE_STATS() \ + stats_->stats->sumR_ += sumR; \ + stats_->stats->sumG_ += sumG; \ + stats_->stats->sumB_ += sumB; void SwStatsCpu::statsBGGR8Line0(const uint8_t *src[]) { @@ -302,15 +288,17 @@ void SwStatsCpu::statsGBRG10PLine0(const uint8_t *src[]) * * This may only be called after a successful setWindow() call. */ -void SwStatsCpu::startFrame([[maybe_unused]] const uint32_t statsBufferId) +void SwStatsCpu::startFrame(const uint32_t statsBufferId) { if (window_.width == 0) LOG(SwStatsCpu, Error) << "Calling startFrame() without setWindow()"; - stats_.sumR_ = 0; - stats_.sumB_ = 0; - stats_.sumG_ = 0; - stats_.yHistogram.fill(0); + auto &s = sharedStats_->at(statsBufferId); + stats_ = std::make_unique(s); + stats_->stats->sumR_ = 0; + stats_->stats->sumB_ = 0; + stats_->stats->sumG_ = 0; + stats_->stats->yHistogram.fill(0); } /** @@ -320,7 +308,6 @@ void SwStatsCpu::startFrame([[maybe_unused]] const uint32_t statsBufferId) */ void SwStatsCpu::finishFrame(uint32_t frame, const uint32_t statsBufferId) { - *(sharedStats_->at(statsBufferId)) = stats_; statsReady.emit(frame, statsBufferId); } diff --git a/src/libcamera/software_isp/swstats_cpu.h b/src/libcamera/software_isp/swstats_cpu.h index fbdea4a3..2b4a98c2 100644 --- a/src/libcamera/software_isp/swstats_cpu.h +++ b/src/libcamera/software_isp/swstats_cpu.h @@ -12,6 +12,7 @@ #pragma once #include +#include #include #include @@ -33,10 +34,6 @@ public: SwStatsCpu(std::unique_ptr>> sharedStats); ~SwStatsCpu() = default; - bool isValid() const { return sharedStats_->begin()->second.fd().isValid(); } - - const SharedFD &getStatsFD() { return sharedStats_->begin()->second.fd(); } - const Size &patternSize() { return patternSize_; } int configure(const StreamConfiguration &inputCfg); @@ -92,7 +89,12 @@ private: unsigned int xShift_; std::unique_ptr>> sharedStats_; - SwIspStats stats_; + struct SwIspStatsRef { + SharedMemObject &stats; + SwIspStatsRef(SharedMemObject &_stats) + : stats(_stats){}; + }; + std::unique_ptr stats_; }; } /* namespace libcamera */ From patchwork Mon Aug 12 11:50:05 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Milan Zamazal X-Patchwork-Id: 20890 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 7860FC323E for ; Mon, 12 Aug 2024 11:50:57 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 177D2633CA; Mon, 12 Aug 2024 13:50:57 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.b="etuepSCM"; dkim-atps=neutral Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 903AA633DB for ; Mon, 12 Aug 2024 13:50:47 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1723463446; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=YIGnnrU9sn832F49PZotPozmJP3zPpgRObwzrRoBQfE=; b=etuepSCMEryT09t/oDtNLX4sO8I+3d4y/bTvxgjqe8i0JMns4LwVbl268Z6kXRMx387TCP faMiRoeBtdh+o7O0Z88Tj8OTM/SHL/ur753NybMr9NKO0YhCs7o1+/lOPqwb/p9tZ0OI+s jVnUT7pY1F+YB6g4XFGxZ7E1Uy+eR80= Received: from mx-prod-mc-01.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-687-PM8kHWjdNMm_BWu7ukP3PQ-1; Mon, 12 Aug 2024 07:50:45 -0400 X-MC-Unique: PM8kHWjdNMm_BWu7ukP3PQ-1 Received: from mx-prod-int-02.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-02.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.15]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-01.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 434971954B16 for ; Mon, 12 Aug 2024 11:50:44 +0000 (UTC) Received: from nuthatch.redhat.com (unknown [10.45.225.57]) by mx-prod-int-02.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id 33E9219772C4; Mon, 12 Aug 2024 11:50:42 +0000 (UTC) From: Milan Zamazal To: libcamera-devel@lists.libcamera.org Cc: Milan Zamazal Subject: [PATCH 16/16] libcamera: software_isp: Remove stats-sharing TODO item Date: Mon, 12 Aug 2024 13:50:05 +0200 Message-ID: <20240812115009.946036-17-mzamazal@redhat.com> In-Reply-To: <20240812115009.946036-1-mzamazal@redhat.com> References: <20240812115009.946036-1-mzamazal@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.0 on 10.30.177.15 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Shared per-frame statistics buffers are implemented, we can remove TODO #2. Signed-off-by: Milan Zamazal --- src/libcamera/software_isp/TODO | 27 --------------------------- 1 file changed, 27 deletions(-) diff --git a/src/libcamera/software_isp/TODO b/src/libcamera/software_isp/TODO index db1115dd..b83c3ba3 100644 --- a/src/libcamera/software_isp/TODO +++ b/src/libcamera/software_isp/TODO @@ -1,30 +1,3 @@ -2. Reconsider stats sharing - ->>> +void SwStatsCpu::finishFrame(void) ->>> +{ ->>> + *sharedStats_ = stats_; ->> ->> Is it more efficient to copy the stats instead of operating directly on ->> the shared memory ? -> -> I inherited doing things this way from Andrey. I kept this because -> we don't really have any synchronization with the IPA reading this. -> -> So the idea is to only touch this when the next set of statistics -> is ready since we don't know when the IPA is done with accessing -> the previous set of statistics ... -> -> This is both something which seems mostly a theoretic problem, -> yet also definitely something which I think we need to fix. -> -> Maybe use a ringbuffer of stats buffers and pass the index into -> the ringbuffer to the emit signal ? - -That would match how we deal with hardware ISPs, and I think that's a -good idea. It will help decoupling the processing side from the IPA. - ---- - 3. Remove statsReady signal > class SwStatsCpu