From patchwork Wed Oct 30 16:48:51 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jacopo Mondi X-Patchwork-Id: 21777 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 996FDC31E9 for ; Wed, 30 Oct 2024 16:49:07 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 9A768653AB; Wed, 30 Oct 2024 17:49:04 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="p40FhJQW"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 6877B6539A for ; Wed, 30 Oct 2024 17:49:02 +0100 (CET) Received: from ideasonboard.com (93-61-96-190.ip145.fastwebnet.it [93.61.96.190]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id E3EFB1083; Wed, 30 Oct 2024 17:48:58 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1730306939; bh=LimbK/+ayXRBlcQOFit0Xdmpq/mWvshsQTZC5cQILbg=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=p40FhJQWjmjFj4QkxJcn1I6fxtBa79CBPxZ4m9tmhqomi2J/hk+VD7U+LLkzE1Gi0 7SOLt1kCLZvGbiKO7XuicZyDWcRfTVIUBItx9rukFiJ6z1bdWA6TBwIl/wdwVe0m0q IaHxWBvLDuT57DjV5FKbaKPZaz6c1AJN8uv0yW0U= From: Jacopo Mondi To: libcamera-devel@lists.libcamera.org Cc: Jacopo Mondi , Stefan Klug , Dan Scally Subject: [PATCH v2 1/2] libipa: FCQueue: Make sure FrameContext#0 is initialized Date: Wed, 30 Oct 2024 17:48:51 +0100 Message-ID: <20241030164853.87586-2-jacopo.mondi@ideasonboard.com> X-Mailer: git-send-email 2.47.0 In-Reply-To: <20241030164853.87586-1-jacopo.mondi@ideasonboard.com> References: <20241030164853.87586-1-jacopo.mondi@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Some IPA modules, like the RkISP1 one, call FCQueue::get(0) at IPA::start() time, before any frame context has been allocated with FCQueue::alloc() called at queueRequest() time. The FCQueue implementation aims to detect when a FrameContext is get() before it is alloc()-ated, Warns about it, and initializes the FrameContext before returning it. In case of frame#0, a get() preceding an alloc() call is not detected as the "frame == frameContext.frame" test returns success, as FrameContexts are zeroed by default. As a result, the first returned FrameContext is not initialized. Explicitly test for frame#0 to make sure the FrameContext is initialized if get(0) is called before alloc(0). To avoid re-initializing a frame context, in case alloc() has been called correctly before get(), introduce an "initialised" state variable that tracks the FrameContext initialisation state. Signed-off-by: Jacopo Mondi Reviewed-by: Daniel Scally Reviewed-by: Stefan Klug --- src/ipa/libipa/fc_queue.h | 21 ++++++++++++++++++++- 1 file changed, 20 insertions(+), 1 deletion(-) diff --git a/src/ipa/libipa/fc_queue.h b/src/ipa/libipa/fc_queue.h index 24d9e82b727d..a1d136521107 100644 --- a/src/ipa/libipa/fc_queue.h +++ b/src/ipa/libipa/fc_queue.h @@ -25,6 +25,7 @@ struct FrameContext { private: template friend class FCQueue; uint32_t frame; + bool initialised = false; }; template @@ -38,8 +39,10 @@ public: void clear() { - for (FrameContext &ctx : contexts_) + for (FrameContext &ctx : contexts_) { + ctx.initialised = false; ctx.frame = 0; + } } FrameContext &alloc(const uint32_t frame) @@ -83,6 +86,21 @@ public: << " has been overwritten by " << frameContext.frame; + if (frame == 0 && !frameContext.initialised) { + /* + * If the IPA calls get() at start() time it will get an + * un-intialized FrameContext as the below "frame == + * frameContext.frame" check will return success because + * FrameContexts are zeroed at creation time. + * + * Make sure the FrameContext gets initialised if get() + * is called before alloc() by the IPA for frame#0. + */ + init(frameContext, frame); + + return frameContext; + } + if (frame == frameContext.frame) return frameContext; @@ -108,6 +126,7 @@ private: { frameContext = {}; frameContext.frame = frame; + frameContext.initialised = true; } std::vector contexts_; From patchwork Wed Oct 30 16:48:52 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jacopo Mondi X-Patchwork-Id: 21778 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 2D97EC330B for ; Wed, 30 Oct 2024 16:49:09 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id AF469653B2; Wed, 30 Oct 2024 17:49:05 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="dszoHZhc"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id D16FA65392 for ; Wed, 30 Oct 2024 17:49:02 +0100 (CET) Received: from ideasonboard.com (93-61-96-190.ip145.fastwebnet.it [93.61.96.190]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 5D0FCB7E; Wed, 30 Oct 2024 17:48:59 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1730306939; bh=ENkjd2vo0r5Yb6A4HNbdTnWQsnT9Jf4U7wNdjEGompo=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=dszoHZhc88DRR1QhyKCMC/qidncj1Eliz/4g8YrxVHL7t2KGW8PO1S1tavmWBVaCb t9Y4WxEwY/ro77jqJS8FKS1r1ByVjY7A6RGbKG1vAVS7OlMooSpSpi1/AcGOHb636J qyjvqHnZ6eDqWbNvVuJ876RFZKjb4A4lNej0vdnA= From: Jacopo Mondi To: libcamera-devel@lists.libcamera.org Cc: Jacopo Mondi , Stefan Klug , Dan Scally Subject: [PATCH v2 2/2] ipa: rkisp1: Have algos initialize FrameContext Date: Wed, 30 Oct 2024 17:48:52 +0100 Message-ID: <20241030164853.87586-3-jacopo.mondi@ideasonboard.com> X-Mailer: git-send-email 2.47.0 In-Reply-To: <20241030164853.87586-1-jacopo.mondi@ideasonboard.com> References: <20241030164853.87586-1-jacopo.mondi@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" The RkISP1 AGC algorithms assumes the metering mode to be "MeteringMatrix" and pre-fill an array of weights associated with it stored in meteringModes_[MeteringMatrix] when intializing the algorithm in parseMeteringModes(). It laters fetches the weights when computing parameters using the FrameContext.agc.meteringMode as index of the meteringModes_ array. When a Request gets queued to the algorithm, the FrameContext.agc.meteringMode index is populated from the algorithm's active state, and optionally updated if the application has specified a different metering mode. In case of Request underrun however, a non-initialized FrameContext gets provided to the algorithm, causing an (unhandled) exception when accessing the meteringModes_ array. To handle the situation when a Request underrun occours and let algorithms initialize a FrameContext to safe defaults: - Add an 'underrun' flag to FrameContext - Add an 'initFrameContext()' function to RkISP1 algorithms - If a FrameContext is get() before being allocated, pass it through the algorithms' initFrameContext() to safely initialize it Signed-off-by: Jacopo Mondi --- src/ipa/libipa/fc_queue.cpp | 6 ++++++ src/ipa/libipa/fc_queue.h | 5 +++++ src/ipa/rkisp1/algorithms/agc.cpp | 8 ++++++++ src/ipa/rkisp1/algorithms/agc.h | 4 ++++ src/ipa/rkisp1/algorithms/algorithm.h | 5 +++++ src/ipa/rkisp1/rkisp1.cpp | 9 +++++++++ 6 files changed, 37 insertions(+) diff --git a/src/ipa/libipa/fc_queue.cpp b/src/ipa/libipa/fc_queue.cpp index 0365e9197748..44f9d866ad1b 100644 --- a/src/ipa/libipa/fc_queue.cpp +++ b/src/ipa/libipa/fc_queue.cpp @@ -36,6 +36,12 @@ namespace ipa { * * \var FrameContext::frame * \brief The frame number + * + * \var FrameContext::underrun + * \brief Boolean flag that reports if the FrameContext has been accessed with + * a FCQeueu::get() call before being FCQueue::alloc()-ated. If the flag is set + * to True then the frame context needs to be initialized by algorithms to safe + * defaults as it won't be initialized with any non-user provided control. */ /** diff --git a/src/ipa/libipa/fc_queue.h b/src/ipa/libipa/fc_queue.h index a1d136521107..d70ca9601bd7 100644 --- a/src/ipa/libipa/fc_queue.h +++ b/src/ipa/libipa/fc_queue.h @@ -22,6 +22,9 @@ template class FCQueue; struct FrameContext { +public: + bool underrun = false; + private: template friend class FCQueue; uint32_t frame; @@ -97,6 +100,7 @@ public: * is called before alloc() by the IPA for frame#0. */ init(frameContext, frame); + frameContext.underrun = true; return frameContext; } @@ -117,6 +121,7 @@ public: << "Obtained an uninitialised FrameContext for " << frame; init(frameContext, frame); + frameContext.underrun = true; return frameContext; } diff --git a/src/ipa/rkisp1/algorithms/agc.cpp b/src/ipa/rkisp1/algorithms/agc.cpp index 301b7ec26508..4122f665b3ee 100644 --- a/src/ipa/rkisp1/algorithms/agc.cpp +++ b/src/ipa/rkisp1/algorithms/agc.cpp @@ -204,6 +204,14 @@ int Agc::configure(IPAContext &context, const IPACameraSensorInfo &configInfo) return 0; } +void Agc::initFrameContext(IPAContext &context, + IPAFrameContext &frameContext) +{ + auto &agc = context.activeState.agc; + + frameContext.agc.meteringMode = agc.meteringMode; +} + /** * \copydoc libcamera::ipa::Algorithm::queueRequest */ diff --git a/src/ipa/rkisp1/algorithms/agc.h b/src/ipa/rkisp1/algorithms/agc.h index aa86f2c5bc21..c1adf91bbc4e 100644 --- a/src/ipa/rkisp1/algorithms/agc.h +++ b/src/ipa/rkisp1/algorithms/agc.h @@ -30,6 +30,10 @@ public: int init(IPAContext &context, const YamlObject &tuningData) override; int configure(IPAContext &context, const IPACameraSensorInfo &configInfo) override; + + void initFrameContext(IPAContext &context, + IPAFrameContext &frameContext) override; + void queueRequest(IPAContext &context, const uint32_t frame, IPAFrameContext &frameContext, diff --git a/src/ipa/rkisp1/algorithms/algorithm.h b/src/ipa/rkisp1/algorithms/algorithm.h index 715cfcd8298b..82603b7b372d 100644 --- a/src/ipa/rkisp1/algorithms/algorithm.h +++ b/src/ipa/rkisp1/algorithms/algorithm.h @@ -23,6 +23,11 @@ public: { } + virtual void initFrameContext([[maybe_unused]] IPAContext &context, + [[maybe_unused]] IPAFrameContext &frameContext) + { + } + bool disabled_; bool supportsRaw_; }; diff --git a/src/ipa/rkisp1/rkisp1.cpp b/src/ipa/rkisp1/rkisp1.cpp index 9e161cabdea4..b743de9ff6af 100644 --- a/src/ipa/rkisp1/rkisp1.cpp +++ b/src/ipa/rkisp1/rkisp1.cpp @@ -353,6 +353,15 @@ void IPARkISP1::processStatsBuffer(const uint32_t frame, const uint32_t bufferId { IPAFrameContext &frameContext = context_.frameContexts.get(frame); + if (frameContext.underrun) { + for (auto const &a : algorithms()) { + Algorithm *algo = static_cast(a.get()); + if (algo->disabled_) + continue; + algo->initFrameContext(context_, frameContext); + } + } + /* * In raw capture mode, the ISP is bypassed and no statistics buffer is * provided.