From patchwork Tue Sep 27 02:36:10 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Laurent Pinchart X-Patchwork-Id: 17412 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 4A52BC0DA4 for ; Tue, 27 Sep 2022 02:37:05 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 095CD62269; Tue, 27 Sep 2022 04:37:05 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1664246225; bh=Snpjz2ADDNDiP6WIRVhkMO0Jamjr3LpMW7rfD3A86mY=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To: From; b=SBcdQqSoAT2J0CGRt0oG/3oQy7+KDM6JBspfDzs+XDvSoIpEXQ/Vi1BHTTNVEOKjN buiJ1ynYxhX8CTZcwsr5J680kyUwBlAqYUBnOKWR0OPKaMGhhBqset4grVSPxqOu/6 lXIGjKz88/h84Fu1rjAaDA098Cw8qy9/YqvXH497LUB6ccdu/W73IsyZRj6pbaRgWq AhQ0f+C1drIQA6qRWyrPnNec/XHoduZlkoXWEkP023alK/5VNeqVsDsr4atYW5WLm8 cY3Mp+eSQyrpFPzyDHqQNWlcQZgBlAsDP8PAOmVhXXB+Y1Mk7wcHXoQSlV0DBSZwrg Vau+TSrOtwW6g== Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 8C76B6225C for ; Tue, 27 Sep 2022 04:37:01 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="dufYZke4"; dkim-atps=neutral Received: from pendragon.ideasonboard.com (62-78-145-57.bb.dnainternet.fi [62.78.145.57]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 0877AE5 for ; Tue, 27 Sep 2022 04:37:00 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1664246221; bh=Snpjz2ADDNDiP6WIRVhkMO0Jamjr3LpMW7rfD3A86mY=; h=From:To:Subject:Date:In-Reply-To:References:From; b=dufYZke4dOwl/hJ4CdNtdUdMJg8tl6SYjRuGHmMx3+AwMhEnEroCnuxP/QFEbJx1o tZ9yjEcdMdWRPsFhCY3xrw/ZoHeWq5ErEoWNa4+vCn6iOi80zGeEfxm3SY4S6n5R66 hRCdP7ENecV9akvy/IJQQISxS0RbQdT0oDqEqYbU= To: libcamera-devel@lists.libcamera.org Date: Tue, 27 Sep 2022 05:36:10 +0300 Message-Id: <20220927023642.12341-2-laurent.pinchart@ideasonboard.com> X-Mailer: git-send-email 2.35.1 In-Reply-To: <20220927023642.12341-1-laurent.pinchart@ideasonboard.com> References: <20220927023642.12341-1-laurent.pinchart@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v5 01/33] ipa: ipu3: Fix style of Doxygen comment blocks X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Laurent Pinchart via libcamera-devel From: Laurent Pinchart Reply-To: Laurent Pinchart Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Fix various issues in Doxygen comment blocks: - \param requires an [in] or [out] tag - \param must come before the body of the documetation - Drop leftover \param for argument that has been removed - Rename coarseSearchStep to kCoarseSearchStep - White space and line wrap Signed-off-by: Laurent Pinchart Reviewed-by: Kieran Bingham Reviewed-by: Jacopo Mondi --- src/ipa/ipu3/algorithms/af.cpp | 23 ++++++++++------------- src/ipa/ipu3/algorithms/blc.cpp | 4 ++-- src/ipa/ipu3/algorithms/tone_mapping.cpp | 10 +++++----- 3 files changed, 17 insertions(+), 20 deletions(-) diff --git a/src/ipa/ipu3/algorithms/af.cpp b/src/ipa/ipu3/algorithms/af.cpp index 4835a0345931..106a7614e2c7 100644 --- a/src/ipa/ipu3/algorithms/af.cpp +++ b/src/ipa/ipu3/algorithms/af.cpp @@ -196,10 +196,10 @@ int Af::configure(IPAContext &context, const IPAConfigInfo &configInfo) /** * \brief AF coarse scan - * - * Find a near focused image using a coarse step. The step is determined by coarseSearchStep. - * * \param[in] context The shared IPA context + * + * Find a near focused image using a coarse step. The step is determined by + * kCoarseSearchStep. */ void Af::afCoarseScan(IPAContext &context) { @@ -223,10 +223,9 @@ void Af::afCoarseScan(IPAContext &context) /** * \brief AF fine scan + * \param[in] context The shared IPA context * * Find an optimum lens position with moving 1 step for each search. - * - * \param[in] context The shared IPA context */ void Af::afFineScan(IPAContext &context) { @@ -244,10 +243,9 @@ void Af::afFineScan(IPAContext &context) /** * \brief AF reset + * \param[in] context The shared IPA context * * Reset all the parameters to start over the AF process. - * - * \param[in] context The shared IPA context */ void Af::afReset(IPAContext &context) { @@ -268,7 +266,7 @@ void Af::afReset(IPAContext &context) /** * \brief AF variance comparison. * \param[in] context The IPA context - * \param min_step The VCM movement step. + * \param[in] min_step The VCM movement step. * * We always pick the largest variance to replace the previous one. The image * with a larger variance also indicates it is a clearer image than previous @@ -343,9 +341,8 @@ void Af::afIgnoreFrameReset() /** * \brief Estimate variance - * \param y_item The AF filter data set from the IPU3 statistics buffer - * \param len The quantity of table item entries which are valid to process - * \param isY1 Selects between filter Y1 or Y2 to calculate the variance + * \param[in] y_items The AF filter data set from the IPU3 statistics buffer + * \param[in] isY1 Selects between filter Y1 or Y2 to calculate the variance * * Calculate the mean of the data set provided by \a y_item, and then calculate * the variance of that data set from the mean. @@ -378,13 +375,13 @@ double Af::afEstimateVariance(Span y_items, bool isY1) /** * \brief Determine out-of-focus situation. - * \param context The IPA context. + * \param[in] context The IPA context. * * Out-of-focus means that the variance change rate for a focused and a new * variance is greater than a threshold. * * \return True if the variance threshold is crossed indicating lost focus, - * false otherwise. + * false otherwise */ bool Af::afIsOutOfFocus(IPAContext context) { diff --git a/src/ipa/ipu3/algorithms/blc.cpp b/src/ipa/ipu3/algorithms/blc.cpp index c561aa858b96..c32427c99549 100644 --- a/src/ipa/ipu3/algorithms/blc.cpp +++ b/src/ipa/ipu3/algorithms/blc.cpp @@ -38,8 +38,8 @@ BlackLevelCorrection::BlackLevelCorrection() /** * \brief Fill in the parameter structure, and enable black level correction - * \param context The shared IPA context - * \param params The IPU3 parameters + * \param[in] context The shared IPA context + * \param[out] params The IPU3 parameters * * Populate the IPU3 parameter structure with the correction values for each * channel and enable the corresponding ImgU block processing. diff --git a/src/ipa/ipu3/algorithms/tone_mapping.cpp b/src/ipa/ipu3/algorithms/tone_mapping.cpp index 49a5558b6faa..c21647e8c51b 100644 --- a/src/ipa/ipu3/algorithms/tone_mapping.cpp +++ b/src/ipa/ipu3/algorithms/tone_mapping.cpp @@ -49,8 +49,8 @@ int ToneMapping::configure(IPAContext &context, /** * \brief Fill in the parameter structure, and enable gamma control - * \param context The shared IPA context - * \param params The IPU3 parameters + * \param[in] context The shared IPA context + * \param[out] params The IPU3 parameters * * Populate the IPU3 parameter structure with our tone mapping look up table and * enable the gamma control module in the processing blocks. @@ -71,9 +71,9 @@ void ToneMapping::prepare([[maybe_unused]] IPAContext &context, /** * \brief Calculate the tone mapping look up table - * \param context The shared IPA context - * \param frameContext The current frame context - * \param stats The IPU3 statistics and ISP results + * \param[in] context The shared IPA context + * \param[in] frameContext The current frame context + * \param[in] stats The IPU3 statistics and ISP results * * The tone mapping look up table is generated as an inverse power curve from * our gamma setting. From patchwork Tue Sep 27 02:36:11 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Laurent Pinchart X-Patchwork-Id: 17413 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 7B97EC327E for ; Tue, 27 Sep 2022 02:37:06 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id D6C9A6226B; Tue, 27 Sep 2022 04:37:05 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1664246225; bh=IgPj/hYSSVv10M89+TU/t0ubOGRQoRKP/Tzn90mvxxg=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To: From; b=wK83TBhyZ37ao8F1Lkb+7NcpM6V7Y1WZ/GDnf1wq9Z/RGkQ3PjOp5y/bfrveLVESE zbVtIrwXrg3uBhA0hgc3v5sPThqGZeWXmu4LLtF1cszlbptux19dLOTDLyRFzidS5I XjecL8h8w1PD0oqsN/fsdQyhl2lO0q0yiyXenZhWFiQ8PL4g+fXT2cmn5eUuSKsd51 CvHlOlBgcFmQ+FU7iVr8x5gQUczTa5gTEs5LLz4F89DGcuC5RTg7CHFOGNy9dwhBD3 Gi1ALCzwz9lBx0zsCkj9YJdN4GBx6Ff4r/VbpKVTupF2K55RVw2wSjxKKePicMyC4h 4oHTurHewwlYQ== Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 110AC6225C for ; Tue, 27 Sep 2022 04:37:03 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="EOdkiwny"; dkim-atps=neutral Received: from pendragon.ideasonboard.com (62-78-145-57.bb.dnainternet.fi [62.78.145.57]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 756C9E5 for ; Tue, 27 Sep 2022 04:37:02 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1664246222; bh=IgPj/hYSSVv10M89+TU/t0ubOGRQoRKP/Tzn90mvxxg=; h=From:To:Subject:Date:In-Reply-To:References:From; b=EOdkiwnynJwP5eFU+nDcIfmnlFmIf7OhorXw29Tz2wz7bylmVhMN1Ger2fLM6/iq+ M0R8RvGbSZ0A5mBFziAzzsQJ60nuYnjCkRJ8QJ7Gyg4mtxDKjM6F7jmd8nIIdvEJfQ 14SGNJXKqa/y3v4HyE8v1JKWtOwKEbwLkrnjL6+8= To: libcamera-devel@lists.libcamera.org Date: Tue, 27 Sep 2022 05:36:11 +0300 Message-Id: <20220927023642.12341-3-laurent.pinchart@ideasonboard.com> X-Mailer: git-send-email 2.35.1 In-Reply-To: <20220927023642.12341-1-laurent.pinchart@ideasonboard.com> References: <20220927023642.12341-1-laurent.pinchart@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v5 02/33] ipa: ipu3: af: Pass context reference to afIsOutOfFocus() X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Laurent Pinchart via libcamera-devel From: Laurent Pinchart Reply-To: Laurent Pinchart Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Avoid copying the whole IPA context by passing a reference to the Af::afIsOutOfFocus() function. Signed-off-by: Laurent Pinchart Reviewed-by: Kieran Bingham Reviewed-by: Jacopo Mondi --- src/ipa/ipu3/algorithms/af.cpp | 2 +- src/ipa/ipu3/algorithms/af.h | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/src/ipa/ipu3/algorithms/af.cpp b/src/ipa/ipu3/algorithms/af.cpp index 106a7614e2c7..9127c24f1287 100644 --- a/src/ipa/ipu3/algorithms/af.cpp +++ b/src/ipa/ipu3/algorithms/af.cpp @@ -383,7 +383,7 @@ double Af::afEstimateVariance(Span y_items, bool isY1) * \return True if the variance threshold is crossed indicating lost focus, * false otherwise */ -bool Af::afIsOutOfFocus(IPAContext context) +bool Af::afIsOutOfFocus(IPAContext &context) { const uint32_t diff_var = std::abs(currentVariance_ - context.activeState.af.maxVariance); diff --git a/src/ipa/ipu3/algorithms/af.h b/src/ipa/ipu3/algorithms/af.h index ccf015f3f8f5..9b93594898bb 100644 --- a/src/ipa/ipu3/algorithms/af.h +++ b/src/ipa/ipu3/algorithms/af.h @@ -44,7 +44,7 @@ private: void afIgnoreFrameReset(); double afEstimateVariance(Span y_items, bool isY1); - bool afIsOutOfFocus(IPAContext context); + bool afIsOutOfFocus(IPAContext &context); /* VCM step configuration. It is the current setting of the VCM step. */ uint32_t focus_; From patchwork Tue Sep 27 02:36:12 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Laurent Pinchart X-Patchwork-Id: 17414 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id BC22FC0DA4 for ; Tue, 27 Sep 2022 02:37:07 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 5E4D56225F; Tue, 27 Sep 2022 04:37:07 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1664246227; bh=YvCdO4ZOpg68WAqLYToe+Em2qdR2OuhUd+ftMHfLec4=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To: From; b=thvEVp0eYvSBPwte+oiyCDAEjmm7VKprCXLSMVbFSjtcDm1IyY/TCI/Aijsd/c9GX 12spVaY0hCKpWthpwBGnuqQR8Rz7GBAG1yLgJarPYlcDySZWKNo5lgjiTOA3+9vAY/ kAKkme6uT6Uh7igwc4UVTWd2Mok/ZmyX/UU/sO0YNKoYWZ3dKt0jaZXgluRVOuUvK0 6I39Qn2Q3qGT11LigPCZUNsw5dRSpwMrS/3bex7GKH+pDpNqiUEk3J1RVNqBVJ05jL WGNR/NYIOU1MmPKDiCnhjJD5Lf7811gdsPBvhyyie5AIoRt7B5JOJmNm/JneECmMPc saHso5Sk9F6SA== Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 55FD06225E for ; Tue, 27 Sep 2022 04:37:04 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="LwPyyLA5"; dkim-atps=neutral Received: from pendragon.ideasonboard.com (62-78-145-57.bb.dnainternet.fi [62.78.145.57]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id D15B447C for ; Tue, 27 Sep 2022 04:37:03 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1664246224; bh=YvCdO4ZOpg68WAqLYToe+Em2qdR2OuhUd+ftMHfLec4=; h=From:To:Subject:Date:In-Reply-To:References:From; b=LwPyyLA5RLVcgssY7fpwnq6BnTZRawaiPZqiOhrrqX0vOaDochx8cKxnVmDDgdfWC 3qhq0wF4GGM2i25FypKa4hxsCmX0veyYAPwUVX/GgKC9ZRYF18I6Z5LwfVs4F8N+ZP EDWFwa3BmNLr8s3J2yg+ia/ami6zyt+yszSDEfHI= To: libcamera-devel@lists.libcamera.org Date: Tue, 27 Sep 2022 05:36:12 +0300 Message-Id: <20220927023642.12341-4-laurent.pinchart@ideasonboard.com> X-Mailer: git-send-email 2.35.1 In-Reply-To: <20220927023642.12341-1-laurent.pinchart@ideasonboard.com> References: <20220927023642.12341-1-laurent.pinchart@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v5 03/33] ipa: libipa: Pass a reference instead of pointer to Algorithm::process() X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Laurent Pinchart via libcamera-devel From: Laurent Pinchart Reply-To: Laurent Pinchart Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Frame contexts will become the core component of IPA modules, always available to functions of the algorithms. To indicate and prepare for this, turn the frame context pointer passed to Algorithm::process() into a reference. The RkISP1 IPA module doesn't use frame contexts yet, so pass a dummy context for now. While at it, drop an unneeded [[maybe_unused]] from Agc::process() and add a missing parameter documentation for the frameContext argument to Awb::process(). Signed-off-by: Laurent Pinchart Reviewed-by: Kieran Bingham Reviewed-by: Jacopo Mondi --- Changes since v5: - Drop extraneous \param frame for Awb::process() - Add missing \param frameContext for Awb::process() --- src/ipa/ipu3/algorithms/af.cpp | 2 +- src/ipa/ipu3/algorithms/af.h | 2 +- src/ipa/ipu3/algorithms/agc.cpp | 8 ++++---- src/ipa/ipu3/algorithms/agc.h | 4 ++-- src/ipa/ipu3/algorithms/awb.cpp | 2 +- src/ipa/ipu3/algorithms/awb.h | 2 +- src/ipa/ipu3/algorithms/tone_mapping.cpp | 2 +- src/ipa/ipu3/algorithms/tone_mapping.h | 2 +- src/ipa/ipu3/ipu3.cpp | 2 +- src/ipa/libipa/algorithm.h | 2 +- src/ipa/rkisp1/algorithms/agc.cpp | 3 ++- src/ipa/rkisp1/algorithms/agc.h | 2 +- src/ipa/rkisp1/algorithms/awb.cpp | 2 +- src/ipa/rkisp1/algorithms/awb.h | 2 +- src/ipa/rkisp1/rkisp1.cpp | 5 ++++- 15 files changed, 23 insertions(+), 19 deletions(-) diff --git a/src/ipa/ipu3/algorithms/af.cpp b/src/ipa/ipu3/algorithms/af.cpp index 9127c24f1287..5ab64bf88313 100644 --- a/src/ipa/ipu3/algorithms/af.cpp +++ b/src/ipa/ipu3/algorithms/af.cpp @@ -417,7 +417,7 @@ bool Af::afIsOutOfFocus(IPAContext &context) * * [1] Hill Climbing Algorithm, https://en.wikipedia.org/wiki/Hill_climbing */ -void Af::process(IPAContext &context, [[maybe_unused]] IPAFrameContext *frameContext, +void Af::process(IPAContext &context, [[maybe_unused]] IPAFrameContext &frameContext, const ipu3_uapi_stats_3a *stats) { /* Evaluate the AF buffer length */ diff --git a/src/ipa/ipu3/algorithms/af.h b/src/ipa/ipu3/algorithms/af.h index 9b93594898bb..29117e8bdd0d 100644 --- a/src/ipa/ipu3/algorithms/af.h +++ b/src/ipa/ipu3/algorithms/af.h @@ -32,7 +32,7 @@ public: void prepare(IPAContext &context, ipu3_uapi_params *params) override; int configure(IPAContext &context, const IPAConfigInfo &configInfo) override; - void process(IPAContext &context, IPAFrameContext *frameContext, + void process(IPAContext &context, IPAFrameContext &frameContext, const ipu3_uapi_stats_3a *stats) override; private: diff --git a/src/ipa/ipu3/algorithms/agc.cpp b/src/ipa/ipu3/algorithms/agc.cpp index ed4809d98007..f8ca29640a44 100644 --- a/src/ipa/ipu3/algorithms/agc.cpp +++ b/src/ipa/ipu3/algorithms/agc.cpp @@ -183,13 +183,13 @@ utils::Duration Agc::filterExposure(utils::Duration exposureValue) * \param[in] yGain The gain calculated based on the relative luminance target * \param[in] iqMeanGain The gain calculated based on the relative luminance target */ -void Agc::computeExposure(IPAContext &context, IPAFrameContext *frameContext, +void Agc::computeExposure(IPAContext &context, IPAFrameContext &frameContext, double yGain, double iqMeanGain) { const IPASessionConfiguration &configuration = context.configuration; /* Get the effective exposure and gain applied on the sensor. */ - uint32_t exposure = frameContext->sensor.exposure; - double analogueGain = frameContext->sensor.gain; + uint32_t exposure = frameContext.sensor.exposure; + double analogueGain = frameContext.sensor.gain; /* Use the highest of the two gain estimates. */ double evGain = std::max(yGain, iqMeanGain); @@ -323,7 +323,7 @@ double Agc::estimateLuminance(IPAActiveState &activeState, * Identify the current image brightness, and use that to estimate the optimal * new exposure and gain for the scene. */ -void Agc::process(IPAContext &context, [[maybe_unused]] IPAFrameContext *frameContext, +void Agc::process(IPAContext &context, IPAFrameContext &frameContext, const ipu3_uapi_stats_3a *stats) { /* diff --git a/src/ipa/ipu3/algorithms/agc.h b/src/ipa/ipu3/algorithms/agc.h index 105ae0f2aac6..876fbbb6b585 100644 --- a/src/ipa/ipu3/algorithms/agc.h +++ b/src/ipa/ipu3/algorithms/agc.h @@ -28,14 +28,14 @@ public: ~Agc() = default; int configure(IPAContext &context, const IPAConfigInfo &configInfo) override; - void process(IPAContext &context, IPAFrameContext *frameContext, + void process(IPAContext &context, IPAFrameContext &frameContext, const ipu3_uapi_stats_3a *stats) override; private: double measureBrightness(const ipu3_uapi_stats_3a *stats, const ipu3_uapi_grid_config &grid) const; utils::Duration filterExposure(utils::Duration currentExposure); - void computeExposure(IPAContext &context, IPAFrameContext *frameContext, + void computeExposure(IPAContext &context, IPAFrameContext &frameContext, double yGain, double iqMeanGain); double estimateLuminance(IPAActiveState &activeState, const ipu3_uapi_grid_config &grid, diff --git a/src/ipa/ipu3/algorithms/awb.cpp b/src/ipa/ipu3/algorithms/awb.cpp index b658ee546b87..128f5d9d5351 100644 --- a/src/ipa/ipu3/algorithms/awb.cpp +++ b/src/ipa/ipu3/algorithms/awb.cpp @@ -387,7 +387,7 @@ void Awb::calculateWBGains(const ipu3_uapi_stats_3a *stats) /** * \copydoc libcamera::ipa::Algorithm::process */ -void Awb::process(IPAContext &context, [[maybe_unused]] IPAFrameContext *frameContext, +void Awb::process(IPAContext &context, [[maybe_unused]] IPAFrameContext &frameContext, const ipu3_uapi_stats_3a *stats) { calculateWBGains(stats); diff --git a/src/ipa/ipu3/algorithms/awb.h b/src/ipa/ipu3/algorithms/awb.h index 0acd21480845..ebbb2d58be7e 100644 --- a/src/ipa/ipu3/algorithms/awb.h +++ b/src/ipa/ipu3/algorithms/awb.h @@ -40,7 +40,7 @@ public: int configure(IPAContext &context, const IPAConfigInfo &configInfo) override; void prepare(IPAContext &context, ipu3_uapi_params *params) override; - void process(IPAContext &context, IPAFrameContext *frameContext, + void process(IPAContext &context, IPAFrameContext &frameContext, const ipu3_uapi_stats_3a *stats) override; private: diff --git a/src/ipa/ipu3/algorithms/tone_mapping.cpp b/src/ipa/ipu3/algorithms/tone_mapping.cpp index c21647e8c51b..24faba9be2b2 100644 --- a/src/ipa/ipu3/algorithms/tone_mapping.cpp +++ b/src/ipa/ipu3/algorithms/tone_mapping.cpp @@ -78,7 +78,7 @@ void ToneMapping::prepare([[maybe_unused]] IPAContext &context, * The tone mapping look up table is generated as an inverse power curve from * our gamma setting. */ -void ToneMapping::process(IPAContext &context, [[maybe_unused]] IPAFrameContext *frameContext, +void ToneMapping::process(IPAContext &context, [[maybe_unused]] IPAFrameContext &frameContext, [[maybe_unused]] const ipu3_uapi_stats_3a *stats) { /* diff --git a/src/ipa/ipu3/algorithms/tone_mapping.h b/src/ipa/ipu3/algorithms/tone_mapping.h index d7d4800628f2..5be62f6ee9bd 100644 --- a/src/ipa/ipu3/algorithms/tone_mapping.h +++ b/src/ipa/ipu3/algorithms/tone_mapping.h @@ -20,7 +20,7 @@ public: int configure(IPAContext &context, const IPAConfigInfo &configInfo) override; void prepare(IPAContext &context, ipu3_uapi_params *params) override; - void process(IPAContext &context, IPAFrameContext *frameContext, + void process(IPAContext &context, IPAFrameContext &frameContext, const ipu3_uapi_stats_3a *stats) override; private: diff --git a/src/ipa/ipu3/ipu3.cpp b/src/ipa/ipu3/ipu3.cpp index e3c1e8160759..f0c92507533c 100644 --- a/src/ipa/ipu3/ipu3.cpp +++ b/src/ipa/ipu3/ipu3.cpp @@ -616,7 +616,7 @@ void IPAIPU3::processStatsBuffer(const uint32_t frame, ControlList ctrls(controls::controls); for (auto const &algo : algorithms()) - algo->process(context_, &frameContext, stats); + algo->process(context_, frameContext, stats); setControls(frame); diff --git a/src/ipa/libipa/algorithm.h b/src/ipa/libipa/algorithm.h index ccc659a63e3b..0fe3d772963a 100644 --- a/src/ipa/libipa/algorithm.h +++ b/src/ipa/libipa/algorithm.h @@ -49,7 +49,7 @@ public: } virtual void process([[maybe_unused]] typename Module::Context &context, - [[maybe_unused]] typename Module::FrameContext *frameContext, + [[maybe_unused]] typename Module::FrameContext &frameContext, [[maybe_unused]] const typename Module::Stats *stats) { } diff --git a/src/ipa/rkisp1/algorithms/agc.cpp b/src/ipa/rkisp1/algorithms/agc.cpp index 708c36ed1af8..112cdaaaf6c7 100644 --- a/src/ipa/rkisp1/algorithms/agc.cpp +++ b/src/ipa/rkisp1/algorithms/agc.cpp @@ -275,13 +275,14 @@ double Agc::measureBrightness(const rkisp1_cif_isp_hist_stat *hist) const /** * \brief Process RkISP1 statistics, and run AGC operations * \param[in] context The shared IPA context + * \param[in] frameContext The current frame context * \param[in] stats The RKISP1 statistics and ISP results * * Identify the current image brightness, and use that to estimate the optimal * new exposure and gain for the scene. */ void Agc::process(IPAContext &context, - [[maybe_unused]] IPAFrameContext *frameContext, + [[maybe_unused]] IPAFrameContext &frameContext, const rkisp1_stat_buffer *stats) { const rkisp1_cif_isp_stat *params = &stats->params; diff --git a/src/ipa/rkisp1/algorithms/agc.h b/src/ipa/rkisp1/algorithms/agc.h index 1c9818b7db2a..6a5723ecbcf8 100644 --- a/src/ipa/rkisp1/algorithms/agc.h +++ b/src/ipa/rkisp1/algorithms/agc.h @@ -27,7 +27,7 @@ public: int configure(IPAContext &context, const IPACameraSensorInfo &configInfo) override; void prepare(IPAContext &context, rkisp1_params_cfg *params) override; - void process(IPAContext &context, IPAFrameContext *frameContext, + void process(IPAContext &context, IPAFrameContext &frameContext, const rkisp1_stat_buffer *stats) override; private: diff --git a/src/ipa/rkisp1/algorithms/awb.cpp b/src/ipa/rkisp1/algorithms/awb.cpp index b12df21de4aa..2bc5d3aa16ab 100644 --- a/src/ipa/rkisp1/algorithms/awb.cpp +++ b/src/ipa/rkisp1/algorithms/awb.cpp @@ -152,7 +152,7 @@ void Awb::queueRequest(IPAContext &context, * \copydoc libcamera::ipa::Algorithm::process */ void Awb::process([[maybe_unused]] IPAContext &context, - [[maybe_unused]] IPAFrameContext *frameCtx, + [[maybe_unused]] IPAFrameContext &frameCtx, const rkisp1_stat_buffer *stats) { const rkisp1_cif_isp_stat *params = &stats->params; diff --git a/src/ipa/rkisp1/algorithms/awb.h b/src/ipa/rkisp1/algorithms/awb.h index 4917267bb99d..fc221acea617 100644 --- a/src/ipa/rkisp1/algorithms/awb.h +++ b/src/ipa/rkisp1/algorithms/awb.h @@ -23,7 +23,7 @@ public: void prepare(IPAContext &context, rkisp1_params_cfg *params) override; void queueRequest(IPAContext &context, const uint32_t frame, const ControlList &controls) override; - void process(IPAContext &context, IPAFrameContext *frameCtx, + void process(IPAContext &context, IPAFrameContext &frameCtx, const rkisp1_stat_buffer *stats) override; private: diff --git a/src/ipa/rkisp1/rkisp1.cpp b/src/ipa/rkisp1/rkisp1.cpp index 3ff1eb48f605..3bc6d4dd2784 100644 --- a/src/ipa/rkisp1/rkisp1.cpp +++ b/src/ipa/rkisp1/rkisp1.cpp @@ -326,8 +326,11 @@ void IPARkISP1::processStatsBuffer(const uint32_t frame, const uint32_t bufferId unsigned int aeState = 0; + /* \todo Obtain the frame context to pass to process from the FCQueue */ + IPAFrameContext frameContext; + for (auto const &algo : algorithms()) - algo->process(context_, nullptr, stats); + algo->process(context_, frameContext, stats); setControls(frame); From patchwork Tue Sep 27 02:36:13 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Laurent Pinchart X-Patchwork-Id: 17415 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 5E800C327E for ; Tue, 27 Sep 2022 02:37:08 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 09E6B6226F; Tue, 27 Sep 2022 04:37:08 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1664246228; bh=KfSOcY/vBJimasK2A6AaFUzNycSQYT3y1QvbpFZkxnk=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To: From; b=MiCf914zmNofFd9pW3EPsL8zv0H49g4zACTatJn+jZFNahS798Z1j7b8coWDLm3my xHkWQbFNIQQXY2vHLCGQpjtgWSfNo7o5M+qM2xytVHadpXEoq+OUzqqPnzX9kyHdrn L0nfGjxwtFYiBgtTG9rO+s5VhEP2T1u2rVflG4ptj4IIAngEObyH/842zHrEdP6bRq EezdYYQPTi7fWzezoDrBQ4kV2KiUxupZ85Gaz67M54rETYAlfPyavDaOMnlLgp0CYE v6FkCR6at7rY0QQr8om5bjZeNqwFpXbNqfQ2SoPqG6FPi/8zZs4l+Gc9GDjt8phMtD I4w0bN+CZKJ+Q== Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id C26046225F for ; Tue, 27 Sep 2022 04:37:05 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="AesojHfz"; dkim-atps=neutral Received: from pendragon.ideasonboard.com (62-78-145-57.bb.dnainternet.fi [62.78.145.57]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 2AB22E5; Tue, 27 Sep 2022 04:37:05 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1664246225; bh=KfSOcY/vBJimasK2A6AaFUzNycSQYT3y1QvbpFZkxnk=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=AesojHfztaijrQGefMr0YrZIeTlq4KYNsplTrp4v6Zm49FPa91IMlZo9P7KC1jR3n czc8Xsh+JCj4lxck+zyYROstUbNPG0JZIwfEcDVunmE5QsEIwYG4NRjaPlvAlehZVF 1Sgw2mZUB2lb3/4b2QSmWIRIrcD3Cv2kIAjZ/Cz8= To: libcamera-devel@lists.libcamera.org Date: Tue, 27 Sep 2022 05:36:13 +0300 Message-Id: <20220927023642.12341-5-laurent.pinchart@ideasonboard.com> X-Mailer: git-send-email 2.35.1 In-Reply-To: <20220927023642.12341-1-laurent.pinchart@ideasonboard.com> References: <20220927023642.12341-1-laurent.pinchart@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v5 04/33] ipa: libipa: Introduce FrameContextQueue X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Laurent Pinchart via libcamera-devel From: Laurent Pinchart Reply-To: Laurent Pinchart Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" From: Umang Jain Introduce a common implementation in libipa to represent the queue of frame contexts. Signed-off-by: Umang Jain Signed-off-by: Kieran Bingham Signed-off-by: Jacopo Mondi Reviewed-by: Kieran Bingham --- Changes since v4: - Documentation improvements - White space fixes - Rework class documentation - Rename FCQueue::init() to FCQueue::alloc() - Only clear frame number in clear() - Add more \todo comments Changes since v3: - Split the IPU3 changes to a separate patch - Use composition instead of inheritance - Use vector instead of array for backend storage - Make the queue size dynamic - Rename initialise() to init() --- src/ipa/libipa/fc_queue.cpp | 118 ++++++++++++++++++++++++++++++++++++ src/ipa/libipa/fc_queue.h | 108 +++++++++++++++++++++++++++++++++ src/ipa/libipa/meson.build | 2 + 3 files changed, 228 insertions(+) create mode 100644 src/ipa/libipa/fc_queue.cpp create mode 100644 src/ipa/libipa/fc_queue.h diff --git a/src/ipa/libipa/fc_queue.cpp b/src/ipa/libipa/fc_queue.cpp new file mode 100644 index 000000000000..57a369512554 --- /dev/null +++ b/src/ipa/libipa/fc_queue.cpp @@ -0,0 +1,118 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2022, Google Inc. + * + * fc_queue.cpp - IPA Frame context queue + */ + +#include "fc_queue.h" + +#include + +namespace libcamera { + +LOG_DEFINE_CATEGORY(FCQueue) + +namespace ipa { + +/** + * \file fc_queue.h + * \brief Queue of per-frame contexts + */ + +/** + * \class FCQueue + * \brief A support class for managing FrameContext instances in IPA modules + * \tparam FrameContext The IPA module-specific FrameContext derived class type + * + * Along with the Module and Algorithm classes, the frame context queue is a + * core component of the libipa infrastructure. It stores per-frame contexts + * used by the Algorithm operations. By centralizing the lifetime management of + * the contexts and implementing safeguards against underflows and overflows, it + * simplifies IPA modules and improves their reliability. + * + * The queue references frame contexts by a monotonically increasing sequence + * number. The FCQueue design assumes that this number matches both the sequence + * number of the corresponding frame, as generated by the camera sensor, and the + * sequence number of the request. This allows IPA modules to obtain the frame + * context from any location where a request or a frame is available. + * + * A frame context normally begins its lifetime when the corresponding request + * is queued, way before the frame is captured by the camera sensor. IPA modules + * allocate the context from the queue at that point, calling alloc() using the + * request number. The queue initializes the context, and the IPA module then + * populates it with data from the request. The context can be later retrieved + * with a call to get(), typically when the IPA module is requested to provide + * sensor or ISP parameters or receives statistics for a frame. The frame number + * is used at that point to identify the context. + * + * If an application fails to queue requests to the camera fast enough, frames + * may be produced by the camera sensor and processed by the IPA module without + * a corresponding request having been queued to the IPA module. This creates an + * underrun condition, where the IPA module will try to get a frame context that + * hasn't been allocated. In this case, the get() function will allocate and + * initialize a context for the frame, and log a message. Algorithms will not + * apply the controls associated with the late request, but should otherwise + * behave correctly. + * + * \todo Mark the frame context with a per-frame control error flag in case of + * underrun, and research how algorithms should handle this. + * + * At its core, the queue uses a circular buffer to avoid dynamic memory + * allocation at runtime. The buffer is pre-allocated with a maximum number of + * entries when the FCQueue instance is constructed. Entries are initialized on + * first use by alloc() or, in underrun conditions, get(). The queue is not + * allowed to overflow, which must be ensured by pipeline handlers never + * queuing more in-flight requests to the IPA module than the queue size. If an + * overflow condition is detected, the queue will log a fatal error. + */ + +/** + * \fn FCQueue::FCQueue(unsigned int size) + * \brief Construct a frame contexts queue of a specified size + * \param[in] size The number of contexts in the queue + */ + +/** + * \fn FCQueue::clear() + * \brief Clear the contexts queue + * + * IPA modules must clear the frame context queue at the beginning of a new + * streaming session, in IPAModule::start(). + * + * \todo Fix any issue this may cause with requests queued before the camera is + * started. + */ + +/** + * \fn FCQueue::alloc(uint32_t frame) + * \brief Allocate and return a FrameContext for the \a frame + * \param[in] frame The frame context sequence number + * + * The first call to obtain a FrameContext from the FCQueue should be handled + * through this function. The FrameContext will be initialised, if not + * initialised already, and returned to the caller. + * + * If the FrameContext was already initialized for this \a frame, a warning will + * be reported and the previously initialized FrameContext is returned. + * + * Frame contexts are expected to be initialised when a Request is first passed + * to the IPA module in IPAModule::queueRequest(). + * + * \return A reference to the FrameContext for sequence \a frame + */ + +/** + * \fn FCQueue::get(uint32_t frame) + * \brief Obtain the FrameContext for the \a frame + * \param[in] frame The frame context sequence number + * + * If the FrameContext is not correctly initialised for the \a frame, it will be + * initialised. + * + * \return A reference to the FrameContext for sequence \a frame + */ + +} /* namespace ipa */ + +} /* namespace libcamera */ diff --git a/src/ipa/libipa/fc_queue.h b/src/ipa/libipa/fc_queue.h new file mode 100644 index 000000000000..4f5cb5d35253 --- /dev/null +++ b/src/ipa/libipa/fc_queue.h @@ -0,0 +1,108 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2022, Google Inc. + * + * fc_queue.h - IPA Frame context queue + */ + +#pragma once + +#include + +#include + +namespace libcamera { + +LOG_DECLARE_CATEGORY(FCQueue) + +namespace ipa { + +template +class FCQueue +{ +public: + FCQueue(unsigned int size) + : contexts_(size) + { + } + + void clear() + { + for (FrameContext &ctx : contexts_) + ctx.frame = 0; + } + + FrameContext &alloc(const uint32_t frame) + { + FrameContext &frameContext = contexts_[frame % contexts_.size()]; + + /* + * Do not re-initialise if a get() call has already fetched this + * frame context to preseve the context. + * + * \todo If the the sequence number of the context to initialise + * is smaller than the sequence number of the queue slot to use, + * it means that we had a serious request underrun and more + * frames than the queue size has been produced since the last + * time the application has queued a request. Does this deserve + * an error condition ? + */ + if (frame != 0 && frame <= frameContext.frame) + LOG(FCQueue, Warning) + << "Frame " << frame << " already initialised"; + else + init(frameContext, frame); + + return frameContext; + } + + FrameContext &get(uint32_t frame) + { + FrameContext &frameContext = contexts_[frame % contexts_.size()]; + + /* + * If the IPA algorithms try to access a frame context slot which + * has been already overwritten by a newer context, it means the + * frame context queue has overflowed and the desired context + * has been forever lost. The pipeline handler shall avoid + * queueing more requests to the IPA than the frame context + * queue size. + */ + if (frame < frameContext.frame) + LOG(FCQueue, Fatal) << "Frame context for " << frame + << " has been overwritten by " + << frameContext.frame; + + if (frame == frameContext.frame) + return frameContext; + + /* + * The frame context has been retrieved before it was + * initialised through the initialise() call. This indicates an + * algorithm attempted to access a Frame context before it was + * queued to the IPA. Controls applied for this request may be + * left unhandled. + * + * \todo Set an error flag for per-frame control errors. + */ + LOG(FCQueue, Warning) + << "Obtained an uninitialised FrameContext for " << frame; + + init(frameContext, frame); + + return frameContext; + } + +private: + void init(FrameContext &frameContext, const uint32_t frame) + { + frameContext = {}; + frameContext.frame = frame; + } + + std::vector contexts_; +}; + +} /* namespace ipa */ + +} /* namespace libcamera */ diff --git a/src/ipa/libipa/meson.build b/src/ipa/libipa/meson.build index fb894bc614af..016b8e0ec9be 100644 --- a/src/ipa/libipa/meson.build +++ b/src/ipa/libipa/meson.build @@ -3,6 +3,7 @@ libipa_headers = files([ 'algorithm.h', 'camera_sensor_helper.h', + 'fc_queue.h', 'histogram.h', 'module.h', ]) @@ -10,6 +11,7 @@ libipa_headers = files([ libipa_sources = files([ 'algorithm.cpp', 'camera_sensor_helper.cpp', + 'fc_queue.cpp', 'histogram.cpp', 'module.cpp', ]) From patchwork Tue Sep 27 02:36:14 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Laurent Pinchart X-Patchwork-Id: 17416 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 2E042C0DA4 for ; Tue, 27 Sep 2022 02:37:11 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id ADEFC62279; Tue, 27 Sep 2022 04:37:10 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1664246230; bh=+QKdoT3OJ4NYtAL13TtxZDMryhUFdej5MafYs5tGmcU=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To: From; b=XtmhJhsfH2Q/ixb57bD7IPT+lMnaZIMMZEd1HEpnGrG2PHLaL5xg7zedVqRdQsxxz NKtYrGUwTW5eFU8OzRmCxCJBsKT7g2aQdCsiL4XNQCTE/txmrqr9Tx+IysE5mFyZsd JbwUGV9/WdvnHZeAB7ls/Q+srPuJ1T1gr6gYVnlWCikOt1gN88sco9TeXXZxav31lk CgK6Nvtz0YfXKvP3TVIlsGuz/RBWS9Y7riAqDB6WhcHyAUD3iPt62NJTVG/3S+crTm wiL9mXAx5OsCvrnRYCIPQ0O4HP4I6ZUZXKWaD9A9WLZpk5O9uqN7BwM5xHIa3mZ3xB B/5LZ61jZD1MQ== Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 7169462270 for ; Tue, 27 Sep 2022 04:37:07 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="VVRVqXvR"; dkim-atps=neutral Received: from pendragon.ideasonboard.com (62-78-145-57.bb.dnainternet.fi [62.78.145.57]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id E0EE647C; Tue, 27 Sep 2022 04:37:06 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1664246227; bh=+QKdoT3OJ4NYtAL13TtxZDMryhUFdej5MafYs5tGmcU=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=VVRVqXvRiK5iFXq965XYyq8kjBcRruHjrRqSgHmhuVOruEBiDq8GkMvUEKu7hqatX LAJWLeyxzbxexEy4cT6s04QHrP9TNPt+a4A7mW945QZJxbRi/RJrabgX3FfutLhd86 S5+lgSGyWAvIb9ZsqKKE4fCVzelv5tnPIDD9xKQc= To: libcamera-devel@lists.libcamera.org Date: Tue, 27 Sep 2022 05:36:14 +0300 Message-Id: <20220927023642.12341-6-laurent.pinchart@ideasonboard.com> X-Mailer: git-send-email 2.35.1 In-Reply-To: <20220927023642.12341-1-laurent.pinchart@ideasonboard.com> References: <20220927023642.12341-1-laurent.pinchart@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v5 05/33] ipa: libipa: Provide a common base for frame contexts X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Laurent Pinchart via libcamera-devel From: Laurent Pinchart Reply-To: Laurent Pinchart Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" From: Kieran Bingham Provide a common FrameContext as a base for IPA modules to inherit from. This will allow having a common set of parameters for every frame context managed by the FCQueue implementation. Signed-off-by: Kieran Bingham Signed-off-by: Laurent Pinchart Reviewed-by: Kieran Bingham Reviewed-by: Jacopo Mondi --- Changes since v4: - Move documentation paragraph from previous patch to this one - Rename IPAFrameContext to FrameContext in documentation Changes since v3: - Rename structure from IPAFrameContext to FrameContext - Move changes to IPA modules to separate patches - Make the FrameContext::frame member private --- src/ipa/libipa/fc_queue.cpp | 22 ++++++++++++++++++++++ src/ipa/libipa/fc_queue.h | 10 ++++++++++ 2 files changed, 32 insertions(+) diff --git a/src/ipa/libipa/fc_queue.cpp b/src/ipa/libipa/fc_queue.cpp index 57a369512554..e812faa505a5 100644 --- a/src/ipa/libipa/fc_queue.cpp +++ b/src/ipa/libipa/fc_queue.cpp @@ -20,6 +20,24 @@ namespace ipa { * \brief Queue of per-frame contexts */ +/** + * \struct FrameContext + * \brief Context for a frame + * + * The frame context stores data specific to a single frame processed by the + * IPA module. Each frame processed by the IPA module has a context associated + * with it, accessible through the Frame Context Queue. + * + * Fields in the frame context should reflect values and controls associated + * with the specific frame as requested by the application, and as configured by + * the hardware. Fields can be read by algorithms to determine if they should + * update any specific action for this frame, and finally to update the metadata + * control lists when the frame is fully completed. + * + * \var FrameContext::frame + * \brief The frame number + */ + /** * \class FCQueue * \brief A support class for managing FrameContext instances in IPA modules @@ -65,6 +83,10 @@ namespace ipa { * allowed to overflow, which must be ensured by pipeline handlers never * queuing more in-flight requests to the IPA module than the queue size. If an * overflow condition is detected, the queue will log a fatal error. + * + * IPA module-specific frame context implementations shall inherit from the + * FrameContext base class to support the minimum required features for a + * FrameContext. */ /** diff --git a/src/ipa/libipa/fc_queue.h b/src/ipa/libipa/fc_queue.h index 4f5cb5d35253..a589e7e1031b 100644 --- a/src/ipa/libipa/fc_queue.h +++ b/src/ipa/libipa/fc_queue.h @@ -7,6 +7,7 @@ #pragma once +#include #include #include @@ -17,6 +18,15 @@ LOG_DECLARE_CATEGORY(FCQueue) namespace ipa { +template +class FCQueue; + +struct FrameContext { +private: + template friend class FCQueue; + uint32_t frame; +}; + template class FCQueue { From patchwork Tue Sep 27 02:36:15 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Laurent Pinchart X-Patchwork-Id: 17417 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 0AD75C327E for ; Tue, 27 Sep 2022 02:37:12 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id A9D4462277; Tue, 27 Sep 2022 04:37:11 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1664246231; bh=m+XJXnp2vANOKxIdcanFln1KLsPm/9pmSWc+6NeV9GI=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To: From; b=g4AbPc80Li8aUg498etUfN2OJ9f1PRDPnJslNYHMSDrFQ86+ygQaVD1gncsu6g63K GanxUv3pJrfLaTDteqOpDQKvMNHhf09ubplqxpuJ1SPWEGBvU7Y6xFEMWIqcz2vJXV cgj/XAE1vMXsq1mY6rjgQlwUw77KMbP0NvHXhMH1U0i9eeYUGsa8KHzrMiX1/wuOXO UBqH0Pr6XbWFLzoDUFk9HEpCOS3qLi51vxYsg1uop1zsRUlV3/4pD/5SiYBgUCDIKB NQibkN22owPcExdGh6AJYVaoaLfYaswjsnyqSQJV+4csprRmTKjSaV9KQgq4rM5D6z xuTQRcUs1Cv1Q== Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id EE0DA62272 for ; Tue, 27 Sep 2022 04:37:08 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="j9HP0eMh"; dkim-atps=neutral Received: from pendragon.ideasonboard.com (62-78-145-57.bb.dnainternet.fi [62.78.145.57]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 6C9C247C; Tue, 27 Sep 2022 04:37:08 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1664246228; bh=m+XJXnp2vANOKxIdcanFln1KLsPm/9pmSWc+6NeV9GI=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=j9HP0eMhK7Qr9Q0Nal3fKm+cCqUm997adAc5Zu+8HReHzXS2wsfb6GqErckMY8fJj dHf5MXrdLLOhCDWEuEADlYA1zp6J/7cztBOx6f1JGG61/UvWb0S/rjlEYaPqhGGiFM HFPQkJBP1TxMiZRk28xkSt3h41MhQmVh96hQylhM= To: libcamera-devel@lists.libcamera.org Date: Tue, 27 Sep 2022 05:36:15 +0300 Message-Id: <20220927023642.12341-7-laurent.pinchart@ideasonboard.com> X-Mailer: git-send-email 2.35.1 In-Reply-To: <20220927023642.12341-1-laurent.pinchart@ideasonboard.com> References: <20220927023642.12341-1-laurent.pinchart@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v5 06/33] ipa: libipa: algorithm: prepare(): Pass frame and frame Context X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Laurent Pinchart via libcamera-devel From: Laurent Pinchart Reply-To: Laurent Pinchart Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" From: Kieran Bingham Pass the current frame number, and the current FrameContext for calls to prepare. Signed-off-by: Kieran Bingham Signed-off-by: Jacopo Mondi Reviewed-by: Umang Jain Signed-off-by: Laurent Pinchart Reviewed-by: Kieran Bingham Reviewed-by: Jacopo Mondi --- Changes since v3: - Use a const uint32_t instead of an unsigned int for the frame number --- src/ipa/ipu3/algorithms/af.cpp | 5 ++++- src/ipa/ipu3/algorithms/af.h | 4 +++- src/ipa/ipu3/algorithms/awb.cpp | 5 ++++- src/ipa/ipu3/algorithms/awb.h | 4 +++- src/ipa/ipu3/algorithms/blc.cpp | 6 +++++- src/ipa/ipu3/algorithms/blc.h | 4 +++- src/ipa/ipu3/algorithms/tone_mapping.cpp | 4 ++++ src/ipa/ipu3/algorithms/tone_mapping.h | 3 ++- src/ipa/ipu3/ipu3.cpp | 4 +++- src/ipa/libipa/algorithm.cpp | 2 ++ src/ipa/libipa/algorithm.h | 3 +++ src/ipa/rkisp1/algorithms/agc.cpp | 5 ++++- src/ipa/rkisp1/algorithms/agc.h | 4 +++- src/ipa/rkisp1/algorithms/awb.cpp | 5 ++++- src/ipa/rkisp1/algorithms/awb.h | 4 +++- src/ipa/rkisp1/algorithms/blc.cpp | 2 ++ src/ipa/rkisp1/algorithms/blc.h | 4 +++- src/ipa/rkisp1/algorithms/cproc.cpp | 2 ++ src/ipa/rkisp1/algorithms/cproc.h | 4 +++- src/ipa/rkisp1/algorithms/dpcc.cpp | 2 ++ src/ipa/rkisp1/algorithms/dpcc.h | 4 +++- src/ipa/rkisp1/algorithms/dpf.cpp | 4 +++- src/ipa/rkisp1/algorithms/dpf.h | 4 +++- src/ipa/rkisp1/algorithms/filter.cpp | 5 ++++- src/ipa/rkisp1/algorithms/filter.h | 4 +++- src/ipa/rkisp1/algorithms/gsl.cpp | 2 ++ src/ipa/rkisp1/algorithms/gsl.h | 4 +++- src/ipa/rkisp1/algorithms/lsc.cpp | 2 ++ src/ipa/rkisp1/algorithms/lsc.h | 4 +++- src/ipa/rkisp1/rkisp1.cpp | 5 ++++- 30 files changed, 92 insertions(+), 22 deletions(-) diff --git a/src/ipa/ipu3/algorithms/af.cpp b/src/ipa/ipu3/algorithms/af.cpp index 5ab64bf88313..bc01b232ec59 100644 --- a/src/ipa/ipu3/algorithms/af.cpp +++ b/src/ipa/ipu3/algorithms/af.cpp @@ -116,7 +116,10 @@ Af::Af() /** * \copydoc libcamera::ipa::Algorithm::prepare */ -void Af::prepare(IPAContext &context, ipu3_uapi_params *params) +void Af::prepare(IPAContext &context, + [[maybe_unused]] const uint32_t frame, + [[maybe_unused]] IPAFrameContext &frameContext, + ipu3_uapi_params *params) { const struct ipu3_uapi_grid_config &grid = context.configuration.af.afGrid; params->acc_param.af.grid_cfg = grid; diff --git a/src/ipa/ipu3/algorithms/af.h b/src/ipa/ipu3/algorithms/af.h index 29117e8bdd0d..fbfd171b6465 100644 --- a/src/ipa/ipu3/algorithms/af.h +++ b/src/ipa/ipu3/algorithms/af.h @@ -30,8 +30,10 @@ public: Af(); ~Af() = default; - void prepare(IPAContext &context, ipu3_uapi_params *params) override; int configure(IPAContext &context, const IPAConfigInfo &configInfo) override; + void prepare(IPAContext &context, const uint32_t frame, + IPAFrameContext &frameContext, + ipu3_uapi_params *params) override; void process(IPAContext &context, IPAFrameContext &frameContext, const ipu3_uapi_stats_3a *stats) override; diff --git a/src/ipa/ipu3/algorithms/awb.cpp b/src/ipa/ipu3/algorithms/awb.cpp index 128f5d9d5351..e6e2951b8abb 100644 --- a/src/ipa/ipu3/algorithms/awb.cpp +++ b/src/ipa/ipu3/algorithms/awb.cpp @@ -429,7 +429,10 @@ constexpr uint16_t Awb::gainValue(double gain) /** * \copydoc libcamera::ipa::Algorithm::prepare */ -void Awb::prepare(IPAContext &context, ipu3_uapi_params *params) +void Awb::prepare(IPAContext &context, + [[maybe_unused]] const uint32_t frame, + [[maybe_unused]] IPAFrameContext &frameContext, + ipu3_uapi_params *params) { /* * Green saturation thresholds are reduced because we are using the diff --git a/src/ipa/ipu3/algorithms/awb.h b/src/ipa/ipu3/algorithms/awb.h index ebbb2d58be7e..e9e816928fb4 100644 --- a/src/ipa/ipu3/algorithms/awb.h +++ b/src/ipa/ipu3/algorithms/awb.h @@ -39,7 +39,9 @@ public: ~Awb(); int configure(IPAContext &context, const IPAConfigInfo &configInfo) override; - void prepare(IPAContext &context, ipu3_uapi_params *params) override; + void prepare(IPAContext &context, const uint32_t frame, + IPAFrameContext &frameContext, + ipu3_uapi_params *params) override; void process(IPAContext &context, IPAFrameContext &frameContext, const ipu3_uapi_stats_3a *stats) override; diff --git a/src/ipa/ipu3/algorithms/blc.cpp b/src/ipa/ipu3/algorithms/blc.cpp index c32427c99549..e838072a344c 100644 --- a/src/ipa/ipu3/algorithms/blc.cpp +++ b/src/ipa/ipu3/algorithms/blc.cpp @@ -39,13 +39,17 @@ BlackLevelCorrection::BlackLevelCorrection() /** * \brief Fill in the parameter structure, and enable black level correction * \param[in] context The shared IPA context + * \param[in] frame The frame context sequence number + * \param[in] frameContext The FrameContext for this frame * \param[out] params The IPU3 parameters * * Populate the IPU3 parameter structure with the correction values for each * channel and enable the corresponding ImgU block processing. */ void BlackLevelCorrection::prepare([[maybe_unused]] IPAContext &context, - ipu3_uapi_params *params) + [[maybe_unused]] const uint32_t frame, + [[maybe_unused]] IPAFrameContext &frameContext, + ipu3_uapi_params *params) { /* * The Optical Black Level correction values diff --git a/src/ipa/ipu3/algorithms/blc.h b/src/ipa/ipu3/algorithms/blc.h index d8da1748baba..292bf67b02ff 100644 --- a/src/ipa/ipu3/algorithms/blc.h +++ b/src/ipa/ipu3/algorithms/blc.h @@ -18,7 +18,9 @@ class BlackLevelCorrection : public Algorithm public: BlackLevelCorrection(); - void prepare(IPAContext &context, ipu3_uapi_params *params) override; + void prepare(IPAContext &context, const uint32_t frame, + IPAFrameContext &frameContext, + ipu3_uapi_params *params) override; }; } /* namespace ipa::ipu3::algorithms */ diff --git a/src/ipa/ipu3/algorithms/tone_mapping.cpp b/src/ipa/ipu3/algorithms/tone_mapping.cpp index 24faba9be2b2..10a2fb1935dc 100644 --- a/src/ipa/ipu3/algorithms/tone_mapping.cpp +++ b/src/ipa/ipu3/algorithms/tone_mapping.cpp @@ -50,12 +50,16 @@ int ToneMapping::configure(IPAContext &context, /** * \brief Fill in the parameter structure, and enable gamma control * \param[in] context The shared IPA context + * \param[in] frame The frame context sequence number + * \param[in] frameContext The FrameContext for this frame * \param[out] params The IPU3 parameters * * Populate the IPU3 parameter structure with our tone mapping look up table and * enable the gamma control module in the processing blocks. */ void ToneMapping::prepare([[maybe_unused]] IPAContext &context, + [[maybe_unused]] const uint32_t frame, + [[maybe_unused]] IPAFrameContext &frameContext, ipu3_uapi_params *params) { /* Copy the calculated LUT into the parameters buffer. */ diff --git a/src/ipa/ipu3/algorithms/tone_mapping.h b/src/ipa/ipu3/algorithms/tone_mapping.h index 5be62f6ee9bd..c24a7e485aaf 100644 --- a/src/ipa/ipu3/algorithms/tone_mapping.h +++ b/src/ipa/ipu3/algorithms/tone_mapping.h @@ -19,7 +19,8 @@ public: ToneMapping(); int configure(IPAContext &context, const IPAConfigInfo &configInfo) override; - void prepare(IPAContext &context, ipu3_uapi_params *params) override; + void prepare(IPAContext &context, const uint32_t frame, + IPAFrameContext &frameContext, ipu3_uapi_params *params) override; void process(IPAContext &context, IPAFrameContext &frameContext, const ipu3_uapi_stats_3a *stats) override; diff --git a/src/ipa/ipu3/ipu3.cpp b/src/ipa/ipu3/ipu3.cpp index f0c92507533c..ac6f66afecb5 100644 --- a/src/ipa/ipu3/ipu3.cpp +++ b/src/ipa/ipu3/ipu3.cpp @@ -572,8 +572,10 @@ void IPAIPU3::fillParamsBuffer(const uint32_t frame, const uint32_t bufferId) */ params->use = {}; + IPAFrameContext &frameContext = context_.frameContexts[frame % kMaxFrameContexts]; + for (auto const &algo : algorithms()) - algo->prepare(context_, params); + algo->prepare(context_, frame, frameContext, params); paramsBufferReady.emit(frame); } diff --git a/src/ipa/libipa/algorithm.cpp b/src/ipa/libipa/algorithm.cpp index 38200e57b780..0992e5de8dd5 100644 --- a/src/ipa/libipa/algorithm.cpp +++ b/src/ipa/libipa/algorithm.cpp @@ -70,6 +70,8 @@ namespace ipa { * \fn Algorithm::prepare() * \brief Fill the \a params buffer with ISP processing parameters for a frame * \param[in] context The shared IPA context + * \param[in] frame The frame context sequence number + * \param[in] frameContext The FrameContext for this frame * \param[out] params The ISP specific parameters. * * This function is called for every frame when the camera is running before it diff --git a/src/ipa/libipa/algorithm.h b/src/ipa/libipa/algorithm.h index 0fe3d772963a..cde2e6399e8e 100644 --- a/src/ipa/libipa/algorithm.h +++ b/src/ipa/libipa/algorithm.h @@ -7,6 +7,7 @@ #pragma once #include +#include #include #include @@ -38,6 +39,8 @@ public: } virtual void prepare([[maybe_unused]] typename Module::Context &context, + [[maybe_unused]] const uint32_t frame, + [[maybe_unused]] typename Module::FrameContext &frameContext, [[maybe_unused]] typename Module::Params *params) { } diff --git a/src/ipa/rkisp1/algorithms/agc.cpp b/src/ipa/rkisp1/algorithms/agc.cpp index 112cdaaaf6c7..603fad183b87 100644 --- a/src/ipa/rkisp1/algorithms/agc.cpp +++ b/src/ipa/rkisp1/algorithms/agc.cpp @@ -323,7 +323,10 @@ void Agc::process(IPAContext &context, /** * \copydoc libcamera::ipa::Algorithm::prepare */ -void Agc::prepare(IPAContext &context, rkisp1_params_cfg *params) +void Agc::prepare(IPAContext &context, + [[maybe_unused]] const uint32_t frame, + [[maybe_unused]] IPAFrameContext &frameContext, + rkisp1_params_cfg *params) { if (context.frameContext.frameCount > 0) return; diff --git a/src/ipa/rkisp1/algorithms/agc.h b/src/ipa/rkisp1/algorithms/agc.h index 6a5723ecbcf8..8724cb602158 100644 --- a/src/ipa/rkisp1/algorithms/agc.h +++ b/src/ipa/rkisp1/algorithms/agc.h @@ -26,7 +26,9 @@ public: ~Agc() = default; int configure(IPAContext &context, const IPACameraSensorInfo &configInfo) override; - void prepare(IPAContext &context, rkisp1_params_cfg *params) override; + void prepare(IPAContext &context, const uint32_t frame, + IPAFrameContext &frameContext, + rkisp1_params_cfg *params) override; void process(IPAContext &context, IPAFrameContext &frameContext, const rkisp1_stat_buffer *stats) override; diff --git a/src/ipa/rkisp1/algorithms/awb.cpp b/src/ipa/rkisp1/algorithms/awb.cpp index 2bc5d3aa16ab..da22a0970a4f 100644 --- a/src/ipa/rkisp1/algorithms/awb.cpp +++ b/src/ipa/rkisp1/algorithms/awb.cpp @@ -74,7 +74,10 @@ uint32_t Awb::estimateCCT(double red, double green, double blue) /** * \copydoc libcamera::ipa::Algorithm::prepare */ -void Awb::prepare(IPAContext &context, rkisp1_params_cfg *params) +void Awb::prepare(IPAContext &context, + [[maybe_unused]] const uint32_t frame, + [[maybe_unused]] IPAFrameContext &frameContext, + rkisp1_params_cfg *params) { params->others.awb_gain_config.gain_green_b = 256 * context.frameContext.awb.gains.green; params->others.awb_gain_config.gain_blue = 256 * context.frameContext.awb.gains.blue; diff --git a/src/ipa/rkisp1/algorithms/awb.h b/src/ipa/rkisp1/algorithms/awb.h index fc221acea617..7ae6c7a9bb10 100644 --- a/src/ipa/rkisp1/algorithms/awb.h +++ b/src/ipa/rkisp1/algorithms/awb.h @@ -20,7 +20,9 @@ public: ~Awb() = default; int configure(IPAContext &context, const IPACameraSensorInfo &configInfo) override; - void prepare(IPAContext &context, rkisp1_params_cfg *params) override; + void prepare(IPAContext &context, const uint32_t frame, + IPAFrameContext &frameContext, + rkisp1_params_cfg *params) override; void queueRequest(IPAContext &context, const uint32_t frame, const ControlList &controls) override; void process(IPAContext &context, IPAFrameContext &frameCtx, diff --git a/src/ipa/rkisp1/algorithms/blc.cpp b/src/ipa/rkisp1/algorithms/blc.cpp index a58569fa2dc2..c45a317ada08 100644 --- a/src/ipa/rkisp1/algorithms/blc.cpp +++ b/src/ipa/rkisp1/algorithms/blc.cpp @@ -66,6 +66,8 @@ int BlackLevelCorrection::init([[maybe_unused]] IPAContext &context, * \copydoc libcamera::ipa::Algorithm::prepare */ void BlackLevelCorrection::prepare(IPAContext &context, + [[maybe_unused]] const uint32_t frame, + [[maybe_unused]] IPAFrameContext &frameContext, rkisp1_params_cfg *params) { if (context.frameContext.frameCount > 0) diff --git a/src/ipa/rkisp1/algorithms/blc.h b/src/ipa/rkisp1/algorithms/blc.h index 5fc3a80fb638..0b1a2d436869 100644 --- a/src/ipa/rkisp1/algorithms/blc.h +++ b/src/ipa/rkisp1/algorithms/blc.h @@ -20,7 +20,9 @@ public: ~BlackLevelCorrection() = default; int init(IPAContext &context, const YamlObject &tuningData) override; - void prepare(IPAContext &context, rkisp1_params_cfg *params) override; + void prepare(IPAContext &context, const uint32_t frame, + IPAFrameContext &frameContext, + rkisp1_params_cfg *params) override; private: bool tuningParameters_; diff --git a/src/ipa/rkisp1/algorithms/cproc.cpp b/src/ipa/rkisp1/algorithms/cproc.cpp index bca5ab6907d6..c0505c68765b 100644 --- a/src/ipa/rkisp1/algorithms/cproc.cpp +++ b/src/ipa/rkisp1/algorithms/cproc.cpp @@ -71,6 +71,8 @@ void ColorProcessing::queueRequest(IPAContext &context, * \copydoc libcamera::ipa::Algorithm::prepare */ void ColorProcessing::prepare(IPAContext &context, + [[maybe_unused]] const uint32_t frame, + [[maybe_unused]] IPAFrameContext &frameContext, rkisp1_params_cfg *params) { auto &cproc = context.frameContext.cproc; diff --git a/src/ipa/rkisp1/algorithms/cproc.h b/src/ipa/rkisp1/algorithms/cproc.h index 4b7e4064d7e8..02c3d9c95488 100644 --- a/src/ipa/rkisp1/algorithms/cproc.h +++ b/src/ipa/rkisp1/algorithms/cproc.h @@ -23,7 +23,9 @@ public: void queueRequest(IPAContext &context, const uint32_t frame, const ControlList &controls) override; - void prepare(IPAContext &context, rkisp1_params_cfg *params) override; + void prepare(IPAContext &context, const uint32_t frame, + IPAFrameContext &frameContext, + rkisp1_params_cfg *params) override; }; } /* namespace ipa::rkisp1::algorithms */ diff --git a/src/ipa/rkisp1/algorithms/dpcc.cpp b/src/ipa/rkisp1/algorithms/dpcc.cpp index fb2f01cdaa3c..5030526769c7 100644 --- a/src/ipa/rkisp1/algorithms/dpcc.cpp +++ b/src/ipa/rkisp1/algorithms/dpcc.cpp @@ -259,6 +259,8 @@ int DefectPixelClusterCorrection::init([[maybe_unused]] IPAContext &context, * \copydoc libcamera::ipa::Algorithm::prepare */ void DefectPixelClusterCorrection::prepare(IPAContext &context, + [[maybe_unused]] const uint32_t frame, + [[maybe_unused]] IPAFrameContext &frameContext, rkisp1_params_cfg *params) { if (context.frameContext.frameCount > 0) diff --git a/src/ipa/rkisp1/algorithms/dpcc.h b/src/ipa/rkisp1/algorithms/dpcc.h index a363f7bee0a7..894c0249f35d 100644 --- a/src/ipa/rkisp1/algorithms/dpcc.h +++ b/src/ipa/rkisp1/algorithms/dpcc.h @@ -20,7 +20,9 @@ public: ~DefectPixelClusterCorrection() = default; int init(IPAContext &context, const YamlObject &tuningData) override; - void prepare(IPAContext &context, rkisp1_params_cfg *params) override; + void prepare(IPAContext &context, const uint32_t frame, + IPAFrameContext &frameContext, + rkisp1_params_cfg *params) override; private: bool initialized_; diff --git a/src/ipa/rkisp1/algorithms/dpf.cpp b/src/ipa/rkisp1/algorithms/dpf.cpp index c93c8361b5da..8286f83facc1 100644 --- a/src/ipa/rkisp1/algorithms/dpf.cpp +++ b/src/ipa/rkisp1/algorithms/dpf.cpp @@ -206,7 +206,9 @@ void Dpf::queueRequest(IPAContext &context, /** * \copydoc libcamera::ipa::Algorithm::prepare */ -void Dpf::prepare(IPAContext &context, rkisp1_params_cfg *params) +void Dpf::prepare(IPAContext &context, [[maybe_unused]] const uint32_t frame, + [[maybe_unused]] IPAFrameContext &frameContext, + rkisp1_params_cfg *params) { if (!initialized_) return; diff --git a/src/ipa/rkisp1/algorithms/dpf.h b/src/ipa/rkisp1/algorithms/dpf.h index 128ebd5e02e3..656d78dee67c 100644 --- a/src/ipa/rkisp1/algorithms/dpf.h +++ b/src/ipa/rkisp1/algorithms/dpf.h @@ -24,7 +24,9 @@ public: int init(IPAContext &context, const YamlObject &tuningData) override; void queueRequest(IPAContext &context, const uint32_t frame, const ControlList &controls) override; - void prepare(IPAContext &context, rkisp1_params_cfg *params) override; + void prepare(IPAContext &context, const uint32_t frame, + IPAFrameContext &frameContext, + rkisp1_params_cfg *params) override; private: bool initialized_; diff --git a/src/ipa/rkisp1/algorithms/filter.cpp b/src/ipa/rkisp1/algorithms/filter.cpp index 8ca10fd1ee9d..02562339d008 100644 --- a/src/ipa/rkisp1/algorithms/filter.cpp +++ b/src/ipa/rkisp1/algorithms/filter.cpp @@ -85,7 +85,10 @@ void Filter::queueRequest(IPAContext &context, /** * \copydoc libcamera::ipa::Algorithm::prepare */ -void Filter::prepare(IPAContext &context, rkisp1_params_cfg *params) +void Filter::prepare(IPAContext &context, + [[maybe_unused]] const uint32_t frame, + [[maybe_unused]] IPAFrameContext &frameContext, + rkisp1_params_cfg *params) { auto &filter = context.frameContext.filter; diff --git a/src/ipa/rkisp1/algorithms/filter.h b/src/ipa/rkisp1/algorithms/filter.h index 9eb170eb7da1..ac69657e2f0f 100644 --- a/src/ipa/rkisp1/algorithms/filter.h +++ b/src/ipa/rkisp1/algorithms/filter.h @@ -23,7 +23,9 @@ public: void queueRequest(IPAContext &context, const uint32_t frame, const ControlList &controls) override; - void prepare(IPAContext &context, rkisp1_params_cfg *params) override; + void prepare(IPAContext &context, const uint32_t frame, + IPAFrameContext &frameContext, + rkisp1_params_cfg *params) override; }; } /* namespace ipa::rkisp1::algorithms */ diff --git a/src/ipa/rkisp1/algorithms/gsl.cpp b/src/ipa/rkisp1/algorithms/gsl.cpp index 2fd1a23d3a9b..dfc76b3d8ffd 100644 --- a/src/ipa/rkisp1/algorithms/gsl.cpp +++ b/src/ipa/rkisp1/algorithms/gsl.cpp @@ -119,6 +119,8 @@ int GammaSensorLinearization::init([[maybe_unused]] IPAContext &context, * \copydoc libcamera::ipa::Algorithm::prepare */ void GammaSensorLinearization::prepare(IPAContext &context, + [[maybe_unused]] const uint32_t frame, + [[maybe_unused]] IPAFrameContext &frameContext, rkisp1_params_cfg *params) { if (context.frameContext.frameCount > 0) diff --git a/src/ipa/rkisp1/algorithms/gsl.h b/src/ipa/rkisp1/algorithms/gsl.h index db287dc280dc..5024b6835949 100644 --- a/src/ipa/rkisp1/algorithms/gsl.h +++ b/src/ipa/rkisp1/algorithms/gsl.h @@ -20,7 +20,9 @@ public: ~GammaSensorLinearization() = default; int init(IPAContext &context, const YamlObject &tuningData) override; - void prepare(IPAContext &context, rkisp1_params_cfg *params) override; + void prepare(IPAContext &context, const uint32_t frame, + IPAFrameContext &frameContext, + rkisp1_params_cfg *params) override; private: bool initialized_; diff --git a/src/ipa/rkisp1/algorithms/lsc.cpp b/src/ipa/rkisp1/algorithms/lsc.cpp index b9efc063ab2e..e4b04136dbaf 100644 --- a/src/ipa/rkisp1/algorithms/lsc.cpp +++ b/src/ipa/rkisp1/algorithms/lsc.cpp @@ -133,6 +133,8 @@ int LensShadingCorrection::configure(IPAContext &context, * \copydoc libcamera::ipa::Algorithm::prepare */ void LensShadingCorrection::prepare(IPAContext &context, + [[maybe_unused]] const uint32_t frame, + [[maybe_unused]] IPAFrameContext &frameContext, rkisp1_params_cfg *params) { if (context.frameContext.frameCount > 0) diff --git a/src/ipa/rkisp1/algorithms/lsc.h b/src/ipa/rkisp1/algorithms/lsc.h index f68602c005c4..da957d3e3103 100644 --- a/src/ipa/rkisp1/algorithms/lsc.h +++ b/src/ipa/rkisp1/algorithms/lsc.h @@ -21,7 +21,9 @@ public: int init(IPAContext &context, const YamlObject &tuningData) override; int configure(IPAContext &context, const IPACameraSensorInfo &configInfo) override; - void prepare(IPAContext &context, rkisp1_params_cfg *params) override; + void prepare(IPAContext &context, const uint32_t frame, + IPAFrameContext &frameContext, + rkisp1_params_cfg *params) override; private: bool initialized_; diff --git a/src/ipa/rkisp1/rkisp1.cpp b/src/ipa/rkisp1/rkisp1.cpp index 3bc6d4dd2784..0932c73b14e4 100644 --- a/src/ipa/rkisp1/rkisp1.cpp +++ b/src/ipa/rkisp1/rkisp1.cpp @@ -298,6 +298,9 @@ void IPARkISP1::queueRequest(const uint32_t frame, const ControlList &controls) void IPARkISP1::fillParamsBuffer(const uint32_t frame, const uint32_t bufferId) { + /* \todo Obtain the frame context to pass to process from the FCQueue */ + IPAFrameContext frameContext; + rkisp1_params_cfg *params = reinterpret_cast( mappedBuffers_.at(bufferId).planes()[0].data()); @@ -306,7 +309,7 @@ void IPARkISP1::fillParamsBuffer(const uint32_t frame, const uint32_t bufferId) memset(params, 0, sizeof(*params)); for (auto const &algo : algorithms()) - algo->prepare(context_, params); + algo->prepare(context_, frame, frameContext, params); paramsBufferReady.emit(frame); context_.frameContext.frameCount++; From patchwork Tue Sep 27 02:36:16 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Laurent Pinchart X-Patchwork-Id: 17418 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id ABEAAC327F for ; Tue, 27 Sep 2022 02:37:12 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 41D7262276; Tue, 27 Sep 2022 04:37:12 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1664246232; bh=sKMpnVRQAsZWXWpIbDuP2QlQPwGHjk5R1xtRvVEcrg4=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To: From; b=0hkV9HEvuc7kn+dDIBpg/jpFI0xANIBwWGpcBVgTaoCeuZEI5p0D6Qn7PQxE2QxkL i1YMMD40Y1/vaWRpOGii/OJ8VczBtL1JMu1W+cSZvSVIgwkvSVaVehksi6XFgqC9OO rpZPynDIbP18BrZFXU4pRQDKRruel9PqChA6JwLQwwdot6cQc6/guaflAmII0hCKpo wPTYlacSxSHKhOAmJZRLWHHKw4DwKP8Yy/tjlmXImtmQQW/VIbNTsI8UVABpcEH+W/ Hm/rTNeDSDRrLgRdZgb7eeohyHmaFAmGPh2FTRrVDKMfukfZJuRh3m9uRz1Z4MjQtu PZB43MrI9e2vw== Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 5113762277 for ; Tue, 27 Sep 2022 04:37:10 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="ZCDeuZ42"; dkim-atps=neutral Received: from pendragon.ideasonboard.com (62-78-145-57.bb.dnainternet.fi [62.78.145.57]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id C5E86E5; Tue, 27 Sep 2022 04:37:09 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1664246230; bh=sKMpnVRQAsZWXWpIbDuP2QlQPwGHjk5R1xtRvVEcrg4=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=ZCDeuZ420ExJSo2pnjp0LiqwePSF8E8wZk7rdvcZYrsOlE9u7dtbDU5QalupwTOMt YvhHCord6tT5izMsbPB3wc/bgTwutARgw5Xx3rKA0YTjReNbvgPnRo/pKHdWGAkdFR Xw0cq+eC369nY71VNM/e9k/Oe7qVlBFIqhWJp0aM= To: libcamera-devel@lists.libcamera.org Date: Tue, 27 Sep 2022 05:36:16 +0300 Message-Id: <20220927023642.12341-8-laurent.pinchart@ideasonboard.com> X-Mailer: git-send-email 2.35.1 In-Reply-To: <20220927023642.12341-1-laurent.pinchart@ideasonboard.com> References: <20220927023642.12341-1-laurent.pinchart@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v5 07/33] ipa: libipa: algorithm: process(): Pass frame number X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Laurent Pinchart via libcamera-devel From: Laurent Pinchart Reply-To: Laurent Pinchart Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" From: Kieran Bingham Pass the frame number of the current frame being processed. Signed-off-by: Kieran Bingham Signed-off-by: Jacopo Mondi Reviewed-by: Umang Jain Signed-off-by: Laurent Pinchart Reviewed-by: Kieran Bingham Reviewed-by: Jacopo Mondi --- Changes since v4: - Update the documentation of ipa::rkisp1::Awb::process() Changes since v3: - Use a const uint32_t instead of an unsigned int for the frame number - Update the documentation of ipa::rkisp1::Agc::process() --- src/ipa/ipu3/algorithms/af.cpp | 4 +++- src/ipa/ipu3/algorithms/af.h | 3 ++- src/ipa/ipu3/algorithms/agc.cpp | 4 +++- src/ipa/ipu3/algorithms/agc.h | 3 ++- src/ipa/ipu3/algorithms/awb.cpp | 3 ++- src/ipa/ipu3/algorithms/awb.h | 3 ++- src/ipa/ipu3/algorithms/tone_mapping.cpp | 4 +++- src/ipa/ipu3/algorithms/tone_mapping.h | 3 ++- src/ipa/ipu3/ipu3.cpp | 2 +- src/ipa/libipa/algorithm.cpp | 1 + src/ipa/libipa/algorithm.h | 1 + src/ipa/rkisp1/algorithms/agc.cpp | 3 ++- src/ipa/rkisp1/algorithms/agc.h | 3 ++- src/ipa/rkisp1/algorithms/awb.cpp | 1 + src/ipa/rkisp1/algorithms/awb.h | 3 ++- src/ipa/rkisp1/rkisp1.cpp | 2 +- 16 files changed, 30 insertions(+), 13 deletions(-) diff --git a/src/ipa/ipu3/algorithms/af.cpp b/src/ipa/ipu3/algorithms/af.cpp index bc01b232ec59..b00a34f9e090 100644 --- a/src/ipa/ipu3/algorithms/af.cpp +++ b/src/ipa/ipu3/algorithms/af.cpp @@ -406,6 +406,7 @@ bool Af::afIsOutOfFocus(IPAContext &context) /** * \brief Determine the max contrast image and lens position. * \param[in] context The IPA context. + * \param[in] frame The frame context sequence number * \param[in] frameContext The current frame context * \param[in] stats The statistics buffer of IPU3. * @@ -420,7 +421,8 @@ bool Af::afIsOutOfFocus(IPAContext &context) * * [1] Hill Climbing Algorithm, https://en.wikipedia.org/wiki/Hill_climbing */ -void Af::process(IPAContext &context, [[maybe_unused]] IPAFrameContext &frameContext, +void Af::process(IPAContext &context, [[maybe_unused]] const uint32_t frame, + [[maybe_unused]] IPAFrameContext &frameContext, const ipu3_uapi_stats_3a *stats) { /* Evaluate the AF buffer length */ diff --git a/src/ipa/ipu3/algorithms/af.h b/src/ipa/ipu3/algorithms/af.h index fbfd171b6465..89d37ac18615 100644 --- a/src/ipa/ipu3/algorithms/af.h +++ b/src/ipa/ipu3/algorithms/af.h @@ -34,7 +34,8 @@ public: void prepare(IPAContext &context, const uint32_t frame, IPAFrameContext &frameContext, ipu3_uapi_params *params) override; - void process(IPAContext &context, IPAFrameContext &frameContext, + void process(IPAContext &context, const uint32_t frame, + IPAFrameContext &frameContext, const ipu3_uapi_stats_3a *stats) override; private: diff --git a/src/ipa/ipu3/algorithms/agc.cpp b/src/ipa/ipu3/algorithms/agc.cpp index f8ca29640a44..a1a3c38ffe84 100644 --- a/src/ipa/ipu3/algorithms/agc.cpp +++ b/src/ipa/ipu3/algorithms/agc.cpp @@ -317,13 +317,15 @@ double Agc::estimateLuminance(IPAActiveState &activeState, /** * \brief Process IPU3 statistics, and run AGC operations * \param[in] context The shared IPA context + * \param[in] frame The current frame sequence number * \param[in] frameContext The current frame context * \param[in] stats The IPU3 statistics and ISP results * * Identify the current image brightness, and use that to estimate the optimal * new exposure and gain for the scene. */ -void Agc::process(IPAContext &context, IPAFrameContext &frameContext, +void Agc::process(IPAContext &context, [[maybe_unused]] const uint32_t frame, + IPAFrameContext &frameContext, const ipu3_uapi_stats_3a *stats) { /* diff --git a/src/ipa/ipu3/algorithms/agc.h b/src/ipa/ipu3/algorithms/agc.h index 876fbbb6b585..59b4b9843c2f 100644 --- a/src/ipa/ipu3/algorithms/agc.h +++ b/src/ipa/ipu3/algorithms/agc.h @@ -28,7 +28,8 @@ public: ~Agc() = default; int configure(IPAContext &context, const IPAConfigInfo &configInfo) override; - void process(IPAContext &context, IPAFrameContext &frameContext, + void process(IPAContext &context, const uint32_t frame, + IPAFrameContext &frameContext, const ipu3_uapi_stats_3a *stats) override; private: diff --git a/src/ipa/ipu3/algorithms/awb.cpp b/src/ipa/ipu3/algorithms/awb.cpp index e6e2951b8abb..0dbd7d4c374f 100644 --- a/src/ipa/ipu3/algorithms/awb.cpp +++ b/src/ipa/ipu3/algorithms/awb.cpp @@ -387,7 +387,8 @@ void Awb::calculateWBGains(const ipu3_uapi_stats_3a *stats) /** * \copydoc libcamera::ipa::Algorithm::process */ -void Awb::process(IPAContext &context, [[maybe_unused]] IPAFrameContext &frameContext, +void Awb::process(IPAContext &context, [[maybe_unused]] const uint32_t frame, + [[maybe_unused]] IPAFrameContext &frameContext, const ipu3_uapi_stats_3a *stats) { calculateWBGains(stats); diff --git a/src/ipa/ipu3/algorithms/awb.h b/src/ipa/ipu3/algorithms/awb.h index e9e816928fb4..28e2d38a711c 100644 --- a/src/ipa/ipu3/algorithms/awb.h +++ b/src/ipa/ipu3/algorithms/awb.h @@ -42,7 +42,8 @@ public: void prepare(IPAContext &context, const uint32_t frame, IPAFrameContext &frameContext, ipu3_uapi_params *params) override; - void process(IPAContext &context, IPAFrameContext &frameContext, + void process(IPAContext &context, const uint32_t frame, + IPAFrameContext &frameContext, const ipu3_uapi_stats_3a *stats) override; private: diff --git a/src/ipa/ipu3/algorithms/tone_mapping.cpp b/src/ipa/ipu3/algorithms/tone_mapping.cpp index 10a2fb1935dc..eac3d4064443 100644 --- a/src/ipa/ipu3/algorithms/tone_mapping.cpp +++ b/src/ipa/ipu3/algorithms/tone_mapping.cpp @@ -76,13 +76,15 @@ void ToneMapping::prepare([[maybe_unused]] IPAContext &context, /** * \brief Calculate the tone mapping look up table * \param[in] context The shared IPA context + * \param[in] frame The current frame sequence number * \param[in] frameContext The current frame context * \param[in] stats The IPU3 statistics and ISP results * * The tone mapping look up table is generated as an inverse power curve from * our gamma setting. */ -void ToneMapping::process(IPAContext &context, [[maybe_unused]] IPAFrameContext &frameContext, +void ToneMapping::process(IPAContext &context, [[maybe_unused]] const uint32_t frame, + [[maybe_unused]] IPAFrameContext &frameContext, [[maybe_unused]] const ipu3_uapi_stats_3a *stats) { /* diff --git a/src/ipa/ipu3/algorithms/tone_mapping.h b/src/ipa/ipu3/algorithms/tone_mapping.h index c24a7e485aaf..822e5168df82 100644 --- a/src/ipa/ipu3/algorithms/tone_mapping.h +++ b/src/ipa/ipu3/algorithms/tone_mapping.h @@ -21,7 +21,8 @@ public: int configure(IPAContext &context, const IPAConfigInfo &configInfo) override; void prepare(IPAContext &context, const uint32_t frame, IPAFrameContext &frameContext, ipu3_uapi_params *params) override; - void process(IPAContext &context, IPAFrameContext &frameContext, + void process(IPAContext &context, const uint32_t frame, + IPAFrameContext &frameContext, const ipu3_uapi_stats_3a *stats) override; private: diff --git a/src/ipa/ipu3/ipu3.cpp b/src/ipa/ipu3/ipu3.cpp index ac6f66afecb5..e5a763fd2b08 100644 --- a/src/ipa/ipu3/ipu3.cpp +++ b/src/ipa/ipu3/ipu3.cpp @@ -618,7 +618,7 @@ void IPAIPU3::processStatsBuffer(const uint32_t frame, ControlList ctrls(controls::controls); for (auto const &algo : algorithms()) - algo->process(context_, frameContext, stats); + algo->process(context_, frame, frameContext, stats); setControls(frame); diff --git a/src/ipa/libipa/algorithm.cpp b/src/ipa/libipa/algorithm.cpp index 0992e5de8dd5..30eab67f71fc 100644 --- a/src/ipa/libipa/algorithm.cpp +++ b/src/ipa/libipa/algorithm.cpp @@ -103,6 +103,7 @@ namespace ipa { * \fn Algorithm::process() * \brief Process ISP statistics, and run algorithm operations * \param[in] context The shared IPA context + * \param[in] frame The frame context sequence number * \param[in] frameContext The current frame's context * \param[in] stats The IPA statistics and ISP results * diff --git a/src/ipa/libipa/algorithm.h b/src/ipa/libipa/algorithm.h index cde2e6399e8e..ee93d260ca3f 100644 --- a/src/ipa/libipa/algorithm.h +++ b/src/ipa/libipa/algorithm.h @@ -52,6 +52,7 @@ public: } virtual void process([[maybe_unused]] typename Module::Context &context, + [[maybe_unused]] const uint32_t frame, [[maybe_unused]] typename Module::FrameContext &frameContext, [[maybe_unused]] const typename Module::Stats *stats) { diff --git a/src/ipa/rkisp1/algorithms/agc.cpp b/src/ipa/rkisp1/algorithms/agc.cpp index 603fad183b87..f9c33f81a074 100644 --- a/src/ipa/rkisp1/algorithms/agc.cpp +++ b/src/ipa/rkisp1/algorithms/agc.cpp @@ -275,13 +275,14 @@ double Agc::measureBrightness(const rkisp1_cif_isp_hist_stat *hist) const /** * \brief Process RkISP1 statistics, and run AGC operations * \param[in] context The shared IPA context + * \param[in] frame The frame context sequence number * \param[in] frameContext The current frame context * \param[in] stats The RKISP1 statistics and ISP results * * Identify the current image brightness, and use that to estimate the optimal * new exposure and gain for the scene. */ -void Agc::process(IPAContext &context, +void Agc::process(IPAContext &context, [[maybe_unused]] const uint32_t frame, [[maybe_unused]] IPAFrameContext &frameContext, const rkisp1_stat_buffer *stats) { diff --git a/src/ipa/rkisp1/algorithms/agc.h b/src/ipa/rkisp1/algorithms/agc.h index 8724cb602158..f115ba2ed85c 100644 --- a/src/ipa/rkisp1/algorithms/agc.h +++ b/src/ipa/rkisp1/algorithms/agc.h @@ -29,7 +29,8 @@ public: void prepare(IPAContext &context, const uint32_t frame, IPAFrameContext &frameContext, rkisp1_params_cfg *params) override; - void process(IPAContext &context, IPAFrameContext &frameContext, + void process(IPAContext &context, const uint32_t frame, + IPAFrameContext &frameContext, const rkisp1_stat_buffer *stats) override; private: diff --git a/src/ipa/rkisp1/algorithms/awb.cpp b/src/ipa/rkisp1/algorithms/awb.cpp index da22a0970a4f..aefba6382807 100644 --- a/src/ipa/rkisp1/algorithms/awb.cpp +++ b/src/ipa/rkisp1/algorithms/awb.cpp @@ -155,6 +155,7 @@ void Awb::queueRequest(IPAContext &context, * \copydoc libcamera::ipa::Algorithm::process */ void Awb::process([[maybe_unused]] IPAContext &context, + [[maybe_unused]] const uint32_t frame, [[maybe_unused]] IPAFrameContext &frameCtx, const rkisp1_stat_buffer *stats) { diff --git a/src/ipa/rkisp1/algorithms/awb.h b/src/ipa/rkisp1/algorithms/awb.h index 7ae6c7a9bb10..806e11a2ec93 100644 --- a/src/ipa/rkisp1/algorithms/awb.h +++ b/src/ipa/rkisp1/algorithms/awb.h @@ -25,7 +25,8 @@ public: rkisp1_params_cfg *params) override; void queueRequest(IPAContext &context, const uint32_t frame, const ControlList &controls) override; - void process(IPAContext &context, IPAFrameContext &frameCtx, + void process(IPAContext &context, const uint32_t frame, + IPAFrameContext &frameCtx, const rkisp1_stat_buffer *stats) override; private: diff --git a/src/ipa/rkisp1/rkisp1.cpp b/src/ipa/rkisp1/rkisp1.cpp index 0932c73b14e4..b2593a584a1a 100644 --- a/src/ipa/rkisp1/rkisp1.cpp +++ b/src/ipa/rkisp1/rkisp1.cpp @@ -333,7 +333,7 @@ void IPARkISP1::processStatsBuffer(const uint32_t frame, const uint32_t bufferId IPAFrameContext frameContext; for (auto const &algo : algorithms()) - algo->process(context_, frameContext, stats); + algo->process(context_, frame, frameContext, stats); setControls(frame); From patchwork Tue Sep 27 02:36:17 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Laurent Pinchart X-Patchwork-Id: 17419 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id F07A9C0DA4 for ; Tue, 27 Sep 2022 02:37:14 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 865B46228A; Tue, 27 Sep 2022 04:37:14 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1664246234; bh=/CIW0eUrjoMlixPe6qEJ+Wgkq7GIJp7JAABKOr7bq4c=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To: From; b=Qxz8NyUrdr4oWdJTFLmZG2l8snGlC11sPXJ5LDyLq26x4ouNeFEXGNKmsAiNiooUf B/oF2r/x8FWb6Vteczl2PJbEXPcSvzI/AvhtrRezjf+8x4fcVV/hvnZqqir+oNL6Jp p/lGX2ZTFeo+VxXYqIbJvmKYdLgMv+y9+IMpr3Trc19VlvcO5k++1/lxOsasGZA1Ev iqNyu7Vv+CrXTET7QZImSSIWSr/tHCuLBDvIgRj7BTLlUO2/dxZOQ49FTyuEAC5rTd y8UAfg6quskthRL/55nJ3JqLvoAzXSWApq6DviRnZLehi3T+ouyuxw9pe/KxDZoxi1 hkgPowW94Hnvg== Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id AE20B6227B for ; Tue, 27 Sep 2022 04:37:11 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="lSq8wkM+"; dkim-atps=neutral Received: from pendragon.ideasonboard.com (62-78-145-57.bb.dnainternet.fi [62.78.145.57]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 320EEE5; Tue, 27 Sep 2022 04:37:11 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1664246231; bh=/CIW0eUrjoMlixPe6qEJ+Wgkq7GIJp7JAABKOr7bq4c=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=lSq8wkM+/LxdaJoraZHZZVe39tSy7R6i+Dl0zFTXWq20zZMErJiknpOX+uOOP8Klb Wr2mqGp2U7nOU1KVzFrxVLz4F+4gm59qsVkGcPV03fdVaofmrPloQVsZ4s25LeVzOF zbQSeNlqOg+Vbqw54U5Q8vBFshVJ+xGO18a5tcv0= To: libcamera-devel@lists.libcamera.org Date: Tue, 27 Sep 2022 05:36:17 +0300 Message-Id: <20220927023642.12341-9-laurent.pinchart@ideasonboard.com> X-Mailer: git-send-email 2.35.1 In-Reply-To: <20220927023642.12341-1-laurent.pinchart@ideasonboard.com> References: <20220927023642.12341-1-laurent.pinchart@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v5 08/33] ipa: libipa: algorithm: queueRequest(): Pass frame context X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Laurent Pinchart via libcamera-devel From: Laurent Pinchart Reply-To: Laurent Pinchart Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" From: Kieran Bingham IPA modules have access to incoming Request's controls list and need to store them in the frame context at queueRequest() time. Pass the frame context to the Algorithm::queueRequest() function. Signed-off-by: Kieran Bingham Signed-off-by: Jacopo Mondi Reviewed-by: Umang Jain Reviewed-by: Kieran Bingham --- Changes since v4: - Rewrite commit message --- src/ipa/libipa/algorithm.cpp | 1 + src/ipa/libipa/algorithm.h | 1 + src/ipa/rkisp1/algorithms/awb.cpp | 1 + src/ipa/rkisp1/algorithms/awb.h | 1 + src/ipa/rkisp1/algorithms/cproc.cpp | 1 + src/ipa/rkisp1/algorithms/cproc.h | 1 + src/ipa/rkisp1/algorithms/dpf.cpp | 1 + src/ipa/rkisp1/algorithms/dpf.h | 1 + src/ipa/rkisp1/algorithms/filter.cpp | 1 + src/ipa/rkisp1/algorithms/filter.h | 1 + src/ipa/rkisp1/rkisp1.cpp | 5 ++++- 11 files changed, 14 insertions(+), 1 deletion(-) diff --git a/src/ipa/libipa/algorithm.cpp b/src/ipa/libipa/algorithm.cpp index 30eab67f71fc..c152b885aee1 100644 --- a/src/ipa/libipa/algorithm.cpp +++ b/src/ipa/libipa/algorithm.cpp @@ -88,6 +88,7 @@ namespace ipa { * \brief Provide control values to the algorithm * \param[in] context The shared IPA context * \param[in] frame The frame number to apply the control values + * \param[in] frameContext The current frame's context * \param[in] controls The list of user controls * * This function is called for each request queued to the camera. It provides diff --git a/src/ipa/libipa/algorithm.h b/src/ipa/libipa/algorithm.h index ee93d260ca3f..d8601f9ccaff 100644 --- a/src/ipa/libipa/algorithm.h +++ b/src/ipa/libipa/algorithm.h @@ -47,6 +47,7 @@ public: virtual void queueRequest([[maybe_unused]] typename Module::Context &context, [[maybe_unused]] const uint32_t frame, + [[maybe_unused]] typename Module::FrameContext &frameContext, [[maybe_unused]] const ControlList &controls) { } diff --git a/src/ipa/rkisp1/algorithms/awb.cpp b/src/ipa/rkisp1/algorithms/awb.cpp index aefba6382807..a23c32015454 100644 --- a/src/ipa/rkisp1/algorithms/awb.cpp +++ b/src/ipa/rkisp1/algorithms/awb.cpp @@ -128,6 +128,7 @@ void Awb::prepare(IPAContext &context, */ void Awb::queueRequest(IPAContext &context, [[maybe_unused]] const uint32_t frame, + [[maybe_unused]] IPAFrameContext &frameContext, const ControlList &controls) { auto &awb = context.frameContext.awb; diff --git a/src/ipa/rkisp1/algorithms/awb.h b/src/ipa/rkisp1/algorithms/awb.h index 806e11a2ec93..6d4a39467038 100644 --- a/src/ipa/rkisp1/algorithms/awb.h +++ b/src/ipa/rkisp1/algorithms/awb.h @@ -24,6 +24,7 @@ public: IPAFrameContext &frameContext, rkisp1_params_cfg *params) override; void queueRequest(IPAContext &context, const uint32_t frame, + IPAFrameContext &frameContext, const ControlList &controls) override; void process(IPAContext &context, const uint32_t frame, IPAFrameContext &frameCtx, diff --git a/src/ipa/rkisp1/algorithms/cproc.cpp b/src/ipa/rkisp1/algorithms/cproc.cpp index c0505c68765b..bc4a07073706 100644 --- a/src/ipa/rkisp1/algorithms/cproc.cpp +++ b/src/ipa/rkisp1/algorithms/cproc.cpp @@ -38,6 +38,7 @@ LOG_DEFINE_CATEGORY(RkISP1CProc) */ void ColorProcessing::queueRequest(IPAContext &context, [[maybe_unused]] const uint32_t frame, + [[maybe_unused]] IPAFrameContext &frameContext, const ControlList &controls) { auto &cproc = context.frameContext.cproc; diff --git a/src/ipa/rkisp1/algorithms/cproc.h b/src/ipa/rkisp1/algorithms/cproc.h index 02c3d9c95488..ba6e901ae110 100644 --- a/src/ipa/rkisp1/algorithms/cproc.h +++ b/src/ipa/rkisp1/algorithms/cproc.h @@ -22,6 +22,7 @@ public: ~ColorProcessing() = default; void queueRequest(IPAContext &context, const uint32_t frame, + IPAFrameContext &frameContext, const ControlList &controls) override; void prepare(IPAContext &context, const uint32_t frame, IPAFrameContext &frameContext, diff --git a/src/ipa/rkisp1/algorithms/dpf.cpp b/src/ipa/rkisp1/algorithms/dpf.cpp index 8286f83facc1..be024fc5fe90 100644 --- a/src/ipa/rkisp1/algorithms/dpf.cpp +++ b/src/ipa/rkisp1/algorithms/dpf.cpp @@ -176,6 +176,7 @@ int Dpf::init([[maybe_unused]] IPAContext &context, */ void Dpf::queueRequest(IPAContext &context, [[maybe_unused]] const uint32_t frame, + [[maybe_unused]] IPAFrameContext &frameContext, const ControlList &controls) { auto &dpf = context.frameContext.dpf; diff --git a/src/ipa/rkisp1/algorithms/dpf.h b/src/ipa/rkisp1/algorithms/dpf.h index 656d78dee67c..e232cad4ee87 100644 --- a/src/ipa/rkisp1/algorithms/dpf.h +++ b/src/ipa/rkisp1/algorithms/dpf.h @@ -23,6 +23,7 @@ public: int init(IPAContext &context, const YamlObject &tuningData) override; void queueRequest(IPAContext &context, const uint32_t frame, + IPAFrameContext &frameContext, const ControlList &controls) override; void prepare(IPAContext &context, const uint32_t frame, IPAFrameContext &frameContext, diff --git a/src/ipa/rkisp1/algorithms/filter.cpp b/src/ipa/rkisp1/algorithms/filter.cpp index 02562339d008..6aa5476aa25e 100644 --- a/src/ipa/rkisp1/algorithms/filter.cpp +++ b/src/ipa/rkisp1/algorithms/filter.cpp @@ -44,6 +44,7 @@ static constexpr uint32_t kFiltModeDefault = 0x000004f2; */ void Filter::queueRequest(IPAContext &context, [[maybe_unused]] const uint32_t frame, + [[maybe_unused]] IPAFrameContext &frameContext, const ControlList &controls) { auto &filter = context.frameContext.filter; diff --git a/src/ipa/rkisp1/algorithms/filter.h b/src/ipa/rkisp1/algorithms/filter.h index ac69657e2f0f..3fd882ea41dc 100644 --- a/src/ipa/rkisp1/algorithms/filter.h +++ b/src/ipa/rkisp1/algorithms/filter.h @@ -22,6 +22,7 @@ public: ~Filter() = default; void queueRequest(IPAContext &context, const uint32_t frame, + IPAFrameContext &frameContext, const ControlList &controls) override; void prepare(IPAContext &context, const uint32_t frame, IPAFrameContext &frameContext, diff --git a/src/ipa/rkisp1/rkisp1.cpp b/src/ipa/rkisp1/rkisp1.cpp index b2593a584a1a..07210d16b5b5 100644 --- a/src/ipa/rkisp1/rkisp1.cpp +++ b/src/ipa/rkisp1/rkisp1.cpp @@ -292,8 +292,11 @@ void IPARkISP1::unmapBuffers(const std::vector &ids) void IPARkISP1::queueRequest(const uint32_t frame, const ControlList &controls) { + /* \todo Obtain the frame context to pass to process from the FCQueue */ + IPAFrameContext frameContext; + for (auto const &algo : algorithms()) - algo->queueRequest(context_, frame, controls); + algo->queueRequest(context_, frame, frameContext, controls); } void IPARkISP1::fillParamsBuffer(const uint32_t frame, const uint32_t bufferId) From patchwork Tue Sep 27 02:36:18 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Laurent Pinchart X-Patchwork-Id: 17420 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 94909C327E for ; Tue, 27 Sep 2022 02:37:15 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 4558062280; Tue, 27 Sep 2022 04:37:15 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1664246235; bh=lWREngrB4gOteSDuh0Ey3KbllXbKoBisx8QRdnhiPA4=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To: From; b=Lx9OIVXxwv7Y3IsPSQIocHXnaTGGqwKTCPXNnl8HYTTbDdGyjJeFGB/HPK1fOlt7m P64gAML4iFpLQ1FJs27SmAo57UfPjPzDWjMhdGhkt6imlRbBc8YNEReAEMb1z8cF8w eIXsX8XRt190eimuU4lacqCWrpXtAGq/msjxSLrtFQMXrJ4dGhCT96eLv9e7fnbH0v JBoBTheS32qaFyFJHzo/6YEhxCgHd+YGI01wFmUJqqk0tZKHR9pNQguTTNVHX68GeI 50BuYEWuga5i+I7C88TbImYybr3mJ+DfoZsQ7r9fTmMeIkJteuZAF+0DvVaB3agzo7 aHMKhJtDnsQ5w== Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 394A362289 for ; Tue, 27 Sep 2022 04:37:13 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="Qrb4wp+x"; dkim-atps=neutral Received: from pendragon.ideasonboard.com (62-78-145-57.bb.dnainternet.fi [62.78.145.57]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id B24CC47C for ; Tue, 27 Sep 2022 04:37:12 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1664246232; bh=lWREngrB4gOteSDuh0Ey3KbllXbKoBisx8QRdnhiPA4=; h=From:To:Subject:Date:In-Reply-To:References:From; b=Qrb4wp+xp+fVJoSJweU3RAnyNN646F/wx4Fl56+dGGfrRqq2M5aRGfvAqjY4oF8Ex 9gOIZXIhfbiVRG7VhU1bOTd07AmRdIGlhHmzyysx3NtF+s5cXKQ9dCQZYxioNwGalM WOdf9xwd+RT0u62xf7esFRopKXJwvGni9OBSeUzk= To: libcamera-devel@lists.libcamera.org Date: Tue, 27 Sep 2022 05:36:18 +0300 Message-Id: <20220927023642.12341-10-laurent.pinchart@ideasonboard.com> X-Mailer: git-send-email 2.35.1 In-Reply-To: <20220927023642.12341-1-laurent.pinchart@ideasonboard.com> References: <20220927023642.12341-1-laurent.pinchart@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v5 09/33] ipa: ipu3: Use base FrameContext class X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Laurent Pinchart via libcamera-devel From: Laurent Pinchart Reply-To: Laurent Pinchart Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Inherit from the base FrameContext class in the IPU3 IPAFrameContext. This allows dropping the frame member, which is now stored in the base class. As the frame member of the base FrameContext class is private, the check that accesses it in IPAIPU3::processStatsBuffer() would fail to compile. As it won't be relevant anymore with the upcoming switch to the FCQueue class, drop it. Signed-off-by: Laurent Pinchart Reviewed-by: Kieran Bingham Reviewed-by: Jacopo Mondi --- src/ipa/ipu3/ipa_context.cpp | 24 ++++-------------------- src/ipa/ipu3/ipa_context.h | 7 ++++--- src/ipa/ipu3/ipu3.cpp | 5 +---- 3 files changed, 9 insertions(+), 27 deletions(-) diff --git a/src/ipa/ipu3/ipa_context.cpp b/src/ipa/ipu3/ipa_context.cpp index 13cdb835ef7f..9cfca0db3a0d 100644 --- a/src/ipa/ipu3/ipa_context.cpp +++ b/src/ipa/ipu3/ipa_context.cpp @@ -35,22 +35,6 @@ namespace libcamera::ipa::ipu3 { * most recently computed by the IPA algorithms. */ -/** - * \struct IPAFrameContext - * \brief Context for a frame - * - * The frame context stores data specific to a single frame processed by the - * IPA. Each frame processed by the IPA has a context associated with it, - * accessible through the IPAContext structure. - * - * Fields in the frame context should reflect values and controls - * associated with the specific frame as requested by the application, and - * as configured by the hardware. Fields can be read by algorithms to - * determine if they should update any specific action for this frame, and - * finally to update the metadata control lists when the frame is fully - * completed. - */ - /** * \struct IPAContext * \brief Global IPA context data shared between all algorithms @@ -188,15 +172,15 @@ IPAFrameContext::IPAFrameContext() = default; /** * \brief Construct a IPAFrameContext instance */ -IPAFrameContext::IPAFrameContext(uint32_t id, const ControlList &reqControls) - : frame(id), frameControls(reqControls) +IPAFrameContext::IPAFrameContext(const ControlList &reqControls) + : frameControls(reqControls) { sensor = {}; } /** - * \var IPAFrameContext::frame - * \brief The frame number + * \struct IPAFrameContext + * \brief IPU3-specific FrameContext * * \var IPAFrameContext::frameControls * \brief Controls sent in by the application while queuing the request diff --git a/src/ipa/ipu3/ipa_context.h b/src/ipa/ipu3/ipa_context.h index 42e11141d3a1..e8fc42769075 100644 --- a/src/ipa/ipu3/ipa_context.h +++ b/src/ipa/ipu3/ipa_context.h @@ -17,6 +17,8 @@ #include #include +#include + namespace libcamera { namespace ipa::ipu3 { @@ -76,16 +78,15 @@ struct IPAActiveState { } toneMapping; }; -struct IPAFrameContext { +struct IPAFrameContext : public FrameContext { IPAFrameContext(); - IPAFrameContext(uint32_t id, const ControlList &reqControls); + IPAFrameContext(const ControlList &reqControls); struct { uint32_t exposure; double gain; } sensor; - uint32_t frame; ControlList frameControls; }; diff --git a/src/ipa/ipu3/ipu3.cpp b/src/ipa/ipu3/ipu3.cpp index e5a763fd2b08..b1b23fd8f927 100644 --- a/src/ipa/ipu3/ipu3.cpp +++ b/src/ipa/ipu3/ipu3.cpp @@ -607,9 +607,6 @@ void IPAIPU3::processStatsBuffer(const uint32_t frame, IPAFrameContext &frameContext = context_.frameContexts[frame % kMaxFrameContexts]; - if (frameContext.frame != frame) - LOG(IPAIPU3, Warning) << "Frame " << frame << " does not match its frame context"; - frameContext.sensor.exposure = sensorControls.get(V4L2_CID_EXPOSURE).get(); frameContext.sensor.gain = camHelper_->gain(sensorControls.get(V4L2_CID_ANALOGUE_GAIN).get()); @@ -654,7 +651,7 @@ void IPAIPU3::processStatsBuffer(const uint32_t frame, void IPAIPU3::queueRequest(const uint32_t frame, const ControlList &controls) { /* \todo Start processing for 'frame' based on 'controls'. */ - context_.frameContexts[frame % kMaxFrameContexts] = { frame, controls }; + context_.frameContexts[frame % kMaxFrameContexts] = { controls }; } /** From patchwork Tue Sep 27 02:36:19 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Laurent Pinchart X-Patchwork-Id: 17421 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 3BE8AC0DA4 for ; Tue, 27 Sep 2022 02:37:18 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id D8E7262293; Tue, 27 Sep 2022 04:37:17 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1664246237; bh=99JUc/pduUEbHGlojTiZe+EJQy/Kb9Fep6BbBUb6jMY=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To: From; b=y29gjel8vFvH9rfPdLMCRNMsPwq+99Khpaa3f8t7KZSItBpW+Ikg9iFk1Ml7l9B4p mczbQG1V7dC2UaTOZ4Z6exlHBSv5r2cLBbPvZK2qZ3+6yChOD1YEACki0OSZkf/Q5+ i7Tor/b2b98YWwkF+KnGkURG6OUZBsYzKWf0wQAiu9KgIuMMjTEFOiYGOd1Pwm/1vI Ciy6cLw/FR+Fo90qrlBKj2GGpUwWhF6Fm+sXYSNK7Af73nesYheYpMj7B4JwdwsgO/ A+8bHwgDZWMX10RtOsKHlBFEpJGcjoVsmefN7xio9CuiEM2HN+RAnSdX/GYu5a5ECx SoKj1JeswA2OQ== Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 8C3C36228B for ; Tue, 27 Sep 2022 04:37:14 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="jSjlLwFt"; dkim-atps=neutral Received: from pendragon.ideasonboard.com (62-78-145-57.bb.dnainternet.fi [62.78.145.57]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 08F9047C for ; Tue, 27 Sep 2022 04:37:13 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1664246234; bh=99JUc/pduUEbHGlojTiZe+EJQy/Kb9Fep6BbBUb6jMY=; h=From:To:Subject:Date:In-Reply-To:References:From; b=jSjlLwFthvSxNeusTNAzzZW0ut0es8rcQtc8TKyFFP1RTfN2xx/DvcK+PyHyyq7WQ qV9niyUJkutDO8PSlyXuJ5RaYplQjsWnUFpbwh2rkuxC4B0xUAYRVMhXGdfiaSrfND HGjClZdbnnzt4qDHfD2m7vc77CsdIWH85SdLjh3Y= To: libcamera-devel@lists.libcamera.org Date: Tue, 27 Sep 2022 05:36:19 +0300 Message-Id: <20220927023642.12341-11-laurent.pinchart@ideasonboard.com> X-Mailer: git-send-email 2.35.1 In-Reply-To: <20220927023642.12341-1-laurent.pinchart@ideasonboard.com> References: <20220927023642.12341-1-laurent.pinchart@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v5 10/33] ipa: ipu3: Use the FCQueue X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Laurent Pinchart via libcamera-devel From: Laurent Pinchart Reply-To: Laurent Pinchart Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Replace the manual ring buffer implementation with the FCQueue class from libipa. Signed-off-by: Laurent Pinchart Reviewed-by: Kieran Bingham Reviewed-by: Jacopo Mondi --- Changes since v4: - Rename FCQueue::init() to alloc() --- src/ipa/ipu3/ipa_context.cpp | 14 -------------- src/ipa/ipu3/ipa_context.h | 10 +--------- src/ipa/ipu3/ipu3.cpp | 32 ++++++++++++++++++++++++-------- 3 files changed, 25 insertions(+), 31 deletions(-) diff --git a/src/ipa/ipu3/ipa_context.cpp b/src/ipa/ipu3/ipa_context.cpp index 9cfca0db3a0d..6904ccbbdf8b 100644 --- a/src/ipa/ipu3/ipa_context.cpp +++ b/src/ipa/ipu3/ipa_context.cpp @@ -164,20 +164,6 @@ namespace libcamera::ipa::ipu3 { * struct ipu3_uapi_gamma_corr_lut for further details. */ -/** - * \brief Default constructor for IPAFrameContext - */ -IPAFrameContext::IPAFrameContext() = default; - -/** - * \brief Construct a IPAFrameContext instance - */ -IPAFrameContext::IPAFrameContext(const ControlList &reqControls) - : frameControls(reqControls) -{ - sensor = {}; -} - /** * \struct IPAFrameContext * \brief IPU3-specific FrameContext diff --git a/src/ipa/ipu3/ipa_context.h b/src/ipa/ipu3/ipa_context.h index e8fc42769075..bfc0196e098a 100644 --- a/src/ipa/ipu3/ipa_context.h +++ b/src/ipa/ipu3/ipa_context.h @@ -8,8 +8,6 @@ #pragma once -#include - #include #include @@ -23,9 +21,6 @@ namespace libcamera { namespace ipa::ipu3 { -/* Maximum number of frame contexts to be held */ -static constexpr uint32_t kMaxFrameContexts = 16; - struct IPASessionConfiguration { struct { ipu3_uapi_grid_config bdsGrid; @@ -79,9 +74,6 @@ struct IPAActiveState { }; struct IPAFrameContext : public FrameContext { - IPAFrameContext(); - IPAFrameContext(const ControlList &reqControls); - struct { uint32_t exposure; double gain; @@ -94,7 +86,7 @@ struct IPAContext { IPASessionConfiguration configuration; IPAActiveState activeState; - std::array frameContexts; + FCQueue frameContexts; }; } /* namespace ipa::ipu3 */ diff --git a/src/ipa/ipu3/ipu3.cpp b/src/ipa/ipu3/ipu3.cpp index b1b23fd8f927..96dc2af2cb73 100644 --- a/src/ipa/ipu3/ipu3.cpp +++ b/src/ipa/ipu3/ipu3.cpp @@ -40,6 +40,8 @@ #include "algorithms/tone_mapping.h" #include "libipa/camera_sensor_helper.h" +#include "ipa_context.h" + /* Minimum grid width, expressed as a number of cells */ static constexpr uint32_t kMinGridWidth = 16; /* Maximum grid width, expressed as a number of cells */ @@ -53,6 +55,9 @@ static constexpr uint32_t kMinCellSizeLog2 = 3; /* log2 of the maximum grid cell width and height, in pixels */ static constexpr uint32_t kMaxCellSizeLog2 = 6; +/* Maximum number of frame contexts to be held */ +static constexpr uint32_t kMaxFrameContexts = 16; + namespace libcamera { LOG_DEFINE_CATEGORY(IPAIPU3) @@ -135,6 +140,8 @@ namespace ipa::ipu3 { class IPAIPU3 : public IPAIPU3Interface, public Module { public: + IPAIPU3(); + int init(const IPASettings &settings, const IPACameraSensorInfo &sensorInfo, const ControlInfoMap &sensorControls, @@ -183,6 +190,11 @@ private: struct IPAContext context_; }; +IPAIPU3::IPAIPU3() + : context_({ {}, {}, { kMaxFrameContexts } }) +{ +} + std::string IPAIPU3::logPrefix() const { return "ipu3"; @@ -205,6 +217,11 @@ void IPAIPU3::updateSessionConfiguration(const ControlInfoMap &sensorControls) int32_t minGain = v4l2Gain.min().get(); int32_t maxGain = v4l2Gain.max().get(); + /* Clear the IPA context before the streaming session. */ + context_.configuration = {}; + context_.activeState = {}; + context_.frameContexts.clear(); + /* * When the AGC computes the new exposure values for a frame, it needs * to know the limits for shutter speed and analogue gain. @@ -382,6 +399,7 @@ int IPAIPU3::start() */ void IPAIPU3::stop() { + context_.frameContexts.clear(); } /** @@ -488,11 +506,6 @@ int IPAIPU3::configure(const IPAConfigInfo &configInfo, calculateBdsGrid(configInfo.bdsOutputSize); - /* Clean IPAActiveState at each reconfiguration. */ - context_.activeState = {}; - IPAFrameContext initFrameContext; - context_.frameContexts.fill(initFrameContext); - if (!validateSensorControls()) { LOG(IPAIPU3, Error) << "Sensor control validation failed."; return -EINVAL; @@ -572,7 +585,7 @@ void IPAIPU3::fillParamsBuffer(const uint32_t frame, const uint32_t bufferId) */ params->use = {}; - IPAFrameContext &frameContext = context_.frameContexts[frame % kMaxFrameContexts]; + IPAFrameContext &frameContext = context_.frameContexts.get(frame); for (auto const &algo : algorithms()) algo->prepare(context_, frame, frameContext, params); @@ -605,7 +618,7 @@ void IPAIPU3::processStatsBuffer(const uint32_t frame, const ipu3_uapi_stats_3a *stats = reinterpret_cast(mem.data()); - IPAFrameContext &frameContext = context_.frameContexts[frame % kMaxFrameContexts]; + IPAFrameContext &frameContext = context_.frameContexts.get(frame); frameContext.sensor.exposure = sensorControls.get(V4L2_CID_EXPOSURE).get(); frameContext.sensor.gain = camHelper_->gain(sensorControls.get(V4L2_CID_ANALOGUE_GAIN).get()); @@ -651,7 +664,10 @@ void IPAIPU3::processStatsBuffer(const uint32_t frame, void IPAIPU3::queueRequest(const uint32_t frame, const ControlList &controls) { /* \todo Start processing for 'frame' based on 'controls'. */ - context_.frameContexts[frame % kMaxFrameContexts] = { controls }; + IPAFrameContext &frameContext = context_.frameContexts.alloc(frame); + + /* \todo Implement queueRequest to each algorithm. */ + frameContext.frameControls = controls; } /** From patchwork Tue Sep 27 02:36:20 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Laurent Pinchart X-Patchwork-Id: 17422 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id BCAD3C327E for ; Tue, 27 Sep 2022 02:37:18 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 8802062290; Tue, 27 Sep 2022 04:37:18 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1664246238; bh=48ozrf+iyxWSMIBRE+QVZz1G+ZWEjToS+6cGvsIwVbo=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To: From; b=RbTImtf+T9DMtld7LtC74PgxPuWpBpvzRTjlXRuStJvzOzuKKQevC4YYQ133297uB ni1D2k4umWFcsLdwmYL51LG0BbU8icENCIuo9DF4o0QTnao35U7zcYZd3Bl1jLvMu2 bQOCAIgkSlT5Wnc4XSsQT0DBbRYQ3GRotyKu4Gg9qV0gdDPBwYTy6Cn03yvNjcmicv mJzhG0rEzVG6+T12YXD+W+oEf+fJr/Ri5UGFfyKdGPdbKUxaTGIkxOSdQ6w+dUtQwe 4+y6Pt84CwGTDqVdcWldcMFbP8ujT4GtWRKn1uj1dGafT6SAzFnM9/Iklc/Xbd110C 8LDjR0HGx2qvg== Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id F1CE16228F for ; Tue, 27 Sep 2022 04:37:15 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="eCPOWuNM"; dkim-atps=neutral Received: from pendragon.ideasonboard.com (62-78-145-57.bb.dnainternet.fi [62.78.145.57]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 76A3A47C for ; Tue, 27 Sep 2022 04:37:15 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1664246235; bh=48ozrf+iyxWSMIBRE+QVZz1G+ZWEjToS+6cGvsIwVbo=; h=From:To:Subject:Date:In-Reply-To:References:From; b=eCPOWuNMvMIxMmXtYDmDU9L+byLgOEvDOz9ORFx2Nzh1ey841DySUaFr6KJvc9D3b EgJMdkwkpZ55crlglmPAmR2lX3BUey02w7qhD0lmBJ2OGtfAse0T71I7hsta5fbyDx O4Xa/v+Ww5JnaxBFwELV9hynXjs/W0EG8JF3onZw= To: libcamera-devel@lists.libcamera.org Date: Tue, 27 Sep 2022 05:36:20 +0300 Message-Id: <20220927023642.12341-12-laurent.pinchart@ideasonboard.com> X-Mailer: git-send-email 2.35.1 In-Reply-To: <20220927023642.12341-1-laurent.pinchart@ideasonboard.com> References: <20220927023642.12341-1-laurent.pinchart@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v5 11/33] ipa: ipu3: Pass controls to algorithm's queueRequest() handler X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Laurent Pinchart via libcamera-devel From: Laurent Pinchart Reply-To: Laurent Pinchart Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Call the Algorithm::queueRequest() function of all algorithms when a request is queued, to pass the request controls to the algorithms. We can now drop the copy of the control list stored in IPAFrameContext as it isn't used anymore. Signed-off-by: Laurent Pinchart Reviewed-by: Kieran Bingham Reviewed-by: Jacopo Mondi --- Changes since v4: - Drop todo comment --- src/ipa/ipu3/ipa_context.cpp | 3 --- src/ipa/ipu3/ipa_context.h | 3 --- src/ipa/ipu3/ipu3.cpp | 5 ++--- 3 files changed, 2 insertions(+), 9 deletions(-) diff --git a/src/ipa/ipu3/ipa_context.cpp b/src/ipa/ipu3/ipa_context.cpp index 6904ccbbdf8b..bd71b615365d 100644 --- a/src/ipa/ipu3/ipa_context.cpp +++ b/src/ipa/ipu3/ipa_context.cpp @@ -168,9 +168,6 @@ namespace libcamera::ipa::ipu3 { * \struct IPAFrameContext * \brief IPU3-specific FrameContext * - * \var IPAFrameContext::frameControls - * \brief Controls sent in by the application while queuing the request - * * \var IPAFrameContext::sensor * \brief Effective sensor values that were applied for the frame * diff --git a/src/ipa/ipu3/ipa_context.h b/src/ipa/ipu3/ipa_context.h index bfc0196e098a..36099353e9f2 100644 --- a/src/ipa/ipu3/ipa_context.h +++ b/src/ipa/ipu3/ipa_context.h @@ -12,7 +12,6 @@ #include -#include #include #include @@ -78,8 +77,6 @@ struct IPAFrameContext : public FrameContext { uint32_t exposure; double gain; } sensor; - - ControlList frameControls; }; struct IPAContext { diff --git a/src/ipa/ipu3/ipu3.cpp b/src/ipa/ipu3/ipu3.cpp index 96dc2af2cb73..d1ea081d595d 100644 --- a/src/ipa/ipu3/ipu3.cpp +++ b/src/ipa/ipu3/ipu3.cpp @@ -663,11 +663,10 @@ void IPAIPU3::processStatsBuffer(const uint32_t frame, */ void IPAIPU3::queueRequest(const uint32_t frame, const ControlList &controls) { - /* \todo Start processing for 'frame' based on 'controls'. */ IPAFrameContext &frameContext = context_.frameContexts.alloc(frame); - /* \todo Implement queueRequest to each algorithm. */ - frameContext.frameControls = controls; + for (auto const &algo : algorithms()) + algo->queueRequest(context_, frame, frameContext, controls); } /** From patchwork Tue Sep 27 02:36:21 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Laurent Pinchart X-Patchwork-Id: 17423 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 67A8FC327F for ; Tue, 27 Sep 2022 02:37:19 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 1999262269; Tue, 27 Sep 2022 04:37:19 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1664246239; bh=m3T2sQgADx/UNRwg4tib9LD5+5I1OEq0ssZTCj9Ak40=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To: From; b=vcyzp4C6VrNDc8mPL0n1J4LaLF2ZauRHXYKNaXma4VN4wLjk6qJs9kP7p5hH3uYjY lnTA2WBgIW1cGSsGM+956M4sMYZSLoeYRRxL7hdhuyhj1JuJ/6tWCAU+OIhnYs1IHN Y02GGjR/VrjqFRBaCitLkIzcpjWLed6iKTWlQGe0gvpMz1RSBcUHgVafnZYd8pBOOU LO4cX/NjX9jqMWmDWZqCzgOHU8LpTvxdha4FCbxKo35QEv0dzV1wqpBEmLw3T58LuD sPCXsGZwTY/zj3yYslJVBUFImhIF89RK36d/cOYLcG9n9L5hUmh3TnIIr9aPqEWiIf 24LCgw/4ibDgg== Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 6D81562262 for ; Tue, 27 Sep 2022 04:37:17 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="JNAMZDcd"; dkim-atps=neutral Received: from pendragon.ideasonboard.com (62-78-145-57.bb.dnainternet.fi [62.78.145.57]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id D1C2BE5; Tue, 27 Sep 2022 04:37:16 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1664246237; bh=m3T2sQgADx/UNRwg4tib9LD5+5I1OEq0ssZTCj9Ak40=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=JNAMZDcdqcLHIUAO2GEx9fsj6MAigFDCswQBICvdLuZSjPOOTjvkCLHFzC75sRPEd DaMRvESQPjTVeUh5studINSZHRcZ39vOnIM6JSwwhMLrEyTAIYNh8YRdMSYzoGZizq sYUKKKg8Eo/EEGlobiV+m64r20/qjTBxaQUqHKx4= To: libcamera-devel@lists.libcamera.org Date: Tue, 27 Sep 2022 05:36:21 +0300 Message-Id: <20220927023642.12341-13-laurent.pinchart@ideasonboard.com> X-Mailer: git-send-email 2.35.1 In-Reply-To: <20220927023642.12341-1-laurent.pinchart@ideasonboard.com> References: <20220927023642.12341-1-laurent.pinchart@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v5 12/33] ipa: rkisp1: Remove unused class member X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Laurent Pinchart via libcamera-devel From: Laurent Pinchart Reply-To: Laurent Pinchart Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" From: Jacopo Mondi The "autoExposure" class member is not used. Remove it. Signed-off-by: Jacopo Mondi Reviewed-by: Laurent Pinchart Reviewed-by: Kieran Bingham --- src/ipa/rkisp1/rkisp1.cpp | 5 ----- 1 file changed, 5 deletions(-) diff --git a/src/ipa/rkisp1/rkisp1.cpp b/src/ipa/rkisp1/rkisp1.cpp index 07210d16b5b5..f7a17459b6c3 100644 --- a/src/ipa/rkisp1/rkisp1.cpp +++ b/src/ipa/rkisp1/rkisp1.cpp @@ -71,9 +71,6 @@ private: ControlInfoMap ctrls_; - /* Camera sensor controls. */ - bool autoExposure_; - /* revision-specific data */ rkisp1_cif_isp_version hwRevision_; unsigned int hwHistBinNMax_; @@ -216,8 +213,6 @@ int IPARkISP1::configure([[maybe_unused]] const IPACameraSensorInfo &info, return -EINVAL; } - autoExposure_ = true; - int32_t minExposure = itExp->second.min().get(); int32_t maxExposure = itExp->second.max().get(); From patchwork Tue Sep 27 02:36:22 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Laurent Pinchart X-Patchwork-Id: 17424 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id C7514C0DA4 for ; Tue, 27 Sep 2022 02:37:21 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 8A6A46229C; Tue, 27 Sep 2022 04:37:21 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1664246241; bh=aMX5dVngiw9idpMcT8CjifyACs0RdcM1BPf9T5FsCyc=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To: From; b=XW1wkG3DIop74arimqt16zq53VzR95HPz371aOCAMoU2dIy8fCQJkaKDIytoyZKJY ELNokTrDJjEk5hN3Y0ftBmYkvEVp2kv2NZLwqJ/XlVSKzgwvSVDkqnoH+sQKt4bjoE VxqNMLOFM5RSIXajXSIbxq7XSwEjxy/d2aug3CSd1Nk6smGSJgf38tsXOtFNCoZUUz cATbBu7hG4rYd/L+BepreeMpxQYw1LLurfSHrPVhIuAPieLWLHClcDoWyRjIackPmL FsinMXmbS5D4HGIjRzY+MCDcPSoh/mCBRAD45hFpYcWqL5/AQHFFe18Qw0RGzeTZbd 0T4HzYi4jTR5Q== Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id ED33C6228D for ; Tue, 27 Sep 2022 04:37:18 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="t7s2jtOe"; dkim-atps=neutral Received: from pendragon.ideasonboard.com (62-78-145-57.bb.dnainternet.fi [62.78.145.57]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 6C318E5 for ; Tue, 27 Sep 2022 04:37:18 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1664246238; bh=aMX5dVngiw9idpMcT8CjifyACs0RdcM1BPf9T5FsCyc=; h=From:To:Subject:Date:In-Reply-To:References:From; b=t7s2jtOeHXHzJNbXCQPpMWbygpRevp1Bq4v8oKuEnwy4BNFq9gPdOx+QYGLoGsojs TVaaP7Kshqxate7lmuHroK/hpYpLavQZpR1znVutXrJPnP7GIfO06qLuXWnZZDsG2z XZ03HNzDCALXB/LPPIXrEhbieTJWELp8JkLPVAi4= To: libcamera-devel@lists.libcamera.org Date: Tue, 27 Sep 2022 05:36:22 +0300 Message-Id: <20220927023642.12341-14-laurent.pinchart@ideasonboard.com> X-Mailer: git-send-email 2.35.1 In-Reply-To: <20220927023642.12341-1-laurent.pinchart@ideasonboard.com> References: <20220927023642.12341-1-laurent.pinchart@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v5 13/33] ipa: rkisp1: Sort documentation of the IPA context X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Laurent Pinchart via libcamera-devel From: Laurent Pinchart Reply-To: Laurent Pinchart Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" The documentation of the IPA context structures is separate from the documentation of the structure members. Sort the documentation block to group members with their structure. Signed-off-by: Laurent Pinchart Reviewed-by: Kieran Bingham Reviewed-by: Jacopo Mondi --- src/ipa/rkisp1/ipa_context.cpp | 62 +++++++++++++++++----------------- 1 file changed, 31 insertions(+), 31 deletions(-) diff --git a/src/ipa/rkisp1/ipa_context.cpp b/src/ipa/rkisp1/ipa_context.cpp index f64af732260c..cc66bc70ac44 100644 --- a/src/ipa/rkisp1/ipa_context.cpp +++ b/src/ipa/rkisp1/ipa_context.cpp @@ -24,37 +24,6 @@ namespace libcamera::ipa::rkisp1 { * may also be updated in the start() operation. */ -/** - * \struct IPAFrameContext - * \brief Per-frame context for algorithms - * - * The frame context stores data specific to a single frame processed by the - * IPA. Each frame processed by the IPA has a context associated with it, - * accessible through the IPAContext structure. - * - * \todo Detail how to access contexts for a particular frame - * - * Each of the fields in the frame context belongs to either a specific - * algorithm, or to the top-level IPA module. A field may be read by any - * algorithm, but should only be written by its owner. - */ - -/** - * \struct IPAContext - * \brief Global IPA context data shared between all algorithms - * - * \var IPAContext::configuration - * \brief The IPA session configuration, immutable during the session - * - * \var IPAContext::frameContext - * \brief The frame context for the frame being processed - * - * \todo While the frame context is supposed to be per-frame, this - * single frame context stores data related to both the current frame - * and the previous frames, with fields being updated as the algorithms - * are run. This needs to be turned into real per-frame data storage. - */ - /** * \var IPASessionConfiguration::agc * \brief AGC parameters configuration of the IPA @@ -115,6 +84,21 @@ namespace libcamera::ipa::rkisp1 { * \brief Sensor output resolution */ +/** + * \struct IPAFrameContext + * \brief Per-frame context for algorithms + * + * The frame context stores data specific to a single frame processed by the + * IPA. Each frame processed by the IPA has a context associated with it, + * accessible through the IPAContext structure. + * + * \todo Detail how to access contexts for a particular frame + * + * Each of the fields in the frame context belongs to either a specific + * algorithm, or to the top-level IPA module. A field may be read by any + * algorithm, but should only be written by its owner. + */ + /** * \var IPAFrameContext::agc * \brief Context for the Automatic Gain Control algorithm @@ -216,4 +200,20 @@ namespace libcamera::ipa::rkisp1 { * Algorithm::prepare() function of all algorithms. */ +/** + * \struct IPAContext + * \brief Global IPA context data shared between all algorithms + * + * \var IPAContext::configuration + * \brief The IPA session configuration, immutable during the session + * + * \var IPAContext::frameContext + * \brief The frame context for the frame being processed + * + * \todo While the frame context is supposed to be per-frame, this + * single frame context stores data related to both the current frame + * and the previous frames, with fields being updated as the algorithms + * are run. This needs to be turned into real per-frame data storage. + */ + } /* namespace libcamera::ipa::rkisp1 */ From patchwork Tue Sep 27 02:36:23 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Laurent Pinchart X-Patchwork-Id: 17425 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 51FBEC327E for ; Tue, 27 Sep 2022 02:37:22 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 0F32E6229F; Tue, 27 Sep 2022 04:37:22 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1664246242; bh=G3WZD+1BmFCDGAS/GMfZ/cVIJiB+/86bgp6jwKelkM0=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To: From; b=Aa3/weZe5IFWy2GNSx9KsuRAT7T+AzfrLfZ6Rn7LwCv7IiGJZuQXole0TnDFpDxR3 5rMxt2xnxYVP+wIgcv1RFrrg+UqH0pTutvFAduBDO6NrmEaA2q8xxZDlbCIg/xi42W grMH8IpuCce9DZvyEWvtQ3x727kVsDJay8GwJj1zV8vCZD5oPM/HQB/LXR23R1B7Ct +Xpj8UtrIyKtPHC+Uz1W2okK64l1xRnGtURUq6R0rXTzrEw0HwxzaleiE+ktMc2vib T+8qgUvh2PhRTXRrANV7QOa+nChB4C6ICVjZDIp7gKIuqVmZ26ibWWNk+fO7VrBPbs Cl/J6n6UsdMXg== Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 56BBC6228D for ; Tue, 27 Sep 2022 04:37:20 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="LXdTAyBZ"; dkim-atps=neutral Received: from pendragon.ideasonboard.com (62-78-145-57.bb.dnainternet.fi [62.78.145.57]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id C3F1947C; Tue, 27 Sep 2022 04:37:19 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1664246240; bh=G3WZD+1BmFCDGAS/GMfZ/cVIJiB+/86bgp6jwKelkM0=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=LXdTAyBZN2QDNsKg0mN7IJF2M6kwus/T3txRMyigyecm/4gf0dOcjVcYpn8rvLEF3 QCrrWm3w46wnhSzxdekhjBZ5+6/324J1MVoKyc7+kiJEVyTQ+Xnl5TT9DJpc55/iBx kOqolRkKcJAvzNiIZipJ4+27UawRSbkNjYNprEEQ= To: libcamera-devel@lists.libcamera.org Date: Tue, 27 Sep 2022 05:36:23 +0300 Message-Id: <20220927023642.12341-15-laurent.pinchart@ideasonboard.com> X-Mailer: git-send-email 2.35.1 In-Reply-To: <20220927023642.12341-1-laurent.pinchart@ideasonboard.com> References: <20220927023642.12341-1-laurent.pinchart@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v5 14/33] ipa: rkisp1: Rename frameContext to activeState X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Laurent Pinchart via libcamera-devel From: Laurent Pinchart Reply-To: Laurent Pinchart Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" From: Kieran Bingham The RkISP1 IPA module creates a single instance of its IPAFrameContext structure, effectively using it more as an active state than a per-frame context. To prepare for the introduction of a real per-frame context, move all the members of the IPAFrameContext structure to a new IPAActiveState structure. The IPAFrameContext becomes effectively unused at runtime, and will be populated back with per-frame data after converting the RkISP1 IPA module to using a frame context queue. The IPAActiveState structure will slowly morph into a different entity as individual algorithm get later ported to the frame context API. While at it, fix a typo in the documentation of the Agc::computeExposure() function that incorrectly refers to the frame context instead of the global context. Signed-off-by: Kieran Bingham Reviewed-by: Umang Jain Reviewed-by: Jacopo Mondi Signed-off-by: Jacopo Mondi Signed-off-by: Laurent Pinchart Reviewed-by: Kieran Bingham Reviewed-by: Jacopo Mondi --- Changes since v3: - Rewrite commit message --- src/ipa/rkisp1/algorithms/agc.cpp | 23 ++++--- src/ipa/rkisp1/algorithms/awb.cpp | 40 ++++++------ src/ipa/rkisp1/algorithms/blc.cpp | 2 +- src/ipa/rkisp1/algorithms/cproc.cpp | 4 +- src/ipa/rkisp1/algorithms/dpcc.cpp | 2 +- src/ipa/rkisp1/algorithms/dpf.cpp | 6 +- src/ipa/rkisp1/algorithms/filter.cpp | 4 +- src/ipa/rkisp1/algorithms/gsl.cpp | 2 +- src/ipa/rkisp1/algorithms/lsc.cpp | 2 +- src/ipa/rkisp1/ipa_context.cpp | 94 +++++++++++++++------------- src/ipa/rkisp1/ipa_context.h | 7 ++- src/ipa/rkisp1/rkisp1.cpp | 12 ++-- 12 files changed, 105 insertions(+), 93 deletions(-) diff --git a/src/ipa/rkisp1/algorithms/agc.cpp b/src/ipa/rkisp1/algorithms/agc.cpp index f9c33f81a074..7219cc5c37ab 100644 --- a/src/ipa/rkisp1/algorithms/agc.cpp +++ b/src/ipa/rkisp1/algorithms/agc.cpp @@ -73,8 +73,8 @@ Agc::Agc() int Agc::configure(IPAContext &context, const IPACameraSensorInfo &configInfo) { /* Configure the default exposure and gain. */ - context.frameContext.agc.gain = std::max(context.configuration.agc.minAnalogueGain, kMinAnalogueGain); - context.frameContext.agc.exposure = 10ms / context.configuration.sensor.lineDuration; + context.activeState.agc.gain = std::max(context.configuration.agc.minAnalogueGain, kMinAnalogueGain); + context.activeState.agc.exposure = 10ms / context.configuration.sensor.lineDuration; /* * According to the RkISP1 documentation: @@ -98,7 +98,10 @@ int Agc::configure(IPAContext &context, const IPACameraSensorInfo &configInfo) context.configuration.agc.measureWindow.h_size = 3 * configInfo.outputSize.width / 4; context.configuration.agc.measureWindow.v_size = 3 * configInfo.outputSize.height / 4; - /* \todo Use actual frame index by populating it in the frameContext. */ + /* + * \todo Use the upcoming per-frame context API that will provide a + * frame index + */ frameCount_ = 0; return 0; } @@ -140,18 +143,18 @@ utils::Duration Agc::filterExposure(utils::Duration exposureValue) /** * \brief Estimate the new exposure and gain values - * \param[inout] frameContext The shared IPA frame Context + * \param[inout] context The shared IPA Context * \param[in] yGain The gain calculated on the current brightness level * \param[in] iqMeanGain The gain calculated based on the relative luminance target */ void Agc::computeExposure(IPAContext &context, double yGain, double iqMeanGain) { IPASessionConfiguration &configuration = context.configuration; - IPAFrameContext &frameContext = context.frameContext; + IPAActiveState &activeState = context.activeState; /* Get the effective exposure and gain applied on the sensor. */ - uint32_t exposure = frameContext.sensor.exposure; - double analogueGain = frameContext.sensor.gain; + uint32_t exposure = activeState.sensor.exposure; + double analogueGain = activeState.sensor.gain; /* Use the highest of the two gain estimates. */ double evGain = std::max(yGain, iqMeanGain); @@ -216,8 +219,8 @@ void Agc::computeExposure(IPAContext &context, double yGain, double iqMeanGain) << stepGain; /* Update the estimated exposure and gain. */ - frameContext.agc.exposure = shutterTime / configuration.sensor.lineDuration; - frameContext.agc.gain = stepGain; + activeState.agc.exposure = shutterTime / configuration.sensor.lineDuration; + activeState.agc.gain = stepGain; } /** @@ -329,7 +332,7 @@ void Agc::prepare(IPAContext &context, [[maybe_unused]] IPAFrameContext &frameContext, rkisp1_params_cfg *params) { - if (context.frameContext.frameCount > 0) + if (context.activeState.frameCount > 0) return; /* Configure the measurement window. */ diff --git a/src/ipa/rkisp1/algorithms/awb.cpp b/src/ipa/rkisp1/algorithms/awb.cpp index a23c32015454..2bd9ef779bc3 100644 --- a/src/ipa/rkisp1/algorithms/awb.cpp +++ b/src/ipa/rkisp1/algorithms/awb.cpp @@ -36,10 +36,10 @@ LOG_DEFINE_CATEGORY(RkISP1Awb) int Awb::configure(IPAContext &context, const IPACameraSensorInfo &configInfo) { - context.frameContext.awb.gains.red = 1.0; - context.frameContext.awb.gains.blue = 1.0; - context.frameContext.awb.gains.green = 1.0; - context.frameContext.awb.autoEnabled = true; + context.activeState.awb.gains.red = 1.0; + context.activeState.awb.gains.blue = 1.0; + context.activeState.awb.gains.green = 1.0; + context.activeState.awb.autoEnabled = true; /* * Define the measurement window for AWB as a centered rectangle @@ -79,16 +79,16 @@ void Awb::prepare(IPAContext &context, [[maybe_unused]] IPAFrameContext &frameContext, rkisp1_params_cfg *params) { - params->others.awb_gain_config.gain_green_b = 256 * context.frameContext.awb.gains.green; - params->others.awb_gain_config.gain_blue = 256 * context.frameContext.awb.gains.blue; - params->others.awb_gain_config.gain_red = 256 * context.frameContext.awb.gains.red; - params->others.awb_gain_config.gain_green_r = 256 * context.frameContext.awb.gains.green; + params->others.awb_gain_config.gain_green_b = 256 * context.activeState.awb.gains.green; + params->others.awb_gain_config.gain_blue = 256 * context.activeState.awb.gains.blue; + params->others.awb_gain_config.gain_red = 256 * context.activeState.awb.gains.red; + params->others.awb_gain_config.gain_green_r = 256 * context.activeState.awb.gains.green; /* Update the gains. */ params->module_cfg_update |= RKISP1_CIF_ISP_MODULE_AWB_GAIN; /* If we already have configured the gains and window, return. */ - if (context.frameContext.frameCount > 0) + if (context.activeState.frameCount > 0) return; /* Configure the gains to apply. */ @@ -131,7 +131,7 @@ void Awb::queueRequest(IPAContext &context, [[maybe_unused]] IPAFrameContext &frameContext, const ControlList &controls) { - auto &awb = context.frameContext.awb; + auto &awb = context.activeState.awb; const auto &awbEnable = controls.get(controls::AwbEnable); if (awbEnable && *awbEnable != awb.autoEnabled) { @@ -162,7 +162,7 @@ void Awb::process([[maybe_unused]] IPAContext &context, { const rkisp1_cif_isp_stat *params = &stats->params; const rkisp1_cif_isp_awb_stat *awb = ¶ms->awb; - IPAFrameContext &frameContext = context.frameContext; + IPAActiveState &activeState = context.activeState; /* Get the YCbCr mean values */ double yMean = awb->awb_mean[0].mean_y_or_g; @@ -194,24 +194,24 @@ void Awb::process([[maybe_unused]] IPAContext &context, /* Filter the values to avoid oscillations. */ double speed = 0.2; - redGain = speed * redGain + (1 - speed) * frameContext.awb.gains.red; - blueGain = speed * blueGain + (1 - speed) * frameContext.awb.gains.blue; + redGain = speed * redGain + (1 - speed) * activeState.awb.gains.red; + blueGain = speed * blueGain + (1 - speed) * activeState.awb.gains.blue; /* * Gain values are unsigned integer value, range 0 to 4 with 8 bit * fractional part. */ - if (frameContext.awb.autoEnabled) { - frameContext.awb.gains.red = std::clamp(redGain, 0.0, 1023.0 / 256); - frameContext.awb.gains.blue = std::clamp(blueGain, 0.0, 1023.0 / 256); + if (activeState.awb.autoEnabled) { + activeState.awb.gains.red = std::clamp(redGain, 0.0, 1023.0 / 256); + activeState.awb.gains.blue = std::clamp(blueGain, 0.0, 1023.0 / 256); } /* Hardcode the green gain to 1.0. */ - frameContext.awb.gains.green = 1.0; + activeState.awb.gains.green = 1.0; - frameContext.awb.temperatureK = estimateCCT(redMean, greenMean, blueMean); + activeState.awb.temperatureK = estimateCCT(redMean, greenMean, blueMean); - LOG(RkISP1Awb, Debug) << "Gain found for red: " << context.frameContext.awb.gains.red - << " and for blue: " << context.frameContext.awb.gains.blue; + LOG(RkISP1Awb, Debug) << "Gain found for red: " << context.activeState.awb.gains.red + << " and for blue: " << context.activeState.awb.gains.blue; } REGISTER_IPA_ALGORITHM(Awb, "Awb") diff --git a/src/ipa/rkisp1/algorithms/blc.cpp b/src/ipa/rkisp1/algorithms/blc.cpp index c45a317ada08..0f7226cf217d 100644 --- a/src/ipa/rkisp1/algorithms/blc.cpp +++ b/src/ipa/rkisp1/algorithms/blc.cpp @@ -70,7 +70,7 @@ void BlackLevelCorrection::prepare(IPAContext &context, [[maybe_unused]] IPAFrameContext &frameContext, rkisp1_params_cfg *params) { - if (context.frameContext.frameCount > 0) + if (context.activeState.frameCount > 0) return; if (!tuningParameters_) diff --git a/src/ipa/rkisp1/algorithms/cproc.cpp b/src/ipa/rkisp1/algorithms/cproc.cpp index bc4a07073706..ea819b2acfcb 100644 --- a/src/ipa/rkisp1/algorithms/cproc.cpp +++ b/src/ipa/rkisp1/algorithms/cproc.cpp @@ -41,7 +41,7 @@ void ColorProcessing::queueRequest(IPAContext &context, [[maybe_unused]] IPAFrameContext &frameContext, const ControlList &controls) { - auto &cproc = context.frameContext.cproc; + auto &cproc = context.activeState.cproc; const auto &brightness = controls.get(controls::Brightness); if (brightness) { @@ -76,7 +76,7 @@ void ColorProcessing::prepare(IPAContext &context, [[maybe_unused]] IPAFrameContext &frameContext, rkisp1_params_cfg *params) { - auto &cproc = context.frameContext.cproc; + auto &cproc = context.activeState.cproc; /* Check if the algorithm configuration has been updated. */ if (!cproc.updateParams) diff --git a/src/ipa/rkisp1/algorithms/dpcc.cpp b/src/ipa/rkisp1/algorithms/dpcc.cpp index 5030526769c7..7c14ace0abee 100644 --- a/src/ipa/rkisp1/algorithms/dpcc.cpp +++ b/src/ipa/rkisp1/algorithms/dpcc.cpp @@ -263,7 +263,7 @@ void DefectPixelClusterCorrection::prepare(IPAContext &context, [[maybe_unused]] IPAFrameContext &frameContext, rkisp1_params_cfg *params) { - if (context.frameContext.frameCount > 0) + if (context.activeState.frameCount > 0) return; if (!initialized_) diff --git a/src/ipa/rkisp1/algorithms/dpf.cpp b/src/ipa/rkisp1/algorithms/dpf.cpp index be024fc5fe90..b8c837c21243 100644 --- a/src/ipa/rkisp1/algorithms/dpf.cpp +++ b/src/ipa/rkisp1/algorithms/dpf.cpp @@ -179,7 +179,7 @@ void Dpf::queueRequest(IPAContext &context, [[maybe_unused]] IPAFrameContext &frameContext, const ControlList &controls) { - auto &dpf = context.frameContext.dpf; + auto &dpf = context.activeState.dpf; const auto &denoise = controls.get(controls::draft::NoiseReductionMode); if (denoise) { @@ -214,9 +214,9 @@ void Dpf::prepare(IPAContext &context, [[maybe_unused]] const uint32_t frame, if (!initialized_) return; - auto &dpf = context.frameContext.dpf; + auto &dpf = context.activeState.dpf; - if (context.frameContext.frameCount == 0) { + if (context.activeState.frameCount == 0) { params->others.dpf_config = config_; params->others.dpf_strength_config = strengthConfig_; diff --git a/src/ipa/rkisp1/algorithms/filter.cpp b/src/ipa/rkisp1/algorithms/filter.cpp index 6aa5476aa25e..837560eb20a6 100644 --- a/src/ipa/rkisp1/algorithms/filter.cpp +++ b/src/ipa/rkisp1/algorithms/filter.cpp @@ -47,7 +47,7 @@ void Filter::queueRequest(IPAContext &context, [[maybe_unused]] IPAFrameContext &frameContext, const ControlList &controls) { - auto &filter = context.frameContext.filter; + auto &filter = context.activeState.filter; const auto &sharpness = controls.get(controls::Sharpness); if (sharpness) { @@ -91,7 +91,7 @@ void Filter::prepare(IPAContext &context, [[maybe_unused]] IPAFrameContext &frameContext, rkisp1_params_cfg *params) { - auto &filter = context.frameContext.filter; + auto &filter = context.activeState.filter; /* Check if the algorithm configuration has been updated. */ if (!filter.updateParams) diff --git a/src/ipa/rkisp1/algorithms/gsl.cpp b/src/ipa/rkisp1/algorithms/gsl.cpp index dfc76b3d8ffd..879ca2973e8a 100644 --- a/src/ipa/rkisp1/algorithms/gsl.cpp +++ b/src/ipa/rkisp1/algorithms/gsl.cpp @@ -123,7 +123,7 @@ void GammaSensorLinearization::prepare(IPAContext &context, [[maybe_unused]] IPAFrameContext &frameContext, rkisp1_params_cfg *params) { - if (context.frameContext.frameCount > 0) + if (context.activeState.frameCount > 0) return; if (!initialized_) diff --git a/src/ipa/rkisp1/algorithms/lsc.cpp b/src/ipa/rkisp1/algorithms/lsc.cpp index e4b04136dbaf..9c717bc5f99e 100644 --- a/src/ipa/rkisp1/algorithms/lsc.cpp +++ b/src/ipa/rkisp1/algorithms/lsc.cpp @@ -137,7 +137,7 @@ void LensShadingCorrection::prepare(IPAContext &context, [[maybe_unused]] IPAFrameContext &frameContext, rkisp1_params_cfg *params) { - if (context.frameContext.frameCount > 0) + if (context.activeState.frameCount > 0) return; if (!initialized_) diff --git a/src/ipa/rkisp1/ipa_context.cpp b/src/ipa/rkisp1/ipa_context.cpp index cc66bc70ac44..d18f4996aad2 100644 --- a/src/ipa/rkisp1/ipa_context.cpp +++ b/src/ipa/rkisp1/ipa_context.cpp @@ -85,114 +85,115 @@ namespace libcamera::ipa::rkisp1 { */ /** - * \struct IPAFrameContext - * \brief Per-frame context for algorithms + * \struct IPAActiveState + * \brief Active state for algorithms * - * The frame context stores data specific to a single frame processed by the - * IPA. Each frame processed by the IPA has a context associated with it, - * accessible through the IPAContext structure. + * The active state stores algorithm-specific data that needs to be shared + * between multiple algorithms and the IPA module. It is accessible through the + * IPAContext structure. * - * \todo Detail how to access contexts for a particular frame + * \todo Split the data contained in this structure between the active state + * and the frame contexts. * - * Each of the fields in the frame context belongs to either a specific + * Each of the fields in the active state belongs to either a specific * algorithm, or to the top-level IPA module. A field may be read by any * algorithm, but should only be written by its owner. */ /** - * \var IPAFrameContext::agc - * \brief Context for the Automatic Gain Control algorithm + * \var IPAActiveState::agc + * \brief State for the Automatic Gain Control algorithm * * The exposure and gain determined are expected to be applied to the sensor * at the earliest opportunity. * - * \var IPAFrameContext::agc.exposure + * \var IPAActiveState::agc.exposure * \brief Exposure time expressed as a number of lines * - * \var IPAFrameContext::agc.gain + * \var IPAActiveState::agc.gain * \brief Analogue gain multiplier * * The gain should be adapted to the sensor specific gain code before applying. */ /** - * \var IPAFrameContext::awb - * \brief Context for the Automatic White Balance algorithm + * \var IPAActiveState::awb + * \brief State for the Automatic White Balance algorithm * - * \struct IPAFrameContext::awb.gains + * \struct IPAActiveState::awb.gains * \brief White balance gains * - * \var IPAFrameContext::awb.gains.red + * \var IPAActiveState::awb.gains.red * \brief White balance gain for R channel * - * \var IPAFrameContext::awb.gains.green + * \var IPAActiveState::awb.gains.green * \brief White balance gain for G channel * - * \var IPAFrameContext::awb.gains.blue + * \var IPAActiveState::awb.gains.blue * \brief White balance gain for B channel * - * \var IPAFrameContext::awb.temperatureK + * \var IPAActiveState::awb.temperatureK * \brief Estimated color temperature * - * \var IPAFrameContext::awb.autoEnabled + * \var IPAActiveState::awb.autoEnabled * \brief Whether the Auto White Balance algorithm is enabled */ /** - * \var IPAFrameContext::cproc - * \brief Context for the Color Processing algorithm + * \var IPAActiveState::cproc + * \brief State for the Color Processing algorithm * - * \struct IPAFrameContext::cproc.brightness + * \struct IPAActiveState::cproc.brightness * \brief Brightness level * - * \var IPAFrameContext::cproc.contrast + * \var IPAActiveState::cproc.contrast * \brief Contrast level * - * \var IPAFrameContext::cproc.saturation + * \var IPAActiveState::cproc.saturation * \brief Saturation level * - * \var IPAFrameContext::cproc.updateParams + * \var IPAActiveState::cproc.updateParams * \brief Indicates if ISP parameters need to be updated */ /** - * \var IPAFrameContext::dpf - * \brief Context for the Denoise Pre-Filter algorithm + * \var IPAActiveState::dpf + * \brief State for the Denoise Pre-Filter algorithm * - * \var IPAFrameContext::dpf.denoise + * \var IPAActiveState::dpf.denoise * \brief Indicates if denoise is activated * - * \var IPAFrameContext::dpf.updateParams + * \var IPAActiveState::dpf.updateParams * \brief Indicates if ISP parameters need to be updated */ /** - * \var IPAFrameContext::filter - * \brief Context for the Filter algorithm + * \var IPAActiveState::filter + * \brief State for the Filter algorithm * - * \struct IPAFrameContext::filter.denoise + * \struct IPAActiveState::filter.denoise * \brief Denoising level * - * \var IPAFrameContext::filter.sharpness + * \var IPAActiveState::filter.sharpness * \brief Sharpness level * - * \var IPAFrameContext::filter.updateParams + * \var IPAActiveState::filter.updateParams * \brief Indicates if ISP parameters need to be updated */ /** - * \var IPAFrameContext::sensor + * \var IPAActiveState::sensor * \brief Effective sensor values * - * \var IPAFrameContext::sensor.exposure + * \var IPAActiveState::sensor.exposure * \brief Exposure time expressed as a number of lines * - * \var IPAFrameContext::sensor.gain + * \var IPAActiveState::sensor.gain * \brief Analogue gain multiplier */ /** - * \var IPAFrameContext::frameCount + * \var IPAActiveState::frameCount * \brief Counter of requests queued to the IPA module * * The counter is reset to 0 when the IPA module is configured, and is @@ -200,6 +201,14 @@ namespace libcamera::ipa::rkisp1 { * Algorithm::prepare() function of all algorithms. */ +/** + * \struct IPAFrameContext + * \brief Per-frame context for algorithms + * + * This structure is currently unused and will be replaced by a real per-frame + * context. + */ + /** * \struct IPAContext * \brief Global IPA context data shared between all algorithms @@ -207,13 +216,10 @@ namespace libcamera::ipa::rkisp1 { * \var IPAContext::configuration * \brief The IPA session configuration, immutable during the session * - * \var IPAContext::frameContext - * \brief The frame context for the frame being processed + * \var IPAContext::activeState + * \brief The IPA active state, storing the latest state for all algorithms * - * \todo While the frame context is supposed to be per-frame, this - * single frame context stores data related to both the current frame - * and the previous frames, with fields being updated as the algorithms - * are run. This needs to be turned into real per-frame data storage. + * \todo Introduce per-frame contexts */ } /* namespace libcamera::ipa::rkisp1 */ diff --git a/src/ipa/rkisp1/ipa_context.h b/src/ipa/rkisp1/ipa_context.h index 854151f2ca75..5590482c6a8c 100644 --- a/src/ipa/rkisp1/ipa_context.h +++ b/src/ipa/rkisp1/ipa_context.h @@ -46,7 +46,7 @@ struct IPASessionConfiguration { } hw; }; -struct IPAFrameContext { +struct IPAActiveState { struct { uint32_t exposure; double gain; @@ -89,9 +89,12 @@ struct IPAFrameContext { unsigned int frameCount; }; +struct IPAFrameContext { +}; + struct IPAContext { IPASessionConfiguration configuration; - IPAFrameContext frameContext; + IPAActiveState activeState; }; } /* namespace ipa::rkisp1 */ diff --git a/src/ipa/rkisp1/rkisp1.cpp b/src/ipa/rkisp1/rkisp1.cpp index f7a17459b6c3..24d5b9647838 100644 --- a/src/ipa/rkisp1/rkisp1.cpp +++ b/src/ipa/rkisp1/rkisp1.cpp @@ -244,7 +244,7 @@ int IPARkISP1::configure([[maybe_unused]] const IPACameraSensorInfo &info, context_.configuration.agc.minAnalogueGain = camHelper_->gain(minGain); context_.configuration.agc.maxAnalogueGain = camHelper_->gain(maxGain); - context_.frameContext.frameCount = 0; + context_.activeState.frameCount = 0; for (auto const &algo : algorithms()) { int ret = algo->configure(context_, info); @@ -310,7 +310,7 @@ void IPARkISP1::fillParamsBuffer(const uint32_t frame, const uint32_t bufferId) algo->prepare(context_, frame, frameContext, params); paramsBufferReady.emit(frame); - context_.frameContext.frameCount++; + context_.activeState.frameCount++; } void IPARkISP1::processStatsBuffer(const uint32_t frame, const uint32_t bufferId, @@ -320,9 +320,9 @@ void IPARkISP1::processStatsBuffer(const uint32_t frame, const uint32_t bufferId reinterpret_cast( mappedBuffers_.at(bufferId).planes()[0].data()); - context_.frameContext.sensor.exposure = + context_.activeState.sensor.exposure = sensorControls.get(V4L2_CID_EXPOSURE).get(); - context_.frameContext.sensor.gain = + context_.activeState.sensor.gain = camHelper_->gain(sensorControls.get(V4L2_CID_ANALOGUE_GAIN).get()); unsigned int aeState = 0; @@ -340,8 +340,8 @@ void IPARkISP1::processStatsBuffer(const uint32_t frame, const uint32_t bufferId void IPARkISP1::setControls(unsigned int frame) { - uint32_t exposure = context_.frameContext.agc.exposure; - uint32_t gain = camHelper_->gainCode(context_.frameContext.agc.gain); + uint32_t exposure = context_.activeState.agc.exposure; + uint32_t gain = camHelper_->gainCode(context_.activeState.agc.gain); ControlList ctrls(ctrls_); ctrls.set(V4L2_CID_EXPOSURE, static_cast(exposure)); From patchwork Tue Sep 27 02:36:24 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Laurent Pinchart X-Patchwork-Id: 17426 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id CAC6FC0DA4 for ; Tue, 27 Sep 2022 02:37:24 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 7FAA5622A5; Tue, 27 Sep 2022 04:37:24 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1664246244; bh=9lxCmbWCn93sXEIe1o8sdocG8NN5IOVflXuad5Vtn+s=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To: From; b=JQ9+TnQ7iI8gyA/54FIyV38WGJVvtDPDD8/z2KqVNNFptfFRGz6apJs1FyM2BzeiZ MM1LHF7QDjjD7hm/VbpASmNfHXeVVXMT8C9dca2RsG0I5GKDLzecL7633ufMvVR0tf srI9dtY6uD8LqXIkU7pKTtLuRNGdcB48UOa8wnkqD1T8labPh4KgKfQg1qWazAys3S PncYlIqeHdhIfoILmGnlerMBqOaIqJC1VIJnyTrONF1sHdRRCcHEq0cvTQRbrxEEPw ytxEvMLMRCRv2BRuBTQ0DOF0i5QgtbK14WMR4oObNGk+TMTfdeiBCN66q2FzRaj06a zOZC/MKKbCy3w== Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id BD2406229E for ; Tue, 27 Sep 2022 04:37:21 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="dZD8Fak0"; dkim-atps=neutral Received: from pendragon.ideasonboard.com (62-78-145-57.bb.dnainternet.fi [62.78.145.57]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 3D4FFE5 for ; Tue, 27 Sep 2022 04:37:21 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1664246241; bh=9lxCmbWCn93sXEIe1o8sdocG8NN5IOVflXuad5Vtn+s=; h=From:To:Subject:Date:In-Reply-To:References:From; b=dZD8Fak0ByhrfP314Qe2iEvYeE4yIzzIsNZgFERsNhhAlAUxeTGXUePkn1Mzv5lm7 oJ++RAnLeF0oGIi5mLyfB0gH/+NbhetxCNdElDpdc4TKXGqYJLebJXyMkacO0f1ysZ FB0TBTtT6o6IGFjTMhSjIsKdSBN+FDd1yfFvo0Nw= To: libcamera-devel@lists.libcamera.org Date: Tue, 27 Sep 2022 05:36:24 +0300 Message-Id: <20220927023642.12341-16-laurent.pinchart@ideasonboard.com> X-Mailer: git-send-email 2.35.1 In-Reply-To: <20220927023642.12341-1-laurent.pinchart@ideasonboard.com> References: <20220927023642.12341-1-laurent.pinchart@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v5 15/33] ipa: rkisp1: Use base FrameContext class X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Laurent Pinchart via libcamera-devel From: Laurent Pinchart Reply-To: Laurent Pinchart Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Inherit from the base FrameContext class in the RkISP1 IPAFrameContext. As the IPAFrameContext is currently unused, this change is a no-op, but it prepares the RkISP1 IPA module for frame context queue support. Signed-off-by: Laurent Pinchart Reviewed-by: Kieran Bingham Reviewed-by: Jacopo Mondi --- src/ipa/rkisp1/ipa_context.h | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/src/ipa/rkisp1/ipa_context.h b/src/ipa/rkisp1/ipa_context.h index 5590482c6a8c..f6b3e6eb951c 100644 --- a/src/ipa/rkisp1/ipa_context.h +++ b/src/ipa/rkisp1/ipa_context.h @@ -14,6 +14,8 @@ #include +#include + namespace libcamera { namespace ipa::rkisp1 { @@ -89,7 +91,7 @@ struct IPAActiveState { unsigned int frameCount; }; -struct IPAFrameContext { +struct IPAFrameContext : public FrameContext { }; struct IPAContext { From patchwork Tue Sep 27 02:36:25 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Laurent Pinchart X-Patchwork-Id: 17427 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id ACD4AC327F for ; Tue, 27 Sep 2022 02:37:25 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 53EEC622A8; Tue, 27 Sep 2022 04:37:25 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1664246245; bh=G/Q9G0Ix9+FJ55HHVnMqC8bFihq3OEZTJmwd/kyxP7c=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To: From; b=cLlsPabPd2fIabYKtfpXqx93Lvv8o3NsAvatlQ+DC9/6B5tDgIqHY71F10dFu7eY3 fdX+pudJOJmKWtYaJuUCBFCaypbE7e0yhAJlsi1+NJiOX2qvbBIXk/DQgwGnA0vmJ/ XbuH0nwhjcJ/hqdZK2OmYxi3Tw/i75OGVdnjqlpGoMeqW9EmNeIlv0YfKn9MUW99Vq CvLVD378gICs6s0iZxuxyeU3RQU+UJ+C3rbwkGa3W+pJcr6HMKiIgyMv1I1z9zzATg o2sR0/SYowUVtMZI3+xhc99Btz3DLrILI7lufJzk12n8HXBVQ2kRjbkfZ6PkE8Jwgj AuEiFFyYjSiSA== Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 42BC462291 for ; Tue, 27 Sep 2022 04:37:23 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="RW5OCZ92"; dkim-atps=neutral Received: from pendragon.ideasonboard.com (62-78-145-57.bb.dnainternet.fi [62.78.145.57]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id B1E314F7; Tue, 27 Sep 2022 04:37:22 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1664246243; bh=G/Q9G0Ix9+FJ55HHVnMqC8bFihq3OEZTJmwd/kyxP7c=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=RW5OCZ929PRS52o7fr/T3ENiFPTMM+nB6+AnwEwbDbTqbVe6wWZJl8bsjrHm4aGWH tFVzBJYtikR7ir1IyCPdX2P3Wl5Zkr5KlQUB4SvRhVdfoo8Pqe9VzCUWH9Y1SA//n/ j7SwzCfxMVVuBhuQCg7i282GI+xWqFBp2fwiNN5c= To: libcamera-devel@lists.libcamera.org Date: Tue, 27 Sep 2022 05:36:25 +0300 Message-Id: <20220927023642.12341-17-laurent.pinchart@ideasonboard.com> X-Mailer: git-send-email 2.35.1 In-Reply-To: <20220927023642.12341-1-laurent.pinchart@ideasonboard.com> References: <20220927023642.12341-1-laurent.pinchart@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v5 16/33] ipa: rkisp1: Convert to use the FCQueue X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Laurent Pinchart via libcamera-devel From: Laurent Pinchart Reply-To: Laurent Pinchart Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" From: Kieran Bingham Establish a queue of FrameContexts using the new FCQueue and use it to supply the FrameContext to the algorithms. The algorithms on the RKISP1 do not use this yet themselves, but are able to do so after the introduction of this patch. Signed-off-by: Kieran Bingham Signed-off-by: Jacopo Mondi Reviewed-by: Umang Jain Reviewed-by: Kieran Bingham Reviewed-by: Jacopo Mondi --- Changes since v4: - Rename FCQueue::init() to alloc() --- src/ipa/rkisp1/ipa_context.cpp | 6 +++--- src/ipa/rkisp1/ipa_context.h | 2 ++ src/ipa/rkisp1/rkisp1.cpp | 34 ++++++++++++++++++++++++---------- 3 files changed, 29 insertions(+), 13 deletions(-) diff --git a/src/ipa/rkisp1/ipa_context.cpp b/src/ipa/rkisp1/ipa_context.cpp index d18f4996aad2..e9846742ee4f 100644 --- a/src/ipa/rkisp1/ipa_context.cpp +++ b/src/ipa/rkisp1/ipa_context.cpp @@ -205,8 +205,7 @@ namespace libcamera::ipa::rkisp1 { * \struct IPAFrameContext * \brief Per-frame context for algorithms * - * This structure is currently unused and will be replaced by a real per-frame - * context. + * \todo Populate the frame context for all algorithms */ /** @@ -219,7 +218,8 @@ namespace libcamera::ipa::rkisp1 { * \var IPAContext::activeState * \brief The IPA active state, storing the latest state for all algorithms * - * \todo Introduce per-frame contexts + * \var IPAContext::frameContexts + * \brief Ring buffer of per-frame contexts */ } /* namespace libcamera::ipa::rkisp1 */ diff --git a/src/ipa/rkisp1/ipa_context.h b/src/ipa/rkisp1/ipa_context.h index f6b3e6eb951c..f6aaefffed52 100644 --- a/src/ipa/rkisp1/ipa_context.h +++ b/src/ipa/rkisp1/ipa_context.h @@ -97,6 +97,8 @@ struct IPAFrameContext : public FrameContext { struct IPAContext { IPASessionConfiguration configuration; IPAActiveState activeState; + + FCQueue frameContexts; }; } /* namespace ipa::rkisp1 */ diff --git a/src/ipa/rkisp1/rkisp1.cpp b/src/ipa/rkisp1/rkisp1.cpp index 24d5b9647838..eb3481949897 100644 --- a/src/ipa/rkisp1/rkisp1.cpp +++ b/src/ipa/rkisp1/rkisp1.cpp @@ -40,13 +40,18 @@ using namespace std::literals::chrono_literals; namespace ipa::rkisp1 { +/* Maximum number of frame contexts to be held */ +static constexpr uint32_t kMaxFrameContexts = 16; + class IPARkISP1 : public IPARkISP1Interface, public Module { public: + IPARkISP1(); + int init(const IPASettings &settings, unsigned int hwRevision, ControlInfoMap *ipaControls) override; int start() override; - void stop() override {} + void stop() override; int configure(const IPACameraSensorInfo &info, const std::map &streamConfig, @@ -100,6 +105,11 @@ const ControlInfoMap::Map rkisp1Controls{ } /* namespace */ +IPARkISP1::IPARkISP1() + : context_({ {}, {}, { kMaxFrameContexts } }) +{ +} + std::string IPARkISP1::logPrefix() const { return "rkisp1"; @@ -186,6 +196,11 @@ int IPARkISP1::start() return 0; } +void IPARkISP1::stop() +{ + context_.frameContexts.clear(); +} + /** * \todo The RkISP1 pipeline currently provides an empty IPACameraSensorInfo * if the connected sensor does not provide enough information to properly @@ -223,8 +238,10 @@ int IPARkISP1::configure([[maybe_unused]] const IPACameraSensorInfo &info, << "Exposure: " << minExposure << "-" << maxExposure << " Gain: " << minGain << "-" << maxGain; - /* Clean context at configuration */ - context_ = {}; + /* Clear the IPA context before the streaming session. */ + context_.configuration = {}; + context_.activeState = {}; + context_.frameContexts.clear(); /* Set the hardware revision for the algorithms. */ context_.configuration.hw.revision = hwRevision_; @@ -287,8 +304,7 @@ void IPARkISP1::unmapBuffers(const std::vector &ids) void IPARkISP1::queueRequest(const uint32_t frame, const ControlList &controls) { - /* \todo Obtain the frame context to pass to process from the FCQueue */ - IPAFrameContext frameContext; + IPAFrameContext &frameContext = context_.frameContexts.alloc(frame); for (auto const &algo : algorithms()) algo->queueRequest(context_, frame, frameContext, controls); @@ -296,8 +312,7 @@ void IPARkISP1::queueRequest(const uint32_t frame, const ControlList &controls) void IPARkISP1::fillParamsBuffer(const uint32_t frame, const uint32_t bufferId) { - /* \todo Obtain the frame context to pass to process from the FCQueue */ - IPAFrameContext frameContext; + IPAFrameContext &frameContext = context_.frameContexts.get(frame); rkisp1_params_cfg *params = reinterpret_cast( @@ -316,6 +331,8 @@ void IPARkISP1::fillParamsBuffer(const uint32_t frame, const uint32_t bufferId) void IPARkISP1::processStatsBuffer(const uint32_t frame, const uint32_t bufferId, const ControlList &sensorControls) { + IPAFrameContext &frameContext = context_.frameContexts.get(frame); + const rkisp1_stat_buffer *stats = reinterpret_cast( mappedBuffers_.at(bufferId).planes()[0].data()); @@ -327,9 +344,6 @@ void IPARkISP1::processStatsBuffer(const uint32_t frame, const uint32_t bufferId unsigned int aeState = 0; - /* \todo Obtain the frame context to pass to process from the FCQueue */ - IPAFrameContext frameContext; - for (auto const &algo : algorithms()) algo->process(context_, frame, frameContext, stats); From patchwork Tue Sep 27 02:36:26 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Laurent Pinchart X-Patchwork-Id: 17428 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 4AFC9C0DA4 for ; Tue, 27 Sep 2022 02:37:27 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 05208622AC; Tue, 27 Sep 2022 04:37:27 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1664246247; bh=3AtyT3ZLp2gghWB5SWqTYKKnJHG04bGQ4OdKMiqVpt0=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To: From; b=pIbst7zX4DMKEkF7G8Usb6dKMGYFY5v+98CR9ASPKhkqk9JNGH2yn3C8SisJvhAuS ZqAMudR0NObB4Lq88nSBtgWYjFCN+YxCb579UGpYHHw2gTkJt0VE0p/kCb9dwHiF5S ssq+lo7mmvOMV/zCZa0OoFOXgCDcnDBEXtPsTJx48TXI2fhttWMd0iB+BZJ8E1mlrb rbWipi6JVPtKGVCVfPTKPS+wVuL6yZgkwD9FFuDDtvPYytouPsdCyAnBi688TOzv5I NS2XhpaC+0FZlqdlJZ/kYfVgmBAOS4gHkuI+SealOGUmCliKiXKhr++0LSapO184lB PCAtpu5+EX/2g== Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 8CFFB622A6 for ; Tue, 27 Sep 2022 04:37:24 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="m9zaWpVF"; dkim-atps=neutral Received: from pendragon.ideasonboard.com (62-78-145-57.bb.dnainternet.fi [62.78.145.57]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 12326E5 for ; Tue, 27 Sep 2022 04:37:24 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1664246244; bh=3AtyT3ZLp2gghWB5SWqTYKKnJHG04bGQ4OdKMiqVpt0=; h=From:To:Subject:Date:In-Reply-To:References:From; b=m9zaWpVFqkVG42IO66MBtrB1Ej8I70Oo1h67RwxY6YhVIrFm9VRWSeTg8ZEQrz0zn 6JifCRAQo4LuaS7LmwI8tkeWWmGvVLAl5NJFzavBtQfSbIVdnmTc8Qa4lOHglTQVOq JTjfWMKdk2v9bL6CTBvuEEHmjzUorDdjQSkk93O0= To: libcamera-devel@lists.libcamera.org Date: Tue, 27 Sep 2022 05:36:26 +0300 Message-Id: <20220927023642.12341-18-laurent.pinchart@ideasonboard.com> X-Mailer: git-send-email 2.35.1 In-Reply-To: <20220927023642.12341-1-laurent.pinchart@ideasonboard.com> References: <20220927023642.12341-1-laurent.pinchart@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v5 17/33] ipa: libipa: Pass FCQueue size as template argument X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Laurent Pinchart via libcamera-devel From: Laurent Pinchart Reply-To: Laurent Pinchart Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" The frame context queue size is meant to be a compile-time constant, not a variable that can change between runs. Pass it to the class as a template argument. Signed-off-by: Laurent Pinchart Reviewed-by: Kieran Bingham --- src/ipa/ipu3/ipa_context.h | 5 ++++- src/ipa/ipu3/ipu3.cpp | 5 +---- src/ipa/libipa/fc_queue.cpp | 6 +++--- src/ipa/libipa/fc_queue.h | 17 ++++++++--------- src/ipa/rkisp1/ipa_context.h | 4 +++- src/ipa/rkisp1/rkisp1.cpp | 4 +--- 6 files changed, 20 insertions(+), 21 deletions(-) diff --git a/src/ipa/ipu3/ipa_context.h b/src/ipa/ipu3/ipa_context.h index 36099353e9f2..bdba3d07d13a 100644 --- a/src/ipa/ipu3/ipa_context.h +++ b/src/ipa/ipu3/ipa_context.h @@ -79,11 +79,14 @@ struct IPAFrameContext : public FrameContext { } sensor; }; +/* Maximum number of frame contexts to be held */ +static constexpr uint32_t kMaxFrameContexts = 16; + struct IPAContext { IPASessionConfiguration configuration; IPAActiveState activeState; - FCQueue frameContexts; + FCQueue frameContexts; }; } /* namespace ipa::ipu3 */ diff --git a/src/ipa/ipu3/ipu3.cpp b/src/ipa/ipu3/ipu3.cpp index d1ea081d595d..f5292a77e4d3 100644 --- a/src/ipa/ipu3/ipu3.cpp +++ b/src/ipa/ipu3/ipu3.cpp @@ -55,9 +55,6 @@ static constexpr uint32_t kMinCellSizeLog2 = 3; /* log2 of the maximum grid cell width and height, in pixels */ static constexpr uint32_t kMaxCellSizeLog2 = 6; -/* Maximum number of frame contexts to be held */ -static constexpr uint32_t kMaxFrameContexts = 16; - namespace libcamera { LOG_DEFINE_CATEGORY(IPAIPU3) @@ -191,7 +188,7 @@ private: }; IPAIPU3::IPAIPU3() - : context_({ {}, {}, { kMaxFrameContexts } }) + : context_({ {}, {}, {} }) { } diff --git a/src/ipa/libipa/fc_queue.cpp b/src/ipa/libipa/fc_queue.cpp index e812faa505a5..9b5702ec5eb0 100644 --- a/src/ipa/libipa/fc_queue.cpp +++ b/src/ipa/libipa/fc_queue.cpp @@ -42,6 +42,7 @@ namespace ipa { * \class FCQueue * \brief A support class for managing FrameContext instances in IPA modules * \tparam FrameContext The IPA module-specific FrameContext derived class type + * \tparam Size The number of contexts in the queue * * Along with the Module and Algorithm classes, the frame context queue is a * core component of the libipa infrastructure. It stores per-frame contexts @@ -90,9 +91,8 @@ namespace ipa { */ /** - * \fn FCQueue::FCQueue(unsigned int size) - * \brief Construct a frame contexts queue of a specified size - * \param[in] size The number of contexts in the queue + * \fn FCQueue::FCQueue() + * \brief Construct a frame contexts queue */ /** diff --git a/src/ipa/libipa/fc_queue.h b/src/ipa/libipa/fc_queue.h index a589e7e1031b..b9c8c6fe5a62 100644 --- a/src/ipa/libipa/fc_queue.h +++ b/src/ipa/libipa/fc_queue.h @@ -7,8 +7,8 @@ #pragma once +#include #include -#include #include @@ -18,21 +18,20 @@ LOG_DECLARE_CATEGORY(FCQueue) namespace ipa { -template +template class FCQueue; struct FrameContext { private: - template friend class FCQueue; + template friend class FCQueue; uint32_t frame; }; -template +template class FCQueue { public: - FCQueue(unsigned int size) - : contexts_(size) + FCQueue() { } @@ -44,7 +43,7 @@ public: FrameContext &alloc(const uint32_t frame) { - FrameContext &frameContext = contexts_[frame % contexts_.size()]; + FrameContext &frameContext = contexts_[frame % Size]; /* * Do not re-initialise if a get() call has already fetched this @@ -68,7 +67,7 @@ public: FrameContext &get(uint32_t frame) { - FrameContext &frameContext = contexts_[frame % contexts_.size()]; + FrameContext &frameContext = contexts_[frame % Size]; /* * If the IPA algorithms try to access a frame context slot which @@ -110,7 +109,7 @@ private: frameContext.frame = frame; } - std::vector contexts_; + std::array contexts_; }; } /* namespace ipa */ diff --git a/src/ipa/rkisp1/ipa_context.h b/src/ipa/rkisp1/ipa_context.h index f6aaefffed52..4481bd2acde6 100644 --- a/src/ipa/rkisp1/ipa_context.h +++ b/src/ipa/rkisp1/ipa_context.h @@ -94,11 +94,13 @@ struct IPAActiveState { struct IPAFrameContext : public FrameContext { }; +static constexpr uint32_t kMaxFrameContexts = 16; + struct IPAContext { IPASessionConfiguration configuration; IPAActiveState activeState; - FCQueue frameContexts; + FCQueue frameContexts; }; } /* namespace ipa::rkisp1 */ diff --git a/src/ipa/rkisp1/rkisp1.cpp b/src/ipa/rkisp1/rkisp1.cpp index eb3481949897..b7c1ecfee4c8 100644 --- a/src/ipa/rkisp1/rkisp1.cpp +++ b/src/ipa/rkisp1/rkisp1.cpp @@ -41,8 +41,6 @@ using namespace std::literals::chrono_literals; namespace ipa::rkisp1 { /* Maximum number of frame contexts to be held */ -static constexpr uint32_t kMaxFrameContexts = 16; - class IPARkISP1 : public IPARkISP1Interface, public Module { public: @@ -106,7 +104,7 @@ const ControlInfoMap::Map rkisp1Controls{ } /* namespace */ IPARkISP1::IPARkISP1() - : context_({ {}, {}, { kMaxFrameContexts } }) + : context_({ {}, {}, {} }) { } From patchwork Tue Sep 27 02:36:27 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Laurent Pinchart X-Patchwork-Id: 17429 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 07219C327E for ; Tue, 27 Sep 2022 02:37:29 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id B0445622B2; Tue, 27 Sep 2022 04:37:28 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1664246248; bh=rDrjId091+fWhPqSCtqNP1jK/fH3P/JmPOUiuveln84=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To: From; b=YC/Qsie4OYK+DQ2CY97cppoFYqq07t4X8DDPcrE/5Edx9iMm1/HRJ16cO9LiPnPjG zAyCYuL827+Dx9usVJfK+Y0fJproLuj0G7Y8X5EruD1fn+o4WTB5EFsY15vzewRiGW V9GlX2sg1fXYFhowrRuS/bAR1ylx+16A3z6G+ChNcrhtULNnAuWXAdgtawWV8YB6QX fblYKgje1NCzHqV6VJ5+k/SxrP8fMiGqQjTwQ2gHC6Nuzz/xXDcCF1UncK+YQPGEtF jp5LdnPzBNDSyTiJeOthEC9F0FKz2s1ENHv69m2Dcfl7euNm5RLe4iRaygpm6giPQn OsIqMG0yqQFXg== Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id D78D4622A9 for ; Tue, 27 Sep 2022 04:37:25 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="Ost+qxll"; dkim-atps=neutral Received: from pendragon.ideasonboard.com (62-78-145-57.bb.dnainternet.fi [62.78.145.57]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 5B67E47C for ; Tue, 27 Sep 2022 04:37:25 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1664246245; bh=rDrjId091+fWhPqSCtqNP1jK/fH3P/JmPOUiuveln84=; h=From:To:Subject:Date:In-Reply-To:References:From; b=Ost+qxllt68RDE3bK7vIYSKdylpVPWl6XL+2z4k7wnThQznq5xb3avUEFYKArM4Qh 0/9usctMYkkDytoM3Nd/Xm8oa3qgoBH/Zw0m2J5vNILvVhnMf4/rV2z4P29L+eaGUJ y91IarnFnuF+Z+Lc1+kEAx5Ofw0e/iyV+2PxKvSA= To: libcamera-devel@lists.libcamera.org Date: Tue, 27 Sep 2022 05:36:27 +0300 Message-Id: <20220927023642.12341-19-laurent.pinchart@ideasonboard.com> X-Mailer: git-send-email 2.35.1 In-Reply-To: <20220927023642.12341-1-laurent.pinchart@ideasonboard.com> References: <20220927023642.12341-1-laurent.pinchart@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v5 18/33] ipa: Disable copy-construction of IPAContext X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Laurent Pinchart via libcamera-devel From: Laurent Pinchart Reply-To: Laurent Pinchart Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" The IPAContext is not meant to be copied. Prevent unintentional pass-by-value by disabling its copy constructor. Signed-off-by: Laurent Pinchart Reviewed-by: Kieran Bingham Reviewed-by: Jacopo Mondi --- src/ipa/ipu3/ipa_context.h | 4 ++++ src/ipa/rkisp1/ipa_context.h | 4 ++++ 2 files changed, 8 insertions(+) diff --git a/src/ipa/ipu3/ipa_context.h b/src/ipa/ipu3/ipa_context.h index bdba3d07d13a..9766330f2f39 100644 --- a/src/ipa/ipu3/ipa_context.h +++ b/src/ipa/ipu3/ipa_context.h @@ -10,6 +10,7 @@ #include +#include #include #include @@ -87,6 +88,9 @@ struct IPAContext { IPAActiveState activeState; FCQueue frameContexts; + +private: + LIBCAMERA_DISABLE_COPY(IPAContext) }; } /* namespace ipa::ipu3 */ diff --git a/src/ipa/rkisp1/ipa_context.h b/src/ipa/rkisp1/ipa_context.h index 4481bd2acde6..9a84b7adda45 100644 --- a/src/ipa/rkisp1/ipa_context.h +++ b/src/ipa/rkisp1/ipa_context.h @@ -10,6 +10,7 @@ #include +#include #include #include @@ -101,6 +102,9 @@ struct IPAContext { IPAActiveState activeState; FCQueue frameContexts; + +private: + LIBCAMERA_DISABLE_COPY(IPAContext) }; } /* namespace ipa::rkisp1 */ From patchwork Tue Sep 27 02:36:28 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Laurent Pinchart X-Patchwork-Id: 17430 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 19062C0DA4 for ; Tue, 27 Sep 2022 02:37:30 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id C0EF9622A4; Tue, 27 Sep 2022 04:37:29 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1664246249; bh=1ZTMVZVGzQtgvJ31xn75MgNd1NZ0xA5ltcPWjKkZ88Q=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To: From; b=rGCRvgyZzeXGzWX4KWPxBM/3SC3ovlTjgQPsU19tfYcE6vIU7vbuiOIaHVyOuGdP0 lNKMAH7uNQ/vNg4Ivu8pQzGB+CZZXPAms5wRUCOqmYbUfbmhXruz/KpkFXcAP/K5jf azFElLeOYJMMh/F2u+sfSpaLkACbUPWfm+K3VAO9fIABsznLLD2xTYA/ZNybkgRq0g 97wnAFX3qaJ4Oy6bGQSJvVMHwkDqCgcVISFzAe6MzeJ2NVxvuahaStfvxVeD95+LAc Fn5eOrUgoQWjrDBzYcap3N0QVF+fGPsDaqTpE52S2RQa32llhbC+E/ffNo4ih8dcA9 mP0oYew1CCPmQ== Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 5BD21622A9 for ; Tue, 27 Sep 2022 04:37:27 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="JprMLA/e"; dkim-atps=neutral Received: from pendragon.ideasonboard.com (62-78-145-57.bb.dnainternet.fi [62.78.145.57]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id D50A5E5 for ; Tue, 27 Sep 2022 04:37:26 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1664246247; bh=1ZTMVZVGzQtgvJ31xn75MgNd1NZ0xA5ltcPWjKkZ88Q=; h=From:To:Subject:Date:In-Reply-To:References:From; b=JprMLA/evL/s8YF0PK8rL3EY9o4PyjkdSQRLsdHJ6QS5BVOVSU//8J5X8CDO8uRiY M3e1tr+k9xrqnZ+D2cJ3IE4CUNEdeWpQDPb9IElkApte61GRDH8zFNfJeZPvGv4FNy XJe8/AvEfXejaLel+Q7EG+uWsGATa3G+i4PUaTiw= To: libcamera-devel@lists.libcamera.org Date: Tue, 27 Sep 2022 05:36:28 +0300 Message-Id: <20220927023642.12341-20-laurent.pinchart@ideasonboard.com> X-Mailer: git-send-email 2.35.1 In-Reply-To: <20220927023642.12341-1-laurent.pinchart@ideasonboard.com> References: <20220927023642.12341-1-laurent.pinchart@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v5 19/33] ipa: rkisp1: Use frame number passed to Algorithm::prepare() X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Laurent Pinchart via libcamera-devel From: Laurent Pinchart Reply-To: Laurent Pinchart Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Now that the Algorithm::prepare() function takes a frame number, we can use it to replace the IPAActiveState::frameCount member. Signed-off-by: Laurent Pinchart Reviewed-by: Kieran Bingham Reviewed-by: Jacopo Mondi --- src/ipa/rkisp1/algorithms/agc.cpp | 5 ++--- src/ipa/rkisp1/algorithms/awb.cpp | 5 ++--- src/ipa/rkisp1/algorithms/blc.cpp | 6 +++--- src/ipa/rkisp1/algorithms/dpcc.cpp | 6 +++--- src/ipa/rkisp1/algorithms/dpf.cpp | 4 ++-- src/ipa/rkisp1/algorithms/gsl.cpp | 6 +++--- src/ipa/rkisp1/algorithms/lsc.cpp | 5 ++--- src/ipa/rkisp1/ipa_context.cpp | 9 --------- src/ipa/rkisp1/ipa_context.h | 2 -- src/ipa/rkisp1/rkisp1.cpp | 3 --- 10 files changed, 17 insertions(+), 34 deletions(-) diff --git a/src/ipa/rkisp1/algorithms/agc.cpp b/src/ipa/rkisp1/algorithms/agc.cpp index 7219cc5c37ab..87fc5d1ffec7 100644 --- a/src/ipa/rkisp1/algorithms/agc.cpp +++ b/src/ipa/rkisp1/algorithms/agc.cpp @@ -327,12 +327,11 @@ void Agc::process(IPAContext &context, [[maybe_unused]] const uint32_t frame, /** * \copydoc libcamera::ipa::Algorithm::prepare */ -void Agc::prepare(IPAContext &context, - [[maybe_unused]] const uint32_t frame, +void Agc::prepare(IPAContext &context, const uint32_t frame, [[maybe_unused]] IPAFrameContext &frameContext, rkisp1_params_cfg *params) { - if (context.activeState.frameCount > 0) + if (frame > 0) return; /* Configure the measurement window. */ diff --git a/src/ipa/rkisp1/algorithms/awb.cpp b/src/ipa/rkisp1/algorithms/awb.cpp index 2bd9ef779bc3..78398927e462 100644 --- a/src/ipa/rkisp1/algorithms/awb.cpp +++ b/src/ipa/rkisp1/algorithms/awb.cpp @@ -74,8 +74,7 @@ uint32_t Awb::estimateCCT(double red, double green, double blue) /** * \copydoc libcamera::ipa::Algorithm::prepare */ -void Awb::prepare(IPAContext &context, - [[maybe_unused]] const uint32_t frame, +void Awb::prepare(IPAContext &context, const uint32_t frame, [[maybe_unused]] IPAFrameContext &frameContext, rkisp1_params_cfg *params) { @@ -88,7 +87,7 @@ void Awb::prepare(IPAContext &context, params->module_cfg_update |= RKISP1_CIF_ISP_MODULE_AWB_GAIN; /* If we already have configured the gains and window, return. */ - if (context.activeState.frameCount > 0) + if (frame > 0) return; /* Configure the gains to apply. */ diff --git a/src/ipa/rkisp1/algorithms/blc.cpp b/src/ipa/rkisp1/algorithms/blc.cpp index 0f7226cf217d..15324fb11861 100644 --- a/src/ipa/rkisp1/algorithms/blc.cpp +++ b/src/ipa/rkisp1/algorithms/blc.cpp @@ -65,12 +65,12 @@ int BlackLevelCorrection::init([[maybe_unused]] IPAContext &context, /** * \copydoc libcamera::ipa::Algorithm::prepare */ -void BlackLevelCorrection::prepare(IPAContext &context, - [[maybe_unused]] const uint32_t frame, +void BlackLevelCorrection::prepare([[maybe_unused]] IPAContext &context, + const uint32_t frame, [[maybe_unused]] IPAFrameContext &frameContext, rkisp1_params_cfg *params) { - if (context.activeState.frameCount > 0) + if (frame > 0) return; if (!tuningParameters_) diff --git a/src/ipa/rkisp1/algorithms/dpcc.cpp b/src/ipa/rkisp1/algorithms/dpcc.cpp index 7c14ace0abee..53aa1d93e42b 100644 --- a/src/ipa/rkisp1/algorithms/dpcc.cpp +++ b/src/ipa/rkisp1/algorithms/dpcc.cpp @@ -258,12 +258,12 @@ int DefectPixelClusterCorrection::init([[maybe_unused]] IPAContext &context, /** * \copydoc libcamera::ipa::Algorithm::prepare */ -void DefectPixelClusterCorrection::prepare(IPAContext &context, - [[maybe_unused]] const uint32_t frame, +void DefectPixelClusterCorrection::prepare([[maybe_unused]] IPAContext &context, + const uint32_t frame, [[maybe_unused]] IPAFrameContext &frameContext, rkisp1_params_cfg *params) { - if (context.activeState.frameCount > 0) + if (frame > 0) return; if (!initialized_) diff --git a/src/ipa/rkisp1/algorithms/dpf.cpp b/src/ipa/rkisp1/algorithms/dpf.cpp index b8c837c21243..b8d0c7b85639 100644 --- a/src/ipa/rkisp1/algorithms/dpf.cpp +++ b/src/ipa/rkisp1/algorithms/dpf.cpp @@ -207,7 +207,7 @@ void Dpf::queueRequest(IPAContext &context, /** * \copydoc libcamera::ipa::Algorithm::prepare */ -void Dpf::prepare(IPAContext &context, [[maybe_unused]] const uint32_t frame, +void Dpf::prepare(IPAContext &context, const uint32_t frame, [[maybe_unused]] IPAFrameContext &frameContext, rkisp1_params_cfg *params) { @@ -216,7 +216,7 @@ void Dpf::prepare(IPAContext &context, [[maybe_unused]] const uint32_t frame, auto &dpf = context.activeState.dpf; - if (context.activeState.frameCount == 0) { + if (frame == 0) { params->others.dpf_config = config_; params->others.dpf_strength_config = strengthConfig_; diff --git a/src/ipa/rkisp1/algorithms/gsl.cpp b/src/ipa/rkisp1/algorithms/gsl.cpp index 879ca2973e8a..9cbad02038c4 100644 --- a/src/ipa/rkisp1/algorithms/gsl.cpp +++ b/src/ipa/rkisp1/algorithms/gsl.cpp @@ -118,12 +118,12 @@ int GammaSensorLinearization::init([[maybe_unused]] IPAContext &context, /** * \copydoc libcamera::ipa::Algorithm::prepare */ -void GammaSensorLinearization::prepare(IPAContext &context, - [[maybe_unused]] const uint32_t frame, +void GammaSensorLinearization::prepare([[maybe_unused]] IPAContext &context, + const uint32_t frame, [[maybe_unused]] IPAFrameContext &frameContext, rkisp1_params_cfg *params) { - if (context.activeState.frameCount > 0) + if (frame > 0) return; if (!initialized_) diff --git a/src/ipa/rkisp1/algorithms/lsc.cpp b/src/ipa/rkisp1/algorithms/lsc.cpp index 9c717bc5f99e..44245caa705e 100644 --- a/src/ipa/rkisp1/algorithms/lsc.cpp +++ b/src/ipa/rkisp1/algorithms/lsc.cpp @@ -132,12 +132,11 @@ int LensShadingCorrection::configure(IPAContext &context, /** * \copydoc libcamera::ipa::Algorithm::prepare */ -void LensShadingCorrection::prepare(IPAContext &context, - [[maybe_unused]] const uint32_t frame, +void LensShadingCorrection::prepare(IPAContext &context, const uint32_t frame, [[maybe_unused]] IPAFrameContext &frameContext, rkisp1_params_cfg *params) { - if (context.activeState.frameCount > 0) + if (frame > 0) return; if (!initialized_) diff --git a/src/ipa/rkisp1/ipa_context.cpp b/src/ipa/rkisp1/ipa_context.cpp index e9846742ee4f..78a785f5f982 100644 --- a/src/ipa/rkisp1/ipa_context.cpp +++ b/src/ipa/rkisp1/ipa_context.cpp @@ -192,15 +192,6 @@ namespace libcamera::ipa::rkisp1 { * \brief Analogue gain multiplier */ -/** - * \var IPAActiveState::frameCount - * \brief Counter of requests queued to the IPA module - * - * The counter is reset to 0 when the IPA module is configured, and is - * incremented for each request being queued, after calling the - * Algorithm::prepare() function of all algorithms. - */ - /** * \struct IPAFrameContext * \brief Per-frame context for algorithms diff --git a/src/ipa/rkisp1/ipa_context.h b/src/ipa/rkisp1/ipa_context.h index 9a84b7adda45..c7041ea2a214 100644 --- a/src/ipa/rkisp1/ipa_context.h +++ b/src/ipa/rkisp1/ipa_context.h @@ -88,8 +88,6 @@ struct IPAActiveState { uint32_t exposure; double gain; } sensor; - - unsigned int frameCount; }; struct IPAFrameContext : public FrameContext { diff --git a/src/ipa/rkisp1/rkisp1.cpp b/src/ipa/rkisp1/rkisp1.cpp index b7c1ecfee4c8..5409c2d5219e 100644 --- a/src/ipa/rkisp1/rkisp1.cpp +++ b/src/ipa/rkisp1/rkisp1.cpp @@ -259,8 +259,6 @@ int IPARkISP1::configure([[maybe_unused]] const IPACameraSensorInfo &info, context_.configuration.agc.minAnalogueGain = camHelper_->gain(minGain); context_.configuration.agc.maxAnalogueGain = camHelper_->gain(maxGain); - context_.activeState.frameCount = 0; - for (auto const &algo : algorithms()) { int ret = algo->configure(context_, info); if (ret) @@ -323,7 +321,6 @@ void IPARkISP1::fillParamsBuffer(const uint32_t frame, const uint32_t bufferId) algo->prepare(context_, frame, frameContext, params); paramsBufferReady.emit(frame); - context_.activeState.frameCount++; } void IPARkISP1::processStatsBuffer(const uint32_t frame, const uint32_t bufferId, From patchwork Tue Sep 27 02:36:29 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Laurent Pinchart X-Patchwork-Id: 17431 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id A09D0C327E for ; Tue, 27 Sep 2022 02:37:31 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 55D366225F; Tue, 27 Sep 2022 04:37:31 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1664246251; bh=0qL8UV+IYV7Thcn8LNEl5Kjr5myNT+7nhu/hzi2ILGw=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To: From; b=TiEUyWafZtmviYFT2ydrXfmpC2cVXXaMWGTGYhkWAaFou6nqAzD93a1ffAOfr0yjr lFxvFdr1Iijp/3W9cnXwvHUHjrn1FIMKsfHRpcVzm8JeVAnu+kwbh8W0yqmnuYHWOG faLQR0NK3huh4OXDEQ15paracuhPjeI1iA4KgnKbiLzzEy2rzDH6lJCOx5eHyZqGrm +5izJb8MBOYj3W+WJ/Dmk+ZWh/QYZcRK2oOMmEh++YnbQMm8dpuI27xOEOXBIzfeE5 2kFffNMTP493oL3UQxp/0NDs6vxJgvAQWR9WlhzmtZFyi/wi3m7tME7suU9Kr5nCwj h3AZBz++DrFRA== Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id BCAEE622B3 for ; Tue, 27 Sep 2022 04:37:28 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="gb7nXwv1"; dkim-atps=neutral Received: from pendragon.ideasonboard.com (62-78-145-57.bb.dnainternet.fi [62.78.145.57]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 3E38F47C for ; Tue, 27 Sep 2022 04:37:28 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1664246248; bh=0qL8UV+IYV7Thcn8LNEl5Kjr5myNT+7nhu/hzi2ILGw=; h=From:To:Subject:Date:In-Reply-To:References:From; b=gb7nXwv1yGtmZQWkgQfwxqWw0H9kHbFmqh28RXFOmB04gZR5i8XXHluM73fDAk0j8 mBF3MaV+zqSYQ7Kq7URbyFs0gu+JiOrLczF/8KLq7Zpit7H3IsdSuUr7db4wG34/Xz 7GZSCo9I6D7//4TCF8bi54hRulQ7H6C9tHCNAPsc= To: libcamera-devel@lists.libcamera.org Date: Tue, 27 Sep 2022 05:36:29 +0300 Message-Id: <20220927023642.12341-21-laurent.pinchart@ideasonboard.com> X-Mailer: git-send-email 2.35.1 In-Reply-To: <20220927023642.12341-1-laurent.pinchart@ideasonboard.com> References: <20220927023642.12341-1-laurent.pinchart@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v5 20/33] ipa: rkisp1: agc: Store per-frame information in frame context X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Laurent Pinchart via libcamera-devel From: Laurent Pinchart Reply-To: Laurent Pinchart Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Rework the algorithm's usage of the active state to store the value of controls for the last queued request in the queueRequest() function, and store a copy of the values in the corresponding frame context. The frame context is used in the prepare() function to populate the ISP parameters with values corresponding to the right frame. Signed-off-by: Laurent Pinchart Reviewed-by: Kieran Bingham Reviewed-by: Jacopo Mondi --- Changes since v4: - Add todo comments related to sensor controls --- src/ipa/rkisp1/algorithms/agc.cpp | 27 +++++++++++++------ src/ipa/rkisp1/algorithms/agc.h | 3 ++- src/ipa/rkisp1/ipa_context.cpp | 43 ++++++++++++++++++++----------- src/ipa/rkisp1/ipa_context.h | 14 ++++++---- src/ipa/rkisp1/rkisp1.cpp | 14 +++++++--- 5 files changed, 68 insertions(+), 33 deletions(-) diff --git a/src/ipa/rkisp1/algorithms/agc.cpp b/src/ipa/rkisp1/algorithms/agc.cpp index 87fc5d1ffec7..4540bde66db4 100644 --- a/src/ipa/rkisp1/algorithms/agc.cpp +++ b/src/ipa/rkisp1/algorithms/agc.cpp @@ -144,17 +144,19 @@ utils::Duration Agc::filterExposure(utils::Duration exposureValue) /** * \brief Estimate the new exposure and gain values * \param[inout] context The shared IPA Context + * \param[in] frameContext The FrameContext for this frame * \param[in] yGain The gain calculated on the current brightness level * \param[in] iqMeanGain The gain calculated based on the relative luminance target */ -void Agc::computeExposure(IPAContext &context, double yGain, double iqMeanGain) +void Agc::computeExposure(IPAContext &context, IPAFrameContext &frameContext, + double yGain, double iqMeanGain) { IPASessionConfiguration &configuration = context.configuration; IPAActiveState &activeState = context.activeState; /* Get the effective exposure and gain applied on the sensor. */ - uint32_t exposure = activeState.sensor.exposure; - double analogueGain = activeState.sensor.gain; + uint32_t exposure = frameContext.sensor.exposure; + double analogueGain = frameContext.sensor.gain; /* Use the highest of the two gain estimates. */ double evGain = std::max(yGain, iqMeanGain); @@ -286,9 +288,16 @@ double Agc::measureBrightness(const rkisp1_cif_isp_hist_stat *hist) const * new exposure and gain for the scene. */ void Agc::process(IPAContext &context, [[maybe_unused]] const uint32_t frame, - [[maybe_unused]] IPAFrameContext &frameContext, - const rkisp1_stat_buffer *stats) + IPAFrameContext &frameContext, const rkisp1_stat_buffer *stats) { + /* + * \todo Verify that the exposure and gain applied by the sensor for + * this frame match what has been requested. This isn't a hard + * requirement for stability of the AGC (the guarantee we need in + * automatic mode is a perfect match between the frame and the values + * we receive), but is important in manual mode. + */ + const rkisp1_cif_isp_stat *params = &stats->params; ASSERT(stats->meas_type & RKISP1_CIF_ISP_STAT_AUTOEXP); @@ -320,7 +329,7 @@ void Agc::process(IPAContext &context, [[maybe_unused]] const uint32_t frame, break; } - computeExposure(context, yGain, iqMeanGain); + computeExposure(context, frameContext, yGain, iqMeanGain); frameCount_++; } @@ -328,9 +337,11 @@ void Agc::process(IPAContext &context, [[maybe_unused]] const uint32_t frame, * \copydoc libcamera::ipa::Algorithm::prepare */ void Agc::prepare(IPAContext &context, const uint32_t frame, - [[maybe_unused]] IPAFrameContext &frameContext, - rkisp1_params_cfg *params) + IPAFrameContext &frameContext, rkisp1_params_cfg *params) { + frameContext.agc.exposure = context.activeState.agc.exposure; + frameContext.agc.gain = context.activeState.agc.gain; + if (frame > 0) return; diff --git a/src/ipa/rkisp1/algorithms/agc.h b/src/ipa/rkisp1/algorithms/agc.h index f115ba2ed85c..9ad5c32fd6f6 100644 --- a/src/ipa/rkisp1/algorithms/agc.h +++ b/src/ipa/rkisp1/algorithms/agc.h @@ -34,7 +34,8 @@ public: const rkisp1_stat_buffer *stats) override; private: - void computeExposure(IPAContext &Context, double yGain, double iqMeanGain); + void computeExposure(IPAContext &Context, IPAFrameContext &frameContext, + double yGain, double iqMeanGain); utils::Duration filterExposure(utils::Duration exposureValue); double estimateLuminance(const rkisp1_cif_isp_ae_stat *ae, double gain); double measureBrightness(const rkisp1_cif_isp_hist_stat *hist) const; diff --git a/src/ipa/rkisp1/ipa_context.cpp b/src/ipa/rkisp1/ipa_context.cpp index 78a785f5f982..c7d5b1b6ec5a 100644 --- a/src/ipa/rkisp1/ipa_context.cpp +++ b/src/ipa/rkisp1/ipa_context.cpp @@ -104,16 +104,13 @@ namespace libcamera::ipa::rkisp1 { * \var IPAActiveState::agc * \brief State for the Automatic Gain Control algorithm * - * The exposure and gain determined are expected to be applied to the sensor - * at the earliest opportunity. + * The exposure and gain are the latest values computed by the AGC algorithm. * * \var IPAActiveState::agc.exposure * \brief Exposure time expressed as a number of lines * * \var IPAActiveState::agc.gain * \brief Analogue gain multiplier - * - * The gain should be adapted to the sensor specific gain code before applying. */ /** @@ -181,17 +178,6 @@ namespace libcamera::ipa::rkisp1 { * \brief Indicates if ISP parameters need to be updated */ -/** - * \var IPAActiveState::sensor - * \brief Effective sensor values - * - * \var IPAActiveState::sensor.exposure - * \brief Exposure time expressed as a number of lines - * - * \var IPAActiveState::sensor.gain - * \brief Analogue gain multiplier - */ - /** * \struct IPAFrameContext * \brief Per-frame context for algorithms @@ -199,6 +185,33 @@ namespace libcamera::ipa::rkisp1 { * \todo Populate the frame context for all algorithms */ +/** + * \var IPAFrameContext::agc + * \brief Automatic Gain Control parameters for this frame + * + * The exposure and gain are provided by the AGC algorithm, and are to be + * applied to the sensor in order to take effect for this frame. + * + * \var IPAFrameContext::agc.exposure + * \brief Exposure time expressed as a number of lines + * + * \var IPAFrameContext::agc.gain + * \brief Analogue gain multiplier + * + * The gain should be adapted to the sensor specific gain code before applying. + */ + +/** + * \var IPAFrameContext::sensor + * \brief Sensor configuration that used been used for this frame + * + * \var IPAFrameContext::sensor.exposure + * \brief Exposure time expressed as a number of lines + * + * \var IPAFrameContext::sensor.gain + * \brief Analogue gain multiplier + */ + /** * \struct IPAContext * \brief Global IPA context data shared between all algorithms diff --git a/src/ipa/rkisp1/ipa_context.h b/src/ipa/rkisp1/ipa_context.h index c7041ea2a214..a4d134e700b5 100644 --- a/src/ipa/rkisp1/ipa_context.h +++ b/src/ipa/rkisp1/ipa_context.h @@ -83,14 +83,18 @@ struct IPAActiveState { uint8_t sharpness; bool updateParams; } filter; - - struct { - uint32_t exposure; - double gain; - } sensor; }; struct IPAFrameContext : public FrameContext { + struct { + uint32_t exposure; + double gain; + } agc; + + struct { + uint32_t exposure; + double gain; + } sensor; }; static constexpr uint32_t kMaxFrameContexts = 16; diff --git a/src/ipa/rkisp1/rkisp1.cpp b/src/ipa/rkisp1/rkisp1.cpp index 5409c2d5219e..6297916cf86d 100644 --- a/src/ipa/rkisp1/rkisp1.cpp +++ b/src/ipa/rkisp1/rkisp1.cpp @@ -332,9 +332,9 @@ void IPARkISP1::processStatsBuffer(const uint32_t frame, const uint32_t bufferId reinterpret_cast( mappedBuffers_.at(bufferId).planes()[0].data()); - context_.activeState.sensor.exposure = + frameContext.sensor.exposure = sensorControls.get(V4L2_CID_EXPOSURE).get(); - context_.activeState.sensor.gain = + frameContext.sensor.gain = camHelper_->gain(sensorControls.get(V4L2_CID_ANALOGUE_GAIN).get()); unsigned int aeState = 0; @@ -349,8 +349,14 @@ void IPARkISP1::processStatsBuffer(const uint32_t frame, const uint32_t bufferId void IPARkISP1::setControls(unsigned int frame) { - uint32_t exposure = context_.activeState.agc.exposure; - uint32_t gain = camHelper_->gainCode(context_.activeState.agc.gain); + /* + * \todo The frame number is most likely wrong here, we need to take + * internal sensor delays and other timing parameters into account. + */ + + IPAFrameContext &frameContext = context_.frameContexts.get(frame); + uint32_t exposure = frameContext.agc.exposure; + uint32_t gain = camHelper_->gainCode(frameContext.agc.gain); ControlList ctrls(ctrls_); ctrls.set(V4L2_CID_EXPOSURE, static_cast(exposure)); From patchwork Tue Sep 27 02:36:30 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Laurent Pinchart X-Patchwork-Id: 17432 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 45797C327F for ; Tue, 27 Sep 2022 02:37:32 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id D53BE622B3; Tue, 27 Sep 2022 04:37:31 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1664246251; bh=pzsNHOyS2d6C8T38VzDWDB0r24r0uuDEsGHDQ8B2HWM=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To: From; b=ploW/gmJpV7HKNnOtcGwT7b++E3v76dazYNwJdenoxZa0yHpToPMmSYKI4/GlOBhg oYeYDiXYrT+28hD4JNc5+QBZ2Yo4ynRX+zVJMvbDkn3SY27VzuB1DcezlnyCKSgZSL R6opOldHQdef9RBuXfhZVNQQZwbGQGU4C6MlSZGnLV3t7gAkJknxpAvQ/Cq2lH1iN+ RUZrD8KLAD123idmzw7DtjJKcRFe29AjJ73gBOpQR+c40iUVd5dSF/xGbjzdTUei68 cj7Znumr0qLCYVcYX3pXFnqQuOVAcXeUUG5PVdLTuzGzz6owvQG5jI92rKG2HAOfub 1KE8d2Fa2ciRw== Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 62E28622B1 for ; Tue, 27 Sep 2022 04:37:30 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="FH/zZbuB"; dkim-atps=neutral Received: from pendragon.ideasonboard.com (62-78-145-57.bb.dnainternet.fi [62.78.145.57]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id B174747C for ; Tue, 27 Sep 2022 04:37:29 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1664246250; bh=pzsNHOyS2d6C8T38VzDWDB0r24r0uuDEsGHDQ8B2HWM=; h=From:To:Subject:Date:In-Reply-To:References:From; b=FH/zZbuBExQizBKcUcsUOZGCRasTbcnVFUZQSmdMEejSartZEGWTV32docNYbyBfs i808aqg83BnWBGrHO3F9Z8k1XxPzLRmb5KbNZEYAMlDkytGTp4kdVVlmc9wkjO+P4O GbEwMtWEoksRO9BW4W9bZfbp4Pdhl4Qd4DAQLwlY= To: libcamera-devel@lists.libcamera.org Date: Tue, 27 Sep 2022 05:36:30 +0300 Message-Id: <20220927023642.12341-22-laurent.pinchart@ideasonboard.com> X-Mailer: git-send-email 2.35.1 In-Reply-To: <20220927023642.12341-1-laurent.pinchart@ideasonboard.com> References: <20220927023642.12341-1-laurent.pinchart@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v5 21/33] ipa: rkisp1: awb: Store per-frame information in frame context X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Laurent Pinchart via libcamera-devel From: Laurent Pinchart Reply-To: Laurent Pinchart Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Rework the algorithm's usage of the active state and frame context to store data in the right place. The active state stores two distinct categories of information: - The consolidated value of all algorithm controls. Requests passed to the queueRequest() function store values for controls that the application wants to modify for that particular frame, and the queueRequest() function updates the active state with those values. The active state thus contains a consolidated view of the value of all controls handled by the algorithm. - The value of parameters computed by the algorithm when running in auto mode. Algorithms running in auto mode compute new parameters every time statistics buffers are received (either synchronously, or possibly in a background thread). The latest computed value of those parameters is stored in the active state in the process() function. The frame context also stores two categories of information: - The value of the controls to be applied to the frame. These values are typically set in the queueRequest() function, from the consolidated control values stored in the active state. The frame context thus stores values for all controls related to the algorithm, not limited to the controls specified in the corresponding request, but consolidated from all requests that have been queued so far. For controls that can be specified manually or computed by the algorithm depending on the operation mode (such as the colour gains), the control value will be stored in the frame context in queueRequest() only when operating in manual mode. When operating in auto mode, the values are computed by the algorithm and stored in the frame context in prepare(), just before being stored in the ISP parameters buffer. The queueRequest() function can also store ancillary data in the frame context, such as flags to indicate if (and what) control values have changed compared to the previous request. - Status information computed by the algorithm for a frame. For instance, the colour temperature estimated by the algorithm from ISP statistics calculated on a frame is stored in the frame context for that frame in the process() function. The active state and frame context thus both contain identical members for most control values, but store values that have a different meaning. Signed-off-by: Laurent Pinchart Reviewed-by: Kieran Bingham --- src/ipa/rkisp1/algorithms/awb.cpp | 75 +++++++++++++++++++------------ src/ipa/rkisp1/ipa_context.cpp | 51 +++++++++++++++++---- src/ipa/rkisp1/ipa_context.h | 25 +++++++++-- 3 files changed, 110 insertions(+), 41 deletions(-) diff --git a/src/ipa/rkisp1/algorithms/awb.cpp b/src/ipa/rkisp1/algorithms/awb.cpp index 78398927e462..e491cf7507e0 100644 --- a/src/ipa/rkisp1/algorithms/awb.cpp +++ b/src/ipa/rkisp1/algorithms/awb.cpp @@ -36,9 +36,12 @@ LOG_DEFINE_CATEGORY(RkISP1Awb) int Awb::configure(IPAContext &context, const IPACameraSensorInfo &configInfo) { - context.activeState.awb.gains.red = 1.0; - context.activeState.awb.gains.blue = 1.0; - context.activeState.awb.gains.green = 1.0; + context.activeState.awb.gains.manual.red = 1.0; + context.activeState.awb.gains.manual.blue = 1.0; + context.activeState.awb.gains.manual.green = 1.0; + context.activeState.awb.gains.automatic.red = 1.0; + context.activeState.awb.gains.automatic.blue = 1.0; + context.activeState.awb.gains.automatic.green = 1.0; context.activeState.awb.autoEnabled = true; /* @@ -75,13 +78,22 @@ uint32_t Awb::estimateCCT(double red, double green, double blue) * \copydoc libcamera::ipa::Algorithm::prepare */ void Awb::prepare(IPAContext &context, const uint32_t frame, - [[maybe_unused]] IPAFrameContext &frameContext, - rkisp1_params_cfg *params) + IPAFrameContext &frameContext, rkisp1_params_cfg *params) { - params->others.awb_gain_config.gain_green_b = 256 * context.activeState.awb.gains.green; - params->others.awb_gain_config.gain_blue = 256 * context.activeState.awb.gains.blue; - params->others.awb_gain_config.gain_red = 256 * context.activeState.awb.gains.red; - params->others.awb_gain_config.gain_green_r = 256 * context.activeState.awb.gains.green; + /* + * This is the latest time we can read the active state. This is the + * most up-to-date automatic values we can read. + */ + if (frameContext.awb.autoEnabled) { + frameContext.awb.gains.red = context.activeState.awb.gains.automatic.red; + frameContext.awb.gains.green = context.activeState.awb.gains.automatic.green; + frameContext.awb.gains.blue = context.activeState.awb.gains.automatic.blue; + } + + params->others.awb_gain_config.gain_green_b = 256 * frameContext.awb.gains.green; + params->others.awb_gain_config.gain_blue = 256 * frameContext.awb.gains.blue; + params->others.awb_gain_config.gain_red = 256 * frameContext.awb.gains.red; + params->others.awb_gain_config.gain_green_r = 256 * frameContext.awb.gains.green; /* Update the gains. */ params->module_cfg_update |= RKISP1_CIF_ISP_MODULE_AWB_GAIN; @@ -127,7 +139,7 @@ void Awb::prepare(IPAContext &context, const uint32_t frame, */ void Awb::queueRequest(IPAContext &context, [[maybe_unused]] const uint32_t frame, - [[maybe_unused]] IPAFrameContext &frameContext, + IPAFrameContext &frameContext, const ControlList &controls) { auto &awb = context.activeState.awb; @@ -142,21 +154,29 @@ void Awb::queueRequest(IPAContext &context, const auto &colourGains = controls.get(controls::ColourGains); if (colourGains && !awb.autoEnabled) { - awb.gains.red = (*colourGains)[0]; - awb.gains.blue = (*colourGains)[1]; + awb.gains.manual.red = (*colourGains)[0]; + awb.gains.manual.blue = (*colourGains)[1]; LOG(RkISP1Awb, Debug) - << "Set colour gains to red: " << awb.gains.red - << ", blue: " << awb.gains.blue; + << "Set colour gains to red: " << awb.gains.manual.red + << ", blue: " << awb.gains.manual.blue; + } + + frameContext.awb.autoEnabled = awb.autoEnabled; + + if (!awb.autoEnabled) { + frameContext.awb.gains.red = awb.gains.manual.red; + frameContext.awb.gains.green = 1.0; + frameContext.awb.gains.blue = awb.gains.manual.blue; } } /** * \copydoc libcamera::ipa::Algorithm::process */ -void Awb::process([[maybe_unused]] IPAContext &context, +void Awb::process(IPAContext &context, [[maybe_unused]] const uint32_t frame, - [[maybe_unused]] IPAFrameContext &frameCtx, + IPAFrameContext &frameContext, const rkisp1_stat_buffer *stats) { const rkisp1_cif_isp_stat *params = &stats->params; @@ -187,30 +207,27 @@ void Awb::process([[maybe_unused]] IPAContext &context, double greenMean = 1.1636 * yMean - 0.4045 * cbMean - 0.7949 * crMean; double blueMean = 1.1636 * yMean + 1.9912 * cbMean - 0.0250 * crMean; + frameContext.awb.temperatureK = estimateCCT(redMean, greenMean, blueMean); + /* Estimate the red and blue gains to apply in a grey world. */ double redGain = greenMean / (redMean + 1); double blueGain = greenMean / (blueMean + 1); /* Filter the values to avoid oscillations. */ double speed = 0.2; - redGain = speed * redGain + (1 - speed) * activeState.awb.gains.red; - blueGain = speed * blueGain + (1 - speed) * activeState.awb.gains.blue; + redGain = speed * redGain + (1 - speed) * activeState.awb.gains.automatic.red; + blueGain = speed * blueGain + (1 - speed) * activeState.awb.gains.automatic.blue; /* * Gain values are unsigned integer value, range 0 to 4 with 8 bit - * fractional part. + * fractional part. Hardcode the green gain to 1.0. */ - if (activeState.awb.autoEnabled) { - activeState.awb.gains.red = std::clamp(redGain, 0.0, 1023.0 / 256); - activeState.awb.gains.blue = std::clamp(blueGain, 0.0, 1023.0 / 256); - } - /* Hardcode the green gain to 1.0. */ - activeState.awb.gains.green = 1.0; + activeState.awb.gains.automatic.red = std::clamp(redGain, 0.0, 1023.0 / 256); + activeState.awb.gains.automatic.blue = std::clamp(blueGain, 0.0, 1023.0 / 256); + activeState.awb.gains.automatic.green = 1.0; - activeState.awb.temperatureK = estimateCCT(redMean, greenMean, blueMean); - - LOG(RkISP1Awb, Debug) << "Gain found for red: " << context.activeState.awb.gains.red - << " and for blue: " << context.activeState.awb.gains.blue; + LOG(RkISP1Awb, Debug) << "Gain found for red: " << activeState.awb.gains.automatic.red + << " and for blue: " << activeState.awb.gains.automatic.blue; } REGISTER_IPA_ALGORITHM(Awb, "Awb") diff --git a/src/ipa/rkisp1/ipa_context.cpp b/src/ipa/rkisp1/ipa_context.cpp index c7d5b1b6ec5a..ba80a0707ef7 100644 --- a/src/ipa/rkisp1/ipa_context.cpp +++ b/src/ipa/rkisp1/ipa_context.cpp @@ -120,17 +120,29 @@ namespace libcamera::ipa::rkisp1 { * \struct IPAActiveState::awb.gains * \brief White balance gains * - * \var IPAActiveState::awb.gains.red - * \brief White balance gain for R channel + * \struct IPAActiveState::awb.gains.manual + * \brief Manual white balance gains (set through requests) * - * \var IPAActiveState::awb.gains.green - * \brief White balance gain for G channel + * \var IPAActiveState::awb.gains.manual.red + * \brief Manual white balance gain for R channel * - * \var IPAActiveState::awb.gains.blue - * \brief White balance gain for B channel + * \var IPAActiveState::awb.gains.manual.green + * \brief Manual white balance gain for G channel * - * \var IPAActiveState::awb.temperatureK - * \brief Estimated color temperature + * \var IPAActiveState::awb.gains.manual.blue + * \brief Manual white balance gain for B channel + * + * \struct IPAActiveState::awb.gains.automatic + * \brief Automatic white balance gains (computed by the algorithm) + * + * \var IPAActiveState::awb.gains.automatic.red + * \brief Automatic white balance gain for R channel + * + * \var IPAActiveState::awb.gains.automatic.green + * \brief Automatic white balance gain for G channel + * + * \var IPAActiveState::awb.gains.automatic.blue + * \brief Automatic white balance gain for B channel * * \var IPAActiveState::awb.autoEnabled * \brief Whether the Auto White Balance algorithm is enabled @@ -201,6 +213,29 @@ namespace libcamera::ipa::rkisp1 { * The gain should be adapted to the sensor specific gain code before applying. */ +/** + * \var IPAFrameContext::awb + * \brief Automatic White Balance parameters for this frame + * + * \struct IPAFrameContext::awb.gains + * \brief White balance gains + * + * \var IPAFrameContext::awb.gains.red + * \brief White balance gain for R channel + * + * \var IPAFrameContext::awb.gains.green + * \brief White balance gain for G channel + * + * \var IPAFrameContext::awb.gains.blue + * \brief White balance gain for B channel + * + * \var IPAFrameContext::awb.temperatureK + * \brief Estimated color temperature + * + * \var IPAFrameContext::awb.autoEnabled + * \brief Whether the Auto White Balance algorithm is enabled + */ + /** * \var IPAFrameContext::sensor * \brief Sensor configuration that used been used for this frame diff --git a/src/ipa/rkisp1/ipa_context.h b/src/ipa/rkisp1/ipa_context.h index a4d134e700b5..f23ebb2c9004 100644 --- a/src/ipa/rkisp1/ipa_context.h +++ b/src/ipa/rkisp1/ipa_context.h @@ -57,12 +57,18 @@ struct IPAActiveState { struct { struct { - double red; - double green; - double blue; + struct { + double red; + double green; + double blue; + } manual; + struct { + double red; + double green; + double blue; + } automatic; } gains; - double temperatureK; bool autoEnabled; } awb; @@ -91,6 +97,17 @@ struct IPAFrameContext : public FrameContext { double gain; } agc; + struct { + struct { + double red; + double green; + double blue; + } gains; + + double temperatureK; + bool autoEnabled; + } awb; + struct { uint32_t exposure; double gain; From patchwork Tue Sep 27 02:36:31 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Laurent Pinchart X-Patchwork-Id: 17433 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 037B1C0DA4 for ; Tue, 27 Sep 2022 02:37:33 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id A97E1622E3; Tue, 27 Sep 2022 04:37:33 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1664246253; bh=1Oxw78c4au4oiPhQy/oqLu6h/iNaB9uN1V5tPWQPJfM=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To: From; b=pTLM6QG7txzUWww5vtdUNgYN+CtoJVxLaMDycYW3LJGipjatzVWciUQVpzVd6FTex Ym2E8vfP9/x2oM1bHHHEhRBkjRySj2/4lSSiQZ/kqtYh/Y5mpexbH/DrE9iZXIMsup FIY3BkFb8r8eltIznWqOOsPiS8v52tZTx2J1qS7/PSWPJWieNT4H7lYTeqGVTyQY5y LnZFbxBOxin1HIYeWtGC/Q3nP/XxIOxA09hRSU+DooIyRyHhr3EMvWtBgsDamwUF1q 2y9elCqtTnVtCYHOLnyLfKhXAobeuYARLDgHPp4hTwOjwTvurtcwdX91FbLrhENGXd 6/I7lTXKzluUA== Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 0FA7762273 for ; Tue, 27 Sep 2022 04:37:33 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="t6p69Mr0"; dkim-atps=neutral Received: from pendragon.ideasonboard.com (62-78-145-57.bb.dnainternet.fi [62.78.145.57]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 2E3C647C for ; Tue, 27 Sep 2022 04:37:31 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1664246252; bh=1Oxw78c4au4oiPhQy/oqLu6h/iNaB9uN1V5tPWQPJfM=; h=From:To:Subject:Date:In-Reply-To:References:From; b=t6p69Mr0WqlVed7FE4L/Eed2ZeH7vk+Qghpet9NvtBRkRdqxAKKrcqhOg3MYgXmOR 86BDRjl1paOThneGpZ7GDss5CYjy46XeIRSEvNaVkG8EfkXEL7o/Aulf+NHS3nyWOI s5Oncj4PVygi5MuHFfpoeDHEpJaqoG7KasDI6lL4= To: libcamera-devel@lists.libcamera.org Date: Tue, 27 Sep 2022 05:36:31 +0300 Message-Id: <20220927023642.12341-23-laurent.pinchart@ideasonboard.com> X-Mailer: git-send-email 2.35.1 In-Reply-To: <20220927023642.12341-1-laurent.pinchart@ideasonboard.com> References: <20220927023642.12341-1-laurent.pinchart@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v5 22/33] ipa: rkisp1: cproc: Store per-frame information in frame context X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Laurent Pinchart via libcamera-devel From: Laurent Pinchart Reply-To: Laurent Pinchart Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Rework the algorithm's usage of the active state, to store the value of controls for the last queued request in the queueRequest() function, and store a copy of the values in the corresponding frame context. The latter is used in the prepare() function to populate the ISP parameters with values corresponding to the right frame. Signed-off-by: Laurent Pinchart Reviewed-by: Kieran Bingham Reviewed-by: Jacopo Mondi --- src/ipa/rkisp1/algorithms/cproc.cpp | 51 ++++++++++++++++++----------- src/ipa/rkisp1/ipa_context.cpp | 21 ++++++++++-- src/ipa/rkisp1/ipa_context.h | 8 ++++- 3 files changed, 56 insertions(+), 24 deletions(-) diff --git a/src/ipa/rkisp1/algorithms/cproc.cpp b/src/ipa/rkisp1/algorithms/cproc.cpp index ea819b2acfcb..eaa56c37f2bb 100644 --- a/src/ipa/rkisp1/algorithms/cproc.cpp +++ b/src/ipa/rkisp1/algorithms/cproc.cpp @@ -38,55 +38,66 @@ LOG_DEFINE_CATEGORY(RkISP1CProc) */ void ColorProcessing::queueRequest(IPAContext &context, [[maybe_unused]] const uint32_t frame, - [[maybe_unused]] IPAFrameContext &frameContext, + IPAFrameContext &frameContext, const ControlList &controls) { auto &cproc = context.activeState.cproc; + bool update = false; const auto &brightness = controls.get(controls::Brightness); if (brightness) { - cproc.brightness = std::clamp(std::lround(*brightness * 128), -128, 127); - cproc.updateParams = true; + int value = std::clamp(std::lround(*brightness * 128), -128, 127); + if (cproc.brightness != value) { + cproc.brightness = value; + update = true; + } - LOG(RkISP1CProc, Debug) << "Set brightness to " << *brightness; + LOG(RkISP1CProc, Debug) << "Set brightness to " << value; } const auto &contrast = controls.get(controls::Contrast); if (contrast) { - cproc.contrast = std::clamp(std::lround(*contrast * 128), 0, 255); - cproc.updateParams = true; + int value = std::clamp(std::lround(*contrast * 128), 0, 255); + if (cproc.contrast != value) { + cproc.contrast = value; + update = true; + } - LOG(RkISP1CProc, Debug) << "Set contrast to " << *contrast; + LOG(RkISP1CProc, Debug) << "Set contrast to " << value; } const auto saturation = controls.get(controls::Saturation); if (saturation) { - cproc.saturation = std::clamp(std::lround(*saturation * 128), 0, 255); - cproc.updateParams = true; + int value = std::clamp(std::lround(*saturation * 128), 0, 255); + if (cproc.saturation != value) { + cproc.saturation = value; + update = true; + } - LOG(RkISP1CProc, Debug) << "Set saturation to " << *saturation; + LOG(RkISP1CProc, Debug) << "Set saturation to " << value; } + + frameContext.cproc.brightness = cproc.brightness; + frameContext.cproc.contrast = cproc.contrast; + frameContext.cproc.saturation = cproc.saturation; + frameContext.cproc.update = update; } /** * \copydoc libcamera::ipa::Algorithm::prepare */ -void ColorProcessing::prepare(IPAContext &context, +void ColorProcessing::prepare([[maybe_unused]] IPAContext &context, [[maybe_unused]] const uint32_t frame, - [[maybe_unused]] IPAFrameContext &frameContext, + IPAFrameContext &frameContext, rkisp1_params_cfg *params) { - auto &cproc = context.activeState.cproc; - /* Check if the algorithm configuration has been updated. */ - if (!cproc.updateParams) + if (!frameContext.cproc.update) return; - cproc.updateParams = false; - - params->others.cproc_config.brightness = cproc.brightness; - params->others.cproc_config.contrast = cproc.contrast; - params->others.cproc_config.sat = cproc.saturation; + params->others.cproc_config.brightness = frameContext.cproc.brightness; + params->others.cproc_config.contrast = frameContext.cproc.contrast; + params->others.cproc_config.sat = frameContext.cproc.saturation; params->module_en_update |= RKISP1_CIF_ISP_MODULE_CPROC; params->module_ens |= RKISP1_CIF_ISP_MODULE_CPROC; diff --git a/src/ipa/rkisp1/ipa_context.cpp b/src/ipa/rkisp1/ipa_context.cpp index ba80a0707ef7..fdc8e7df0b76 100644 --- a/src/ipa/rkisp1/ipa_context.cpp +++ b/src/ipa/rkisp1/ipa_context.cpp @@ -160,9 +160,6 @@ namespace libcamera::ipa::rkisp1 { * * \var IPAActiveState::cproc.saturation * \brief Saturation level - * - * \var IPAActiveState::cproc.updateParams - * \brief Indicates if ISP parameters need to be updated */ /** @@ -236,6 +233,24 @@ namespace libcamera::ipa::rkisp1 { * \brief Whether the Auto White Balance algorithm is enabled */ +/** + * \var IPAFrameContext::cproc + * \brief Color Processing parameters for this frame + * + * \struct IPAFrameContext::cproc.brightness + * \brief Brightness level + * + * \var IPAFrameContext::cproc.contrast + * \brief Contrast level + * + * \var IPAFrameContext::cproc.saturation + * \brief Saturation level + * + * \var IPAFrameContext::cproc.update + * \brief Indicates if the color processing parameters have been updated + * compared to the previous frame + */ + /** * \var IPAFrameContext::sensor * \brief Sensor configuration that used been used for this frame diff --git a/src/ipa/rkisp1/ipa_context.h b/src/ipa/rkisp1/ipa_context.h index f23ebb2c9004..cc217aec511a 100644 --- a/src/ipa/rkisp1/ipa_context.h +++ b/src/ipa/rkisp1/ipa_context.h @@ -76,7 +76,6 @@ struct IPAActiveState { int8_t brightness; uint8_t contrast; uint8_t saturation; - bool updateParams; } cproc; struct { @@ -108,6 +107,13 @@ struct IPAFrameContext : public FrameContext { bool autoEnabled; } awb; + struct { + int8_t brightness; + uint8_t contrast; + uint8_t saturation; + bool update; + } cproc; + struct { uint32_t exposure; double gain; From patchwork Tue Sep 27 02:36:32 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Laurent Pinchart X-Patchwork-Id: 17434 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 9816DC0DA4 for ; Tue, 27 Sep 2022 02:37:38 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 52574622B6; Tue, 27 Sep 2022 04:37:38 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1664246258; bh=fM0v5GdYx5Qzd+oE/S32nz8rJN8nsoZ4NM3CXVXGwrQ=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To: From; b=IacQ7ENzYBxzWPBV91hKiCPOtmSm8LME/zCVOW8Rs5mXv7Om4CZyEzwfz32qgMz/s OhJXqq1mY3T/je4PsUHDfSWdb0ahG0Th9c7L4QR7RtLRcxPoPO9gpJnt1uRQN/m09L kMB/EKz/+A6WhYw3L7MbavKM4ZNBc4RmHsLcWBhKyKwbL71Ecj6xSFrv5yjhcfC0re ztDYn2kzdNiyNYxQSZxIcG44/ZiiroLitzTazMRqopXQB3FYrApQVHC2Jk/lhYpARY oFPWcUVFoDlzEYZg01kCt6mVb4xfifcKWg+xT4VnbOdziKroVqPXeD9EOH8HZZb/am PrdSVXXbW/D5A== Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 51A3B61F79 for ; Tue, 27 Sep 2022 04:37:36 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="BcMLDe2V"; dkim-atps=neutral Received: from pendragon.ideasonboard.com (62-78-145-57.bb.dnainternet.fi [62.78.145.57]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 7462FE5 for ; Tue, 27 Sep 2022 04:37:35 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1664246256; bh=fM0v5GdYx5Qzd+oE/S32nz8rJN8nsoZ4NM3CXVXGwrQ=; h=From:To:Subject:Date:In-Reply-To:References:From; b=BcMLDe2VYNdLrnw7EzVuzCn5UecXl9wBZy4ROmjrjdFo/WU12MJsqe5zlIPP/o4Rk Dy+jWTy7bbyN9pLpMYLqfNq3f0y0iQCYmBxz79Ygh2SUKerKAb1ahjxI7ladLml9aS 5nUR0vwsoTw7jnE1sA7mhW9X+9Q2S0C7N1OjYc6s= To: libcamera-devel@lists.libcamera.org Date: Tue, 27 Sep 2022 05:36:32 +0300 Message-Id: <20220927023642.12341-24-laurent.pinchart@ideasonboard.com> X-Mailer: git-send-email 2.35.1 In-Reply-To: <20220927023642.12341-1-laurent.pinchart@ideasonboard.com> References: <20220927023642.12341-1-laurent.pinchart@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v5 23/33] ipa: rkisp1: dpf: Store per-frame information in frame context X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Laurent Pinchart via libcamera-devel From: Laurent Pinchart Reply-To: Laurent Pinchart Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Rework the algorithm's usage of the active state, to store the value of controls for the last queued request in the queueRequest() function, and store a copy of the values in the corresponding frame context. The latter is used in the prepare() function to populate the ISP parameters with values corresponding to the right frame. Signed-off-by: Laurent Pinchart Reviewed-by: Kieran Bingham Reviewed-by: Jacopo Mondi --- Changes since v4: - Only update parameters when values have changed --- src/ipa/rkisp1/algorithms/dpf.cpp | 30 +++++++++++++++++------------- src/ipa/rkisp1/ipa_context.cpp | 15 ++++++++++++--- src/ipa/rkisp1/ipa_context.h | 6 +++++- 3 files changed, 34 insertions(+), 17 deletions(-) diff --git a/src/ipa/rkisp1/algorithms/dpf.cpp b/src/ipa/rkisp1/algorithms/dpf.cpp index b8d0c7b85639..f7bc371da52e 100644 --- a/src/ipa/rkisp1/algorithms/dpf.cpp +++ b/src/ipa/rkisp1/algorithms/dpf.cpp @@ -176,10 +176,11 @@ int Dpf::init([[maybe_unused]] IPAContext &context, */ void Dpf::queueRequest(IPAContext &context, [[maybe_unused]] const uint32_t frame, - [[maybe_unused]] IPAFrameContext &frameContext, + IPAFrameContext &frameContext, const ControlList &controls) { auto &dpf = context.activeState.dpf; + bool update = false; const auto &denoise = controls.get(controls::draft::NoiseReductionMode); if (denoise) { @@ -187,35 +188,40 @@ void Dpf::queueRequest(IPAContext &context, switch (*denoise) { case controls::draft::NoiseReductionModeOff: - dpf.denoise = false; - dpf.updateParams = true; + if (dpf.denoise) { + dpf.denoise = false; + update = true; + } break; case controls::draft::NoiseReductionModeMinimal: case controls::draft::NoiseReductionModeHighQuality: case controls::draft::NoiseReductionModeFast: - dpf.denoise = true; - dpf.updateParams = true; + if (!dpf.denoise) { + dpf.denoise = true; + update = true; + } break; default: LOG(RkISP1Dpf, Error) << "Unsupported denoise value " << *denoise; + break; } } + + frameContext.dpf.denoise = dpf.denoise; + frameContext.dpf.update = update; } /** * \copydoc libcamera::ipa::Algorithm::prepare */ void Dpf::prepare(IPAContext &context, const uint32_t frame, - [[maybe_unused]] IPAFrameContext &frameContext, - rkisp1_params_cfg *params) + IPAFrameContext &frameContext, rkisp1_params_cfg *params) { if (!initialized_) return; - auto &dpf = context.activeState.dpf; - if (frame == 0) { params->others.dpf_config = config_; params->others.dpf_strength_config = strengthConfig_; @@ -245,12 +251,10 @@ void Dpf::prepare(IPAContext &context, const uint32_t frame, RKISP1_CIF_ISP_MODULE_DPF_STRENGTH; } - if (dpf.updateParams) { + if (frameContext.dpf.update) { params->module_en_update |= RKISP1_CIF_ISP_MODULE_DPF; - if (dpf.denoise) + if (frameContext.dpf.denoise) params->module_ens |= RKISP1_CIF_ISP_MODULE_DPF; - - dpf.updateParams = false; } } diff --git a/src/ipa/rkisp1/ipa_context.cpp b/src/ipa/rkisp1/ipa_context.cpp index fdc8e7df0b76..c7cd007b2428 100644 --- a/src/ipa/rkisp1/ipa_context.cpp +++ b/src/ipa/rkisp1/ipa_context.cpp @@ -168,9 +168,6 @@ namespace libcamera::ipa::rkisp1 { * * \var IPAActiveState::dpf.denoise * \brief Indicates if denoise is activated - * - * \var IPAActiveState::dpf.updateParams - * \brief Indicates if ISP parameters need to be updated */ /** @@ -251,6 +248,18 @@ namespace libcamera::ipa::rkisp1 { * compared to the previous frame */ +/** + * \var IPAFrameContext::dpf + * \brief Denoise Pre-Filter parameters for this frame + * + * \var IPAFrameContext::dpf.denoise + * \brief Indicates if denoise is activated + * + * \var IPAFrameContext::dpf.update + * \brief Indicates if the denoise pre-filter parameters have been updated + * compared to the previous frame + */ + /** * \var IPAFrameContext::sensor * \brief Sensor configuration that used been used for this frame diff --git a/src/ipa/rkisp1/ipa_context.h b/src/ipa/rkisp1/ipa_context.h index cc217aec511a..81b632a15960 100644 --- a/src/ipa/rkisp1/ipa_context.h +++ b/src/ipa/rkisp1/ipa_context.h @@ -80,7 +80,6 @@ struct IPAActiveState { struct { bool denoise; - bool updateParams; } dpf; struct { @@ -114,6 +113,11 @@ struct IPAFrameContext : public FrameContext { bool update; } cproc; + struct { + bool denoise; + bool update; + } dpf; + struct { uint32_t exposure; double gain; From patchwork Tue Sep 27 02:36:33 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Laurent Pinchart X-Patchwork-Id: 17435 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 6AE3FC0DA4 for ; Tue, 27 Sep 2022 02:37:41 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 0564E622E8; Tue, 27 Sep 2022 04:37:41 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1664246261; bh=UlmoKdWfvwRreP4dtGHvpH32FV97sx3ax0ZcEdVQmGU=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To: From; b=1CB9txdd8pJpMFjIceK+G2TrYQCCWMGKdYv6TKPmjStyd9yznhk5CwNHwxgs56fQ8 SyY6HMzNGpEhZ8f/Idyu3i/Ubua8/NAr9QMg0laAmwYSFevM/9lzh13oJzqyiNwE0Y XKol3xdITxqBxzgMXcCA9/Ytx9gka+uvZoG8jzFdhDSf/ASPqYcU27+0Fq20EE9miE Dn71K/IPOOCtK1SyKM3AuQeWhcMyRLs0eZqqJLfixsbj9NOxxo+OpgQOhB42rS0v+2 sR1uh8TOihSukCaURKez8dtFregd93kpMrBQ824y42QZg+pC4bSa62wceX3Ddzybp5 8eEDP1nUBUaqQ== Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id B7AB762268 for ; Tue, 27 Sep 2022 04:37:37 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="VusSu2EU"; dkim-atps=neutral Received: from pendragon.ideasonboard.com (62-78-145-57.bb.dnainternet.fi [62.78.145.57]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 30FA0E5 for ; Tue, 27 Sep 2022 04:37:37 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1664246257; bh=UlmoKdWfvwRreP4dtGHvpH32FV97sx3ax0ZcEdVQmGU=; h=From:To:Subject:Date:In-Reply-To:References:From; b=VusSu2EUhgIwMq31HAyRZPDBLm15c1BS1pvqPZlb2HWZ9C2Jnf6lirWbZxvjTB1IM 8Ncki0/Fs3cLzDBcN9SgoRHhn4U8qQP8nnc9SGtZANQ8dYMncUhsQfIevUp9ydL/O5 xRa3qg77tyCV6zcV50UdDQlfR+homop95UyDxyHw= To: libcamera-devel@lists.libcamera.org Date: Tue, 27 Sep 2022 05:36:33 +0300 Message-Id: <20220927023642.12341-25-laurent.pinchart@ideasonboard.com> X-Mailer: git-send-email 2.35.1 In-Reply-To: <20220927023642.12341-1-laurent.pinchart@ideasonboard.com> References: <20220927023642.12341-1-laurent.pinchart@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v5 24/33] ipa: rkisp1: filter: Store per-frame information in frame context X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Laurent Pinchart via libcamera-devel From: Laurent Pinchart Reply-To: Laurent Pinchart Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Rework the algorithm's usage of the active state, to store the value of controls for the last queued request in the queueRequest() function, and store a copy of the values in the corresponding frame context. The latter is used in the prepare() function to populate the ISP parameters with values corresponding to the right frame. Signed-off-by: Laurent Pinchart Reviewed-by: Kieran Bingham Reviewed-by: Jacopo Mondi --- Changes since v4: - Only update parameters when values have changed --- src/ipa/rkisp1/algorithms/filter.cpp | 49 +++++++++++++++++----------- src/ipa/rkisp1/ipa_context.cpp | 18 ++++++++-- src/ipa/rkisp1/ipa_context.h | 7 +++- 3 files changed, 51 insertions(+), 23 deletions(-) diff --git a/src/ipa/rkisp1/algorithms/filter.cpp b/src/ipa/rkisp1/algorithms/filter.cpp index 837560eb20a6..4b89c05a52c4 100644 --- a/src/ipa/rkisp1/algorithms/filter.cpp +++ b/src/ipa/rkisp1/algorithms/filter.cpp @@ -44,15 +44,20 @@ static constexpr uint32_t kFiltModeDefault = 0x000004f2; */ void Filter::queueRequest(IPAContext &context, [[maybe_unused]] const uint32_t frame, - [[maybe_unused]] IPAFrameContext &frameContext, + IPAFrameContext &frameContext, const ControlList &controls) { auto &filter = context.activeState.filter; + bool update = false; const auto &sharpness = controls.get(controls::Sharpness); if (sharpness) { - filter.sharpness = std::round(std::clamp(*sharpness, 0.0f, 10.0f)); - filter.updateParams = true; + unsigned int value = std::round(std::clamp(*sharpness, 0.0f, 10.0f)); + + if (filter.sharpness != value) { + filter.sharpness = value; + update = true; + } LOG(RkISP1Filter, Debug) << "Set sharpness to " << *sharpness; } @@ -63,42 +68,48 @@ void Filter::queueRequest(IPAContext &context, switch (*denoise) { case controls::draft::NoiseReductionModeOff: - filter.denoise = 0; - filter.updateParams = true; + if (filter.denoise != 0) { + filter.denoise = 0; + update = true; + } break; case controls::draft::NoiseReductionModeMinimal: - filter.denoise = 1; - filter.updateParams = true; + if (filter.denoise != 1) { + filter.denoise = 1; + update = true; + } break; case controls::draft::NoiseReductionModeHighQuality: case controls::draft::NoiseReductionModeFast: - filter.denoise = 3; - filter.updateParams = true; + if (filter.denoise != 3) { + filter.denoise = 3; + update = true; + } break; default: LOG(RkISP1Filter, Error) << "Unsupported denoise value " << *denoise; + break; } } + + frameContext.filter.denoise = filter.denoise; + frameContext.filter.sharpness = filter.sharpness; + frameContext.filter.update = update; } /** * \copydoc libcamera::ipa::Algorithm::prepare */ -void Filter::prepare(IPAContext &context, +void Filter::prepare([[maybe_unused]] IPAContext &context, [[maybe_unused]] const uint32_t frame, - [[maybe_unused]] IPAFrameContext &frameContext, - rkisp1_params_cfg *params) + IPAFrameContext &frameContext, rkisp1_params_cfg *params) { - auto &filter = context.activeState.filter; - /* Check if the algorithm configuration has been updated. */ - if (!filter.updateParams) + if (!frameContext.filter.update) return; - filter.updateParams = false; - static constexpr uint16_t filt_fac_sh0[] = { 0x04, 0x07, 0x0a, 0x0c, 0x10, 0x14, 0x1a, 0x1e, 0x24, 0x2a, 0x30 }; @@ -147,8 +158,8 @@ void Filter::prepare(IPAContext &context, 0, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3 }; - uint8_t denoise = filter.denoise; - uint8_t sharpness = filter.sharpness; + uint8_t denoise = frameContext.filter.denoise; + uint8_t sharpness = frameContext.filter.sharpness; auto &flt_config = params->others.flt_config; flt_config.fac_sh0 = filt_fac_sh0[sharpness]; diff --git a/src/ipa/rkisp1/ipa_context.cpp b/src/ipa/rkisp1/ipa_context.cpp index c7cd007b2428..4380634e272d 100644 --- a/src/ipa/rkisp1/ipa_context.cpp +++ b/src/ipa/rkisp1/ipa_context.cpp @@ -179,9 +179,6 @@ namespace libcamera::ipa::rkisp1 { * * \var IPAActiveState::filter.sharpness * \brief Sharpness level - * - * \var IPAActiveState::filter.updateParams - * \brief Indicates if ISP parameters need to be updated */ /** @@ -260,6 +257,21 @@ namespace libcamera::ipa::rkisp1 { * compared to the previous frame */ +/** + * \var IPAFrameContext::filter + * \brief Filter parameters for this frame + * + * \struct IPAFrameContext::filter.denoise + * \brief Denoising level + * + * \var IPAFrameContext::filter.sharpness + * \brief Sharpness level + * + * \var IPAFrameContext::filter.updateParams + * \brief Indicates if the filter parameters have been updated compared to the + * previous frame + */ + /** * \var IPAFrameContext::sensor * \brief Sensor configuration that used been used for this frame diff --git a/src/ipa/rkisp1/ipa_context.h b/src/ipa/rkisp1/ipa_context.h index 81b632a15960..9a16dd6cf108 100644 --- a/src/ipa/rkisp1/ipa_context.h +++ b/src/ipa/rkisp1/ipa_context.h @@ -85,7 +85,6 @@ struct IPAActiveState { struct { uint8_t denoise; uint8_t sharpness; - bool updateParams; } filter; }; @@ -118,6 +117,12 @@ struct IPAFrameContext : public FrameContext { bool update; } dpf; + struct { + uint8_t denoise; + uint8_t sharpness; + bool update; + } filter; + struct { uint32_t exposure; double gain; From patchwork Tue Sep 27 02:36:34 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Laurent Pinchart X-Patchwork-Id: 17436 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 44E29C327E for ; Tue, 27 Sep 2022 02:37:42 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id F26D962278; Tue, 27 Sep 2022 04:37:41 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1664246262; bh=DAKwY6U4JDfQRn2KCymAmlS4p4TPatrnAha6ECJUbm8=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To: From; b=MYIFvzovhJfzLjSOx2awnevS1+ue1AW2tJHk97LEO2rqFlFLJejhDHpJUyxjMh2py puVWbg+clih0DEAh2SX6I8v1vMk7KLlBwg/gnmUfVmIUUjAm8VpAJchZKNYiBDuqiy QXEpwEpURIl3kRrmsPBWlj1Y1tDWogbdcABSuqrLwf/ZrSAu3G8VzUNYvVuQ2m79R0 kRPb39lx5s96sul/lfdBCAX1tFvMzr2PKC/fdzybBkF7aH8HDoeXyetM6u1XgZsjj5 gVt0rVe8/dKdPQ6CJg6NuvXSZFfiGexa76bSDhaau8MAa7ghvW/nh0ooMyfbgr53GP dFS20tmOrxoPQ== Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 310AF62273 for ; Tue, 27 Sep 2022 04:37:39 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="k93u81/M"; dkim-atps=neutral Received: from pendragon.ideasonboard.com (62-78-145-57.bb.dnainternet.fi [62.78.145.57]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id AAD93E5 for ; Tue, 27 Sep 2022 04:37:38 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1664246258; bh=DAKwY6U4JDfQRn2KCymAmlS4p4TPatrnAha6ECJUbm8=; h=From:To:Subject:Date:In-Reply-To:References:From; b=k93u81/M2YrrvSlA18C2caruA3w8jvFvSar2LYEzseZ3wtybs8rZQpIeBzSGtnseY qUGwT9aPkN/u+HlNU+zng7OS733/nqkwlWn1s+VvPphP+Gdh3aJsB+ueE14EHVHire 7aA6f8qzYm/+VcBhnGKIOHOjOJogH2r2zI9BN370= To: libcamera-devel@lists.libcamera.org Date: Tue, 27 Sep 2022 05:36:34 +0300 Message-Id: <20220927023642.12341-26-laurent.pinchart@ideasonboard.com> X-Mailer: git-send-email 2.35.1 In-Reply-To: <20220927023642.12341-1-laurent.pinchart@ideasonboard.com> References: <20220927023642.12341-1-laurent.pinchart@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v5 25/33] ipa: rkisp1: Document the active state and frame context X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Laurent Pinchart via libcamera-devel From: Laurent Pinchart Reply-To: Laurent Pinchart Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Now that data used by algorithms has been partitioned between the active state and frame context, we have a better view of the role of each of those structures. Document them appropriately. Signed-off-by: Laurent Pinchart Reviewed-by: Kieran Bingham Reviewed-by: Jacopo Mondi --- Changes since v4: - Fix typo in documentation --- src/ipa/rkisp1/ipa_context.cpp | 55 ++++++++++++++++++++++++++++------ 1 file changed, 46 insertions(+), 9 deletions(-) diff --git a/src/ipa/rkisp1/ipa_context.cpp b/src/ipa/rkisp1/ipa_context.cpp index 4380634e272d..2126ed1d0acb 100644 --- a/src/ipa/rkisp1/ipa_context.cpp +++ b/src/ipa/rkisp1/ipa_context.cpp @@ -88,16 +88,28 @@ namespace libcamera::ipa::rkisp1 { * \struct IPAActiveState * \brief Active state for algorithms * - * The active state stores algorithm-specific data that needs to be shared - * between multiple algorithms and the IPA module. It is accessible through the - * IPAContext structure. + * The active state contains all algorithm-specific data that needs to be + * maintained by algorithms across frames. Unlike the session configuration, + * the active state is mutable and constantly updated by algorithms. The active + * state is accessible through the IPAContext structure. * - * \todo Split the data contained in this structure between the active state - * and the frame contexts. + * The active state stores two distinct categories of information: * - * Each of the fields in the active state belongs to either a specific - * algorithm, or to the top-level IPA module. A field may be read by any - * algorithm, but should only be written by its owner. + * - The consolidated value of all algorithm controls. Requests passed to + * the queueRequest() function store values for controls that the + * application wants to modify for that particular frame, and the + * queueRequest() function updates the active state with those values. + * The active state thus contains a consolidated view of the value of all + * controls handled by the algorithm. + * + * - The value of parameters computed by the algorithm when running in auto + * mode. Algorithms running in auto mode compute new parameters every + * time statistics buffers are received (either synchronously, or + * possibly in a background thread). The latest computed value of those + * parameters is stored in the active state in the process() function. + * + * Each of the members in the active state belongs to a specific algorithm. A + * member may be read by any algorithm, but shall only be written by its owner. */ /** @@ -185,7 +197,32 @@ namespace libcamera::ipa::rkisp1 { * \struct IPAFrameContext * \brief Per-frame context for algorithms * - * \todo Populate the frame context for all algorithms + * The frame context stores two distinct categories of information: + * + * - The value of the controls to be applied to the frame. These values are + * typically set in the queueRequest() function, from the consolidated + * control values stored in the active state. The frame context thus stores + * values for all controls related to the algorithm, not limited to the + * controls specified in the corresponding request, but consolidated from all + * requests that have been queued so far. + * + * For controls that can be set manually or computed by an algorithm + * (depending on the algorithm operation mode), such as for instance the + * colour gains for the AWB algorithm, the control value will be stored in + * the frame context in the queueRequest() function only when operating in + * manual mode. When operating in auto mode, the values are computed by the + * algorithm in process(), stored in the active state, and copied to the + * frame context in prepare(), just before being stored in the ISP parameters + * buffer. + * + * The queueRequest() function can also store ancillary data in the frame + * context, such as flags to indicate if (and what) control values have + * changed compared to the previous request. + * + * - Status information computed by the algorithm for a frame. For instance, + * the colour temperature estimated by the AWB algorithm from ISP statistics + * calculated on a frame is stored in the frame context for that frame in + * the process() function. */ /** From patchwork Tue Sep 27 02:36:35 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Laurent Pinchart X-Patchwork-Id: 17437 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id BEA69C0DA4 for ; Tue, 27 Sep 2022 02:37:42 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 8074B622E7; Tue, 27 Sep 2022 04:37:42 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1664246262; bh=GcSa94Jrazj/EAi+vuDxoDwbhhojdBEJ8UWXSrd03HA=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To: From; b=cUq8F6Tq3TcsRZPgQWaNDyli6X9ok/TgbqLksOSlEGtHI08cq/SVRBVKufvGqs+vj gPa0q9ILTM/1xPLQbTPur0sTthNGPl6PlVd80stRBykzU8p/s+UvKZXm1gBkeMZDrW TVGhxTNnLYw88KB3dwltToLiBpiXDeKn06NlSjDLiUGRMhburWz1tr0J9H90a3Dpfb 4Wxd127SIYh+5k4u5PG+4TdK6Lpb/ggBmG7lSQL8Ypy3JsyiG+pnlFt4CP3YHKiTfA ZzYSXcfQ4gYhK3BeHoJC1PIql1rZl/XNg51dknZKP4NmabkEDynKSfM/qi1onAEVYO 7Oaq8EnlqDP7Q== Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id B50F76227A for ; Tue, 27 Sep 2022 04:37:40 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="aYn8SXbl"; dkim-atps=neutral Received: from pendragon.ideasonboard.com (62-78-145-57.bb.dnainternet.fi [62.78.145.57]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 38607E5 for ; Tue, 27 Sep 2022 04:37:40 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1664246260; bh=GcSa94Jrazj/EAi+vuDxoDwbhhojdBEJ8UWXSrd03HA=; h=From:To:Subject:Date:In-Reply-To:References:From; b=aYn8SXblmJUy7BsusBKQqnnzfVFnMEbVOPeLCfKNYS/RU17yV2hZTnLLqXXBWOs/D NmL44y9qydpsvRy04P4uyyUOcnYdpfo8G9eq8eq4SJ2qz7KL7xcnXGKynTxIvKlOFL TFAOFlHZoQ+2mDbTM1EVlEWYby7QjgQFIH9X4ILU= To: libcamera-devel@lists.libcamera.org Date: Tue, 27 Sep 2022 05:36:35 +0300 Message-Id: <20220927023642.12341-27-laurent.pinchart@ideasonboard.com> X-Mailer: git-send-email 2.35.1 In-Reply-To: <20220927023642.12341-1-laurent.pinchart@ideasonboard.com> References: <20220927023642.12341-1-laurent.pinchart@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v5 26/33] ipa: rkisp1: awb: Use frame context to fix gains calculations X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Laurent Pinchart via libcamera-devel From: Laurent Pinchart Reply-To: Laurent Pinchart Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" The AWB statistics are computed after the ISP applies the colour gains. This means that the red, green and blue means do not match the data coming directly from the sensor, but are multiplied by the colour gains that were used for the frame on which the statistics have been computed. The AWB algorithm needs to take this into account when calculating the colour gains for the next frame. Do so by dividing the means by the gains that were applied to the frame, retrieved from the frame context. Signed-off-by: Laurent Pinchart Reviewed-by: Kieran Bingham Reviewed-by: Jacopo Mondi --- src/ipa/rkisp1/algorithms/awb.cpp | 9 +++++++++ 1 file changed, 9 insertions(+) diff --git a/src/ipa/rkisp1/algorithms/awb.cpp b/src/ipa/rkisp1/algorithms/awb.cpp index e491cf7507e0..694d97cccc07 100644 --- a/src/ipa/rkisp1/algorithms/awb.cpp +++ b/src/ipa/rkisp1/algorithms/awb.cpp @@ -207,6 +207,15 @@ void Awb::process(IPAContext &context, double greenMean = 1.1636 * yMean - 0.4045 * cbMean - 0.7949 * crMean; double blueMean = 1.1636 * yMean + 1.9912 * cbMean - 0.0250 * crMean; + /* + * The ISP computes the AWB means after applying the colour gains, + * divide by the gains that were used to get the raw means from the + * sensor. + */ + redMean /= frameContext.awb.gains.red; + greenMean /= frameContext.awb.gains.green; + blueMean /= frameContext.awb.gains.blue; + frameContext.awb.temperatureK = estimateCCT(redMean, greenMean, blueMean); /* Estimate the red and blue gains to apply in a grey world. */ From patchwork Tue Sep 27 02:36:36 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Laurent Pinchart X-Patchwork-Id: 17438 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 5C8B4C327E for ; Tue, 27 Sep 2022 02:37:45 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 0AD59622EF; Tue, 27 Sep 2022 04:37:45 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1664246265; bh=VcoWLaP0G21qo8Zj/0ZRKtilZuv0yfvdBPF5HEhjo5k=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To: From; b=WmCwT82JBKUcBEH6hjLG9ywDFpw6+CFR7n5TnUQWGD8IJ5CGem8ngWMj+7P5IY4TA rs3ps1TAazYOX6qN4peUYXkddOny9/88OoN8MpOYd1zm3HXTgCGZGc1lNEeCp87foo cxZXGSkiree3cH04VLrYVv/57brxenx80t0TJQW3PclvDyi9SJO/UPMkZR4ufE0NKI iNlP9sdDVcqZJxkCYFaQiTyvIM1m7vw6/MTzN2n2KkuTJ/a6TQZNRVYuIdaKIdByCs 5COSDoDk7qrF58WZihwI5r0/rJPqCJG7CEpiivon8ENM8rvQAQjMKp4yW1DPv9gR6d Sj3PodMM+e7Sw== Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 4A8CF622EA for ; Tue, 27 Sep 2022 04:37:42 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="HjY2+Yai"; dkim-atps=neutral Received: from pendragon.ideasonboard.com (62-78-145-57.bb.dnainternet.fi [62.78.145.57]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id A6458E5 for ; Tue, 27 Sep 2022 04:37:41 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1664246262; bh=VcoWLaP0G21qo8Zj/0ZRKtilZuv0yfvdBPF5HEhjo5k=; h=From:To:Subject:Date:In-Reply-To:References:From; b=HjY2+YaiM1WoqE88pqnh4oJPxc0DYv9i3Pv/xHftkpxRZTm3IrxHmegEt577+egMl 0/BiAujE9wPGmvo8A/tJMaOxfcLiS9UtxhAuaRtU3XeyLLBFAZymnD1AxK6O0ucLIV XIXuT7WmvyFx23NOkOAZwBfQvhRPwNSmaRos1KTI= To: libcamera-devel@lists.libcamera.org Date: Tue, 27 Sep 2022 05:36:36 +0300 Message-Id: <20220927023642.12341-28-laurent.pinchart@ideasonboard.com> X-Mailer: git-send-email 2.35.1 In-Reply-To: <20220927023642.12341-1-laurent.pinchart@ideasonboard.com> References: <20220927023642.12341-1-laurent.pinchart@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v5 27/33] ipa: rkisp1: awb: Store color temperature as an integer X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Laurent Pinchart via libcamera-devel From: Laurent Pinchart Reply-To: Laurent Pinchart Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" The color temperature doesn't need floating point precision, and is calculated by Awb::estimateCCT() as an unsigned integer. Store it with the same data type in the frame context. Signed-off-by: Laurent Pinchart Reviewed-by: Kieran Bingham Reviewed-by: Jacopo Mondi --- src/ipa/rkisp1/ipa_context.h | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/ipa/rkisp1/ipa_context.h b/src/ipa/rkisp1/ipa_context.h index 9a16dd6cf108..d0692f872960 100644 --- a/src/ipa/rkisp1/ipa_context.h +++ b/src/ipa/rkisp1/ipa_context.h @@ -101,7 +101,7 @@ struct IPAFrameContext : public FrameContext { double blue; } gains; - double temperatureK; + unsigned int temperatureK; bool autoEnabled; } awb; From patchwork Tue Sep 27 02:36:37 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Laurent Pinchart X-Patchwork-Id: 17439 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id DAFDDC0DA4 for ; Tue, 27 Sep 2022 02:37:45 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 99053622ED; Tue, 27 Sep 2022 04:37:45 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1664246265; bh=BRr1yWkgRo95TeUJpCd/hiAsq3WVLt7xHy3IY8/QPJs=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To:Cc: From; b=SpwSAx3WWywpjql11SotVDUQaJoWvAtkE2WO8KCdt/FU0khUw2mboy/nS+c+bg2EB WFSbjnEDUt9S/O20CjMVDukQwUO+bEcY9O86YstB1sIXQPfMLIpCz/0+L6aA6wzncw xfuVTUgmGzvPlmpQq6FUrin7do3PR67aa0Iz9PNhbK3c3QHW1gbM0VfyTFpp1VJSS6 5gy8PzllYSxe+wkhi9L2oHbTU1/v6z8c2WY3yFtYs+orKTdJRHZSpW6glVykE0hjdx U72BG0iohnRjGIMkVgEOuf8f+gvRqHtyI8ibk/h2pSQOH+5ZjRN0j7ruWFJl9s/msk an7Gd0tmTZfaA== Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id AC564622EA for ; Tue, 27 Sep 2022 04:37:43 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="MQ2orzer"; dkim-atps=neutral Received: from pendragon.ideasonboard.com (62-78-145-57.bb.dnainternet.fi [62.78.145.57]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 30444823; Tue, 27 Sep 2022 04:37:43 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1664246263; bh=BRr1yWkgRo95TeUJpCd/hiAsq3WVLt7xHy3IY8/QPJs=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=MQ2orzerlD8VyQizcSGmpFyIz3nyRhLOkrPLJ2EwnqbwF+YEM+98PM23e+h+SClEB FGPTcb2JhDWNgXOCmqQhiC1B37WhwRnoq3uLhgKz4DPJcJAiXHgVu4tZX7SiYbfanD PWd69wkWfiQdmx2Tb7fz0SWe5ENUKwCr909dJlKQ= To: libcamera-devel@lists.libcamera.org Date: Tue, 27 Sep 2022 05:36:37 +0300 Message-Id: <20220927023642.12341-29-laurent.pinchart@ideasonboard.com> X-Mailer: git-send-email 2.35.1 In-Reply-To: <20220927023642.12341-1-laurent.pinchart@ideasonboard.com> References: <20220927023642.12341-1-laurent.pinchart@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v5 28/33] ipa: rkisp1: awb: Add support for RGB means X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Laurent Pinchart via libcamera-devel From: Laurent Pinchart Reply-To: Laurent Pinchart Cc: Quentin Schulz Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" From: Quentin Schulz RkISP actually supports two modes for color means, RGB and YCbCr. The variables where the means are stored are identically named regardless of the color means mode that's been selected. Since the gains are computed in RGB mode, a conversion needs to be done when the mode is YCbCr, which is unnecessary when RGB mode is selected. This adds support for RGB means mode too, by checking at runtime which mode is selected at a given time. The default is still set to YCbCr mode for now. Cc: Quentin Schulz Signed-off-by: Quentin Schulz Signed-off-by: Laurent Pinchart Reviewed-by: Kieran Bingham Reviewed-by: Jacopo Mondi --- Changes since v4: - Fix grammar mistake and spelling error Changes since v1: - Fix compilation error by storing RGB mode flag in Awb class - Set ISP AWB configuration based on the selected mode --- src/ipa/rkisp1/algorithms/awb.cpp | 130 +++++++++++++++++++----------- src/ipa/rkisp1/algorithms/awb.h | 4 +- 2 files changed, 86 insertions(+), 48 deletions(-) diff --git a/src/ipa/rkisp1/algorithms/awb.cpp b/src/ipa/rkisp1/algorithms/awb.cpp index 694d97cccc07..cde5b981df61 100644 --- a/src/ipa/rkisp1/algorithms/awb.cpp +++ b/src/ipa/rkisp1/algorithms/awb.cpp @@ -30,6 +30,11 @@ namespace ipa::rkisp1::algorithms { LOG_DEFINE_CATEGORY(RkISP1Awb) +Awb::Awb() + : rgbMode_(false) +{ +} + /** * \copydoc libcamera::ipa::Algorithm::configure */ @@ -98,38 +103,60 @@ void Awb::prepare(IPAContext &context, const uint32_t frame, /* Update the gains. */ params->module_cfg_update |= RKISP1_CIF_ISP_MODULE_AWB_GAIN; - /* If we already have configured the gains and window, return. */ + /* If we have already set the AWB measurement parameters, return. */ if (frame > 0) return; - /* Configure the gains to apply. */ + rkisp1_cif_isp_awb_meas_config &awb_config = params->meas.awb_meas_config; + + /* Configure the measure window for AWB. */ + awb_config.awb_wnd = context.configuration.awb.measureWindow; + + /* Number of frames to use to estimate the means (0 means 1 frame). */ + awb_config.frames = 0; + + /* Select RGB or YCbCr means measurement. */ + if (rgbMode_) { + awb_config.awb_mode = RKISP1_CIF_ISP_AWB_MODE_RGB; + + /* + * For RGB-based measurements, pixels are selected with maximum + * red, green and blue thresholds that are set in the + * awb_ref_cr, awb_min_y and awb_ref_cb respectively. The other + * values are not used, set them to 0. + */ + awb_config.awb_ref_cr = 250; + awb_config.min_y = 250; + awb_config.awb_ref_cb = 250; + + awb_config.max_y = 0; + awb_config.min_c = 0; + awb_config.max_csum = 0; + } else { + awb_config.awb_mode = RKISP1_CIF_ISP_AWB_MODE_YCBCR; + + /* Set the reference Cr and Cb (AWB target) to white. */ + awb_config.awb_ref_cb = 128; + awb_config.awb_ref_cr = 128; + + /* + * Filter out pixels based on luminance and chrominance values. + * The acceptable luma values are specified as a [16, 250] + * range, while the acceptable chroma values are specified with + * a minimum of 16 and a maximum Cb+Cr sum of 250. + */ + awb_config.min_y = 16; + awb_config.max_y = 250; + awb_config.min_c = 16; + awb_config.max_csum = 250; + } + + /* Enable the AWB gains. */ params->module_en_update |= RKISP1_CIF_ISP_MODULE_AWB_GAIN; - /* Update the ISP to apply the gains configured. */ params->module_ens |= RKISP1_CIF_ISP_MODULE_AWB_GAIN; - /* Configure the measure window for AWB. */ - params->meas.awb_meas_config.awb_wnd = context.configuration.awb.measureWindow; - /* - * Measure Y, Cr and Cb means. - * \todo RGB is not working, the kernel seems to not configure it ? - */ - params->meas.awb_meas_config.awb_mode = RKISP1_CIF_ISP_AWB_MODE_YCBCR; - /* Reference Cr and Cb. */ - params->meas.awb_meas_config.awb_ref_cb = 128; - params->meas.awb_meas_config.awb_ref_cr = 128; - /* Y values to include are between min_y and max_y only. */ - params->meas.awb_meas_config.min_y = 16; - params->meas.awb_meas_config.max_y = 250; - /* Maximum Cr+Cb value to take into account for awb. */ - params->meas.awb_meas_config.max_csum = 250; - /* Minimum Cr and Cb values to take into account. */ - params->meas.awb_meas_config.min_c = 16; - /* Number of frames to use to estimate the mean (0 means 1 frame). */ - params->meas.awb_meas_config.frames = 0; - - /* Update AWB measurement unit configuration. */ + /* Update the AWB measurement parameters and enable the AWB module. */ params->module_cfg_update |= RKISP1_CIF_ISP_MODULE_AWB; - /* Make sure the ISP is measuring the means for the next frame. */ params->module_en_update |= RKISP1_CIF_ISP_MODULE_AWB; params->module_ens |= RKISP1_CIF_ISP_MODULE_AWB; } @@ -182,30 +209,39 @@ void Awb::process(IPAContext &context, const rkisp1_cif_isp_stat *params = &stats->params; const rkisp1_cif_isp_awb_stat *awb = ¶ms->awb; IPAActiveState &activeState = context.activeState; + double greenMean; + double redMean; + double blueMean; - /* Get the YCbCr mean values */ - double yMean = awb->awb_mean[0].mean_y_or_g; - double crMean = awb->awb_mean[0].mean_cr_or_r; - double cbMean = awb->awb_mean[0].mean_cb_or_b; + if (rgbMode_) { + greenMean = awb->awb_mean[0].mean_y_or_g; + redMean = awb->awb_mean[0].mean_cr_or_r; + blueMean = awb->awb_mean[0].mean_cb_or_b; + } else { + /* Get the YCbCr mean values */ + double yMean = awb->awb_mean[0].mean_y_or_g; + double cbMean = awb->awb_mean[0].mean_cb_or_b; + double crMean = awb->awb_mean[0].mean_cr_or_r; - /* - * Convert from YCbCr to RGB. - * The hardware uses the following formulas: - * Y = 16 + 0.2500 R + 0.5000 G + 0.1094 B - * Cb = 128 - 0.1406 R - 0.2969 G + 0.4375 B - * Cr = 128 + 0.4375 R - 0.3750 G - 0.0625 B - * - * The inverse matrix is thus: - * [[1,1636, -0,0623, 1,6008] - * [1,1636, -0,4045, -0,7949] - * [1,1636, 1,9912, -0,0250]] - */ - yMean -= 16; - cbMean -= 128; - crMean -= 128; - double redMean = 1.1636 * yMean - 0.0623 * cbMean + 1.6008 * crMean; - double greenMean = 1.1636 * yMean - 0.4045 * cbMean - 0.7949 * crMean; - double blueMean = 1.1636 * yMean + 1.9912 * cbMean - 0.0250 * crMean; + /* + * Convert from YCbCr to RGB. + * The hardware uses the following formulas: + * Y = 16 + 0.2500 R + 0.5000 G + 0.1094 B + * Cb = 128 - 0.1406 R - 0.2969 G + 0.4375 B + * Cr = 128 + 0.4375 R - 0.3750 G - 0.0625 B + * + * The inverse matrix is thus: + * [[1,1636, -0,0623, 1,6008] + * [1,1636, -0,4045, -0,7949] + * [1,1636, 1,9912, -0,0250]] + */ + yMean -= 16; + cbMean -= 128; + crMean -= 128; + redMean = 1.1636 * yMean - 0.0623 * cbMean + 1.6008 * crMean; + greenMean = 1.1636 * yMean - 0.4045 * cbMean - 0.7949 * crMean; + blueMean = 1.1636 * yMean + 1.9912 * cbMean - 0.0250 * crMean; + } /* * The ISP computes the AWB means after applying the colour gains, diff --git a/src/ipa/rkisp1/algorithms/awb.h b/src/ipa/rkisp1/algorithms/awb.h index 6d4a39467038..d76b538288ec 100644 --- a/src/ipa/rkisp1/algorithms/awb.h +++ b/src/ipa/rkisp1/algorithms/awb.h @@ -16,7 +16,7 @@ namespace ipa::rkisp1::algorithms { class Awb : public Algorithm { public: - Awb() = default; + Awb(); ~Awb() = default; int configure(IPAContext &context, const IPACameraSensorInfo &configInfo) override; @@ -32,6 +32,8 @@ public: private: uint32_t estimateCCT(double red, double green, double blue); + + bool rgbMode_; }; } /* namespace ipa::rkisp1::algorithms */ From patchwork Tue Sep 27 02:36:38 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Laurent Pinchart X-Patchwork-Id: 17440 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 480E1C327E for ; Tue, 27 Sep 2022 02:37:47 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 1089B6236A; Tue, 27 Sep 2022 04:37:47 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1664246267; bh=lBdRG10Dae6rR63qGMuMOGUHyO2yRK3Bsr7qu44b0aY=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To: From; b=p0u+T8ikBfL8STqY1kjfwkw7EZAbgbX+iRtavMQARZ6I5S8Kt4DPhHqTzJMy+bfB4 4VMQ4lKdTpC8b9b5+svkh9QEBAmY+i37vqEkMqipR4KZbzt9aYG2xL3nryYPkmZUee dpeKcJbGrL73okXWSolQmi8IGWt7UXDIclAxxmkJhOub/y5kmPxkurvLcSpUfYvOFa yo7NvT5HwxQYr0VCMjCyyTj+CJpRjV5vyLeqjCzawiuSOIgC9umK9Wu3D8fnUrpa9I B6KG4EvfwxroSAWeqkXVBJ3FtcOeD/pnr0wQxUNdsM/672dn05i0b2wOrpfMV2XQL7 fmeeQ4wpci5Sg== Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 0FD94622F1 for ; Tue, 27 Sep 2022 04:37:45 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="FRVTpTdK"; dkim-atps=neutral Received: from pendragon.ideasonboard.com (62-78-145-57.bb.dnainternet.fi [62.78.145.57]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 89759E5 for ; Tue, 27 Sep 2022 04:37:44 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1664246264; bh=lBdRG10Dae6rR63qGMuMOGUHyO2yRK3Bsr7qu44b0aY=; h=From:To:Subject:Date:In-Reply-To:References:From; b=FRVTpTdKUsxdjz1gNDvM2k0wyuHAA/B4y5b9ElIPY8Bj+Ltyn4u8wHQdzKItBuMt/ LYt17LbcMruXZGOiLtxEUioqzuSlaXJ9Su/g5xnS4th2C5h4bh39l49qJJZZ7Id1Qv cuB0OAg+AbfPf1lLpFFs8WCFH6Udu83GEYKBZM4g= To: libcamera-devel@lists.libcamera.org Date: Tue, 27 Sep 2022 05:36:38 +0300 Message-Id: <20220927023642.12341-30-laurent.pinchart@ideasonboard.com> X-Mailer: git-send-email 2.35.1 In-Reply-To: <20220927023642.12341-1-laurent.pinchart@ideasonboard.com> References: <20220927023642.12341-1-laurent.pinchart@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v5 29/33] ipa: rkisp1: awb: Log means, gains and temperature in debug message X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Laurent Pinchart via libcamera-devel From: Laurent Pinchart Reply-To: Laurent Pinchart Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Extend the debug message in Awb::process() to log the means and color temperature in addition to the gains. This is useful for debugging the algorithm behaviour. While at it, set the showpoint flag to print a fixed number of digits after the decimal point, making logs more readable. Signed-off-by: Laurent Pinchart Reviewed-by: Kieran Bingham Reviewed-by: Jacopo Mondi --- src/ipa/rkisp1/algorithms/awb.cpp | 9 +++++++-- 1 file changed, 7 insertions(+), 2 deletions(-) diff --git a/src/ipa/rkisp1/algorithms/awb.cpp b/src/ipa/rkisp1/algorithms/awb.cpp index cde5b981df61..139c8239504f 100644 --- a/src/ipa/rkisp1/algorithms/awb.cpp +++ b/src/ipa/rkisp1/algorithms/awb.cpp @@ -9,6 +9,7 @@ #include #include +#include #include @@ -271,8 +272,12 @@ void Awb::process(IPAContext &context, activeState.awb.gains.automatic.blue = std::clamp(blueGain, 0.0, 1023.0 / 256); activeState.awb.gains.automatic.green = 1.0; - LOG(RkISP1Awb, Debug) << "Gain found for red: " << activeState.awb.gains.automatic.red - << " and for blue: " << activeState.awb.gains.automatic.blue; + LOG(RkISP1Awb, Debug) << std::showpoint + << "Means [" << redMean << ", " << greenMean << ", " << blueMean + << "], gains [" << activeState.awb.gains.automatic.red << ", " + << activeState.awb.gains.automatic.green << ", " + << activeState.awb.gains.automatic.blue << "], temp " + << frameContext.awb.temperatureK << "K"; } REGISTER_IPA_ALGORITHM(Awb, "Awb") From patchwork Tue Sep 27 02:36:39 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Laurent Pinchart X-Patchwork-Id: 17441 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id E23B7C0DA4 for ; Tue, 27 Sep 2022 02:37:49 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id A830862379; Tue, 27 Sep 2022 04:37:49 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1664246269; bh=MVOLInGX5yVAIA9XLQDFWLFjXKy+lGpTeefmyExcEhw=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To: From; b=cJKcLu0lJsipo2kA5CF373Fk7Had8BVGO4ztXo0nbcZgR+wN9op4myiNLd6chn6qy KAV5yTcYvqj94DRKlQyVzlur6iuhVVKXNJt5xjqxw9lfe6kh8iDxoXQBNH2j6725Fg eyUqXgZTh/6P9dFXp1I8NfHHQWA/4JFCRMo/6m1D++ZiZhMHNB4YCJlOSzPj7WVIZ9 uOOQ+4p6um4Uj9pQ/6eXh0DpFtba40Zjlp1vQUrsHG5QjSVgfZRZO/+Mg5NVfOQfnI ERRUM7b0AV2t+PvDb/PwBYBpw93/MdxzzYPVuozpiKRThtWFWjxC6UlEfhaiU9EVvD jfgtSRHjvTjaw== Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 99768622EE for ; Tue, 27 Sep 2022 04:37:46 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="l9yeaTN2"; dkim-atps=neutral Received: from pendragon.ideasonboard.com (62-78-145-57.bb.dnainternet.fi [62.78.145.57]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 1DFC24F7 for ; Tue, 27 Sep 2022 04:37:46 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1664246266; bh=MVOLInGX5yVAIA9XLQDFWLFjXKy+lGpTeefmyExcEhw=; h=From:To:Subject:Date:In-Reply-To:References:From; b=l9yeaTN2xVfcGM/cxv+qjmuCEceMMHoffo2LoQLPeX/CeQgpwQ7r2r2521WKUADXa QwU2elsBw+PXFKag2S95yC2UuQ6nPNnlIIRAFqCiI1ShrlciYVge4ug3eUM0aLM1gf aohVbK+GRJ7L8wC7rvO4388wtm6kSeCKZhe7cnhk= To: libcamera-devel@lists.libcamera.org Date: Tue, 27 Sep 2022 05:36:39 +0300 Message-Id: <20220927023642.12341-31-laurent.pinchart@ideasonboard.com> X-Mailer: git-send-email 2.35.1 In-Reply-To: <20220927023642.12341-1-laurent.pinchart@ideasonboard.com> References: <20220927023642.12341-1-laurent.pinchart@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v5 30/33] ipa: rkisp1: awb: Prevent RGB means from being negative X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Laurent Pinchart via libcamera-devel From: Laurent Pinchart Reply-To: Laurent Pinchart Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Due to hardware rounding errors in the YCbCr means, the calculated RGB means may be negative. This would lead to negative gains, messing up calculation. Prevent this by clamping the means to positive values. Signed-off-by: Laurent Pinchart Reviewed-by: Kieran Bingham Reviewed-by: Jacopo Mondi --- src/ipa/rkisp1/algorithms/awb.cpp | 10 ++++++++++ 1 file changed, 10 insertions(+) diff --git a/src/ipa/rkisp1/algorithms/awb.cpp b/src/ipa/rkisp1/algorithms/awb.cpp index 139c8239504f..59664d09b84c 100644 --- a/src/ipa/rkisp1/algorithms/awb.cpp +++ b/src/ipa/rkisp1/algorithms/awb.cpp @@ -242,6 +242,16 @@ void Awb::process(IPAContext &context, redMean = 1.1636 * yMean - 0.0623 * cbMean + 1.6008 * crMean; greenMean = 1.1636 * yMean - 0.4045 * cbMean - 0.7949 * crMean; blueMean = 1.1636 * yMean + 1.9912 * cbMean - 0.0250 * crMean; + + /* + * Due to hardware rounding errors in the YCbCr means, the + * calculated RGB means may be negative. This would lead to + * negative gains, messing up calculation. Prevent this by + * clamping the means to positive values. + */ + redMean = std::max(redMean, 0.0); + greenMean = std::max(greenMean, 0.0); + blueMean = std::max(blueMean, 0.0); } /* From patchwork Tue Sep 27 02:36:40 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Laurent Pinchart X-Patchwork-Id: 17442 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 57BEAC327E for ; Tue, 27 Sep 2022 02:37:50 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 1A17C62378; Tue, 27 Sep 2022 04:37:50 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1664246270; bh=fRiz4PGXGBvt4bzwxnX/+k7/Gm/HEywmgBG7nxHurcQ=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To: From; b=XQoslnuQd53mmVMQo3z2wWu8Pyh5AVhcMj2nZT46Vf43PJh8XSpKIfzSYHcRQ5/2g CSm7XSsWKjomozdxLRjywOjgbOEV/VO6Ao3qjojcj0GzKV0pDslUOyq0wFzTyAmwGK 6vYzHMxquSx0dCA+NZPRSzWilEwqEExGE9zqA7F6Xqyl0Taz/0kuoFZ7Eob4OP+7EL bY/KoJ7f/SNZlgW+/V8dtOvzh+q2Lp+WKMQiVRO9i+Bzu9rg7p7RQsR/qvwLWAM1Xh IOdDpu7EN7VQh1H8URVtT40md4ZKPfpE6VsXXnDDuufIgjyYFfBjxCJOugtc1x+JQc zL9MdHfDUsPzg== Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 0D7CE62295 for ; Tue, 27 Sep 2022 04:37:48 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="G+jS83vt"; dkim-atps=neutral Received: from pendragon.ideasonboard.com (62-78-145-57.bb.dnainternet.fi [62.78.145.57]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 85A42E5 for ; Tue, 27 Sep 2022 04:37:47 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1664246267; bh=fRiz4PGXGBvt4bzwxnX/+k7/Gm/HEywmgBG7nxHurcQ=; h=From:To:Subject:Date:In-Reply-To:References:From; b=G+jS83vt+79paWYUyP4UxddnXA+P6yDBONwajEyPM0/GBbYHGBzKWcF5/OOESkNUX lWEzb1hNgLsPZJyEgOMYAUWRQIVxNCB5SQbmzuJnkGy4X6vqW3fQ3JbEiLYtiaMbqO 8Xh/UZHWysFBr1jJqalBo6qX4fR3r5GxE4/AQ3f0= To: libcamera-devel@lists.libcamera.org Date: Tue, 27 Sep 2022 05:36:40 +0300 Message-Id: <20220927023642.12341-32-laurent.pinchart@ideasonboard.com> X-Mailer: git-send-email 2.35.1 In-Reply-To: <20220927023642.12341-1-laurent.pinchart@ideasonboard.com> References: <20220927023642.12341-1-laurent.pinchart@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v5 31/33] ipa: rkisp1: awb: Clamp gains to prevent divisions by zero X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Laurent Pinchart via libcamera-devel From: Laurent Pinchart Reply-To: Laurent Pinchart Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" The gain values are currently clamped to the range [0.0, 3.996] used by the hardware. A zero value makes little sense, as it would completely remove the contribution of the corresponding color channel from the AWB accumulators, but worse, would lead to divisions by zero when calculating the raw means in subsequent iterations. Prevent this by setting the minimum gain value to 1/256. While at it, clamp the gain values before filtering them, to improve the stability of the control loop. Signed-off-by: Laurent Pinchart Reviewed-by: Kieran Bingham Reviewed-by: Jacopo Mondi --- Changes since v4: - Fix typo in comment --- src/ipa/rkisp1/algorithms/awb.cpp | 22 +++++++++++++++------- 1 file changed, 15 insertions(+), 7 deletions(-) diff --git a/src/ipa/rkisp1/algorithms/awb.cpp b/src/ipa/rkisp1/algorithms/awb.cpp index 59664d09b84c..a3066fbb1994 100644 --- a/src/ipa/rkisp1/algorithms/awb.cpp +++ b/src/ipa/rkisp1/algorithms/awb.cpp @@ -265,21 +265,29 @@ void Awb::process(IPAContext &context, frameContext.awb.temperatureK = estimateCCT(redMean, greenMean, blueMean); - /* Estimate the red and blue gains to apply in a grey world. */ + /* + * Estimate the red and blue gains to apply in a grey world. The green + * gain is hardcoded to 1.0. + */ double redGain = greenMean / (redMean + 1); double blueGain = greenMean / (blueMean + 1); + /* + * Clamp the gain values to the hardware, which expresses gains as Q2.8 + * unsigned integer values. Set the minimum just above zero to avoid + * divisions by zero when computing the raw means in subsequent + * iterations. + */ + redGain = std::clamp(redGain, 1.0 / 256, 1023.0 / 256); + blueGain = std::clamp(blueGain, 1.0 / 256, 1023.0 / 256); + /* Filter the values to avoid oscillations. */ double speed = 0.2; redGain = speed * redGain + (1 - speed) * activeState.awb.gains.automatic.red; blueGain = speed * blueGain + (1 - speed) * activeState.awb.gains.automatic.blue; - /* - * Gain values are unsigned integer value, range 0 to 4 with 8 bit - * fractional part. Hardcode the green gain to 1.0. - */ - activeState.awb.gains.automatic.red = std::clamp(redGain, 0.0, 1023.0 / 256); - activeState.awb.gains.automatic.blue = std::clamp(blueGain, 0.0, 1023.0 / 256); + activeState.awb.gains.automatic.red = redGain; + activeState.awb.gains.automatic.blue = blueGain; activeState.awb.gains.automatic.green = 1.0; LOG(RkISP1Awb, Debug) << std::showpoint From patchwork Tue Sep 27 02:36:41 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Laurent Pinchart X-Patchwork-Id: 17443 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id D572CC0DA4 for ; Tue, 27 Sep 2022 02:37:51 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 9AD0E62369; Tue, 27 Sep 2022 04:37:51 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1664246271; bh=3B+zhvvDYOWFaJ5PLdGBMmgUzAOCaswyHUmOwZPzXX0=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To: From; b=U4ySHf8TKPiZxVPaXY3Bwj5hdFBv3VW+owgCgcQl7EpPPYXtg9bzn2GooOo9WH2uV gncH4MnYFwaX/bA0EhVkII+4RPjZ6xIH8RWbC867ABQhfLwFhCQsbOrTqm3FETSKs9 moxElfxYHcX9LSchhU8M5UQtduNiPUv2ZdnCY5XFf+SQTYfu7HwOrt/m1erxudBGKw IL0dZkj4XSmbsX8OfGAr0C15x8+PckQuGMtBuC+WhUhSUwtRF6AXNSJsXtg9ob5k4m lhLC0wmnVbvTlLDEdOswfaBPQLxIUDW2/5fUFPkd/p5oQ2JinEWUtY5VMnRMDTq/AJ Myz2R6kTMD2gw== Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 92760622EE for ; Tue, 27 Sep 2022 04:37:49 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="M1hqnLsE"; dkim-atps=neutral Received: from pendragon.ideasonboard.com (62-78-145-57.bb.dnainternet.fi [62.78.145.57]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 097D0E5 for ; Tue, 27 Sep 2022 04:37:48 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1664246269; bh=3B+zhvvDYOWFaJ5PLdGBMmgUzAOCaswyHUmOwZPzXX0=; h=From:To:Subject:Date:In-Reply-To:References:From; b=M1hqnLsEWlJFl06t8xOi1rQciiQ3UMZGsDQiDvGGvwDMfN1wcHfi7YrhPiQ9SB2oe Dawgadp4nwJr0mZtU0subUkpfHZnkTv61orfRtgGZF9CgRQlZxt3T7KbsCXx92ejdM nNmAdmoGXYmuvcQmCNCShCw/deWFdf0eArwJeyIY= To: libcamera-devel@lists.libcamera.org Date: Tue, 27 Sep 2022 05:36:41 +0300 Message-Id: <20220927023642.12341-33-laurent.pinchart@ideasonboard.com> X-Mailer: git-send-email 2.35.1 In-Reply-To: <20220927023642.12341-1-laurent.pinchart@ideasonboard.com> References: <20220927023642.12341-1-laurent.pinchart@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v5 32/33] ipa: rkisp1: awb: Freeze AWB when means are too small X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Laurent Pinchart via libcamera-devel From: Laurent Pinchart Reply-To: Laurent Pinchart Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" When the RGB means are too small, gains and color temperature can't be meaningfully calculated. Freeze the AWB in that case, using the previously calculated values. Signed-off-by: Laurent Pinchart Reviewed-by: Kieran Bingham Reviewed-by: Jacopo Mondi --- src/ipa/rkisp1/algorithms/awb.cpp | 17 ++++++++++++++++- src/ipa/rkisp1/ipa_context.cpp | 3 +++ src/ipa/rkisp1/ipa_context.h | 1 + 3 files changed, 20 insertions(+), 1 deletion(-) diff --git a/src/ipa/rkisp1/algorithms/awb.cpp b/src/ipa/rkisp1/algorithms/awb.cpp index a3066fbb1994..eb32cd722071 100644 --- a/src/ipa/rkisp1/algorithms/awb.cpp +++ b/src/ipa/rkisp1/algorithms/awb.cpp @@ -31,6 +31,9 @@ namespace ipa::rkisp1::algorithms { LOG_DEFINE_CATEGORY(RkISP1Awb) +/* Minimum mean value below which AWB can't operate. */ +constexpr double kMeanMinThreshold = 2.0; + Awb::Awb() : rgbMode_(false) { @@ -263,7 +266,17 @@ void Awb::process(IPAContext &context, greenMean /= frameContext.awb.gains.green; blueMean /= frameContext.awb.gains.blue; - frameContext.awb.temperatureK = estimateCCT(redMean, greenMean, blueMean); + /* + * If the means are too small we don't have enough information to + * meaningfully calculate gains. Freeze the algorithm in that case. + */ + if (redMean < kMeanMinThreshold && greenMean < kMeanMinThreshold && + blueMean < kMeanMinThreshold) { + frameContext.awb.temperatureK = activeState.awb.temperatureK; + return; + } + + activeState.awb.temperatureK = estimateCCT(redMean, greenMean, blueMean); /* * Estimate the red and blue gains to apply in a grey world. The green @@ -290,6 +303,8 @@ void Awb::process(IPAContext &context, activeState.awb.gains.automatic.blue = blueGain; activeState.awb.gains.automatic.green = 1.0; + frameContext.awb.temperatureK = activeState.awb.temperatureK; + LOG(RkISP1Awb, Debug) << std::showpoint << "Means [" << redMean << ", " << greenMean << ", " << blueMean << "], gains [" << activeState.awb.gains.automatic.red << ", " diff --git a/src/ipa/rkisp1/ipa_context.cpp b/src/ipa/rkisp1/ipa_context.cpp index 2126ed1d0acb..b00dc29c1713 100644 --- a/src/ipa/rkisp1/ipa_context.cpp +++ b/src/ipa/rkisp1/ipa_context.cpp @@ -156,6 +156,9 @@ namespace libcamera::ipa::rkisp1 { * \var IPAActiveState::awb.gains.automatic.blue * \brief Automatic white balance gain for B channel * + * \var IPAActiveState::awb.temperatureK + * \brief Estimated color temperature + * * \var IPAActiveState::awb.autoEnabled * \brief Whether the Auto White Balance algorithm is enabled */ diff --git a/src/ipa/rkisp1/ipa_context.h b/src/ipa/rkisp1/ipa_context.h index d0692f872960..0677c8c35d73 100644 --- a/src/ipa/rkisp1/ipa_context.h +++ b/src/ipa/rkisp1/ipa_context.h @@ -69,6 +69,7 @@ struct IPAActiveState { } automatic; } gains; + unsigned int temperatureK; bool autoEnabled; } awb; From patchwork Tue Sep 27 02:36:42 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Laurent Pinchart X-Patchwork-Id: 17444 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 46C3BC0DA4 for ; Tue, 27 Sep 2022 02:37:54 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 12A5362383; Tue, 27 Sep 2022 04:37:54 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1664246274; bh=cYsCTrk5122uifEVEu5V4F9/P4MluZ7OC+U6xLpN+uI=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To: From; b=z2P+FFyWiDso1Zpib7J/T9JY9f9XYuPkLUD0enXAdd0vca46meBPe3OAYx2phK2In i4DwPfZ4fz7GS7mA4FbHISZSRmy6zKCTLg8BpARkeGZ14/8sdTZNO5ctm0l54kWt6/ 0F+o4EMjMu86VVGSVFqE3pDy6+4TF3nScRusQY754FPK8FlVSOQzCc0olXoe9PdC96 lec4dBOWPqNeKL/qOAGGXcJ1CMOgmqVO5mOJ3We6FSynIBB0Z+FiBFdHACGNPdF/8m kwRClwUlDbUg4U5Hm6Cez7ghmb8GwNBtPX44LnvnllQa7AV15WdnvfEYe93cDe0b2i C1uMXRx8IEZOA== Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 18AE362369 for ; Tue, 27 Sep 2022 04:37:51 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="cubKMqwS"; dkim-atps=neutral Received: from pendragon.ideasonboard.com (62-78-145-57.bb.dnainternet.fi [62.78.145.57]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 88EA647C for ; Tue, 27 Sep 2022 04:37:50 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1664246270; bh=cYsCTrk5122uifEVEu5V4F9/P4MluZ7OC+U6xLpN+uI=; h=From:To:Subject:Date:In-Reply-To:References:From; b=cubKMqwSV5fL98iiRR+XOsxi2zOe+EM5vebfPEhEA/o/2jBiQJsPGjDhRIwCLPKGY AYCXAJjMMDr6X2PaZ4F2IUpuyDP2My1Itoyb1/gqwsOjMZQdmrDuRmuNsA0OykEzGr z82hamIy1r9ilBuAJDkAA9o9e0ge7kTCclHUEMlc= To: libcamera-devel@lists.libcamera.org Date: Tue, 27 Sep 2022 05:36:42 +0300 Message-Id: <20220927023642.12341-34-laurent.pinchart@ideasonboard.com> X-Mailer: git-send-email 2.35.1 In-Reply-To: <20220927023642.12341-1-laurent.pinchart@ideasonboard.com> References: <20220927023642.12341-1-laurent.pinchart@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v5 33/33] ipa: rkisp1: awb: Remove bias from gain calculation X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Laurent Pinchart via libcamera-devel From: Laurent Pinchart Reply-To: Laurent Pinchart Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" The red and blue gains are computed by dividing the green mean by the red and blue means respectively. An offset of 1 is added to the dividers to avoid divisions by zero. This introduces a bias in the gain values. Fix it by clamping the divisors to a minimum of 1.0 instead of adding an offset. Signed-off-by: Laurent Pinchart Reviewed-by: Kieran Bingham Reviewed-by: Jacopo Mondi --- src/ipa/rkisp1/algorithms/awb.cpp | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-) diff --git a/src/ipa/rkisp1/algorithms/awb.cpp b/src/ipa/rkisp1/algorithms/awb.cpp index eb32cd722071..3349948a3acf 100644 --- a/src/ipa/rkisp1/algorithms/awb.cpp +++ b/src/ipa/rkisp1/algorithms/awb.cpp @@ -280,10 +280,11 @@ void Awb::process(IPAContext &context, /* * Estimate the red and blue gains to apply in a grey world. The green - * gain is hardcoded to 1.0. + * gain is hardcoded to 1.0. Avoid divisions by zero by clamping the + * divisor to a minimum value of 1.0. */ - double redGain = greenMean / (redMean + 1); - double blueGain = greenMean / (blueMean + 1); + double redGain = greenMean / std::max(redMean, 1.0); + double blueGain = greenMean / std::max(blueMean, 1.0); /* * Clamp the gain values to the hardware, which expresses gains as Q2.8