From patchwork Thu Sep 8 01:41:29 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Laurent Pinchart X-Patchwork-Id: 17310 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 7C170C327D for ; Thu, 8 Sep 2022 01:42:21 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 33C4062090; Thu, 8 Sep 2022 03:42:21 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1662601341; bh=1H7Yhw29Av1nCbAcY+MzMe9hiOH0+OQVs1mKKAQ0hbk=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To: From; b=ytK/Hy4/mppRarhMVeYH6foi6vM58E2g6wrqARZabmw2oANSeU7IgnqBiOsRdFWTQ SjmIbea8ih1nh+wQ+fKeyOEql7pm6RsyhTirF28YSDZl0mzBsTR2sfLb94VwfxhIHs PuSboK0lYX68hyd1nOIfYw06+o5rDNB64508a7yuJnu+9mMdubCCu5EdIs+Db7sdxn 9RVZ2rHdjFnAU/ZQHMEKS/bPA3FsVAp+Jhc+6vV6BDUN9/n1VxgThZIshZw2KY76Uf 776XEzSTczy9+bp+zCJg/FeiMiayxz8F3LWkDR+EZN7/HlvyPL62IjDeuFhw3MUssV ij6HdUH+CaSbg== Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 216706208F for ; Thu, 8 Sep 2022 03:42:19 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="luTfHtcY"; dkim-atps=neutral Received: from pendragon.ideasonboard.com (62-78-145-57.bb.dnainternet.fi [62.78.145.57]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id A1CE86CC for ; Thu, 8 Sep 2022 03:42:18 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1662601338; bh=1H7Yhw29Av1nCbAcY+MzMe9hiOH0+OQVs1mKKAQ0hbk=; h=From:To:Subject:Date:In-Reply-To:References:From; b=luTfHtcYBmnYlUTd3cqxNOio2GcduMAb/ecUEeq28L2LnRiLSmi5p7cpWITdudhMu G0u4N9NfSWcxurhn6NlHqFgrahSshWVEZbM8mj3Vxke69sJizhhcaLNG66lIgvckzk 60+Ad5nWXdUYu9i4DpVEeyEm+5Utxf/TOcTWLVU8= To: libcamera-devel@lists.libcamera.org Date: Thu, 8 Sep 2022 04:41:29 +0300 Message-Id: <20220908014200.28728-2-laurent.pinchart@ideasonboard.com> X-Mailer: git-send-email 2.35.1 In-Reply-To: <20220908014200.28728-1-laurent.pinchart@ideasonboard.com> References: <20220908014200.28728-1-laurent.pinchart@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v4 01/32] ipa: ipu3: Fix style of Doxygen comment blocks X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Laurent Pinchart via libcamera-devel From: Laurent Pinchart Reply-To: Laurent Pinchart Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Fix various issues in Doxygen comment blocks: - \param requires an [in] or [out] tag - \param must come before the body of the documetation - Rename coarseSearchStep to kCoarseSearchStep - White space and line wrap Signed-off-by: Laurent Pinchart Reviewed-by: Kieran Bingham Reviewed-by: Jacopo Mondi --- src/ipa/ipu3/algorithms/af.cpp | 23 ++++++++++------------- src/ipa/ipu3/algorithms/blc.cpp | 4 ++-- src/ipa/ipu3/algorithms/tone_mapping.cpp | 10 +++++----- 3 files changed, 17 insertions(+), 20 deletions(-) diff --git a/src/ipa/ipu3/algorithms/af.cpp b/src/ipa/ipu3/algorithms/af.cpp index 4835a0345931..106a7614e2c7 100644 --- a/src/ipa/ipu3/algorithms/af.cpp +++ b/src/ipa/ipu3/algorithms/af.cpp @@ -196,10 +196,10 @@ int Af::configure(IPAContext &context, const IPAConfigInfo &configInfo) /** * \brief AF coarse scan - * - * Find a near focused image using a coarse step. The step is determined by coarseSearchStep. - * * \param[in] context The shared IPA context + * + * Find a near focused image using a coarse step. The step is determined by + * kCoarseSearchStep. */ void Af::afCoarseScan(IPAContext &context) { @@ -223,10 +223,9 @@ void Af::afCoarseScan(IPAContext &context) /** * \brief AF fine scan + * \param[in] context The shared IPA context * * Find an optimum lens position with moving 1 step for each search. - * - * \param[in] context The shared IPA context */ void Af::afFineScan(IPAContext &context) { @@ -244,10 +243,9 @@ void Af::afFineScan(IPAContext &context) /** * \brief AF reset + * \param[in] context The shared IPA context * * Reset all the parameters to start over the AF process. - * - * \param[in] context The shared IPA context */ void Af::afReset(IPAContext &context) { @@ -268,7 +266,7 @@ void Af::afReset(IPAContext &context) /** * \brief AF variance comparison. * \param[in] context The IPA context - * \param min_step The VCM movement step. + * \param[in] min_step The VCM movement step. * * We always pick the largest variance to replace the previous one. The image * with a larger variance also indicates it is a clearer image than previous @@ -343,9 +341,8 @@ void Af::afIgnoreFrameReset() /** * \brief Estimate variance - * \param y_item The AF filter data set from the IPU3 statistics buffer - * \param len The quantity of table item entries which are valid to process - * \param isY1 Selects between filter Y1 or Y2 to calculate the variance + * \param[in] y_items The AF filter data set from the IPU3 statistics buffer + * \param[in] isY1 Selects between filter Y1 or Y2 to calculate the variance * * Calculate the mean of the data set provided by \a y_item, and then calculate * the variance of that data set from the mean. @@ -378,13 +375,13 @@ double Af::afEstimateVariance(Span y_items, bool isY1) /** * \brief Determine out-of-focus situation. - * \param context The IPA context. + * \param[in] context The IPA context. * * Out-of-focus means that the variance change rate for a focused and a new * variance is greater than a threshold. * * \return True if the variance threshold is crossed indicating lost focus, - * false otherwise. + * false otherwise */ bool Af::afIsOutOfFocus(IPAContext context) { diff --git a/src/ipa/ipu3/algorithms/blc.cpp b/src/ipa/ipu3/algorithms/blc.cpp index c561aa858b96..c32427c99549 100644 --- a/src/ipa/ipu3/algorithms/blc.cpp +++ b/src/ipa/ipu3/algorithms/blc.cpp @@ -38,8 +38,8 @@ BlackLevelCorrection::BlackLevelCorrection() /** * \brief Fill in the parameter structure, and enable black level correction - * \param context The shared IPA context - * \param params The IPU3 parameters + * \param[in] context The shared IPA context + * \param[out] params The IPU3 parameters * * Populate the IPU3 parameter structure with the correction values for each * channel and enable the corresponding ImgU block processing. diff --git a/src/ipa/ipu3/algorithms/tone_mapping.cpp b/src/ipa/ipu3/algorithms/tone_mapping.cpp index 49a5558b6faa..c21647e8c51b 100644 --- a/src/ipa/ipu3/algorithms/tone_mapping.cpp +++ b/src/ipa/ipu3/algorithms/tone_mapping.cpp @@ -49,8 +49,8 @@ int ToneMapping::configure(IPAContext &context, /** * \brief Fill in the parameter structure, and enable gamma control - * \param context The shared IPA context - * \param params The IPU3 parameters + * \param[in] context The shared IPA context + * \param[out] params The IPU3 parameters * * Populate the IPU3 parameter structure with our tone mapping look up table and * enable the gamma control module in the processing blocks. @@ -71,9 +71,9 @@ void ToneMapping::prepare([[maybe_unused]] IPAContext &context, /** * \brief Calculate the tone mapping look up table - * \param context The shared IPA context - * \param frameContext The current frame context - * \param stats The IPU3 statistics and ISP results + * \param[in] context The shared IPA context + * \param[in] frameContext The current frame context + * \param[in] stats The IPU3 statistics and ISP results * * The tone mapping look up table is generated as an inverse power curve from * our gamma setting. From patchwork Thu Sep 8 01:41:30 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Laurent Pinchart X-Patchwork-Id: 17311 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 1043BC3272 for ; Thu, 8 Sep 2022 01:42:23 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id B37376209B; Thu, 8 Sep 2022 03:42:22 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1662601342; bh=E9b69VVLHcujLlP+GqAxNm3BvaWK50WgqW9V8xE4yIg=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To: From; b=WswsMMv0vE+wkvbit9BGOtj+fMQsT3ov8wcv/HbHPioUu826iqoqiA/kh/p2Ub4ix TU9xLJevKpv7vf8k/v7Bj1+9wO6FZsrEjmlRXQIln96d1ftEKXH7dab2f/VEbd9k2v TSaJkuO/qrrpFUFC/BXblJtoax5WQT8hRQhfWsNViruZJX39lBHxrphtcOBBC4IRu2 j4eUJqkrOPEmIEC7aGIkT9ZVZaVD2dpA072WVjcgQ6w37X+9P0Uyu/CUGspdcYh36B 8n23kZ7YVQk/ZFmCuUI6cbyRpSo7NWZPyAEqU90QzsYt4BiJQYMUptJK0m2c/+rPdd dC9HRij7kqXdA== Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 91A8B6208F for ; Thu, 8 Sep 2022 03:42:20 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="mJTKMlLI"; dkim-atps=neutral Received: from pendragon.ideasonboard.com (62-78-145-57.bb.dnainternet.fi [62.78.145.57]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 1A2736CC for ; Thu, 8 Sep 2022 03:42:20 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1662601340; bh=E9b69VVLHcujLlP+GqAxNm3BvaWK50WgqW9V8xE4yIg=; h=From:To:Subject:Date:In-Reply-To:References:From; b=mJTKMlLIH4HCu3n3zcj/lluGQTuhJv2kWkU+CO0i5mlNNpJDW5+7c8ol+UyvwcfGo fNAT+Wo1WPbrLW+Mg37Gjt8aToMeOfHZB6olf6zxB8bfIatFeyRcBKtXC2BGlBi2X6 K6cQbJz4Vg7CtLbj30gFvQvJPT8pWmrXJBec9iXs= To: libcamera-devel@lists.libcamera.org Date: Thu, 8 Sep 2022 04:41:30 +0300 Message-Id: <20220908014200.28728-3-laurent.pinchart@ideasonboard.com> X-Mailer: git-send-email 2.35.1 In-Reply-To: <20220908014200.28728-1-laurent.pinchart@ideasonboard.com> References: <20220908014200.28728-1-laurent.pinchart@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v4 02/32] ipa: ipu3: af: Pass context reference to afIsOutOfFocus() X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Laurent Pinchart via libcamera-devel From: Laurent Pinchart Reply-To: Laurent Pinchart Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Avoid copying the whole IPA context by passing a reference to the Af::afIsOutOfFocus() function. Signed-off-by: Laurent Pinchart Reviewed-by: Kieran Bingham Reviewed-by: Jacopo Mondi --- src/ipa/ipu3/algorithms/af.cpp | 2 +- src/ipa/ipu3/algorithms/af.h | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/src/ipa/ipu3/algorithms/af.cpp b/src/ipa/ipu3/algorithms/af.cpp index 106a7614e2c7..9127c24f1287 100644 --- a/src/ipa/ipu3/algorithms/af.cpp +++ b/src/ipa/ipu3/algorithms/af.cpp @@ -383,7 +383,7 @@ double Af::afEstimateVariance(Span y_items, bool isY1) * \return True if the variance threshold is crossed indicating lost focus, * false otherwise */ -bool Af::afIsOutOfFocus(IPAContext context) +bool Af::afIsOutOfFocus(IPAContext &context) { const uint32_t diff_var = std::abs(currentVariance_ - context.activeState.af.maxVariance); diff --git a/src/ipa/ipu3/algorithms/af.h b/src/ipa/ipu3/algorithms/af.h index ccf015f3f8f5..9b93594898bb 100644 --- a/src/ipa/ipu3/algorithms/af.h +++ b/src/ipa/ipu3/algorithms/af.h @@ -44,7 +44,7 @@ private: void afIgnoreFrameReset(); double afEstimateVariance(Span y_items, bool isY1); - bool afIsOutOfFocus(IPAContext context); + bool afIsOutOfFocus(IPAContext &context); /* VCM step configuration. It is the current setting of the VCM step. */ uint32_t focus_; From patchwork Thu Sep 8 01:41:31 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Laurent Pinchart X-Patchwork-Id: 17312 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id F1530C327D for ; Thu, 8 Sep 2022 01:42:23 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 7F2D9620A0; Thu, 8 Sep 2022 03:42:23 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1662601343; bh=KJu6IsdzV26VsRhbfPSj7BnobvZix4ays/KgwnSkByg=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To: From; b=e69CZ7H6/XVd79pscVLlgT3P7+kkEQOVaUheM4P3znZneNTJVFVhgHgSe9uFYgtpA jovYIG7B9rwsHcuU+MzQpaz+3Y0YSfYsYXDpicau8mBchZbhDY8KwmoOLwtPum/JRm qsQyuJsEZlelyOxAoGRNFX0QBHeDEzZoTeOkc7q8WFXFR21h2/6HD5ZCBJrxwqV+h+ Vg3NTr22Wmm04nahGrT/zKTPGJ94pEIe8qYJ/RptkX8hcv/87n1vORGHh967GBwi28 1CSkHl6d9BEljQu+kFpeogD0styLgOFI2zJjGrkwMgQ91DjDDB8H+a9h2AEbvVA3MF g+5rPTiXZN/rw== Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id E630962096 for ; Thu, 8 Sep 2022 03:42:21 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="bUtrq0y/"; dkim-atps=neutral Received: from pendragon.ideasonboard.com (62-78-145-57.bb.dnainternet.fi [62.78.145.57]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 7177C6CC for ; Thu, 8 Sep 2022 03:42:21 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1662601341; bh=KJu6IsdzV26VsRhbfPSj7BnobvZix4ays/KgwnSkByg=; h=From:To:Subject:Date:In-Reply-To:References:From; b=bUtrq0y/SVxj2x4ZJpwkbdAmiFk8zlXlikGydoCCtQ4y7fD7fWA5y8eJ+wDykPU4Z bp7l9dJEGHJTaZr1b0mbdyq8eUrz4Fr9HN5l2n9ty0T4e29QMvbSHS5IzlJCVCiNwh +17Y94ZP9G/DUvL0mZryrNu68DssPy0dR4LUso28= To: libcamera-devel@lists.libcamera.org Date: Thu, 8 Sep 2022 04:41:31 +0300 Message-Id: <20220908014200.28728-4-laurent.pinchart@ideasonboard.com> X-Mailer: git-send-email 2.35.1 In-Reply-To: <20220908014200.28728-1-laurent.pinchart@ideasonboard.com> References: <20220908014200.28728-1-laurent.pinchart@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v4 03/32] ipa: libipa: Pass a reference instead of pointer to Algorithm::process() X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Laurent Pinchart via libcamera-devel From: Laurent Pinchart Reply-To: Laurent Pinchart Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Frame contexts will become the core component of IPA modules, always available to functions of the algorithms. To indicate and prepare for this, turn the frame context pointer passed to Algorithm::process() into a reference. The RkISP1 IPA module doesn't use frame contexts yet, so pass a dummy context for now. While at it, drop an unneeded [[maybe_unused]] from Agc::process(). Signed-off-by: Laurent Pinchart Reviewed-by: Kieran Bingham Reviewed-by: Jacopo Mondi --- src/ipa/ipu3/algorithms/af.cpp | 2 +- src/ipa/ipu3/algorithms/af.h | 2 +- src/ipa/ipu3/algorithms/agc.cpp | 8 ++++---- src/ipa/ipu3/algorithms/agc.h | 4 ++-- src/ipa/ipu3/algorithms/awb.cpp | 2 +- src/ipa/ipu3/algorithms/awb.h | 2 +- src/ipa/ipu3/algorithms/tone_mapping.cpp | 2 +- src/ipa/ipu3/algorithms/tone_mapping.h | 2 +- src/ipa/ipu3/ipu3.cpp | 2 +- src/ipa/libipa/algorithm.h | 2 +- src/ipa/rkisp1/algorithms/agc.cpp | 3 ++- src/ipa/rkisp1/algorithms/agc.h | 2 +- src/ipa/rkisp1/algorithms/awb.cpp | 2 +- src/ipa/rkisp1/algorithms/awb.h | 2 +- src/ipa/rkisp1/rkisp1.cpp | 5 ++++- 15 files changed, 23 insertions(+), 19 deletions(-) diff --git a/src/ipa/ipu3/algorithms/af.cpp b/src/ipa/ipu3/algorithms/af.cpp index 9127c24f1287..5ab64bf88313 100644 --- a/src/ipa/ipu3/algorithms/af.cpp +++ b/src/ipa/ipu3/algorithms/af.cpp @@ -417,7 +417,7 @@ bool Af::afIsOutOfFocus(IPAContext &context) * * [1] Hill Climbing Algorithm, https://en.wikipedia.org/wiki/Hill_climbing */ -void Af::process(IPAContext &context, [[maybe_unused]] IPAFrameContext *frameContext, +void Af::process(IPAContext &context, [[maybe_unused]] IPAFrameContext &frameContext, const ipu3_uapi_stats_3a *stats) { /* Evaluate the AF buffer length */ diff --git a/src/ipa/ipu3/algorithms/af.h b/src/ipa/ipu3/algorithms/af.h index 9b93594898bb..29117e8bdd0d 100644 --- a/src/ipa/ipu3/algorithms/af.h +++ b/src/ipa/ipu3/algorithms/af.h @@ -32,7 +32,7 @@ public: void prepare(IPAContext &context, ipu3_uapi_params *params) override; int configure(IPAContext &context, const IPAConfigInfo &configInfo) override; - void process(IPAContext &context, IPAFrameContext *frameContext, + void process(IPAContext &context, IPAFrameContext &frameContext, const ipu3_uapi_stats_3a *stats) override; private: diff --git a/src/ipa/ipu3/algorithms/agc.cpp b/src/ipa/ipu3/algorithms/agc.cpp index ed4809d98007..f8ca29640a44 100644 --- a/src/ipa/ipu3/algorithms/agc.cpp +++ b/src/ipa/ipu3/algorithms/agc.cpp @@ -183,13 +183,13 @@ utils::Duration Agc::filterExposure(utils::Duration exposureValue) * \param[in] yGain The gain calculated based on the relative luminance target * \param[in] iqMeanGain The gain calculated based on the relative luminance target */ -void Agc::computeExposure(IPAContext &context, IPAFrameContext *frameContext, +void Agc::computeExposure(IPAContext &context, IPAFrameContext &frameContext, double yGain, double iqMeanGain) { const IPASessionConfiguration &configuration = context.configuration; /* Get the effective exposure and gain applied on the sensor. */ - uint32_t exposure = frameContext->sensor.exposure; - double analogueGain = frameContext->sensor.gain; + uint32_t exposure = frameContext.sensor.exposure; + double analogueGain = frameContext.sensor.gain; /* Use the highest of the two gain estimates. */ double evGain = std::max(yGain, iqMeanGain); @@ -323,7 +323,7 @@ double Agc::estimateLuminance(IPAActiveState &activeState, * Identify the current image brightness, and use that to estimate the optimal * new exposure and gain for the scene. */ -void Agc::process(IPAContext &context, [[maybe_unused]] IPAFrameContext *frameContext, +void Agc::process(IPAContext &context, IPAFrameContext &frameContext, const ipu3_uapi_stats_3a *stats) { /* diff --git a/src/ipa/ipu3/algorithms/agc.h b/src/ipa/ipu3/algorithms/agc.h index 105ae0f2aac6..876fbbb6b585 100644 --- a/src/ipa/ipu3/algorithms/agc.h +++ b/src/ipa/ipu3/algorithms/agc.h @@ -28,14 +28,14 @@ public: ~Agc() = default; int configure(IPAContext &context, const IPAConfigInfo &configInfo) override; - void process(IPAContext &context, IPAFrameContext *frameContext, + void process(IPAContext &context, IPAFrameContext &frameContext, const ipu3_uapi_stats_3a *stats) override; private: double measureBrightness(const ipu3_uapi_stats_3a *stats, const ipu3_uapi_grid_config &grid) const; utils::Duration filterExposure(utils::Duration currentExposure); - void computeExposure(IPAContext &context, IPAFrameContext *frameContext, + void computeExposure(IPAContext &context, IPAFrameContext &frameContext, double yGain, double iqMeanGain); double estimateLuminance(IPAActiveState &activeState, const ipu3_uapi_grid_config &grid, diff --git a/src/ipa/ipu3/algorithms/awb.cpp b/src/ipa/ipu3/algorithms/awb.cpp index b658ee546b87..128f5d9d5351 100644 --- a/src/ipa/ipu3/algorithms/awb.cpp +++ b/src/ipa/ipu3/algorithms/awb.cpp @@ -387,7 +387,7 @@ void Awb::calculateWBGains(const ipu3_uapi_stats_3a *stats) /** * \copydoc libcamera::ipa::Algorithm::process */ -void Awb::process(IPAContext &context, [[maybe_unused]] IPAFrameContext *frameContext, +void Awb::process(IPAContext &context, [[maybe_unused]] IPAFrameContext &frameContext, const ipu3_uapi_stats_3a *stats) { calculateWBGains(stats); diff --git a/src/ipa/ipu3/algorithms/awb.h b/src/ipa/ipu3/algorithms/awb.h index 0acd21480845..ebbb2d58be7e 100644 --- a/src/ipa/ipu3/algorithms/awb.h +++ b/src/ipa/ipu3/algorithms/awb.h @@ -40,7 +40,7 @@ public: int configure(IPAContext &context, const IPAConfigInfo &configInfo) override; void prepare(IPAContext &context, ipu3_uapi_params *params) override; - void process(IPAContext &context, IPAFrameContext *frameContext, + void process(IPAContext &context, IPAFrameContext &frameContext, const ipu3_uapi_stats_3a *stats) override; private: diff --git a/src/ipa/ipu3/algorithms/tone_mapping.cpp b/src/ipa/ipu3/algorithms/tone_mapping.cpp index c21647e8c51b..24faba9be2b2 100644 --- a/src/ipa/ipu3/algorithms/tone_mapping.cpp +++ b/src/ipa/ipu3/algorithms/tone_mapping.cpp @@ -78,7 +78,7 @@ void ToneMapping::prepare([[maybe_unused]] IPAContext &context, * The tone mapping look up table is generated as an inverse power curve from * our gamma setting. */ -void ToneMapping::process(IPAContext &context, [[maybe_unused]] IPAFrameContext *frameContext, +void ToneMapping::process(IPAContext &context, [[maybe_unused]] IPAFrameContext &frameContext, [[maybe_unused]] const ipu3_uapi_stats_3a *stats) { /* diff --git a/src/ipa/ipu3/algorithms/tone_mapping.h b/src/ipa/ipu3/algorithms/tone_mapping.h index d7d4800628f2..5be62f6ee9bd 100644 --- a/src/ipa/ipu3/algorithms/tone_mapping.h +++ b/src/ipa/ipu3/algorithms/tone_mapping.h @@ -20,7 +20,7 @@ public: int configure(IPAContext &context, const IPAConfigInfo &configInfo) override; void prepare(IPAContext &context, ipu3_uapi_params *params) override; - void process(IPAContext &context, IPAFrameContext *frameContext, + void process(IPAContext &context, IPAFrameContext &frameContext, const ipu3_uapi_stats_3a *stats) override; private: diff --git a/src/ipa/ipu3/ipu3.cpp b/src/ipa/ipu3/ipu3.cpp index e3c1e8160759..f0c92507533c 100644 --- a/src/ipa/ipu3/ipu3.cpp +++ b/src/ipa/ipu3/ipu3.cpp @@ -616,7 +616,7 @@ void IPAIPU3::processStatsBuffer(const uint32_t frame, ControlList ctrls(controls::controls); for (auto const &algo : algorithms()) - algo->process(context_, &frameContext, stats); + algo->process(context_, frameContext, stats); setControls(frame); diff --git a/src/ipa/libipa/algorithm.h b/src/ipa/libipa/algorithm.h index ccc659a63e3b..0fe3d772963a 100644 --- a/src/ipa/libipa/algorithm.h +++ b/src/ipa/libipa/algorithm.h @@ -49,7 +49,7 @@ public: } virtual void process([[maybe_unused]] typename Module::Context &context, - [[maybe_unused]] typename Module::FrameContext *frameContext, + [[maybe_unused]] typename Module::FrameContext &frameContext, [[maybe_unused]] const typename Module::Stats *stats) { } diff --git a/src/ipa/rkisp1/algorithms/agc.cpp b/src/ipa/rkisp1/algorithms/agc.cpp index 708c36ed1af8..7642796d69a3 100644 --- a/src/ipa/rkisp1/algorithms/agc.cpp +++ b/src/ipa/rkisp1/algorithms/agc.cpp @@ -275,13 +275,14 @@ double Agc::measureBrightness(const rkisp1_cif_isp_hist_stat *hist) const /** * \brief Process RkISP1 statistics, and run AGC operations * \param[in] context The shared IPA context + * \param[in] frame The frame context sequence number * \param[in] stats The RKISP1 statistics and ISP results * * Identify the current image brightness, and use that to estimate the optimal * new exposure and gain for the scene. */ void Agc::process(IPAContext &context, - [[maybe_unused]] IPAFrameContext *frameContext, + [[maybe_unused]] IPAFrameContext &frameContext, const rkisp1_stat_buffer *stats) { const rkisp1_cif_isp_stat *params = &stats->params; diff --git a/src/ipa/rkisp1/algorithms/agc.h b/src/ipa/rkisp1/algorithms/agc.h index 1c9818b7db2a..6a5723ecbcf8 100644 --- a/src/ipa/rkisp1/algorithms/agc.h +++ b/src/ipa/rkisp1/algorithms/agc.h @@ -27,7 +27,7 @@ public: int configure(IPAContext &context, const IPACameraSensorInfo &configInfo) override; void prepare(IPAContext &context, rkisp1_params_cfg *params) override; - void process(IPAContext &context, IPAFrameContext *frameContext, + void process(IPAContext &context, IPAFrameContext &frameContext, const rkisp1_stat_buffer *stats) override; private: diff --git a/src/ipa/rkisp1/algorithms/awb.cpp b/src/ipa/rkisp1/algorithms/awb.cpp index b12df21de4aa..2bc5d3aa16ab 100644 --- a/src/ipa/rkisp1/algorithms/awb.cpp +++ b/src/ipa/rkisp1/algorithms/awb.cpp @@ -152,7 +152,7 @@ void Awb::queueRequest(IPAContext &context, * \copydoc libcamera::ipa::Algorithm::process */ void Awb::process([[maybe_unused]] IPAContext &context, - [[maybe_unused]] IPAFrameContext *frameCtx, + [[maybe_unused]] IPAFrameContext &frameCtx, const rkisp1_stat_buffer *stats) { const rkisp1_cif_isp_stat *params = &stats->params; diff --git a/src/ipa/rkisp1/algorithms/awb.h b/src/ipa/rkisp1/algorithms/awb.h index 4917267bb99d..fc221acea617 100644 --- a/src/ipa/rkisp1/algorithms/awb.h +++ b/src/ipa/rkisp1/algorithms/awb.h @@ -23,7 +23,7 @@ public: void prepare(IPAContext &context, rkisp1_params_cfg *params) override; void queueRequest(IPAContext &context, const uint32_t frame, const ControlList &controls) override; - void process(IPAContext &context, IPAFrameContext *frameCtx, + void process(IPAContext &context, IPAFrameContext &frameCtx, const rkisp1_stat_buffer *stats) override; private: diff --git a/src/ipa/rkisp1/rkisp1.cpp b/src/ipa/rkisp1/rkisp1.cpp index 3ff1eb48f605..3bc6d4dd2784 100644 --- a/src/ipa/rkisp1/rkisp1.cpp +++ b/src/ipa/rkisp1/rkisp1.cpp @@ -326,8 +326,11 @@ void IPARkISP1::processStatsBuffer(const uint32_t frame, const uint32_t bufferId unsigned int aeState = 0; + /* \todo Obtain the frame context to pass to process from the FCQueue */ + IPAFrameContext frameContext; + for (auto const &algo : algorithms()) - algo->process(context_, nullptr, stats); + algo->process(context_, frameContext, stats); setControls(frame); From patchwork Thu Sep 8 01:41:32 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Laurent Pinchart X-Patchwork-Id: 17313 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 64E1EC3272 for ; Thu, 8 Sep 2022 01:42:25 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 14DD26209E; Thu, 8 Sep 2022 03:42:25 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1662601345; bh=Lfvv4mWBWOcWKc2TlDW7eM3J7wXgI98zO/kEion9OG0=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To: From; b=sQK/HcbBS/c46no/Yreju6heygCAj/5HkzzWrOoi+D3UFmk4glgO1IlSDDVwRrPBx dAor8OOS4plYigoGNUoOToiPiy2o0tUfYrMwMAKr3nsRJl4+xdsqDm0ntpBxfmJJ2e shL0jezivC7Z1d84aXF/GWPF8HnMF3VLIpURdmHK4dM7lOextI5RpKu5FlKsCkv3Q8 LcnuooTjH+ep3Yuwv6mpJ/9a+mrw6lnMRIcEFZmXk+sfQ1Tvb/EzWqKpNJDAPAoEuy ybJ/7zD7nB8Ate+giuDIxOTIGiL11ZVDOMYTpGv8LymoRYlE30M7OTYbTENmXB+7Ew Y8YRsT16xNOag== Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 6B9766209F for ; Thu, 8 Sep 2022 03:42:23 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="Wm5DGkOU"; dkim-atps=neutral Received: from pendragon.ideasonboard.com (62-78-145-57.bb.dnainternet.fi [62.78.145.57]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id CB60A6CC; Thu, 8 Sep 2022 03:42:22 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1662601343; bh=Lfvv4mWBWOcWKc2TlDW7eM3J7wXgI98zO/kEion9OG0=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=Wm5DGkOUgNShYLRsOOWzs68dm0hDxrVnszvABqzESKPFTAayz8dMeXKpIaPlpaz4b VFYiSTiAF5aqc2JD2LwOPG9SRA+3qe1bLl36tqSoOOpxZI1mWpxJGPOJ9fSk9zMN+/ T73UAKJaACLf2IVOdW2fRUa6UzlzHDfY3y6BvACU= To: libcamera-devel@lists.libcamera.org Date: Thu, 8 Sep 2022 04:41:32 +0300 Message-Id: <20220908014200.28728-5-laurent.pinchart@ideasonboard.com> X-Mailer: git-send-email 2.35.1 In-Reply-To: <20220908014200.28728-1-laurent.pinchart@ideasonboard.com> References: <20220908014200.28728-1-laurent.pinchart@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v4 04/32] ipa: libipa: Introduce FrameContextQueue X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Laurent Pinchart via libcamera-devel From: Laurent Pinchart Reply-To: Laurent Pinchart Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" From: Umang Jain Introduce a common implementation in libipa to represent the queue of frame contexts. Signed-off-by: Umang Jain Signed-off-by: Kieran Bingham Signed-off-by: Jacopo Mondi Reviewed-by: Kieran Bingham --- Changes since v3: - Split the IPU3 changes to a separate patch - Use composition instead of inheritance - Use vector instead of array for backend storage - Make the queue size dynamic - Rename initialise() to init() --- src/ipa/libipa/fc_queue.cpp | 117 ++++++++++++++++++++++++++++++++++++ src/ipa/libipa/fc_queue.h | 108 +++++++++++++++++++++++++++++++++ src/ipa/libipa/meson.build | 2 + 3 files changed, 227 insertions(+) create mode 100644 src/ipa/libipa/fc_queue.cpp create mode 100644 src/ipa/libipa/fc_queue.h diff --git a/src/ipa/libipa/fc_queue.cpp b/src/ipa/libipa/fc_queue.cpp new file mode 100644 index 000000000000..b81d497e255a --- /dev/null +++ b/src/ipa/libipa/fc_queue.cpp @@ -0,0 +1,117 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2022, Google Inc. + * + * fc_queue.cpp - IPA Frame context queue + */ + +#include "fc_queue.h" + +#include + +namespace libcamera { + +LOG_DEFINE_CATEGORY(FCQueue) + +namespace ipa { + +/** + * \file fc_queue.h + * \brief Queue to access per-frame Context + */ + +/** + * \class FCQueue + * \brief A support class for queueing FrameContext instances through the IPA + * \tparam FrameContext The IPA specific FrameContext derived class type + * + * The frame context queue provides a simple circular buffer implementation to + * store IPA specific context for each frame through its lifetime within the + * IPA. + * + * FrameContext instances are expected to be referenced by a monotonically + * increasing sequence count corresponding to a frame sequence number. + * + * A FrameContext is initialised for a given frame when the corresponding + * Request is first queued into the IPA. From that point on the FrameContext can + * be obtained by the IPA and its algorithms by referencing it from the frame + * sequence number. + * + * A frame sequence number from the image source must correspond to the request + * sequence number for this implementation to be supported in an IPA. It is + * expected that the same sequence number will be used to reference the context + * of the frame from the point of queueing the request, specifying controls for + * a given frame, and processing of any ISP related controls and statistics for + * the same corresponding image. + * + * IPA specific frame context implementations shall inherit from the + * IPAFrameContext base class to support the minimum required features for a + * FrameContext. + * + * FrameContexts are overwritten when they are recycled and re-initialised by + * the first access made on them by either init(frame) or get(frame). It is + * required that the number of available slots in the frame context queue is + * larger or equal to the maximum number of in-flight requests a pipeline can + * handle to avoid overwriting frame context instances that still have to be + * processed. + * + * In the case an application does not queue requests to the camera fast + * enough, frames can be produced and processed by the IPA without a + * corresponding Request being queued. In this case the IPA algorithm + * will try to access the FrameContext with a call to get() before it + * had been initialized at queueRequest() time. In this case, there + * is no way the controls associated with the late Request could be + * applied in time, as the frame as already been processed by the IPA. + * + * \todo Set an error flag for per-frame control errors. + */ + +/** + * \fn FCQueue::FCQueue(unsigned int size) + * \brief Construct a frame context queue of a specified size + * \param[in] size The number of contexts in the queue + */ + +/** + * \fn FCQueue::clear() + * \brief Clear the context queue + * + * Reset the queue and ensure all FrameContext slots are re-initialised. + * IPA modules are expected to clear the frame context queue at the beginning of + * a new streaming session, in IPAModule::start(). + */ + +/** + * \fn FCQueue::init(uint32_t frame) + * \brief Initialize and return the Frame Context at slot specified by \a frame + * \param[in] frame The frame context sequence number + * + * The first call to obtain a FrameContext from the FCQueue should be handled + * through this call. The FrameContext will be initialised for the frame and + * returned to the caller if it was not already initialised. + * + * If the FrameContext was already initialized for this sequence, a warning will + * be reported and the previously initialized FrameContext is returned. + * + * Frame contexts are expected to be initialised when a Request is first passed + * to the IPA module in IPAModule::queueRequest(). + * + * \return A reference to the FrameContext for sequence \a frame + */ + +/** + * \fn FCQueue::get() + * \brief Obtain the Frame Context at slot specified by \a frame + * \param[in] frame The frame context sequence number + * + * Obtains an existing FrameContext from the queue and returns it to the caller. + * + * If the FrameContext is not correctly initialised for the \a frame, it will be + * initialised. + * + * \return A reference to the FrameContext for sequence \a frame + */ + +} /* namespace ipa */ + +} /* namespace libcamera */ diff --git a/src/ipa/libipa/fc_queue.h b/src/ipa/libipa/fc_queue.h new file mode 100644 index 000000000000..0f3af0f3a189 --- /dev/null +++ b/src/ipa/libipa/fc_queue.h @@ -0,0 +1,108 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2022, Google Inc. + * + * fc_queue.h - IPA Frame context queue + */ + +#pragma once + +#include +#include + +#include + +namespace libcamera { + +LOG_DECLARE_CATEGORY(FCQueue) + +namespace ipa { + +template +class FCQueue +{ +public: + FCQueue(unsigned int size) + : contexts_(size) + { + } + + void clear() + { + std::fill(contexts_.begin(), contexts_.end(), FrameContext{}); + } + + FrameContext &init(const uint32_t frame) + { + FrameContext &frameContext = contexts_[frame % contexts_.size()]; + + /* + * Do not re-initialise if a get() call has already fetched this + * frame context to preseve the context. + * + * \todo If the the sequence number of the context to initialise + * is smaller than the sequence number of the queue slot to use, + * it means that we had a serious request underrun and more + * frames than the queue size has been produced since the last + * time the application has queued a request. Does this deserve + * an error condition ? + */ + if (frame != 0 && frame <= frameContext.frame) + LOG(FCQueue, Warning) + << "Frame " << frame << " already initialised"; + else + init(frameContext, frame); + + return frameContext; + } + + FrameContext &get(uint32_t frame) + { + FrameContext &frameContext = contexts_[frame % contexts_.size()]; + + /* + * If the IPA algorithms try to access a frame context slot which + * has been already overwritten by a newer context, it means the + * frame context queue has overflowed and the desired context + * has been forever lost. The pipeline handler shall avoid + * queueing more requests to the IPA than the frame context + * queue size. + */ + if (frame < frameContext.frame) + LOG(FCQueue, Fatal) << "Frame Context for " << frame + << " has been overwritten by " + << frameContext.frame; + + if (frame == frameContext.frame) + return frameContext; + + /* + * The frame context has been retrieved before it was + * initialised through the initialise() call. This indicates an + * algorithm attempted to access a Frame context before it was + * queued to the IPA. Controls applied for this request may be + * left unhandled. + * + * \todo Set an error flag for per-frame control errors. + */ + LOG(FCQueue, Warning) + << "Obtained an uninitialised FrameContext for " << frame; + + init(frameContext, frame); + + return frameContext; + } + +private: + void init(FrameContext &frameContext, const uint32_t frame) + { + frameContext = {}; + frameContext.frame = frame; + } + + std::vector contexts_; +}; + +} /* namespace ipa */ + +} /* namespace libcamera */ diff --git a/src/ipa/libipa/meson.build b/src/ipa/libipa/meson.build index fb894bc614af..016b8e0ec9be 100644 --- a/src/ipa/libipa/meson.build +++ b/src/ipa/libipa/meson.build @@ -3,6 +3,7 @@ libipa_headers = files([ 'algorithm.h', 'camera_sensor_helper.h', + 'fc_queue.h', 'histogram.h', 'module.h', ]) @@ -10,6 +11,7 @@ libipa_headers = files([ libipa_sources = files([ 'algorithm.cpp', 'camera_sensor_helper.cpp', + 'fc_queue.cpp', 'histogram.cpp', 'module.cpp', ]) From patchwork Thu Sep 8 01:41:33 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Laurent Pinchart X-Patchwork-Id: 17314 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id C46DAC327D for ; Thu, 8 Sep 2022 01:42:26 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 89910620A8; Thu, 8 Sep 2022 03:42:26 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1662601346; bh=XHI/YYMDnKXaWp7Try6x/5ydzAneF2irlbrDSiVsVL0=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To: From; b=hlYjlbR5o0jODCWnszneRehcNpznXk+n7TTG4a2a9Bg6c0xmAskvBjd3pTwi/x1ig LNGdHUqhIDWwNLm0bynIQt83HNp+tS6CPJdWl3y+7c+kAaBhnXM5hIABekQStyuQiF w5WeZq4GdXTsO/6ANV/u5+L3gjdasnaIcIXQpcd+Z/xMHJuGHMQFAkBAJ2ZAHly0lO y5qaCpGXlc4wUtX+hUdhGS1TojDodV6uBobz5xwrsZ/Rd+mZSwX1ltMSjh5jfxHZw2 jI1iX7CKB50KDrGWCPyn9VESP2p2SB6x7SmCONF2rwZPIFlrjg+zjgcis6SjS05Fxk miJK9651+VgZw== Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id A7D0A6209E for ; Thu, 8 Sep 2022 03:42:24 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="sZ9xvGnc"; dkim-atps=neutral Received: from pendragon.ideasonboard.com (62-78-145-57.bb.dnainternet.fi [62.78.145.57]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 300D7888; Thu, 8 Sep 2022 03:42:24 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1662601344; bh=XHI/YYMDnKXaWp7Try6x/5ydzAneF2irlbrDSiVsVL0=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=sZ9xvGncsKHVhJgW7XllurqbUAZNaarWZjRZXaADj9ttgM5N9ujWCOSDo9hBmy4lQ sEqE9FarL7058rtjp4zV6a4JaDDM7wJ0/zYDwmCk4IXaX0mgbTRNqdJeXgM1TJafvh VP208xq42Yi+oA3mPrpEWVdH+U/YNBiwF7bWL3MM= To: libcamera-devel@lists.libcamera.org Date: Thu, 8 Sep 2022 04:41:33 +0300 Message-Id: <20220908014200.28728-6-laurent.pinchart@ideasonboard.com> X-Mailer: git-send-email 2.35.1 In-Reply-To: <20220908014200.28728-1-laurent.pinchart@ideasonboard.com> References: <20220908014200.28728-1-laurent.pinchart@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v4 05/32] ipa: libipa: Provide a common base for frame contexts X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Laurent Pinchart via libcamera-devel From: Laurent Pinchart Reply-To: Laurent Pinchart Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" From: Kieran Bingham Provide a common FrameContext as a base for IPA modules to inherit from. This will allow having a common set of parameters for every frame context managed by the FCQueue implementation. Signed-off-by: Kieran Bingham Reviewed-by: Kieran Bingham Reviewed-by: Jacopo Mondi --- Changes since v3: - Rename structure from IPAFrameContext to FrameContext - Move changes to IPA modules to separate patches - Make the FrameContext::frame member private --- src/ipa/libipa/fc_queue.cpp | 18 ++++++++++++++++++ src/ipa/libipa/fc_queue.h | 10 ++++++++++ 2 files changed, 28 insertions(+) diff --git a/src/ipa/libipa/fc_queue.cpp b/src/ipa/libipa/fc_queue.cpp index b81d497e255a..8f61c85131c9 100644 --- a/src/ipa/libipa/fc_queue.cpp +++ b/src/ipa/libipa/fc_queue.cpp @@ -20,6 +20,24 @@ namespace ipa { * \brief Queue to access per-frame Context */ +/** + * \struct FrameContext + * \brief Context for a frame + * + * The frame context stores data specific to a single frame processed by the + * IPA. Each frame processed by the IPA has a context associated with it, + * accessible through the Frame Context Queue. + * + * Fields in the frame context should reflect values and controls associated + * with the specific frame as requested by the application, and as configured by + * the hardware. Fields can be read by algorithms to determine if they should + * update any specific action for this frame, and finally to update the metadata + * control lists when the frame is fully completed. + * + * \var FrameContext::frame + * \brief The frame number + */ + /** * \class FCQueue * \brief A support class for queueing FrameContext instances through the IPA diff --git a/src/ipa/libipa/fc_queue.h b/src/ipa/libipa/fc_queue.h index 0f3af0f3a189..d8a5b8297a70 100644 --- a/src/ipa/libipa/fc_queue.h +++ b/src/ipa/libipa/fc_queue.h @@ -8,6 +8,7 @@ #pragma once #include +#include #include #include @@ -18,6 +19,15 @@ LOG_DECLARE_CATEGORY(FCQueue) namespace ipa { +template +class FCQueue; + +struct FrameContext { +private: + template friend class FCQueue; + uint32_t frame; +}; + template class FCQueue { From patchwork Thu Sep 8 01:41:34 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Laurent Pinchart X-Patchwork-Id: 17315 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 54270C3272 for ; Thu, 8 Sep 2022 01:42:29 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 0AA7D620AE; Thu, 8 Sep 2022 03:42:29 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1662601349; bh=PjM5F25fh2xkqfLem6G67AjIXpfzMI0rq80UtYxMTAY=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To: From; b=oKVa0vEY7MKbARtHBoL5hwZQrCnxTLLiI+ueVIHXxlE8C4s3JqcXtkm4/ACU+EO9t EWMHBv9zZ6W6T4fnij3vS6lSDlzRoqxjVs98WO0HFkpnxVZHxb5CdGz/rlXA2XIsp5 mMoXqT17v0PH2HV8BWvN8GOqvdq/NvfbSOqX0m2YDGjyogIMUzGlgjgaAEf4scju1h t1vQzC+KZWg5wwzznx+74QLR96ZFKDTmfGCum7B0VtvZoNyYpyragja6F9QARtyxN+ 2/3hiVHtqwy1XC4Be8TNR++sxWbOOmIne4Q39zX8POByOWbvSglarWxtWTOxma9miz 3RT1SCUZAls/Q== Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 5E23862095 for ; Thu, 8 Sep 2022 03:42:26 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="mTUnJHDq"; dkim-atps=neutral Received: from pendragon.ideasonboard.com (62-78-145-57.bb.dnainternet.fi [62.78.145.57]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id C69E9888; Thu, 8 Sep 2022 03:42:25 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1662601346; bh=PjM5F25fh2xkqfLem6G67AjIXpfzMI0rq80UtYxMTAY=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=mTUnJHDqRRvkQm7qXiLoBSlq08mSx4kdNyyYbOtIko3LwrsnHkJGT9bKZuHDQBPl8 xtjg04gYVSFRi2P4Rrsg6Aac270+5/Rcf6gA/NAJ4cG0HXNTzX1Ru9z/92N7UOMP3S XVgiV73M9lxO9W1Mjjnh39K9AY/GGSAqpDi80szw= To: libcamera-devel@lists.libcamera.org Date: Thu, 8 Sep 2022 04:41:34 +0300 Message-Id: <20220908014200.28728-7-laurent.pinchart@ideasonboard.com> X-Mailer: git-send-email 2.35.1 In-Reply-To: <20220908014200.28728-1-laurent.pinchart@ideasonboard.com> References: <20220908014200.28728-1-laurent.pinchart@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v4 06/32] ipa: libipa: algorithm: prepare(): Pass frame and frame Context X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Laurent Pinchart via libcamera-devel From: Laurent Pinchart Reply-To: Laurent Pinchart Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" From: Kieran Bingham Pass the current frame number, and the current FrameContext for calls to prepare. Signed-off-by: Kieran Bingham Signed-off-by: Jacopo Mondi Reviewed-by: Umang Jain Reviewed-by: Kieran Bingham Reviewed-by: Jacopo Mondi --- Changes since v3: - Use a const uint32_t instead of an unsigned int for the frame number --- src/ipa/ipu3/algorithms/af.cpp | 5 ++++- src/ipa/ipu3/algorithms/af.h | 4 +++- src/ipa/ipu3/algorithms/awb.cpp | 5 ++++- src/ipa/ipu3/algorithms/awb.h | 4 +++- src/ipa/ipu3/algorithms/blc.cpp | 6 +++++- src/ipa/ipu3/algorithms/blc.h | 4 +++- src/ipa/ipu3/algorithms/tone_mapping.cpp | 4 ++++ src/ipa/ipu3/algorithms/tone_mapping.h | 3 ++- src/ipa/ipu3/ipu3.cpp | 4 +++- src/ipa/libipa/algorithm.cpp | 2 ++ src/ipa/libipa/algorithm.h | 3 +++ src/ipa/rkisp1/algorithms/agc.cpp | 5 ++++- src/ipa/rkisp1/algorithms/agc.h | 4 +++- src/ipa/rkisp1/algorithms/awb.cpp | 5 ++++- src/ipa/rkisp1/algorithms/awb.h | 4 +++- src/ipa/rkisp1/algorithms/blc.cpp | 2 ++ src/ipa/rkisp1/algorithms/blc.h | 4 +++- src/ipa/rkisp1/algorithms/cproc.cpp | 2 ++ src/ipa/rkisp1/algorithms/cproc.h | 4 +++- src/ipa/rkisp1/algorithms/dpcc.cpp | 2 ++ src/ipa/rkisp1/algorithms/dpcc.h | 4 +++- src/ipa/rkisp1/algorithms/dpf.cpp | 4 +++- src/ipa/rkisp1/algorithms/dpf.h | 4 +++- src/ipa/rkisp1/algorithms/filter.cpp | 5 ++++- src/ipa/rkisp1/algorithms/filter.h | 4 +++- src/ipa/rkisp1/algorithms/gsl.cpp | 2 ++ src/ipa/rkisp1/algorithms/gsl.h | 4 +++- src/ipa/rkisp1/algorithms/lsc.cpp | 2 ++ src/ipa/rkisp1/algorithms/lsc.h | 4 +++- src/ipa/rkisp1/rkisp1.cpp | 5 ++++- 30 files changed, 92 insertions(+), 22 deletions(-) diff --git a/src/ipa/ipu3/algorithms/af.cpp b/src/ipa/ipu3/algorithms/af.cpp index 5ab64bf88313..bc01b232ec59 100644 --- a/src/ipa/ipu3/algorithms/af.cpp +++ b/src/ipa/ipu3/algorithms/af.cpp @@ -116,7 +116,10 @@ Af::Af() /** * \copydoc libcamera::ipa::Algorithm::prepare */ -void Af::prepare(IPAContext &context, ipu3_uapi_params *params) +void Af::prepare(IPAContext &context, + [[maybe_unused]] const uint32_t frame, + [[maybe_unused]] IPAFrameContext &frameContext, + ipu3_uapi_params *params) { const struct ipu3_uapi_grid_config &grid = context.configuration.af.afGrid; params->acc_param.af.grid_cfg = grid; diff --git a/src/ipa/ipu3/algorithms/af.h b/src/ipa/ipu3/algorithms/af.h index 29117e8bdd0d..fbfd171b6465 100644 --- a/src/ipa/ipu3/algorithms/af.h +++ b/src/ipa/ipu3/algorithms/af.h @@ -30,8 +30,10 @@ public: Af(); ~Af() = default; - void prepare(IPAContext &context, ipu3_uapi_params *params) override; int configure(IPAContext &context, const IPAConfigInfo &configInfo) override; + void prepare(IPAContext &context, const uint32_t frame, + IPAFrameContext &frameContext, + ipu3_uapi_params *params) override; void process(IPAContext &context, IPAFrameContext &frameContext, const ipu3_uapi_stats_3a *stats) override; diff --git a/src/ipa/ipu3/algorithms/awb.cpp b/src/ipa/ipu3/algorithms/awb.cpp index 128f5d9d5351..e6e2951b8abb 100644 --- a/src/ipa/ipu3/algorithms/awb.cpp +++ b/src/ipa/ipu3/algorithms/awb.cpp @@ -429,7 +429,10 @@ constexpr uint16_t Awb::gainValue(double gain) /** * \copydoc libcamera::ipa::Algorithm::prepare */ -void Awb::prepare(IPAContext &context, ipu3_uapi_params *params) +void Awb::prepare(IPAContext &context, + [[maybe_unused]] const uint32_t frame, + [[maybe_unused]] IPAFrameContext &frameContext, + ipu3_uapi_params *params) { /* * Green saturation thresholds are reduced because we are using the diff --git a/src/ipa/ipu3/algorithms/awb.h b/src/ipa/ipu3/algorithms/awb.h index ebbb2d58be7e..e9e816928fb4 100644 --- a/src/ipa/ipu3/algorithms/awb.h +++ b/src/ipa/ipu3/algorithms/awb.h @@ -39,7 +39,9 @@ public: ~Awb(); int configure(IPAContext &context, const IPAConfigInfo &configInfo) override; - void prepare(IPAContext &context, ipu3_uapi_params *params) override; + void prepare(IPAContext &context, const uint32_t frame, + IPAFrameContext &frameContext, + ipu3_uapi_params *params) override; void process(IPAContext &context, IPAFrameContext &frameContext, const ipu3_uapi_stats_3a *stats) override; diff --git a/src/ipa/ipu3/algorithms/blc.cpp b/src/ipa/ipu3/algorithms/blc.cpp index c32427c99549..e838072a344c 100644 --- a/src/ipa/ipu3/algorithms/blc.cpp +++ b/src/ipa/ipu3/algorithms/blc.cpp @@ -39,13 +39,17 @@ BlackLevelCorrection::BlackLevelCorrection() /** * \brief Fill in the parameter structure, and enable black level correction * \param[in] context The shared IPA context + * \param[in] frame The frame context sequence number + * \param[in] frameContext The FrameContext for this frame * \param[out] params The IPU3 parameters * * Populate the IPU3 parameter structure with the correction values for each * channel and enable the corresponding ImgU block processing. */ void BlackLevelCorrection::prepare([[maybe_unused]] IPAContext &context, - ipu3_uapi_params *params) + [[maybe_unused]] const uint32_t frame, + [[maybe_unused]] IPAFrameContext &frameContext, + ipu3_uapi_params *params) { /* * The Optical Black Level correction values diff --git a/src/ipa/ipu3/algorithms/blc.h b/src/ipa/ipu3/algorithms/blc.h index d8da1748baba..292bf67b02ff 100644 --- a/src/ipa/ipu3/algorithms/blc.h +++ b/src/ipa/ipu3/algorithms/blc.h @@ -18,7 +18,9 @@ class BlackLevelCorrection : public Algorithm public: BlackLevelCorrection(); - void prepare(IPAContext &context, ipu3_uapi_params *params) override; + void prepare(IPAContext &context, const uint32_t frame, + IPAFrameContext &frameContext, + ipu3_uapi_params *params) override; }; } /* namespace ipa::ipu3::algorithms */ diff --git a/src/ipa/ipu3/algorithms/tone_mapping.cpp b/src/ipa/ipu3/algorithms/tone_mapping.cpp index 24faba9be2b2..10a2fb1935dc 100644 --- a/src/ipa/ipu3/algorithms/tone_mapping.cpp +++ b/src/ipa/ipu3/algorithms/tone_mapping.cpp @@ -50,12 +50,16 @@ int ToneMapping::configure(IPAContext &context, /** * \brief Fill in the parameter structure, and enable gamma control * \param[in] context The shared IPA context + * \param[in] frame The frame context sequence number + * \param[in] frameContext The FrameContext for this frame * \param[out] params The IPU3 parameters * * Populate the IPU3 parameter structure with our tone mapping look up table and * enable the gamma control module in the processing blocks. */ void ToneMapping::prepare([[maybe_unused]] IPAContext &context, + [[maybe_unused]] const uint32_t frame, + [[maybe_unused]] IPAFrameContext &frameContext, ipu3_uapi_params *params) { /* Copy the calculated LUT into the parameters buffer. */ diff --git a/src/ipa/ipu3/algorithms/tone_mapping.h b/src/ipa/ipu3/algorithms/tone_mapping.h index 5be62f6ee9bd..c24a7e485aaf 100644 --- a/src/ipa/ipu3/algorithms/tone_mapping.h +++ b/src/ipa/ipu3/algorithms/tone_mapping.h @@ -19,7 +19,8 @@ public: ToneMapping(); int configure(IPAContext &context, const IPAConfigInfo &configInfo) override; - void prepare(IPAContext &context, ipu3_uapi_params *params) override; + void prepare(IPAContext &context, const uint32_t frame, + IPAFrameContext &frameContext, ipu3_uapi_params *params) override; void process(IPAContext &context, IPAFrameContext &frameContext, const ipu3_uapi_stats_3a *stats) override; diff --git a/src/ipa/ipu3/ipu3.cpp b/src/ipa/ipu3/ipu3.cpp index f0c92507533c..ac6f66afecb5 100644 --- a/src/ipa/ipu3/ipu3.cpp +++ b/src/ipa/ipu3/ipu3.cpp @@ -572,8 +572,10 @@ void IPAIPU3::fillParamsBuffer(const uint32_t frame, const uint32_t bufferId) */ params->use = {}; + IPAFrameContext &frameContext = context_.frameContexts[frame % kMaxFrameContexts]; + for (auto const &algo : algorithms()) - algo->prepare(context_, params); + algo->prepare(context_, frame, frameContext, params); paramsBufferReady.emit(frame); } diff --git a/src/ipa/libipa/algorithm.cpp b/src/ipa/libipa/algorithm.cpp index 38200e57b780..0992e5de8dd5 100644 --- a/src/ipa/libipa/algorithm.cpp +++ b/src/ipa/libipa/algorithm.cpp @@ -70,6 +70,8 @@ namespace ipa { * \fn Algorithm::prepare() * \brief Fill the \a params buffer with ISP processing parameters for a frame * \param[in] context The shared IPA context + * \param[in] frame The frame context sequence number + * \param[in] frameContext The FrameContext for this frame * \param[out] params The ISP specific parameters. * * This function is called for every frame when the camera is running before it diff --git a/src/ipa/libipa/algorithm.h b/src/ipa/libipa/algorithm.h index 0fe3d772963a..cde2e6399e8e 100644 --- a/src/ipa/libipa/algorithm.h +++ b/src/ipa/libipa/algorithm.h @@ -7,6 +7,7 @@ #pragma once #include +#include #include #include @@ -38,6 +39,8 @@ public: } virtual void prepare([[maybe_unused]] typename Module::Context &context, + [[maybe_unused]] const uint32_t frame, + [[maybe_unused]] typename Module::FrameContext &frameContext, [[maybe_unused]] typename Module::Params *params) { } diff --git a/src/ipa/rkisp1/algorithms/agc.cpp b/src/ipa/rkisp1/algorithms/agc.cpp index 7642796d69a3..0d22d9dc98fb 100644 --- a/src/ipa/rkisp1/algorithms/agc.cpp +++ b/src/ipa/rkisp1/algorithms/agc.cpp @@ -323,7 +323,10 @@ void Agc::process(IPAContext &context, /** * \copydoc libcamera::ipa::Algorithm::prepare */ -void Agc::prepare(IPAContext &context, rkisp1_params_cfg *params) +void Agc::prepare(IPAContext &context, + [[maybe_unused]] const uint32_t frame, + [[maybe_unused]] IPAFrameContext &frameContext, + rkisp1_params_cfg *params) { if (context.frameContext.frameCount > 0) return; diff --git a/src/ipa/rkisp1/algorithms/agc.h b/src/ipa/rkisp1/algorithms/agc.h index 6a5723ecbcf8..8724cb602158 100644 --- a/src/ipa/rkisp1/algorithms/agc.h +++ b/src/ipa/rkisp1/algorithms/agc.h @@ -26,7 +26,9 @@ public: ~Agc() = default; int configure(IPAContext &context, const IPACameraSensorInfo &configInfo) override; - void prepare(IPAContext &context, rkisp1_params_cfg *params) override; + void prepare(IPAContext &context, const uint32_t frame, + IPAFrameContext &frameContext, + rkisp1_params_cfg *params) override; void process(IPAContext &context, IPAFrameContext &frameContext, const rkisp1_stat_buffer *stats) override; diff --git a/src/ipa/rkisp1/algorithms/awb.cpp b/src/ipa/rkisp1/algorithms/awb.cpp index 2bc5d3aa16ab..da22a0970a4f 100644 --- a/src/ipa/rkisp1/algorithms/awb.cpp +++ b/src/ipa/rkisp1/algorithms/awb.cpp @@ -74,7 +74,10 @@ uint32_t Awb::estimateCCT(double red, double green, double blue) /** * \copydoc libcamera::ipa::Algorithm::prepare */ -void Awb::prepare(IPAContext &context, rkisp1_params_cfg *params) +void Awb::prepare(IPAContext &context, + [[maybe_unused]] const uint32_t frame, + [[maybe_unused]] IPAFrameContext &frameContext, + rkisp1_params_cfg *params) { params->others.awb_gain_config.gain_green_b = 256 * context.frameContext.awb.gains.green; params->others.awb_gain_config.gain_blue = 256 * context.frameContext.awb.gains.blue; diff --git a/src/ipa/rkisp1/algorithms/awb.h b/src/ipa/rkisp1/algorithms/awb.h index fc221acea617..7ae6c7a9bb10 100644 --- a/src/ipa/rkisp1/algorithms/awb.h +++ b/src/ipa/rkisp1/algorithms/awb.h @@ -20,7 +20,9 @@ public: ~Awb() = default; int configure(IPAContext &context, const IPACameraSensorInfo &configInfo) override; - void prepare(IPAContext &context, rkisp1_params_cfg *params) override; + void prepare(IPAContext &context, const uint32_t frame, + IPAFrameContext &frameContext, + rkisp1_params_cfg *params) override; void queueRequest(IPAContext &context, const uint32_t frame, const ControlList &controls) override; void process(IPAContext &context, IPAFrameContext &frameCtx, diff --git a/src/ipa/rkisp1/algorithms/blc.cpp b/src/ipa/rkisp1/algorithms/blc.cpp index a58569fa2dc2..c45a317ada08 100644 --- a/src/ipa/rkisp1/algorithms/blc.cpp +++ b/src/ipa/rkisp1/algorithms/blc.cpp @@ -66,6 +66,8 @@ int BlackLevelCorrection::init([[maybe_unused]] IPAContext &context, * \copydoc libcamera::ipa::Algorithm::prepare */ void BlackLevelCorrection::prepare(IPAContext &context, + [[maybe_unused]] const uint32_t frame, + [[maybe_unused]] IPAFrameContext &frameContext, rkisp1_params_cfg *params) { if (context.frameContext.frameCount > 0) diff --git a/src/ipa/rkisp1/algorithms/blc.h b/src/ipa/rkisp1/algorithms/blc.h index 5fc3a80fb638..0b1a2d436869 100644 --- a/src/ipa/rkisp1/algorithms/blc.h +++ b/src/ipa/rkisp1/algorithms/blc.h @@ -20,7 +20,9 @@ public: ~BlackLevelCorrection() = default; int init(IPAContext &context, const YamlObject &tuningData) override; - void prepare(IPAContext &context, rkisp1_params_cfg *params) override; + void prepare(IPAContext &context, const uint32_t frame, + IPAFrameContext &frameContext, + rkisp1_params_cfg *params) override; private: bool tuningParameters_; diff --git a/src/ipa/rkisp1/algorithms/cproc.cpp b/src/ipa/rkisp1/algorithms/cproc.cpp index bca5ab6907d6..c0505c68765b 100644 --- a/src/ipa/rkisp1/algorithms/cproc.cpp +++ b/src/ipa/rkisp1/algorithms/cproc.cpp @@ -71,6 +71,8 @@ void ColorProcessing::queueRequest(IPAContext &context, * \copydoc libcamera::ipa::Algorithm::prepare */ void ColorProcessing::prepare(IPAContext &context, + [[maybe_unused]] const uint32_t frame, + [[maybe_unused]] IPAFrameContext &frameContext, rkisp1_params_cfg *params) { auto &cproc = context.frameContext.cproc; diff --git a/src/ipa/rkisp1/algorithms/cproc.h b/src/ipa/rkisp1/algorithms/cproc.h index 4b7e4064d7e8..02c3d9c95488 100644 --- a/src/ipa/rkisp1/algorithms/cproc.h +++ b/src/ipa/rkisp1/algorithms/cproc.h @@ -23,7 +23,9 @@ public: void queueRequest(IPAContext &context, const uint32_t frame, const ControlList &controls) override; - void prepare(IPAContext &context, rkisp1_params_cfg *params) override; + void prepare(IPAContext &context, const uint32_t frame, + IPAFrameContext &frameContext, + rkisp1_params_cfg *params) override; }; } /* namespace ipa::rkisp1::algorithms */ diff --git a/src/ipa/rkisp1/algorithms/dpcc.cpp b/src/ipa/rkisp1/algorithms/dpcc.cpp index fb2f01cdaa3c..5030526769c7 100644 --- a/src/ipa/rkisp1/algorithms/dpcc.cpp +++ b/src/ipa/rkisp1/algorithms/dpcc.cpp @@ -259,6 +259,8 @@ int DefectPixelClusterCorrection::init([[maybe_unused]] IPAContext &context, * \copydoc libcamera::ipa::Algorithm::prepare */ void DefectPixelClusterCorrection::prepare(IPAContext &context, + [[maybe_unused]] const uint32_t frame, + [[maybe_unused]] IPAFrameContext &frameContext, rkisp1_params_cfg *params) { if (context.frameContext.frameCount > 0) diff --git a/src/ipa/rkisp1/algorithms/dpcc.h b/src/ipa/rkisp1/algorithms/dpcc.h index a363f7bee0a7..894c0249f35d 100644 --- a/src/ipa/rkisp1/algorithms/dpcc.h +++ b/src/ipa/rkisp1/algorithms/dpcc.h @@ -20,7 +20,9 @@ public: ~DefectPixelClusterCorrection() = default; int init(IPAContext &context, const YamlObject &tuningData) override; - void prepare(IPAContext &context, rkisp1_params_cfg *params) override; + void prepare(IPAContext &context, const uint32_t frame, + IPAFrameContext &frameContext, + rkisp1_params_cfg *params) override; private: bool initialized_; diff --git a/src/ipa/rkisp1/algorithms/dpf.cpp b/src/ipa/rkisp1/algorithms/dpf.cpp index c93c8361b5da..8286f83facc1 100644 --- a/src/ipa/rkisp1/algorithms/dpf.cpp +++ b/src/ipa/rkisp1/algorithms/dpf.cpp @@ -206,7 +206,9 @@ void Dpf::queueRequest(IPAContext &context, /** * \copydoc libcamera::ipa::Algorithm::prepare */ -void Dpf::prepare(IPAContext &context, rkisp1_params_cfg *params) +void Dpf::prepare(IPAContext &context, [[maybe_unused]] const uint32_t frame, + [[maybe_unused]] IPAFrameContext &frameContext, + rkisp1_params_cfg *params) { if (!initialized_) return; diff --git a/src/ipa/rkisp1/algorithms/dpf.h b/src/ipa/rkisp1/algorithms/dpf.h index 128ebd5e02e3..656d78dee67c 100644 --- a/src/ipa/rkisp1/algorithms/dpf.h +++ b/src/ipa/rkisp1/algorithms/dpf.h @@ -24,7 +24,9 @@ public: int init(IPAContext &context, const YamlObject &tuningData) override; void queueRequest(IPAContext &context, const uint32_t frame, const ControlList &controls) override; - void prepare(IPAContext &context, rkisp1_params_cfg *params) override; + void prepare(IPAContext &context, const uint32_t frame, + IPAFrameContext &frameContext, + rkisp1_params_cfg *params) override; private: bool initialized_; diff --git a/src/ipa/rkisp1/algorithms/filter.cpp b/src/ipa/rkisp1/algorithms/filter.cpp index 8ca10fd1ee9d..02562339d008 100644 --- a/src/ipa/rkisp1/algorithms/filter.cpp +++ b/src/ipa/rkisp1/algorithms/filter.cpp @@ -85,7 +85,10 @@ void Filter::queueRequest(IPAContext &context, /** * \copydoc libcamera::ipa::Algorithm::prepare */ -void Filter::prepare(IPAContext &context, rkisp1_params_cfg *params) +void Filter::prepare(IPAContext &context, + [[maybe_unused]] const uint32_t frame, + [[maybe_unused]] IPAFrameContext &frameContext, + rkisp1_params_cfg *params) { auto &filter = context.frameContext.filter; diff --git a/src/ipa/rkisp1/algorithms/filter.h b/src/ipa/rkisp1/algorithms/filter.h index 9eb170eb7da1..ac69657e2f0f 100644 --- a/src/ipa/rkisp1/algorithms/filter.h +++ b/src/ipa/rkisp1/algorithms/filter.h @@ -23,7 +23,9 @@ public: void queueRequest(IPAContext &context, const uint32_t frame, const ControlList &controls) override; - void prepare(IPAContext &context, rkisp1_params_cfg *params) override; + void prepare(IPAContext &context, const uint32_t frame, + IPAFrameContext &frameContext, + rkisp1_params_cfg *params) override; }; } /* namespace ipa::rkisp1::algorithms */ diff --git a/src/ipa/rkisp1/algorithms/gsl.cpp b/src/ipa/rkisp1/algorithms/gsl.cpp index 2fd1a23d3a9b..dfc76b3d8ffd 100644 --- a/src/ipa/rkisp1/algorithms/gsl.cpp +++ b/src/ipa/rkisp1/algorithms/gsl.cpp @@ -119,6 +119,8 @@ int GammaSensorLinearization::init([[maybe_unused]] IPAContext &context, * \copydoc libcamera::ipa::Algorithm::prepare */ void GammaSensorLinearization::prepare(IPAContext &context, + [[maybe_unused]] const uint32_t frame, + [[maybe_unused]] IPAFrameContext &frameContext, rkisp1_params_cfg *params) { if (context.frameContext.frameCount > 0) diff --git a/src/ipa/rkisp1/algorithms/gsl.h b/src/ipa/rkisp1/algorithms/gsl.h index db287dc280dc..5024b6835949 100644 --- a/src/ipa/rkisp1/algorithms/gsl.h +++ b/src/ipa/rkisp1/algorithms/gsl.h @@ -20,7 +20,9 @@ public: ~GammaSensorLinearization() = default; int init(IPAContext &context, const YamlObject &tuningData) override; - void prepare(IPAContext &context, rkisp1_params_cfg *params) override; + void prepare(IPAContext &context, const uint32_t frame, + IPAFrameContext &frameContext, + rkisp1_params_cfg *params) override; private: bool initialized_; diff --git a/src/ipa/rkisp1/algorithms/lsc.cpp b/src/ipa/rkisp1/algorithms/lsc.cpp index b9efc063ab2e..e4b04136dbaf 100644 --- a/src/ipa/rkisp1/algorithms/lsc.cpp +++ b/src/ipa/rkisp1/algorithms/lsc.cpp @@ -133,6 +133,8 @@ int LensShadingCorrection::configure(IPAContext &context, * \copydoc libcamera::ipa::Algorithm::prepare */ void LensShadingCorrection::prepare(IPAContext &context, + [[maybe_unused]] const uint32_t frame, + [[maybe_unused]] IPAFrameContext &frameContext, rkisp1_params_cfg *params) { if (context.frameContext.frameCount > 0) diff --git a/src/ipa/rkisp1/algorithms/lsc.h b/src/ipa/rkisp1/algorithms/lsc.h index f68602c005c4..da957d3e3103 100644 --- a/src/ipa/rkisp1/algorithms/lsc.h +++ b/src/ipa/rkisp1/algorithms/lsc.h @@ -21,7 +21,9 @@ public: int init(IPAContext &context, const YamlObject &tuningData) override; int configure(IPAContext &context, const IPACameraSensorInfo &configInfo) override; - void prepare(IPAContext &context, rkisp1_params_cfg *params) override; + void prepare(IPAContext &context, const uint32_t frame, + IPAFrameContext &frameContext, + rkisp1_params_cfg *params) override; private: bool initialized_; diff --git a/src/ipa/rkisp1/rkisp1.cpp b/src/ipa/rkisp1/rkisp1.cpp index 3bc6d4dd2784..0932c73b14e4 100644 --- a/src/ipa/rkisp1/rkisp1.cpp +++ b/src/ipa/rkisp1/rkisp1.cpp @@ -298,6 +298,9 @@ void IPARkISP1::queueRequest(const uint32_t frame, const ControlList &controls) void IPARkISP1::fillParamsBuffer(const uint32_t frame, const uint32_t bufferId) { + /* \todo Obtain the frame context to pass to process from the FCQueue */ + IPAFrameContext frameContext; + rkisp1_params_cfg *params = reinterpret_cast( mappedBuffers_.at(bufferId).planes()[0].data()); @@ -306,7 +309,7 @@ void IPARkISP1::fillParamsBuffer(const uint32_t frame, const uint32_t bufferId) memset(params, 0, sizeof(*params)); for (auto const &algo : algorithms()) - algo->prepare(context_, params); + algo->prepare(context_, frame, frameContext, params); paramsBufferReady.emit(frame); context_.frameContext.frameCount++; From patchwork Thu Sep 8 01:41:35 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Laurent Pinchart X-Patchwork-Id: 17316 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id DD7A9C327D for ; Thu, 8 Sep 2022 01:42:29 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 8F664620A6; Thu, 8 Sep 2022 03:42:29 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1662601349; bh=RgQ0EimBXyyZ7oUBujLkXwHHuR6LacOnWdWazuG00yM=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To: From; b=mUMmxVT0C9hNuGE+g9O+O+QdGKMXaycpZd0qlag9rx4TibPAaBz6KlnTbRORSVmcx AHLiYpaokW3z1lvFPksLDJJq6gQEMIp62eXNJL0sIk24tEBZ4gRnGqGxdjCKLG2gbJ Z4UBszU+dkFbuOjXaWa7H4Uzuaj1oiU0wb65nvkKh7U++4e4atI8NF5NxFmmKQ1Jjr ERiX9L5VlrKcCY665CDYVhu8XgtsFA0T0nYk4KaSgoqegXjQrgfxWsMcyk882FcCs/ bbqSH3GZcK2mQmlvVV9HeRi6YjRK85HyHHZWh8eYn5bJAzFRoXkOMCaMFa9EhJ2wVB tHt/YHqZkmj5w== Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id BAB8762095 for ; Thu, 8 Sep 2022 03:42:27 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="PfTF/Kf3"; dkim-atps=neutral Received: from pendragon.ideasonboard.com (62-78-145-57.bb.dnainternet.fi [62.78.145.57]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 42377888; Thu, 8 Sep 2022 03:42:27 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1662601347; bh=RgQ0EimBXyyZ7oUBujLkXwHHuR6LacOnWdWazuG00yM=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=PfTF/Kf35Elt3GxfURo2Rd41U3yOKNAbDIxxi4VWZgV0csinxldEshWpbgH21AiYB jkXq12WuD90aTwIVncodT9hnHzzdR7T7MlRGoamN1PUXBTyLx2vA4dce46+x11/btS 1kxkriKbVaJEHS/58IXj2K2cKj5xA9/IdKiU8UHk= To: libcamera-devel@lists.libcamera.org Date: Thu, 8 Sep 2022 04:41:35 +0300 Message-Id: <20220908014200.28728-8-laurent.pinchart@ideasonboard.com> X-Mailer: git-send-email 2.35.1 In-Reply-To: <20220908014200.28728-1-laurent.pinchart@ideasonboard.com> References: <20220908014200.28728-1-laurent.pinchart@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v4 07/32] ipa: libipa: algorithm: process(): Pass frame number X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Laurent Pinchart via libcamera-devel From: Laurent Pinchart Reply-To: Laurent Pinchart Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" From: Kieran Bingham Pass the frame number of the current frame being processed. Signed-off-by: Kieran Bingham Signed-off-by: Jacopo Mondi Reviewed-by: Umang Jain Reviewed-by: Kieran Bingham Reviewed-by: Jacopo Mondi --- Changes since v3: - Use a const uint32_t instead of an unsigned int for the frame number - Update the documentation of ipa::rkisp1::Agc::process() --- src/ipa/ipu3/algorithms/af.cpp | 4 +++- src/ipa/ipu3/algorithms/af.h | 3 ++- src/ipa/ipu3/algorithms/agc.cpp | 4 +++- src/ipa/ipu3/algorithms/agc.h | 3 ++- src/ipa/ipu3/algorithms/awb.cpp | 3 ++- src/ipa/ipu3/algorithms/awb.h | 3 ++- src/ipa/ipu3/algorithms/tone_mapping.cpp | 4 +++- src/ipa/ipu3/algorithms/tone_mapping.h | 3 ++- src/ipa/ipu3/ipu3.cpp | 2 +- src/ipa/libipa/algorithm.cpp | 1 + src/ipa/libipa/algorithm.h | 1 + src/ipa/rkisp1/algorithms/agc.cpp | 2 +- src/ipa/rkisp1/algorithms/agc.h | 3 ++- src/ipa/rkisp1/algorithms/awb.cpp | 1 + src/ipa/rkisp1/algorithms/awb.h | 3 ++- src/ipa/rkisp1/rkisp1.cpp | 2 +- 16 files changed, 29 insertions(+), 13 deletions(-) diff --git a/src/ipa/ipu3/algorithms/af.cpp b/src/ipa/ipu3/algorithms/af.cpp index bc01b232ec59..b00a34f9e090 100644 --- a/src/ipa/ipu3/algorithms/af.cpp +++ b/src/ipa/ipu3/algorithms/af.cpp @@ -406,6 +406,7 @@ bool Af::afIsOutOfFocus(IPAContext &context) /** * \brief Determine the max contrast image and lens position. * \param[in] context The IPA context. + * \param[in] frame The frame context sequence number * \param[in] frameContext The current frame context * \param[in] stats The statistics buffer of IPU3. * @@ -420,7 +421,8 @@ bool Af::afIsOutOfFocus(IPAContext &context) * * [1] Hill Climbing Algorithm, https://en.wikipedia.org/wiki/Hill_climbing */ -void Af::process(IPAContext &context, [[maybe_unused]] IPAFrameContext &frameContext, +void Af::process(IPAContext &context, [[maybe_unused]] const uint32_t frame, + [[maybe_unused]] IPAFrameContext &frameContext, const ipu3_uapi_stats_3a *stats) { /* Evaluate the AF buffer length */ diff --git a/src/ipa/ipu3/algorithms/af.h b/src/ipa/ipu3/algorithms/af.h index fbfd171b6465..89d37ac18615 100644 --- a/src/ipa/ipu3/algorithms/af.h +++ b/src/ipa/ipu3/algorithms/af.h @@ -34,7 +34,8 @@ public: void prepare(IPAContext &context, const uint32_t frame, IPAFrameContext &frameContext, ipu3_uapi_params *params) override; - void process(IPAContext &context, IPAFrameContext &frameContext, + void process(IPAContext &context, const uint32_t frame, + IPAFrameContext &frameContext, const ipu3_uapi_stats_3a *stats) override; private: diff --git a/src/ipa/ipu3/algorithms/agc.cpp b/src/ipa/ipu3/algorithms/agc.cpp index f8ca29640a44..a1a3c38ffe84 100644 --- a/src/ipa/ipu3/algorithms/agc.cpp +++ b/src/ipa/ipu3/algorithms/agc.cpp @@ -317,13 +317,15 @@ double Agc::estimateLuminance(IPAActiveState &activeState, /** * \brief Process IPU3 statistics, and run AGC operations * \param[in] context The shared IPA context + * \param[in] frame The current frame sequence number * \param[in] frameContext The current frame context * \param[in] stats The IPU3 statistics and ISP results * * Identify the current image brightness, and use that to estimate the optimal * new exposure and gain for the scene. */ -void Agc::process(IPAContext &context, IPAFrameContext &frameContext, +void Agc::process(IPAContext &context, [[maybe_unused]] const uint32_t frame, + IPAFrameContext &frameContext, const ipu3_uapi_stats_3a *stats) { /* diff --git a/src/ipa/ipu3/algorithms/agc.h b/src/ipa/ipu3/algorithms/agc.h index 876fbbb6b585..59b4b9843c2f 100644 --- a/src/ipa/ipu3/algorithms/agc.h +++ b/src/ipa/ipu3/algorithms/agc.h @@ -28,7 +28,8 @@ public: ~Agc() = default; int configure(IPAContext &context, const IPAConfigInfo &configInfo) override; - void process(IPAContext &context, IPAFrameContext &frameContext, + void process(IPAContext &context, const uint32_t frame, + IPAFrameContext &frameContext, const ipu3_uapi_stats_3a *stats) override; private: diff --git a/src/ipa/ipu3/algorithms/awb.cpp b/src/ipa/ipu3/algorithms/awb.cpp index e6e2951b8abb..0dbd7d4c374f 100644 --- a/src/ipa/ipu3/algorithms/awb.cpp +++ b/src/ipa/ipu3/algorithms/awb.cpp @@ -387,7 +387,8 @@ void Awb::calculateWBGains(const ipu3_uapi_stats_3a *stats) /** * \copydoc libcamera::ipa::Algorithm::process */ -void Awb::process(IPAContext &context, [[maybe_unused]] IPAFrameContext &frameContext, +void Awb::process(IPAContext &context, [[maybe_unused]] const uint32_t frame, + [[maybe_unused]] IPAFrameContext &frameContext, const ipu3_uapi_stats_3a *stats) { calculateWBGains(stats); diff --git a/src/ipa/ipu3/algorithms/awb.h b/src/ipa/ipu3/algorithms/awb.h index e9e816928fb4..28e2d38a711c 100644 --- a/src/ipa/ipu3/algorithms/awb.h +++ b/src/ipa/ipu3/algorithms/awb.h @@ -42,7 +42,8 @@ public: void prepare(IPAContext &context, const uint32_t frame, IPAFrameContext &frameContext, ipu3_uapi_params *params) override; - void process(IPAContext &context, IPAFrameContext &frameContext, + void process(IPAContext &context, const uint32_t frame, + IPAFrameContext &frameContext, const ipu3_uapi_stats_3a *stats) override; private: diff --git a/src/ipa/ipu3/algorithms/tone_mapping.cpp b/src/ipa/ipu3/algorithms/tone_mapping.cpp index 10a2fb1935dc..eac3d4064443 100644 --- a/src/ipa/ipu3/algorithms/tone_mapping.cpp +++ b/src/ipa/ipu3/algorithms/tone_mapping.cpp @@ -76,13 +76,15 @@ void ToneMapping::prepare([[maybe_unused]] IPAContext &context, /** * \brief Calculate the tone mapping look up table * \param[in] context The shared IPA context + * \param[in] frame The current frame sequence number * \param[in] frameContext The current frame context * \param[in] stats The IPU3 statistics and ISP results * * The tone mapping look up table is generated as an inverse power curve from * our gamma setting. */ -void ToneMapping::process(IPAContext &context, [[maybe_unused]] IPAFrameContext &frameContext, +void ToneMapping::process(IPAContext &context, [[maybe_unused]] const uint32_t frame, + [[maybe_unused]] IPAFrameContext &frameContext, [[maybe_unused]] const ipu3_uapi_stats_3a *stats) { /* diff --git a/src/ipa/ipu3/algorithms/tone_mapping.h b/src/ipa/ipu3/algorithms/tone_mapping.h index c24a7e485aaf..822e5168df82 100644 --- a/src/ipa/ipu3/algorithms/tone_mapping.h +++ b/src/ipa/ipu3/algorithms/tone_mapping.h @@ -21,7 +21,8 @@ public: int configure(IPAContext &context, const IPAConfigInfo &configInfo) override; void prepare(IPAContext &context, const uint32_t frame, IPAFrameContext &frameContext, ipu3_uapi_params *params) override; - void process(IPAContext &context, IPAFrameContext &frameContext, + void process(IPAContext &context, const uint32_t frame, + IPAFrameContext &frameContext, const ipu3_uapi_stats_3a *stats) override; private: diff --git a/src/ipa/ipu3/ipu3.cpp b/src/ipa/ipu3/ipu3.cpp index ac6f66afecb5..e5a763fd2b08 100644 --- a/src/ipa/ipu3/ipu3.cpp +++ b/src/ipa/ipu3/ipu3.cpp @@ -618,7 +618,7 @@ void IPAIPU3::processStatsBuffer(const uint32_t frame, ControlList ctrls(controls::controls); for (auto const &algo : algorithms()) - algo->process(context_, frameContext, stats); + algo->process(context_, frame, frameContext, stats); setControls(frame); diff --git a/src/ipa/libipa/algorithm.cpp b/src/ipa/libipa/algorithm.cpp index 0992e5de8dd5..30eab67f71fc 100644 --- a/src/ipa/libipa/algorithm.cpp +++ b/src/ipa/libipa/algorithm.cpp @@ -103,6 +103,7 @@ namespace ipa { * \fn Algorithm::process() * \brief Process ISP statistics, and run algorithm operations * \param[in] context The shared IPA context + * \param[in] frame The frame context sequence number * \param[in] frameContext The current frame's context * \param[in] stats The IPA statistics and ISP results * diff --git a/src/ipa/libipa/algorithm.h b/src/ipa/libipa/algorithm.h index cde2e6399e8e..ee93d260ca3f 100644 --- a/src/ipa/libipa/algorithm.h +++ b/src/ipa/libipa/algorithm.h @@ -52,6 +52,7 @@ public: } virtual void process([[maybe_unused]] typename Module::Context &context, + [[maybe_unused]] const uint32_t frame, [[maybe_unused]] typename Module::FrameContext &frameContext, [[maybe_unused]] const typename Module::Stats *stats) { diff --git a/src/ipa/rkisp1/algorithms/agc.cpp b/src/ipa/rkisp1/algorithms/agc.cpp index 0d22d9dc98fb..4c246d71a739 100644 --- a/src/ipa/rkisp1/algorithms/agc.cpp +++ b/src/ipa/rkisp1/algorithms/agc.cpp @@ -281,7 +281,7 @@ double Agc::measureBrightness(const rkisp1_cif_isp_hist_stat *hist) const * Identify the current image brightness, and use that to estimate the optimal * new exposure and gain for the scene. */ -void Agc::process(IPAContext &context, +void Agc::process(IPAContext &context, [[maybe_unused]] const uint32_t frame, [[maybe_unused]] IPAFrameContext &frameContext, const rkisp1_stat_buffer *stats) { diff --git a/src/ipa/rkisp1/algorithms/agc.h b/src/ipa/rkisp1/algorithms/agc.h index 8724cb602158..f115ba2ed85c 100644 --- a/src/ipa/rkisp1/algorithms/agc.h +++ b/src/ipa/rkisp1/algorithms/agc.h @@ -29,7 +29,8 @@ public: void prepare(IPAContext &context, const uint32_t frame, IPAFrameContext &frameContext, rkisp1_params_cfg *params) override; - void process(IPAContext &context, IPAFrameContext &frameContext, + void process(IPAContext &context, const uint32_t frame, + IPAFrameContext &frameContext, const rkisp1_stat_buffer *stats) override; private: diff --git a/src/ipa/rkisp1/algorithms/awb.cpp b/src/ipa/rkisp1/algorithms/awb.cpp index da22a0970a4f..aefba6382807 100644 --- a/src/ipa/rkisp1/algorithms/awb.cpp +++ b/src/ipa/rkisp1/algorithms/awb.cpp @@ -155,6 +155,7 @@ void Awb::queueRequest(IPAContext &context, * \copydoc libcamera::ipa::Algorithm::process */ void Awb::process([[maybe_unused]] IPAContext &context, + [[maybe_unused]] const uint32_t frame, [[maybe_unused]] IPAFrameContext &frameCtx, const rkisp1_stat_buffer *stats) { diff --git a/src/ipa/rkisp1/algorithms/awb.h b/src/ipa/rkisp1/algorithms/awb.h index 7ae6c7a9bb10..806e11a2ec93 100644 --- a/src/ipa/rkisp1/algorithms/awb.h +++ b/src/ipa/rkisp1/algorithms/awb.h @@ -25,7 +25,8 @@ public: rkisp1_params_cfg *params) override; void queueRequest(IPAContext &context, const uint32_t frame, const ControlList &controls) override; - void process(IPAContext &context, IPAFrameContext &frameCtx, + void process(IPAContext &context, const uint32_t frame, + IPAFrameContext &frameCtx, const rkisp1_stat_buffer *stats) override; private: diff --git a/src/ipa/rkisp1/rkisp1.cpp b/src/ipa/rkisp1/rkisp1.cpp index 0932c73b14e4..b2593a584a1a 100644 --- a/src/ipa/rkisp1/rkisp1.cpp +++ b/src/ipa/rkisp1/rkisp1.cpp @@ -333,7 +333,7 @@ void IPARkISP1::processStatsBuffer(const uint32_t frame, const uint32_t bufferId IPAFrameContext frameContext; for (auto const &algo : algorithms()) - algo->process(context_, frameContext, stats); + algo->process(context_, frame, frameContext, stats); setControls(frame); From patchwork Thu Sep 8 01:41:36 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Laurent Pinchart X-Patchwork-Id: 17317 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 6F931C3272 for ; Thu, 8 Sep 2022 01:42:31 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 22818620B2; Thu, 8 Sep 2022 03:42:31 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1662601351; bh=2SBmxcjPXeieBOv7B6wLXDN09vgXOa2LSsP84CgBLjc=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To: From; b=upGitaWlVQBUYi9Hc2brjM0diHvZ8f4Jxa4kc4loIMvgoxF6wi8bhC/tdXIabkbNO 4aQrgBM+SJiNbXPVeHtjEuc3CwUFmUH3n8DzkBYFsXJf/OzuGdUX5rQ3AYHJ4a78O+ ehm1PduVC/hJoXVjvF5JXVESNmdcgj64eJWQfc7iLRILPzCNRaIn/VgT7xhpWu/Mgr zemlkOyVG3szU1kSNeDnHFO5TnV/hGuAcjm22RYHQiCAixKssRXhu39x4SjZmZxTHq Wjtk93ZXrfCZtK0MW2Jyrb/OmegL2hycoNE8QBfEo1+M3bWQahr1Djly3RPCEa41hV LjZtII+XsyFVA== Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 0D140620AF for ; Thu, 8 Sep 2022 03:42:29 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="VP70YO5I"; dkim-atps=neutral Received: from pendragon.ideasonboard.com (62-78-145-57.bb.dnainternet.fi [62.78.145.57]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 89DE56CC; Thu, 8 Sep 2022 03:42:28 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1662601348; bh=2SBmxcjPXeieBOv7B6wLXDN09vgXOa2LSsP84CgBLjc=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=VP70YO5ItrlkNh08SLrxtF84/Ka6wsgXGgE9rp6cRTYRQzdpc05DZyykEsrKS1+QC t/SkDKfqlXJUmR7rfhS51yFF44r3ehFCn4Dcjad4bWhkP9PLY1KG0Q0vb3vpcqYLBh Mv8D1HZd/9ZmpmkFCGzmLKtb9UqsTcU+ambKOmS4= To: libcamera-devel@lists.libcamera.org Date: Thu, 8 Sep 2022 04:41:36 +0300 Message-Id: <20220908014200.28728-9-laurent.pinchart@ideasonboard.com> X-Mailer: git-send-email 2.35.1 In-Reply-To: <20220908014200.28728-1-laurent.pinchart@ideasonboard.com> References: <20220908014200.28728-1-laurent.pinchart@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v4 08/32] ipa: libipa: algorithm: queueRequest(): Pass frame context X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Laurent Pinchart via libcamera-devel From: Laurent Pinchart Reply-To: Laurent Pinchart Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" From: Kieran Bingham IPA modules have access to incoming Request's controls list and need to store them in the frame context at queueRequest() time. As each algorithm is expected to have ownership of a certain set of controls it should have access to the frame context where they have been stored, and where the algorithm computation results can be stored to later populate metadata. Now that both the IPU3 and RkISP1 IPA modules comply with the same interface, propagate the queueRequest() call to algorithms in the IPU3 IPA module. Signed-off-by: Kieran Bingham Signed-off-by: Jacopo Mondi Reviewed-by: Umang Jain Reviewed-by: Kieran Bingham --- src/ipa/libipa/algorithm.cpp | 1 + src/ipa/libipa/algorithm.h | 1 + src/ipa/rkisp1/algorithms/awb.cpp | 1 + src/ipa/rkisp1/algorithms/awb.h | 1 + src/ipa/rkisp1/algorithms/cproc.cpp | 1 + src/ipa/rkisp1/algorithms/cproc.h | 1 + src/ipa/rkisp1/algorithms/dpf.cpp | 1 + src/ipa/rkisp1/algorithms/dpf.h | 1 + src/ipa/rkisp1/algorithms/filter.cpp | 1 + src/ipa/rkisp1/algorithms/filter.h | 1 + src/ipa/rkisp1/rkisp1.cpp | 5 ++++- 11 files changed, 14 insertions(+), 1 deletion(-) diff --git a/src/ipa/libipa/algorithm.cpp b/src/ipa/libipa/algorithm.cpp index 30eab67f71fc..c152b885aee1 100644 --- a/src/ipa/libipa/algorithm.cpp +++ b/src/ipa/libipa/algorithm.cpp @@ -88,6 +88,7 @@ namespace ipa { * \brief Provide control values to the algorithm * \param[in] context The shared IPA context * \param[in] frame The frame number to apply the control values + * \param[in] frameContext The current frame's context * \param[in] controls The list of user controls * * This function is called for each request queued to the camera. It provides diff --git a/src/ipa/libipa/algorithm.h b/src/ipa/libipa/algorithm.h index ee93d260ca3f..d8601f9ccaff 100644 --- a/src/ipa/libipa/algorithm.h +++ b/src/ipa/libipa/algorithm.h @@ -47,6 +47,7 @@ public: virtual void queueRequest([[maybe_unused]] typename Module::Context &context, [[maybe_unused]] const uint32_t frame, + [[maybe_unused]] typename Module::FrameContext &frameContext, [[maybe_unused]] const ControlList &controls) { } diff --git a/src/ipa/rkisp1/algorithms/awb.cpp b/src/ipa/rkisp1/algorithms/awb.cpp index aefba6382807..a23c32015454 100644 --- a/src/ipa/rkisp1/algorithms/awb.cpp +++ b/src/ipa/rkisp1/algorithms/awb.cpp @@ -128,6 +128,7 @@ void Awb::prepare(IPAContext &context, */ void Awb::queueRequest(IPAContext &context, [[maybe_unused]] const uint32_t frame, + [[maybe_unused]] IPAFrameContext &frameContext, const ControlList &controls) { auto &awb = context.frameContext.awb; diff --git a/src/ipa/rkisp1/algorithms/awb.h b/src/ipa/rkisp1/algorithms/awb.h index 806e11a2ec93..6d4a39467038 100644 --- a/src/ipa/rkisp1/algorithms/awb.h +++ b/src/ipa/rkisp1/algorithms/awb.h @@ -24,6 +24,7 @@ public: IPAFrameContext &frameContext, rkisp1_params_cfg *params) override; void queueRequest(IPAContext &context, const uint32_t frame, + IPAFrameContext &frameContext, const ControlList &controls) override; void process(IPAContext &context, const uint32_t frame, IPAFrameContext &frameCtx, diff --git a/src/ipa/rkisp1/algorithms/cproc.cpp b/src/ipa/rkisp1/algorithms/cproc.cpp index c0505c68765b..bc4a07073706 100644 --- a/src/ipa/rkisp1/algorithms/cproc.cpp +++ b/src/ipa/rkisp1/algorithms/cproc.cpp @@ -38,6 +38,7 @@ LOG_DEFINE_CATEGORY(RkISP1CProc) */ void ColorProcessing::queueRequest(IPAContext &context, [[maybe_unused]] const uint32_t frame, + [[maybe_unused]] IPAFrameContext &frameContext, const ControlList &controls) { auto &cproc = context.frameContext.cproc; diff --git a/src/ipa/rkisp1/algorithms/cproc.h b/src/ipa/rkisp1/algorithms/cproc.h index 02c3d9c95488..ba6e901ae110 100644 --- a/src/ipa/rkisp1/algorithms/cproc.h +++ b/src/ipa/rkisp1/algorithms/cproc.h @@ -22,6 +22,7 @@ public: ~ColorProcessing() = default; void queueRequest(IPAContext &context, const uint32_t frame, + IPAFrameContext &frameContext, const ControlList &controls) override; void prepare(IPAContext &context, const uint32_t frame, IPAFrameContext &frameContext, diff --git a/src/ipa/rkisp1/algorithms/dpf.cpp b/src/ipa/rkisp1/algorithms/dpf.cpp index 8286f83facc1..be024fc5fe90 100644 --- a/src/ipa/rkisp1/algorithms/dpf.cpp +++ b/src/ipa/rkisp1/algorithms/dpf.cpp @@ -176,6 +176,7 @@ int Dpf::init([[maybe_unused]] IPAContext &context, */ void Dpf::queueRequest(IPAContext &context, [[maybe_unused]] const uint32_t frame, + [[maybe_unused]] IPAFrameContext &frameContext, const ControlList &controls) { auto &dpf = context.frameContext.dpf; diff --git a/src/ipa/rkisp1/algorithms/dpf.h b/src/ipa/rkisp1/algorithms/dpf.h index 656d78dee67c..e232cad4ee87 100644 --- a/src/ipa/rkisp1/algorithms/dpf.h +++ b/src/ipa/rkisp1/algorithms/dpf.h @@ -23,6 +23,7 @@ public: int init(IPAContext &context, const YamlObject &tuningData) override; void queueRequest(IPAContext &context, const uint32_t frame, + IPAFrameContext &frameContext, const ControlList &controls) override; void prepare(IPAContext &context, const uint32_t frame, IPAFrameContext &frameContext, diff --git a/src/ipa/rkisp1/algorithms/filter.cpp b/src/ipa/rkisp1/algorithms/filter.cpp index 02562339d008..6aa5476aa25e 100644 --- a/src/ipa/rkisp1/algorithms/filter.cpp +++ b/src/ipa/rkisp1/algorithms/filter.cpp @@ -44,6 +44,7 @@ static constexpr uint32_t kFiltModeDefault = 0x000004f2; */ void Filter::queueRequest(IPAContext &context, [[maybe_unused]] const uint32_t frame, + [[maybe_unused]] IPAFrameContext &frameContext, const ControlList &controls) { auto &filter = context.frameContext.filter; diff --git a/src/ipa/rkisp1/algorithms/filter.h b/src/ipa/rkisp1/algorithms/filter.h index ac69657e2f0f..3fd882ea41dc 100644 --- a/src/ipa/rkisp1/algorithms/filter.h +++ b/src/ipa/rkisp1/algorithms/filter.h @@ -22,6 +22,7 @@ public: ~Filter() = default; void queueRequest(IPAContext &context, const uint32_t frame, + IPAFrameContext &frameContext, const ControlList &controls) override; void prepare(IPAContext &context, const uint32_t frame, IPAFrameContext &frameContext, diff --git a/src/ipa/rkisp1/rkisp1.cpp b/src/ipa/rkisp1/rkisp1.cpp index b2593a584a1a..07210d16b5b5 100644 --- a/src/ipa/rkisp1/rkisp1.cpp +++ b/src/ipa/rkisp1/rkisp1.cpp @@ -292,8 +292,11 @@ void IPARkISP1::unmapBuffers(const std::vector &ids) void IPARkISP1::queueRequest(const uint32_t frame, const ControlList &controls) { + /* \todo Obtain the frame context to pass to process from the FCQueue */ + IPAFrameContext frameContext; + for (auto const &algo : algorithms()) - algo->queueRequest(context_, frame, controls); + algo->queueRequest(context_, frame, frameContext, controls); } void IPARkISP1::fillParamsBuffer(const uint32_t frame, const uint32_t bufferId) From patchwork Thu Sep 8 01:41:37 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Laurent Pinchart X-Patchwork-Id: 17318 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id D213CC327E for ; Thu, 8 Sep 2022 01:42:31 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 882CF620AD; Thu, 8 Sep 2022 03:42:31 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1662601351; bh=9WeMQgLeNyxB5edUAp1qiDP6ehK4JQAlX4CfMPpflBU=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To: From; b=OPo1uo27F6coZhWBa+DS9xBs2SF/P5w8UCZR+EGZrB/3ggA6d59Reu0WiWbvu9fs/ dNfwmoK7DKVymkr14Rg+p/+TzRuFICKes5IYMkeoB3QVy0XUXPEQcP3miLsVWYnf1o OjTGLkrJn3RkJ6LYobQ4Xufjvj+4YojcbvhGLHCBXe1bmqmO2ENd/F53hdXxUGyUrv ySpWLe+XDscYdXSWmzqclI6x1tB5EFXzQrBF2UAssZ8MQJRJPSp5VtSvmDMc8aS0/B 0XkbZXMSo4tNlQwlVs3UCtbuKJq0AUZ9vcglFa763InxchkGxkbFZWhtwlLcM6EN8h pjtvamuHscVWg== Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 5E78C620B2 for ; Thu, 8 Sep 2022 03:42:30 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="Bb++AA9M"; dkim-atps=neutral Received: from pendragon.ideasonboard.com (62-78-145-57.bb.dnainternet.fi [62.78.145.57]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id DD83D888 for ; Thu, 8 Sep 2022 03:42:29 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1662601350; bh=9WeMQgLeNyxB5edUAp1qiDP6ehK4JQAlX4CfMPpflBU=; h=From:To:Subject:Date:In-Reply-To:References:From; b=Bb++AA9MvZq2OA+PsRV8Cw3uuAsYhiM0ppjkjrjNHkTZ63KPhioopoR+8ZxstM3vr An6/H5Er/uuLqac12uuKy1ZrTIivQgoV+OBTMLzr5sYjUIjHEjl69CWzofS1FXZfdT BUycEDlvQ0HpzsUKi5rNdQ+tfw8rhGPF0Eqtrwhw= To: libcamera-devel@lists.libcamera.org Date: Thu, 8 Sep 2022 04:41:37 +0300 Message-Id: <20220908014200.28728-10-laurent.pinchart@ideasonboard.com> X-Mailer: git-send-email 2.35.1 In-Reply-To: <20220908014200.28728-1-laurent.pinchart@ideasonboard.com> References: <20220908014200.28728-1-laurent.pinchart@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v4 09/32] ipa: ipu3: Use base FrameContext class X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Laurent Pinchart via libcamera-devel From: Laurent Pinchart Reply-To: Laurent Pinchart Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Inherit from the base FrameContext class in the IPU3 IPAFrameContext. This allows dropping the frame member, which is now stored in the base class. Signed-off-by: Laurent Pinchart Reviewed-by: Kieran Bingham Reviewed-by: Jacopo Mondi --- src/ipa/ipu3/ipa_context.cpp | 24 ++++-------------------- src/ipa/ipu3/ipa_context.h | 7 ++++--- src/ipa/ipu3/ipu3.cpp | 5 +---- 3 files changed, 9 insertions(+), 27 deletions(-) diff --git a/src/ipa/ipu3/ipa_context.cpp b/src/ipa/ipu3/ipa_context.cpp index 13cdb835ef7f..9cfca0db3a0d 100644 --- a/src/ipa/ipu3/ipa_context.cpp +++ b/src/ipa/ipu3/ipa_context.cpp @@ -35,22 +35,6 @@ namespace libcamera::ipa::ipu3 { * most recently computed by the IPA algorithms. */ -/** - * \struct IPAFrameContext - * \brief Context for a frame - * - * The frame context stores data specific to a single frame processed by the - * IPA. Each frame processed by the IPA has a context associated with it, - * accessible through the IPAContext structure. - * - * Fields in the frame context should reflect values and controls - * associated with the specific frame as requested by the application, and - * as configured by the hardware. Fields can be read by algorithms to - * determine if they should update any specific action for this frame, and - * finally to update the metadata control lists when the frame is fully - * completed. - */ - /** * \struct IPAContext * \brief Global IPA context data shared between all algorithms @@ -188,15 +172,15 @@ IPAFrameContext::IPAFrameContext() = default; /** * \brief Construct a IPAFrameContext instance */ -IPAFrameContext::IPAFrameContext(uint32_t id, const ControlList &reqControls) - : frame(id), frameControls(reqControls) +IPAFrameContext::IPAFrameContext(const ControlList &reqControls) + : frameControls(reqControls) { sensor = {}; } /** - * \var IPAFrameContext::frame - * \brief The frame number + * \struct IPAFrameContext + * \brief IPU3-specific FrameContext * * \var IPAFrameContext::frameControls * \brief Controls sent in by the application while queuing the request diff --git a/src/ipa/ipu3/ipa_context.h b/src/ipa/ipu3/ipa_context.h index 42e11141d3a1..e8fc42769075 100644 --- a/src/ipa/ipu3/ipa_context.h +++ b/src/ipa/ipu3/ipa_context.h @@ -17,6 +17,8 @@ #include #include +#include + namespace libcamera { namespace ipa::ipu3 { @@ -76,16 +78,15 @@ struct IPAActiveState { } toneMapping; }; -struct IPAFrameContext { +struct IPAFrameContext : public FrameContext { IPAFrameContext(); - IPAFrameContext(uint32_t id, const ControlList &reqControls); + IPAFrameContext(const ControlList &reqControls); struct { uint32_t exposure; double gain; } sensor; - uint32_t frame; ControlList frameControls; }; diff --git a/src/ipa/ipu3/ipu3.cpp b/src/ipa/ipu3/ipu3.cpp index e5a763fd2b08..b1b23fd8f927 100644 --- a/src/ipa/ipu3/ipu3.cpp +++ b/src/ipa/ipu3/ipu3.cpp @@ -607,9 +607,6 @@ void IPAIPU3::processStatsBuffer(const uint32_t frame, IPAFrameContext &frameContext = context_.frameContexts[frame % kMaxFrameContexts]; - if (frameContext.frame != frame) - LOG(IPAIPU3, Warning) << "Frame " << frame << " does not match its frame context"; - frameContext.sensor.exposure = sensorControls.get(V4L2_CID_EXPOSURE).get(); frameContext.sensor.gain = camHelper_->gain(sensorControls.get(V4L2_CID_ANALOGUE_GAIN).get()); @@ -654,7 +651,7 @@ void IPAIPU3::processStatsBuffer(const uint32_t frame, void IPAIPU3::queueRequest(const uint32_t frame, const ControlList &controls) { /* \todo Start processing for 'frame' based on 'controls'. */ - context_.frameContexts[frame % kMaxFrameContexts] = { frame, controls }; + context_.frameContexts[frame % kMaxFrameContexts] = { controls }; } /** From patchwork Thu Sep 8 01:41:38 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Laurent Pinchart X-Patchwork-Id: 17319 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 99E1DC3272 for ; Thu, 8 Sep 2022 01:42:34 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 4FFE0620BC; Thu, 8 Sep 2022 03:42:34 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1662601354; bh=twb0vBHWIbEjPFQZq4LX3sNNY9X05vblLMGHZXpuI5A=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To: From; b=hGLpYBK34TJKfvTsxqi0AEw9dc5BjSfjmdfLFYhK/+eIYb4KyjFNJDe1K1OyLCdTT bDCxztXBuUAt6dp1Wsn4xnCb3D4ApGx5moSXL1PYBluBZ5IZ8/5AaIlaA47PCoiqky 5y83TsFgHtvs0oc5s28chdmjymS8lK8OJ1155HUpaxGKGv8JDk1NcXKuvEzNPxL0vC 7WKNJ3lptQGuzcWW9jLYLwWrtoGnTi570it2gDMQe2umUtLwuwSfDZhso5qHJ+a2Vz LTRJAvXVt7sqVmFFCheP4rmgd7ign3xbK8Gqsog1CjT7RKx2jABk3noGXMU0xAMB7i b9bXp2QaROriw== Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id BD317620BC for ; Thu, 8 Sep 2022 03:42:31 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="R3qOErDX"; dkim-atps=neutral Received: from pendragon.ideasonboard.com (62-78-145-57.bb.dnainternet.fi [62.78.145.57]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 431526CC for ; Thu, 8 Sep 2022 03:42:31 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1662601351; bh=twb0vBHWIbEjPFQZq4LX3sNNY9X05vblLMGHZXpuI5A=; h=From:To:Subject:Date:In-Reply-To:References:From; b=R3qOErDXHFvsJ1udnFbM24MODwMyX2TEPCKayI3WRcUYW+utsCi4zM2XWujV6BWcJ bm3SYJ1Fr5KnbsfcdYHWSAdw+D6mkPGt3cr6gYs5eGgTZGAZ9GzccCcJEF+gyPyHpe ltkeynZDFPKp14fdybtY0qhySXoX7AGZAPX+xqUo= To: libcamera-devel@lists.libcamera.org Date: Thu, 8 Sep 2022 04:41:38 +0300 Message-Id: <20220908014200.28728-11-laurent.pinchart@ideasonboard.com> X-Mailer: git-send-email 2.35.1 In-Reply-To: <20220908014200.28728-1-laurent.pinchart@ideasonboard.com> References: <20220908014200.28728-1-laurent.pinchart@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v4 10/32] ipa: ipu3: Use the FCQueue X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Laurent Pinchart via libcamera-devel From: Laurent Pinchart Reply-To: Laurent Pinchart Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Replace the manual ring buffer implementation with the FCQueue class from libipa. Signed-off-by: Laurent Pinchart Reviewed-by: Kieran Bingham Reviewed-by: Jacopo Mondi --- src/ipa/ipu3/ipa_context.cpp | 14 -------------- src/ipa/ipu3/ipa_context.h | 10 +--------- src/ipa/ipu3/ipu3.cpp | 32 ++++++++++++++++++++++++-------- 3 files changed, 25 insertions(+), 31 deletions(-) diff --git a/src/ipa/ipu3/ipa_context.cpp b/src/ipa/ipu3/ipa_context.cpp index 9cfca0db3a0d..6904ccbbdf8b 100644 --- a/src/ipa/ipu3/ipa_context.cpp +++ b/src/ipa/ipu3/ipa_context.cpp @@ -164,20 +164,6 @@ namespace libcamera::ipa::ipu3 { * struct ipu3_uapi_gamma_corr_lut for further details. */ -/** - * \brief Default constructor for IPAFrameContext - */ -IPAFrameContext::IPAFrameContext() = default; - -/** - * \brief Construct a IPAFrameContext instance - */ -IPAFrameContext::IPAFrameContext(const ControlList &reqControls) - : frameControls(reqControls) -{ - sensor = {}; -} - /** * \struct IPAFrameContext * \brief IPU3-specific FrameContext diff --git a/src/ipa/ipu3/ipa_context.h b/src/ipa/ipu3/ipa_context.h index e8fc42769075..bfc0196e098a 100644 --- a/src/ipa/ipu3/ipa_context.h +++ b/src/ipa/ipu3/ipa_context.h @@ -8,8 +8,6 @@ #pragma once -#include - #include #include @@ -23,9 +21,6 @@ namespace libcamera { namespace ipa::ipu3 { -/* Maximum number of frame contexts to be held */ -static constexpr uint32_t kMaxFrameContexts = 16; - struct IPASessionConfiguration { struct { ipu3_uapi_grid_config bdsGrid; @@ -79,9 +74,6 @@ struct IPAActiveState { }; struct IPAFrameContext : public FrameContext { - IPAFrameContext(); - IPAFrameContext(const ControlList &reqControls); - struct { uint32_t exposure; double gain; @@ -94,7 +86,7 @@ struct IPAContext { IPASessionConfiguration configuration; IPAActiveState activeState; - std::array frameContexts; + FCQueue frameContexts; }; } /* namespace ipa::ipu3 */ diff --git a/src/ipa/ipu3/ipu3.cpp b/src/ipa/ipu3/ipu3.cpp index b1b23fd8f927..8158ca0883e8 100644 --- a/src/ipa/ipu3/ipu3.cpp +++ b/src/ipa/ipu3/ipu3.cpp @@ -40,6 +40,8 @@ #include "algorithms/tone_mapping.h" #include "libipa/camera_sensor_helper.h" +#include "ipa_context.h" + /* Minimum grid width, expressed as a number of cells */ static constexpr uint32_t kMinGridWidth = 16; /* Maximum grid width, expressed as a number of cells */ @@ -53,6 +55,9 @@ static constexpr uint32_t kMinCellSizeLog2 = 3; /* log2 of the maximum grid cell width and height, in pixels */ static constexpr uint32_t kMaxCellSizeLog2 = 6; +/* Maximum number of frame contexts to be held */ +static constexpr uint32_t kMaxFrameContexts = 16; + namespace libcamera { LOG_DEFINE_CATEGORY(IPAIPU3) @@ -135,6 +140,8 @@ namespace ipa::ipu3 { class IPAIPU3 : public IPAIPU3Interface, public Module { public: + IPAIPU3(); + int init(const IPASettings &settings, const IPACameraSensorInfo &sensorInfo, const ControlInfoMap &sensorControls, @@ -183,6 +190,11 @@ private: struct IPAContext context_; }; +IPAIPU3::IPAIPU3() + : context_({ {}, {}, { kMaxFrameContexts } }) +{ +} + std::string IPAIPU3::logPrefix() const { return "ipu3"; @@ -205,6 +217,11 @@ void IPAIPU3::updateSessionConfiguration(const ControlInfoMap &sensorControls) int32_t minGain = v4l2Gain.min().get(); int32_t maxGain = v4l2Gain.max().get(); + /* Clear the IPA context before the streaming session. */ + context_.configuration = {}; + context_.activeState = {}; + context_.frameContexts.clear(); + /* * When the AGC computes the new exposure values for a frame, it needs * to know the limits for shutter speed and analogue gain. @@ -382,6 +399,7 @@ int IPAIPU3::start() */ void IPAIPU3::stop() { + context_.frameContexts.clear(); } /** @@ -488,11 +506,6 @@ int IPAIPU3::configure(const IPAConfigInfo &configInfo, calculateBdsGrid(configInfo.bdsOutputSize); - /* Clean IPAActiveState at each reconfiguration. */ - context_.activeState = {}; - IPAFrameContext initFrameContext; - context_.frameContexts.fill(initFrameContext); - if (!validateSensorControls()) { LOG(IPAIPU3, Error) << "Sensor control validation failed."; return -EINVAL; @@ -572,7 +585,7 @@ void IPAIPU3::fillParamsBuffer(const uint32_t frame, const uint32_t bufferId) */ params->use = {}; - IPAFrameContext &frameContext = context_.frameContexts[frame % kMaxFrameContexts]; + IPAFrameContext &frameContext = context_.frameContexts.get(frame); for (auto const &algo : algorithms()) algo->prepare(context_, frame, frameContext, params); @@ -605,7 +618,7 @@ void IPAIPU3::processStatsBuffer(const uint32_t frame, const ipu3_uapi_stats_3a *stats = reinterpret_cast(mem.data()); - IPAFrameContext &frameContext = context_.frameContexts[frame % kMaxFrameContexts]; + IPAFrameContext &frameContext = context_.frameContexts.get(frame); frameContext.sensor.exposure = sensorControls.get(V4L2_CID_EXPOSURE).get(); frameContext.sensor.gain = camHelper_->gain(sensorControls.get(V4L2_CID_ANALOGUE_GAIN).get()); @@ -651,7 +664,10 @@ void IPAIPU3::processStatsBuffer(const uint32_t frame, void IPAIPU3::queueRequest(const uint32_t frame, const ControlList &controls) { /* \todo Start processing for 'frame' based on 'controls'. */ - context_.frameContexts[frame % kMaxFrameContexts] = { controls }; + IPAFrameContext &frameContext = context_.frameContexts.init(frame); + + /* \todo Implement queueRequest to each algorithm. */ + frameContext.frameControls = controls; } /** From patchwork Thu Sep 8 01:41:39 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Laurent Pinchart X-Patchwork-Id: 17320 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 1280BC327D for ; Thu, 8 Sep 2022 01:42:35 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id BBBC8620C3; Thu, 8 Sep 2022 03:42:34 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1662601354; bh=99rJq+scIF3Wd+NrFT3kX9FI6zV87UX/otE3vmV/Lgc=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To: From; b=cIaQt+TDVOJJoV+e4j7FEUJLS5cxTyjZiZa/pr/KpLdp7+l9Hu+f6Z06JubrKg3ME QeWxPvqscIXVelzlgNYhYrwwNZoujWe9Eb9JZ5UsE3iM0g7wpx1iwrIPskLfDLY/sO t7A4tv6FIso0oBd8jJ+kCBd4B0UNhH91r9HIORRYNUKBDEmgRilGd636a7tpMg9Cw/ oVeM5UQmvEKUEd0mhE5H9OBV42zIsEkCBhq+k9ZqjfSYbW79xJIf8Ax1+kZO4LsCGA ykVSFoQPZ6nQldIWKGlCG1Iirjb0+Ebx3Csy+hSO+vg/2Kcpfy23/ZaesHtbk3LOoV rqjPLHCgeQ5og== Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 41A91620B8 for ; Thu, 8 Sep 2022 03:42:33 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="coUehksp"; dkim-atps=neutral Received: from pendragon.ideasonboard.com (62-78-145-57.bb.dnainternet.fi [62.78.145.57]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id BFAB9888 for ; Thu, 8 Sep 2022 03:42:32 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1662601353; bh=99rJq+scIF3Wd+NrFT3kX9FI6zV87UX/otE3vmV/Lgc=; h=From:To:Subject:Date:In-Reply-To:References:From; b=coUehkspEFMiTkP6W9ZC6yxwsJ0A9khT35r48RLSoJOahnQ3uwUDz8EZObAmoi/iA NGy88G/J5zJ5ReTAOVHSpGXUgL+PM9CuxaKPD0YAdq6NMdzCinO6Uc+Fxwts0s+m92 lA5rP2GCmAV8oDTZ2exoeQfTUWlwChVHS5bxV6p4= To: libcamera-devel@lists.libcamera.org Date: Thu, 8 Sep 2022 04:41:39 +0300 Message-Id: <20220908014200.28728-12-laurent.pinchart@ideasonboard.com> X-Mailer: git-send-email 2.35.1 In-Reply-To: <20220908014200.28728-1-laurent.pinchart@ideasonboard.com> References: <20220908014200.28728-1-laurent.pinchart@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v4 11/32] ipa: ipu3: Pass controls to algorithm's queueRequest() handler X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Laurent Pinchart via libcamera-devel From: Laurent Pinchart Reply-To: Laurent Pinchart Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" The the Algorithm::queueRequest() function of all algorithms when a request is queue, to pass the request controls to the algorithms. We can now drop the copy of the control list stored in IPAFrameContext as it isn't used anymore. Signed-off-by: Laurent Pinchart Reviewed-by: Kieran Bingham --- src/ipa/ipu3/ipa_context.cpp | 3 --- src/ipa/ipu3/ipa_context.h | 3 --- src/ipa/ipu3/ipu3.cpp | 4 ++-- 3 files changed, 2 insertions(+), 8 deletions(-) diff --git a/src/ipa/ipu3/ipa_context.cpp b/src/ipa/ipu3/ipa_context.cpp index 6904ccbbdf8b..bd71b615365d 100644 --- a/src/ipa/ipu3/ipa_context.cpp +++ b/src/ipa/ipu3/ipa_context.cpp @@ -168,9 +168,6 @@ namespace libcamera::ipa::ipu3 { * \struct IPAFrameContext * \brief IPU3-specific FrameContext * - * \var IPAFrameContext::frameControls - * \brief Controls sent in by the application while queuing the request - * * \var IPAFrameContext::sensor * \brief Effective sensor values that were applied for the frame * diff --git a/src/ipa/ipu3/ipa_context.h b/src/ipa/ipu3/ipa_context.h index bfc0196e098a..36099353e9f2 100644 --- a/src/ipa/ipu3/ipa_context.h +++ b/src/ipa/ipu3/ipa_context.h @@ -12,7 +12,6 @@ #include -#include #include #include @@ -78,8 +77,6 @@ struct IPAFrameContext : public FrameContext { uint32_t exposure; double gain; } sensor; - - ControlList frameControls; }; struct IPAContext { diff --git a/src/ipa/ipu3/ipu3.cpp b/src/ipa/ipu3/ipu3.cpp index 8158ca0883e8..844ab6de03c7 100644 --- a/src/ipa/ipu3/ipu3.cpp +++ b/src/ipa/ipu3/ipu3.cpp @@ -666,8 +666,8 @@ void IPAIPU3::queueRequest(const uint32_t frame, const ControlList &controls) /* \todo Start processing for 'frame' based on 'controls'. */ IPAFrameContext &frameContext = context_.frameContexts.init(frame); - /* \todo Implement queueRequest to each algorithm. */ - frameContext.frameControls = controls; + for (auto const &algo : algorithms()) + algo->queueRequest(context_, frame, frameContext, controls); } /** From patchwork Thu Sep 8 01:41:40 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Laurent Pinchart X-Patchwork-Id: 17321 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 7AB6AC3272 for ; Thu, 8 Sep 2022 01:42:36 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 32F18620C5; Thu, 8 Sep 2022 03:42:36 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1662601356; bh=LbHoDg6Ka3CHlxAv9U+Xv8AKdMpJk+ndXS3/VWsa1oo=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To: From; b=zRa1ZHpe5vPIuMasO2bcVil8dsqA6XDG8A283GJGFcDjIGLhBIQsZ09n27v2byaGe OlhEOBFzYAWqmCq2XUkfAaFNJ/eyiklOLibmB5GXZzXFB5hSFM66BYxf3/GjZLYWrk wrZfKEA+2z2NawEEKn5QAR8CmfuV2ACkKLjZ/A6FXxtB/Vgl16J2zGn98RxrUpF55d cexwBVTBxIvEsoX0XZJDG3n/jewjFXfL86zYPt+BsT/KdkLdVETqP7NF5QEAEYmm9J ayOhpDd4nvSFtKMXcBXRBkHUGEd8pgdyZxmBS/MjqW6PRvxHUe2SYLUSGphJCRl8Fc Efjsfh5Y7LY4g== Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id AEEE2620BE for ; Thu, 8 Sep 2022 03:42:34 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="ujEAOXo+"; dkim-atps=neutral Received: from pendragon.ideasonboard.com (62-78-145-57.bb.dnainternet.fi [62.78.145.57]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 344BD6CC; Thu, 8 Sep 2022 03:42:34 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1662601354; bh=LbHoDg6Ka3CHlxAv9U+Xv8AKdMpJk+ndXS3/VWsa1oo=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=ujEAOXo+4PpvsAIT0+6BJUtqyudJrspJj7ynSlCTY5WGK5y4P2lKN1UQVyhnz2gZ/ C66SkydBIhP0YjXGq6AuVlpIe1fZCSrXk2+psSbbFhWJzFWNnQ4S6p2eNa+iTY4MXE yYuaI5SoHTUUzwMjZKXPHEBMCuMweQKI1hDeU3zY= To: libcamera-devel@lists.libcamera.org Date: Thu, 8 Sep 2022 04:41:40 +0300 Message-Id: <20220908014200.28728-13-laurent.pinchart@ideasonboard.com> X-Mailer: git-send-email 2.35.1 In-Reply-To: <20220908014200.28728-1-laurent.pinchart@ideasonboard.com> References: <20220908014200.28728-1-laurent.pinchart@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v4 12/32] ipa: rkisp1: Remove unused class member X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Laurent Pinchart via libcamera-devel From: Laurent Pinchart Reply-To: Laurent Pinchart Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" From: Jacopo Mondi The "autoExposure" class member is not used. Remove it. Signed-off-by: Jacopo Mondi Reviewed-by: Kieran Bingham --- src/ipa/rkisp1/rkisp1.cpp | 5 ----- 1 file changed, 5 deletions(-) diff --git a/src/ipa/rkisp1/rkisp1.cpp b/src/ipa/rkisp1/rkisp1.cpp index 07210d16b5b5..f7a17459b6c3 100644 --- a/src/ipa/rkisp1/rkisp1.cpp +++ b/src/ipa/rkisp1/rkisp1.cpp @@ -71,9 +71,6 @@ private: ControlInfoMap ctrls_; - /* Camera sensor controls. */ - bool autoExposure_; - /* revision-specific data */ rkisp1_cif_isp_version hwRevision_; unsigned int hwHistBinNMax_; @@ -216,8 +213,6 @@ int IPARkISP1::configure([[maybe_unused]] const IPACameraSensorInfo &info, return -EINVAL; } - autoExposure_ = true; - int32_t minExposure = itExp->second.min().get(); int32_t maxExposure = itExp->second.max().get(); From patchwork Thu Sep 8 01:41:41 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Laurent Pinchart X-Patchwork-Id: 17322 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id F018DC327D for ; Thu, 8 Sep 2022 01:42:37 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id A28DD620CA; Thu, 8 Sep 2022 03:42:37 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1662601357; bh=N7T7B+FpWNY9q84INn+Mg4WQ1LCrieqE2TwIP0e0YQc=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To: From; b=hOfQA0AVBA3R2NSpidgFdQpBNfg08oaYado5uSojQWPalqroBHs9+RCo2aoEIO5SC 7WD67hZRvTlO0IvLkz0FR/DWiuSFxje6wtJhN4ANUSQtjoYd1mXB+Az1bIgg/Rfbnx v4tttNj3adAqX/kga5tcebhmf2XEGozBLJk8biOwMRYIA0xjz+H28q7OPmidBCS2EB fG52Z9qbQ6+V0VHjQ0HNPIzEOUU0CFQplXaiGhBoA/zGkt7xJWX+bHKauBKbgD+Gfs fpiUFQHimxtR02VFvbIDOUokaLS2tvejPiL6G6HS51MEg9KzJK/Fi4lo7EghiLqSrA 0nbtG4r5bWl6g== Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 0B957620B7 for ; Thu, 8 Sep 2022 03:42:36 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="OuX/9ier"; dkim-atps=neutral Received: from pendragon.ideasonboard.com (62-78-145-57.bb.dnainternet.fi [62.78.145.57]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 860038F1 for ; Thu, 8 Sep 2022 03:42:35 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1662601355; bh=N7T7B+FpWNY9q84INn+Mg4WQ1LCrieqE2TwIP0e0YQc=; h=From:To:Subject:Date:In-Reply-To:References:From; b=OuX/9ierMSzahFfAPEMEsDFgyl9iugvbHc3LgEEfsGVACZhfJPxEFI/ZUDpsOxZ3z Qn5g2wmtQs3oqeHSnffjfm2wyFmQb4JTfM4q3yVp53MxFmrPbEJsd+NpY5HR/MZRV2 3KQbea5o9KyVy8gsE6c3FxEulhqE16C1mJ/CxWTc= To: libcamera-devel@lists.libcamera.org Date: Thu, 8 Sep 2022 04:41:41 +0300 Message-Id: <20220908014200.28728-14-laurent.pinchart@ideasonboard.com> X-Mailer: git-send-email 2.35.1 In-Reply-To: <20220908014200.28728-1-laurent.pinchart@ideasonboard.com> References: <20220908014200.28728-1-laurent.pinchart@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v4 13/32] ipa: rkisp1: Sort documentation of the IPA context X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Laurent Pinchart via libcamera-devel From: Laurent Pinchart Reply-To: Laurent Pinchart Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" The documentation of the IPA context structures is separate from the documentation of the structure members. Sort the documentation block to group members with their structure. Signed-off-by: Laurent Pinchart Reviewed-by: Kieran Bingham Reviewed-by: Jacopo Mondi --- src/ipa/rkisp1/ipa_context.cpp | 62 +++++++++++++++++----------------- 1 file changed, 31 insertions(+), 31 deletions(-) diff --git a/src/ipa/rkisp1/ipa_context.cpp b/src/ipa/rkisp1/ipa_context.cpp index f64af732260c..cc66bc70ac44 100644 --- a/src/ipa/rkisp1/ipa_context.cpp +++ b/src/ipa/rkisp1/ipa_context.cpp @@ -24,37 +24,6 @@ namespace libcamera::ipa::rkisp1 { * may also be updated in the start() operation. */ -/** - * \struct IPAFrameContext - * \brief Per-frame context for algorithms - * - * The frame context stores data specific to a single frame processed by the - * IPA. Each frame processed by the IPA has a context associated with it, - * accessible through the IPAContext structure. - * - * \todo Detail how to access contexts for a particular frame - * - * Each of the fields in the frame context belongs to either a specific - * algorithm, or to the top-level IPA module. A field may be read by any - * algorithm, but should only be written by its owner. - */ - -/** - * \struct IPAContext - * \brief Global IPA context data shared between all algorithms - * - * \var IPAContext::configuration - * \brief The IPA session configuration, immutable during the session - * - * \var IPAContext::frameContext - * \brief The frame context for the frame being processed - * - * \todo While the frame context is supposed to be per-frame, this - * single frame context stores data related to both the current frame - * and the previous frames, with fields being updated as the algorithms - * are run. This needs to be turned into real per-frame data storage. - */ - /** * \var IPASessionConfiguration::agc * \brief AGC parameters configuration of the IPA @@ -115,6 +84,21 @@ namespace libcamera::ipa::rkisp1 { * \brief Sensor output resolution */ +/** + * \struct IPAFrameContext + * \brief Per-frame context for algorithms + * + * The frame context stores data specific to a single frame processed by the + * IPA. Each frame processed by the IPA has a context associated with it, + * accessible through the IPAContext structure. + * + * \todo Detail how to access contexts for a particular frame + * + * Each of the fields in the frame context belongs to either a specific + * algorithm, or to the top-level IPA module. A field may be read by any + * algorithm, but should only be written by its owner. + */ + /** * \var IPAFrameContext::agc * \brief Context for the Automatic Gain Control algorithm @@ -216,4 +200,20 @@ namespace libcamera::ipa::rkisp1 { * Algorithm::prepare() function of all algorithms. */ +/** + * \struct IPAContext + * \brief Global IPA context data shared between all algorithms + * + * \var IPAContext::configuration + * \brief The IPA session configuration, immutable during the session + * + * \var IPAContext::frameContext + * \brief The frame context for the frame being processed + * + * \todo While the frame context is supposed to be per-frame, this + * single frame context stores data related to both the current frame + * and the previous frames, with fields being updated as the algorithms + * are run. This needs to be turned into real per-frame data storage. + */ + } /* namespace libcamera::ipa::rkisp1 */ From patchwork Thu Sep 8 01:41:42 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Laurent Pinchart X-Patchwork-Id: 17323 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id A0056C3272 for ; Thu, 8 Sep 2022 01:42:39 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 4C4BE620BF; Thu, 8 Sep 2022 03:42:39 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1662601359; bh=NNP1nHInqW9nN1yI4ryt1Q07Vy/XAlUalSCnCLKcU+4=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To: From; b=s5ocolsWQwBg678bzSZakLBK7zfvTeVCAqCqR7u7EVxRqKgwZuICDkqf8F7dLebro ASETiWHqc/0g6XQnYcScG75cLDuhmn5/GXEnQMbWkE59ViXhKM8qaYyryQDxgw8y6Q xTY5dt75K1WB7eArjukeUhxuc8oO46/C9ZXn/KbtRuANk2Rin+WfJ4K8JuQKFwJ6ef 3u+PIc7ns3CR+/HtfOClgcVR4wHFNExMcIMoZyESOuD/SMUVQjIJTaADMk9d7mBWN3 OHTFq2IhbEoa9minZdralV8OfEvdJJfkmipo9tWev/MM7b60SOZzoCfgsDWdVlbdmo 9am7UYIdHZUVw== Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 92A49620C9 for ; Thu, 8 Sep 2022 03:42:37 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="uoS87SLP"; dkim-atps=neutral Received: from pendragon.ideasonboard.com (62-78-145-57.bb.dnainternet.fi [62.78.145.57]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 2754A888; Thu, 8 Sep 2022 03:42:37 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1662601357; bh=NNP1nHInqW9nN1yI4ryt1Q07Vy/XAlUalSCnCLKcU+4=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=uoS87SLPr5f4Z65r34Czc+IzoqXmqgTqOIyTFLoDff08j9HAQDGblcDuhaEObXkAu Kd/us70u2CbwkFl7SYN/xSfEEu6de+ZVMJBaW/YjAzRzFf4YVenrkIwZoWlbfQwrqk 8fd6uHMZJ2G8sVsJeNZhZjMTsSSmsRN2FD2F//4Y= To: libcamera-devel@lists.libcamera.org Date: Thu, 8 Sep 2022 04:41:42 +0300 Message-Id: <20220908014200.28728-15-laurent.pinchart@ideasonboard.com> X-Mailer: git-send-email 2.35.1 In-Reply-To: <20220908014200.28728-1-laurent.pinchart@ideasonboard.com> References: <20220908014200.28728-1-laurent.pinchart@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v4 14/32] ipa: rkisp1: Rename frameContext to activeState X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Laurent Pinchart via libcamera-devel From: Laurent Pinchart Reply-To: Laurent Pinchart Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" From: Kieran Bingham The RkISP1 IPA module creates a single instance of its IPAFrameContext structure, effectively using it more as an active state than a per-frame context. To prepare for the introduction of a real per-frame context, move all the members of the IPAFrameContext structure to a new IPAActiveState structure. The IPAFrameContext becomes effectively unused at runtime, and will be populated back with per-frame data after converting the RkISP1 IPA module to using a frame context queue. The IPAActiveState structure will slowly morph into a different entity as individual algorithm get later ported to the frame context API. While at it, fix a typo in the documentation of the Agc::computeExposure() function that incorrectly refers to the frame context instead of the global context. Signed-off-by: Kieran Bingham Reviewed-by: Umang Jain Reviewed-by: Jacopo Mondi Signed-off-by: Jacopo Mondi Reviewed-by: Kieran Bingham Reviewed-by: Jacopo Mondi --- Changes since v3: - Rewrite commit message --- src/ipa/rkisp1/algorithms/agc.cpp | 23 ++++--- src/ipa/rkisp1/algorithms/awb.cpp | 40 ++++++------ src/ipa/rkisp1/algorithms/blc.cpp | 2 +- src/ipa/rkisp1/algorithms/cproc.cpp | 4 +- src/ipa/rkisp1/algorithms/dpcc.cpp | 2 +- src/ipa/rkisp1/algorithms/dpf.cpp | 6 +- src/ipa/rkisp1/algorithms/filter.cpp | 4 +- src/ipa/rkisp1/algorithms/gsl.cpp | 2 +- src/ipa/rkisp1/algorithms/lsc.cpp | 2 +- src/ipa/rkisp1/ipa_context.cpp | 94 +++++++++++++++------------- src/ipa/rkisp1/ipa_context.h | 7 ++- src/ipa/rkisp1/rkisp1.cpp | 12 ++-- 12 files changed, 105 insertions(+), 93 deletions(-) diff --git a/src/ipa/rkisp1/algorithms/agc.cpp b/src/ipa/rkisp1/algorithms/agc.cpp index 4c246d71a739..606f13e776a3 100644 --- a/src/ipa/rkisp1/algorithms/agc.cpp +++ b/src/ipa/rkisp1/algorithms/agc.cpp @@ -73,8 +73,8 @@ Agc::Agc() int Agc::configure(IPAContext &context, const IPACameraSensorInfo &configInfo) { /* Configure the default exposure and gain. */ - context.frameContext.agc.gain = std::max(context.configuration.agc.minAnalogueGain, kMinAnalogueGain); - context.frameContext.agc.exposure = 10ms / context.configuration.sensor.lineDuration; + context.activeState.agc.gain = std::max(context.configuration.agc.minAnalogueGain, kMinAnalogueGain); + context.activeState.agc.exposure = 10ms / context.configuration.sensor.lineDuration; /* * According to the RkISP1 documentation: @@ -98,7 +98,10 @@ int Agc::configure(IPAContext &context, const IPACameraSensorInfo &configInfo) context.configuration.agc.measureWindow.h_size = 3 * configInfo.outputSize.width / 4; context.configuration.agc.measureWindow.v_size = 3 * configInfo.outputSize.height / 4; - /* \todo Use actual frame index by populating it in the frameContext. */ + /* + * \todo Use the upcoming per-frame context API that will provide a + * frame index + */ frameCount_ = 0; return 0; } @@ -140,18 +143,18 @@ utils::Duration Agc::filterExposure(utils::Duration exposureValue) /** * \brief Estimate the new exposure and gain values - * \param[inout] frameContext The shared IPA frame Context + * \param[inout] context The shared IPA Context * \param[in] yGain The gain calculated on the current brightness level * \param[in] iqMeanGain The gain calculated based on the relative luminance target */ void Agc::computeExposure(IPAContext &context, double yGain, double iqMeanGain) { IPASessionConfiguration &configuration = context.configuration; - IPAFrameContext &frameContext = context.frameContext; + IPAActiveState &activeState = context.activeState; /* Get the effective exposure and gain applied on the sensor. */ - uint32_t exposure = frameContext.sensor.exposure; - double analogueGain = frameContext.sensor.gain; + uint32_t exposure = activeState.sensor.exposure; + double analogueGain = activeState.sensor.gain; /* Use the highest of the two gain estimates. */ double evGain = std::max(yGain, iqMeanGain); @@ -216,8 +219,8 @@ void Agc::computeExposure(IPAContext &context, double yGain, double iqMeanGain) << stepGain; /* Update the estimated exposure and gain. */ - frameContext.agc.exposure = shutterTime / configuration.sensor.lineDuration; - frameContext.agc.gain = stepGain; + activeState.agc.exposure = shutterTime / configuration.sensor.lineDuration; + activeState.agc.gain = stepGain; } /** @@ -328,7 +331,7 @@ void Agc::prepare(IPAContext &context, [[maybe_unused]] IPAFrameContext &frameContext, rkisp1_params_cfg *params) { - if (context.frameContext.frameCount > 0) + if (context.activeState.frameCount > 0) return; /* Configure the measurement window. */ diff --git a/src/ipa/rkisp1/algorithms/awb.cpp b/src/ipa/rkisp1/algorithms/awb.cpp index a23c32015454..2bd9ef779bc3 100644 --- a/src/ipa/rkisp1/algorithms/awb.cpp +++ b/src/ipa/rkisp1/algorithms/awb.cpp @@ -36,10 +36,10 @@ LOG_DEFINE_CATEGORY(RkISP1Awb) int Awb::configure(IPAContext &context, const IPACameraSensorInfo &configInfo) { - context.frameContext.awb.gains.red = 1.0; - context.frameContext.awb.gains.blue = 1.0; - context.frameContext.awb.gains.green = 1.0; - context.frameContext.awb.autoEnabled = true; + context.activeState.awb.gains.red = 1.0; + context.activeState.awb.gains.blue = 1.0; + context.activeState.awb.gains.green = 1.0; + context.activeState.awb.autoEnabled = true; /* * Define the measurement window for AWB as a centered rectangle @@ -79,16 +79,16 @@ void Awb::prepare(IPAContext &context, [[maybe_unused]] IPAFrameContext &frameContext, rkisp1_params_cfg *params) { - params->others.awb_gain_config.gain_green_b = 256 * context.frameContext.awb.gains.green; - params->others.awb_gain_config.gain_blue = 256 * context.frameContext.awb.gains.blue; - params->others.awb_gain_config.gain_red = 256 * context.frameContext.awb.gains.red; - params->others.awb_gain_config.gain_green_r = 256 * context.frameContext.awb.gains.green; + params->others.awb_gain_config.gain_green_b = 256 * context.activeState.awb.gains.green; + params->others.awb_gain_config.gain_blue = 256 * context.activeState.awb.gains.blue; + params->others.awb_gain_config.gain_red = 256 * context.activeState.awb.gains.red; + params->others.awb_gain_config.gain_green_r = 256 * context.activeState.awb.gains.green; /* Update the gains. */ params->module_cfg_update |= RKISP1_CIF_ISP_MODULE_AWB_GAIN; /* If we already have configured the gains and window, return. */ - if (context.frameContext.frameCount > 0) + if (context.activeState.frameCount > 0) return; /* Configure the gains to apply. */ @@ -131,7 +131,7 @@ void Awb::queueRequest(IPAContext &context, [[maybe_unused]] IPAFrameContext &frameContext, const ControlList &controls) { - auto &awb = context.frameContext.awb; + auto &awb = context.activeState.awb; const auto &awbEnable = controls.get(controls::AwbEnable); if (awbEnable && *awbEnable != awb.autoEnabled) { @@ -162,7 +162,7 @@ void Awb::process([[maybe_unused]] IPAContext &context, { const rkisp1_cif_isp_stat *params = &stats->params; const rkisp1_cif_isp_awb_stat *awb = ¶ms->awb; - IPAFrameContext &frameContext = context.frameContext; + IPAActiveState &activeState = context.activeState; /* Get the YCbCr mean values */ double yMean = awb->awb_mean[0].mean_y_or_g; @@ -194,24 +194,24 @@ void Awb::process([[maybe_unused]] IPAContext &context, /* Filter the values to avoid oscillations. */ double speed = 0.2; - redGain = speed * redGain + (1 - speed) * frameContext.awb.gains.red; - blueGain = speed * blueGain + (1 - speed) * frameContext.awb.gains.blue; + redGain = speed * redGain + (1 - speed) * activeState.awb.gains.red; + blueGain = speed * blueGain + (1 - speed) * activeState.awb.gains.blue; /* * Gain values are unsigned integer value, range 0 to 4 with 8 bit * fractional part. */ - if (frameContext.awb.autoEnabled) { - frameContext.awb.gains.red = std::clamp(redGain, 0.0, 1023.0 / 256); - frameContext.awb.gains.blue = std::clamp(blueGain, 0.0, 1023.0 / 256); + if (activeState.awb.autoEnabled) { + activeState.awb.gains.red = std::clamp(redGain, 0.0, 1023.0 / 256); + activeState.awb.gains.blue = std::clamp(blueGain, 0.0, 1023.0 / 256); } /* Hardcode the green gain to 1.0. */ - frameContext.awb.gains.green = 1.0; + activeState.awb.gains.green = 1.0; - frameContext.awb.temperatureK = estimateCCT(redMean, greenMean, blueMean); + activeState.awb.temperatureK = estimateCCT(redMean, greenMean, blueMean); - LOG(RkISP1Awb, Debug) << "Gain found for red: " << context.frameContext.awb.gains.red - << " and for blue: " << context.frameContext.awb.gains.blue; + LOG(RkISP1Awb, Debug) << "Gain found for red: " << context.activeState.awb.gains.red + << " and for blue: " << context.activeState.awb.gains.blue; } REGISTER_IPA_ALGORITHM(Awb, "Awb") diff --git a/src/ipa/rkisp1/algorithms/blc.cpp b/src/ipa/rkisp1/algorithms/blc.cpp index c45a317ada08..0f7226cf217d 100644 --- a/src/ipa/rkisp1/algorithms/blc.cpp +++ b/src/ipa/rkisp1/algorithms/blc.cpp @@ -70,7 +70,7 @@ void BlackLevelCorrection::prepare(IPAContext &context, [[maybe_unused]] IPAFrameContext &frameContext, rkisp1_params_cfg *params) { - if (context.frameContext.frameCount > 0) + if (context.activeState.frameCount > 0) return; if (!tuningParameters_) diff --git a/src/ipa/rkisp1/algorithms/cproc.cpp b/src/ipa/rkisp1/algorithms/cproc.cpp index bc4a07073706..ea819b2acfcb 100644 --- a/src/ipa/rkisp1/algorithms/cproc.cpp +++ b/src/ipa/rkisp1/algorithms/cproc.cpp @@ -41,7 +41,7 @@ void ColorProcessing::queueRequest(IPAContext &context, [[maybe_unused]] IPAFrameContext &frameContext, const ControlList &controls) { - auto &cproc = context.frameContext.cproc; + auto &cproc = context.activeState.cproc; const auto &brightness = controls.get(controls::Brightness); if (brightness) { @@ -76,7 +76,7 @@ void ColorProcessing::prepare(IPAContext &context, [[maybe_unused]] IPAFrameContext &frameContext, rkisp1_params_cfg *params) { - auto &cproc = context.frameContext.cproc; + auto &cproc = context.activeState.cproc; /* Check if the algorithm configuration has been updated. */ if (!cproc.updateParams) diff --git a/src/ipa/rkisp1/algorithms/dpcc.cpp b/src/ipa/rkisp1/algorithms/dpcc.cpp index 5030526769c7..7c14ace0abee 100644 --- a/src/ipa/rkisp1/algorithms/dpcc.cpp +++ b/src/ipa/rkisp1/algorithms/dpcc.cpp @@ -263,7 +263,7 @@ void DefectPixelClusterCorrection::prepare(IPAContext &context, [[maybe_unused]] IPAFrameContext &frameContext, rkisp1_params_cfg *params) { - if (context.frameContext.frameCount > 0) + if (context.activeState.frameCount > 0) return; if (!initialized_) diff --git a/src/ipa/rkisp1/algorithms/dpf.cpp b/src/ipa/rkisp1/algorithms/dpf.cpp index be024fc5fe90..b8c837c21243 100644 --- a/src/ipa/rkisp1/algorithms/dpf.cpp +++ b/src/ipa/rkisp1/algorithms/dpf.cpp @@ -179,7 +179,7 @@ void Dpf::queueRequest(IPAContext &context, [[maybe_unused]] IPAFrameContext &frameContext, const ControlList &controls) { - auto &dpf = context.frameContext.dpf; + auto &dpf = context.activeState.dpf; const auto &denoise = controls.get(controls::draft::NoiseReductionMode); if (denoise) { @@ -214,9 +214,9 @@ void Dpf::prepare(IPAContext &context, [[maybe_unused]] const uint32_t frame, if (!initialized_) return; - auto &dpf = context.frameContext.dpf; + auto &dpf = context.activeState.dpf; - if (context.frameContext.frameCount == 0) { + if (context.activeState.frameCount == 0) { params->others.dpf_config = config_; params->others.dpf_strength_config = strengthConfig_; diff --git a/src/ipa/rkisp1/algorithms/filter.cpp b/src/ipa/rkisp1/algorithms/filter.cpp index 6aa5476aa25e..837560eb20a6 100644 --- a/src/ipa/rkisp1/algorithms/filter.cpp +++ b/src/ipa/rkisp1/algorithms/filter.cpp @@ -47,7 +47,7 @@ void Filter::queueRequest(IPAContext &context, [[maybe_unused]] IPAFrameContext &frameContext, const ControlList &controls) { - auto &filter = context.frameContext.filter; + auto &filter = context.activeState.filter; const auto &sharpness = controls.get(controls::Sharpness); if (sharpness) { @@ -91,7 +91,7 @@ void Filter::prepare(IPAContext &context, [[maybe_unused]] IPAFrameContext &frameContext, rkisp1_params_cfg *params) { - auto &filter = context.frameContext.filter; + auto &filter = context.activeState.filter; /* Check if the algorithm configuration has been updated. */ if (!filter.updateParams) diff --git a/src/ipa/rkisp1/algorithms/gsl.cpp b/src/ipa/rkisp1/algorithms/gsl.cpp index dfc76b3d8ffd..879ca2973e8a 100644 --- a/src/ipa/rkisp1/algorithms/gsl.cpp +++ b/src/ipa/rkisp1/algorithms/gsl.cpp @@ -123,7 +123,7 @@ void GammaSensorLinearization::prepare(IPAContext &context, [[maybe_unused]] IPAFrameContext &frameContext, rkisp1_params_cfg *params) { - if (context.frameContext.frameCount > 0) + if (context.activeState.frameCount > 0) return; if (!initialized_) diff --git a/src/ipa/rkisp1/algorithms/lsc.cpp b/src/ipa/rkisp1/algorithms/lsc.cpp index e4b04136dbaf..9c717bc5f99e 100644 --- a/src/ipa/rkisp1/algorithms/lsc.cpp +++ b/src/ipa/rkisp1/algorithms/lsc.cpp @@ -137,7 +137,7 @@ void LensShadingCorrection::prepare(IPAContext &context, [[maybe_unused]] IPAFrameContext &frameContext, rkisp1_params_cfg *params) { - if (context.frameContext.frameCount > 0) + if (context.activeState.frameCount > 0) return; if (!initialized_) diff --git a/src/ipa/rkisp1/ipa_context.cpp b/src/ipa/rkisp1/ipa_context.cpp index cc66bc70ac44..d18f4996aad2 100644 --- a/src/ipa/rkisp1/ipa_context.cpp +++ b/src/ipa/rkisp1/ipa_context.cpp @@ -85,114 +85,115 @@ namespace libcamera::ipa::rkisp1 { */ /** - * \struct IPAFrameContext - * \brief Per-frame context for algorithms + * \struct IPAActiveState + * \brief Active state for algorithms * - * The frame context stores data specific to a single frame processed by the - * IPA. Each frame processed by the IPA has a context associated with it, - * accessible through the IPAContext structure. + * The active state stores algorithm-specific data that needs to be shared + * between multiple algorithms and the IPA module. It is accessible through the + * IPAContext structure. * - * \todo Detail how to access contexts for a particular frame + * \todo Split the data contained in this structure between the active state + * and the frame contexts. * - * Each of the fields in the frame context belongs to either a specific + * Each of the fields in the active state belongs to either a specific * algorithm, or to the top-level IPA module. A field may be read by any * algorithm, but should only be written by its owner. */ /** - * \var IPAFrameContext::agc - * \brief Context for the Automatic Gain Control algorithm + * \var IPAActiveState::agc + * \brief State for the Automatic Gain Control algorithm * * The exposure and gain determined are expected to be applied to the sensor * at the earliest opportunity. * - * \var IPAFrameContext::agc.exposure + * \var IPAActiveState::agc.exposure * \brief Exposure time expressed as a number of lines * - * \var IPAFrameContext::agc.gain + * \var IPAActiveState::agc.gain * \brief Analogue gain multiplier * * The gain should be adapted to the sensor specific gain code before applying. */ /** - * \var IPAFrameContext::awb - * \brief Context for the Automatic White Balance algorithm + * \var IPAActiveState::awb + * \brief State for the Automatic White Balance algorithm * - * \struct IPAFrameContext::awb.gains + * \struct IPAActiveState::awb.gains * \brief White balance gains * - * \var IPAFrameContext::awb.gains.red + * \var IPAActiveState::awb.gains.red * \brief White balance gain for R channel * - * \var IPAFrameContext::awb.gains.green + * \var IPAActiveState::awb.gains.green * \brief White balance gain for G channel * - * \var IPAFrameContext::awb.gains.blue + * \var IPAActiveState::awb.gains.blue * \brief White balance gain for B channel * - * \var IPAFrameContext::awb.temperatureK + * \var IPAActiveState::awb.temperatureK * \brief Estimated color temperature * - * \var IPAFrameContext::awb.autoEnabled + * \var IPAActiveState::awb.autoEnabled * \brief Whether the Auto White Balance algorithm is enabled */ /** - * \var IPAFrameContext::cproc - * \brief Context for the Color Processing algorithm + * \var IPAActiveState::cproc + * \brief State for the Color Processing algorithm * - * \struct IPAFrameContext::cproc.brightness + * \struct IPAActiveState::cproc.brightness * \brief Brightness level * - * \var IPAFrameContext::cproc.contrast + * \var IPAActiveState::cproc.contrast * \brief Contrast level * - * \var IPAFrameContext::cproc.saturation + * \var IPAActiveState::cproc.saturation * \brief Saturation level * - * \var IPAFrameContext::cproc.updateParams + * \var IPAActiveState::cproc.updateParams * \brief Indicates if ISP parameters need to be updated */ /** - * \var IPAFrameContext::dpf - * \brief Context for the Denoise Pre-Filter algorithm + * \var IPAActiveState::dpf + * \brief State for the Denoise Pre-Filter algorithm * - * \var IPAFrameContext::dpf.denoise + * \var IPAActiveState::dpf.denoise * \brief Indicates if denoise is activated * - * \var IPAFrameContext::dpf.updateParams + * \var IPAActiveState::dpf.updateParams * \brief Indicates if ISP parameters need to be updated */ /** - * \var IPAFrameContext::filter - * \brief Context for the Filter algorithm + * \var IPAActiveState::filter + * \brief State for the Filter algorithm * - * \struct IPAFrameContext::filter.denoise + * \struct IPAActiveState::filter.denoise * \brief Denoising level * - * \var IPAFrameContext::filter.sharpness + * \var IPAActiveState::filter.sharpness * \brief Sharpness level * - * \var IPAFrameContext::filter.updateParams + * \var IPAActiveState::filter.updateParams * \brief Indicates if ISP parameters need to be updated */ /** - * \var IPAFrameContext::sensor + * \var IPAActiveState::sensor * \brief Effective sensor values * - * \var IPAFrameContext::sensor.exposure + * \var IPAActiveState::sensor.exposure * \brief Exposure time expressed as a number of lines * - * \var IPAFrameContext::sensor.gain + * \var IPAActiveState::sensor.gain * \brief Analogue gain multiplier */ /** - * \var IPAFrameContext::frameCount + * \var IPAActiveState::frameCount * \brief Counter of requests queued to the IPA module * * The counter is reset to 0 when the IPA module is configured, and is @@ -200,6 +201,14 @@ namespace libcamera::ipa::rkisp1 { * Algorithm::prepare() function of all algorithms. */ +/** + * \struct IPAFrameContext + * \brief Per-frame context for algorithms + * + * This structure is currently unused and will be replaced by a real per-frame + * context. + */ + /** * \struct IPAContext * \brief Global IPA context data shared between all algorithms @@ -207,13 +216,10 @@ namespace libcamera::ipa::rkisp1 { * \var IPAContext::configuration * \brief The IPA session configuration, immutable during the session * - * \var IPAContext::frameContext - * \brief The frame context for the frame being processed + * \var IPAContext::activeState + * \brief The IPA active state, storing the latest state for all algorithms * - * \todo While the frame context is supposed to be per-frame, this - * single frame context stores data related to both the current frame - * and the previous frames, with fields being updated as the algorithms - * are run. This needs to be turned into real per-frame data storage. + * \todo Introduce per-frame contexts */ } /* namespace libcamera::ipa::rkisp1 */ diff --git a/src/ipa/rkisp1/ipa_context.h b/src/ipa/rkisp1/ipa_context.h index 854151f2ca75..5590482c6a8c 100644 --- a/src/ipa/rkisp1/ipa_context.h +++ b/src/ipa/rkisp1/ipa_context.h @@ -46,7 +46,7 @@ struct IPASessionConfiguration { } hw; }; -struct IPAFrameContext { +struct IPAActiveState { struct { uint32_t exposure; double gain; @@ -89,9 +89,12 @@ struct IPAFrameContext { unsigned int frameCount; }; +struct IPAFrameContext { +}; + struct IPAContext { IPASessionConfiguration configuration; - IPAFrameContext frameContext; + IPAActiveState activeState; }; } /* namespace ipa::rkisp1 */ diff --git a/src/ipa/rkisp1/rkisp1.cpp b/src/ipa/rkisp1/rkisp1.cpp index f7a17459b6c3..24d5b9647838 100644 --- a/src/ipa/rkisp1/rkisp1.cpp +++ b/src/ipa/rkisp1/rkisp1.cpp @@ -244,7 +244,7 @@ int IPARkISP1::configure([[maybe_unused]] const IPACameraSensorInfo &info, context_.configuration.agc.minAnalogueGain = camHelper_->gain(minGain); context_.configuration.agc.maxAnalogueGain = camHelper_->gain(maxGain); - context_.frameContext.frameCount = 0; + context_.activeState.frameCount = 0; for (auto const &algo : algorithms()) { int ret = algo->configure(context_, info); @@ -310,7 +310,7 @@ void IPARkISP1::fillParamsBuffer(const uint32_t frame, const uint32_t bufferId) algo->prepare(context_, frame, frameContext, params); paramsBufferReady.emit(frame); - context_.frameContext.frameCount++; + context_.activeState.frameCount++; } void IPARkISP1::processStatsBuffer(const uint32_t frame, const uint32_t bufferId, @@ -320,9 +320,9 @@ void IPARkISP1::processStatsBuffer(const uint32_t frame, const uint32_t bufferId reinterpret_cast( mappedBuffers_.at(bufferId).planes()[0].data()); - context_.frameContext.sensor.exposure = + context_.activeState.sensor.exposure = sensorControls.get(V4L2_CID_EXPOSURE).get(); - context_.frameContext.sensor.gain = + context_.activeState.sensor.gain = camHelper_->gain(sensorControls.get(V4L2_CID_ANALOGUE_GAIN).get()); unsigned int aeState = 0; @@ -340,8 +340,8 @@ void IPARkISP1::processStatsBuffer(const uint32_t frame, const uint32_t bufferId void IPARkISP1::setControls(unsigned int frame) { - uint32_t exposure = context_.frameContext.agc.exposure; - uint32_t gain = camHelper_->gainCode(context_.frameContext.agc.gain); + uint32_t exposure = context_.activeState.agc.exposure; + uint32_t gain = camHelper_->gainCode(context_.activeState.agc.gain); ControlList ctrls(ctrls_); ctrls.set(V4L2_CID_EXPOSURE, static_cast(exposure)); From patchwork Thu Sep 8 01:41:43 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Laurent Pinchart X-Patchwork-Id: 17324 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 0D7A6C3272 for ; Thu, 8 Sep 2022 01:42:41 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id BC0CD620D5; Thu, 8 Sep 2022 03:42:40 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1662601360; bh=8j/YBvvGgMJbJ53tc0F+C0YHVXUyQt69ZY96MUcTtMs=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To: From; b=FJo+qguEZidXieHEm+rxirpzG/7P78yekX/KOZ+joNkTqW0bvyKGo+734W1HVz+f7 h3pf5b9PjJ+FNluXF6IjStoGm4cs7PVs3f7IQOTK6c0o30eboYEl0JT7JIRxsOsGO/ c3BoVtjguCtptZfm+k0r2r3n1kTqCX6i9RfJ/mEaaY2k+l5hzSNih1oCW/2TmIUU3o mWbM09mZyMUF4fptSVHDsFc17+WMv5Xyd9RFjaw6lC6Mi7n5sW38Vm8tqdXMlXpYy3 Pn6w2FdYnVQf6vrvGm06uyrgG1HSTNrzcK4vxZElOfHLVYq6bf3hlk4BfLtSnyp9U8 fqGst8fUOGkIw== Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id DAADC620BE for ; Thu, 8 Sep 2022 03:42:38 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="JWynqFBG"; dkim-atps=neutral Received: from pendragon.ideasonboard.com (62-78-145-57.bb.dnainternet.fi [62.78.145.57]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 671D0888 for ; Thu, 8 Sep 2022 03:42:38 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1662601358; bh=8j/YBvvGgMJbJ53tc0F+C0YHVXUyQt69ZY96MUcTtMs=; h=From:To:Subject:Date:In-Reply-To:References:From; b=JWynqFBGsC5BszpQuMVGjlj+uhYpodBV4lxh7cbu/YihPppzJeUx75Eddd47sxuFs /F8CwSylwhPhPRBx6EKwV7PC3at32pdc11guklAWDqUQVffZyPOtDvXm7QA29RAs6o Fql1lNnw7ZjDhexljLoULshbOMz+kTGqyiIOSgmU= To: libcamera-devel@lists.libcamera.org Date: Thu, 8 Sep 2022 04:41:43 +0300 Message-Id: <20220908014200.28728-16-laurent.pinchart@ideasonboard.com> X-Mailer: git-send-email 2.35.1 In-Reply-To: <20220908014200.28728-1-laurent.pinchart@ideasonboard.com> References: <20220908014200.28728-1-laurent.pinchart@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v4 15/32] ipa: rkisp1: Use base FrameContext class X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Laurent Pinchart via libcamera-devel From: Laurent Pinchart Reply-To: Laurent Pinchart Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Inherit from the base FrameContext class in the RkISP1 IPAFrameContext. As the IPAFrameContext is currently unused, this change is a no-op, but it prepares the RkISP1 IPA module for frame context queue support. Signed-off-by: Laurent Pinchart Reviewed-by: Kieran Bingham Reviewed-by: Jacopo Mondi --- src/ipa/rkisp1/ipa_context.h | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/src/ipa/rkisp1/ipa_context.h b/src/ipa/rkisp1/ipa_context.h index 5590482c6a8c..f6b3e6eb951c 100644 --- a/src/ipa/rkisp1/ipa_context.h +++ b/src/ipa/rkisp1/ipa_context.h @@ -14,6 +14,8 @@ #include +#include + namespace libcamera { namespace ipa::rkisp1 { @@ -89,7 +91,7 @@ struct IPAActiveState { unsigned int frameCount; }; -struct IPAFrameContext { +struct IPAFrameContext : public FrameContext { }; struct IPAContext { From patchwork Thu Sep 8 01:41:44 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Laurent Pinchart X-Patchwork-Id: 17325 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 78085C3272 for ; Thu, 8 Sep 2022 01:42:42 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 2E0BB620D1; Thu, 8 Sep 2022 03:42:42 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1662601362; bh=e8ubAQjZZlGxHT/f4tPvWfGlTAbsi9q/Re1zIR8ZtZw=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To: From; b=1hAkBwbbHw4inbx8fsRpZIUsTKotQRMNkrQhkXvw4aSbWZ++rcqxUVNDENBlYRf2F EUmjBpG5xiaVPEnV59lEzugetgLnSV9D62lsf5ASXKD5iRjDNkU3iA3+/6pRVEQiy3 xclqIF+OT9sv/WTmRKsSiyW4fn5kX4Bem0xBbpHWt6YO+SMd9RNq/W8tphG85Z/9z1 hqPhS5eEd6/WFa4mo5R/ukhzAmL66L2Fe/cDtCM2Haw+BQKadY0YJORqmRx9mbFxtg YBKbaoXkhQBL+CHV5qqMoK67w1xvzcjy/DQyJ6x76nMokg3YcwAlm/zECSxtt6nU38 VlOU5rOT/Br8A== Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 57C57620C9 for ; Thu, 8 Sep 2022 03:42:40 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="metfnIXb"; dkim-atps=neutral Received: from pendragon.ideasonboard.com (62-78-145-57.bb.dnainternet.fi [62.78.145.57]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id D723E888; Thu, 8 Sep 2022 03:42:39 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1662601360; bh=e8ubAQjZZlGxHT/f4tPvWfGlTAbsi9q/Re1zIR8ZtZw=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=metfnIXb8wIRGccy0/SLkjNZLhQAiDBGZD6d4gocD0XcEyKJhDqMYWekMHTau/QPC s6Im5OK5lmAoKWj8zsF0qqJXJ7CwYtFvBHc4accoF561sjy/ev9qFEkmo67xlPYi8x Ss6nYxXJnAT65JBWrxRpDgiMJz/SE/PwnOH5iBsI= To: libcamera-devel@lists.libcamera.org Date: Thu, 8 Sep 2022 04:41:44 +0300 Message-Id: <20220908014200.28728-17-laurent.pinchart@ideasonboard.com> X-Mailer: git-send-email 2.35.1 In-Reply-To: <20220908014200.28728-1-laurent.pinchart@ideasonboard.com> References: <20220908014200.28728-1-laurent.pinchart@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v4 16/32] ipa: rkisp1: Convert to use the FCQueue X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Laurent Pinchart via libcamera-devel From: Laurent Pinchart Reply-To: Laurent Pinchart Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" From: Kieran Bingham Establish a queue of FrameContexts using the new FCQueue and use it to supply the FrameContext to the algorithms. The algorithms on the RKISP1 do not use this yet themselves, but are able to do so after the introduction of this patch. Signed-off-by: Kieran Bingham Signed-off-by: Jacopo Mondi Reviewed-by: Umang Jain Reviewed-by: Kieran Bingham Reviewed-by: Jacopo Mondi --- src/ipa/rkisp1/ipa_context.cpp | 6 +++--- src/ipa/rkisp1/ipa_context.h | 2 ++ src/ipa/rkisp1/rkisp1.cpp | 34 ++++++++++++++++++++++++---------- 3 files changed, 29 insertions(+), 13 deletions(-) diff --git a/src/ipa/rkisp1/ipa_context.cpp b/src/ipa/rkisp1/ipa_context.cpp index d18f4996aad2..e9846742ee4f 100644 --- a/src/ipa/rkisp1/ipa_context.cpp +++ b/src/ipa/rkisp1/ipa_context.cpp @@ -205,8 +205,7 @@ namespace libcamera::ipa::rkisp1 { * \struct IPAFrameContext * \brief Per-frame context for algorithms * - * This structure is currently unused and will be replaced by a real per-frame - * context. + * \todo Populate the frame context for all algorithms */ /** @@ -219,7 +218,8 @@ namespace libcamera::ipa::rkisp1 { * \var IPAContext::activeState * \brief The IPA active state, storing the latest state for all algorithms * - * \todo Introduce per-frame contexts + * \var IPAContext::frameContexts + * \brief Ring buffer of per-frame contexts */ } /* namespace libcamera::ipa::rkisp1 */ diff --git a/src/ipa/rkisp1/ipa_context.h b/src/ipa/rkisp1/ipa_context.h index f6b3e6eb951c..f6aaefffed52 100644 --- a/src/ipa/rkisp1/ipa_context.h +++ b/src/ipa/rkisp1/ipa_context.h @@ -97,6 +97,8 @@ struct IPAFrameContext : public FrameContext { struct IPAContext { IPASessionConfiguration configuration; IPAActiveState activeState; + + FCQueue frameContexts; }; } /* namespace ipa::rkisp1 */ diff --git a/src/ipa/rkisp1/rkisp1.cpp b/src/ipa/rkisp1/rkisp1.cpp index 24d5b9647838..c5ed0bb21f67 100644 --- a/src/ipa/rkisp1/rkisp1.cpp +++ b/src/ipa/rkisp1/rkisp1.cpp @@ -40,13 +40,18 @@ using namespace std::literals::chrono_literals; namespace ipa::rkisp1 { +/* Maximum number of frame contexts to be held */ +static constexpr uint32_t kMaxFrameContexts = 16; + class IPARkISP1 : public IPARkISP1Interface, public Module { public: + IPARkISP1(); + int init(const IPASettings &settings, unsigned int hwRevision, ControlInfoMap *ipaControls) override; int start() override; - void stop() override {} + void stop() override; int configure(const IPACameraSensorInfo &info, const std::map &streamConfig, @@ -100,6 +105,11 @@ const ControlInfoMap::Map rkisp1Controls{ } /* namespace */ +IPARkISP1::IPARkISP1() + : context_({ {}, {}, { kMaxFrameContexts } }) +{ +} + std::string IPARkISP1::logPrefix() const { return "rkisp1"; @@ -186,6 +196,11 @@ int IPARkISP1::start() return 0; } +void IPARkISP1::stop() +{ + context_.frameContexts.clear(); +} + /** * \todo The RkISP1 pipeline currently provides an empty IPACameraSensorInfo * if the connected sensor does not provide enough information to properly @@ -223,8 +238,10 @@ int IPARkISP1::configure([[maybe_unused]] const IPACameraSensorInfo &info, << "Exposure: " << minExposure << "-" << maxExposure << " Gain: " << minGain << "-" << maxGain; - /* Clean context at configuration */ - context_ = {}; + /* Clear the IPA context before the streaming session. */ + context_.configuration = {}; + context_.activeState = {}; + context_.frameContexts.clear(); /* Set the hardware revision for the algorithms. */ context_.configuration.hw.revision = hwRevision_; @@ -287,8 +304,7 @@ void IPARkISP1::unmapBuffers(const std::vector &ids) void IPARkISP1::queueRequest(const uint32_t frame, const ControlList &controls) { - /* \todo Obtain the frame context to pass to process from the FCQueue */ - IPAFrameContext frameContext; + IPAFrameContext &frameContext = context_.frameContexts.init(frame); for (auto const &algo : algorithms()) algo->queueRequest(context_, frame, frameContext, controls); @@ -296,8 +312,7 @@ void IPARkISP1::queueRequest(const uint32_t frame, const ControlList &controls) void IPARkISP1::fillParamsBuffer(const uint32_t frame, const uint32_t bufferId) { - /* \todo Obtain the frame context to pass to process from the FCQueue */ - IPAFrameContext frameContext; + IPAFrameContext &frameContext = context_.frameContexts.get(frame); rkisp1_params_cfg *params = reinterpret_cast( @@ -316,6 +331,8 @@ void IPARkISP1::fillParamsBuffer(const uint32_t frame, const uint32_t bufferId) void IPARkISP1::processStatsBuffer(const uint32_t frame, const uint32_t bufferId, const ControlList &sensorControls) { + IPAFrameContext &frameContext = context_.frameContexts.get(frame); + const rkisp1_stat_buffer *stats = reinterpret_cast( mappedBuffers_.at(bufferId).planes()[0].data()); @@ -327,9 +344,6 @@ void IPARkISP1::processStatsBuffer(const uint32_t frame, const uint32_t bufferId unsigned int aeState = 0; - /* \todo Obtain the frame context to pass to process from the FCQueue */ - IPAFrameContext frameContext; - for (auto const &algo : algorithms()) algo->process(context_, frame, frameContext, stats); From patchwork Thu Sep 8 01:41:45 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Laurent Pinchart X-Patchwork-Id: 17326 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 43FECC3272 for ; Thu, 8 Sep 2022 01:42:45 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id CDB2C620E0; Thu, 8 Sep 2022 03:42:44 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1662601365; bh=zx7tti0zOqP2dfT/d3VLBIJWzOTnQBnNhpE2Sie286c=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To: From; b=1wswWTcbHRCAiCBANRI+gAVtChYsUkfUiE9d1SdSiUfQ8PGG3vVWUET2Qo160RCSK vDFcsMEzODT6d91T6Z1AhuJnEB8Xt/a6cCHaWVC7m+U3zducojkCVx8/K1px0EB4Wi R/uNlJo4NXY1NzKPkcae9JfdwrrAkbWH6McsMn/g9oouU49aplEVZ3/ACVKyTDiprB DHk4rpCt3Pi4w0R2XVYllXbIWgK2on8LW+TZqIyjnSTz6nTYdfEY7fh+NGP65rTpO1 HpxcTL2r7zUWse/0KQSFQCVeOXkA5vIXF6919GUESuBpv24XqvQkOuZFQAQsZIfU/F DPZ04LLcnhscg== Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id B92FF620D1 for ; Thu, 8 Sep 2022 03:42:41 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="BMyzFTrl"; dkim-atps=neutral Received: from pendragon.ideasonboard.com (62-78-145-57.bb.dnainternet.fi [62.78.145.57]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 20D6A6CC; Thu, 8 Sep 2022 03:42:41 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1662601361; bh=zx7tti0zOqP2dfT/d3VLBIJWzOTnQBnNhpE2Sie286c=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=BMyzFTrlkO7RwXQx/RHR99GgKrWJasjnmgf5/rnzZChMYbaT4NaxBlPhNiB0TZU8v 4tDKg8IBZap4sIvAvH8E41D2uQFk9slD6TZxYtr8qnzwEDrxTrSGkMOO2QKrASbJyb d6fP0pv2gJl1cCZsyUVpl6rAhn7k08CTkcvqaE8Q= To: libcamera-devel@lists.libcamera.org Date: Thu, 8 Sep 2022 04:41:45 +0300 Message-Id: <20220908014200.28728-18-laurent.pinchart@ideasonboard.com> X-Mailer: git-send-email 2.35.1 In-Reply-To: <20220908014200.28728-1-laurent.pinchart@ideasonboard.com> References: <20220908014200.28728-1-laurent.pinchart@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v4 17/32] ipa: Rename IPAFrameContext structures X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Laurent Pinchart via libcamera-devel From: Laurent Pinchart Reply-To: Laurent Pinchart Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" From: Kieran Bingham Rename the IPAFrameContext structures in the IPU3 and RkISP1 IPA modules to IPU3FrameContext and RkISP1FrameContext respectively, to avoid ambiguities. Signed-off-by: Kieran Bingham Reviewed-by: Jacopo Mondi --- src/ipa/ipu3/algorithms/af.cpp | 4 ++-- src/ipa/ipu3/algorithms/af.h | 4 ++-- src/ipa/ipu3/algorithms/agc.cpp | 4 ++-- src/ipa/ipu3/algorithms/agc.h | 4 ++-- src/ipa/ipu3/algorithms/awb.cpp | 4 ++-- src/ipa/ipu3/algorithms/awb.h | 4 ++-- src/ipa/ipu3/algorithms/blc.cpp | 2 +- src/ipa/ipu3/algorithms/blc.h | 2 +- src/ipa/ipu3/algorithms/tone_mapping.cpp | 4 ++-- src/ipa/ipu3/algorithms/tone_mapping.h | 4 ++-- src/ipa/ipu3/ipa_context.cpp | 10 +++++----- src/ipa/ipu3/ipa_context.h | 4 ++-- src/ipa/ipu3/ipu3.cpp | 6 +++--- src/ipa/ipu3/module.h | 2 +- src/ipa/rkisp1/algorithms/agc.cpp | 4 ++-- src/ipa/rkisp1/algorithms/agc.h | 4 ++-- src/ipa/rkisp1/algorithms/awb.cpp | 6 +++--- src/ipa/rkisp1/algorithms/awb.h | 6 +++--- src/ipa/rkisp1/algorithms/blc.cpp | 2 +- src/ipa/rkisp1/algorithms/blc.h | 2 +- src/ipa/rkisp1/algorithms/cproc.cpp | 4 ++-- src/ipa/rkisp1/algorithms/cproc.h | 4 ++-- src/ipa/rkisp1/algorithms/dpcc.cpp | 2 +- src/ipa/rkisp1/algorithms/dpcc.h | 2 +- src/ipa/rkisp1/algorithms/dpf.cpp | 4 ++-- src/ipa/rkisp1/algorithms/dpf.h | 4 ++-- src/ipa/rkisp1/algorithms/filter.cpp | 4 ++-- src/ipa/rkisp1/algorithms/filter.h | 4 ++-- src/ipa/rkisp1/algorithms/gsl.cpp | 2 +- src/ipa/rkisp1/algorithms/gsl.h | 2 +- src/ipa/rkisp1/algorithms/lsc.cpp | 2 +- src/ipa/rkisp1/algorithms/lsc.h | 2 +- src/ipa/rkisp1/ipa_context.cpp | 2 +- src/ipa/rkisp1/ipa_context.h | 4 ++-- src/ipa/rkisp1/module.h | 2 +- src/ipa/rkisp1/rkisp1.cpp | 6 +++--- 36 files changed, 66 insertions(+), 66 deletions(-) diff --git a/src/ipa/ipu3/algorithms/af.cpp b/src/ipa/ipu3/algorithms/af.cpp index b00a34f9e090..0370619fb0d7 100644 --- a/src/ipa/ipu3/algorithms/af.cpp +++ b/src/ipa/ipu3/algorithms/af.cpp @@ -118,7 +118,7 @@ Af::Af() */ void Af::prepare(IPAContext &context, [[maybe_unused]] const uint32_t frame, - [[maybe_unused]] IPAFrameContext &frameContext, + [[maybe_unused]] IPU3FrameContext &frameContext, ipu3_uapi_params *params) { const struct ipu3_uapi_grid_config &grid = context.configuration.af.afGrid; @@ -422,7 +422,7 @@ bool Af::afIsOutOfFocus(IPAContext &context) * [1] Hill Climbing Algorithm, https://en.wikipedia.org/wiki/Hill_climbing */ void Af::process(IPAContext &context, [[maybe_unused]] const uint32_t frame, - [[maybe_unused]] IPAFrameContext &frameContext, + [[maybe_unused]] IPU3FrameContext &frameContext, const ipu3_uapi_stats_3a *stats) { /* Evaluate the AF buffer length */ diff --git a/src/ipa/ipu3/algorithms/af.h b/src/ipa/ipu3/algorithms/af.h index 89d37ac18615..4727221b595a 100644 --- a/src/ipa/ipu3/algorithms/af.h +++ b/src/ipa/ipu3/algorithms/af.h @@ -32,10 +32,10 @@ public: int configure(IPAContext &context, const IPAConfigInfo &configInfo) override; void prepare(IPAContext &context, const uint32_t frame, - IPAFrameContext &frameContext, + IPU3FrameContext &frameContext, ipu3_uapi_params *params) override; void process(IPAContext &context, const uint32_t frame, - IPAFrameContext &frameContext, + IPU3FrameContext &frameContext, const ipu3_uapi_stats_3a *stats) override; private: diff --git a/src/ipa/ipu3/algorithms/agc.cpp b/src/ipa/ipu3/algorithms/agc.cpp index a1a3c38ffe84..fc5810218a4a 100644 --- a/src/ipa/ipu3/algorithms/agc.cpp +++ b/src/ipa/ipu3/algorithms/agc.cpp @@ -183,7 +183,7 @@ utils::Duration Agc::filterExposure(utils::Duration exposureValue) * \param[in] yGain The gain calculated based on the relative luminance target * \param[in] iqMeanGain The gain calculated based on the relative luminance target */ -void Agc::computeExposure(IPAContext &context, IPAFrameContext &frameContext, +void Agc::computeExposure(IPAContext &context, IPU3FrameContext &frameContext, double yGain, double iqMeanGain) { const IPASessionConfiguration &configuration = context.configuration; @@ -325,7 +325,7 @@ double Agc::estimateLuminance(IPAActiveState &activeState, * new exposure and gain for the scene. */ void Agc::process(IPAContext &context, [[maybe_unused]] const uint32_t frame, - IPAFrameContext &frameContext, + IPU3FrameContext &frameContext, const ipu3_uapi_stats_3a *stats) { /* diff --git a/src/ipa/ipu3/algorithms/agc.h b/src/ipa/ipu3/algorithms/agc.h index 59b4b9843c2f..726c9d905026 100644 --- a/src/ipa/ipu3/algorithms/agc.h +++ b/src/ipa/ipu3/algorithms/agc.h @@ -29,14 +29,14 @@ public: int configure(IPAContext &context, const IPAConfigInfo &configInfo) override; void process(IPAContext &context, const uint32_t frame, - IPAFrameContext &frameContext, + IPU3FrameContext &frameContext, const ipu3_uapi_stats_3a *stats) override; private: double measureBrightness(const ipu3_uapi_stats_3a *stats, const ipu3_uapi_grid_config &grid) const; utils::Duration filterExposure(utils::Duration currentExposure); - void computeExposure(IPAContext &context, IPAFrameContext &frameContext, + void computeExposure(IPAContext &context, IPU3FrameContext &frameContext, double yGain, double iqMeanGain); double estimateLuminance(IPAActiveState &activeState, const ipu3_uapi_grid_config &grid, diff --git a/src/ipa/ipu3/algorithms/awb.cpp b/src/ipa/ipu3/algorithms/awb.cpp index 0dbd7d4c374f..289e9a05f08a 100644 --- a/src/ipa/ipu3/algorithms/awb.cpp +++ b/src/ipa/ipu3/algorithms/awb.cpp @@ -388,7 +388,7 @@ void Awb::calculateWBGains(const ipu3_uapi_stats_3a *stats) * \copydoc libcamera::ipa::Algorithm::process */ void Awb::process(IPAContext &context, [[maybe_unused]] const uint32_t frame, - [[maybe_unused]] IPAFrameContext &frameContext, + [[maybe_unused]] IPU3FrameContext &frameContext, const ipu3_uapi_stats_3a *stats) { calculateWBGains(stats); @@ -432,7 +432,7 @@ constexpr uint16_t Awb::gainValue(double gain) */ void Awb::prepare(IPAContext &context, [[maybe_unused]] const uint32_t frame, - [[maybe_unused]] IPAFrameContext &frameContext, + [[maybe_unused]] IPU3FrameContext &frameContext, ipu3_uapi_params *params) { /* diff --git a/src/ipa/ipu3/algorithms/awb.h b/src/ipa/ipu3/algorithms/awb.h index 28e2d38a711c..6662aa6ffe04 100644 --- a/src/ipa/ipu3/algorithms/awb.h +++ b/src/ipa/ipu3/algorithms/awb.h @@ -40,10 +40,10 @@ public: int configure(IPAContext &context, const IPAConfigInfo &configInfo) override; void prepare(IPAContext &context, const uint32_t frame, - IPAFrameContext &frameContext, + IPU3FrameContext &frameContext, ipu3_uapi_params *params) override; void process(IPAContext &context, const uint32_t frame, - IPAFrameContext &frameContext, + IPU3FrameContext &frameContext, const ipu3_uapi_stats_3a *stats) override; private: diff --git a/src/ipa/ipu3/algorithms/blc.cpp b/src/ipa/ipu3/algorithms/blc.cpp index e838072a344c..ca03b280b66c 100644 --- a/src/ipa/ipu3/algorithms/blc.cpp +++ b/src/ipa/ipu3/algorithms/blc.cpp @@ -48,7 +48,7 @@ BlackLevelCorrection::BlackLevelCorrection() */ void BlackLevelCorrection::prepare([[maybe_unused]] IPAContext &context, [[maybe_unused]] const uint32_t frame, - [[maybe_unused]] IPAFrameContext &frameContext, + [[maybe_unused]] IPU3FrameContext &frameContext, ipu3_uapi_params *params) { /* diff --git a/src/ipa/ipu3/algorithms/blc.h b/src/ipa/ipu3/algorithms/blc.h index 292bf67b02ff..b7cc55909e71 100644 --- a/src/ipa/ipu3/algorithms/blc.h +++ b/src/ipa/ipu3/algorithms/blc.h @@ -19,7 +19,7 @@ public: BlackLevelCorrection(); void prepare(IPAContext &context, const uint32_t frame, - IPAFrameContext &frameContext, + IPU3FrameContext &frameContext, ipu3_uapi_params *params) override; }; diff --git a/src/ipa/ipu3/algorithms/tone_mapping.cpp b/src/ipa/ipu3/algorithms/tone_mapping.cpp index eac3d4064443..a8bd05d935cf 100644 --- a/src/ipa/ipu3/algorithms/tone_mapping.cpp +++ b/src/ipa/ipu3/algorithms/tone_mapping.cpp @@ -59,7 +59,7 @@ int ToneMapping::configure(IPAContext &context, */ void ToneMapping::prepare([[maybe_unused]] IPAContext &context, [[maybe_unused]] const uint32_t frame, - [[maybe_unused]] IPAFrameContext &frameContext, + [[maybe_unused]] IPU3FrameContext &frameContext, ipu3_uapi_params *params) { /* Copy the calculated LUT into the parameters buffer. */ @@ -84,7 +84,7 @@ void ToneMapping::prepare([[maybe_unused]] IPAContext &context, * our gamma setting. */ void ToneMapping::process(IPAContext &context, [[maybe_unused]] const uint32_t frame, - [[maybe_unused]] IPAFrameContext &frameContext, + [[maybe_unused]] IPU3FrameContext &frameContext, [[maybe_unused]] const ipu3_uapi_stats_3a *stats) { /* diff --git a/src/ipa/ipu3/algorithms/tone_mapping.h b/src/ipa/ipu3/algorithms/tone_mapping.h index 822e5168df82..e8d7f593d513 100644 --- a/src/ipa/ipu3/algorithms/tone_mapping.h +++ b/src/ipa/ipu3/algorithms/tone_mapping.h @@ -20,9 +20,9 @@ public: int configure(IPAContext &context, const IPAConfigInfo &configInfo) override; void prepare(IPAContext &context, const uint32_t frame, - IPAFrameContext &frameContext, ipu3_uapi_params *params) override; + IPU3FrameContext &frameContext, ipu3_uapi_params *params) override; void process(IPAContext &context, const uint32_t frame, - IPAFrameContext &frameContext, + IPU3FrameContext &frameContext, const ipu3_uapi_stats_3a *stats) override; private: diff --git a/src/ipa/ipu3/ipa_context.cpp b/src/ipa/ipu3/ipa_context.cpp index bd71b615365d..89cb87c66d9f 100644 --- a/src/ipa/ipu3/ipa_context.cpp +++ b/src/ipa/ipu3/ipa_context.cpp @@ -43,7 +43,7 @@ namespace libcamera::ipa::ipu3 { * \brief The IPA session configuration, immutable during the session * * \var IPAContext::frameContexts - * \brief Ring buffer of the IPAFrameContext(s) + * \brief Ring buffer of the IPU3FrameContext(s) * * \var IPAContext::activeState * \brief The current state of IPA algorithms @@ -165,16 +165,16 @@ namespace libcamera::ipa::ipu3 { */ /** - * \struct IPAFrameContext + * \struct IPU3FrameContext * \brief IPU3-specific FrameContext * - * \var IPAFrameContext::sensor + * \var IPU3FrameContext::sensor * \brief Effective sensor values that were applied for the frame * - * \var IPAFrameContext::sensor.exposure + * \var IPU3FrameContext::sensor.exposure * \brief Exposure time expressed as a number of lines * - * \var IPAFrameContext::sensor.gain + * \var IPU3FrameContext::sensor.gain * \brief Analogue gain multiplier */ diff --git a/src/ipa/ipu3/ipa_context.h b/src/ipa/ipu3/ipa_context.h index 36099353e9f2..b8a7ad24a032 100644 --- a/src/ipa/ipu3/ipa_context.h +++ b/src/ipa/ipu3/ipa_context.h @@ -72,7 +72,7 @@ struct IPAActiveState { } toneMapping; }; -struct IPAFrameContext : public FrameContext { +struct IPU3FrameContext : public FrameContext { struct { uint32_t exposure; double gain; @@ -83,7 +83,7 @@ struct IPAContext { IPASessionConfiguration configuration; IPAActiveState activeState; - FCQueue frameContexts; + FCQueue frameContexts; }; } /* namespace ipa::ipu3 */ diff --git a/src/ipa/ipu3/ipu3.cpp b/src/ipa/ipu3/ipu3.cpp index 844ab6de03c7..d850aa46bc28 100644 --- a/src/ipa/ipu3/ipu3.cpp +++ b/src/ipa/ipu3/ipu3.cpp @@ -585,7 +585,7 @@ void IPAIPU3::fillParamsBuffer(const uint32_t frame, const uint32_t bufferId) */ params->use = {}; - IPAFrameContext &frameContext = context_.frameContexts.get(frame); + IPU3FrameContext &frameContext = context_.frameContexts.get(frame); for (auto const &algo : algorithms()) algo->prepare(context_, frame, frameContext, params); @@ -618,7 +618,7 @@ void IPAIPU3::processStatsBuffer(const uint32_t frame, const ipu3_uapi_stats_3a *stats = reinterpret_cast(mem.data()); - IPAFrameContext &frameContext = context_.frameContexts.get(frame); + IPU3FrameContext &frameContext = context_.frameContexts.get(frame); frameContext.sensor.exposure = sensorControls.get(V4L2_CID_EXPOSURE).get(); frameContext.sensor.gain = camHelper_->gain(sensorControls.get(V4L2_CID_ANALOGUE_GAIN).get()); @@ -664,7 +664,7 @@ void IPAIPU3::processStatsBuffer(const uint32_t frame, void IPAIPU3::queueRequest(const uint32_t frame, const ControlList &controls) { /* \todo Start processing for 'frame' based on 'controls'. */ - IPAFrameContext &frameContext = context_.frameContexts.init(frame); + IPU3FrameContext &frameContext = context_.frameContexts.init(frame); for (auto const &algo : algorithms()) algo->queueRequest(context_, frame, frameContext, controls); diff --git a/src/ipa/ipu3/module.h b/src/ipa/ipu3/module.h index d94fc4594871..6d0d50f615d8 100644 --- a/src/ipa/ipu3/module.h +++ b/src/ipa/ipu3/module.h @@ -19,7 +19,7 @@ namespace libcamera { namespace ipa::ipu3 { -using Module = ipa::Module; } /* namespace ipa::ipu3 */ diff --git a/src/ipa/rkisp1/algorithms/agc.cpp b/src/ipa/rkisp1/algorithms/agc.cpp index 606f13e776a3..e9fab0222ac3 100644 --- a/src/ipa/rkisp1/algorithms/agc.cpp +++ b/src/ipa/rkisp1/algorithms/agc.cpp @@ -285,7 +285,7 @@ double Agc::measureBrightness(const rkisp1_cif_isp_hist_stat *hist) const * new exposure and gain for the scene. */ void Agc::process(IPAContext &context, [[maybe_unused]] const uint32_t frame, - [[maybe_unused]] IPAFrameContext &frameContext, + [[maybe_unused]] RkISP1FrameContext &frameContext, const rkisp1_stat_buffer *stats) { const rkisp1_cif_isp_stat *params = &stats->params; @@ -328,7 +328,7 @@ void Agc::process(IPAContext &context, [[maybe_unused]] const uint32_t frame, */ void Agc::prepare(IPAContext &context, [[maybe_unused]] const uint32_t frame, - [[maybe_unused]] IPAFrameContext &frameContext, + [[maybe_unused]] RkISP1FrameContext &frameContext, rkisp1_params_cfg *params) { if (context.activeState.frameCount > 0) diff --git a/src/ipa/rkisp1/algorithms/agc.h b/src/ipa/rkisp1/algorithms/agc.h index f115ba2ed85c..be8932040c8e 100644 --- a/src/ipa/rkisp1/algorithms/agc.h +++ b/src/ipa/rkisp1/algorithms/agc.h @@ -27,10 +27,10 @@ public: int configure(IPAContext &context, const IPACameraSensorInfo &configInfo) override; void prepare(IPAContext &context, const uint32_t frame, - IPAFrameContext &frameContext, + RkISP1FrameContext &frameContext, rkisp1_params_cfg *params) override; void process(IPAContext &context, const uint32_t frame, - IPAFrameContext &frameContext, + RkISP1FrameContext &frameContext, const rkisp1_stat_buffer *stats) override; private: diff --git a/src/ipa/rkisp1/algorithms/awb.cpp b/src/ipa/rkisp1/algorithms/awb.cpp index 2bd9ef779bc3..d3e9aa5334cf 100644 --- a/src/ipa/rkisp1/algorithms/awb.cpp +++ b/src/ipa/rkisp1/algorithms/awb.cpp @@ -76,7 +76,7 @@ uint32_t Awb::estimateCCT(double red, double green, double blue) */ void Awb::prepare(IPAContext &context, [[maybe_unused]] const uint32_t frame, - [[maybe_unused]] IPAFrameContext &frameContext, + [[maybe_unused]] RkISP1FrameContext &frameContext, rkisp1_params_cfg *params) { params->others.awb_gain_config.gain_green_b = 256 * context.activeState.awb.gains.green; @@ -128,7 +128,7 @@ void Awb::prepare(IPAContext &context, */ void Awb::queueRequest(IPAContext &context, [[maybe_unused]] const uint32_t frame, - [[maybe_unused]] IPAFrameContext &frameContext, + [[maybe_unused]] RkISP1FrameContext &frameContext, const ControlList &controls) { auto &awb = context.activeState.awb; @@ -157,7 +157,7 @@ void Awb::queueRequest(IPAContext &context, */ void Awb::process([[maybe_unused]] IPAContext &context, [[maybe_unused]] const uint32_t frame, - [[maybe_unused]] IPAFrameContext &frameCtx, + [[maybe_unused]] RkISP1FrameContext &frameCtx, const rkisp1_stat_buffer *stats) { const rkisp1_cif_isp_stat *params = &stats->params; diff --git a/src/ipa/rkisp1/algorithms/awb.h b/src/ipa/rkisp1/algorithms/awb.h index 6d4a39467038..9fd156d8aff9 100644 --- a/src/ipa/rkisp1/algorithms/awb.h +++ b/src/ipa/rkisp1/algorithms/awb.h @@ -21,13 +21,13 @@ public: int configure(IPAContext &context, const IPACameraSensorInfo &configInfo) override; void prepare(IPAContext &context, const uint32_t frame, - IPAFrameContext &frameContext, + RkISP1FrameContext &frameContext, rkisp1_params_cfg *params) override; void queueRequest(IPAContext &context, const uint32_t frame, - IPAFrameContext &frameContext, + RkISP1FrameContext &frameContext, const ControlList &controls) override; void process(IPAContext &context, const uint32_t frame, - IPAFrameContext &frameCtx, + RkISP1FrameContext &frameCtx, const rkisp1_stat_buffer *stats) override; private: diff --git a/src/ipa/rkisp1/algorithms/blc.cpp b/src/ipa/rkisp1/algorithms/blc.cpp index 0f7226cf217d..c4aea0e828ca 100644 --- a/src/ipa/rkisp1/algorithms/blc.cpp +++ b/src/ipa/rkisp1/algorithms/blc.cpp @@ -67,7 +67,7 @@ int BlackLevelCorrection::init([[maybe_unused]] IPAContext &context, */ void BlackLevelCorrection::prepare(IPAContext &context, [[maybe_unused]] const uint32_t frame, - [[maybe_unused]] IPAFrameContext &frameContext, + [[maybe_unused]] RkISP1FrameContext &frameContext, rkisp1_params_cfg *params) { if (context.activeState.frameCount > 0) diff --git a/src/ipa/rkisp1/algorithms/blc.h b/src/ipa/rkisp1/algorithms/blc.h index 0b1a2d436869..e57c8c7a95c5 100644 --- a/src/ipa/rkisp1/algorithms/blc.h +++ b/src/ipa/rkisp1/algorithms/blc.h @@ -21,7 +21,7 @@ public: int init(IPAContext &context, const YamlObject &tuningData) override; void prepare(IPAContext &context, const uint32_t frame, - IPAFrameContext &frameContext, + RkISP1FrameContext &frameContext, rkisp1_params_cfg *params) override; private: diff --git a/src/ipa/rkisp1/algorithms/cproc.cpp b/src/ipa/rkisp1/algorithms/cproc.cpp index ea819b2acfcb..22a70e0b70c7 100644 --- a/src/ipa/rkisp1/algorithms/cproc.cpp +++ b/src/ipa/rkisp1/algorithms/cproc.cpp @@ -38,7 +38,7 @@ LOG_DEFINE_CATEGORY(RkISP1CProc) */ void ColorProcessing::queueRequest(IPAContext &context, [[maybe_unused]] const uint32_t frame, - [[maybe_unused]] IPAFrameContext &frameContext, + [[maybe_unused]] RkISP1FrameContext &frameContext, const ControlList &controls) { auto &cproc = context.activeState.cproc; @@ -73,7 +73,7 @@ void ColorProcessing::queueRequest(IPAContext &context, */ void ColorProcessing::prepare(IPAContext &context, [[maybe_unused]] const uint32_t frame, - [[maybe_unused]] IPAFrameContext &frameContext, + [[maybe_unused]] RkISP1FrameContext &frameContext, rkisp1_params_cfg *params) { auto &cproc = context.activeState.cproc; diff --git a/src/ipa/rkisp1/algorithms/cproc.h b/src/ipa/rkisp1/algorithms/cproc.h index ba6e901ae110..4e9a17a58800 100644 --- a/src/ipa/rkisp1/algorithms/cproc.h +++ b/src/ipa/rkisp1/algorithms/cproc.h @@ -22,10 +22,10 @@ public: ~ColorProcessing() = default; void queueRequest(IPAContext &context, const uint32_t frame, - IPAFrameContext &frameContext, + RkISP1FrameContext &frameContext, const ControlList &controls) override; void prepare(IPAContext &context, const uint32_t frame, - IPAFrameContext &frameContext, + RkISP1FrameContext &frameContext, rkisp1_params_cfg *params) override; }; diff --git a/src/ipa/rkisp1/algorithms/dpcc.cpp b/src/ipa/rkisp1/algorithms/dpcc.cpp index 7c14ace0abee..b61eee022a68 100644 --- a/src/ipa/rkisp1/algorithms/dpcc.cpp +++ b/src/ipa/rkisp1/algorithms/dpcc.cpp @@ -260,7 +260,7 @@ int DefectPixelClusterCorrection::init([[maybe_unused]] IPAContext &context, */ void DefectPixelClusterCorrection::prepare(IPAContext &context, [[maybe_unused]] const uint32_t frame, - [[maybe_unused]] IPAFrameContext &frameContext, + [[maybe_unused]] RkISP1FrameContext &frameContext, rkisp1_params_cfg *params) { if (context.activeState.frameCount > 0) diff --git a/src/ipa/rkisp1/algorithms/dpcc.h b/src/ipa/rkisp1/algorithms/dpcc.h index 894c0249f35d..4c242e99f8ce 100644 --- a/src/ipa/rkisp1/algorithms/dpcc.h +++ b/src/ipa/rkisp1/algorithms/dpcc.h @@ -21,7 +21,7 @@ public: int init(IPAContext &context, const YamlObject &tuningData) override; void prepare(IPAContext &context, const uint32_t frame, - IPAFrameContext &frameContext, + RkISP1FrameContext &frameContext, rkisp1_params_cfg *params) override; private: diff --git a/src/ipa/rkisp1/algorithms/dpf.cpp b/src/ipa/rkisp1/algorithms/dpf.cpp index b8c837c21243..8eafbbf85124 100644 --- a/src/ipa/rkisp1/algorithms/dpf.cpp +++ b/src/ipa/rkisp1/algorithms/dpf.cpp @@ -176,7 +176,7 @@ int Dpf::init([[maybe_unused]] IPAContext &context, */ void Dpf::queueRequest(IPAContext &context, [[maybe_unused]] const uint32_t frame, - [[maybe_unused]] IPAFrameContext &frameContext, + [[maybe_unused]] RkISP1FrameContext &frameContext, const ControlList &controls) { auto &dpf = context.activeState.dpf; @@ -208,7 +208,7 @@ void Dpf::queueRequest(IPAContext &context, * \copydoc libcamera::ipa::Algorithm::prepare */ void Dpf::prepare(IPAContext &context, [[maybe_unused]] const uint32_t frame, - [[maybe_unused]] IPAFrameContext &frameContext, + [[maybe_unused]] RkISP1FrameContext &frameContext, rkisp1_params_cfg *params) { if (!initialized_) diff --git a/src/ipa/rkisp1/algorithms/dpf.h b/src/ipa/rkisp1/algorithms/dpf.h index e232cad4ee87..dd465a52ba0a 100644 --- a/src/ipa/rkisp1/algorithms/dpf.h +++ b/src/ipa/rkisp1/algorithms/dpf.h @@ -23,10 +23,10 @@ public: int init(IPAContext &context, const YamlObject &tuningData) override; void queueRequest(IPAContext &context, const uint32_t frame, - IPAFrameContext &frameContext, + RkISP1FrameContext &frameContext, const ControlList &controls) override; void prepare(IPAContext &context, const uint32_t frame, - IPAFrameContext &frameContext, + RkISP1FrameContext &frameContext, rkisp1_params_cfg *params) override; private: diff --git a/src/ipa/rkisp1/algorithms/filter.cpp b/src/ipa/rkisp1/algorithms/filter.cpp index 837560eb20a6..e64bd6a6d68f 100644 --- a/src/ipa/rkisp1/algorithms/filter.cpp +++ b/src/ipa/rkisp1/algorithms/filter.cpp @@ -44,7 +44,7 @@ static constexpr uint32_t kFiltModeDefault = 0x000004f2; */ void Filter::queueRequest(IPAContext &context, [[maybe_unused]] const uint32_t frame, - [[maybe_unused]] IPAFrameContext &frameContext, + [[maybe_unused]] RkISP1FrameContext &frameContext, const ControlList &controls) { auto &filter = context.activeState.filter; @@ -88,7 +88,7 @@ void Filter::queueRequest(IPAContext &context, */ void Filter::prepare(IPAContext &context, [[maybe_unused]] const uint32_t frame, - [[maybe_unused]] IPAFrameContext &frameContext, + [[maybe_unused]] RkISP1FrameContext &frameContext, rkisp1_params_cfg *params) { auto &filter = context.activeState.filter; diff --git a/src/ipa/rkisp1/algorithms/filter.h b/src/ipa/rkisp1/algorithms/filter.h index 3fd882ea41dc..c340e824722d 100644 --- a/src/ipa/rkisp1/algorithms/filter.h +++ b/src/ipa/rkisp1/algorithms/filter.h @@ -22,10 +22,10 @@ public: ~Filter() = default; void queueRequest(IPAContext &context, const uint32_t frame, - IPAFrameContext &frameContext, + RkISP1FrameContext &frameContext, const ControlList &controls) override; void prepare(IPAContext &context, const uint32_t frame, - IPAFrameContext &frameContext, + RkISP1FrameContext &frameContext, rkisp1_params_cfg *params) override; }; diff --git a/src/ipa/rkisp1/algorithms/gsl.cpp b/src/ipa/rkisp1/algorithms/gsl.cpp index 879ca2973e8a..ed3104c98e48 100644 --- a/src/ipa/rkisp1/algorithms/gsl.cpp +++ b/src/ipa/rkisp1/algorithms/gsl.cpp @@ -120,7 +120,7 @@ int GammaSensorLinearization::init([[maybe_unused]] IPAContext &context, */ void GammaSensorLinearization::prepare(IPAContext &context, [[maybe_unused]] const uint32_t frame, - [[maybe_unused]] IPAFrameContext &frameContext, + [[maybe_unused]] RkISP1FrameContext &frameContext, rkisp1_params_cfg *params) { if (context.activeState.frameCount > 0) diff --git a/src/ipa/rkisp1/algorithms/gsl.h b/src/ipa/rkisp1/algorithms/gsl.h index 5024b6835949..5a951f07e56d 100644 --- a/src/ipa/rkisp1/algorithms/gsl.h +++ b/src/ipa/rkisp1/algorithms/gsl.h @@ -21,7 +21,7 @@ public: int init(IPAContext &context, const YamlObject &tuningData) override; void prepare(IPAContext &context, const uint32_t frame, - IPAFrameContext &frameContext, + RkISP1FrameContext &frameContext, rkisp1_params_cfg *params) override; private: diff --git a/src/ipa/rkisp1/algorithms/lsc.cpp b/src/ipa/rkisp1/algorithms/lsc.cpp index 9c717bc5f99e..ed5b988599b8 100644 --- a/src/ipa/rkisp1/algorithms/lsc.cpp +++ b/src/ipa/rkisp1/algorithms/lsc.cpp @@ -134,7 +134,7 @@ int LensShadingCorrection::configure(IPAContext &context, */ void LensShadingCorrection::prepare(IPAContext &context, [[maybe_unused]] const uint32_t frame, - [[maybe_unused]] IPAFrameContext &frameContext, + [[maybe_unused]] RkISP1FrameContext &frameContext, rkisp1_params_cfg *params) { if (context.activeState.frameCount > 0) diff --git a/src/ipa/rkisp1/algorithms/lsc.h b/src/ipa/rkisp1/algorithms/lsc.h index da957d3e3103..b4f18e565085 100644 --- a/src/ipa/rkisp1/algorithms/lsc.h +++ b/src/ipa/rkisp1/algorithms/lsc.h @@ -22,7 +22,7 @@ public: int init(IPAContext &context, const YamlObject &tuningData) override; int configure(IPAContext &context, const IPACameraSensorInfo &configInfo) override; void prepare(IPAContext &context, const uint32_t frame, - IPAFrameContext &frameContext, + RkISP1FrameContext &frameContext, rkisp1_params_cfg *params) override; private: diff --git a/src/ipa/rkisp1/ipa_context.cpp b/src/ipa/rkisp1/ipa_context.cpp index e9846742ee4f..80fae271a6dc 100644 --- a/src/ipa/rkisp1/ipa_context.cpp +++ b/src/ipa/rkisp1/ipa_context.cpp @@ -202,7 +202,7 @@ namespace libcamera::ipa::rkisp1 { */ /** - * \struct IPAFrameContext + * \struct RkISP1FrameContext * \brief Per-frame context for algorithms * * \todo Populate the frame context for all algorithms diff --git a/src/ipa/rkisp1/ipa_context.h b/src/ipa/rkisp1/ipa_context.h index f6aaefffed52..9d89cf0160fc 100644 --- a/src/ipa/rkisp1/ipa_context.h +++ b/src/ipa/rkisp1/ipa_context.h @@ -91,14 +91,14 @@ struct IPAActiveState { unsigned int frameCount; }; -struct IPAFrameContext : public FrameContext { +struct RkISP1FrameContext : public FrameContext { }; struct IPAContext { IPASessionConfiguration configuration; IPAActiveState activeState; - FCQueue frameContexts; + FCQueue frameContexts; }; } /* namespace ipa::rkisp1 */ diff --git a/src/ipa/rkisp1/module.h b/src/ipa/rkisp1/module.h index 89f83208a75c..594d5c9d685a 100644 --- a/src/ipa/rkisp1/module.h +++ b/src/ipa/rkisp1/module.h @@ -19,7 +19,7 @@ namespace libcamera { namespace ipa::rkisp1 { -using Module = ipa::Module; } /* namespace ipa::rkisp1 */ diff --git a/src/ipa/rkisp1/rkisp1.cpp b/src/ipa/rkisp1/rkisp1.cpp index c5ed0bb21f67..538401ac89ac 100644 --- a/src/ipa/rkisp1/rkisp1.cpp +++ b/src/ipa/rkisp1/rkisp1.cpp @@ -304,7 +304,7 @@ void IPARkISP1::unmapBuffers(const std::vector &ids) void IPARkISP1::queueRequest(const uint32_t frame, const ControlList &controls) { - IPAFrameContext &frameContext = context_.frameContexts.init(frame); + RkISP1FrameContext &frameContext = context_.frameContexts.init(frame); for (auto const &algo : algorithms()) algo->queueRequest(context_, frame, frameContext, controls); @@ -312,7 +312,7 @@ void IPARkISP1::queueRequest(const uint32_t frame, const ControlList &controls) void IPARkISP1::fillParamsBuffer(const uint32_t frame, const uint32_t bufferId) { - IPAFrameContext &frameContext = context_.frameContexts.get(frame); + RkISP1FrameContext &frameContext = context_.frameContexts.get(frame); rkisp1_params_cfg *params = reinterpret_cast( @@ -331,7 +331,7 @@ void IPARkISP1::fillParamsBuffer(const uint32_t frame, const uint32_t bufferId) void IPARkISP1::processStatsBuffer(const uint32_t frame, const uint32_t bufferId, const ControlList &sensorControls) { - IPAFrameContext &frameContext = context_.frameContexts.get(frame); + RkISP1FrameContext &frameContext = context_.frameContexts.get(frame); const rkisp1_stat_buffer *stats = reinterpret_cast( From patchwork Thu Sep 8 01:41:46 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Laurent Pinchart X-Patchwork-Id: 17327 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id C8309C327D for ; Thu, 8 Sep 2022 01:42:45 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 753DB620DF; Thu, 8 Sep 2022 03:42:45 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1662601365; bh=3v11PVcUtrPKZMP+rWuEgyvaNHa3x3/J5D0Re907CSc=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To: From; b=k5GNpP05j5H6RltVT7R1OjFj9X0yvqX2ieNKvcI6NfM1srD+vWW7bsgpLCyWfFIGF kv3IZRrs3rI+QN3aumf+sAZV4sfoPIYtFTWTWU7Sm4WC6HSTflvI1JVEpmjuSzGiC6 V1/OTGeRx4DmDS7q7DgkcN2iNg5gmW64XDrRnCgkSh8yRoWFeuD/980AJUdLNx9Qxs P5x8x+r22uI/fnS6OaYoQuyoISGvVY8nNnC6HQleSUg02sCysFMxNNJO2k842aH/ks ND/ed7VIjvcoqKNi4L5KC7Ld9CGhZc7iW04qYnZVkZaGGbOCry0nkISIdEVGgj2ORb QsTeJbR3umATA== Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 41F2E620D4 for ; Thu, 8 Sep 2022 03:42:43 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="wZG7RgUl"; dkim-atps=neutral Received: from pendragon.ideasonboard.com (62-78-145-57.bb.dnainternet.fi [62.78.145.57]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id BB29C6CC for ; Thu, 8 Sep 2022 03:42:42 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1662601363; bh=3v11PVcUtrPKZMP+rWuEgyvaNHa3x3/J5D0Re907CSc=; h=From:To:Subject:Date:In-Reply-To:References:From; b=wZG7RgUlfKjt4Emoh8XLhuOZR1zp5mPN0PmOgbJHrKUDZZa9TFCTz27ElWhoLfxpl cbd6wQosg2yXQxbNxcqiTL+t5zyH0Dy1FeSHk9RgUGsTMybH6SKjqehJU7+tejibEd fLhjenOh1MvD9Q8aTN1Vz5sWJJCQh4W8ilaghdx8= To: libcamera-devel@lists.libcamera.org Date: Thu, 8 Sep 2022 04:41:46 +0300 Message-Id: <20220908014200.28728-19-laurent.pinchart@ideasonboard.com> X-Mailer: git-send-email 2.35.1 In-Reply-To: <20220908014200.28728-1-laurent.pinchart@ideasonboard.com> References: <20220908014200.28728-1-laurent.pinchart@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v4 18/32] ipa: rkisp1: Use frame number passed to Algorithm::prepare() X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Laurent Pinchart via libcamera-devel From: Laurent Pinchart Reply-To: Laurent Pinchart Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Now that the Algorithm::prepare() function takes a frame number, we can use it to replace the IPAActiveState::frameCount member. Signed-off-by: Laurent Pinchart Reviewed-by: Kieran Bingham Reviewed-by: Jacopo Mondi --- src/ipa/rkisp1/algorithms/agc.cpp | 5 ++--- src/ipa/rkisp1/algorithms/awb.cpp | 5 ++--- src/ipa/rkisp1/algorithms/blc.cpp | 6 +++--- src/ipa/rkisp1/algorithms/dpcc.cpp | 6 +++--- src/ipa/rkisp1/algorithms/dpf.cpp | 4 ++-- src/ipa/rkisp1/algorithms/gsl.cpp | 6 +++--- src/ipa/rkisp1/algorithms/lsc.cpp | 5 ++--- src/ipa/rkisp1/ipa_context.cpp | 9 --------- src/ipa/rkisp1/ipa_context.h | 2 -- src/ipa/rkisp1/rkisp1.cpp | 3 --- 10 files changed, 17 insertions(+), 34 deletions(-) diff --git a/src/ipa/rkisp1/algorithms/agc.cpp b/src/ipa/rkisp1/algorithms/agc.cpp index e9fab0222ac3..4124c3272bc7 100644 --- a/src/ipa/rkisp1/algorithms/agc.cpp +++ b/src/ipa/rkisp1/algorithms/agc.cpp @@ -326,12 +326,11 @@ void Agc::process(IPAContext &context, [[maybe_unused]] const uint32_t frame, /** * \copydoc libcamera::ipa::Algorithm::prepare */ -void Agc::prepare(IPAContext &context, - [[maybe_unused]] const uint32_t frame, +void Agc::prepare(IPAContext &context, const uint32_t frame, [[maybe_unused]] RkISP1FrameContext &frameContext, rkisp1_params_cfg *params) { - if (context.activeState.frameCount > 0) + if (frame > 0) return; /* Configure the measurement window. */ diff --git a/src/ipa/rkisp1/algorithms/awb.cpp b/src/ipa/rkisp1/algorithms/awb.cpp index d3e9aa5334cf..a0bbe5cb8afa 100644 --- a/src/ipa/rkisp1/algorithms/awb.cpp +++ b/src/ipa/rkisp1/algorithms/awb.cpp @@ -74,8 +74,7 @@ uint32_t Awb::estimateCCT(double red, double green, double blue) /** * \copydoc libcamera::ipa::Algorithm::prepare */ -void Awb::prepare(IPAContext &context, - [[maybe_unused]] const uint32_t frame, +void Awb::prepare(IPAContext &context, const uint32_t frame, [[maybe_unused]] RkISP1FrameContext &frameContext, rkisp1_params_cfg *params) { @@ -88,7 +87,7 @@ void Awb::prepare(IPAContext &context, params->module_cfg_update |= RKISP1_CIF_ISP_MODULE_AWB_GAIN; /* If we already have configured the gains and window, return. */ - if (context.activeState.frameCount > 0) + if (frame > 0) return; /* Configure the gains to apply. */ diff --git a/src/ipa/rkisp1/algorithms/blc.cpp b/src/ipa/rkisp1/algorithms/blc.cpp index c4aea0e828ca..500ce94294a2 100644 --- a/src/ipa/rkisp1/algorithms/blc.cpp +++ b/src/ipa/rkisp1/algorithms/blc.cpp @@ -65,12 +65,12 @@ int BlackLevelCorrection::init([[maybe_unused]] IPAContext &context, /** * \copydoc libcamera::ipa::Algorithm::prepare */ -void BlackLevelCorrection::prepare(IPAContext &context, - [[maybe_unused]] const uint32_t frame, +void BlackLevelCorrection::prepare([[maybe_unused]] IPAContext &context, + const uint32_t frame, [[maybe_unused]] RkISP1FrameContext &frameContext, rkisp1_params_cfg *params) { - if (context.activeState.frameCount > 0) + if (frame > 0) return; if (!tuningParameters_) diff --git a/src/ipa/rkisp1/algorithms/dpcc.cpp b/src/ipa/rkisp1/algorithms/dpcc.cpp index b61eee022a68..f59d48180eb5 100644 --- a/src/ipa/rkisp1/algorithms/dpcc.cpp +++ b/src/ipa/rkisp1/algorithms/dpcc.cpp @@ -258,12 +258,12 @@ int DefectPixelClusterCorrection::init([[maybe_unused]] IPAContext &context, /** * \copydoc libcamera::ipa::Algorithm::prepare */ -void DefectPixelClusterCorrection::prepare(IPAContext &context, - [[maybe_unused]] const uint32_t frame, +void DefectPixelClusterCorrection::prepare([[maybe_unused]] IPAContext &context, + const uint32_t frame, [[maybe_unused]] RkISP1FrameContext &frameContext, rkisp1_params_cfg *params) { - if (context.activeState.frameCount > 0) + if (frame > 0) return; if (!initialized_) diff --git a/src/ipa/rkisp1/algorithms/dpf.cpp b/src/ipa/rkisp1/algorithms/dpf.cpp index 8eafbbf85124..94c35c36570c 100644 --- a/src/ipa/rkisp1/algorithms/dpf.cpp +++ b/src/ipa/rkisp1/algorithms/dpf.cpp @@ -207,7 +207,7 @@ void Dpf::queueRequest(IPAContext &context, /** * \copydoc libcamera::ipa::Algorithm::prepare */ -void Dpf::prepare(IPAContext &context, [[maybe_unused]] const uint32_t frame, +void Dpf::prepare(IPAContext &context, const uint32_t frame, [[maybe_unused]] RkISP1FrameContext &frameContext, rkisp1_params_cfg *params) { @@ -216,7 +216,7 @@ void Dpf::prepare(IPAContext &context, [[maybe_unused]] const uint32_t frame, auto &dpf = context.activeState.dpf; - if (context.activeState.frameCount == 0) { + if (frame == 0) { params->others.dpf_config = config_; params->others.dpf_strength_config = strengthConfig_; diff --git a/src/ipa/rkisp1/algorithms/gsl.cpp b/src/ipa/rkisp1/algorithms/gsl.cpp index ed3104c98e48..44303594bd47 100644 --- a/src/ipa/rkisp1/algorithms/gsl.cpp +++ b/src/ipa/rkisp1/algorithms/gsl.cpp @@ -118,12 +118,12 @@ int GammaSensorLinearization::init([[maybe_unused]] IPAContext &context, /** * \copydoc libcamera::ipa::Algorithm::prepare */ -void GammaSensorLinearization::prepare(IPAContext &context, - [[maybe_unused]] const uint32_t frame, +void GammaSensorLinearization::prepare([[maybe_unused]] IPAContext &context, + const uint32_t frame, [[maybe_unused]] RkISP1FrameContext &frameContext, rkisp1_params_cfg *params) { - if (context.activeState.frameCount > 0) + if (frame > 0) return; if (!initialized_) diff --git a/src/ipa/rkisp1/algorithms/lsc.cpp b/src/ipa/rkisp1/algorithms/lsc.cpp index ed5b988599b8..20111c342bb3 100644 --- a/src/ipa/rkisp1/algorithms/lsc.cpp +++ b/src/ipa/rkisp1/algorithms/lsc.cpp @@ -132,12 +132,11 @@ int LensShadingCorrection::configure(IPAContext &context, /** * \copydoc libcamera::ipa::Algorithm::prepare */ -void LensShadingCorrection::prepare(IPAContext &context, - [[maybe_unused]] const uint32_t frame, +void LensShadingCorrection::prepare(IPAContext &context, const uint32_t frame, [[maybe_unused]] RkISP1FrameContext &frameContext, rkisp1_params_cfg *params) { - if (context.activeState.frameCount > 0) + if (frame > 0) return; if (!initialized_) diff --git a/src/ipa/rkisp1/ipa_context.cpp b/src/ipa/rkisp1/ipa_context.cpp index 80fae271a6dc..211428717838 100644 --- a/src/ipa/rkisp1/ipa_context.cpp +++ b/src/ipa/rkisp1/ipa_context.cpp @@ -192,15 +192,6 @@ namespace libcamera::ipa::rkisp1 { * \brief Analogue gain multiplier */ -/** - * \var IPAActiveState::frameCount - * \brief Counter of requests queued to the IPA module - * - * The counter is reset to 0 when the IPA module is configured, and is - * incremented for each request being queued, after calling the - * Algorithm::prepare() function of all algorithms. - */ - /** * \struct RkISP1FrameContext * \brief Per-frame context for algorithms diff --git a/src/ipa/rkisp1/ipa_context.h b/src/ipa/rkisp1/ipa_context.h index 9d89cf0160fc..4e38ef40ed3c 100644 --- a/src/ipa/rkisp1/ipa_context.h +++ b/src/ipa/rkisp1/ipa_context.h @@ -87,8 +87,6 @@ struct IPAActiveState { uint32_t exposure; double gain; } sensor; - - unsigned int frameCount; }; struct RkISP1FrameContext : public FrameContext { diff --git a/src/ipa/rkisp1/rkisp1.cpp b/src/ipa/rkisp1/rkisp1.cpp index 538401ac89ac..4f6e0b0f87b9 100644 --- a/src/ipa/rkisp1/rkisp1.cpp +++ b/src/ipa/rkisp1/rkisp1.cpp @@ -261,8 +261,6 @@ int IPARkISP1::configure([[maybe_unused]] const IPACameraSensorInfo &info, context_.configuration.agc.minAnalogueGain = camHelper_->gain(minGain); context_.configuration.agc.maxAnalogueGain = camHelper_->gain(maxGain); - context_.activeState.frameCount = 0; - for (auto const &algo : algorithms()) { int ret = algo->configure(context_, info); if (ret) @@ -325,7 +323,6 @@ void IPARkISP1::fillParamsBuffer(const uint32_t frame, const uint32_t bufferId) algo->prepare(context_, frame, frameContext, params); paramsBufferReady.emit(frame); - context_.activeState.frameCount++; } void IPARkISP1::processStatsBuffer(const uint32_t frame, const uint32_t bufferId, From patchwork Thu Sep 8 01:41:47 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Laurent Pinchart X-Patchwork-Id: 17328 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 691EFC3272 for ; Thu, 8 Sep 2022 01:42:47 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 1A627620DD; Thu, 8 Sep 2022 03:42:47 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1662601367; bh=z122C3wGPVnmSw1BpNS36W2k/1kgikwymrCKG6k6QhM=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To: From; b=dQF8bJbjBf3SLIg0DifO+7YYPLKYaVM5FCIy4j7xIX6mxGYjAfswPYYdb3DEUOFh2 +JGoj00x6thYoiBuolrT9xEL7unt2E6vcFFxt6I4+Rkkj3QULkSygjHtO06q0fJhUy eHr4Q2pFBqhxUo/hq312dWf6DjsphBHRfFjliwPiwf/QEA2jp97kRjZe67nJ46vLYx uC9P9ikHt1KwkyuvxgzaGnPuwK6Y6XJd3TK4TTR5W5+cwrz8+mSOwqTgLqQ2J+YJ8O rWLZUK3kpVpJ6pdCok3iPAprIOThxSe96tKxCoycuu4Xn53iEwQ+PAPYXsXPFP2qT/ bvlm5fjNhn5+A== Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id B1E8A620DF for ; Thu, 8 Sep 2022 03:42:44 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="oqJl8HNN"; dkim-atps=neutral Received: from pendragon.ideasonboard.com (62-78-145-57.bb.dnainternet.fi [62.78.145.57]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 403F26CC for ; Thu, 8 Sep 2022 03:42:44 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1662601364; bh=z122C3wGPVnmSw1BpNS36W2k/1kgikwymrCKG6k6QhM=; h=From:To:Subject:Date:In-Reply-To:References:From; b=oqJl8HNNtW3fCKrfhvONj4AxVRK4F1LCbpfR+bB6TUwonLsMokONp4TbqhSlJikpF CpJzuKRAUCj0gsUqZ/pi/twokMT9A2hvbeqw2gS6L8CZ/fNWc6zS9vWKqGuvEaj2Ls +xH1467gQTGsWagEskO1NfWSCKf0Ez7GddIykLuo= To: libcamera-devel@lists.libcamera.org Date: Thu, 8 Sep 2022 04:41:47 +0300 Message-Id: <20220908014200.28728-20-laurent.pinchart@ideasonboard.com> X-Mailer: git-send-email 2.35.1 In-Reply-To: <20220908014200.28728-1-laurent.pinchart@ideasonboard.com> References: <20220908014200.28728-1-laurent.pinchart@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v4 19/32] ipa: rkisp1: agc: Store per-frame information in frame context X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Laurent Pinchart via libcamera-devel From: Laurent Pinchart Reply-To: Laurent Pinchart Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Rework the algorithm's usage of the active state to store the value of controls for the last queued request in the queueRequest() function, and store a copy of the values in the corresponding frame context. The frame context is used in the prepare() function to populate the ISP parameters with values corresponding to the right frame. Signed-off-by: Laurent Pinchart Reviewed-by: Kieran Bingham --- src/ipa/rkisp1/algorithms/agc.cpp | 18 ++++++++----- src/ipa/rkisp1/algorithms/agc.h | 3 ++- src/ipa/rkisp1/ipa_context.cpp | 43 ++++++++++++++++++++----------- src/ipa/rkisp1/ipa_context.h | 14 ++++++---- src/ipa/rkisp1/rkisp1.cpp | 9 ++++--- 5 files changed, 55 insertions(+), 32 deletions(-) diff --git a/src/ipa/rkisp1/algorithms/agc.cpp b/src/ipa/rkisp1/algorithms/agc.cpp index 4124c3272bc7..e7198169d11b 100644 --- a/src/ipa/rkisp1/algorithms/agc.cpp +++ b/src/ipa/rkisp1/algorithms/agc.cpp @@ -144,17 +144,19 @@ utils::Duration Agc::filterExposure(utils::Duration exposureValue) /** * \brief Estimate the new exposure and gain values * \param[inout] context The shared IPA Context + * \param[in] frameContext The FrameContext for this frame * \param[in] yGain The gain calculated on the current brightness level * \param[in] iqMeanGain The gain calculated based on the relative luminance target */ -void Agc::computeExposure(IPAContext &context, double yGain, double iqMeanGain) +void Agc::computeExposure(IPAContext &context, RkISP1FrameContext &frameContext, + double yGain, double iqMeanGain) { IPASessionConfiguration &configuration = context.configuration; IPAActiveState &activeState = context.activeState; /* Get the effective exposure and gain applied on the sensor. */ - uint32_t exposure = activeState.sensor.exposure; - double analogueGain = activeState.sensor.gain; + uint32_t exposure = frameContext.sensor.exposure; + double analogueGain = frameContext.sensor.gain; /* Use the highest of the two gain estimates. */ double evGain = std::max(yGain, iqMeanGain); @@ -285,7 +287,7 @@ double Agc::measureBrightness(const rkisp1_cif_isp_hist_stat *hist) const * new exposure and gain for the scene. */ void Agc::process(IPAContext &context, [[maybe_unused]] const uint32_t frame, - [[maybe_unused]] RkISP1FrameContext &frameContext, + RkISP1FrameContext &frameContext, const rkisp1_stat_buffer *stats) { const rkisp1_cif_isp_stat *params = &stats->params; @@ -319,7 +321,7 @@ void Agc::process(IPAContext &context, [[maybe_unused]] const uint32_t frame, break; } - computeExposure(context, yGain, iqMeanGain); + computeExposure(context, frameContext, yGain, iqMeanGain); frameCount_++; } @@ -327,9 +329,11 @@ void Agc::process(IPAContext &context, [[maybe_unused]] const uint32_t frame, * \copydoc libcamera::ipa::Algorithm::prepare */ void Agc::prepare(IPAContext &context, const uint32_t frame, - [[maybe_unused]] RkISP1FrameContext &frameContext, - rkisp1_params_cfg *params) + RkISP1FrameContext &frameContext, rkisp1_params_cfg *params) { + frameContext.agc.exposure = context.activeState.agc.exposure; + frameContext.agc.gain = context.activeState.agc.gain; + if (frame > 0) return; diff --git a/src/ipa/rkisp1/algorithms/agc.h b/src/ipa/rkisp1/algorithms/agc.h index be8932040c8e..d6c6fb130fc1 100644 --- a/src/ipa/rkisp1/algorithms/agc.h +++ b/src/ipa/rkisp1/algorithms/agc.h @@ -34,7 +34,8 @@ public: const rkisp1_stat_buffer *stats) override; private: - void computeExposure(IPAContext &Context, double yGain, double iqMeanGain); + void computeExposure(IPAContext &Context, RkISP1FrameContext &frameContext, + double yGain, double iqMeanGain); utils::Duration filterExposure(utils::Duration exposureValue); double estimateLuminance(const rkisp1_cif_isp_ae_stat *ae, double gain); double measureBrightness(const rkisp1_cif_isp_hist_stat *hist) const; diff --git a/src/ipa/rkisp1/ipa_context.cpp b/src/ipa/rkisp1/ipa_context.cpp index 211428717838..88e0631c8d39 100644 --- a/src/ipa/rkisp1/ipa_context.cpp +++ b/src/ipa/rkisp1/ipa_context.cpp @@ -104,16 +104,13 @@ namespace libcamera::ipa::rkisp1 { * \var IPAActiveState::agc * \brief State for the Automatic Gain Control algorithm * - * The exposure and gain determined are expected to be applied to the sensor - * at the earliest opportunity. + * The exposure and gain are the latest values computed by the AGC algorithm. * * \var IPAActiveState::agc.exposure * \brief Exposure time expressed as a number of lines * * \var IPAActiveState::agc.gain * \brief Analogue gain multiplier - * - * The gain should be adapted to the sensor specific gain code before applying. */ /** @@ -181,17 +178,6 @@ namespace libcamera::ipa::rkisp1 { * \brief Indicates if ISP parameters need to be updated */ -/** - * \var IPAActiveState::sensor - * \brief Effective sensor values - * - * \var IPAActiveState::sensor.exposure - * \brief Exposure time expressed as a number of lines - * - * \var IPAActiveState::sensor.gain - * \brief Analogue gain multiplier - */ - /** * \struct RkISP1FrameContext * \brief Per-frame context for algorithms @@ -199,6 +185,33 @@ namespace libcamera::ipa::rkisp1 { * \todo Populate the frame context for all algorithms */ +/** + * \var RkISP1FrameContext::agc + * \brief Automatic Gain Control parameters for this frame + * + * The exposure and gain are provided by the AGC algorithm, and are to be + * applied to the sensor in order to take effect for this frame. + * + * \var RkISP1FrameContext::agc.exposure + * \brief Exposure time expressed as a number of lines + * + * \var RkISP1FrameContext::agc.gain + * \brief Analogue gain multiplier + * + * The gain should be adapted to the sensor specific gain code before applying. + */ + +/** + * \var RkISP1FrameContext::sensor + * \brief Sensor configuration that used been used for this frame + * + * \var RkISP1FrameContext::sensor.exposure + * \brief Exposure time expressed as a number of lines + * + * \var RkISP1FrameContext::sensor.gain + * \brief Analogue gain multiplier + */ + /** * \struct IPAContext * \brief Global IPA context data shared between all algorithms diff --git a/src/ipa/rkisp1/ipa_context.h b/src/ipa/rkisp1/ipa_context.h index 4e38ef40ed3c..ecf993cd22d7 100644 --- a/src/ipa/rkisp1/ipa_context.h +++ b/src/ipa/rkisp1/ipa_context.h @@ -82,14 +82,18 @@ struct IPAActiveState { uint8_t sharpness; bool updateParams; } filter; - - struct { - uint32_t exposure; - double gain; - } sensor; }; struct RkISP1FrameContext : public FrameContext { + struct { + uint32_t exposure; + double gain; + } agc; + + struct { + uint32_t exposure; + double gain; + } sensor; }; struct IPAContext { diff --git a/src/ipa/rkisp1/rkisp1.cpp b/src/ipa/rkisp1/rkisp1.cpp index 4f6e0b0f87b9..a23cfc7fb24d 100644 --- a/src/ipa/rkisp1/rkisp1.cpp +++ b/src/ipa/rkisp1/rkisp1.cpp @@ -334,9 +334,9 @@ void IPARkISP1::processStatsBuffer(const uint32_t frame, const uint32_t bufferId reinterpret_cast( mappedBuffers_.at(bufferId).planes()[0].data()); - context_.activeState.sensor.exposure = + frameContext.sensor.exposure = sensorControls.get(V4L2_CID_EXPOSURE).get(); - context_.activeState.sensor.gain = + frameContext.sensor.gain = camHelper_->gain(sensorControls.get(V4L2_CID_ANALOGUE_GAIN).get()); unsigned int aeState = 0; @@ -351,8 +351,9 @@ void IPARkISP1::processStatsBuffer(const uint32_t frame, const uint32_t bufferId void IPARkISP1::setControls(unsigned int frame) { - uint32_t exposure = context_.activeState.agc.exposure; - uint32_t gain = camHelper_->gainCode(context_.activeState.agc.gain); + RkISP1FrameContext &frameContext = context_.frameContexts.get(frame); + uint32_t exposure = frameContext.agc.exposure; + uint32_t gain = camHelper_->gainCode(frameContext.agc.gain); ControlList ctrls(ctrls_); ctrls.set(V4L2_CID_EXPOSURE, static_cast(exposure)); From patchwork Thu Sep 8 01:41:48 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Laurent Pinchart X-Patchwork-Id: 17329 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id C926AC327D for ; Thu, 8 Sep 2022 01:42:48 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 82A33620E8; Thu, 8 Sep 2022 03:42:48 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1662601368; bh=m4uVtVyQVnfTsDWeTz9KqdBImnz4wr5OVhCYAgdQSI0=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To: From; b=dOWdT+xt//He4HBwuU2yEFEBORdk1ONliotffr+gF5S/lXjEJ4xlxfc0SpG5SyKjw y0DHAkUXEfUf29YBuxULgU1KnlcMITtxGAV+BkhGLWAdlD0zdBEIObVCGBdaihv5dQ EGdxqdljZTAlS04OB7g0XlkOflCKCdot5oDXFAuxbzM4+jmVybEXr6gjW8tGjnAiC0 0Y3ZHivmxV/yIy1eFKhwxw0M6AJ6CBj2OAkS9kCamDWD7NODHlyurg62qKMux7u0+Y rSARfvliyXslseWKfkFjnWXxUM8BLYQ+MQC2nQKbuQhseZtobN+9e9WJEQ03OkVkvL cI9xvs87pWuiw== Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 0F7D3620DD for ; Thu, 8 Sep 2022 03:42:46 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="wSsCvzFe"; dkim-atps=neutral Received: from pendragon.ideasonboard.com (62-78-145-57.bb.dnainternet.fi [62.78.145.57]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 8AE3D888 for ; Thu, 8 Sep 2022 03:42:45 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1662601365; bh=m4uVtVyQVnfTsDWeTz9KqdBImnz4wr5OVhCYAgdQSI0=; h=From:To:Subject:Date:In-Reply-To:References:From; b=wSsCvzFeQo5njJipCFgXM4zRaLUCzRH/OFN2EAIKjpyEgUr3qqNkTbW5CNvXB6zhP +45OO66YoLouI/WYSuDtlrlm0wS0oRVpG8UkCffATzcu9yi0n8XrdsEekZbN4ENtpk 4o4ACQF0BZ4pHTaQ9dWw/mVmeuktoxi/CB97rFSo= To: libcamera-devel@lists.libcamera.org Date: Thu, 8 Sep 2022 04:41:48 +0300 Message-Id: <20220908014200.28728-21-laurent.pinchart@ideasonboard.com> X-Mailer: git-send-email 2.35.1 In-Reply-To: <20220908014200.28728-1-laurent.pinchart@ideasonboard.com> References: <20220908014200.28728-1-laurent.pinchart@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v4 20/32] ipa: rkisp1: awb: Store per-frame information in frame context X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Laurent Pinchart via libcamera-devel From: Laurent Pinchart Reply-To: Laurent Pinchart Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Rework the algorithm's usage of the active state and frame context to store data in the right place. The active state stores two distinct categories of information: - The consolidated value of all algorithm controls. Requests passed to the queueRequest() function store values for controls that the application wants to modify for that particular frame, and the queueRequest() function updates the active state with those values. The active state thus contains a consolidated view of the value of all controls handled by the algorithm. - The value of parameters computed by the algorithm when running in auto mode. Algorithms running in auto mode compute new parameters every time statistics buffers are received (either synchronously, or possibly in a background thread). The latest computed value of those parameters is stored in the active state in the process() function. The frame context also stores two categories of information: - The value of the controls to be applied to the frame. These values are typically set in the queueRequest() function, from the consolidated control values stored in the active state. The frame context thus stores values for all controls related to the algorithm, not limited to the controls specified in the corresponding request, but consolidated from all requests that have been queued so far. For controls that can be specified manually or computed by the algorithm depending on the operation mode (such as the colour gains), the control value will be stored in the frame context in queueRequest() only when operating in manual mode. When operating in auto mode, the values are computed by the algorithm and stored in the frame context in prepare(), just before being stored in the ISP parameters buffer. The queueRequest() function can also store ancillary data in the frame context, such as flags to indicate if (and what) control values have changed compared to the previous request. - Status information computed by the algorithm for a frame. For instance, the colour temperature estimated by the algorithm from ISP statistics calculated on a frame is stored in the frame context for that frame in the process() function. The active state and frame context thus both contain identical members for most control values, but store values that have a different meaning. Signed-off-by: Laurent Pinchart Reviewed-by: Kieran Bingham --- src/ipa/rkisp1/algorithms/awb.cpp | 75 +++++++++++++++++++------------ src/ipa/rkisp1/ipa_context.cpp | 51 +++++++++++++++++---- src/ipa/rkisp1/ipa_context.h | 25 +++++++++-- 3 files changed, 110 insertions(+), 41 deletions(-) diff --git a/src/ipa/rkisp1/algorithms/awb.cpp b/src/ipa/rkisp1/algorithms/awb.cpp index a0bbe5cb8afa..bb0f6c27fc7d 100644 --- a/src/ipa/rkisp1/algorithms/awb.cpp +++ b/src/ipa/rkisp1/algorithms/awb.cpp @@ -36,9 +36,12 @@ LOG_DEFINE_CATEGORY(RkISP1Awb) int Awb::configure(IPAContext &context, const IPACameraSensorInfo &configInfo) { - context.activeState.awb.gains.red = 1.0; - context.activeState.awb.gains.blue = 1.0; - context.activeState.awb.gains.green = 1.0; + context.activeState.awb.gains.manual.red = 1.0; + context.activeState.awb.gains.manual.blue = 1.0; + context.activeState.awb.gains.manual.green = 1.0; + context.activeState.awb.gains.automatic.red = 1.0; + context.activeState.awb.gains.automatic.blue = 1.0; + context.activeState.awb.gains.automatic.green = 1.0; context.activeState.awb.autoEnabled = true; /* @@ -75,13 +78,22 @@ uint32_t Awb::estimateCCT(double red, double green, double blue) * \copydoc libcamera::ipa::Algorithm::prepare */ void Awb::prepare(IPAContext &context, const uint32_t frame, - [[maybe_unused]] RkISP1FrameContext &frameContext, - rkisp1_params_cfg *params) + RkISP1FrameContext &frameContext, rkisp1_params_cfg *params) { - params->others.awb_gain_config.gain_green_b = 256 * context.activeState.awb.gains.green; - params->others.awb_gain_config.gain_blue = 256 * context.activeState.awb.gains.blue; - params->others.awb_gain_config.gain_red = 256 * context.activeState.awb.gains.red; - params->others.awb_gain_config.gain_green_r = 256 * context.activeState.awb.gains.green; + /* + * This is the latest time we can read the active state. This is the + * most up-to-date automatic values we can read. + */ + if (frameContext.awb.autoEnabled) { + frameContext.awb.gains.red = context.activeState.awb.gains.automatic.red; + frameContext.awb.gains.green = context.activeState.awb.gains.automatic.green; + frameContext.awb.gains.blue = context.activeState.awb.gains.automatic.blue; + } + + params->others.awb_gain_config.gain_green_b = 256 * frameContext.awb.gains.green; + params->others.awb_gain_config.gain_blue = 256 * frameContext.awb.gains.blue; + params->others.awb_gain_config.gain_red = 256 * frameContext.awb.gains.red; + params->others.awb_gain_config.gain_green_r = 256 * frameContext.awb.gains.green; /* Update the gains. */ params->module_cfg_update |= RKISP1_CIF_ISP_MODULE_AWB_GAIN; @@ -127,7 +139,7 @@ void Awb::prepare(IPAContext &context, const uint32_t frame, */ void Awb::queueRequest(IPAContext &context, [[maybe_unused]] const uint32_t frame, - [[maybe_unused]] RkISP1FrameContext &frameContext, + RkISP1FrameContext &frameContext, const ControlList &controls) { auto &awb = context.activeState.awb; @@ -142,21 +154,29 @@ void Awb::queueRequest(IPAContext &context, const auto &colourGains = controls.get(controls::ColourGains); if (colourGains && !awb.autoEnabled) { - awb.gains.red = (*colourGains)[0]; - awb.gains.blue = (*colourGains)[1]; + awb.gains.manual.red = (*colourGains)[0]; + awb.gains.manual.blue = (*colourGains)[1]; LOG(RkISP1Awb, Debug) - << "Set colour gains to red: " << awb.gains.red - << ", blue: " << awb.gains.blue; + << "Set colour gains to red: " << awb.gains.manual.red + << ", blue: " << awb.gains.manual.blue; + } + + frameContext.awb.autoEnabled = awb.autoEnabled; + + if (!awb.autoEnabled) { + frameContext.awb.gains.red = awb.gains.manual.red; + frameContext.awb.gains.green = 1.0; + frameContext.awb.gains.blue = awb.gains.manual.blue; } } /** * \copydoc libcamera::ipa::Algorithm::process */ -void Awb::process([[maybe_unused]] IPAContext &context, +void Awb::process(IPAContext &context, [[maybe_unused]] const uint32_t frame, - [[maybe_unused]] RkISP1FrameContext &frameCtx, + RkISP1FrameContext &frameContext, const rkisp1_stat_buffer *stats) { const rkisp1_cif_isp_stat *params = &stats->params; @@ -187,30 +207,27 @@ void Awb::process([[maybe_unused]] IPAContext &context, double greenMean = 1.1636 * yMean - 0.4045 * cbMean - 0.7949 * crMean; double blueMean = 1.1636 * yMean + 1.9912 * cbMean - 0.0250 * crMean; + frameContext.awb.temperatureK = estimateCCT(redMean, greenMean, blueMean); + /* Estimate the red and blue gains to apply in a grey world. */ double redGain = greenMean / (redMean + 1); double blueGain = greenMean / (blueMean + 1); /* Filter the values to avoid oscillations. */ double speed = 0.2; - redGain = speed * redGain + (1 - speed) * activeState.awb.gains.red; - blueGain = speed * blueGain + (1 - speed) * activeState.awb.gains.blue; + redGain = speed * redGain + (1 - speed) * activeState.awb.gains.automatic.red; + blueGain = speed * blueGain + (1 - speed) * activeState.awb.gains.automatic.blue; /* * Gain values are unsigned integer value, range 0 to 4 with 8 bit - * fractional part. + * fractional part. Hardcode the green gain to 1.0. */ - if (activeState.awb.autoEnabled) { - activeState.awb.gains.red = std::clamp(redGain, 0.0, 1023.0 / 256); - activeState.awb.gains.blue = std::clamp(blueGain, 0.0, 1023.0 / 256); - } - /* Hardcode the green gain to 1.0. */ - activeState.awb.gains.green = 1.0; + activeState.awb.gains.automatic.red = std::clamp(redGain, 0.0, 1023.0 / 256); + activeState.awb.gains.automatic.blue = std::clamp(blueGain, 0.0, 1023.0 / 256); + activeState.awb.gains.automatic.green = 1.0; - activeState.awb.temperatureK = estimateCCT(redMean, greenMean, blueMean); - - LOG(RkISP1Awb, Debug) << "Gain found for red: " << context.activeState.awb.gains.red - << " and for blue: " << context.activeState.awb.gains.blue; + LOG(RkISP1Awb, Debug) << "Gain found for red: " << activeState.awb.gains.automatic.red + << " and for blue: " << activeState.awb.gains.automatic.blue; } REGISTER_IPA_ALGORITHM(Awb, "Awb") diff --git a/src/ipa/rkisp1/ipa_context.cpp b/src/ipa/rkisp1/ipa_context.cpp index 88e0631c8d39..cd1443e1ed46 100644 --- a/src/ipa/rkisp1/ipa_context.cpp +++ b/src/ipa/rkisp1/ipa_context.cpp @@ -120,17 +120,29 @@ namespace libcamera::ipa::rkisp1 { * \struct IPAActiveState::awb.gains * \brief White balance gains * - * \var IPAActiveState::awb.gains.red - * \brief White balance gain for R channel + * \struct IPAActiveState::awb.gains.manual + * \brief Manual white balance gains (set through requests) * - * \var IPAActiveState::awb.gains.green - * \brief White balance gain for G channel + * \var IPAActiveState::awb.gains.manual.red + * \brief Manual white balance gain for R channel * - * \var IPAActiveState::awb.gains.blue - * \brief White balance gain for B channel + * \var IPAActiveState::awb.gains.manual.green + * \brief Manual white balance gain for G channel * - * \var IPAActiveState::awb.temperatureK - * \brief Estimated color temperature + * \var IPAActiveState::awb.gains.manual.blue + * \brief Manual white balance gain for B channel + * + * \struct IPAActiveState::awb.gains.automatic + * \brief Automatic white balance gains (computed by the algorithm) + * + * \var IPAActiveState::awb.gains.automatic.red + * \brief Automatic white balance gain for R channel + * + * \var IPAActiveState::awb.gains.automatic.green + * \brief Automatic white balance gain for G channel + * + * \var IPAActiveState::awb.gains.automatic.blue + * \brief Automatic white balance gain for B channel * * \var IPAActiveState::awb.autoEnabled * \brief Whether the Auto White Balance algorithm is enabled @@ -201,6 +213,29 @@ namespace libcamera::ipa::rkisp1 { * The gain should be adapted to the sensor specific gain code before applying. */ +/** + * \var RkISP1FrameContext::awb + * \brief Automatic White Balance parameters for this frame + * + * \struct RkISP1FrameContext::awb.gains + * \brief White balance gains + * + * \var RkISP1FrameContext::awb.gains.red + * \brief White balance gain for R channel + * + * \var RkISP1FrameContext::awb.gains.green + * \brief White balance gain for G channel + * + * \var RkISP1FrameContext::awb.gains.blue + * \brief White balance gain for B channel + * + * \var RkISP1FrameContext::awb.temperatureK + * \brief Estimated color temperature + * + * \var RkISP1FrameContext::awb.autoEnabled + * \brief Whether the Auto White Balance algorithm is enabled + */ + /** * \var RkISP1FrameContext::sensor * \brief Sensor configuration that used been used for this frame diff --git a/src/ipa/rkisp1/ipa_context.h b/src/ipa/rkisp1/ipa_context.h index ecf993cd22d7..d97aae9a97b4 100644 --- a/src/ipa/rkisp1/ipa_context.h +++ b/src/ipa/rkisp1/ipa_context.h @@ -56,12 +56,18 @@ struct IPAActiveState { struct { struct { - double red; - double green; - double blue; + struct { + double red; + double green; + double blue; + } manual; + struct { + double red; + double green; + double blue; + } automatic; } gains; - double temperatureK; bool autoEnabled; } awb; @@ -90,6 +96,17 @@ struct RkISP1FrameContext : public FrameContext { double gain; } agc; + struct { + struct { + double red; + double green; + double blue; + } gains; + + double temperatureK; + bool autoEnabled; + } awb; + struct { uint32_t exposure; double gain; From patchwork Thu Sep 8 01:41:49 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Laurent Pinchart X-Patchwork-Id: 17330 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 44A8DC3272 for ; Thu, 8 Sep 2022 01:42:49 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 0B22B620E5; Thu, 8 Sep 2022 03:42:49 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1662601369; bh=tO0gsFWQuTe0wdKJLk3/87qbHIdw/Lnh0RzFFOyT1i0=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To: From; b=ZNUhnNRggvYRmPP0bcoM9CiFtGwqk21jgexSplcxnoYcEhrh7ZPu6OVOz3bXEyXgv U7cJNzxl+nTGMU2DrdABVmC81i9LU6HW6P/JQsTAe+yarI8HvGk349MYzNI2wVQVCi nBiYZJtVP3MHSRJ1ktKQXUzCImray74GOrDUklvzGhntznuV373FcMCB3f5hcCDBbC rJx+UduARnlw2MssW3WMpw11L1JO5DFQuGqm+V9mrULrHNEecJBoxY99pxzdRwWi8W z2fgRqncDsiCDsS6ycjZfqnkGUXVuzV6Y3dj7BRwsraxENMnFUE84uLWUs1FZ3/EyK ZsdPDLnpVTn6A== Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 5D2E9620E7 for ; Thu, 8 Sep 2022 03:42:47 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="CNgF287f"; dkim-atps=neutral Received: from pendragon.ideasonboard.com (62-78-145-57.bb.dnainternet.fi [62.78.145.57]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id DE7176CC for ; Thu, 8 Sep 2022 03:42:46 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1662601367; bh=tO0gsFWQuTe0wdKJLk3/87qbHIdw/Lnh0RzFFOyT1i0=; h=From:To:Subject:Date:In-Reply-To:References:From; b=CNgF287fCCGe3O6JV3EyQin5CW/VrJ3UXULxo8Qg1hXoFrI/zOZ6yiONG22hmdyDa Zw+RArKckD/hWWMMK9aUP/18Nq3BYE4nOX04oeHARxIbhraUbJCifs5KLG/rv/iw4g Fy2Se5FCg1BhdqeiRpwT6PWuW+qI0U4nqgEh6wL0= To: libcamera-devel@lists.libcamera.org Date: Thu, 8 Sep 2022 04:41:49 +0300 Message-Id: <20220908014200.28728-22-laurent.pinchart@ideasonboard.com> X-Mailer: git-send-email 2.35.1 In-Reply-To: <20220908014200.28728-1-laurent.pinchart@ideasonboard.com> References: <20220908014200.28728-1-laurent.pinchart@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v4 21/32] ipa: rkisp1: cproc: Store per-frame information in frame context X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Laurent Pinchart via libcamera-devel From: Laurent Pinchart Reply-To: Laurent Pinchart Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Rework the algorithm's usage of the active state, to store the value of controls for the last queued request in the queueRequest() function, and store a copy of the values in the corresponding frame context. The latter is used in the prepare() function to populate the ISP parameters with values corresponding to the right frame. Signed-off-by: Laurent Pinchart Reviewed-by: Kieran Bingham Reviewed-by: Jacopo Mondi --- src/ipa/rkisp1/algorithms/cproc.cpp | 51 ++++++++++++++++++----------- src/ipa/rkisp1/ipa_context.cpp | 21 ++++++++++-- src/ipa/rkisp1/ipa_context.h | 8 ++++- 3 files changed, 56 insertions(+), 24 deletions(-) diff --git a/src/ipa/rkisp1/algorithms/cproc.cpp b/src/ipa/rkisp1/algorithms/cproc.cpp index 22a70e0b70c7..9c442cd56b3f 100644 --- a/src/ipa/rkisp1/algorithms/cproc.cpp +++ b/src/ipa/rkisp1/algorithms/cproc.cpp @@ -38,55 +38,66 @@ LOG_DEFINE_CATEGORY(RkISP1CProc) */ void ColorProcessing::queueRequest(IPAContext &context, [[maybe_unused]] const uint32_t frame, - [[maybe_unused]] RkISP1FrameContext &frameContext, + RkISP1FrameContext &frameContext, const ControlList &controls) { auto &cproc = context.activeState.cproc; + bool update = false; const auto &brightness = controls.get(controls::Brightness); if (brightness) { - cproc.brightness = std::clamp(std::lround(*brightness * 128), -128, 127); - cproc.updateParams = true; + int value = std::clamp(std::lround(*brightness * 128), -128, 127); + if (cproc.brightness != value) { + cproc.brightness = value; + update = true; + } - LOG(RkISP1CProc, Debug) << "Set brightness to " << *brightness; + LOG(RkISP1CProc, Debug) << "Set brightness to " << value; } const auto &contrast = controls.get(controls::Contrast); if (contrast) { - cproc.contrast = std::clamp(std::lround(*contrast * 128), 0, 255); - cproc.updateParams = true; + int value = std::clamp(std::lround(*contrast * 128), 0, 255); + if (cproc.contrast != value) { + cproc.contrast = value; + update = true; + } - LOG(RkISP1CProc, Debug) << "Set contrast to " << *contrast; + LOG(RkISP1CProc, Debug) << "Set contrast to " << value; } const auto saturation = controls.get(controls::Saturation); if (saturation) { - cproc.saturation = std::clamp(std::lround(*saturation * 128), 0, 255); - cproc.updateParams = true; + int value = std::clamp(std::lround(*saturation * 128), 0, 255); + if (cproc.saturation != value) { + cproc.saturation = value; + update = true; + } - LOG(RkISP1CProc, Debug) << "Set saturation to " << *saturation; + LOG(RkISP1CProc, Debug) << "Set saturation to " << value; } + + frameContext.cproc.brightness = cproc.brightness; + frameContext.cproc.contrast = cproc.contrast; + frameContext.cproc.saturation = cproc.saturation; + frameContext.cproc.update = update; } /** * \copydoc libcamera::ipa::Algorithm::prepare */ -void ColorProcessing::prepare(IPAContext &context, +void ColorProcessing::prepare([[maybe_unused]] IPAContext &context, [[maybe_unused]] const uint32_t frame, - [[maybe_unused]] RkISP1FrameContext &frameContext, + RkISP1FrameContext &frameContext, rkisp1_params_cfg *params) { - auto &cproc = context.activeState.cproc; - /* Check if the algorithm configuration has been updated. */ - if (!cproc.updateParams) + if (!frameContext.cproc.update) return; - cproc.updateParams = false; - - params->others.cproc_config.brightness = cproc.brightness; - params->others.cproc_config.contrast = cproc.contrast; - params->others.cproc_config.sat = cproc.saturation; + params->others.cproc_config.brightness = frameContext.cproc.brightness; + params->others.cproc_config.contrast = frameContext.cproc.contrast; + params->others.cproc_config.sat = frameContext.cproc.saturation; params->module_en_update |= RKISP1_CIF_ISP_MODULE_CPROC; params->module_ens |= RKISP1_CIF_ISP_MODULE_CPROC; diff --git a/src/ipa/rkisp1/ipa_context.cpp b/src/ipa/rkisp1/ipa_context.cpp index cd1443e1ed46..936b77709417 100644 --- a/src/ipa/rkisp1/ipa_context.cpp +++ b/src/ipa/rkisp1/ipa_context.cpp @@ -160,9 +160,6 @@ namespace libcamera::ipa::rkisp1 { * * \var IPAActiveState::cproc.saturation * \brief Saturation level - * - * \var IPAActiveState::cproc.updateParams - * \brief Indicates if ISP parameters need to be updated */ /** @@ -236,6 +233,24 @@ namespace libcamera::ipa::rkisp1 { * \brief Whether the Auto White Balance algorithm is enabled */ +/** + * \var RkISP1FrameContext::cproc + * \brief Color Processing parameters for this frame + * + * \struct RkISP1FrameContext::cproc.brightness + * \brief Brightness level + * + * \var RkISP1FrameContext::cproc.contrast + * \brief Contrast level + * + * \var RkISP1FrameContext::cproc.saturation + * \brief Saturation level + * + * \var RkISP1FrameContext::cproc.update + * \brief Indicates if the color processing parameters have been updated + * compared to the previous frame + */ + /** * \var RkISP1FrameContext::sensor * \brief Sensor configuration that used been used for this frame diff --git a/src/ipa/rkisp1/ipa_context.h b/src/ipa/rkisp1/ipa_context.h index d97aae9a97b4..78edb607d039 100644 --- a/src/ipa/rkisp1/ipa_context.h +++ b/src/ipa/rkisp1/ipa_context.h @@ -75,7 +75,6 @@ struct IPAActiveState { int8_t brightness; uint8_t contrast; uint8_t saturation; - bool updateParams; } cproc; struct { @@ -107,6 +106,13 @@ struct RkISP1FrameContext : public FrameContext { bool autoEnabled; } awb; + struct { + int8_t brightness; + uint8_t contrast; + uint8_t saturation; + bool update; + } cproc; + struct { uint32_t exposure; double gain; From patchwork Thu Sep 8 01:41:50 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Laurent Pinchart X-Patchwork-Id: 17331 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id CEAC5C3272 for ; Thu, 8 Sep 2022 01:42:51 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 7EE79620F2; Thu, 8 Sep 2022 03:42:51 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1662601371; bh=jn9Ivw3zGegi1jns95YpkXteO+PkORxCdYTrqghI6UM=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To: From; b=tlqgEYPPPeAbDgpqEWE3d+JlboMlBUfKeHfTZ9SI51h4nypbNvmfHtcDHph7asK9Q kVLoZ7hKt/oQ9bdGFz1tpmE020R2bP3PSiZp+o9Hn76Ko+jDSYOXqN3ZbAfADTBZUo /XyfpIkRXVfRVVWwcgLd3SW8soKA9zFfM1iPm2DO6BKtvjyGEH16UNpd6dQz6CC47+ JxBT2isz0jWkxM46ujbQwGJ1OLudXOM//YhppwvaI9UuWwaXzyw37WXophj2hIT6jm YzocJUrqgurDEvvCuQnIMYoQrPKaDJkbUE6uie/rX4nKbXIqbNNdVl5eyp8fyP0BcD zvigyZgrpM8mw== Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id B6D33620E1 for ; Thu, 8 Sep 2022 03:42:48 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="P0OGZ3JE"; dkim-atps=neutral Received: from pendragon.ideasonboard.com (62-78-145-57.bb.dnainternet.fi [62.78.145.57]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 4408D888 for ; Thu, 8 Sep 2022 03:42:48 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1662601368; bh=jn9Ivw3zGegi1jns95YpkXteO+PkORxCdYTrqghI6UM=; h=From:To:Subject:Date:In-Reply-To:References:From; b=P0OGZ3JEr8HvsgH2FrRDYBREnGJLYWLPCz5KtEAHxJvPW/dSR154iwmhE8oxpeJhe VeRNpJXHzjpeBi13EG+n5ZbpoC3L8sM/gmOdyVnqayaQeswR4oUYwpvO/0dLOVNqsD bKG9yj+xUD1JDNcyJ8O4G5I4Mn0cnrBlQFwAXrVY= To: libcamera-devel@lists.libcamera.org Date: Thu, 8 Sep 2022 04:41:50 +0300 Message-Id: <20220908014200.28728-23-laurent.pinchart@ideasonboard.com> X-Mailer: git-send-email 2.35.1 In-Reply-To: <20220908014200.28728-1-laurent.pinchart@ideasonboard.com> References: <20220908014200.28728-1-laurent.pinchart@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v4 22/32] ipa: rkisp1: dpf: Store per-frame information in frame context X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Laurent Pinchart via libcamera-devel From: Laurent Pinchart Reply-To: Laurent Pinchart Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Rework the algorithm's usage of the active state, to store the value of controls for the last queued request in the queueRequest() function, and store a copy of the values in the corresponding frame context. The latter is used in the prepare() function to populate the ISP parameters with values corresponding to the right frame. Signed-off-by: Laurent Pinchart Reviewed-by: Kieran Bingham Reviewed-by: Jacopo Mondi --- src/ipa/rkisp1/algorithms/dpf.cpp | 22 +++++++++++----------- src/ipa/rkisp1/ipa_context.cpp | 15 ++++++++++++--- src/ipa/rkisp1/ipa_context.h | 6 +++++- 3 files changed, 28 insertions(+), 15 deletions(-) diff --git a/src/ipa/rkisp1/algorithms/dpf.cpp b/src/ipa/rkisp1/algorithms/dpf.cpp index 94c35c36570c..5d44480c5059 100644 --- a/src/ipa/rkisp1/algorithms/dpf.cpp +++ b/src/ipa/rkisp1/algorithms/dpf.cpp @@ -176,10 +176,11 @@ int Dpf::init([[maybe_unused]] IPAContext &context, */ void Dpf::queueRequest(IPAContext &context, [[maybe_unused]] const uint32_t frame, - [[maybe_unused]] RkISP1FrameContext &frameContext, + RkISP1FrameContext &frameContext, const ControlList &controls) { auto &dpf = context.activeState.dpf; + bool update = false; const auto &denoise = controls.get(controls::draft::NoiseReductionMode); if (denoise) { @@ -188,34 +189,35 @@ void Dpf::queueRequest(IPAContext &context, switch (*denoise) { case controls::draft::NoiseReductionModeOff: dpf.denoise = false; - dpf.updateParams = true; + update = true; break; case controls::draft::NoiseReductionModeMinimal: case controls::draft::NoiseReductionModeHighQuality: case controls::draft::NoiseReductionModeFast: dpf.denoise = true; - dpf.updateParams = true; + update = true; break; default: LOG(RkISP1Dpf, Error) << "Unsupported denoise value " << *denoise; + break; } } + + frameContext.dpf.denoise = dpf.denoise; + frameContext.dpf.update = update; } /** * \copydoc libcamera::ipa::Algorithm::prepare */ void Dpf::prepare(IPAContext &context, const uint32_t frame, - [[maybe_unused]] RkISP1FrameContext &frameContext, - rkisp1_params_cfg *params) + RkISP1FrameContext &frameContext, rkisp1_params_cfg *params) { if (!initialized_) return; - auto &dpf = context.activeState.dpf; - if (frame == 0) { params->others.dpf_config = config_; params->others.dpf_strength_config = strengthConfig_; @@ -245,12 +247,10 @@ void Dpf::prepare(IPAContext &context, const uint32_t frame, RKISP1_CIF_ISP_MODULE_DPF_STRENGTH; } - if (dpf.updateParams) { + if (frameContext.dpf.update) { params->module_en_update |= RKISP1_CIF_ISP_MODULE_DPF; - if (dpf.denoise) + if (frameContext.dpf.denoise) params->module_ens |= RKISP1_CIF_ISP_MODULE_DPF; - - dpf.updateParams = false; } } diff --git a/src/ipa/rkisp1/ipa_context.cpp b/src/ipa/rkisp1/ipa_context.cpp index 936b77709417..b0210a978559 100644 --- a/src/ipa/rkisp1/ipa_context.cpp +++ b/src/ipa/rkisp1/ipa_context.cpp @@ -168,9 +168,6 @@ namespace libcamera::ipa::rkisp1 { * * \var IPAActiveState::dpf.denoise * \brief Indicates if denoise is activated - * - * \var IPAActiveState::dpf.updateParams - * \brief Indicates if ISP parameters need to be updated */ /** @@ -251,6 +248,18 @@ namespace libcamera::ipa::rkisp1 { * compared to the previous frame */ +/** + * \var RkISP1FrameContext::dpf + * \brief Denoise Pre-Filter parameters for this frame + * + * \var RkISP1FrameContext::dpf.denoise + * \brief Indicates if denoise is activated + * + * \var RkISP1FrameContext::dpf.update + * \brief Indicates if the denoise pre-filter parameters have been updated + * compared to the previous frame + */ + /** * \var RkISP1FrameContext::sensor * \brief Sensor configuration that used been used for this frame diff --git a/src/ipa/rkisp1/ipa_context.h b/src/ipa/rkisp1/ipa_context.h index 78edb607d039..c22e1f099c23 100644 --- a/src/ipa/rkisp1/ipa_context.h +++ b/src/ipa/rkisp1/ipa_context.h @@ -79,7 +79,6 @@ struct IPAActiveState { struct { bool denoise; - bool updateParams; } dpf; struct { @@ -113,6 +112,11 @@ struct RkISP1FrameContext : public FrameContext { bool update; } cproc; + struct { + bool denoise; + bool update; + } dpf; + struct { uint32_t exposure; double gain; From patchwork Thu Sep 8 01:41:51 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Laurent Pinchart X-Patchwork-Id: 17332 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id A5CCDC327D for ; Thu, 8 Sep 2022 01:42:52 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 67006620EB; Thu, 8 Sep 2022 03:42:52 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1662601372; bh=9ZhAZnf6lr0krKmTVy/pD4FI0CBNPWwai1Lq3wgarBE=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To: From; b=B211SLpB/WgiFuD6rfZL1Kh3LQSCsiItxfxPvjtuHzmCSs0DZv3x/Tg/NfT+E7ogP WlsIuixj7UtE8359lgqTJYxyo+Ga1T0IU78QWoJyRY9dZVKrBOVRJWMPZCS/zvuL/Z hPdyvtbhZUROFemX2vv6jGN35q94KjSkgTitHWN4AO6ayBMF3oNR/axw7dUQXG0Wzu pZtsv2ByJG1a0y8KWQPSy2Tq1yRUeOFANgYiTj0oE6pqXGpL9sHLZuXVdD20SPqL3Y TMhJ79JBnD1YKm0+CZ8QyJiGyrgVTriSz9+AiXVOC8OxkebPY39PBfJ4Qg8+1PAtnD U84qtgk8hNLzw== Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 24DA1620E1 for ; Thu, 8 Sep 2022 03:42:50 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="nMfWpzuz"; dkim-atps=neutral Received: from pendragon.ideasonboard.com (62-78-145-57.bb.dnainternet.fi [62.78.145.57]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id A456C8F1 for ; Thu, 8 Sep 2022 03:42:49 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1662601369; bh=9ZhAZnf6lr0krKmTVy/pD4FI0CBNPWwai1Lq3wgarBE=; h=From:To:Subject:Date:In-Reply-To:References:From; b=nMfWpzuzHhaf9kIosHbXyI2ZwnIMpZxgQ6GbTTZC0IYNtnLY5PWOY2cBRUGLMrdVG u9FWFcnbXy3I/kvhKZuQ9SAila+SrzveYeMGOsxLGpoWqqpK2AccCt1hDA2eyZ8CjP +5BivHjY3qV7akuKsj3Nm7thas4LUCuhygRHQnhI= To: libcamera-devel@lists.libcamera.org Date: Thu, 8 Sep 2022 04:41:51 +0300 Message-Id: <20220908014200.28728-24-laurent.pinchart@ideasonboard.com> X-Mailer: git-send-email 2.35.1 In-Reply-To: <20220908014200.28728-1-laurent.pinchart@ideasonboard.com> References: <20220908014200.28728-1-laurent.pinchart@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v4 23/32] ipa: rkisp1: filter: Store per-frame information in frame context X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Laurent Pinchart via libcamera-devel From: Laurent Pinchart Reply-To: Laurent Pinchart Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Rework the algorithm's usage of the active state, to store the value of controls for the last queued request in the queueRequest() function, and store a copy of the values in the corresponding frame context. The latter is used in the prepare() function to populate the ISP parameters with values corresponding to the right frame. Signed-off-by: Laurent Pinchart Reviewed-by: Kieran Bingham Reviewed-by: Jacopo Mondi --- src/ipa/rkisp1/algorithms/filter.cpp | 30 +++++++++++++++------------- src/ipa/rkisp1/ipa_context.cpp | 18 ++++++++++++++--- src/ipa/rkisp1/ipa_context.h | 7 ++++++- 3 files changed, 37 insertions(+), 18 deletions(-) diff --git a/src/ipa/rkisp1/algorithms/filter.cpp b/src/ipa/rkisp1/algorithms/filter.cpp index e64bd6a6d68f..ac743729e247 100644 --- a/src/ipa/rkisp1/algorithms/filter.cpp +++ b/src/ipa/rkisp1/algorithms/filter.cpp @@ -44,15 +44,16 @@ static constexpr uint32_t kFiltModeDefault = 0x000004f2; */ void Filter::queueRequest(IPAContext &context, [[maybe_unused]] const uint32_t frame, - [[maybe_unused]] RkISP1FrameContext &frameContext, + RkISP1FrameContext &frameContext, const ControlList &controls) { auto &filter = context.activeState.filter; + bool update = false; const auto &sharpness = controls.get(controls::Sharpness); if (sharpness) { filter.sharpness = std::round(std::clamp(*sharpness, 0.0f, 10.0f)); - filter.updateParams = true; + update = true; LOG(RkISP1Filter, Debug) << "Set sharpness to " << *sharpness; } @@ -64,41 +65,42 @@ void Filter::queueRequest(IPAContext &context, switch (*denoise) { case controls::draft::NoiseReductionModeOff: filter.denoise = 0; - filter.updateParams = true; + update = true; break; case controls::draft::NoiseReductionModeMinimal: filter.denoise = 1; - filter.updateParams = true; + update = true; break; case controls::draft::NoiseReductionModeHighQuality: case controls::draft::NoiseReductionModeFast: filter.denoise = 3; - filter.updateParams = true; + update = true; break; default: LOG(RkISP1Filter, Error) << "Unsupported denoise value " << *denoise; + break; } } + + frameContext.filter.denoise = filter.denoise; + frameContext.filter.sharpness = filter.sharpness; + frameContext.filter.update = update; } /** * \copydoc libcamera::ipa::Algorithm::prepare */ -void Filter::prepare(IPAContext &context, +void Filter::prepare([[maybe_unused]] IPAContext &context, [[maybe_unused]] const uint32_t frame, - [[maybe_unused]] RkISP1FrameContext &frameContext, + RkISP1FrameContext &frameContext, rkisp1_params_cfg *params) { - auto &filter = context.activeState.filter; - /* Check if the algorithm configuration has been updated. */ - if (!filter.updateParams) + if (!frameContext.filter.update) return; - filter.updateParams = false; - static constexpr uint16_t filt_fac_sh0[] = { 0x04, 0x07, 0x0a, 0x0c, 0x10, 0x14, 0x1a, 0x1e, 0x24, 0x2a, 0x30 }; @@ -147,8 +149,8 @@ void Filter::prepare(IPAContext &context, 0, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3 }; - uint8_t denoise = filter.denoise; - uint8_t sharpness = filter.sharpness; + uint8_t denoise = frameContext.filter.denoise; + uint8_t sharpness = frameContext.filter.sharpness; auto &flt_config = params->others.flt_config; flt_config.fac_sh0 = filt_fac_sh0[sharpness]; diff --git a/src/ipa/rkisp1/ipa_context.cpp b/src/ipa/rkisp1/ipa_context.cpp index b0210a978559..b2628ef73d49 100644 --- a/src/ipa/rkisp1/ipa_context.cpp +++ b/src/ipa/rkisp1/ipa_context.cpp @@ -179,9 +179,6 @@ namespace libcamera::ipa::rkisp1 { * * \var IPAActiveState::filter.sharpness * \brief Sharpness level - * - * \var IPAActiveState::filter.updateParams - * \brief Indicates if ISP parameters need to be updated */ /** @@ -260,6 +257,21 @@ namespace libcamera::ipa::rkisp1 { * compared to the previous frame */ +/** + * \var RkISP1FrameContext::filter + * \brief Filter parameters for this frame + * + * \struct RkISP1FrameContext::filter.denoise + * \brief Denoising level + * + * \var RkISP1FrameContext::filter.sharpness + * \brief Sharpness level + * + * \var RkISP1FrameContext::filter.updateParams + * \brief Indicates if the filter parameters have been updated compared to the + * previous frame + */ + /** * \var RkISP1FrameContext::sensor * \brief Sensor configuration that used been used for this frame diff --git a/src/ipa/rkisp1/ipa_context.h b/src/ipa/rkisp1/ipa_context.h index c22e1f099c23..c15b908afcc8 100644 --- a/src/ipa/rkisp1/ipa_context.h +++ b/src/ipa/rkisp1/ipa_context.h @@ -84,7 +84,6 @@ struct IPAActiveState { struct { uint8_t denoise; uint8_t sharpness; - bool updateParams; } filter; }; @@ -117,6 +116,12 @@ struct RkISP1FrameContext : public FrameContext { bool update; } dpf; + struct { + uint8_t denoise; + uint8_t sharpness; + bool update; + } filter; + struct { uint32_t exposure; double gain; From patchwork Thu Sep 8 01:41:52 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Laurent Pinchart X-Patchwork-Id: 17333 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 371D4C3272 for ; Thu, 8 Sep 2022 01:42:54 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id DD0FF620F1; Thu, 8 Sep 2022 03:42:53 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1662601373; bh=lir+T+wpRYOiNU4TK0RI1MEqMh5BYXbnxK6geybiTVY=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To: From; b=jc+4xEArCV/4xv9yH+1aCUaG8+WSSfPDlYF2pQxZJtgDgMUbuvTwEXGNTfbk1DUxs ogK6sYBMeUq+Wc4xZfkDqgfHH/U0rNortSz6HWdBbANTUD0uioLZk12nfGaI2gblv6 XPnI3zyxhUX/oIuNj31p73Xo/KVdfxGhYHuzgpwstIqAMeM9pxbpYScDmeh0Acshwc MOBGFvoicVtKZkA5O7DyLHt8R6gswECwryoa+/Te2DbvlG1ZXMsstiBMhZ3SdtHUye xMIjrGqY4YYBwfNI4HttGkpCPTUpR+0yCXaT/YXtqAzKykfYLiB0Qi6ONFTxm5tCza 7w3u9nbJVoy7g== Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 5C6C562094 for ; Thu, 8 Sep 2022 03:42:51 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="rhiugPjO"; dkim-atps=neutral Received: from pendragon.ideasonboard.com (62-78-145-57.bb.dnainternet.fi [62.78.145.57]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id E0C046CC for ; Thu, 8 Sep 2022 03:42:50 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1662601371; bh=lir+T+wpRYOiNU4TK0RI1MEqMh5BYXbnxK6geybiTVY=; h=From:To:Subject:Date:In-Reply-To:References:From; b=rhiugPjO9K8+o956ntxyYtszgIZ2C+1gJZymcOjY/3VQS/LvksbTQYZ4jLKFxj+mU /93w3LMnP4SgqxDfw0UjUO3jrWKM6U8gq5+FhiS3fiTt4kCeeXbgscLtlUNOh/zPqj SFbQxc98wzvGISPgvHODpuQiLb+AWi5CKGjgB5Yg= To: libcamera-devel@lists.libcamera.org Date: Thu, 8 Sep 2022 04:41:52 +0300 Message-Id: <20220908014200.28728-25-laurent.pinchart@ideasonboard.com> X-Mailer: git-send-email 2.35.1 In-Reply-To: <20220908014200.28728-1-laurent.pinchart@ideasonboard.com> References: <20220908014200.28728-1-laurent.pinchart@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v4 24/32] ipa: rkisp1: Document the active state and frame context X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Laurent Pinchart via libcamera-devel From: Laurent Pinchart Reply-To: Laurent Pinchart Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Now that data used by algorithms has been partitioned between the active state and frame context, we have a better view of the role of each of those structures. Document them appropriately. Signed-off-by: Laurent Pinchart Reviewed-by: Kieran Bingham Reviewed-by: Jacopo Mondi --- src/ipa/rkisp1/ipa_context.cpp | 55 ++++++++++++++++++++++++++++------ 1 file changed, 46 insertions(+), 9 deletions(-) diff --git a/src/ipa/rkisp1/ipa_context.cpp b/src/ipa/rkisp1/ipa_context.cpp index b2628ef73d49..335cb32c538d 100644 --- a/src/ipa/rkisp1/ipa_context.cpp +++ b/src/ipa/rkisp1/ipa_context.cpp @@ -88,16 +88,28 @@ namespace libcamera::ipa::rkisp1 { * \struct IPAActiveState * \brief Active state for algorithms * - * The active state stores algorithm-specific data that needs to be shared - * between multiple algorithms and the IPA module. It is accessible through the - * IPAContext structure. + * The active state contains all algorithm-specific data that need to be + * maintained by algorithms across frames. Unlike the session configuration, + * the active state is mutable and constantly updated by algorithms. The active + * state is accessible through the IPAContext structure. * - * \todo Split the data contained in this structure between the active state - * and the frame contexts. + * The active state stores two distinct categories of information: * - * Each of the fields in the active state belongs to either a specific - * algorithm, or to the top-level IPA module. A field may be read by any - * algorithm, but should only be written by its owner. + * - The consolidated value of all algorithm controls. Requests passed to + * the queueRequest() function store values for controls that the + * application wants to modify for that particular frame, and the + * queueRequest() function updates the active state with those values. + * The active state thus contains a consolidated view of the value of all + * controls handled by the algorithm. + * + * - The value of parameters computed by the algorithm when running in auto + * mode. Algorithms running in auto mode compute new parameters every + * time statistics buffers are received (either synchronously, or + * possibly in a background thread). The latest computed value of those + * parameters is stored in the active state in the process() function. + * + * Each of the members in the active state belongs to a specific algorithm. A + * member may be read by any algorithm, but shall only be written by its owner. */ /** @@ -185,7 +197,32 @@ namespace libcamera::ipa::rkisp1 { * \struct RkISP1FrameContext * \brief Per-frame context for algorithms * - * \todo Populate the frame context for all algorithms + * The frame context stores two distinct categories of information: + * + * - The value of the controls to be applied to the frame. These values are + * typically set in the queueRequest() function, from the consolidated + * control values stored in the active state. The frame context thus stores + * values for all controls related to the algorithm, not limited to the + * controls specified in the corresponding request, but consolidated from all + * requests that have been queued so far. + * + * For controls that can be set manually or computed by an algorithm + * (depending on the algorithm operation mode), such as for instance the + * colour gains for the AWB algorithm, the control value will be stored in + * the frame context in the queueRequest() function only when operating in + * manual mode. When operating in auto mode, the values are computed by the + * algorithm in process(), stored in the active state, and copied to the + * frame context in prepare(), just before being stored in the ISP parameters + * buffer. + * + * The queueRequest() function can also store ancillary data in the frame + * context, such as flags to indicate if (and what) control values have + * changed compared to the previous request. + * + * - Status information computed by the algorithm for a frame. For instance, + * the colour temperature estimated by the AWB algorithm from ISP statistics + * calculated on a frame is stored in the frame context for that frame in + * the process() function. */ /** From patchwork Thu Sep 8 01:41:53 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Laurent Pinchart X-Patchwork-Id: 17334 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 9B3D8C327E for ; Thu, 8 Sep 2022 01:42:54 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 463E9620FD; Thu, 8 Sep 2022 03:42:54 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1662601374; bh=q0RfxFnDYGcvT+madUJOOXtY91wIaKNuVkh+9HZ6tYE=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To: From; b=AvMM1/ulGwJ74qoBk3TDT2PPUn59HvsmiKozy4vivVyepO9GSVQlEsge8yx5ZifEk AY9mzwjThLIQ5L6FFHpvjsIoYQTAhAQ/UvtWI4eyafoP2VZ8Ao7GiVt8p454umMgji utxHykvTl/FTvACrgg2EsZhV/bw+aUqQuZPkcHA2zkAlbuk5kb3Ol2YS/lWXkxfvdE ShBbA9Q9kYaue9wYtyZ7uuPqy2ornvlfxuOLZ+sirMEfz7XRINRSY4PGvjJvWMmjt9 X8OATSEmiL8PhF91JO8VrzvmhULidNhuEPxfiV9YszctVwbPg90JGK+W1YvMEl4ttp CNqHwFJ8ZSeyQ== Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id D6827620F1 for ; Thu, 8 Sep 2022 03:42:52 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="ItjpbF9x"; dkim-atps=neutral Received: from pendragon.ideasonboard.com (62-78-145-57.bb.dnainternet.fi [62.78.145.57]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 63BC2888 for ; Thu, 8 Sep 2022 03:42:52 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1662601372; bh=q0RfxFnDYGcvT+madUJOOXtY91wIaKNuVkh+9HZ6tYE=; h=From:To:Subject:Date:In-Reply-To:References:From; b=ItjpbF9xKKLgJYFlwSyJOAW4OaP2Guv1vVuUemGIGcdT8jmpVuCqB3tCEU5zGcmqG jDvVX43BflpDmwfCZKPHulVfvp4CQHYFepFp5ok3nb6+Eq57xrl+1xkbacWifpWqau 4XntrpMelRMTVCkBHzOh7UR11LinmhzQtgQ2wE98= To: libcamera-devel@lists.libcamera.org Date: Thu, 8 Sep 2022 04:41:53 +0300 Message-Id: <20220908014200.28728-26-laurent.pinchart@ideasonboard.com> X-Mailer: git-send-email 2.35.1 In-Reply-To: <20220908014200.28728-1-laurent.pinchart@ideasonboard.com> References: <20220908014200.28728-1-laurent.pinchart@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v4 25/32] ipa: rkisp1: awb: Use frame context to fix gains calculations X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Laurent Pinchart via libcamera-devel From: Laurent Pinchart Reply-To: Laurent Pinchart Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" The AWB statistics are computed after the ISP applies the colour gains. This means that the red, green and blue means do not match the data coming directly from the sensor, but are multiplied by the colour gains that were used for the frame on which the statistics have been computed. The AWB algorithm needs to take this into account when calculating the colour gains for the next frame. Do so by dividing the means by the gains that were applied to the frame, retrieved from the frame context. Signed-off-by: Laurent Pinchart Reviewed-by: Kieran Bingham Reviewed-by: Jacopo Mondi --- src/ipa/rkisp1/algorithms/awb.cpp | 9 +++++++++ 1 file changed, 9 insertions(+) diff --git a/src/ipa/rkisp1/algorithms/awb.cpp b/src/ipa/rkisp1/algorithms/awb.cpp index bb0f6c27fc7d..b711e93b73ba 100644 --- a/src/ipa/rkisp1/algorithms/awb.cpp +++ b/src/ipa/rkisp1/algorithms/awb.cpp @@ -207,6 +207,15 @@ void Awb::process(IPAContext &context, double greenMean = 1.1636 * yMean - 0.4045 * cbMean - 0.7949 * crMean; double blueMean = 1.1636 * yMean + 1.9912 * cbMean - 0.0250 * crMean; + /* + * The ISP computes the AWB means after applying the colour gains, + * divide by the gains that were used to get the raw means from the + * sensor. + */ + redMean /= frameContext.awb.gains.red; + greenMean /= frameContext.awb.gains.green; + blueMean /= frameContext.awb.gains.blue; + frameContext.awb.temperatureK = estimateCCT(redMean, greenMean, blueMean); /* Estimate the red and blue gains to apply in a grey world. */ From patchwork Thu Sep 8 01:41:54 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Laurent Pinchart X-Patchwork-Id: 17335 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 5B72BC3272 for ; Thu, 8 Sep 2022 01:42:57 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 0A13662102; Thu, 8 Sep 2022 03:42:57 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1662601377; bh=v3IeW1SqF7Gjhs0enIUlpe53Kc7QXui/tKAfw4nrhSw=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To: From; b=FAm53P+w5h63taXMXWHltmOt9RzWtW6fS1TpSuzcnc4L6xHjaz5dZQluMOBjdEn1z 9ukhdI3TuwzZXcE63uYasMnHTx2zQuNLPDiML3FkrQvAflMdI7tvwUhfg5lA4OpHy4 FyBAN6vS+wf/XDphEfFRq2WosyLczSC+m5sb8CUNH3cPwtPNm6EEv8T6h1iajNFKxs 8L07Ywh/+I/+H7KuiAWN8luIo0v0PUStrCHX8bM4zQnUy13NCjn1Xo51jFLJfgDQxa l1nkx0Fzt1MCv6eESubze6Pebi53tGuE6PuObELtX2nGvJEsS7xbPogFp5GKMJ6lzO v3noff1ijpgkw== Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 43105620FC for ; Thu, 8 Sep 2022 03:42:54 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="cOLZlwLK"; dkim-atps=neutral Received: from pendragon.ideasonboard.com (62-78-145-57.bb.dnainternet.fi [62.78.145.57]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id BDD8C6CC for ; Thu, 8 Sep 2022 03:42:53 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1662601374; bh=v3IeW1SqF7Gjhs0enIUlpe53Kc7QXui/tKAfw4nrhSw=; h=From:To:Subject:Date:In-Reply-To:References:From; b=cOLZlwLKNy3BSlu7q1lRNe1+wjvO8+cAxXMTKy0SBjaANsw7Pfofl2qTCnru0wZTi ioL15DExJS9jPMd05Iid+PFg+wu7ZRTbHmzhylw0QbUy6dVXltqXkJFDwK3u0vD5s3 xfDhUczwmLQBYj5Q43p3DsX/IVe2P56UX/BuKNwg= To: libcamera-devel@lists.libcamera.org Date: Thu, 8 Sep 2022 04:41:54 +0300 Message-Id: <20220908014200.28728-27-laurent.pinchart@ideasonboard.com> X-Mailer: git-send-email 2.35.1 In-Reply-To: <20220908014200.28728-1-laurent.pinchart@ideasonboard.com> References: <20220908014200.28728-1-laurent.pinchart@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v4 26/32] ipa: rkisp1: awb: Store color temperature as an integer X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Laurent Pinchart via libcamera-devel From: Laurent Pinchart Reply-To: Laurent Pinchart Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" The color temperature doesn't need floating point precision, and is calculated by Awb::estimateCCT() as an unsigned integer. Store it with the same data type in the frame context. Signed-off-by: Laurent Pinchart Reviewed-by: Kieran Bingham Reviewed-by: Jacopo Mondi --- src/ipa/rkisp1/ipa_context.h | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/ipa/rkisp1/ipa_context.h b/src/ipa/rkisp1/ipa_context.h index c15b908afcc8..53787320e565 100644 --- a/src/ipa/rkisp1/ipa_context.h +++ b/src/ipa/rkisp1/ipa_context.h @@ -100,7 +100,7 @@ struct RkISP1FrameContext : public FrameContext { double blue; } gains; - double temperatureK; + unsigned int temperatureK; bool autoEnabled; } awb; From patchwork Thu Sep 8 01:41:55 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Laurent Pinchart X-Patchwork-Id: 17336 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id BCB66C327D for ; Thu, 8 Sep 2022 01:42:57 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 837F062100; Thu, 8 Sep 2022 03:42:57 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1662601377; bh=vnC1O9P9yN43lp++s9PPrSzIj+Ios4qwMqahe4dKzi4=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To:Cc: From; b=miQgbX80ktrqGe3fISBzWQ/WC3eSmZsNID7uuxEYbv03gZ6x2Ltlkmg17HuvtbOYR PHBW1PoSoaHO3f5PGhkIBPw5hTKHKCz5Sd6tMNI4zG5LuBf+CxLq3ym7CSBPaERkfN dTjzSuhn6GEHt1JZMslHQ0wEEccaD+uMqdfOVnir3OfsPezj+uRIIuvt8ABIZh/98q 6WPr0QaIWCo5tYSoBLPTQimg0uNDaVnYEkXtCQj8pG2GYvAReWVX+xwALCukTv46/G nGbw96w6Heh6kdnWtw9NIXsRBuzAgUNOdCHGEUAXrza7EjMJ+n2VLPImgquwkl6TiV zR3CDmF2jKcwQ== Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id AE836620F9 for ; Thu, 8 Sep 2022 03:42:55 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="mPuqTj6p"; dkim-atps=neutral Received: from pendragon.ideasonboard.com (62-78-145-57.bb.dnainternet.fi [62.78.145.57]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 36976888; Thu, 8 Sep 2022 03:42:55 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1662601375; bh=vnC1O9P9yN43lp++s9PPrSzIj+Ios4qwMqahe4dKzi4=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=mPuqTj6plo8aRAeg3JjPhCaOuuiTh0V4Wt5hc7wmtTjm2Y+zHYIgbMq9nDe4vQrxd xKuqkEjwD1LTyUfpi+o8ZU36VLtSH5PlvxTGwZV0gEFRcWgEsQOPmHyszr2k4Uz8+x JlViPI6oOOO0MgKUKJ4REVI3fEU15p9v8nzil62s= To: libcamera-devel@lists.libcamera.org Date: Thu, 8 Sep 2022 04:41:55 +0300 Message-Id: <20220908014200.28728-28-laurent.pinchart@ideasonboard.com> X-Mailer: git-send-email 2.35.1 In-Reply-To: <20220908014200.28728-1-laurent.pinchart@ideasonboard.com> References: <20220908014200.28728-1-laurent.pinchart@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v4 27/32] ipa: rkisp1: awb: Add support for RGB means X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Laurent Pinchart via libcamera-devel From: Laurent Pinchart Reply-To: Laurent Pinchart Cc: Quentin Schulz Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" From: Quentin Schulz RkISP actually supports two modes for color means, RGB and YCbCr. The variables where the means are stored are identically named regardless of the color means mode that's been selected. Since the gains are computed in RGB mode, a conversion needs to be done when the mode is YCbCr, which is unnecessary when RGB mode is selected. This adds support for RGB means mode too, by checking at runtime which mode is selected at a given time. The default is still set to YCbCr mode for now. Cc: Quentin Schulz Signed-off-by: Quentin Schulz Signed-off-by: Laurent Pinchart Reviewed-by: Kieran Bingham Reviewed-by: Jacopo Mondi --- Changes since v1: - Fix compilation error by storing RGB mode flag in Awb class - Set ISP AWB configuration based on the selected mode --- src/ipa/rkisp1/algorithms/awb.cpp | 130 +++++++++++++++++++----------- src/ipa/rkisp1/algorithms/awb.h | 4 +- 2 files changed, 86 insertions(+), 48 deletions(-) diff --git a/src/ipa/rkisp1/algorithms/awb.cpp b/src/ipa/rkisp1/algorithms/awb.cpp index b711e93b73ba..2ff67ed98221 100644 --- a/src/ipa/rkisp1/algorithms/awb.cpp +++ b/src/ipa/rkisp1/algorithms/awb.cpp @@ -30,6 +30,11 @@ namespace ipa::rkisp1::algorithms { LOG_DEFINE_CATEGORY(RkISP1Awb) +Awb::Awb() + : rgbMode_(false) +{ +} + /** * \copydoc libcamera::ipa::Algorithm::configure */ @@ -98,38 +103,60 @@ void Awb::prepare(IPAContext &context, const uint32_t frame, /* Update the gains. */ params->module_cfg_update |= RKISP1_CIF_ISP_MODULE_AWB_GAIN; - /* If we already have configured the gains and window, return. */ + /* If we already have set the AWB measurement parameters, return. */ if (frame > 0) return; - /* Configure the gains to apply. */ + rkisp1_cif_isp_awb_meas_config &awb_config = params->meas.awb_meas_config; + + /* Configure the measure window for AWB. */ + awb_config.awb_wnd = context.configuration.awb.measureWindow; + + /* Number of frames to use to estimate the means (0 means 1 frame). */ + awb_config.frames = 0; + + /* Select RGB or YCbCr means measurement. */ + if (rgbMode_) { + awb_config.awb_mode = RKISP1_CIF_ISP_AWB_MODE_RGB; + + /* + * For RGB-based measurements, pixels are selected with maximum + * red, green and blue thresholds that are set in the + * awb_ref_cr, awb_min_y and awb_ref_cb respectively. The other + * values are not used, set them to 0. + */ + awb_config.awb_ref_cr = 250; + awb_config.min_y = 250; + awb_config.awb_ref_cb = 250; + + awb_config.max_y = 0; + awb_config.min_c = 0; + awb_config.max_csum = 0; + } else { + awb_config.awb_mode = RKISP1_CIF_ISP_AWB_MODE_YCBCR; + + /* Set the reference Cr and Cb (AWB target) to white. */ + awb_config.awb_ref_cb = 128; + awb_config.awb_ref_cr = 128; + + /* + * Filter out pixels based on luminance and chrominance values. + * The acceptable luma values are specified as a [16, 250] + * range, while the acceptable chrome values are specified with + * a minimum of 16 and a maximum Cb+Cr sum of 250. + */ + awb_config.min_y = 16; + awb_config.max_y = 250; + awb_config.min_c = 16; + awb_config.max_csum = 250; + } + + /* Enable the AWB gains. */ params->module_en_update |= RKISP1_CIF_ISP_MODULE_AWB_GAIN; - /* Update the ISP to apply the gains configured. */ params->module_ens |= RKISP1_CIF_ISP_MODULE_AWB_GAIN; - /* Configure the measure window for AWB. */ - params->meas.awb_meas_config.awb_wnd = context.configuration.awb.measureWindow; - /* - * Measure Y, Cr and Cb means. - * \todo RGB is not working, the kernel seems to not configure it ? - */ - params->meas.awb_meas_config.awb_mode = RKISP1_CIF_ISP_AWB_MODE_YCBCR; - /* Reference Cr and Cb. */ - params->meas.awb_meas_config.awb_ref_cb = 128; - params->meas.awb_meas_config.awb_ref_cr = 128; - /* Y values to include are between min_y and max_y only. */ - params->meas.awb_meas_config.min_y = 16; - params->meas.awb_meas_config.max_y = 250; - /* Maximum Cr+Cb value to take into account for awb. */ - params->meas.awb_meas_config.max_csum = 250; - /* Minimum Cr and Cb values to take into account. */ - params->meas.awb_meas_config.min_c = 16; - /* Number of frames to use to estimate the mean (0 means 1 frame). */ - params->meas.awb_meas_config.frames = 0; - - /* Update AWB measurement unit configuration. */ + /* Update the AWB measurement parameters and enable the AWB module. */ params->module_cfg_update |= RKISP1_CIF_ISP_MODULE_AWB; - /* Make sure the ISP is measuring the means for the next frame. */ params->module_en_update |= RKISP1_CIF_ISP_MODULE_AWB; params->module_ens |= RKISP1_CIF_ISP_MODULE_AWB; } @@ -182,30 +209,39 @@ void Awb::process(IPAContext &context, const rkisp1_cif_isp_stat *params = &stats->params; const rkisp1_cif_isp_awb_stat *awb = ¶ms->awb; IPAActiveState &activeState = context.activeState; + double greenMean; + double redMean; + double blueMean; - /* Get the YCbCr mean values */ - double yMean = awb->awb_mean[0].mean_y_or_g; - double crMean = awb->awb_mean[0].mean_cr_or_r; - double cbMean = awb->awb_mean[0].mean_cb_or_b; + if (rgbMode_) { + greenMean = awb->awb_mean[0].mean_y_or_g; + redMean = awb->awb_mean[0].mean_cr_or_r; + blueMean = awb->awb_mean[0].mean_cb_or_b; + } else { + /* Get the YCbCr mean values */ + double yMean = awb->awb_mean[0].mean_y_or_g; + double cbMean = awb->awb_mean[0].mean_cb_or_b; + double crMean = awb->awb_mean[0].mean_cr_or_r; - /* - * Convert from YCbCr to RGB. - * The hardware uses the following formulas: - * Y = 16 + 0.2500 R + 0.5000 G + 0.1094 B - * Cb = 128 - 0.1406 R - 0.2969 G + 0.4375 B - * Cr = 128 + 0.4375 R - 0.3750 G - 0.0625 B - * - * The inverse matrix is thus: - * [[1,1636, -0,0623, 1,6008] - * [1,1636, -0,4045, -0,7949] - * [1,1636, 1,9912, -0,0250]] - */ - yMean -= 16; - cbMean -= 128; - crMean -= 128; - double redMean = 1.1636 * yMean - 0.0623 * cbMean + 1.6008 * crMean; - double greenMean = 1.1636 * yMean - 0.4045 * cbMean - 0.7949 * crMean; - double blueMean = 1.1636 * yMean + 1.9912 * cbMean - 0.0250 * crMean; + /* + * Convert from YCbCr to RGB. + * The hardware uses the following formulas: + * Y = 16 + 0.2500 R + 0.5000 G + 0.1094 B + * Cb = 128 - 0.1406 R - 0.2969 G + 0.4375 B + * Cr = 128 + 0.4375 R - 0.3750 G - 0.0625 B + * + * The inverse matrix is thus: + * [[1,1636, -0,0623, 1,6008] + * [1,1636, -0,4045, -0,7949] + * [1,1636, 1,9912, -0,0250]] + */ + yMean -= 16; + cbMean -= 128; + crMean -= 128; + redMean = 1.1636 * yMean - 0.0623 * cbMean + 1.6008 * crMean; + greenMean = 1.1636 * yMean - 0.4045 * cbMean - 0.7949 * crMean; + blueMean = 1.1636 * yMean + 1.9912 * cbMean - 0.0250 * crMean; + } /* * The ISP computes the AWB means after applying the colour gains, diff --git a/src/ipa/rkisp1/algorithms/awb.h b/src/ipa/rkisp1/algorithms/awb.h index 9fd156d8aff9..b1b861d930d1 100644 --- a/src/ipa/rkisp1/algorithms/awb.h +++ b/src/ipa/rkisp1/algorithms/awb.h @@ -16,7 +16,7 @@ namespace ipa::rkisp1::algorithms { class Awb : public Algorithm { public: - Awb() = default; + Awb(); ~Awb() = default; int configure(IPAContext &context, const IPACameraSensorInfo &configInfo) override; @@ -32,6 +32,8 @@ public: private: uint32_t estimateCCT(double red, double green, double blue); + + bool rgbMode_; }; } /* namespace ipa::rkisp1::algorithms */ From patchwork Thu Sep 8 01:41:56 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Laurent Pinchart X-Patchwork-Id: 17337 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 60400C3272 for ; Thu, 8 Sep 2022 01:42:59 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 101A462108; Thu, 8 Sep 2022 03:42:59 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1662601379; bh=Eq/b+TndUsp7IzNW5h5xamI3p+PT21otPxoaCrymkIQ=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To: From; b=Ok5oooLg8T2x5po0qmzw1VIdPAx+FxAr1uEWK4zWpCcRUOcMDGIt5awD6xYseY/Hh PtGSLzQ4EuCIaeaYqabRAlRQk6JAfiebCYg7SpQEXonj8GToO+LjefqQnKk5pEnuZT zTwXf2gsjTVxsWbevkWFk7JHzV+YWYQ3D5cpqMYkHM/tB96kCttPQ7ZwQ2ynD0bnuT nF6hNFjnbO3yc3F1e++W9l47vQsR5ow5+nYJMWSznxa+q2EJ0u8eGpjYXg6QyeShyO cDNU917kTj6h7Ly2QFv32WNia8fNWYZdAtjeqA5bdv2RHjkiiinaeWzlixS7CyxbSe OIuZY5X0caI1A== Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 1762A62101 for ; Thu, 8 Sep 2022 03:42:57 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="DHCZ0m9N"; dkim-atps=neutral Received: from pendragon.ideasonboard.com (62-78-145-57.bb.dnainternet.fi [62.78.145.57]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 924DA6CC for ; Thu, 8 Sep 2022 03:42:56 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1662601376; bh=Eq/b+TndUsp7IzNW5h5xamI3p+PT21otPxoaCrymkIQ=; h=From:To:Subject:Date:In-Reply-To:References:From; b=DHCZ0m9NXvAboDN8Y/gFd7o1LqAmOLgHTt6/BW+/G+3b0BMn2GBpG4j8akFSivGQh XKE5S9VdvfLu6PuzGroYe2Ijy13N5c+F6c/2JD+zWpRqmQBOsCwEULeFX6Gh859Wd1 RWs+TaJDAibqoP1elxV+EpT5p+n/grIHlHeibF/I= To: libcamera-devel@lists.libcamera.org Date: Thu, 8 Sep 2022 04:41:56 +0300 Message-Id: <20220908014200.28728-29-laurent.pinchart@ideasonboard.com> X-Mailer: git-send-email 2.35.1 In-Reply-To: <20220908014200.28728-1-laurent.pinchart@ideasonboard.com> References: <20220908014200.28728-1-laurent.pinchart@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v4 28/32] ipa: rkisp1: awb: Log means, gains and temperature in debug message X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Laurent Pinchart via libcamera-devel From: Laurent Pinchart Reply-To: Laurent Pinchart Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Extend the debug message in Awb::process() to log the means and color temperature in addition to the gains. This is useful for debugging the algorithm behaviour. While at it, set the showpoint flag to print a fixed number of digits after the decimal point, making logs more readable. Signed-off-by: Laurent Pinchart Reviewed-by: Kieran Bingham Reviewed-by: Jacopo Mondi --- src/ipa/rkisp1/algorithms/awb.cpp | 9 +++++++-- 1 file changed, 7 insertions(+), 2 deletions(-) diff --git a/src/ipa/rkisp1/algorithms/awb.cpp b/src/ipa/rkisp1/algorithms/awb.cpp index 2ff67ed98221..ed91e9277a16 100644 --- a/src/ipa/rkisp1/algorithms/awb.cpp +++ b/src/ipa/rkisp1/algorithms/awb.cpp @@ -9,6 +9,7 @@ #include #include +#include #include @@ -271,8 +272,12 @@ void Awb::process(IPAContext &context, activeState.awb.gains.automatic.blue = std::clamp(blueGain, 0.0, 1023.0 / 256); activeState.awb.gains.automatic.green = 1.0; - LOG(RkISP1Awb, Debug) << "Gain found for red: " << activeState.awb.gains.automatic.red - << " and for blue: " << activeState.awb.gains.automatic.blue; + LOG(RkISP1Awb, Debug) << std::showpoint + << "Means [" << redMean << ", " << greenMean << ", " << blueMean + << "], gains [" << activeState.awb.gains.automatic.red << ", " + << activeState.awb.gains.automatic.green << ", " + << activeState.awb.gains.automatic.blue << "], temp " + << frameContext.awb.temperatureK << "K"; } REGISTER_IPA_ALGORITHM(Awb, "Awb") From patchwork Thu Sep 8 01:41:57 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Laurent Pinchart X-Patchwork-Id: 17338 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id ECDFDC327D for ; Thu, 8 Sep 2022 01:43:00 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 9ACA16210E; Thu, 8 Sep 2022 03:43:00 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1662601380; bh=7UvmQ3qSqI6o2mLkcewEM7KdUj4KP3LM/xCTrOi3RAg=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To: From; b=aoa6Xg6eWpxq+DFRzS6N+eCE9BIT3PJjiTpuPfkinw41Qd6xQ9eiCxtpyf72uQLdm K+BKpGi/3HCcuOAdtwlXyPXN8bjGgoA10I0gXkaw0rYokbm0K/c4dEtohzSCevlkR+ ov4PNK0CJRanGQ+VjDRY6UeDuKDyGhljN5SLuwnmaKBj/bAKQK0vm//q0ayC2ID529 U7jiTCYr4lWDlMZIaKDDhXP1EcXihx4kYlfOhj4+Ars6yaH1xcNx2yDa9j/ntS0l6S Mo8JPkfxeUR/h3qp6dJXspI1Gk+z3t5MevE0Mw/Wk8pJ+Vd4KQbVAYaz35cQNX12I6 53johcu1G6D9g== Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 6C0C5620FC for ; Thu, 8 Sep 2022 03:42:58 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="tEtScMoF"; dkim-atps=neutral Received: from pendragon.ideasonboard.com (62-78-145-57.bb.dnainternet.fi [62.78.145.57]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id E362C8F1 for ; Thu, 8 Sep 2022 03:42:57 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1662601378; bh=7UvmQ3qSqI6o2mLkcewEM7KdUj4KP3LM/xCTrOi3RAg=; h=From:To:Subject:Date:In-Reply-To:References:From; b=tEtScMoFV83L2ZG2QJlkYel6G0CI4bSbKEQF57IHik5esgtmZZ9m9tLWyqDcDfDpt nNlsT1SmlqXVtb8IQ3eE6Oh5z1PVZoZ7/fPFfdl3Cg6zYeRKjfiNvWHUlnxHTVXZEC qwX9xOGHE54Ih1yKX52uGXaBeEj+xIQPWV+xRS+0= To: libcamera-devel@lists.libcamera.org Date: Thu, 8 Sep 2022 04:41:57 +0300 Message-Id: <20220908014200.28728-30-laurent.pinchart@ideasonboard.com> X-Mailer: git-send-email 2.35.1 In-Reply-To: <20220908014200.28728-1-laurent.pinchart@ideasonboard.com> References: <20220908014200.28728-1-laurent.pinchart@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v4 29/32] ipa: rkisp1: awb: Prevent RGB means from being negative X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Laurent Pinchart via libcamera-devel From: Laurent Pinchart Reply-To: Laurent Pinchart Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Due to hardware rounding errors in the YCbCr means, the calculated RGB means may be negative. This would lead to negative gains, messing up calculation. Prevent this by clamping the means to positive values. Signed-off-by: Laurent Pinchart Reviewed-by: Kieran Bingham Reviewed-by: Jacopo Mondi --- src/ipa/rkisp1/algorithms/awb.cpp | 10 ++++++++++ 1 file changed, 10 insertions(+) diff --git a/src/ipa/rkisp1/algorithms/awb.cpp b/src/ipa/rkisp1/algorithms/awb.cpp index ed91e9277a16..de54c4d24650 100644 --- a/src/ipa/rkisp1/algorithms/awb.cpp +++ b/src/ipa/rkisp1/algorithms/awb.cpp @@ -242,6 +242,16 @@ void Awb::process(IPAContext &context, redMean = 1.1636 * yMean - 0.0623 * cbMean + 1.6008 * crMean; greenMean = 1.1636 * yMean - 0.4045 * cbMean - 0.7949 * crMean; blueMean = 1.1636 * yMean + 1.9912 * cbMean - 0.0250 * crMean; + + /* + * Due to hardware rounding errors in the YCbCr means, the + * calculated RGB means may be negative. This would lead to + * negative gains, messing up calculation. Prevent this by + * clamping the means to positive values. + */ + redMean = std::max(redMean, 0.0); + greenMean = std::max(greenMean, 0.0); + blueMean = std::max(blueMean, 0.0); } /* From patchwork Thu Sep 8 01:41:58 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Laurent Pinchart X-Patchwork-Id: 17339 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 5A384C3272 for ; Thu, 8 Sep 2022 01:43:01 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 1F1566210D; Thu, 8 Sep 2022 03:43:01 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1662601381; bh=Lpem7NLXtEcHUFpWg1GO9dStYi5oKp1xPNZDA+oPcm4=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To: From; b=qRO6e/3Fs50wS76cDVmQPbNHzHirOBsVYMfUUXdRD+1uUfsCiGwYsedJalylL/hXQ 9SqSGdwb37dtrkUzwHWy8y6/UlrHlvrqi+5PBrujnYI0TEm0hBqFZ378hOcPt3OhTQ n/dlsvGbelDlCRcpZG/HVmfGmjlx32OQBGNanXmgqv+T/BpAti0hM7dUt/3FojagYR SV3QU3v+F6JWy+Kgh5/ngEJCs7zdVm9HK4VOqA+CJ+d8E3lPcXx2R0jYLLQ8aEMCnv baFMuKLEUzYLP3/g8rJZAxlQlQbfmAKQHUgIMRetvRJbt0WpAXPeuHA792a2NCjhRe j3Xs4UCwQkshg== Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id EBA4F62104 for ; Thu, 8 Sep 2022 03:42:59 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="TItHRTNl"; dkim-atps=neutral Received: from pendragon.ideasonboard.com (62-78-145-57.bb.dnainternet.fi [62.78.145.57]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 68E426CC for ; Thu, 8 Sep 2022 03:42:59 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1662601379; bh=Lpem7NLXtEcHUFpWg1GO9dStYi5oKp1xPNZDA+oPcm4=; h=From:To:Subject:Date:In-Reply-To:References:From; b=TItHRTNlsv7kTY93qvo3RhtILIABoSn5nI9YaRyHqWFVOVF8e+32wte6GHk+Sq+LQ mjPfLgzcsps4D0egttgM8j4RAAbFYGPzln79VUZ985V9mtSSl89ei9yBGHBlVkWJIz By07KW0jVcwgZYHLHGiE0e3fIPFXMHCeVkfoV70w= To: libcamera-devel@lists.libcamera.org Date: Thu, 8 Sep 2022 04:41:58 +0300 Message-Id: <20220908014200.28728-31-laurent.pinchart@ideasonboard.com> X-Mailer: git-send-email 2.35.1 In-Reply-To: <20220908014200.28728-1-laurent.pinchart@ideasonboard.com> References: <20220908014200.28728-1-laurent.pinchart@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v4 30/32] ipa: rkisp1: awb: Clamp gains to prevent divisions by zero X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Laurent Pinchart via libcamera-devel From: Laurent Pinchart Reply-To: Laurent Pinchart Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" The gain values are currently clamped to the range [0.0, 3.996] used by the hardware. A zero value makes little sense, as it would completely remove the contribution of the corresponding color channel from the AWB accumulators, but worse, would lead to divisions by zero when calculating the raw means in subsequent iterations. Prevent this by setting the minimum gain value to 1/256. While at it, clamp the gain values before filtering them, to improve the stability of the control loop. Signed-off-by: Laurent Pinchart Reviewed-by: Kieran Bingham Reviewed-by: Jacopo Mondi --- src/ipa/rkisp1/algorithms/awb.cpp | 22 +++++++++++++++------- 1 file changed, 15 insertions(+), 7 deletions(-) diff --git a/src/ipa/rkisp1/algorithms/awb.cpp b/src/ipa/rkisp1/algorithms/awb.cpp index de54c4d24650..5f2535688c93 100644 --- a/src/ipa/rkisp1/algorithms/awb.cpp +++ b/src/ipa/rkisp1/algorithms/awb.cpp @@ -265,21 +265,29 @@ void Awb::process(IPAContext &context, frameContext.awb.temperatureK = estimateCCT(redMean, greenMean, blueMean); - /* Estimate the red and blue gains to apply in a grey world. */ + /* + * Estimate the red and blue gains to apply in a grey world. The green + * gain is hardcoded to 0. + */ double redGain = greenMean / (redMean + 1); double blueGain = greenMean / (blueMean + 1); + /* + * Clamp the gain values to the hardware, which expresses gains as Q2.8 + * unsigned integer values. Set the minimum just above zero to avoid + * divisions by zero when computing the raw means in subsequent + * iterations. + */ + redGain = std::clamp(redGain, 1.0 / 256, 1023.0 / 256); + blueGain = std::clamp(blueGain, 1.0 / 256, 1023.0 / 256); + /* Filter the values to avoid oscillations. */ double speed = 0.2; redGain = speed * redGain + (1 - speed) * activeState.awb.gains.automatic.red; blueGain = speed * blueGain + (1 - speed) * activeState.awb.gains.automatic.blue; - /* - * Gain values are unsigned integer value, range 0 to 4 with 8 bit - * fractional part. Hardcode the green gain to 1.0. - */ - activeState.awb.gains.automatic.red = std::clamp(redGain, 0.0, 1023.0 / 256); - activeState.awb.gains.automatic.blue = std::clamp(blueGain, 0.0, 1023.0 / 256); + activeState.awb.gains.automatic.red = redGain; + activeState.awb.gains.automatic.blue = blueGain; activeState.awb.gains.automatic.green = 1.0; LOG(RkISP1Awb, Debug) << std::showpoint From patchwork Thu Sep 8 01:41:59 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Laurent Pinchart X-Patchwork-Id: 17340 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id EA956C3272 for ; Thu, 8 Sep 2022 01:43:03 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id A3D4362113; Thu, 8 Sep 2022 03:43:03 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1662601383; bh=FPQ0oy+GR46vpCePLbhpjzaqSYBF4YKVAzLdR6O5vcA=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To: From; b=rctIiXFSwaOrFbOSjblLonQvvj4xw8pkqsJkSVJYLg7GOBB6+vo5DdCAApyAED+F7 CP8A6nE/Yd+U+UrfIvDNLaYShLJN0A95Ng/npkkg3ipvHz9SJAZUkG1R0nc9mcKKGx 3FPqcBlYWcroQpnNsy83sRSSNSIs+9axZtuebOKmFgEltRVyUJbYe5Maxe9Zfv11xK M20YIHqjf7SYJJN68WnvmqVY3CoN9YPZ8nJ5e2QtuIdTQDdo2qm3DfV+eVw7zvR7ze qYwrTPXMnod7HXPYNzbHddwhH8KVUzQfx+GDp/8SuLRuOM49ai0FK7k/htyVOhEAVU 81H22TeltA1uQ== Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 6E85562101 for ; Thu, 8 Sep 2022 03:43:01 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="CU0RXV67"; dkim-atps=neutral Received: from pendragon.ideasonboard.com (62-78-145-57.bb.dnainternet.fi [62.78.145.57]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id EB2A76CC for ; Thu, 8 Sep 2022 03:43:00 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1662601381; bh=FPQ0oy+GR46vpCePLbhpjzaqSYBF4YKVAzLdR6O5vcA=; h=From:To:Subject:Date:In-Reply-To:References:From; b=CU0RXV67jyAwHZBzCrBB+/K/rECokn55n0uqW/7cVtwwH28lwyGzuRmV9Lz6lQC4P 26SuzaAW8Ggu5QowqQ1PsVyV1cpDGr8n0vne92bnogmUgV/ziM87C9SFQ9gsxoyx3d 5rW/0vyHGNfadAcnAQix8rrZTOt61KjVwrD4qaMA= To: libcamera-devel@lists.libcamera.org Date: Thu, 8 Sep 2022 04:41:59 +0300 Message-Id: <20220908014200.28728-32-laurent.pinchart@ideasonboard.com> X-Mailer: git-send-email 2.35.1 In-Reply-To: <20220908014200.28728-1-laurent.pinchart@ideasonboard.com> References: <20220908014200.28728-1-laurent.pinchart@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v4 31/32] ipa: rkisp1: awb: Freeze AWB when means are too small X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Laurent Pinchart via libcamera-devel From: Laurent Pinchart Reply-To: Laurent Pinchart Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" When the RGB means are too small, gains and color temperature can't be meaningfully calculated. Freeze the AWB in that case, using the previously calculated values. Signed-off-by: Laurent Pinchart Reviewed-by: Kieran Bingham --- src/ipa/rkisp1/algorithms/awb.cpp | 17 ++++++++++++++++- src/ipa/rkisp1/ipa_context.cpp | 3 +++ src/ipa/rkisp1/ipa_context.h | 1 + 3 files changed, 20 insertions(+), 1 deletion(-) diff --git a/src/ipa/rkisp1/algorithms/awb.cpp b/src/ipa/rkisp1/algorithms/awb.cpp index 5f2535688c93..404ad66e6953 100644 --- a/src/ipa/rkisp1/algorithms/awb.cpp +++ b/src/ipa/rkisp1/algorithms/awb.cpp @@ -31,6 +31,9 @@ namespace ipa::rkisp1::algorithms { LOG_DEFINE_CATEGORY(RkISP1Awb) +/* Minimum mean value below which AWB can't operate. */ +constexpr double kMeanMinThreshold = 2.0; + Awb::Awb() : rgbMode_(false) { @@ -263,7 +266,17 @@ void Awb::process(IPAContext &context, greenMean /= frameContext.awb.gains.green; blueMean /= frameContext.awb.gains.blue; - frameContext.awb.temperatureK = estimateCCT(redMean, greenMean, blueMean); + /* + * If the means are too small we don't have enough information to + * meaningfully calculate gains. Freeze the algorithm in that case. + */ + if (redMean < kMeanMinThreshold && greenMean < kMeanMinThreshold && + blueMean < kMeanMinThreshold) { + frameContext.awb.temperatureK = activeState.awb.temperatureK; + return; + } + + activeState.awb.temperatureK = estimateCCT(redMean, greenMean, blueMean); /* * Estimate the red and blue gains to apply in a grey world. The green @@ -290,6 +303,8 @@ void Awb::process(IPAContext &context, activeState.awb.gains.automatic.blue = blueGain; activeState.awb.gains.automatic.green = 1.0; + frameContext.awb.temperatureK = activeState.awb.temperatureK; + LOG(RkISP1Awb, Debug) << std::showpoint << "Means [" << redMean << ", " << greenMean << ", " << blueMean << "], gains [" << activeState.awb.gains.automatic.red << ", " diff --git a/src/ipa/rkisp1/ipa_context.cpp b/src/ipa/rkisp1/ipa_context.cpp index 335cb32c538d..9cac05bec79b 100644 --- a/src/ipa/rkisp1/ipa_context.cpp +++ b/src/ipa/rkisp1/ipa_context.cpp @@ -156,6 +156,9 @@ namespace libcamera::ipa::rkisp1 { * \var IPAActiveState::awb.gains.automatic.blue * \brief Automatic white balance gain for B channel * + * \var IPAActiveState::awb.temperatureK + * \brief Estimated color temperature + * * \var IPAActiveState::awb.autoEnabled * \brief Whether the Auto White Balance algorithm is enabled */ diff --git a/src/ipa/rkisp1/ipa_context.h b/src/ipa/rkisp1/ipa_context.h index 53787320e565..d0bc9090a8f1 100644 --- a/src/ipa/rkisp1/ipa_context.h +++ b/src/ipa/rkisp1/ipa_context.h @@ -68,6 +68,7 @@ struct IPAActiveState { } automatic; } gains; + unsigned int temperatureK; bool autoEnabled; } awb; From patchwork Thu Sep 8 01:42:00 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Laurent Pinchart X-Patchwork-Id: 17341 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 6C46CC327D for ; Thu, 8 Sep 2022 01:43:04 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 13C686209D; Thu, 8 Sep 2022 03:43:04 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1662601384; bh=cxjIAz7fxkLpqHRIBc5gwql1/d+wyi2dDUmV7Xjien8=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To: From; b=h/zTC3nfok7fB02mjWg4X83b+DPn01Pxm1UqCZC7AxaxE/9/GNfDDoCYNDVCXERks EcEfnPAuvASp2h58yQqmRzvCHbd0QijdCARg169vs0BilDRJakM4kHjAieGiOEwsA5 NtKoL12Fch7EQlHmQ5tbzT7wNdQyPShT9vmnVHEd59E3Ob+3hh9lPYt8epTzJZpNrp ColAFGHsooVtzZ1ViCjIOKV5XoFY7Nzhs4GgMqXaqVz6Ic7yVt4EYDooQ5bx9utb4C Tv9yLkkHwEU0S6eebivdZY0GTz6f12OEkTaW1BPTMpK4KKZeee9kKfKZhVyeCv89VD 8CjR858M3TdpA== Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id E6D436209D for ; Thu, 8 Sep 2022 03:43:02 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="sdB8TXB0"; dkim-atps=neutral Received: from pendragon.ideasonboard.com (62-78-145-57.bb.dnainternet.fi [62.78.145.57]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 73246888 for ; Thu, 8 Sep 2022 03:43:02 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1662601382; bh=cxjIAz7fxkLpqHRIBc5gwql1/d+wyi2dDUmV7Xjien8=; h=From:To:Subject:Date:In-Reply-To:References:From; b=sdB8TXB0VjoPe7zLwH1FirGynywDyi6rI489+Z3AIRJTGld05bsgRjC/kuFDsCEae G3AQ20QroDV1YwZcVAbI+97zIMDo1hgC4ceaVdyUIYr3nguAmfVYztjX96ioeM7cbD iC9oi18eDMHb2gx4zAPU6Lfwo5NZ9XdOpcnL0os4= To: libcamera-devel@lists.libcamera.org Date: Thu, 8 Sep 2022 04:42:00 +0300 Message-Id: <20220908014200.28728-33-laurent.pinchart@ideasonboard.com> X-Mailer: git-send-email 2.35.1 In-Reply-To: <20220908014200.28728-1-laurent.pinchart@ideasonboard.com> References: <20220908014200.28728-1-laurent.pinchart@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v4 32/32] ipa: rkisp1: awb: Remove bias from gain calculation X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Laurent Pinchart via libcamera-devel From: Laurent Pinchart Reply-To: Laurent Pinchart Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" The red and blue gains are computed by dividing the green mean by the red and blue means respectively. An offset of 1 is added to the dividers to avoid divisions by zero. This introduces a bias in the gain values. Fix it by clamping the divisors to a minimum of 1.0 instead of adding an offset. Signed-off-by: Laurent Pinchart Reviewed-by: Kieran Bingham --- src/ipa/rkisp1/algorithms/awb.cpp | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-) diff --git a/src/ipa/rkisp1/algorithms/awb.cpp b/src/ipa/rkisp1/algorithms/awb.cpp index 404ad66e6953..fe1ba09a99a2 100644 --- a/src/ipa/rkisp1/algorithms/awb.cpp +++ b/src/ipa/rkisp1/algorithms/awb.cpp @@ -280,10 +280,11 @@ void Awb::process(IPAContext &context, /* * Estimate the red and blue gains to apply in a grey world. The green - * gain is hardcoded to 0. + * gain is hardcoded to 0. Avoid divisions by zero by clamping the + * divisor to a minimum value of 1.0. */ - double redGain = greenMean / (redMean + 1); - double blueGain = greenMean / (blueMean + 1); + double redGain = greenMean / std::max(redMean, 1.0); + double blueGain = greenMean / std::max(blueMean, 1.0); /* * Clamp the gain values to the hardware, which expresses gains as Q2.8