From patchwork Tue Nov 16 16:26:11 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Laurent Pinchart X-Patchwork-Id: 14617 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 3BB51BDB1C for ; Tue, 16 Nov 2021 16:26:48 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 6676E6037A; Tue, 16 Nov 2021 17:26:46 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="DnNFoUc1"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id D1F796032C for ; Tue, 16 Nov 2021 17:26:42 +0100 (CET) Received: from localhost.localdomain (117.145-247-81.adsl-dyn.isp.belgacom.be [81.247.145.117]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 7B7A69CA for ; Tue, 16 Nov 2021 17:26:42 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1637080002; bh=JshyZB4bxYhmsY5/BvtNc33N4gSN80H5eB26x845pxc=; h=From:To:Subject:Date:In-Reply-To:References:From; b=DnNFoUc1dv+9TU5hpjlgY3bAXom3k2CpA5bpEOEJYBHY0YOqvNgkxbuEWFgZQjuqP faHrNkYIBFkw5TLvLtxIXt4QghJO8YqVSzniO4fdwfouepvXqTBnCPxMjgxJbU2JET Kl2My8zrAl+w3F3epS4BlWWjbFWdm0ABSuKGrH5g= From: Laurent Pinchart To: libcamera-devel@lists.libcamera.org Date: Tue, 16 Nov 2021 18:26:11 +0200 Message-Id: <20211116162615.27777-2-laurent.pinchart@ideasonboard.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20211116162615.27777-1-laurent.pinchart@ideasonboard.com> References: <20211116162615.27777-1-laurent.pinchart@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH 1/5] ipa: ipu3: agc: Drop kMaxLuminance constant X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" The kMaxLuminance constant is badly named, it's not a maximum luminance, but the maximum integer value output by the AWB statistics engine for per-channel averages. The constant is used in a single place, hardcoding the value is actually more readable. Signed-off-by: Laurent Pinchart Reviewed-by: Jean-Michel Hautbois Reviewed-by: Kieran Bingham --- src/ipa/ipu3/algorithms/agc.cpp | 6 +----- 1 file changed, 1 insertion(+), 5 deletions(-) diff --git a/src/ipa/ipu3/algorithms/agc.cpp b/src/ipa/ipu3/algorithms/agc.cpp index bd02c474611c..43a39ffd57d6 100644 --- a/src/ipa/ipu3/algorithms/agc.cpp +++ b/src/ipa/ipu3/algorithms/agc.cpp @@ -61,9 +61,6 @@ static constexpr double kEvGainTarget = 0.5; /* Number of frames to wait before calculating stats on minimum exposure */ static constexpr uint32_t kNumStartupFrames = 10; -/* Maximum luminance used for brightness normalization */ -static constexpr uint32_t kMaxLuminance = 255; - /* * Normalized luma value target. * @@ -298,8 +295,7 @@ double Agc::computeInitialY(IPAFrameContext &frameContext, greenSum * frameContext.awb.gains.green * .587 + blueSum * frameContext.awb.gains.blue * .114; - /* Return the normalized relative luminance. */ - return Y_sum / (grid.height * grid.width) / kMaxLuminance; + return Y_sum / (grid.height * grid.width) / 255; } /** From patchwork Tue Nov 16 16:26:12 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Laurent Pinchart X-Patchwork-Id: 14618 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 6BF89BDB1C for ; Tue, 16 Nov 2021 16:26:49 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 74E1760398; Tue, 16 Nov 2021 17:26:47 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="QGt3etA3"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id F34D46033C for ; Tue, 16 Nov 2021 17:26:42 +0100 (CET) Received: from localhost.localdomain (117.145-247-81.adsl-dyn.isp.belgacom.be [81.247.145.117]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id ABDFD1449 for ; Tue, 16 Nov 2021 17:26:42 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1637080002; bh=yikkiY1KKoBAZHU6oA+rNxIXX8MYRYJlY7WOnWpBLlU=; h=From:To:Subject:Date:In-Reply-To:References:From; b=QGt3etA3RakkQlQaTrMN0u+Trv0hDW9w2rozz9IFEFb7sjKsEY0uF1OVydlBmFJ+Y 88spzqI748Qk3CBGX3VFVx3As7pYSN16EXv2qjUFTG7ltXsQiSFU6WWzPYE2eMdHt7 ZfhAhsVE1ZwmeMMFy7GU4ufel7cWB88mkWyEu/YE= From: Laurent Pinchart To: libcamera-devel@lists.libcamera.org Date: Tue, 16 Nov 2021 18:26:12 +0200 Message-Id: <20211116162615.27777-3-laurent.pinchart@ideasonboard.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20211116162615.27777-1-laurent.pinchart@ideasonboard.com> References: <20211116162615.27777-1-laurent.pinchart@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH 2/5] ipa: ipu3: agc: Standardize vocabulary on "relative luminance" X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" The AGC computes the average relative luminance of the frame and calls the value "normalized luma", "brightness" or "initialY". The latter is the most accurate term, as the relative luminance is abbreviated Y, but the "initial" prefix isn't accurate. Standardize the vocabulary on "relative luminance" in code and comments, abbreviating it to Y when needed. While at it, rename variables to match the libcamera coding style. Signed-off-by: Laurent Pinchart --- src/ipa/ipu3/algorithms/agc.cpp | 44 ++++++++++++++++----------------- src/ipa/ipu3/algorithms/agc.h | 8 +++--- 2 files changed, 26 insertions(+), 26 deletions(-) diff --git a/src/ipa/ipu3/algorithms/agc.cpp b/src/ipa/ipu3/algorithms/agc.cpp index 43a39ffd57d6..d5840fb0aa97 100644 --- a/src/ipa/ipu3/algorithms/agc.cpp +++ b/src/ipa/ipu3/algorithms/agc.cpp @@ -62,12 +62,12 @@ static constexpr double kEvGainTarget = 0.5; static constexpr uint32_t kNumStartupFrames = 10; /* - * Normalized luma value target. + * Relative luminance target. * * It's a number that's chosen so that, when the camera points at a grey * target, the resulting image brightness is considered right. */ -static constexpr double kNormalizedLumaTarget = 0.16; +static constexpr double kRelativeLuminanceTarget = 0.16; Agc::Agc() : frameCount_(0), iqMean_(0.0), lineDuration_(0s), minShutterSpeed_(0s), @@ -250,12 +250,12 @@ void Agc::computeExposure(IPAFrameContext &frameContext, double currentYGain) } /** - * \brief Estimate the average brightness of the frame + * \brief Estimate the relative luminance of the frame with a given gain * \param[in] frameContext The shared IPA frame context * \param[in] grid The grid used to store the statistics in the IPU3 * \param[in] stats The IPU3 statistics and ISP results * \param[in] currentYGain The gain calculated on the current brightness level - * \return The normalized luma + * \return The relative luminance * * Luma is the weighted sum of gamma-compressed R′G′B′ components of a color * video. The luma values are normalized as 0.0 to 1.0, with 1.0 being a @@ -263,12 +263,12 @@ void Agc::computeExposure(IPAFrameContext &frameContext, double currentYGain) * luma here. * * More detailed information can be found in: - * https://en.wikipedia.org/wiki/Luma_(video) + * https://en.wikipedia.org/wiki/Relative_luminance */ -double Agc::computeInitialY(IPAFrameContext &frameContext, - const ipu3_uapi_grid_config &grid, - const ipu3_uapi_stats_3a *stats, - double currentYGain) +double Agc::estimateLuminance(IPAFrameContext &frameContext, + const ipu3_uapi_grid_config &grid, + const ipu3_uapi_stats_3a *stats, + double currentYGain) { double redSum = 0, greenSum = 0, blueSum = 0; @@ -288,14 +288,14 @@ double Agc::computeInitialY(IPAFrameContext &frameContext, } /* - * Estimate the sum of the brightness values, weighted with the gains - * applied on the channels in AWB as the Rec. 601 luma. + * Compute the relative luminance using the Rec. 601 formula, and + * normalize it. */ - double Y_sum = redSum * frameContext.awb.gains.red * .299 + - greenSum * frameContext.awb.gains.green * .587 + - blueSum * frameContext.awb.gains.blue * .114; + double ySum = redSum * frameContext.awb.gains.red * .299 + + greenSum * frameContext.awb.gains.green * .587 + + blueSum * frameContext.awb.gains.blue * .114; - return Y_sum / (grid.height * grid.width) / 255; + return ySum / (grid.height * grid.width) / 255; } /** @@ -311,22 +311,22 @@ void Agc::process(IPAContext &context, const ipu3_uapi_stats_3a *stats) measureBrightness(stats, context.configuration.grid.bdsGrid); double currentYGain = 1.0; - double targetY = kNormalizedLumaTarget; + double yTarget = kRelativeLuminanceTarget; /* * Do this calculation a few times as brightness increase can be * non-linear when there are saturated regions. */ - for (int i = 0; i < 8; i++) { - double initialY = computeInitialY(context.frameContext, + for (unsigned int i = 0; i < 8; i++) { + double yValue = estimateLuminance(context.frameContext, context.configuration.grid.bdsGrid, stats, currentYGain); - double extra_gain = std::min(10.0, targetY / (initialY + .001)); + double extra_gain = std::min(10.0, yTarget / (yValue + .001)); currentYGain *= extra_gain; - LOG(IPU3Agc, Debug) << "Initial Y " << initialY - << " target " << targetY - << " gives gain " << currentYGain; + LOG(IPU3Agc, Debug) << "Y value: " << yValue + << ", Y target: " << yTarget + << ", gives gain " << currentYGain; if (extra_gain < 1.01) break; } diff --git a/src/ipa/ipu3/algorithms/agc.h b/src/ipa/ipu3/algorithms/agc.h index 31c5a6e519d4..943c354a820e 100644 --- a/src/ipa/ipu3/algorithms/agc.h +++ b/src/ipa/ipu3/algorithms/agc.h @@ -35,10 +35,10 @@ private: const ipu3_uapi_grid_config &grid); void filterExposure(); void computeExposure(IPAFrameContext &frameContext, double currentYGain); - double computeInitialY(IPAFrameContext &frameContext, - const ipu3_uapi_grid_config &grid, - const ipu3_uapi_stats_3a *stats, - double currentYGain); + double estimateLuminance(IPAFrameContext &frameContext, + const ipu3_uapi_grid_config &grid, + const ipu3_uapi_stats_3a *stats, + double currentYGain); uint64_t frameCount_; From patchwork Tue Nov 16 16:26:13 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Laurent Pinchart X-Patchwork-Id: 14619 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 63EABC324E for ; Tue, 16 Nov 2021 16:26:50 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 8A35E603F9; Tue, 16 Nov 2021 17:26:48 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="Te5nJLfC"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 2D06360376 for ; Tue, 16 Nov 2021 17:26:43 +0100 (CET) Received: from localhost.localdomain (117.145-247-81.adsl-dyn.isp.belgacom.be [81.247.145.117]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id DEA4DE7 for ; Tue, 16 Nov 2021 17:26:42 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1637080002; bh=CAvTCZMzSAt/NgQqUvP7TCITy9Za6FjWuWY9gVMHEjU=; h=From:To:Subject:Date:In-Reply-To:References:From; b=Te5nJLfCQEBpedYPaeBZZHft2A0UfwCXzPomFHppdkdH8+7I78fcTcFk2yMf3jxzt IHrDKqM7Ah91gN6RQ+rzPnD877Y65eUtg8Al0bh99Q2Ovgn7quuWar5wuE5vmwOBSz HTeOXd1nXOtQMLsQpq3QryBKOEcHN+k1XzIkcVZY= From: Laurent Pinchart To: libcamera-devel@lists.libcamera.org Date: Tue, 16 Nov 2021 18:26:13 +0200 Message-Id: <20211116162615.27777-4-laurent.pinchart@ideasonboard.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20211116162615.27777-1-laurent.pinchart@ideasonboard.com> References: <20211116162615.27777-1-laurent.pinchart@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH 3/5] ipa: ipu3: agc: Rename currentYGain X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" The "current" prefix in the currentYGain variable name is confusing: - In Agc::estimateLuminance(), the variable contains the gain to be applied to the image, which is neither a "current" gain nor a "Y" gain. Rename it to "gain". - In Agc::computeExposure(), the variable contains the gain computed by the relative luminance method, so rename it to "yGain". While at it, rename variables to match the libcamera coding style. Signed-off-by: Laurent Pinchart --- src/ipa/ipu3/algorithms/agc.cpp | 32 ++++++++++++++++---------------- src/ipa/ipu3/algorithms/agc.h | 4 ++-- 2 files changed, 18 insertions(+), 18 deletions(-) diff --git a/src/ipa/ipu3/algorithms/agc.cpp b/src/ipa/ipu3/algorithms/agc.cpp index d5840fb0aa97..6aab9fd5ebb5 100644 --- a/src/ipa/ipu3/algorithms/agc.cpp +++ b/src/ipa/ipu3/algorithms/agc.cpp @@ -173,9 +173,9 @@ void Agc::filterExposure() /** * \brief Estimate the new exposure and gain values * \param[inout] frameContext The shared IPA frame Context - * \param[in] currentYGain The gain calculated on the current brightness level + * \param[in] yGain The gain calculated based on the relative luminance target */ -void Agc::computeExposure(IPAFrameContext &frameContext, double currentYGain) +void Agc::computeExposure(IPAFrameContext &frameContext, double yGain) { /* Get the effective exposure and gain applied on the sensor. */ uint32_t exposure = frameContext.sensor.exposure; @@ -189,8 +189,8 @@ void Agc::computeExposure(IPAFrameContext &frameContext, double currentYGain) */ double evGain = kEvGainTarget * knumHistogramBins / iqMean_; - if (evGain < currentYGain) - evGain = currentYGain; + if (evGain < yGain) + evGain = yGain; /* Consider within 1% of the target as correctly exposed */ if (std::abs(evGain - 1.0) < 0.01) @@ -254,7 +254,7 @@ void Agc::computeExposure(IPAFrameContext &frameContext, double currentYGain) * \param[in] frameContext The shared IPA frame context * \param[in] grid The grid used to store the statistics in the IPU3 * \param[in] stats The IPU3 statistics and ISP results - * \param[in] currentYGain The gain calculated on the current brightness level + * \param[in] gain The analogue gain to apply to the frame * \return The relative luminance * * Luma is the weighted sum of gamma-compressed R′G′B′ components of a color @@ -268,7 +268,7 @@ void Agc::computeExposure(IPAFrameContext &frameContext, double currentYGain) double Agc::estimateLuminance(IPAFrameContext &frameContext, const ipu3_uapi_grid_config &grid, const ipu3_uapi_stats_3a *stats, - double currentYGain) + double gain) { double redSum = 0, greenSum = 0, blueSum = 0; @@ -281,9 +281,9 @@ double Agc::estimateLuminance(IPAFrameContext &frameContext, &stats->awb_raw_buffer.meta_data[cellPosition] ); - redSum += cell->R_avg * currentYGain; - greenSum += (cell->Gr_avg + cell->Gb_avg) / 2 * currentYGain; - blueSum += cell->B_avg * currentYGain; + redSum += cell->R_avg * gain; + greenSum += (cell->Gr_avg + cell->Gb_avg) / 2 * gain; + blueSum += cell->B_avg * gain; } } @@ -310,7 +310,7 @@ void Agc::process(IPAContext &context, const ipu3_uapi_stats_3a *stats) { measureBrightness(stats, context.configuration.grid.bdsGrid); - double currentYGain = 1.0; + double yGain = 1.0; double yTarget = kRelativeLuminanceTarget; /* @@ -320,18 +320,18 @@ void Agc::process(IPAContext &context, const ipu3_uapi_stats_3a *stats) for (unsigned int i = 0; i < 8; i++) { double yValue = estimateLuminance(context.frameContext, context.configuration.grid.bdsGrid, - stats, currentYGain); - double extra_gain = std::min(10.0, yTarget / (yValue + .001)); + stats, yGain); + double extraGain = std::min(10.0, yTarget / (yValue + .001)); - currentYGain *= extra_gain; + yGain *= extraGain; LOG(IPU3Agc, Debug) << "Y value: " << yValue << ", Y target: " << yTarget - << ", gives gain " << currentYGain; - if (extra_gain < 1.01) + << ", gives gain " << yGain; + if (extraGain < 1.01) break; } - computeExposure(context.frameContext, currentYGain); + computeExposure(context.frameContext, yGain); frameCount_++; } diff --git a/src/ipa/ipu3/algorithms/agc.h b/src/ipa/ipu3/algorithms/agc.h index 943c354a820e..0c868d6737f1 100644 --- a/src/ipa/ipu3/algorithms/agc.h +++ b/src/ipa/ipu3/algorithms/agc.h @@ -34,11 +34,11 @@ private: void measureBrightness(const ipu3_uapi_stats_3a *stats, const ipu3_uapi_grid_config &grid); void filterExposure(); - void computeExposure(IPAFrameContext &frameContext, double currentYGain); + void computeExposure(IPAFrameContext &frameContext, double yGain); double estimateLuminance(IPAFrameContext &frameContext, const ipu3_uapi_grid_config &grid, const ipu3_uapi_stats_3a *stats, - double currentYGain); + double gain); uint64_t frameCount_; From patchwork Tue Nov 16 16:26:14 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Laurent Pinchart X-Patchwork-Id: 14620 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 42510BDB1C for ; Tue, 16 Nov 2021 16:26:51 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 9097F603BE; Tue, 16 Nov 2021 17:26:49 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="XtVRIp2M"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 54B7C6037A for ; Tue, 16 Nov 2021 17:26:43 +0100 (CET) Received: from localhost.localdomain (117.145-247-81.adsl-dyn.isp.belgacom.be [81.247.145.117]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 19B509CA for ; Tue, 16 Nov 2021 17:26:43 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1637080003; bh=ZnZ8fVY1xj8ztC8Zei3e73ubKzZ8ZUeEbhNt+nY4bPg=; h=From:To:Subject:Date:In-Reply-To:References:From; b=XtVRIp2MbIPduXfeyWbVwIIL3tG0sM77vsldQNQ1TtmKOGnDiLeMxr5l9oWLjRMq+ x7kPg4s3+Xf/xsTUwmCw55V6iGnMTdT1dSsuqsyNa+rdnc63meKwfRolR6Hpl6TwA6 aGRiwg5TmNV/NCF1PlsLUxbK1Na7cz/jpgoiXI0c= From: Laurent Pinchart To: libcamera-devel@lists.libcamera.org Date: Tue, 16 Nov 2021 18:26:14 +0200 Message-Id: <20211116162615.27777-5-laurent.pinchart@ideasonboard.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20211116162615.27777-1-laurent.pinchart@ideasonboard.com> References: <20211116162615.27777-1-laurent.pinchart@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH 4/5] ipa: ipu3: agc: Return the inter-quantile mean from measureBrightness() X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" The inter-quantile mean is a value that is computed as part of the AGC run. It doesn't need to be stored in a member variable. Return it from measureBrightness(), which makes the flow of data easier to follow. Signed-off-by: Laurent Pinchart Reviewed-by: Jean-Michel Hautbois Reviewed-by: Kieran Bingham --- src/ipa/ipu3/algorithms/agc.cpp | 53 ++++++++++++++++++--------------- src/ipa/ipu3/algorithms/agc.h | 9 +++--- 2 files changed, 33 insertions(+), 29 deletions(-) diff --git a/src/ipa/ipu3/algorithms/agc.cpp b/src/ipa/ipu3/algorithms/agc.cpp index 6aab9fd5ebb5..71398fdd96a6 100644 --- a/src/ipa/ipu3/algorithms/agc.cpp +++ b/src/ipa/ipu3/algorithms/agc.cpp @@ -70,7 +70,7 @@ static constexpr uint32_t kNumStartupFrames = 10; static constexpr double kRelativeLuminanceTarget = 0.16; Agc::Agc() - : frameCount_(0), iqMean_(0.0), lineDuration_(0s), minShutterSpeed_(0s), + : frameCount_(0), lineDuration_(0s), minShutterSpeed_(0s), maxShutterSpeed_(0s), filteredExposure_(0s), currentExposure_(0s) { } @@ -108,9 +108,10 @@ int Agc::configure(IPAContext &context, const IPAConfigInfo &configInfo) * \brief Estimate the mean value of the top 2% of the histogram * \param[in] stats The statistics computed by the ImgU * \param[in] grid The grid used to store the statistics in the IPU3 + * \return The mean value of the top 2% of the histogram */ -void Agc::measureBrightness(const ipu3_uapi_stats_3a *stats, - const ipu3_uapi_grid_config &grid) +double Agc::measureBrightness(const ipu3_uapi_stats_3a *stats, + const ipu3_uapi_grid_config &grid) const { /* Initialise the histogram array */ uint32_t hist[knumHistogramBins] = { 0 }; @@ -135,8 +136,8 @@ void Agc::measureBrightness(const ipu3_uapi_stats_3a *stats, } } - /* Estimate the quantile mean of the top 2% of the histogram */ - iqMean_ = Histogram(Span(hist)).interQuantileMean(0.98, 1.0); + /* Estimate the quantile mean of the top 2% of the histogram. */ + return Histogram(Span(hist)).interQuantileMean(0.98, 1.0); } /** @@ -174,28 +175,22 @@ void Agc::filterExposure() * \brief Estimate the new exposure and gain values * \param[inout] frameContext The shared IPA frame Context * \param[in] yGain The gain calculated based on the relative luminance target + * \param[in] iqMeanGain The gain calculated based on the relative luminance target */ -void Agc::computeExposure(IPAFrameContext &frameContext, double yGain) +void Agc::computeExposure(IPAFrameContext &frameContext, double yGain, + double iqMeanGain) { /* Get the effective exposure and gain applied on the sensor. */ uint32_t exposure = frameContext.sensor.exposure; double analogueGain = frameContext.sensor.gain; - /* - * Estimate the gain needed to have the proportion of pixels in a given - * desired range. iqMean_ returns the mean value of the top 2% of the - * cumulative histogram, and we want it to be as close as possible to a - * configured target. - */ - double evGain = kEvGainTarget * knumHistogramBins / iqMean_; - - if (evGain < yGain) - evGain = yGain; + /* Use the highest of the two gain estimates. */ + double evGain = std::max(yGain, iqMeanGain); /* Consider within 1% of the target as correctly exposed */ if (std::abs(evGain - 1.0) < 0.01) - LOG(IPU3Agc, Debug) << "We are well exposed (iqMean = " - << iqMean_ << ")"; + LOG(IPU3Agc, Debug) << "We are well exposed (evGain = " + << evGain << ")"; /* extracted from Rpi::Agc::computeTargetExposure */ @@ -308,15 +303,25 @@ double Agc::estimateLuminance(IPAFrameContext &frameContext, */ void Agc::process(IPAContext &context, const ipu3_uapi_stats_3a *stats) { - measureBrightness(stats, context.configuration.grid.bdsGrid); + /* + * Estimate the gain needed to have the proportion of pixels in a given + * desired range. iqMean is the mean value of the top 2% of the + * cumulative histogram, and we want it to be as close as possible to a + * configured target. + */ + double iqMean = measureBrightness(stats, context.configuration.grid.bdsGrid); + double iqMeanGain = kEvGainTarget * knumHistogramBins / iqMean; + /* + * Estimate the gain needed to achieve a relative luminance target. To + * account for non-linearity caused by saturation, the value needs to be + * estimated in an iterative process, as multiplying by a gain will not + * increase the relative luminance by the same factor if some image + * regions are saturated. + */ double yGain = 1.0; double yTarget = kRelativeLuminanceTarget; - /* - * Do this calculation a few times as brightness increase can be - * non-linear when there are saturated regions. - */ for (unsigned int i = 0; i < 8; i++) { double yValue = estimateLuminance(context.frameContext, context.configuration.grid.bdsGrid, @@ -331,7 +336,7 @@ void Agc::process(IPAContext &context, const ipu3_uapi_stats_3a *stats) break; } - computeExposure(context.frameContext, yGain); + computeExposure(context.frameContext, yGain, iqMeanGain); frameCount_++; } diff --git a/src/ipa/ipu3/algorithms/agc.h b/src/ipa/ipu3/algorithms/agc.h index 0c868d6737f1..a04a81fb8eae 100644 --- a/src/ipa/ipu3/algorithms/agc.h +++ b/src/ipa/ipu3/algorithms/agc.h @@ -31,10 +31,11 @@ public: void process(IPAContext &context, const ipu3_uapi_stats_3a *stats) override; private: - void measureBrightness(const ipu3_uapi_stats_3a *stats, - const ipu3_uapi_grid_config &grid); + double measureBrightness(const ipu3_uapi_stats_3a *stats, + const ipu3_uapi_grid_config &grid) const; void filterExposure(); - void computeExposure(IPAFrameContext &frameContext, double yGain); + void computeExposure(IPAFrameContext &frameContext, double yGain, + double iqMeanGain); double estimateLuminance(IPAFrameContext &frameContext, const ipu3_uapi_grid_config &grid, const ipu3_uapi_stats_3a *stats, @@ -42,8 +43,6 @@ private: uint64_t frameCount_; - double iqMean_; - utils::Duration lineDuration_; utils::Duration minShutterSpeed_; utils::Duration maxShutterSpeed_; From patchwork Tue Nov 16 16:26:15 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Laurent Pinchart X-Patchwork-Id: 14621 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id D9284C324F for ; Tue, 16 Nov 2021 16:26:51 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 9A4696037A; Tue, 16 Nov 2021 17:26:50 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="N44gLHUv"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 88BF86038A for ; Tue, 16 Nov 2021 17:26:43 +0100 (CET) Received: from localhost.localdomain (117.145-247-81.adsl-dyn.isp.belgacom.be [81.247.145.117]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 4A968E7 for ; Tue, 16 Nov 2021 17:26:43 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1637080003; bh=45PdbriSu7VhWwqIlkK5J/wQLTFcQuaxEfq6m9OuXvg=; h=From:To:Subject:Date:In-Reply-To:References:From; b=N44gLHUvqtxerIvYAJocK8gLakm4IJvIyD4H8TOL5pu0H3ZH2d3okfok1CZYnW/4D 7vOwJeU8uqtpIZagAgDBBkswcxSEtu10AVcoI97fPsGx3mQR8VBSOTMB4+1COI2S2V DIbkodeaw1Ly1IatOKDjxYS4KSeqyJEzd4FI8/yc= From: Laurent Pinchart To: libcamera-devel@lists.libcamera.org Date: Tue, 16 Nov 2021 18:26:15 +0200 Message-Id: <20211116162615.27777-6-laurent.pinchart@ideasonboard.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20211116162615.27777-1-laurent.pinchart@ideasonboard.com> References: <20211116162615.27777-1-laurent.pinchart@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH 5/5] ipa: ipu3: agc: Saturate the averages when computing relative luminance X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" The relative luminance is calculated using an iterative process to account for saturation in the sensor, as multiplying pixels by a gain doesn't increase the relative luminance by the same factor if some regions are saturated. Relative luminance estimation doesn't apply a saturation, which produces a value that doesn't match what the sensor will output, and defeats the point of the iterative process. Fix it. Fixes: f8f07f9468c6 ("ipa: ipu3: agc: Improve gain calculation") Signed-off-by: Laurent Pinchart Tested-by: Jean-Michel Hautbois Reviewed-by: Jean-Michel Hautbois Tested-by: Kieran Bingham Reviewed-by: Kieran Bingham --- src/ipa/ipu3/algorithms/agc.cpp | 25 ++++++++++++++++++------- 1 file changed, 18 insertions(+), 7 deletions(-) diff --git a/src/ipa/ipu3/algorithms/agc.cpp b/src/ipa/ipu3/algorithms/agc.cpp index 71398fdd96a6..46aa1b14a100 100644 --- a/src/ipa/ipu3/algorithms/agc.cpp +++ b/src/ipa/ipu3/algorithms/agc.cpp @@ -252,10 +252,19 @@ void Agc::computeExposure(IPAFrameContext &frameContext, double yGain, * \param[in] gain The analogue gain to apply to the frame * \return The relative luminance * - * Luma is the weighted sum of gamma-compressed R′G′B′ components of a color - * video. The luma values are normalized as 0.0 to 1.0, with 1.0 being a - * theoretical perfect reflector of 100% reference white. We use the Rec. 601 - * luma here. + * This function estimates the average relative luminance of the frame that + * would be output by the sensor if an additional analogue \a gain was applied. + * + * The estimation is based on the AWB statistics for the current frame. Red, + * green and blue averages for all cells are first multiplied by the gain, and + * then saturated to approximate the sensor behaviour at high brightness + * values. The approximation is quitte rough, as it doesn't take into account + * non-linearities when approaching saturation. + * + * The relative luminance (Y) is computed from the linear RGB components using + * the Rec. 601 formula. The values is normalized to the [0.0, 1.0] range, + * where 1.0 corresponds to a theoretical perfect reflector of 100% reference + * white. * * More detailed information can be found in: * https://en.wikipedia.org/wiki/Relative_luminance @@ -267,6 +276,7 @@ double Agc::estimateLuminance(IPAFrameContext &frameContext, { double redSum = 0, greenSum = 0, blueSum = 0; + /* Sum the per-channel averages, saturated to 255. */ for (unsigned int cellY = 0; cellY < grid.height; cellY++) { for (unsigned int cellX = 0; cellX < grid.width; cellX++) { uint32_t cellPosition = cellY * stride_ + cellX; @@ -275,10 +285,11 @@ double Agc::estimateLuminance(IPAFrameContext &frameContext, reinterpret_cast( &stats->awb_raw_buffer.meta_data[cellPosition] ); + const uint8_t G_avg = (cell->Gr_avg + cell->Gb_avg) / 2; - redSum += cell->R_avg * gain; - greenSum += (cell->Gr_avg + cell->Gb_avg) / 2 * gain; - blueSum += cell->B_avg * gain; + redSum += std::min(cell->R_avg * gain, 255.0); + greenSum += std::min(G_avg * gain, 255.0); + blueSum += std::min(cell->B_avg * gain, 255.0); } }