From patchwork Mon Nov 8 13:13:35 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jean-Michel Hautbois X-Patchwork-Id: 14476 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id C9FEDBDB1C for ; Mon, 8 Nov 2021 13:14:10 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id D44A76039C; Mon, 8 Nov 2021 14:14:09 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="afwnBodz"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id AF16360368 for ; Mon, 8 Nov 2021 14:14:00 +0100 (CET) Received: from tatooine.ideasonboard.com (unknown [IPv6:2a01:e0a:169:7140:c2bb:76d0:68d7:a9a5]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 6EF71110E; Mon, 8 Nov 2021 14:14:00 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1636377240; bh=8ssxltMYMTz012AgiKESipgfnNQY25+vOCry7kbaFLQ=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=afwnBodzIkU8zNr+snHIN2+U3thd4qH8rRCh2Zf6bPYp96m91uOylp56ODpXFfG5z ddf02XjaTk+hj+W6n/ebws+65h2lvDERUjj+NmsTuon5OZY5PqYygNWCW7F1DlpfMK QB7X2JDiuOiKxejBqz9xK/VbzLYVSDNnfL4721s8= From: Jean-Michel Hautbois To: libcamera-devel@lists.libcamera.org Date: Mon, 8 Nov 2021 14:13:35 +0100 Message-Id: <20211108131350.130665-8-jeanmichel.hautbois@ideasonboard.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20211108131350.130665-1-jeanmichel.hautbois@ideasonboard.com> References: <20211108131350.130665-1-jeanmichel.hautbois@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH 07/22] ipa: ipu3: agc: Improve gain calculation X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" When an image is partially saturated, its brightness is not increasing linearly when the shutter time or gain increases. It is a big issue with a backlight as the algorithm is fading to darkness right now. Introduce a function to estimate the brightness of the frame, based on the current exposure/gain and loop on it several times to estimate it again and approach the non linear function. Inspired-by: 7de5506c30b3 ("libcamera: src: ipa: raspberrypi: agc: Improve gain update calculation for partly saturated images") Signed-off-by: Jean-Michel Hautbois --- src/ipa/ipu3/algorithms/agc.cpp | 77 ++++++++++++++++++++++++++++++++- src/ipa/ipu3/algorithms/agc.h | 6 ++- 2 files changed, 80 insertions(+), 3 deletions(-) diff --git a/src/ipa/ipu3/algorithms/agc.cpp b/src/ipa/ipu3/algorithms/agc.cpp index 119a7938..e4048d40 100644 --- a/src/ipa/ipu3/algorithms/agc.cpp +++ b/src/ipa/ipu3/algorithms/agc.cpp @@ -186,8 +186,9 @@ void Agc::filterExposure() * \brief Estimate the new exposure and gain values * \param[inout] exposure The exposure value reference as a number of lines * \param[inout] gain The gain reference to be updated + * \param[in] currentYGain The gain calculated on the current brightness level */ -void Agc::computeExposure(uint32_t &exposure, double &analogueGain) +void Agc::computeExposure(uint32_t &exposure, double &analogueGain, double currentYGain) { /* Estimate the gain needed to have the proportion wanted */ double evGain = kEvGainTarget * knumHistogramBins / iqMean_; @@ -206,6 +207,14 @@ void Agc::computeExposure(uint32_t &exposure, double &analogueGain) << " Gain " << analogueGain << " Needed ev gain " << evGain; + if (evGain < currentYGain) + evGain = currentYGain; + + if (std::abs(evGain - 1.0) < 0.01) { + LOG(IPU3Agc, Debug) << "We are well exposed (iqMean = " + << iqMean_ << ")"; + } + /* * Calculate the current exposure value for the scene as the latest * exposure value applied multiplied by the new estimated gain. @@ -253,6 +262,48 @@ void Agc::computeExposure(uint32_t &exposure, double &analogueGain) prevExposureValue_ = shutterTime * analogueGain; } +/** + * \brief Estimate the average brightness of the frame + * \param[in] context The shared IPA context + * \param[in] grid The grid used to store the statistics in the IPU3 + * \param[in] stats The IPU3 statistics and ISP results + * \param[in] currentYGain The gain calculated on the current brightness level + */ +double Agc::computeInitialY(IPAFrameContext &frameContext, + const ipu3_uapi_grid_config &grid, + const ipu3_uapi_stats_3a *stats, + double currentYGain) +{ + double redSum = 0, greenSum = 0, blueSum = 0; + + for (unsigned int cellY = 0; cellY < grid.height; cellY++) { + for (unsigned int cellX = 0; cellX < grid.width; cellX++) { + uint32_t cellPosition = cellY * stride_ + cellX; + + const ipu3_uapi_awb_set_item *cell = + reinterpret_cast( + &stats->awb_raw_buffer.meta_data[cellPosition] + ); + + redSum += cell->R_avg * currentYGain; + greenSum += (cell->Gr_avg + cell->Gb_avg) / 2 * currentYGain; + blueSum += cell->B_avg * currentYGain; + } + } + + /* + * Estimate the sum of the brightness values, weighted with the gains + * applied on the channels in AWB. + */ + double Y_sum = redSum * frameContext.awb.gains.red * .299 + + greenSum * frameContext.awb.gains.green * .587 + + blueSum * frameContext.awb.gains.blue * .114; + + /* And return the average brightness */ + return Y_sum / (grid.height * grid.width); +} + + /** * \brief Process IPU3 statistics, and run AGC operations * \param[in] context The shared IPA context @@ -267,7 +318,29 @@ void Agc::process(IPAContext &context, const ipu3_uapi_stats_3a *stats) uint32_t &exposure = context.frameContext.agc.exposure; double &analogueGain = context.frameContext.agc.gain; measureBrightness(stats, context.configuration.grid.bdsGrid); - computeExposure(exposure, analogueGain); + + double currentYGain = 1.0; + /* \todo: the target Y needs to be grabbed from a configuration */ + double targetY = 60; + /* + * Do this calculation a few times as brightness increase can be + * non-linear when there are saturated regions. + */ + for (int i = 0; i < 8; i++) { + double initialY = computeInitialY(context.frameContext, + context.configuration.grid.bdsGrid, + stats, currentYGain); + double extra_gain = std::min(10.0, targetY / (initialY + .001)); + + currentYGain *= extra_gain; + LOG(IPU3Agc, Debug) << "Initial Y " << initialY + << " target " << targetY + << " gives gain " << currentYGain; + if (extra_gain < 1.01) + break; + } + + computeExposure(exposure, analogueGain, currentYGain); frameCount_++; } diff --git a/src/ipa/ipu3/algorithms/agc.h b/src/ipa/ipu3/algorithms/agc.h index 69e0b831..0a9152a9 100644 --- a/src/ipa/ipu3/algorithms/agc.h +++ b/src/ipa/ipu3/algorithms/agc.h @@ -34,7 +34,11 @@ private: void measureBrightness(const ipu3_uapi_stats_3a *stats, const ipu3_uapi_grid_config &grid); void filterExposure(); - void computeExposure(uint32_t &exposure, double &gain); + void computeExposure(uint32_t &exposure, double &gain, double currentYGain); + double computeInitialY(IPAFrameContext &frameContext, + const ipu3_uapi_grid_config &grid, + const ipu3_uapi_stats_3a *stats, + double currentYGain); uint64_t frameCount_; uint64_t lastFrame_;