[{"id":20958,"web_url":"https://patchwork.libcamera.org/comment/20958/","msgid":"","date":"2021-11-15T15:39:34","subject":"Re: [libcamera-devel] [PATCH v5 04/14] ipa: ipu3: agc: Limit the\n\tnumber of saturated cells","submitter":{"id":2,"url":"https://patchwork.libcamera.org/api/people/2/","name":"Laurent Pinchart","email":"laurent.pinchart@ideasonboard.com"},"content":"Hi Jean-Michel,\n\nThank you for the patch.\n\nOn Sat, Nov 13, 2021 at 09:49:37AM +0100, Jean-Michel Hautbois wrote:\n> When the histogram is calculated, we check if a cell is saturated or not\n> before cumulating its green value. This is wrong, and it can lead to an\n> empty histogram in case of a fully saturated frame.\n> \n> Use a constant to limit the amount of pixels within a cell before\n> considering it saturated. If at the end of the loop we still have an\n> empty histogram, then make it a fully saturated one.\n\nDo we need this threshold ? For AWB handling saturation is important,\nbut for AGC, what will happen if we compute the histogram using all\ncells regardless of saturation ?\n\n> Bug: https://bugs.libcamera.org/show_bug.cgi?id=84\n> Signed-off-by: Jean-Michel Hautbois \n> Reviewed-by: Kieran Bingham \n> Reviewed-by: Paul Elder \n> ---\n> src/ipa/ipu3/algorithms/agc.cpp | 16 ++++++++++++++--\n> 1 file changed, 14 insertions(+), 2 deletions(-)\n> \n> diff --git a/src/ipa/ipu3/algorithms/agc.cpp b/src/ipa/ipu3/algorithms/agc.cpp\n> index 5723f6f3..83aa3676 100644\n> --- a/src/ipa/ipu3/algorithms/agc.cpp\n> +++ b/src/ipa/ipu3/algorithms/agc.cpp\n> @@ -63,6 +63,12 @@ static constexpr uint32_t knumHistogramBins = 256;\n> /* Target value to reach for the top 2% of the histogram */\n> static constexpr double kEvGainTarget = 0.5;\n> \n> +/*\n> + * Maximum ratio of saturated pixels in a cell for the cell to be considered\n> + * non-saturated and counted by the AGC algorithm.\n> + */\n> +static constexpr uint32_t kMinCellsPerZoneRatio = 255 * 20 / 100;\n> +\n> Agc::Agc()\n> \t: frameCount_(0), lastFrame_(0), iqMean_(0.0), lineDuration_(0s),\n> \t minExposureLines_(0), maxExposureLines_(0), filteredExposure_(0s),\n> @@ -124,7 +130,7 @@ void Agc::measureBrightness(const ipu3_uapi_stats_3a *stats,\n> \t\t\t\t\t&stats->awb_raw_buffer.meta_data[cellPosition]\n> \t\t\t\t);\n> \n> -\t\t\tif (cell->sat_ratio == 0) {\n> +\t\t\tif (cell->sat_ratio <= kMinCellsPerZoneRatio) {\n> \t\t\t\tuint8_t gr = cell->Gr_avg;\n> \t\t\t\tuint8_t gb = cell->Gb_avg;\n> \t\t\t\t/*\n> @@ -137,8 +143,14 @@ void Agc::measureBrightness(const ipu3_uapi_stats_3a *stats,\n> \t\t}\n> \t}\n> \n> +\tHistogram cumulativeHist = Histogram(Span(hist));\n> \t/* Estimate the quantile mean of the top 2% of the histogram */\n> -\tiqMean_ = Histogram(Span(hist)).interQuantileMean(0.98, 1.0);\n> +\tif (cumulativeHist.total() == 0) {\n> +\t\t/* Force the value as histogram is empty */\n> +\t\tiqMean_ = knumHistogramBins - 0.5;\n> +\t} else {\n> +\t\tiqMean_ = cumulativeHist.interQuantileMean(0.98, 1.0);\n> +\t}\n> }\n> \n> /**","headers":{"Return-Path":"","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id 3D45EBDB1C\n\tfor ;\n\tMon, 15 Nov 2021 15:39:59 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id 9BCFC6036C;\n\tMon, 15 Nov 2021 16:39:58 +0100 (CET)","from perceval.ideasonboard.com (perceval.ideasonboard.com\n\t[213.167.242.64])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id 2487D60233\n\tfor ;\n\tMon, 15 Nov 2021 16:39:57 +0100 (CET)","from pendragon.ideasonboard.com\n\t(117.145-247-81.adsl-dyn.isp.belgacom.be [81.247.145.117])\n\tby perceval.ideasonboard.com (Postfix) with ESMTPSA id A88979CA;\n\tMon, 15 Nov 2021 16:39:56 +0100 (CET)"],"Authentication-Results":"lancelot.ideasonboard.com; dkim=pass (1024-bit key;\n\tunprotected) header.d=ideasonboard.com header.i=@ideasonboard.com\n\theader.b=\"QqiSDnHQ\"; dkim-atps=neutral","DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com;\n\ts=mail; t=1636990796;\n\tbh=j+2ayPEweBeevIgPh9DtyGa3YPyVY2rgKq1QqXyVGYw=;\n\th=Date:From:To:Cc:Subject:References:In-Reply-To:From;\n\tb=QqiSDnHQrv4cu7RTrUCg88+TBsU0ec5HGf4imMwLMR4QrBPKUsUUo0K1w54p5qw6P\n\tfmqPmU5LXBk/9moVQB0QIn223d/MIx2PcuHNnYPFtc07zIADulbJSOy42u14xO3nNb\n\tMq/Fpq7KjWqOFVM6wkbRI+yKcfiKbEpWlVLJH8ew=","Date":"Mon, 15 Nov 2021 17:39:34 +0200","From":"Laurent Pinchart ","To":"Jean-Michel Hautbois ","Message-ID":"","References":"<20211113084947.21892-1-jeanmichel.hautbois@ideasonboard.com>\n\t<20211113084947.21892-5-jeanmichel.hautbois@ideasonboard.com>","MIME-Version":"1.0","Content-Type":"text/plain; charset=utf-8","Content-Disposition":"inline","In-Reply-To":"<20211113084947.21892-5-jeanmichel.hautbois@ideasonboard.com>","Subject":"Re: [libcamera-devel] [PATCH v5 04/14] ipa: ipu3: agc: Limit the\n\tnumber of saturated cells","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Cc":"libcamera-devel@lists.libcamera.org","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" "}},{"id":20962,"web_url":"https://patchwork.libcamera.org/comment/20962/","msgid":"<0c667fd8-ee9e-dd0d-b036-980c70a15aba@ideasonboard.com>","date":"2021-11-16T07:54:10","subject":"Re: [libcamera-devel] [PATCH v5 04/14] ipa: ipu3: agc: Limit the\n\tnumber of saturated cells","submitter":{"id":75,"url":"https://patchwork.libcamera.org/api/people/75/","name":"Jean-Michel Hautbois","email":"jeanmichel.hautbois@ideasonboard.com"},"content":"Hi Laurent,\n\nOn 15/11/2021 16:39, Laurent Pinchart wrote:\n> Hi Jean-Michel,\n> \n> Thank you for the patch.\n> \n> On Sat, Nov 13, 2021 at 09:49:37AM +0100, Jean-Michel Hautbois wrote:\n>> When the histogram is calculated, we check if a cell is saturated or not\n>> before cumulating its green value. This is wrong, and it can lead to an\n>> empty histogram in case of a fully saturated frame.\n>>\n>> Use a constant to limit the amount of pixels within a cell before\n>> considering it saturated. If at the end of the loop we still have an\n>> empty histogram, then make it a fully saturated one.\n> \n> Do we need this threshold ? For AWB handling saturation is important,\n> but for AGC, what will happen if we compute the histogram using all\n> cells regardless of saturation ?\n\nI think we need this until the \"ipa: ipu3: agc: Improve gain \ncalculation\" is applied. So, on master, I think we can probably remove \nit now.\n\n> \n>> Bug: https://bugs.libcamera.org/show_bug.cgi?id=84\n>> Signed-off-by: Jean-Michel Hautbois \n>> Reviewed-by: Kieran Bingham \n>> Reviewed-by: Paul Elder \n>> ---\n>> src/ipa/ipu3/algorithms/agc.cpp | 16 ++++++++++++++--\n>> 1 file changed, 14 insertions(+), 2 deletions(-)\n>>\n>> diff --git a/src/ipa/ipu3/algorithms/agc.cpp b/src/ipa/ipu3/algorithms/agc.cpp\n>> index 5723f6f3..83aa3676 100644\n>> --- a/src/ipa/ipu3/algorithms/agc.cpp\n>> +++ b/src/ipa/ipu3/algorithms/agc.cpp\n>> @@ -63,6 +63,12 @@ static constexpr uint32_t knumHistogramBins = 256;\n>> /* Target value to reach for the top 2% of the histogram */\n>> static constexpr double kEvGainTarget = 0.5;\n>> \n>> +/*\n>> + * Maximum ratio of saturated pixels in a cell for the cell to be considered\n>> + * non-saturated and counted by the AGC algorithm.\n>> + */\n>> +static constexpr uint32_t kMinCellsPerZoneRatio = 255 * 20 / 100;\n>> +\n>> Agc::Agc()\n>> \t: frameCount_(0), lastFrame_(0), iqMean_(0.0), lineDuration_(0s),\n>> \t minExposureLines_(0), maxExposureLines_(0), filteredExposure_(0s),\n>> @@ -124,7 +130,7 @@ void Agc::measureBrightness(const ipu3_uapi_stats_3a *stats,\n>> \t\t\t\t\t&stats->awb_raw_buffer.meta_data[cellPosition]\n>> \t\t\t\t);\n>> \n>> -\t\t\tif (cell->sat_ratio == 0) {\n>> +\t\t\tif (cell->sat_ratio <= kMinCellsPerZoneRatio) {\n>> \t\t\t\tuint8_t gr = cell->Gr_avg;\n>> \t\t\t\tuint8_t gb = cell->Gb_avg;\n>> \t\t\t\t/*\n>> @@ -137,8 +143,14 @@ void Agc::measureBrightness(const ipu3_uapi_stats_3a *stats,\n>> \t\t}\n>> \t}\n>> \n>> +\tHistogram cumulativeHist = Histogram(Span(hist));\n>> \t/* Estimate the quantile mean of the top 2% of the histogram */\n>> -\tiqMean_ = Histogram(Span(hist)).interQuantileMean(0.98, 1.0);\n>> +\tif (cumulativeHist.total() == 0) {\n>> +\t\t/* Force the value as histogram is empty */\n>> +\t\tiqMean_ = knumHistogramBins - 0.5;\n>> +\t} else {\n>> +\t\tiqMean_ = cumulativeHist.interQuantileMean(0.98, 1.0);\n>> +\t}\n>> }\n>> \n>> /**\n>","headers":{"Return-Path":"","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id 56240BF415\n\tfor ;\n\tTue, 16 Nov 2021 07:54:15 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id A7AC860230;\n\tTue, 16 Nov 2021 08:54:14 +0100 (CET)","from perceval.ideasonboard.com (perceval.ideasonboard.com\n\t[213.167.242.64])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id 61B2860128\n\tfor ;\n\tTue, 16 Nov 2021 08:54:13 +0100 (CET)","from [IPV6:2a01:e0a:169:7140:c966:1d53:a935:70a0] (unknown\n\t[IPv6:2a01:e0a:169:7140:c966:1d53:a935:70a0])\n\tby perceval.ideasonboard.com (Postfix) with ESMTPSA id 13FB1466;\n\tTue, 16 Nov 2021 08:54:13 +0100 (CET)"],"Authentication-Results":"lancelot.ideasonboard.com; dkim=pass (1024-bit key;\n\tunprotected) header.d=ideasonboard.com header.i=@ideasonboard.com\n\theader.b=\"kaFjN6gV\"; dkim-atps=neutral","DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com;\n\ts=mail; t=1637049253;\n\tbh=4BFGx1Y7VAsxaeZd7QNdGR08j9aZRPLGlTyuMuG10KA=;\n\th=Date:Subject:To:Cc:References:From:In-Reply-To:From;\n\tb=kaFjN6gVcaiugZqaVP5hRL3DtYUCHDlXFRFjIkOsCVIzCqTx3NfRkx9tKVcorIJab\n\tHXYJyGGujLN1S51NUVjpOhEhCgOnmUzpbc9KaN6gxvYSauCGHVtyye1AYqvTW+dYYE\n\tqCc6VLVak7rTvl4VpOzgojgT4Z3eXCu6hjPNPIio=","Message-ID":"<0c667fd8-ee9e-dd0d-b036-980c70a15aba@ideasonboard.com>","Date":"Tue, 16 Nov 2021 08:54:10 +0100","MIME-Version":"1.0","User-Agent":"Mozilla/5.0 (X11; Linux x86_64; rv:91.0) Gecko/20100101\n\tThunderbird/91.2.1","Content-Language":"en-US","To":"Laurent Pinchart ","References":"<20211113084947.21892-1-jeanmichel.hautbois@ideasonboard.com>\n\t<20211113084947.21892-5-jeanmichel.hautbois@ideasonboard.com>\n\t","From":"Jean-Michel Hautbois ","In-Reply-To":"","Content-Type":"text/plain; charset=UTF-8; format=flowed","Content-Transfer-Encoding":"7bit","Subject":"Re: [libcamera-devel] [PATCH v5 04/14] ipa: ipu3: agc: Limit the\n\tnumber of saturated cells","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Cc":"libcamera-devel@lists.libcamera.org","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" "}}]