[{"id":25044,"web_url":"https://patchwork.libcamera.org/comment/25044/","msgid":"<166371757953.18961.3389671865776993038@Monstersaurus>","date":"2022-09-20T23:46:19","subject":"Re: [libcamera-devel] [PATCH v4 31/32] ipa: rkisp1: awb: Freeze AWB\n\twhen means are too small","submitter":{"id":4,"url":"https://patchwork.libcamera.org/api/people/4/","name":"Kieran Bingham","email":"kieran.bingham@ideasonboard.com"},"content":"Quoting Laurent Pinchart via libcamera-devel (2022-09-08 02:41:59)\n> When the RGB means are too small, gains and color temperature can't be\n> meaningfully calculated. Freeze the AWB in that case, using the\n> previously calculated values.\n> \n> Signed-off-by: Laurent Pinchart \n> ---\n> src/ipa/rkisp1/algorithms/awb.cpp | 17 ++++++++++++++++-\n> src/ipa/rkisp1/ipa_context.cpp | 3 +++\n> src/ipa/rkisp1/ipa_context.h | 1 +\n> 3 files changed, 20 insertions(+), 1 deletion(-)\n> \n> diff --git a/src/ipa/rkisp1/algorithms/awb.cpp b/src/ipa/rkisp1/algorithms/awb.cpp\n> index 5f2535688c93..404ad66e6953 100644\n> --- a/src/ipa/rkisp1/algorithms/awb.cpp\n> +++ b/src/ipa/rkisp1/algorithms/awb.cpp\n> @@ -31,6 +31,9 @@ namespace ipa::rkisp1::algorithms {\n> \n> LOG_DEFINE_CATEGORY(RkISP1Awb)\n> \n> +/* Minimum mean value below which AWB can't operate. */\n> +constexpr double kMeanMinThreshold = 2.0;\n\nWill this be a tuning parameter? Is it something to calibrate later? or\nbetter document the expected ranges that should be used here?\n\nWhat has led to the choice of 2.0, - is it simply that we can't estimate\na colour temparature from a near black / dark image?\n\nIf so - should we have a representation of overall brightness?\n\n\n\n> +\n> Awb::Awb()\n> : rgbMode_(false)\n> {\n> @@ -263,7 +266,17 @@ void Awb::process(IPAContext &context,\n> greenMean /= frameContext.awb.gains.green;\n> blueMean /= frameContext.awb.gains.blue;\n> \n> - frameContext.awb.temperatureK = estimateCCT(redMean, greenMean, blueMean);\n> + /*\n> + * If the means are too small we don't have enough information to\n> + * meaningfully calculate gains. Freeze the algorithm in that case.\n> + */\n> + if (redMean < kMeanMinThreshold && greenMean < kMeanMinThreshold &&\n> + blueMean < kMeanMinThreshold) {\n> + frameContext.awb.temperatureK = activeState.awb.temperatureK;\n> + return;\n> + }\n> +\n> + activeState.awb.temperatureK = estimateCCT(redMean, greenMean, blueMean);\n\nAha - good, here's the active state view of the colour temperature.\n\n\nWhile I have questions about how the threshold is determined, I think\nthis is still good to get in.\n\n\nReviewed-by: Kieran Bingham \n\n> \n> /*\n> * Estimate the red and blue gains to apply in a grey world. The green\n> @@ -290,6 +303,8 @@ void Awb::process(IPAContext &context,\n> activeState.awb.gains.automatic.blue = blueGain;\n> activeState.awb.gains.automatic.green = 1.0;\n> \n> + frameContext.awb.temperatureK = activeState.awb.temperatureK;\n> +\n> LOG(RkISP1Awb, Debug) << std::showpoint\n> << \"Means [\" << redMean << \", \" << greenMean << \", \" << blueMean\n> << \"], gains [\" << activeState.awb.gains.automatic.red << \", \"\n> diff --git a/src/ipa/rkisp1/ipa_context.cpp b/src/ipa/rkisp1/ipa_context.cpp\n> index 335cb32c538d..9cac05bec79b 100644\n> --- a/src/ipa/rkisp1/ipa_context.cpp\n> +++ b/src/ipa/rkisp1/ipa_context.cpp\n> @@ -156,6 +156,9 @@ namespace libcamera::ipa::rkisp1 {\n> * \\var IPAActiveState::awb.gains.automatic.blue\n> * \\brief Automatic white balance gain for B channel\n> *\n> + * \\var IPAActiveState::awb.temperatureK\n> + * \\brief Estimated color temperature\n> + *\n> * \\var IPAActiveState::awb.autoEnabled\n> * \\brief Whether the Auto White Balance algorithm is enabled\n> */\n> diff --git a/src/ipa/rkisp1/ipa_context.h b/src/ipa/rkisp1/ipa_context.h\n> index 53787320e565..d0bc9090a8f1 100644\n> --- a/src/ipa/rkisp1/ipa_context.h\n> +++ b/src/ipa/rkisp1/ipa_context.h\n> @@ -68,6 +68,7 @@ struct IPAActiveState {\n> } automatic;\n> } gains;\n> \n> + unsigned int temperatureK;\n> bool autoEnabled;\n> } awb;\n> \n> -- \n> Regards,\n> \n> Laurent Pinchart\n>","headers":{"Return-Path":"","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id 6B1AEC3272\n\tfor ;\n\tTue, 20 Sep 2022 23:46:24 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id 81268621D9;\n\tWed, 21 Sep 2022 01:46:23 +0200 (CEST)","from perceval.ideasonboard.com (perceval.ideasonboard.com\n\t[IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id 6985F61F7D\n\tfor ;\n\tWed, 21 Sep 2022 01:46:22 +0200 (CEST)","from pendragon.ideasonboard.com\n\t(cpc89244-aztw30-2-0-cust3082.18-1.cable.virginm.net [86.31.172.11])\n\tby perceval.ideasonboard.com (Postfix) with ESMTPSA id E1E19415;\n\tWed, 21 Sep 2022 01:46:21 +0200 (CEST)"],"DKIM-Signature":["v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org;\n\ts=mail; t=1663717583;\n\tbh=KhRRlOOynBo3eSjFcoRJ/nCzUHA4Dggb3cgvwcvcmRk=;\n\th=In-Reply-To:References:To:Date:Subject:List-Id:List-Unsubscribe:\n\tList-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To:\n\tFrom;\n\tb=f0WfWKNcOSg662gAJBl3ojIdCpmkENjnfBb0wlQPN3B12+hPbiv0LkMkhNJlOcksn\n\tlMG2babEHG6sIJ0nsj/OwMULKKoCube6kei7miyPs7x8NMuE/QACw+QQ56dQNxHXQH\n\t6cvRTPB6qjiVtmiBsMWUUyVA6EVIQteRze597Qk7JCEGNrVvZzveriXgNCljK4Nt9B\n\tk6tVg+/rcF0930dCJUsaY/rtJbr/3pj/R84KwlWWF45u1te0xZrDwx4QjVlQngpUFx\n\tq//V+ZnMgiyOts2Wlnx4ktdbcHbRYNbwEAtOiMP7EuSCQNRrtzGTIEf4HpNgtl5KKs\n\tIGmiNdE/qdeWw==","v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com;\n\ts=mail; t=1663717582;\n\tbh=KhRRlOOynBo3eSjFcoRJ/nCzUHA4Dggb3cgvwcvcmRk=;\n\th=In-Reply-To:References:Subject:From:To:Date:From;\n\tb=AAYnxK+e6aT9wsOclxuIVEaaSwmAfp/al4ejEnUHXUgysAU2AmMalVG9p0X9rcrQi\n\tj8ko8gCFHvPIY0vUmjiXivINLFspXmCj1b5ju/tmQOIebU1dXdW9I2hkAN1c+8V3Ej\n\tAoHQGkkewRGRDyoHTyWJE2k+26/r81OR1GoQ7XWM="],"Authentication-Results":"lancelot.ideasonboard.com; dkim=pass (1024-bit key; \n\tunprotected) header.d=ideasonboard.com\n\theader.i=@ideasonboard.com\n\theader.b=\"AAYnxK+e\"; dkim-atps=neutral","Content-Type":"text/plain; charset=\"utf-8\"","MIME-Version":"1.0","Content-Transfer-Encoding":"quoted-printable","In-Reply-To":"<20220908014200.28728-32-laurent.pinchart@ideasonboard.com>","References":"<20220908014200.28728-1-laurent.pinchart@ideasonboard.com>\n\t<20220908014200.28728-32-laurent.pinchart@ideasonboard.com>","To":"Laurent Pinchart ,\n\tlibcamera-devel@lists.libcamera.org","Date":"Wed, 21 Sep 2022 00:46:19 +0100","Message-ID":"<166371757953.18961.3389671865776993038@Monstersaurus>","User-Agent":"alot/0.10","Subject":"Re: [libcamera-devel] [PATCH v4 31/32] ipa: rkisp1: awb: Freeze AWB\n\twhen means are too small","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","From":"Kieran Bingham via libcamera-devel\n\t","Reply-To":"Kieran Bingham ","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" "}},{"id":25081,"web_url":"https://patchwork.libcamera.org/comment/25081/","msgid":"<20220922104524.svbjcrkiduis4eke@uno.localdomain>","date":"2022-09-22T10:45:24","subject":"Re: [libcamera-devel] [PATCH v4 31/32] ipa: rkisp1: awb: Freeze AWB\n\twhen means are too small","submitter":{"id":3,"url":"https://patchwork.libcamera.org/api/people/3/","name":"Jacopo Mondi","email":"jacopo@jmondi.org"},"content":"Hi Laurent\n\nOn Thu, Sep 08, 2022 at 04:41:59AM +0300, Laurent Pinchart via libcamera-devel wrote:\n> When the RGB means are too small, gains and color temperature can't be\n> meaningfully calculated. Freeze the AWB in that case, using the\n> previously calculated values.\n>\n> Signed-off-by: Laurent Pinchart \n> ---\n> src/ipa/rkisp1/algorithms/awb.cpp | 17 ++++++++++++++++-\n> src/ipa/rkisp1/ipa_context.cpp | 3 +++\n> src/ipa/rkisp1/ipa_context.h | 1 +\n> 3 files changed, 20 insertions(+), 1 deletion(-)\n>\n> diff --git a/src/ipa/rkisp1/algorithms/awb.cpp b/src/ipa/rkisp1/algorithms/awb.cpp\n> index 5f2535688c93..404ad66e6953 100644\n> --- a/src/ipa/rkisp1/algorithms/awb.cpp\n> +++ b/src/ipa/rkisp1/algorithms/awb.cpp\n> @@ -31,6 +31,9 @@ namespace ipa::rkisp1::algorithms {\n>\n> LOG_DEFINE_CATEGORY(RkISP1Awb)\n>\n> +/* Minimum mean value below which AWB can't operate. */\n> +constexpr double kMeanMinThreshold = 2.0;\n> +\n\nIs this a software constraint specified somewhere ?\n\n> Awb::Awb()\n> \t: rgbMode_(false)\n> {\n> @@ -263,7 +266,17 @@ void Awb::process(IPAContext &context,\n> \tgreenMean /= frameContext.awb.gains.green;\n> \tblueMean /= frameContext.awb.gains.blue;\n>\n> -\tframeContext.awb.temperatureK = estimateCCT(redMean, greenMean, blueMean);\n> +\t/*\n> +\t * If the means are too small we don't have enough information to\n> +\t * meaningfully calculate gains. Freeze the algorithm in that case.\n> +\t */\n> +\tif (redMean < kMeanMinThreshold && greenMean < kMeanMinThreshold &&\n> +\t blueMean < kMeanMinThreshold) {\n> +\t\tframeContext.awb.temperatureK = activeState.awb.temperatureK;\n> +\t\treturn;\n> +\t}\n> +\n> +\tactiveState.awb.temperatureK = estimateCCT(redMean, greenMean, blueMean);\n>\n> \t/*\n> \t * Estimate the red and blue gains to apply in a grey world. The green\n> @@ -290,6 +303,8 @@ void Awb::process(IPAContext &context,\n> \tactiveState.awb.gains.automatic.blue = blueGain;\n> \tactiveState.awb.gains.automatic.green = 1.0;\n>\n> +\tframeContext.awb.temperatureK = activeState.awb.temperatureK;\n> +\n> \tLOG(RkISP1Awb, Debug) << std::showpoint\n> \t\t<< \"Means [\" << redMean << \", \" << greenMean << \", \" << blueMean\n> \t\t<< \"], gains [\" << activeState.awb.gains.automatic.red << \", \"\n> diff --git a/src/ipa/rkisp1/ipa_context.cpp b/src/ipa/rkisp1/ipa_context.cpp\n> index 335cb32c538d..9cac05bec79b 100644\n> --- a/src/ipa/rkisp1/ipa_context.cpp\n> +++ b/src/ipa/rkisp1/ipa_context.cpp\n> @@ -156,6 +156,9 @@ namespace libcamera::ipa::rkisp1 {\n> * \\var IPAActiveState::awb.gains.automatic.blue\n> * \\brief Automatic white balance gain for B channel\n> *\n> + * \\var IPAActiveState::awb.temperatureK\n> + * \\brief Estimated color temperature\n> + *\n> * \\var IPAActiveState::awb.autoEnabled\n> * \\brief Whether the Auto White Balance algorithm is enabled\n> */\n> diff --git a/src/ipa/rkisp1/ipa_context.h b/src/ipa/rkisp1/ipa_context.h\n> index 53787320e565..d0bc9090a8f1 100644\n> --- a/src/ipa/rkisp1/ipa_context.h\n> +++ b/src/ipa/rkisp1/ipa_context.h\n> @@ -68,6 +68,7 @@ struct IPAActiveState {\n> \t\t\t} automatic;\n> \t\t} gains;\n>\n> +\t\tunsigned int temperatureK;\n> \t\tbool autoEnabled;\n> \t} awb;\n>\n> --\n> Regards,\n>\n> Laurent Pinchart\n>","headers":{"Return-Path":"","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id E7185C0DA4\n\tfor ;\n\tThu, 22 Sep 2022 10:45:28 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id A4D8C62209;\n\tThu, 22 Sep 2022 12:45:28 +0200 (CEST)","from relay9-d.mail.gandi.net (relay9-d.mail.gandi.net\n\t[IPv6:2001:4b98:dc4:8::229])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id B84FB6219A\n\tfor ;\n\tThu, 22 Sep 2022 12:45:26 +0200 (CEST)","(Authenticated sender: jacopo@jmondi.org)\n\tby mail.gandi.net (Postfix) with ESMTPSA id 0BD72FF80C;\n\tThu, 22 Sep 2022 10:45:25 +0000 (UTC)"],"DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org;\n\ts=mail; t=1663843528;\n\tbh=1yk/F0DB1JXNtUobh9ruh0o+MVoJZNWXyz8QYKc1BrQ=;\n\th=Date:To:References:In-Reply-To:Subject:List-Id:List-Unsubscribe:\n\tList-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To:Cc:\n\tFrom;\n\tb=hpz5hR3iw84J7JK4SwYtYrpWMJJhqc5q/tN8SEbRzR8CMY0+8X9qF/0xGCfJvNJGN\n\tjKFrLur5CnMVa+64H0+cNAnYZ1wY7DU+w9FxeHkFiBqf3YqC8rrFIpLtFnU5qjic00\n\ttV6PO5ELL0xwCk0ZSuSSpo5WmzRYIIiMl5BqODPN51SstNbvYnX1StGjkomXHN47YX\n\tyJYoUSoVKfbJCjG9zprgdN0h+QmGHaKBKG3/LYy4jK1OPx2+9pv/nS/mK1q1MgWFe0\n\t5XtwMvRNlbBebzY9JMAHWoouqwTW450h7gltMrza7d6Wxy+v63CNPwrkKVcmxQVAdd\n\t378CVZFxW/W1g==","Date":"Thu, 22 Sep 2022 12:45:24 +0200","To":"Laurent Pinchart ","Message-ID":"<20220922104524.svbjcrkiduis4eke@uno.localdomain>","References":"<20220908014200.28728-1-laurent.pinchart@ideasonboard.com>\n\t<20220908014200.28728-32-laurent.pinchart@ideasonboard.com>","MIME-Version":"1.0","Content-Type":"text/plain; charset=utf-8","Content-Disposition":"inline","In-Reply-To":"<20220908014200.28728-32-laurent.pinchart@ideasonboard.com>","Subject":"Re: [libcamera-devel] [PATCH v4 31/32] ipa: rkisp1: awb: Freeze AWB\n\twhen means are too small","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","From":"Jacopo Mondi via libcamera-devel ","Reply-To":"Jacopo Mondi ","Cc":"libcamera-devel@lists.libcamera.org","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" "}},{"id":25103,"web_url":"https://patchwork.libcamera.org/comment/25103/","msgid":"","date":"2022-09-22T23:21:35","subject":"Re: [libcamera-devel] [PATCH v4 31/32] ipa: rkisp1: awb: Freeze AWB\n\twhen means are too small","submitter":{"id":2,"url":"https://patchwork.libcamera.org/api/people/2/","name":"Laurent Pinchart","email":"laurent.pinchart@ideasonboard.com"},"content":"Hi Kieran,\n\nOn Wed, Sep 21, 2022 at 12:46:19AM +0100, Kieran Bingham wrote:\n> Quoting Laurent Pinchart via libcamera-devel (2022-09-08 02:41:59)\n> > When the RGB means are too small, gains and color temperature can't be\n> > meaningfully calculated. Freeze the AWB in that case, using the\n> > previously calculated values.\n> > \n> > Signed-off-by: Laurent Pinchart \n> > ---\n> > src/ipa/rkisp1/algorithms/awb.cpp | 17 ++++++++++++++++-\n> > src/ipa/rkisp1/ipa_context.cpp | 3 +++\n> > src/ipa/rkisp1/ipa_context.h | 1 +\n> > 3 files changed, 20 insertions(+), 1 deletion(-)\n> > \n> > diff --git a/src/ipa/rkisp1/algorithms/awb.cpp b/src/ipa/rkisp1/algorithms/awb.cpp\n> > index 5f2535688c93..404ad66e6953 100644\n> > --- a/src/ipa/rkisp1/algorithms/awb.cpp\n> > +++ b/src/ipa/rkisp1/algorithms/awb.cpp\n> > @@ -31,6 +31,9 @@ namespace ipa::rkisp1::algorithms {\n> > \n> > LOG_DEFINE_CATEGORY(RkISP1Awb)\n> > \n> > +/* Minimum mean value below which AWB can't operate. */\n> > +constexpr double kMeanMinThreshold = 2.0;\n> \n> Will this be a tuning parameter? Is it something to calibrate later? or\n> better document the expected ranges that should be used here?\n> \n> What has led to the choice of 2.0, - is it simply that we can't estimate\n> a colour temparature from a near black / dark image?\n> \n> If so - should we have a representation of overall brightness?\n\nThe means are supposed to be in the 0-255 range. A value below 2 is just\ntoo low to do any meaningful computation. We'll probably need to adjust\nthis, as I think the threshold is too low, but I haven't spent any time\non that. I don't think it should be a tuning parameter, it's just a\nthreshold to avoid numerical instability.\n\n> > +\n> > Awb::Awb()\n> > : rgbMode_(false)\n> > {\n> > @@ -263,7 +266,17 @@ void Awb::process(IPAContext &context,\n> > greenMean /= frameContext.awb.gains.green;\n> > blueMean /= frameContext.awb.gains.blue;\n> > \n> > - frameContext.awb.temperatureK = estimateCCT(redMean, greenMean, blueMean);\n> > + /*\n> > + * If the means are too small we don't have enough information to\n> > + * meaningfully calculate gains. Freeze the algorithm in that case.\n> > + */\n> > + if (redMean < kMeanMinThreshold && greenMean < kMeanMinThreshold &&\n> > + blueMean < kMeanMinThreshold) {\n> > + frameContext.awb.temperatureK = activeState.awb.temperatureK;\n> > + return;\n> > + }\n> > +\n> > + activeState.awb.temperatureK = estimateCCT(redMean, greenMean, blueMean);\n> \n> Aha - good, here's the active state view of the colour temperature.\n> \n> \n> While I have questions about how the threshold is determined, I think\n> this is still good to get in.\n> \n> \n> Reviewed-by: Kieran Bingham \n> \n> > \n> > /*\n> > * Estimate the red and blue gains to apply in a grey world. The green\n> > @@ -290,6 +303,8 @@ void Awb::process(IPAContext &context,\n> > activeState.awb.gains.automatic.blue = blueGain;\n> > activeState.awb.gains.automatic.green = 1.0;\n> > \n> > + frameContext.awb.temperatureK = activeState.awb.temperatureK;\n> > +\n> > LOG(RkISP1Awb, Debug) << std::showpoint\n> > << \"Means [\" << redMean << \", \" << greenMean << \", \" << blueMean\n> > << \"], gains [\" << activeState.awb.gains.automatic.red << \", \"\n> > diff --git a/src/ipa/rkisp1/ipa_context.cpp b/src/ipa/rkisp1/ipa_context.cpp\n> > index 335cb32c538d..9cac05bec79b 100644\n> > --- a/src/ipa/rkisp1/ipa_context.cpp\n> > +++ b/src/ipa/rkisp1/ipa_context.cpp\n> > @@ -156,6 +156,9 @@ namespace libcamera::ipa::rkisp1 {\n> > * \\var IPAActiveState::awb.gains.automatic.blue\n> > * \\brief Automatic white balance gain for B channel\n> > *\n> > + * \\var IPAActiveState::awb.temperatureK\n> > + * \\brief Estimated color temperature\n> > + *\n> > * \\var IPAActiveState::awb.autoEnabled\n> > * \\brief Whether the Auto White Balance algorithm is enabled\n> > */\n> > diff --git a/src/ipa/rkisp1/ipa_context.h b/src/ipa/rkisp1/ipa_context.h\n> > index 53787320e565..d0bc9090a8f1 100644\n> > --- a/src/ipa/rkisp1/ipa_context.h\n> > +++ b/src/ipa/rkisp1/ipa_context.h\n> > @@ -68,6 +68,7 @@ struct IPAActiveState {\n> > } automatic;\n> > } gains;\n> > \n> > + unsigned int temperatureK;\n> > bool autoEnabled;\n> > } awb;\n> >","headers":{"Return-Path":"","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id A3F3BBD16B\n\tfor ;\n\tThu, 22 Sep 2022 23:21:53 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id CE4BA62225;\n\tFri, 23 Sep 2022 01:21:52 +0200 (CEST)","from perceval.ideasonboard.com (perceval.ideasonboard.com\n\t[IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id 6A33F61F7B\n\tfor ;\n\tFri, 23 Sep 2022 01:21:51 +0200 (CEST)","from pendragon.ideasonboard.com (62-78-145-57.bb.dnainternet.fi\n\t[62.78.145.57])\n\tby perceval.ideasonboard.com (Postfix) with ESMTPSA id BBEF84F7;\n\tFri, 23 Sep 2022 01:21:50 +0200 (CEST)"],"DKIM-Signature":["v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org;\n\ts=mail; t=1663888912;\n\tbh=p9syFFh7ofYM4L1Yy/QmzXqISXbLhwJKci0nrH4lZ3g=;\n\th=Date:To:References:In-Reply-To:Subject:List-Id:List-Unsubscribe:\n\tList-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To:Cc:\n\tFrom;\n\tb=DmrbTEgNFwuKvJJyzLLoiLatJhxIDfTMRisMWTUOudOhXM+w6x6eeNGwuH8JXsITm\n\t6d6THQ3AQh8RjIaOBwfHA6WpmUiK6SDcrUAKA3nOOBXtblUXGDnrraJd5Xl/TuDY+F\n\tEd8t4NZau+RKQdoxK0rD5T3pWOhtdFfPMOmGfv2gNcGhh7aswgIvVOwfAyreybW3XT\n\tLXOSq3897DsEn4LVJwIWqdfkPcilvypqk2ct0kpI+08y8IUz10hergh8I/2kIDVqaa\n\tDISht7DFsByF3ZVTbS8feue7EoHCl91DA9+osskKRiwgq/PVeP/oWbib6ih+pBR/qK\n\tlFPHb43ah3bIA==","v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com;\n\ts=mail; t=1663888911;\n\tbh=p9syFFh7ofYM4L1Yy/QmzXqISXbLhwJKci0nrH4lZ3g=;\n\th=Date:From:To:Cc:Subject:References:In-Reply-To:From;\n\tb=dlQIvRid79c9yZ6LvE620fOwb1eMmX9zwiptIv/qdiqyBpORH/Ahjcx6Aali6YEAD\n\tmRC5z0NRN6kr6ZMXpGIEYw+uTyZuE9pdqHc3ikmUJHT1KJH0v9JJT8mMY/BNqTrDbh\n\tSCiX4iNjILpmqrRIgREL9SorV4vfBPWdg2V+8qEY="],"Authentication-Results":"lancelot.ideasonboard.com; dkim=pass (1024-bit key; \n\tunprotected) header.d=ideasonboard.com\n\theader.i=@ideasonboard.com\n\theader.b=\"dlQIvRid\"; dkim-atps=neutral","Date":"Fri, 23 Sep 2022 02:21:35 +0300","To":"Kieran Bingham ","Message-ID":"","References":"<20220908014200.28728-1-laurent.pinchart@ideasonboard.com>\n\t<20220908014200.28728-32-laurent.pinchart@ideasonboard.com>\n\t<166371757953.18961.3389671865776993038@Monstersaurus>","MIME-Version":"1.0","Content-Type":"text/plain; charset=utf-8","Content-Disposition":"inline","In-Reply-To":"<166371757953.18961.3389671865776993038@Monstersaurus>","Subject":"Re: [libcamera-devel] [PATCH v4 31/32] ipa: rkisp1: awb: Freeze AWB\n\twhen means are too small","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","From":"Laurent Pinchart via libcamera-devel\n\t","Reply-To":"Laurent Pinchart ","Cc":"libcamera-devel@lists.libcamera.org","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" "}},{"id":25104,"web_url":"https://patchwork.libcamera.org/comment/25104/","msgid":"","date":"2022-09-22T23:23:59","subject":"Re: [libcamera-devel] [PATCH v4 31/32] ipa: rkisp1: awb: Freeze AWB\n\twhen means are too small","submitter":{"id":2,"url":"https://patchwork.libcamera.org/api/people/2/","name":"Laurent Pinchart","email":"laurent.pinchart@ideasonboard.com"},"content":"Hi Jacopo,\n\nOn Thu, Sep 22, 2022 at 12:45:24PM +0200, Jacopo Mondi wrote:\n> On Thu, Sep 08, 2022 at 04:41:59AM +0300, Laurent Pinchart via libcamera-devel wrote:\n> > When the RGB means are too small, gains and color temperature can't be\n> > meaningfully calculated. Freeze the AWB in that case, using the\n> > previously calculated values.\n> >\n> > Signed-off-by: Laurent Pinchart \n> > ---\n> > src/ipa/rkisp1/algorithms/awb.cpp | 17 ++++++++++++++++-\n> > src/ipa/rkisp1/ipa_context.cpp | 3 +++\n> > src/ipa/rkisp1/ipa_context.h | 1 +\n> > 3 files changed, 20 insertions(+), 1 deletion(-)\n> >\n> > diff --git a/src/ipa/rkisp1/algorithms/awb.cpp b/src/ipa/rkisp1/algorithms/awb.cpp\n> > index 5f2535688c93..404ad66e6953 100644\n> > --- a/src/ipa/rkisp1/algorithms/awb.cpp\n> > +++ b/src/ipa/rkisp1/algorithms/awb.cpp\n> > @@ -31,6 +31,9 @@ namespace ipa::rkisp1::algorithms {\n> >\n> > LOG_DEFINE_CATEGORY(RkISP1Awb)\n> >\n> > +/* Minimum mean value below which AWB can't operate. */\n> > +constexpr double kMeanMinThreshold = 2.0;\n> > +\n> \n> Is this a software constraint specified somewhere ?\n\nWe compute the red and blue gains as\n\n\tgreenMean / (redMean + 1)\n\tgreenMean / (blueMean + 1)\n\nThe means are in the 0-255 range. A value lower than 2 is just too low\nto give any meaningful result. I expect we'll need to increase the\nthreshold actually, this is just an initial value to avoid numerical\ninstability in the calculations.\n\n> > Awb::Awb()\n> > \t: rgbMode_(false)\n> > {\n> > @@ -263,7 +266,17 @@ void Awb::process(IPAContext &context,\n> > \tgreenMean /= frameContext.awb.gains.green;\n> > \tblueMean /= frameContext.awb.gains.blue;\n> >\n> > -\tframeContext.awb.temperatureK = estimateCCT(redMean, greenMean, blueMean);\n> > +\t/*\n> > +\t * If the means are too small we don't have enough information to\n> > +\t * meaningfully calculate gains. Freeze the algorithm in that case.\n> > +\t */\n> > +\tif (redMean < kMeanMinThreshold && greenMean < kMeanMinThreshold &&\n> > +\t blueMean < kMeanMinThreshold) {\n> > +\t\tframeContext.awb.temperatureK = activeState.awb.temperatureK;\n> > +\t\treturn;\n> > +\t}\n> > +\n> > +\tactiveState.awb.temperatureK = estimateCCT(redMean, greenMean, blueMean);\n> >\n> > \t/*\n> > \t * Estimate the red and blue gains to apply in a grey world. The green\n> > @@ -290,6 +303,8 @@ void Awb::process(IPAContext &context,\n> > \tactiveState.awb.gains.automatic.blue = blueGain;\n> > \tactiveState.awb.gains.automatic.green = 1.0;\n> >\n> > +\tframeContext.awb.temperatureK = activeState.awb.temperatureK;\n> > +\n> > \tLOG(RkISP1Awb, Debug) << std::showpoint\n> > \t\t<< \"Means [\" << redMean << \", \" << greenMean << \", \" << blueMean\n> > \t\t<< \"], gains [\" << activeState.awb.gains.automatic.red << \", \"\n> > diff --git a/src/ipa/rkisp1/ipa_context.cpp b/src/ipa/rkisp1/ipa_context.cpp\n> > index 335cb32c538d..9cac05bec79b 100644\n> > --- a/src/ipa/rkisp1/ipa_context.cpp\n> > +++ b/src/ipa/rkisp1/ipa_context.cpp\n> > @@ -156,6 +156,9 @@ namespace libcamera::ipa::rkisp1 {\n> > * \\var IPAActiveState::awb.gains.automatic.blue\n> > * \\brief Automatic white balance gain for B channel\n> > *\n> > + * \\var IPAActiveState::awb.temperatureK\n> > + * \\brief Estimated color temperature\n> > + *\n> > * \\var IPAActiveState::awb.autoEnabled\n> > * \\brief Whether the Auto White Balance algorithm is enabled\n> > */\n> > diff --git a/src/ipa/rkisp1/ipa_context.h b/src/ipa/rkisp1/ipa_context.h\n> > index 53787320e565..d0bc9090a8f1 100644\n> > --- a/src/ipa/rkisp1/ipa_context.h\n> > +++ b/src/ipa/rkisp1/ipa_context.h\n> > @@ -68,6 +68,7 @@ struct IPAActiveState {\n> > \t\t\t} automatic;\n> > \t\t} gains;\n> >\n> > +\t\tunsigned int temperatureK;\n> > \t\tbool autoEnabled;\n> > \t} awb;\n> >","headers":{"Return-Path":"","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id A74C5C0DA4\n\tfor ;\n\tThu, 22 Sep 2022 23:24:17 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id B417B62226;\n\tFri, 23 Sep 2022 01:24:16 +0200 (CEST)","from perceval.ideasonboard.com (perceval.ideasonboard.com\n\t[213.167.242.64])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id A203461F7B\n\tfor ;\n\tFri, 23 Sep 2022 01:24:15 +0200 (CEST)","from pendragon.ideasonboard.com (62-78-145-57.bb.dnainternet.fi\n\t[62.78.145.57])\n\tby perceval.ideasonboard.com (Postfix) with ESMTPSA id 0D8274F7;\n\tFri, 23 Sep 2022 01:24:14 +0200 (CEST)"],"DKIM-Signature":["v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org;\n\ts=mail; t=1663889056;\n\tbh=OHQPiExzPxSJj7C6Dd8/uLyyko2pyLdbYRC7CIZy1aI=;\n\th=Date:To:References:In-Reply-To:Subject:List-Id:List-Unsubscribe:\n\tList-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To:Cc:\n\tFrom;\n\tb=psVFPh1YCmW8o8vXP6Kj4E8hjhFDG9XdN7C3dd6LSraoLVAETQlJx2KHCDHlJ7UmE\n\t6nvpPS6fbzLk+AM//T78os7iz9rExNVkT6N+WzOAtqye0nQ5djEv3eU9/klBRVX6UZ\n\t30nbg6NwXV/pyog14Nq2BP2zXL02PcQyApumIwrsdPsftQxh0qF3si/TYzXhqLE8U5\n\t5rMkLr8DXGoHXKFMxbVtJ5mBAlgPd6cxsavBeNVI+KJ8PnWQN5rCYbjwckZQlRuCbI\n\thaQNr+lh+RFwnuTX7/8SUPwaMNP4Y9Yuo07iAQSofWNUb2u3yQqmqwIWLWdpZeju20\n\tE2RtUWLvgM2nA==","v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com;\n\ts=mail; t=1663889055;\n\tbh=OHQPiExzPxSJj7C6Dd8/uLyyko2pyLdbYRC7CIZy1aI=;\n\th=Date:From:To:Cc:Subject:References:In-Reply-To:From;\n\tb=sD50QJAOzyLWfSg+whveOicFaszXm82eIL7253dVB0I01hpJdoakZZdDXuIDD09/j\n\tI2zdOGOzvSHgDFM65+wVRWS9UaoBOdUT5XjbjzJwfQUgCuvG55rV8bUOy5bAboM6Ot\n\tb4ejaPj3QWnKiQhTihn9H1wbI+855jsaucCM/5AQ="],"Authentication-Results":"lancelot.ideasonboard.com; dkim=pass (1024-bit key; \n\tunprotected) header.d=ideasonboard.com\n\theader.i=@ideasonboard.com\n\theader.b=\"sD50QJAO\"; dkim-atps=neutral","Date":"Fri, 23 Sep 2022 02:23:59 +0300","To":"Jacopo Mondi ","Message-ID":"","References":"<20220908014200.28728-1-laurent.pinchart@ideasonboard.com>\n\t<20220908014200.28728-32-laurent.pinchart@ideasonboard.com>\n\t<20220922104524.svbjcrkiduis4eke@uno.localdomain>","MIME-Version":"1.0","Content-Type":"text/plain; charset=utf-8","Content-Disposition":"inline","In-Reply-To":"<20220922104524.svbjcrkiduis4eke@uno.localdomain>","Subject":"Re: [libcamera-devel] [PATCH v4 31/32] ipa: rkisp1: awb: Freeze AWB\n\twhen means are too small","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","From":"Laurent Pinchart via libcamera-devel\n\t","Reply-To":"Laurent Pinchart ","Cc":"libcamera-devel@lists.libcamera.org","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" "}}]