[{"id":21092,"web_url":"https://patchwork.libcamera.org/comment/21092/","msgid":"","date":"2021-11-22T06:41:32","subject":"Re: [libcamera-devel] [PATCH v2 2/5] ipa: ipu3: agc: Standardize\n\tvocabulary on \"relative luminance\"","submitter":{"id":75,"url":"https://patchwork.libcamera.org/api/people/75/","name":"Jean-Michel Hautbois","email":"jeanmichel.hautbois@ideasonboard.com"},"content":"Hi Laurent,\n\nOn 19/11/2021 22:02, Laurent Pinchart wrote:\n> The AGC computes the average relative luminance of the frame and calls\n> the value \"normalized luma\", \"brightness\" or \"initialY\". The latter is\n> the most accurate term, as the relative luminance is abbreviated Y, but\n> the \"initial\" prefix isn't accurate.\n> \n> Standardize the vocabulary on \"relative luminance\" in code and comments,\n> abbreviating it to Y when needed.\n> \n> While at it, rename variables to match the libcamera coding style.\n> \n> Signed-off-by: Laurent Pinchart \nReviewed-by: Jean-Michel Hautbois \n> ---\n> Changes since v1:\n> \n> - Improve comment about relative luminance computation\n> ---\n> src/ipa/ipu3/algorithms/agc.cpp | 44 ++++++++++++++++-----------------\n> src/ipa/ipu3/algorithms/agc.h | 8 +++---\n> 2 files changed, 26 insertions(+), 26 deletions(-)\n> \n> diff --git a/src/ipa/ipu3/algorithms/agc.cpp b/src/ipa/ipu3/algorithms/agc.cpp\n> index 43a39ffd57d6..9cd2ded501ed 100644\n> --- a/src/ipa/ipu3/algorithms/agc.cpp\n> +++ b/src/ipa/ipu3/algorithms/agc.cpp\n> @@ -62,12 +62,12 @@ static constexpr double kEvGainTarget = 0.5;\n> static constexpr uint32_t kNumStartupFrames = 10;\n> \n> /*\n> - * Normalized luma value target.\n> + * Relative luminance target.\n> *\n> * It's a number that's chosen so that, when the camera points at a grey\n> * target, the resulting image brightness is considered right.\n> */\n> -static constexpr double kNormalizedLumaTarget = 0.16;\n> +static constexpr double kRelativeLuminanceTarget = 0.16;\n> \n> Agc::Agc()\n> \t: frameCount_(0), iqMean_(0.0), lineDuration_(0s), minShutterSpeed_(0s),\n> @@ -250,12 +250,12 @@ void Agc::computeExposure(IPAFrameContext &frameContext, double currentYGain)\n> }\n> \n> /**\n> - * \\brief Estimate the average brightness of the frame\n> + * \\brief Estimate the relative luminance of the frame with a given gain\n> * \\param[in] frameContext The shared IPA frame context\n> * \\param[in] grid The grid used to store the statistics in the IPU3\n> * \\param[in] stats The IPU3 statistics and ISP results\n> * \\param[in] currentYGain The gain calculated on the current brightness level\n> - * \\return The normalized luma\n> + * \\return The relative luminance\n> *\n> * Luma is the weighted sum of gamma-compressed R′G′B′ components of a color\n> * video. The luma values are normalized as 0.0 to 1.0, with 1.0 being a\n> @@ -263,12 +263,12 @@ void Agc::computeExposure(IPAFrameContext &frameContext, double currentYGain)\n> * luma here.\n> *\n> * More detailed information can be found in:\n> - * https://en.wikipedia.org/wiki/Luma_(video)\n> + * https://en.wikipedia.org/wiki/Relative_luminance\n> */\n> -double Agc::computeInitialY(IPAFrameContext &frameContext,\n> -\t\t\t const ipu3_uapi_grid_config &grid,\n> -\t\t\t const ipu3_uapi_stats_3a *stats,\n> -\t\t\t double currentYGain)\n> +double Agc::estimateLuminance(IPAFrameContext &frameContext,\n> +\t\t\t const ipu3_uapi_grid_config &grid,\n> +\t\t\t const ipu3_uapi_stats_3a *stats,\n> +\t\t\t double currentYGain)\n> {\n> \tdouble redSum = 0, greenSum = 0, blueSum = 0;\n> \n> @@ -288,14 +288,14 @@ double Agc::computeInitialY(IPAFrameContext &frameContext,\n> \t}\n> \n> \t/*\n> -\t * Estimate the sum of the brightness values, weighted with the gains\n> -\t * applied on the channels in AWB as the Rec. 601 luma.\n> +\t * Apply the AWB gains to approximate colours correctly, use the Rec.\n> +\t * 601 formula to calculate the relative luminance, and normalize it.\n> \t */\n> -\tdouble Y_sum = redSum * frameContext.awb.gains.red * .299 +\n> -\t\t greenSum * frameContext.awb.gains.green * .587 +\n> -\t\t blueSum * frameContext.awb.gains.blue * .114;\n> +\tdouble ySum = redSum * frameContext.awb.gains.red * 0.299\n> +\t\t + greenSum * frameContext.awb.gains.green * 0.587\n> +\t\t + blueSum * frameContext.awb.gains.blue * 0.114;\n> \n> -\treturn Y_sum / (grid.height * grid.width) / 255;\n> +\treturn ySum / (grid.height * grid.width) / 255;\n> }\n> \n> /**\n> @@ -311,22 +311,22 @@ void Agc::process(IPAContext &context, const ipu3_uapi_stats_3a *stats)\n> \tmeasureBrightness(stats, context.configuration.grid.bdsGrid);\n> \n> \tdouble currentYGain = 1.0;\n> -\tdouble targetY = kNormalizedLumaTarget;\n> +\tdouble yTarget = kRelativeLuminanceTarget;\n> \n> \t/*\n> \t * Do this calculation a few times as brightness increase can be\n> \t * non-linear when there are saturated regions.\n> \t */\n> -\tfor (int i = 0; i < 8; i++) {\n> -\t\tdouble initialY = computeInitialY(context.frameContext,\n> +\tfor (unsigned int i = 0; i < 8; i++) {\n> +\t\tdouble yValue = estimateLuminance(context.frameContext,\n> \t\t\t\t\t\t context.configuration.grid.bdsGrid,\n> \t\t\t\t\t\t stats, currentYGain);\n> -\t\tdouble extra_gain = std::min(10.0, targetY / (initialY + .001));\n> +\t\tdouble extra_gain = std::min(10.0, yTarget / (yValue + .001));\n> \n> \t\tcurrentYGain *= extra_gain;\n> -\t\tLOG(IPU3Agc, Debug) << \"Initial Y \" << initialY\n> -\t\t\t\t << \" target \" << targetY\n> -\t\t\t\t << \" gives gain \" << currentYGain;\n> +\t\tLOG(IPU3Agc, Debug) << \"Y value: \" << yValue\n> +\t\t\t\t << \", Y target: \" << yTarget\n> +\t\t\t\t << \", gives gain \" << currentYGain;\n> \t\tif (extra_gain < 1.01)\n> \t\t\tbreak;\n> \t}\n> diff --git a/src/ipa/ipu3/algorithms/agc.h b/src/ipa/ipu3/algorithms/agc.h\n> index 31c5a6e519d4..943c354a820e 100644\n> --- a/src/ipa/ipu3/algorithms/agc.h\n> +++ b/src/ipa/ipu3/algorithms/agc.h\n> @@ -35,10 +35,10 @@ private:\n> \t\t\t const ipu3_uapi_grid_config &grid);\n> \tvoid filterExposure();\n> \tvoid computeExposure(IPAFrameContext &frameContext, double currentYGain);\n> -\tdouble computeInitialY(IPAFrameContext &frameContext,\n> -\t\t\t const ipu3_uapi_grid_config &grid,\n> -\t\t\t const ipu3_uapi_stats_3a *stats,\n> -\t\t\t double currentYGain);\n> +\tdouble estimateLuminance(IPAFrameContext &frameContext,\n> +\t\t\t\t const ipu3_uapi_grid_config &grid,\n> +\t\t\t\t const ipu3_uapi_stats_3a *stats,\n> +\t\t\t\t double currentYGain);\n> \n> \tuint64_t frameCount_;\n> \n>","headers":{"Return-Path":"","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id 902EEBF415\n\tfor ;\n\tMon, 22 Nov 2021 06:41:37 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id 8B2AA6036F;\n\tMon, 22 Nov 2021 07:41:36 +0100 (CET)","from perceval.ideasonboard.com (perceval.ideasonboard.com\n\t[213.167.242.64])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id 2EB6460228\n\tfor ;\n\tMon, 22 Nov 2021 07:41:35 +0100 (CET)","from [IPV6:2a01:e0a:169:7140:933b:1c23:4e2b:3c0f] (unknown\n\t[IPv6:2a01:e0a:169:7140:933b:1c23:4e2b:3c0f])\n\tby perceval.ideasonboard.com (Postfix) with ESMTPSA id C3CCD92A;\n\tMon, 22 Nov 2021 07:41:34 +0100 (CET)"],"Authentication-Results":"lancelot.ideasonboard.com; dkim=pass (1024-bit key;\n\tunprotected) header.d=ideasonboard.com header.i=@ideasonboard.com\n\theader.b=\"Ts4C9TG7\"; dkim-atps=neutral","DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com;\n\ts=mail; t=1637563294;\n\tbh=HzRnrhUYtjo6IqVG2Jm3kvVRjjMDuZ8B0oqHBoe2Vtk=;\n\th=Date:Subject:To:References:From:In-Reply-To:From;\n\tb=Ts4C9TG73Pvi+n62Ak+wjyzRVXhMkpN1DBMIojrx69K6t1IeIXXClQ9gsBJ53DZJ8\n\tD2tjFn9AFE+alTyH3zyAhhcdcBsmHqIEOjsG9/9rb9uUcUaKgx6yMSzsE/C3W92s7B\n\tuK89x6fcSmJj0/ZmdCinZ4mwXf1tewl8XiWuxfXQ=","Message-ID":"","Date":"Mon, 22 Nov 2021 07:41:32 +0100","MIME-Version":"1.0","User-Agent":"Mozilla/5.0 (X11; Linux x86_64; rv:91.0) Gecko/20100101\n\tThunderbird/91.3.1","Content-Language":"en-US","To":"Laurent Pinchart ,\n\tlibcamera-devel@lists.libcamera.org","References":"<20211119210239.18540-1-laurent.pinchart@ideasonboard.com>\n\t<20211119210239.18540-3-laurent.pinchart@ideasonboard.com>","From":"Jean-Michel Hautbois ","In-Reply-To":"<20211119210239.18540-3-laurent.pinchart@ideasonboard.com>","Content-Type":"text/plain; charset=UTF-8; format=flowed","Content-Transfer-Encoding":"8bit","Subject":"Re: [libcamera-devel] [PATCH v2 2/5] ipa: ipu3: agc: Standardize\n\tvocabulary on \"relative luminance\"","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" "}},{"id":21098,"web_url":"https://patchwork.libcamera.org/comment/21098/","msgid":"<163758764666.1089182.18366636681902161708@Monstersaurus>","date":"2021-11-22T13:27:26","subject":"Re: [libcamera-devel] [PATCH v2 2/5] ipa: ipu3: agc: Standardize\n\tvocabulary on \"relative luminance\"","submitter":{"id":4,"url":"https://patchwork.libcamera.org/api/people/4/","name":"Kieran Bingham","email":"kieran.bingham@ideasonboard.com"},"content":"Quoting Laurent Pinchart (2021-11-19 21:02:36)\n> The AGC computes the average relative luminance of the frame and calls\n> the value \"normalized luma\", \"brightness\" or \"initialY\". The latter is\n> the most accurate term, as the relative luminance is abbreviated Y, but\n> the \"initial\" prefix isn't accurate.\n> \n> Standardize the vocabulary on \"relative luminance\" in code and comments,\n> abbreviating it to Y when needed.\n> \n> While at it, rename variables to match the libcamera coding style.\n> \n> Signed-off-by: Laurent Pinchart \n\n\nReviewed-by: Kieran Bingham \n\n> ---\n> Changes since v1:\n> \n> - Improve comment about relative luminance computation\n> ---\n> src/ipa/ipu3/algorithms/agc.cpp | 44 ++++++++++++++++-----------------\n> src/ipa/ipu3/algorithms/agc.h | 8 +++---\n> 2 files changed, 26 insertions(+), 26 deletions(-)\n> \n> diff --git a/src/ipa/ipu3/algorithms/agc.cpp b/src/ipa/ipu3/algorithms/agc.cpp\n> index 43a39ffd57d6..9cd2ded501ed 100644\n> --- a/src/ipa/ipu3/algorithms/agc.cpp\n> +++ b/src/ipa/ipu3/algorithms/agc.cpp\n> @@ -62,12 +62,12 @@ static constexpr double kEvGainTarget = 0.5;\n> static constexpr uint32_t kNumStartupFrames = 10;\n> \n> /*\n> - * Normalized luma value target.\n> + * Relative luminance target.\n> *\n> * It's a number that's chosen so that, when the camera points at a grey\n> * target, the resulting image brightness is considered right.\n> */\n> -static constexpr double kNormalizedLumaTarget = 0.16;\n> +static constexpr double kRelativeLuminanceTarget = 0.16;\n> \n> Agc::Agc()\n> : frameCount_(0), iqMean_(0.0), lineDuration_(0s), minShutterSpeed_(0s),\n> @@ -250,12 +250,12 @@ void Agc::computeExposure(IPAFrameContext &frameContext, double currentYGain)\n> }\n> \n> /**\n> - * \\brief Estimate the average brightness of the frame\n> + * \\brief Estimate the relative luminance of the frame with a given gain\n> * \\param[in] frameContext The shared IPA frame context\n> * \\param[in] grid The grid used to store the statistics in the IPU3\n> * \\param[in] stats The IPU3 statistics and ISP results\n> * \\param[in] currentYGain The gain calculated on the current brightness level\n> - * \\return The normalized luma\n> + * \\return The relative luminance\n> *\n> * Luma is the weighted sum of gamma-compressed R′G′B′ components of a color\n> * video. The luma values are normalized as 0.0 to 1.0, with 1.0 being a\n> @@ -263,12 +263,12 @@ void Agc::computeExposure(IPAFrameContext &frameContext, double currentYGain)\n> * luma here.\n> *\n> * More detailed information can be found in:\n> - * https://en.wikipedia.org/wiki/Luma_(video)\n> + * https://en.wikipedia.org/wiki/Relative_luminance\n> */\n> -double Agc::computeInitialY(IPAFrameContext &frameContext,\n> - const ipu3_uapi_grid_config &grid,\n> - const ipu3_uapi_stats_3a *stats,\n> - double currentYGain)\n> +double Agc::estimateLuminance(IPAFrameContext &frameContext,\n> + const ipu3_uapi_grid_config &grid,\n> + const ipu3_uapi_stats_3a *stats,\n> + double currentYGain)\n> {\n> double redSum = 0, greenSum = 0, blueSum = 0;\n> \n> @@ -288,14 +288,14 @@ double Agc::computeInitialY(IPAFrameContext &frameContext,\n> }\n> \n> /*\n> - * Estimate the sum of the brightness values, weighted with the gains\n> - * applied on the channels in AWB as the Rec. 601 luma.\n> + * Apply the AWB gains to approximate colours correctly, use the Rec.\n> + * 601 formula to calculate the relative luminance, and normalize it.\n> */\n> - double Y_sum = redSum * frameContext.awb.gains.red * .299 +\n> - greenSum * frameContext.awb.gains.green * .587 +\n> - blueSum * frameContext.awb.gains.blue * .114;\n> + double ySum = redSum * frameContext.awb.gains.red * 0.299\n> + + greenSum * frameContext.awb.gains.green * 0.587\n> + + blueSum * frameContext.awb.gains.blue * 0.114;\n> \n> - return Y_sum / (grid.height * grid.width) / 255;\n> + return ySum / (grid.height * grid.width) / 255;\n> }\n> \n> /**\n> @@ -311,22 +311,22 @@ void Agc::process(IPAContext &context, const ipu3_uapi_stats_3a *stats)\n> measureBrightness(stats, context.configuration.grid.bdsGrid);\n> \n> double currentYGain = 1.0;\n> - double targetY = kNormalizedLumaTarget;\n> + double yTarget = kRelativeLuminanceTarget;\n> \n> /*\n> * Do this calculation a few times as brightness increase can be\n> * non-linear when there are saturated regions.\n> */\n> - for (int i = 0; i < 8; i++) {\n> - double initialY = computeInitialY(context.frameContext,\n> + for (unsigned int i = 0; i < 8; i++) {\n> + double yValue = estimateLuminance(context.frameContext,\n> context.configuration.grid.bdsGrid,\n> stats, currentYGain);\n> - double extra_gain = std::min(10.0, targetY / (initialY + .001));\n> + double extra_gain = std::min(10.0, yTarget / (yValue + .001));\n> \n> currentYGain *= extra_gain;\n> - LOG(IPU3Agc, Debug) << \"Initial Y \" << initialY\n> - << \" target \" << targetY\n> - << \" gives gain \" << currentYGain;\n> + LOG(IPU3Agc, Debug) << \"Y value: \" << yValue\n> + << \", Y target: \" << yTarget\n> + << \", gives gain \" << currentYGain;\n> if (extra_gain < 1.01)\n> break;\n> }\n> diff --git a/src/ipa/ipu3/algorithms/agc.h b/src/ipa/ipu3/algorithms/agc.h\n> index 31c5a6e519d4..943c354a820e 100644\n> --- a/src/ipa/ipu3/algorithms/agc.h\n> +++ b/src/ipa/ipu3/algorithms/agc.h\n> @@ -35,10 +35,10 @@ private:\n> const ipu3_uapi_grid_config &grid);\n> void filterExposure();\n> void computeExposure(IPAFrameContext &frameContext, double currentYGain);\n> - double computeInitialY(IPAFrameContext &frameContext,\n> - const ipu3_uapi_grid_config &grid,\n> - const ipu3_uapi_stats_3a *stats,\n> - double currentYGain);\n> + double estimateLuminance(IPAFrameContext &frameContext,\n> + const ipu3_uapi_grid_config &grid,\n> + const ipu3_uapi_stats_3a *stats,\n> + double currentYGain);\n> \n> uint64_t frameCount_;\n> \n> -- \n> Regards,\n> \n> Laurent Pinchart\n>","headers":{"Return-Path":"","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id 64C12BDB13\n\tfor ;\n\tMon, 22 Nov 2021 13:27:31 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id 9890B6036F;\n\tMon, 22 Nov 2021 14:27:30 +0100 (CET)","from perceval.ideasonboard.com (perceval.ideasonboard.com\n\t[IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id 7AC8C60230\n\tfor ;\n\tMon, 22 Nov 2021 14:27:29 +0100 (CET)","from pendragon.ideasonboard.com\n\t(cpc89244-aztw30-2-0-cust3082.18-1.cable.virginm.net [86.31.172.11])\n\tby perceval.ideasonboard.com (Postfix) with ESMTPSA id 10AB51B40;\n\tMon, 22 Nov 2021 14:27:29 +0100 (CET)"],"Authentication-Results":"lancelot.ideasonboard.com; dkim=pass (1024-bit key;\n\tunprotected) header.d=ideasonboard.com header.i=@ideasonboard.com\n\theader.b=\"DGGYH3gn\"; dkim-atps=neutral","DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com;\n\ts=mail; t=1637587649;\n\tbh=8ebEuz/V57ZKqnux9ni9j42nV/dmgd6fIerySX/oadQ=;\n\th=In-Reply-To:References:Subject:From:To:Date:From;\n\tb=DGGYH3gnNGaHXKLsvLGBiSgXGsd/dcn30hh5Jiapii6B9CNIodZOGYyymsO8eKXvd\n\tG4z75Lg/qjKhm6hOUauq8d97L0f5WYShUS84LVyWntH/e5wHeukdVP4g2d/MhtEBCE\n\tm7XbkMYaKPQKkEppGuL2+kkPXZiAGTKzCZLm3ZNs=","Content-Type":"text/plain; charset=\"utf-8\"","MIME-Version":"1.0","Content-Transfer-Encoding":"quoted-printable","In-Reply-To":"<20211119210239.18540-3-laurent.pinchart@ideasonboard.com>","References":"<20211119210239.18540-1-laurent.pinchart@ideasonboard.com>\n\t<20211119210239.18540-3-laurent.pinchart@ideasonboard.com>","From":"Kieran Bingham ","To":"Laurent Pinchart ,\n\tlibcamera-devel@lists.libcamera.org","Date":"Mon, 22 Nov 2021 13:27:26 +0000","Message-ID":"<163758764666.1089182.18366636681902161708@Monstersaurus>","User-Agent":"alot/0.10","Subject":"Re: [libcamera-devel] [PATCH v2 2/5] ipa: ipu3: agc: Standardize\n\tvocabulary on \"relative luminance\"","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" "}}]