{"id":13333,"url":"https://patchwork.libcamera.org/api/patches/13333/?format=json","web_url":"https://patchwork.libcamera.org/patch/13333/","project":{"id":1,"url":"https://patchwork.libcamera.org/api/projects/1/?format=json","name":"libcamera","link_name":"libcamera","list_id":"libcamera_core","list_email":"libcamera-devel@lists.libcamera.org","web_url":"","scm_url":"","webscm_url":""},"msgid":"<20210812165243.276977-6-jeanmichel.hautbois@ideasonboard.com>","date":"2021-08-12T16:52:38","name":"[libcamera-devel,v2,05/10] ipa: ipu3: Introduce a modular contrast algorithm","commit_ref":null,"pull_url":null,"state":"superseded","archived":false,"hash":"58491ab909cfb9e142a6d53a2f3ef8083b935f20","submitter":{"id":75,"url":"https://patchwork.libcamera.org/api/people/75/?format=json","name":"Jean-Michel Hautbois","email":"jeanmichel.hautbois@ideasonboard.com"},"delegate":null,"mbox":"https://patchwork.libcamera.org/patch/13333/mbox/","series":[{"id":2347,"url":"https://patchwork.libcamera.org/api/series/2347/?format=json","web_url":"https://patchwork.libcamera.org/project/libcamera/list/?series=2347","date":"2021-08-12T16:52:33","name":"IPU3: Introduce modularity for algorithms","version":2,"mbox":"https://patchwork.libcamera.org/series/2347/mbox/"}],"comments":"https://patchwork.libcamera.org/api/patches/13333/comments/","check":"pending","checks":"https://patchwork.libcamera.org/api/patches/13333/checks/","tags":{},"headers":{"Return-Path":"","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id A31CABD87D\n\tfor ;\n\tThu, 12 Aug 2021 16:52:54 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id 6D542688D2;\n\tThu, 12 Aug 2021 18:52:54 +0200 (CEST)","from perceval.ideasonboard.com (perceval.ideasonboard.com\n\t[213.167.242.64])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id BC33268864\n\tfor ;\n\tThu, 12 Aug 2021 18:52:47 +0200 (CEST)","from tatooine.ideasonboard.com (unknown\n\t[IPv6:2a01:e0a:169:7140:3d6d:7284:3c48:5edc])\n\tby perceval.ideasonboard.com (Postfix) with ESMTPSA id 7294F49A;\n\tThu, 12 Aug 2021 18:52:47 +0200 (CEST)"],"Authentication-Results":"lancelot.ideasonboard.com;\n\tdkim=fail reason=\"signature verification failed\" (1024-bit key;\n\tunprotected) header.d=ideasonboard.com header.i=@ideasonboard.com\n\theader.b=\"De0rCV2V\"; dkim-atps=neutral","DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com;\n\ts=mail; t=1628787167;\n\tbh=Ma+KxMqwSLTIzxe5tm03n4LkUT0lfkzLgc5Q0BtVa4o=;\n\th=From:To:Cc:Subject:Date:In-Reply-To:References:From;\n\tb=De0rCV2VnDCtylOmEoPvRIt6wxB0clfbYBcBaP3Chpf1d+ksnYpFHrp/aazHK0Yhj\n\t3tbUWtgME66cFPYGEpLEg+H2UF9lJTRKA1TKmHOP95biPAham8BGu9NghDYlnB5U9U\n\tD4qwRa598b57h7mRIvLaJDAJwLbxBIZ8eNaRR83U=","From":"Jean-Michel Hautbois ","To":"libcamera-devel@lists.libcamera.org","Date":"Thu, 12 Aug 2021 18:52:38 +0200","Message-Id":"<20210812165243.276977-6-jeanmichel.hautbois@ideasonboard.com>","X-Mailer":"git-send-email 2.30.2","In-Reply-To":"<20210812165243.276977-1-jeanmichel.hautbois@ideasonboard.com>","References":"<20210812165243.276977-1-jeanmichel.hautbois@ideasonboard.com>","MIME-Version":"1.0","Content-Transfer-Encoding":"8bit","Subject":"[libcamera-devel] [PATCH v2 05/10] ipa: ipu3: Introduce a modular\n\tcontrast algorithm","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" "},"content":"Introduce a new algorithm to manage the contrast handling of the IPU3.\n\nThe initial algorithm is chosen to configure the gamma contrast curve\nwhich moves the implementation out of AWB for simplicity. As it is\ninitialised with a default gamma value of 1.1, there is no need to use\nthe default table at init time anymore.\n\nThis demonstrates the way to use process() call when the EventStatReady\ncomes in. The function calculates the LUT in the context of a frame, and\nwhen prepare() is called, the parameters are filled with the updated\nvalues.\n\nAGC needs to be modified to take the new process interface into account.\n\nSigned-off-by: Kieran Bingham \nSigned-off-by: Jean-Michel Hautbois \n---\n src/ipa/ipu3/algorithms/algorithm.h | 2 ++\n src/ipa/ipu3/algorithms/contrast.cpp | 51 ++++++++++++++++++++++++++++\n src/ipa/ipu3/algorithms/contrast.h | 33 ++++++++++++++++++\n src/ipa/ipu3/algorithms/meson.build | 3 +-\n src/ipa/ipu3/ipa_context.h | 8 +++++\n src/ipa/ipu3/ipu3.cpp | 12 +++++--\n src/ipa/ipu3/ipu3_agc.cpp | 9 +++--\n src/ipa/ipu3/ipu3_agc.h | 5 +--\n src/ipa/ipu3/ipu3_awb.cpp | 41 ++--------------------\n src/ipa/ipu3/ipu3_awb.h | 2 +-\n 10 files changed, 114 insertions(+), 52 deletions(-)\n create mode 100644 src/ipa/ipu3/algorithms/contrast.cpp\n create mode 100644 src/ipa/ipu3/algorithms/contrast.h","diff":"diff --git a/src/ipa/ipu3/algorithms/algorithm.h b/src/ipa/ipu3/algorithms/algorithm.h\nindex b571f55a..13e5f2d4 100644\n--- a/src/ipa/ipu3/algorithms/algorithm.h\n+++ b/src/ipa/ipu3/algorithms/algorithm.h\n@@ -28,6 +28,8 @@ public:\n \t}\n \n \tvirtual void prepare([[maybe_unused]] IPAContext &context, [[maybe_unused]] ipu3_uapi_params ¶ms) {}\n+\n+\tvirtual void process([[maybe_unused]] IPAContext &context, [[maybe_unused]] const ipu3_uapi_stats_3a *stats) {}\n };\n \n } /* namespace ipa::ipu3 */\ndiff --git a/src/ipa/ipu3/algorithms/contrast.cpp b/src/ipa/ipu3/algorithms/contrast.cpp\nnew file mode 100644\nindex 00000000..bc00490f\n--- /dev/null\n+++ b/src/ipa/ipu3/algorithms/contrast.cpp\n@@ -0,0 +1,51 @@\n+/* SPDX-License-Identifier: LGPL-2.1-or-later */\n+/*\n+ * Copyright (C) 2021, Ideas On Board\n+ *\n+ * constrast.cpp - IPU3 Contrast and Gamma control\n+ */\n+\n+#include \"contrast.h\"\n+\n+#include \n+\n+#include \n+\n+namespace libcamera {\n+\n+namespace ipa::ipu3::algorithms {\n+\n+LOG_DEFINE_CATEGORY(IPU3Contrast)\n+\n+Contrast::Contrast()\n+\t: gamma_(1.0)\n+{\n+}\n+\n+void Contrast::prepare([[maybe_unused]] IPAContext &context, ipu3_uapi_params ¶ms)\n+{\n+\tparams.use.acc_gamma = 1;\n+\tparams.acc_param.gamma.gc_ctrl.enable = 1;\n+\n+\tfor (uint32_t i = 0; i < IPU3_UAPI_GAMMA_CORR_LUT_ENTRIES; i++)\n+\t\tparams.acc_param.gamma.gc_lut.lut[i] = context.frameContext.contrast.lut[i];\n+}\n+\n+void Contrast::process([[maybe_unused]] IPAContext &context, [[maybe_unused]] const ipu3_uapi_stats_3a *stats)\n+{\n+\t/* Limit the gamma effect for now */\n+\tgamma_ = 1.1;\n+\n+\t/* Plot the gamma curve into the look up table */\n+\tfor (uint32_t i = 0; i < IPU3_UAPI_GAMMA_CORR_LUT_ENTRIES; i++) {\n+\t\tdouble j = i / 255.0;\n+\t\tdouble gamma = std::pow(j, 1.0 / gamma_);\n+\n+\t\t/* The maximum value 255 is represented on 13 bits in the IPU3 */\n+\t\tcontext.frameContext.contrast.lut[i] = gamma * 8191;\n+\t}\n+}\n+\n+} /* namespace ipa::ipu3::algorithms */\n+\n+} /* namespace libcamera */\ndiff --git a/src/ipa/ipu3/algorithms/contrast.h b/src/ipa/ipu3/algorithms/contrast.h\nnew file mode 100644\nindex 00000000..50cb626f\n--- /dev/null\n+++ b/src/ipa/ipu3/algorithms/contrast.h\n@@ -0,0 +1,33 @@\n+/* SPDX-License-Identifier: LGPL-2.1-or-later */\n+/*\n+ * Copyright (C) 2021, Google\n+ *\n+ * constrast.h - IPU3 Contrast and Gamma control\n+ */\n+#ifndef __LIBCAMERA_IPU3_ALGORITHMS_CONTRAST_H__\n+#define __LIBCAMERA_IPU3_ALGORITHMS_CONTRAST_H__\n+\n+#include \"algorithm.h\"\n+\n+namespace libcamera {\n+\n+namespace ipa::ipu3::algorithms {\n+\n+class Contrast : public Algorithm\n+{\n+public:\n+\tContrast();\n+\t~Contrast() = default;\n+\n+\tvoid prepare(IPAContext &context, ipu3_uapi_params ¶ms) override;\n+\tvoid process(IPAContext &context, const ipu3_uapi_stats_3a *stats) override;\n+\n+private:\n+\tdouble gamma_;\n+};\n+\n+} /* namespace ipa::ipu3::algorithms */\n+\n+} /* namespace libcamera */\n+\n+#endif /* __LIBCAMERA_IPU3_ALGORITHMS_CONTRAST_H__ */\ndiff --git a/src/ipa/ipu3/algorithms/meson.build b/src/ipa/ipu3/algorithms/meson.build\nindex 3fb3ce56..69a59096 100644\n--- a/src/ipa/ipu3/algorithms/meson.build\n+++ b/src/ipa/ipu3/algorithms/meson.build\n@@ -1,5 +1,6 @@\n # SPDX-License-Identifier: CC0-1.0\n \n ipu3_ipa_algorithms = files([\n- 'grid.cpp',\n+ 'contrast.cpp',\n+ 'grid.cpp',\n ])\ndiff --git a/src/ipa/ipu3/ipa_context.h b/src/ipa/ipu3/ipa_context.h\nindex 90b2f2c2..e995c663 100644\n--- a/src/ipa/ipu3/ipa_context.h\n+++ b/src/ipa/ipu3/ipa_context.h\n@@ -33,6 +33,14 @@ struct IPAConfiguration {\n * lifetime, though for now a single instance is used.\n */\n struct IPAFrameContext {\n+\tstruct Agc {\n+\t\tuint32_t exposure;\n+\t\tdouble gain;\n+\t} agc;\n+\n+\tstruct Contrast {\n+\t\tuint16_t lut[IPU3_UAPI_GAMMA_CORR_LUT_ENTRIES];\n+\t} contrast;\n };\n \n /* Global context of the IPA */\ndiff --git a/src/ipa/ipu3/ipu3.cpp b/src/ipa/ipu3/ipu3.cpp\nindex 394a7a45..e09adfc3 100644\n--- a/src/ipa/ipu3/ipu3.cpp\n+++ b/src/ipa/ipu3/ipu3.cpp\n@@ -30,6 +30,7 @@\n #include \"libcamera/internal/mapped_framebuffer.h\"\n \n #include \"algorithms/algorithm.h\"\n+#include \"algorithms/contrast.h\"\n #include \"algorithms/grid.h\"\n #include \"ipu3_agc.h\"\n #include \"ipu3_awb.h\"\n@@ -167,6 +168,7 @@ int IPAIPU3::init(const IPASettings &settings,\n \t*ipaControls = ControlInfoMap(std::move(controls), controls::controls);\n \t/* Construct our Algorithms */\n \talgorithms_.emplace_back(new algorithms::Grid());\n+\talgorithms_.emplace_back(new algorithms::Contrast());\n \n \treturn 0;\n }\n@@ -309,7 +311,7 @@ void IPAIPU3::fillParams(unsigned int frame, ipu3_uapi_params *params)\n \t\talgo->prepare(context_, params_);\n \n \tif (agcAlgo_->updateControls())\n-\t\tawbAlgo_->updateWbParameters(params_, agcAlgo_->gamma());\n+\t\tawbAlgo_->updateWbParameters(params_);\n \n \t*params = params_;\n \n@@ -324,9 +326,15 @@ void IPAIPU3::parseStatistics(unsigned int frame,\n \t\t\t [[maybe_unused]] const ipu3_uapi_stats_3a *stats)\n {\n \tControlList ctrls(controls::controls);\n+\tfor (auto const &algo : algorithms_)\n+\t\talgo->process(context_, stats);\n \n \tdouble gain = camHelper_->gain(gain_);\n-\tagcAlgo_->process(stats, exposure_, gain);\n+\tcontext_.frameContext.agc.exposure = exposure_;\n+\tcontext_.frameContext.agc.gain = gain;\n+\tagcAlgo_->process(context_, stats);\n+\texposure_ = context_.frameContext.agc.exposure;\n+\tgain = context_.frameContext.agc.gain;\n \tgain_ = camHelper_->gainCode(gain);\n \n \tawbAlgo_->calculateWBGains(stats);\ndiff --git a/src/ipa/ipu3/ipu3_agc.cpp b/src/ipa/ipu3/ipu3_agc.cpp\nindex 408eb849..c6782ff4 100644\n--- a/src/ipa/ipu3/ipu3_agc.cpp\n+++ b/src/ipa/ipu3/ipu3_agc.cpp\n@@ -52,7 +52,7 @@ static constexpr uint8_t kCellSize = 8;\n \n IPU3Agc::IPU3Agc()\n \t: frameCount_(0), lastFrame_(0), converged_(false),\n-\t updateControls_(false), iqMean_(0.0), gamma_(1.0),\n+\t updateControls_(false), iqMean_(0.0),\n \t lineDuration_(0s), maxExposureTime_(0s),\n \t prevExposure_(0s), prevExposureNoDg_(0s),\n \t currentExposure_(0s), currentExposureNoDg_(0s)\n@@ -104,9 +104,6 @@ void IPU3Agc::processBrightness(const ipu3_uapi_stats_3a *stats)\n \t\t}\n \t}\n \n-\t/* Limit the gamma effect for now */\n-\tgamma_ = 1.1;\n-\n \t/* Estimate the quantile mean of the top 2% of the histogram */\n \tiqMean_ = Histogram(Span(hist)).interQuantileMean(0.98, 1.0);\n }\n@@ -193,8 +190,10 @@ void IPU3Agc::lockExposureGain(uint32_t &exposure, double &gain)\n \tlastFrame_ = frameCount_;\n }\n \n-void IPU3Agc::process(const ipu3_uapi_stats_3a *stats, uint32_t &exposure, double &gain)\n+void IPU3Agc::process(IPAContext &context, const ipu3_uapi_stats_3a *stats)\n {\n+\tuint32_t &exposure = context.frameContext.agc.exposure;\n+\tdouble &gain = context.frameContext.agc.gain;\n \tprocessBrightness(stats);\n \tlockExposureGain(exposure, gain);\n \tframeCount_++;\ndiff --git a/src/ipa/ipu3/ipu3_agc.h b/src/ipa/ipu3/ipu3_agc.h\nindex f00b98d6..8b32e52f 100644\n--- a/src/ipa/ipu3/ipu3_agc.h\n+++ b/src/ipa/ipu3/ipu3_agc.h\n@@ -30,11 +30,9 @@ public:\n \t~IPU3Agc() = default;\n \n \tvoid initialise(struct ipu3_uapi_grid_config &bdsGrid, const IPACameraSensorInfo &sensorInfo);\n-\tvoid process(const ipu3_uapi_stats_3a *stats, uint32_t &exposure, double &gain);\n+\tvoid process(IPAContext &context, const ipu3_uapi_stats_3a *stats) override;\n \tbool converged() { return converged_; }\n \tbool updateControls() { return updateControls_; }\n-\t/* \\todo Use a metadata exchange between IPAs */\n-\tdouble gamma() { return gamma_; }\n \n private:\n \tvoid processBrightness(const ipu3_uapi_stats_3a *stats);\n@@ -50,7 +48,6 @@ private:\n \tbool updateControls_;\n \n \tdouble iqMean_;\n-\tdouble gamma_;\n \n \tDuration lineDuration_;\n \tDuration maxExposureTime_;\ndiff --git a/src/ipa/ipu3/ipu3_awb.cpp b/src/ipa/ipu3/ipu3_awb.cpp\nindex 911760b3..0f3bcdc9 100644\n--- a/src/ipa/ipu3/ipu3_awb.cpp\n+++ b/src/ipa/ipu3/ipu3_awb.cpp\n@@ -114,31 +114,6 @@ static const struct ipu3_uapi_ccm_mat_config imguCssCcmDefault = {\n \t0, 0, 8191, 0\n };\n \n-/* Default settings for Gamma correction */\n-const struct ipu3_uapi_gamma_corr_lut imguCssGammaLut = { {\n-\t63, 79, 95, 111, 127, 143, 159, 175, 191, 207, 223, 239, 255, 271, 287,\n-\t303, 319, 335, 351, 367, 383, 399, 415, 431, 447, 463, 479, 495, 511,\n-\t527, 543, 559, 575, 591, 607, 623, 639, 655, 671, 687, 703, 719, 735,\n-\t751, 767, 783, 799, 815, 831, 847, 863, 879, 895, 911, 927, 943, 959,\n-\t975, 991, 1007, 1023, 1039, 1055, 1071, 1087, 1103, 1119, 1135, 1151,\n-\t1167, 1183, 1199, 1215, 1231, 1247, 1263, 1279, 1295, 1311, 1327, 1343,\n-\t1359, 1375, 1391, 1407, 1423, 1439, 1455, 1471, 1487, 1503, 1519, 1535,\n-\t1551, 1567, 1583, 1599, 1615, 1631, 1647, 1663, 1679, 1695, 1711, 1727,\n-\t1743, 1759, 1775, 1791, 1807, 1823, 1839, 1855, 1871, 1887, 1903, 1919,\n-\t1935, 1951, 1967, 1983, 1999, 2015, 2031, 2047, 2063, 2079, 2095, 2111,\n-\t2143, 2175, 2207, 2239, 2271, 2303, 2335, 2367, 2399, 2431, 2463, 2495,\n-\t2527, 2559, 2591, 2623, 2655, 2687, 2719, 2751, 2783, 2815, 2847, 2879,\n-\t2911, 2943, 2975, 3007, 3039, 3071, 3103, 3135, 3167, 3199, 3231, 3263,\n-\t3295, 3327, 3359, 3391, 3423, 3455, 3487, 3519, 3551, 3583, 3615, 3647,\n-\t3679, 3711, 3743, 3775, 3807, 3839, 3871, 3903, 3935, 3967, 3999, 4031,\n-\t4063, 4095, 4127, 4159, 4223, 4287, 4351, 4415, 4479, 4543, 4607, 4671,\n-\t4735, 4799, 4863, 4927, 4991, 5055, 5119, 5183, 5247, 5311, 5375, 5439,\n-\t5503, 5567, 5631, 5695, 5759, 5823, 5887, 5951, 6015, 6079, 6143, 6207,\n-\t6271, 6335, 6399, 6463, 6527, 6591, 6655, 6719, 6783, 6847, 6911, 6975,\n-\t7039, 7103, 7167, 7231, 7295, 7359, 7423, 7487, 7551, 7615, 7679, 7743,\n-\t7807, 7871, 7935, 7999, 8063, 8127, 8191\n-} };\n-\n IPU3Awb::IPU3Awb()\n \t: Algorithm()\n {\n@@ -179,10 +154,6 @@ void IPU3Awb::initialise(ipu3_uapi_params ¶ms, const Size &bdsOutputSize, st\n \tparams.use.acc_ccm = 1;\n \tparams.acc_param.ccm = imguCssCcmDefault;\n \n-\tparams.use.acc_gamma = 1;\n-\tparams.acc_param.gamma.gc_lut = imguCssGammaLut;\n-\tparams.acc_param.gamma.gc_ctrl.enable = 1;\n-\n \tzones_.reserve(kAwbStatsSizeX * kAwbStatsSizeY);\n }\n \n@@ -332,7 +303,7 @@ void IPU3Awb::calculateWBGains(const ipu3_uapi_stats_3a *stats)\n \t}\n }\n \n-void IPU3Awb::updateWbParameters(ipu3_uapi_params ¶ms, double agcGamma)\n+void IPU3Awb::updateWbParameters(ipu3_uapi_params ¶ms)\n {\n \t/*\n \t * Green gains should not be touched and considered 1.\n@@ -344,18 +315,10 @@ void IPU3Awb::updateWbParameters(ipu3_uapi_params ¶ms, double agcGamma)\n \tparams.acc_param.bnr.wb_gains.b = 4096 * asyncResults_.blueGain;\n \tparams.acc_param.bnr.wb_gains.gb = 16;\n \n-\tLOG(IPU3Awb, Debug) << \"Color temperature estimated: \" << asyncResults_.temperatureK\n-\t\t\t << \" and gamma calculated: \" << agcGamma;\n+\tLOG(IPU3Awb, Debug) << \"Color temperature estimated: \" << asyncResults_.temperatureK;\n \n \t/* The CCM matrix may change when color temperature will be used */\n \tparams.acc_param.ccm = imguCssCcmDefault;\n-\n-\tfor (uint32_t i = 0; i < 256; i++) {\n-\t\tdouble j = i / 255.0;\n-\t\tdouble gamma = std::pow(j, 1.0 / agcGamma);\n-\t\t/* The maximum value 255 is represented on 13 bits in the IPU3 */\n-\t\tparams.acc_param.gamma.gc_lut.lut[i] = gamma * 8191;\n-\t}\n }\n \n } /* namespace ipa::ipu3 */\ndiff --git a/src/ipa/ipu3/ipu3_awb.h b/src/ipa/ipu3/ipu3_awb.h\nindex ea2d4320..eeb2920b 100644\n--- a/src/ipa/ipu3/ipu3_awb.h\n+++ b/src/ipa/ipu3/ipu3_awb.h\n@@ -31,7 +31,7 @@ public:\n \n \tvoid initialise(ipu3_uapi_params ¶ms, const Size &bdsOutputSize, struct ipu3_uapi_grid_config &bdsGrid);\n \tvoid calculateWBGains(const ipu3_uapi_stats_3a *stats);\n-\tvoid updateWbParameters(ipu3_uapi_params ¶ms, double agcGamma);\n+\tvoid updateWbParameters(ipu3_uapi_params ¶ms);\n \n \tstruct Ipu3AwbCell {\n \t\tunsigned char greenRedAvg;\n","prefixes":["libcamera-devel","v2","05/10"]}