Patch Detail
Show a patch.
GET /api/patches/23173/?format=api
{ "id": 23173, "url": "https://patchwork.libcamera.org/api/patches/23173/?format=api", "web_url": "https://patchwork.libcamera.org/patch/23173/", "project": { "id": 1, "url": "https://patchwork.libcamera.org/api/projects/1/?format=api", "name": "libcamera", "link_name": "libcamera", "list_id": "libcamera_core", "list_email": "libcamera-devel@lists.libcamera.org", "web_url": "", "scm_url": "", "webscm_url": "" }, "msgid": "<20250411130423.2164577-7-stefan.klug@ideasonboard.com>", "date": "2025-04-11T13:04:13", "name": "[6/8] ipa: rkisp1: Add WDR algorithm", "commit_ref": null, "pull_url": null, "state": "superseded", "archived": false, "hash": "5f998fd148d75a57d9b4647f58c225f4c06740e6", "submitter": { "id": 184, "url": "https://patchwork.libcamera.org/api/people/184/?format=api", "name": "Stefan Klug", "email": "stefan.klug@ideasonboard.com" }, "delegate": null, "mbox": "https://patchwork.libcamera.org/patch/23173/mbox/", "series": [ { "id": 5122, "url": "https://patchwork.libcamera.org/api/series/5122/?format=api", "web_url": "https://patchwork.libcamera.org/project/libcamera/list/?series=5122", "date": "2025-04-11T13:04:07", "name": "Implement WDR algorithm", "version": 1, "mbox": "https://patchwork.libcamera.org/series/5122/mbox/" } ], "comments": "https://patchwork.libcamera.org/api/patches/23173/comments/", "check": "pending", "checks": "https://patchwork.libcamera.org/api/patches/23173/checks/", "tags": {}, "headers": { "Return-Path": "<libcamera-devel-bounces@lists.libcamera.org>", "X-Original-To": "parsemail@patchwork.libcamera.org", "Delivered-To": "parsemail@patchwork.libcamera.org", "Received": [ "from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id D7B79C3213\n\tfor <parsemail@patchwork.libcamera.org>;\n\tFri, 11 Apr 2025 13:04:45 +0000 (UTC)", "from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id 8F2AF68AB0;\n\tFri, 11 Apr 2025 15:04:45 +0200 (CEST)", "from perceval.ideasonboard.com (perceval.ideasonboard.com\n\t[IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id AB56068A90\n\tfor <libcamera-devel@lists.libcamera.org>;\n\tFri, 11 Apr 2025 15:04:43 +0200 (CEST)", "from ideasonboard.com (unknown\n\t[IPv6:2a00:6020:448c:6c00:5b21:2ad5:1023:7179])\n\tby perceval.ideasonboard.com (Postfix) with ESMTPSA id 4F0A0735;\n\tFri, 11 Apr 2025 15:02:44 +0200 (CEST)" ], "Authentication-Results": "lancelot.ideasonboard.com; dkim=pass (1024-bit key;\n\tunprotected) header.d=ideasonboard.com header.i=@ideasonboard.com\n\theader.b=\"f/JDnwQf\"; dkim-atps=neutral", "DKIM-Signature": "v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com;\n\ts=mail; t=1744376564;\n\tbh=QRY09doo8IaDxRa25m74bcMS89llj5PFJXtpfClJxFY=;\n\th=From:To:Cc:Subject:Date:In-Reply-To:References:From;\n\tb=f/JDnwQfPKiCewA3sWfU/3oqS6PoNukLLEwvmoX0JHbL/Jf7cK1Ib/BwkCc3q7NRo\n\tamAhE0C34hEG37aRe7/KUE9am7O9eyo02uFz2gV+ks6CYqQH4sfiFtETNuZL1aiphK\n\t0FCvvwjK/jkfacFhf6VISEvj7SNPvC3rbidLN3z4=", "From": "Stefan Klug <stefan.klug@ideasonboard.com>", "To": "libcamera-devel@lists.libcamera.org", "Cc": "Stefan Klug <stefan.klug@ideasonboard.com>", "Subject": "[PATCH 6/8] ipa: rkisp1: Add WDR algorithm", "Date": "Fri, 11 Apr 2025 15:04:13 +0200", "Message-ID": "<20250411130423.2164577-7-stefan.klug@ideasonboard.com>", "X-Mailer": "git-send-email 2.43.0", "In-Reply-To": "<20250411130423.2164577-1-stefan.klug@ideasonboard.com>", "References": "<20250411130423.2164577-1-stefan.klug@ideasonboard.com>", "MIME-Version": "1.0", "Content-Transfer-Encoding": "8bit", "X-BeenThere": "libcamera-devel@lists.libcamera.org", "X-Mailman-Version": "2.1.29", "Precedence": "list", "List-Id": "<libcamera-devel.lists.libcamera.org>", "List-Unsubscribe": "<https://lists.libcamera.org/options/libcamera-devel>,\n\t<mailto:libcamera-devel-request@lists.libcamera.org?subject=unsubscribe>", "List-Archive": "<https://lists.libcamera.org/pipermail/libcamera-devel/>", "List-Post": "<mailto:libcamera-devel@lists.libcamera.org>", "List-Help": "<mailto:libcamera-devel-request@lists.libcamera.org?subject=help>", "List-Subscribe": "<https://lists.libcamera.org/listinfo/libcamera-devel>,\n\t<mailto:libcamera-devel-request@lists.libcamera.org?subject=subscribe>", "Errors-To": "libcamera-devel-bounces@lists.libcamera.org", "Sender": "\"libcamera-devel\" <libcamera-devel-bounces@lists.libcamera.org>" }, "content": "Add a WDR algorithm to do global tone mapping. Global tone mapping is\nused to increase the perceived dynamic range of a image. The typical\neffect is that in areas that are normally overexposed additional\nstructure gets visible.\n\nThe overall idea is that the algorithm applies an exposure value\ncorrection to underexpose the image to the point where only a small\nnumber of saturated pixels is left. This artificial underexposure is\nthen mitigated by applying a tone mapping curve.\n\nThis algorithm implements 4 tone mapping strategies:\n- Linear\n- Power\n- Exponential\n- Histogram equalization\n\nSigned-off-by: Stefan Klug <stefan.klug@ideasonboard.com>\n---\n src/ipa/rkisp1/algorithms/meson.build | 1 +\n src/ipa/rkisp1/algorithms/wdr.cpp | 502 ++++++++++++++++++++++++++\n src/ipa/rkisp1/algorithms/wdr.h | 63 ++++\n src/ipa/rkisp1/ipa_context.h | 10 +\n src/ipa/rkisp1/params.cpp | 1 +\n src/ipa/rkisp1/params.h | 2 +\n src/libcamera/control_ids_debug.yaml | 7 +-\n src/libcamera/control_ids_draft.yaml | 67 ++++\n 8 files changed, 652 insertions(+), 1 deletion(-)\n create mode 100644 src/ipa/rkisp1/algorithms/wdr.cpp\n create mode 100644 src/ipa/rkisp1/algorithms/wdr.h", "diff": "diff --git a/src/ipa/rkisp1/algorithms/meson.build b/src/ipa/rkisp1/algorithms/meson.build\nindex c66b0b70b82f..ff9e26d2684b 100644\n--- a/src/ipa/rkisp1/algorithms/meson.build\n+++ b/src/ipa/rkisp1/algorithms/meson.build\n@@ -13,4 +13,5 @@ rkisp1_ipa_algorithms = files([\n 'gsl.cpp',\n 'lsc.cpp',\n 'lux.cpp',\n+ 'wdr.cpp',\n ])\ndiff --git a/src/ipa/rkisp1/algorithms/wdr.cpp b/src/ipa/rkisp1/algorithms/wdr.cpp\nnew file mode 100644\nindex 000000000000..b342750c54a8\n--- /dev/null\n+++ b/src/ipa/rkisp1/algorithms/wdr.cpp\n@@ -0,0 +1,502 @@\n+/* SPDX-License-Identifier: LGPL-2.1-or-later */\n+/*\n+ * Copyright (C) 2025, Ideas On Board\n+ *\n+ * RkISP1 Wide Dynamic Range control\n+ */\n+\n+#include \"wdr.h\"\n+\n+#include <libcamera/base/log.h>\n+#include <libcamera/base/utils.h>\n+\n+#include \"libcamera/internal/yaml_parser.h\"\n+\n+#include <libipa/histogram.h>\n+#include <libipa/pwl.h>\n+\n+#include \"linux/rkisp1-config.h\"\n+\n+/**\n+ * \\file wdr.h\n+ */\n+\n+namespace libcamera {\n+\n+namespace ipa::rkisp1::algorithms {\n+\n+/**\n+ * \\class WideDynamicRange\n+ * \\brief RkISP1 Wide Dynamic Range algorithm\n+ *\n+ * This algorithm implements automatic global tone mapping for the RkISP1.\n+ * Global tone mapping is done by the GWDR hardware block and applies\n+ * a global tone mapping curve to the image to increase the perceived dynamic\n+ * range. Imagine a indoor scene with bright outside visible through the windows.\n+ * With normal exposure settings, the windwos will be completely saturated and\n+ * no structure (sky/clouds) will be visible because the AEGC has to increase\n+ * overall exposure to reach a certain level of mean brightness. In WDR mode,\n+ * the algorithm will artifically reduce the exposure time so that the texture\n+ * and colours become visible in the former saturated areas. Then the global\n+ * tone mapping curve is applied to mitigate the loss of brightness.\n+ *\n+ * Calculating that tone mapping curve is the most difficult part. This\n+ * algorithm implements four tone mapping strategies:\n+ * - Linear: The tone mapping curve is a combination of two linear functions\n+ * with one kneepoint\n+ * - Power: The tone mapping curve follows a power function\n+ * - Exponential: The tone mapping curve follows an exponential function\n+ * - HistogramEqualization: The tone mapping curve tries to equalize the\n+ * histogram\n+ *\n+ * The overall strategy is the same in all cases: A negative exposure value is\n+ * applied to the AEGC regulation until the number of nearly saturated pixels go\n+ * below a given threshold (controllable via WdrMaxBrightPixels, default is 2%)\n+ * or the MinExposureValue specified in the tuning file is reached.\n+ *\n+ * The global tone mapping curve is then calculated so that it accounts for the\n+ * reduction of brightness due to the negative exposure value. As the result of\n+ * tone mapping is very difficult to quantize and as it is by definition a\n+ * lossy process there is not a single \"correct\" solution.\n+ *\n+ * The approach taken here is based on the sinmple linear model. The kneepoint\n+ * of the curve is calculated so that a 50% grey pixel in the normal exposed\n+ * image ends up at 50% grey in the WDR image again. So the first section of the\n+ * curve has a gain of pow(2, - WDR-EV) up to 50% level. The second part of the\n+ * curve compresses the remaining range. Using the WdrStrength control, the gain\n+ * of that initial section can be further adjusted.\n+ *\n+ * In the Power and Exponential modes, the curves are calculated so that they\n+ * pass through that kneepoint.\n+ *\n+ * The histogram equalization mode tries to equalize the histogram of the\n+ * image and acts independently of the calculated exposure value.\n+ *\n+ * \\code{.unparsed}\n+ * algorithms:\n+ * - WideDynamicRange:\n+ * ExposureConstraint:\n+ * MaxBrightPixels: 0.02\n+ * yTarget: 0.95\n+ * MinExposureValue: -4.0\n+ * \\endcode\n+ */\n+\n+LOG_DEFINE_CATEGORY(RkISP1Wdr)\n+\n+static constexpr unsigned int kTonecurveXIntervals = RKISP1_CIF_ISP_WDR_CURVE_NUM_INTERV;\n+\n+/*\n+ * Increasing interval sizes. The intervals are crafted so that they sum\n+ * up to 4096. This results in better fitting curves than the constant intervals\n+ * (all entries are 4)\n+ */\n+static constexpr std::array<int, kTonecurveXIntervals> kLoglikeIntervals = {\n+\t{ 0, 0, 0, 0, 1, 1, 2, 2, 2, 2, 3, 3, 3, 3, 3, 4,\n+\t 4, 4, 4, 4, 4, 4, 4, 4, 4, 5, 5, 5, 5, 5, 6, 6 }\n+};\n+\n+WideDynamicRange::WideDynamicRange()\n+{\n+}\n+\n+/**\n+ * \\copydoc libcamera::ipa::Algorithm::init\n+ */\n+int WideDynamicRange::init([[maybe_unused]] IPAContext &context,\n+\t\t\t [[maybe_unused]] const YamlObject &tuningData)\n+{\n+\ttoneCurveIntervalValues_ = kLoglikeIntervals;\n+\n+\t/* Calculate a list of normed x values */\n+\ttoneCurveX_[0] = 0.0;\n+\tint lastValue = 0;\n+\tfor (unsigned int i = 1; i < toneCurveX_.size(); i++) {\n+\t\tlastValue += std::pow(2, toneCurveIntervalValues_[i - 1] + 3);\n+\t\tlastValue = std::min(lastValue, 4096);\n+\t\ttoneCurveX_[i] = lastValue / 4096.0;\n+\t}\n+\n+\tstrength_ = 1.0;\n+\tmode_ = controls::draft::WdrOff;\n+\texposureConstraintMaxBrightPixels_ = 0.02;\n+\texposureConstraintY_ = 0.95;\n+\tminExposureValue_ = -4.0;\n+\n+\tconst auto &constraint = tuningData[\"ExposureConstraint\"];\n+\tif (!constraint.isDictionary()) {\n+\t\tLOG(RkISP1Wdr, Warning)\n+\t\t\t<< \"ExposureConstraint not found in tuning data.\"\n+\t\t\t \"Using default values MaxBrightPixels: \"\n+\t\t\t<< exposureConstraintMaxBrightPixels_\n+\t\t\t<< \" yTarget: \" << exposureConstraintY_;\n+\t} else {\n+\t\texposureConstraintMaxBrightPixels_ =\n+\t\t\tconstraint[\"MaxBrightPixels\"].get<double>().value_or(0.98);\n+\t\texposureConstraintY_ = constraint[\"yTarget\"].get<double>().value_or(0.95);\n+\t}\n+\n+\tconst auto &minExp = tuningData[\"MinExposureValue\"];\n+\tminExposureValue_ = minExp.get<double>().value_or(minExposureValue_);\n+\tif (!minExp) {\n+\t\tLOG(RkISP1Wdr, Warning)\n+\t\t\t<< \"MinExposureValue not found in tuning data.\"\n+\t\t\t \"Using default value \"\n+\t\t\t<< minExposureValue_;\n+\t}\n+\n+\tcontext.ctrlMap[&controls::draft::WdrMode] =\n+\t\tControlInfo(controls::draft::WdrModeValues, controls::draft::WdrOff);\n+\tcontext.ctrlMap[&controls::draft::WdrStrength] =\n+\t\tControlInfo(0.0f, 2.0f, static_cast<float>(strength_));\n+\tcontext.ctrlMap[&controls::draft::WdrMaxBrightPixels] =\n+\t\tControlInfo(0.0f, 1.0f, static_cast<float>(exposureConstraintMaxBrightPixels_));\n+\n+\tapplyCompensationLinear(1.0, 0.0);\n+\n+\treturn 0;\n+}\n+\n+void WideDynamicRange::applyHistogramEqualization(double strength)\n+{\n+\tif (hist_.empty())\n+\t\treturn;\n+\n+\tstrength *= 0.65;\n+\n+\t/*\n+\t * In a fully equalized histogram, all bins have the same value. Try\n+\t * to equalize the histogram by applying a gain or damping depending on\n+\t * the distance of the actual bin value from that norm.\n+\t */\n+\tstd::vector<double> gains;\n+\tgains.resize(hist_.size());\n+\tdouble sum = 0;\n+\tdouble norm = 1.0 / (gains.size());\n+\tfor (unsigned i = 0; i < hist_.size(); i++) {\n+\t\tdouble diff = 1.0 + strength * (hist_[i] - norm) / norm;\n+\t\t/* Todo: fix for large intensity */\n+\t\t//diff = std::max(diff, 0.0);\n+\t\tgains[i] = diff;\n+\t\tsum += diff;\n+\t}\n+\t/* Never amplify the last entry. */\n+\tgains.back() = std::max(gains.back(), 1.0);\n+\n+\tdouble scale = gains.size() / sum;\n+\tfor (auto &v : gains)\n+\t\tv *= scale;\n+\n+\tPwl pwl;\n+\tdouble step = 1.0 / gains.size();\n+\tdouble lastX = 0;\n+\tdouble lastY = 0;\n+\n+\tpwl.append(lastX, lastY);\n+\tfor (unsigned int i = 0; i < gains.size() - 1; i++) {\n+\t\tlastY += gains[i] * step;\n+\t\tlastX += step;\n+\t\tpwl.append(lastX, lastY);\n+\t}\n+\tpwl.append(1.0, 1.0);\n+\n+\tfor (unsigned int i = 0; i < toneCurveX_.size(); i++) {\n+\t\ttoneCurveY_[i] = pwl.eval(toneCurveX_[i]);\n+\t}\n+}\n+\n+Vector<double, 2> WideDynamicRange::kneePoint(double gain, double strength)\n+{\n+\tgain = std::pow(gain, strength);\n+\tdouble y = 0.5;\n+\tdouble x = y / gain;\n+\t//double x = 1 / (gain+1);\n+\t//double y = gain*x;\n+\n+\treturn { { x, y } };\n+}\n+\n+void WideDynamicRange::applyCompensationLinear(double gain, double strength)\n+{\n+\tauto kp = kneePoint(gain, strength);\n+\tdouble g1 = kp[1] / kp[0];\n+\tdouble g2 = (kp.y() - 1) / (kp.x() - 1);\n+\n+\tfor (unsigned int i = 0; i < toneCurveX_.size(); i++) {\n+\t\tdouble x = toneCurveX_[i];\n+\t\tdouble y;\n+\t\tif (x < kp.x()) {\n+\t\t\ty = g1 * x;\n+\t\t} else {\n+\t\t\ty = g2 * x + 1 - g2;\n+\t\t}\n+\t\ttoneCurveY_[i] = y;\n+\t}\n+}\n+\n+void WideDynamicRange::applyCompensationPower(double gain, double strength)\n+{\n+\tdouble e = 1.0;\n+\tif (strength > 1e-6) {\n+\t\tauto kp = kneePoint(gain, strength);\n+\t\t/* Calculate an exponent to go through the knee point. */\n+\t\te = log(kp[1]) / log(kp[0]);\n+\t}\n+\n+\tfor (unsigned int i = 0; i < toneCurveX_.size(); i++) {\n+\t\ttoneCurveY_[i] = std::pow(toneCurveX_[i], e);\n+\t}\n+}\n+\n+void WideDynamicRange::applyCompensationExponential(double gain, double strength)\n+{\n+\tdouble k = 0.1;\n+\tauto kp = kneePoint(gain, strength);\n+\tdouble kx = kp[0];\n+\tdouble ky = kp[1];\n+\n+\tif (kx > ky) {\n+\t\tLOG(RkISP1Wdr, Warning) << \"Invalid knee point: \" << kp;\n+\t\tkx = ky;\n+\t}\n+\n+\t/*\n+\t * The exponential curve is based on the function proposed by Glozman\n+\t * et al. in\n+\t * S. Glozman, T. Kats, and O. Yadid-Pecht, \"Exponent Operator Based\n+\t * Tone Mapping Algorithm for Color Wide Dynamic Range Images,\" 2011.\n+\t *\n+\t * That function uses a k factor as parameter for the WDR compression\n+\t * curve:\n+\t * k=0: maximum compression\n+\t * k=infinity: linear curve\n+\t *\n+\t * To calculate a k factor that results in a curve that passes through\n+\t * the kneepoint, the equation needs to be solved for k after inserting\n+\t * the kneepoint. This can be formulated as search for a zero point.\n+\t * Unfortunately there is no closed solution for that transformation.\n+\t * Using newton's method to approximate the value is numerically\n+\t * unstable.\n+\t *\n+\t * Luckily the function only crosses the x axis once and for the set of\n+\t * possible kneepoints, a negative and a positive point can be guessed.\n+\t * The approximation is then implemented using bisection.\n+\t */\n+\tif (std::abs(kx - ky) < 0.001) {\n+\t\tk = 1e8;\n+\t} else {\n+\t\tdouble kl = 0.0001;\n+\t\tdouble kh = 1000;\n+\n+\t\tauto fk = [=](double v) {\n+\t\t\treturn exp(-kx / v) - ky * exp(-1.0 / v) + ky - 1.0;\n+\t\t};\n+\n+\t\tASSERT(fk(kl) < 0);\n+\t\tASSERT(fk(kh) > 0);\n+\n+\t\tk = kh / 10.0;\n+\t\twhile (fk(k) > 0) {\n+\t\t\tkh = k;\n+\t\t\tk /= 10.0;\n+\t\t}\n+\n+\t\tdo {\n+\t\t\tk = (kl + kh) / 2;\n+\t\t\tif (fk(k) < 0)\n+\t\t\t\tkl = k;\n+\t\t\telse\n+\t\t\t\tkh = k;\n+\t\t} while (std::abs(kh - kl) > 1e-3);\n+\t}\n+\n+\tdouble a = 1.0 / (1.0 - std::exp(-1.0 / k));\n+\tfor (unsigned int i = 0; i < toneCurveX_.size(); i++) {\n+\t\ttoneCurveY_[i] = a * (1.0 - std::exp(-toneCurveX_[i] / k));\n+\t}\n+}\n+\n+/**\n+ * \\copydoc libcamera::ipa::Algorithm::queueRequest\n+ */\n+void WideDynamicRange::queueRequest([[maybe_unused]] IPAContext &context,\n+\t\t\t\t [[maybe_unused]] const uint32_t frame,\n+\t\t\t\t IPAFrameContext &frameContext,\n+\t\t\t\t const ControlList &controls)\n+{\n+\tstrength_ = controls.get(controls::draft::WdrStrength).value_or(strength_);\n+\texposureConstraintMaxBrightPixels_ =\n+\t\tcontrols.get(controls::draft::WdrMaxBrightPixels)\n+\t\t\t.value_or(exposureConstraintMaxBrightPixels_);\n+\n+\tconst auto &mode = controls.get(controls::draft::WdrMode);\n+\tif (mode) {\n+\t\tmode_ = static_cast<controls::draft::WdrModeEnum>(*mode);\n+\t}\n+\n+\tframeContext.wdr.mode = mode_;\n+}\n+\n+/**\n+ * \\copydoc libcamera::ipa::Algorithm::prepare\n+ */\n+void WideDynamicRange::prepare(IPAContext &context,\n+\t\t\t [[maybe_unused]] const uint32_t frame,\n+\t\t\t IPAFrameContext &frameContext,\n+\t\t\t RkISP1Params *params)\n+{\n+\tif (!params) {\n+\t\tLOG(RkISP1Wdr, Warning) << \"Params is null\";\n+\t\treturn;\n+\t}\n+\n+\tauto config = params->block<BlockType::Wdr>();\n+\tif (!config) {\n+\t\tLOG(RkISP1Wdr, Warning) << \"Wdr block is invalid\";\n+\t\treturn;\n+\t}\n+\n+\tauto mode = frameContext.wdr.mode;\n+\n+\tconfig.setEnabled(mode != controls::draft::WdrOff);\n+\n+\tdouble comp = 0;\n+\n+\t/*\n+\t * Todo: This overwrites frameContext.agc.exposureValue so that\n+\t * in the next call to Agc::process() that exposureValue get's\n+\t * applied. In the future it is planned to move the exposure\n+\t * calculations from Agc::process() to Agc::prepare(). In this\n+\t * case, we need to ensure that this code get's called early enough.\n+\t */\n+\tif (mode != controls::draft::WdrOff) {\n+\t\tframeContext.wdr.wdrExposureValue = context.activeState.wdr.exposureValue;\n+\t\tframeContext.wdr.agcExposureValue = frameContext.agc.exposureValue;\n+\n+\t\t/*\n+\t\t * When WDR is enabled, the maxBrightPixels constraint is always\n+\t\t * active. This is problematic when the user sets a positive\n+\t\t * exposureValue. As this would mean that the negative WDR\n+\t\t * exposure value and the user provided positive value should\n+\t\t * cancel each other out. But as the regulation is not absolute\n+\t\t * and only dependent on the measured changes in the histogram,\n+\t\t * reducing the wdr EV would only lead to an even stronger pull\n+\t\t * from regulation. Positive user supplied EVs must therefore be\n+\t\t * handled using the wdr curve.\n+\t\t */\n+\t\tif (frameContext.wdr.wdrExposureValue < 0) {\n+\t\t\tframeContext.agc.exposureValue =\n+\t\t\t\tstd::min(frameContext.agc.exposureValue,\n+\t\t\t\t\t frameContext.wdr.wdrExposureValue);\n+\t\t\tcomp = frameContext.agc.exposureValue - frameContext.wdr.agcExposureValue;\n+\t\t}\n+\t}\n+\n+\t/* Calculate how much EV we need to compensate with the WDR curve. */\n+\tdouble gain = pow(2.0, -comp);\n+\n+\tif (mode == controls::draft::WdrOff) {\n+\t\tapplyCompensationLinear(1.0, 0.0);\n+\t} else if (mode == controls::draft::WdrLinear) {\n+\t\tapplyCompensationLinear(gain, strength_);\n+\t} else if (mode == controls::draft::WdrPower) {\n+\t\tapplyCompensationPower(gain, strength_);\n+\t} else if (mode == controls::draft::WdrExponential) {\n+\t\tapplyCompensationExponential(gain, strength_);\n+\t} else if (mode == controls::draft::WdrHistogramEqualization) {\n+\t\tapplyHistogramEqualization(strength_);\n+\t}\n+\n+\t//reset value\n+\tconfig->dmin_strength = 0x10;\n+\tconfig->dmin_thresh = 0;\n+\n+\tfor (unsigned int i = 0; i < kTonecurveXIntervals; i++) {\n+\t\tint v = toneCurveIntervalValues_[i];\n+\t\tconfig->tone_curve.dY[i / 8] |= (v & 0x07) << ((i % 8) * 4);\n+\t}\n+\n+\tstd::vector<Point> debugCurve;\n+\n+\t/*\n+\t * Fix the curve to adhere to the hardware constraints. Don't apply a\n+\t * constraint on the first element, which is most likely zero anyways.\n+\t */\n+\tint lastY = toneCurveY_[0] * 4096.0;\n+\tfor (unsigned int i = 0; i < toneCurveX_.size(); i++) {\n+\t\tint diff = static_cast<int>(toneCurveY_[i] * 4096.0) - lastY;\n+\t\tdiff = std::clamp(diff, -2048, 2048);\n+\t\tlastY = lastY + diff;\n+\t\tconfig->tone_curve.ym[i] = lastY;\n+\t\tdebugCurve.push_back({ static_cast<int>(toneCurveX_[i] * 4096.0),\n+\t\t\t\t lastY });\n+\t}\n+\n+\tcontext.debugMetadata.set<Span<const Point>>(controls::debug::WdrCurve, debugCurve);\n+}\n+\n+void WideDynamicRange::process(IPAContext &context, const uint32_t frame,\n+\t\t\t IPAFrameContext &frameContext,\n+\t\t\t const rkisp1_stat_buffer *stats,\n+\t\t\t ControlList &metadata)\n+{\n+\tif (!stats || !(stats->meas_type & RKISP1_CIF_ISP_STAT_HIST)) {\n+\t\tLOG(RkISP1Wdr, Error) << \"No histogram data in statistics\";\n+\t\treturn;\n+\t}\n+\n+\tif (frame == 0) {\n+\t\tpid_.setStandardParameters(1, 5.0, 3.0);\n+\t\tpid_.setOutputLimits(minExposureValue_, 0.0);\n+\t\tcontext.activeState.wdr.exposureValue = 0.0;\n+\t}\n+\n+\tpid_.setTarget(exposureConstraintY_);\n+\n+\tconst rkisp1_cif_isp_stat *params = &stats->params;\n+\tauto mode = frameContext.wdr.mode;\n+\n+\tmetadata.set(controls::draft::WdrMode, mode);\n+\n+\tHistogram cumHist({ params->hist.hist_bins, context.hw->numHistogramBins },\n+\t\t\t [](uint32_t x) { return x >> 4; });\n+\n+\tstd::vector<double> hist;\n+\tdouble sum = 0;\n+\tfor (unsigned i = 0; i < context.hw->numHistogramBins; i++) {\n+\t\tdouble v = params->hist.hist_bins[i] >> 4;\n+\t\thist.push_back(v);\n+\t\tsum += v;\n+\t}\n+\n+\t/* Scale so that the entries sum up to 1. */\n+\tdouble scale = 1.0 / sum;\n+\tfor (auto &v : hist)\n+\t\tv *= scale;\n+\thist_.swap(hist);\n+\n+\tdouble mean = cumHist.quantile(1.0 - exposureConstraintMaxBrightPixels_) /\n+\t\t cumHist.bins();\n+\tLOG(RkISP1Wdr, Debug) << \"Mean y of bright pixels: \" << mean;\n+\n+\tif (mode == controls::draft::WdrOff) {\n+\t\tmetadata.set(controls::draft::WdrExposureValue, 0);\n+\t\treturn;\n+\t}\n+\n+\tcontext.activeState.wdr.exposureValue = pid_.process(mean);\n+\tLOG(RkISP1Wdr, Debug) << \"Active state WDR ev: \" << context.activeState.wdr.exposureValue;\n+\tmetadata.set(controls::draft::WdrExposureValue, frameContext.wdr.wdrExposureValue);\n+\n+\t/*\n+\t * We overwrote agc exposure value in prepare() to create an\n+\t * underexposure for WDR. Report the original exposure value in metadata.\n+\t */\n+\tmetadata.set(controls::ExposureValue, frameContext.wdr.agcExposureValue);\n+}\n+\n+REGISTER_IPA_ALGORITHM(WideDynamicRange, \"WideDynamicRange\")\n+\n+} /* namespace ipa::rkisp1::algorithms */\n+\n+} /* namespace libcamera */\ndiff --git a/src/ipa/rkisp1/algorithms/wdr.h b/src/ipa/rkisp1/algorithms/wdr.h\nnew file mode 100644\nindex 000000000000..d9bf9e9c433e\n--- /dev/null\n+++ b/src/ipa/rkisp1/algorithms/wdr.h\n@@ -0,0 +1,63 @@\n+/* SPDX-License-Identifier: LGPL-2.1-or-later */\n+/*\n+ * Copyright (C) 2021-2022, Ideas On Board\n+ *\n+ * RkISP1 Wide Dynamic Range control\n+ */\n+\n+#pragma once\n+\n+#include <libcamera/control_ids.h>\n+\n+#include \"libcamera/internal/pid_controller.h\"\n+\n+#include \"linux/rkisp1-config.h\"\n+\n+#include \"algorithm.h\"\n+\n+namespace libcamera {\n+\n+namespace ipa::rkisp1::algorithms {\n+\n+class WideDynamicRange : public Algorithm\n+{\n+public:\n+\tWideDynamicRange();\n+\t~WideDynamicRange() = default;\n+\n+\tint init(IPAContext &context, const YamlObject &tuningData) override;\n+\n+\tvoid queueRequest(IPAContext &context, const uint32_t frame,\n+\t\t\t IPAFrameContext &frameContext,\n+\t\t\t const ControlList &controls) override;\n+\tvoid prepare(IPAContext &context, const uint32_t frame,\n+\t\t IPAFrameContext &frameContext,\n+\t\t RkISP1Params *params) override;\n+\tvoid process(IPAContext &context, const uint32_t frame,\n+\t\t IPAFrameContext &frameContext,\n+\t\t const rkisp1_stat_buffer *stats,\n+\t\t ControlList &metadata) override;\n+\n+private:\n+\tVector<double, 2> kneePoint(double gain, double strength);\n+\tvoid applyCompensationLinear(double gain, double strength);\n+\tvoid applyCompensationPower(double gain, double strength);\n+\tvoid applyCompensationExponential(double gain, double strength);\n+\tvoid applyHistogramEqualization(double strength);\n+\n+\tdouble exposureConstraintMaxBrightPixels_;\n+\tdouble exposureConstraintY_;\n+\tdouble minExposureValue_;\n+\tdouble strength_;\n+\n+\tcontrols::draft::WdrModeEnum mode_;\n+\tstd::vector<double> hist_;\n+\tPidController pid_;\n+\n+\tstd::array<int, RKISP1_CIF_ISP_WDR_CURVE_NUM_INTERV> toneCurveIntervalValues_;\n+\tstd::array<double, RKISP1_CIF_ISP_WDR_CURVE_NUM_INTERV + 1> toneCurveX_;\n+\tstd::array<double, RKISP1_CIF_ISP_WDR_CURVE_NUM_INTERV + 1> toneCurveY_;\n+};\n+\n+} /* namespace ipa::rkisp1::algorithms */\n+} /* namespace libcamera */\ndiff --git a/src/ipa/rkisp1/ipa_context.h b/src/ipa/rkisp1/ipa_context.h\nindex 182203880ac9..ce7a3fb7f2bf 100644\n--- a/src/ipa/rkisp1/ipa_context.h\n+++ b/src/ipa/rkisp1/ipa_context.h\n@@ -125,6 +125,10 @@ struct IPAActiveState {\n \tstruct {\n \t\tdouble gamma;\n \t} goc;\n+\n+\tstruct {\n+\t\tdouble exposureValue;\n+\t} wdr;\n };\n \n struct IPAFrameContext : public FrameContext {\n@@ -188,6 +192,12 @@ struct IPAFrameContext : public FrameContext {\n \tstruct {\n \t\tdouble lux;\n \t} lux;\n+\n+\tstruct {\n+\t\tcontrols::draft::WdrModeEnum mode;\n+\t\tdouble wdrExposureValue;\n+\t\tdouble agcExposureValue;\n+\t} wdr;\n };\n \n struct IPAContext {\ndiff --git a/src/ipa/rkisp1/params.cpp b/src/ipa/rkisp1/params.cpp\nindex b4a889e415fc..fccfa430b13d 100644\n--- a/src/ipa/rkisp1/params.cpp\n+++ b/src/ipa/rkisp1/params.cpp\n@@ -74,6 +74,7 @@ const std::map<BlockType, BlockTypeInfo> kBlockTypeInfo = {\n \tRKISP1_BLOCK_TYPE_ENTRY_EXT(CompandBls, COMPAND_BLS, compand_bls),\n \tRKISP1_BLOCK_TYPE_ENTRY_EXT(CompandExpand, COMPAND_EXPAND, compand_curve),\n \tRKISP1_BLOCK_TYPE_ENTRY_EXT(CompandCompress, COMPAND_COMPRESS, compand_curve),\n+\tRKISP1_BLOCK_TYPE_ENTRY_EXT(Wdr, WDR, wdr),\n };\n \n } /* namespace */\ndiff --git a/src/ipa/rkisp1/params.h b/src/ipa/rkisp1/params.h\nindex 04b06c2a6266..a31c54b4df53 100644\n--- a/src/ipa/rkisp1/params.h\n+++ b/src/ipa/rkisp1/params.h\n@@ -40,6 +40,7 @@ enum class BlockType {\n \tCompandBls,\n \tCompandExpand,\n \tCompandCompress,\n+\tWdr,\n };\n \n namespace details {\n@@ -74,6 +75,7 @@ RKISP1_DEFINE_BLOCK_TYPE(Afc, afc)\n RKISP1_DEFINE_BLOCK_TYPE(CompandBls, compand_bls)\n RKISP1_DEFINE_BLOCK_TYPE(CompandExpand, compand_curve)\n RKISP1_DEFINE_BLOCK_TYPE(CompandCompress, compand_curve)\n+RKISP1_DEFINE_BLOCK_TYPE(Wdr, wdr)\n \n } /* namespace details */\n \ndiff --git a/src/libcamera/control_ids_debug.yaml b/src/libcamera/control_ids_debug.yaml\nindex 797532712099..9489e677402e 100644\n--- a/src/libcamera/control_ids_debug.yaml\n+++ b/src/libcamera/control_ids_debug.yaml\n@@ -3,4 +3,9 @@\n %YAML 1.1\n ---\n vendor: debug\n-controls: []\n+controls:\n+- WdrCurve:\n+ type: Point\n+ direction: out\n+ description: Debug control WdrCurve found in src/ipa/rkisp1/algorithms/wdr.cpp\n+ size: '[n]'\ndiff --git a/src/libcamera/control_ids_draft.yaml b/src/libcamera/control_ids_draft.yaml\nindex 03309eeac34f..d53d181bbf47 100644\n--- a/src/libcamera/control_ids_draft.yaml\n+++ b/src/libcamera/control_ids_draft.yaml\n@@ -293,5 +293,72 @@ controls:\n \n Currently identical to ANDROID_STATISTICS_FACE_IDS.\n size: [n]\n+ - WdrMode:\n+ type: int32_t\n+ direction: inout\n+ description: |\n+ Set the WDR mode.\n+\n+ The WDR mode is used to select the algorithm used for global tone\n+ mapping. It will automatically reduce the exposure time of the sensor\n+ so that there are only a small number of saturated pixels in the image.\n+ The algorithm then compensate for the loss of brightness by applying a\n+ global tone mapping curve to the image.\n+ enum:\n+ - name: WdrOff\n+ value: 0\n+ description: Wdr is disabled.\n+ - name: WdrLinear\n+ value: 1\n+ description:\n+ Apply a linear global tone mapping curve.\n+\n+ The curve with two linear sections is applied. This produces good\n+ results at the expense of a slightly artificial look.\n+ - name: WdrPower\n+ value: 2\n+ description: |\n+ Apply a power global tone mapping curve.\n+\n+ This curve has high gain values on the dark areas of an image and\n+ high compression values on the bright area. It therefore tends to\n+ produce noticeable noise artifacts.\n+ - name: WdrExponential\n+ value: 3\n+ description: |\n+ Apply a exponential global tone mapping curve.\n+\n+ This curve has lower gain values in dark areas compared to the power\n+ curve but produces a more natural look compared to the linear curve.\n+ It is therefore the best choice for most scenes.\n+ - name: WdrHistogramEqualization\n+ value: 4\n+ description: |\n+ Apply histogram equalization.\n+\n+ This curve preserves most of the information of the image at the\n+ expense of a very artificial look. It is therefore best suited for\n+ technical analysis.\n+ - WdrStrength:\n+ type: float\n+ direction: in\n+ description: |\n+ Specify the strength of the wdr algorithm. The exact meaning of this\n+ value is specific to the algorithm in use. Usually a value of 0 means no\n+ global tone mapping is applied. A values of 1 is the default value and\n+ the correct value for most scenes. A value above 1 increases the global\n+ tone mapping effect and can lead to unrealistic image effects,\n+ - WdrMaxBrightPixels:\n+ type: float\n+ direction: in\n+ description: |\n+ Percentage of allowed (nearly) saturated pixels.\n+ - WdrExposureValue:\n+ type: float\n+ direction: out\n+ description: |\n+ Return the exposure value compensation applied by the WDR algorithm.\n+\n+ \\sa ExposureValue\n \n ...\n", "prefixes": [ "6/8" ] }