{"id":15613,"url":"https://patchwork.libcamera.org/api/1.1/patches/15613/?format=json","web_url":"https://patchwork.libcamera.org/patch/15613/","project":{"id":1,"url":"https://patchwork.libcamera.org/api/1.1/projects/1/?format=json","name":"libcamera","link_name":"libcamera","list_id":"libcamera_core","list_email":"libcamera-devel@lists.libcamera.org","web_url":"","scm_url":"","webscm_url":""},"msgid":"<20220404152152.1779553-1-naush@raspberrypi.com>","date":"2022-04-04T15:21:52","name":"[libcamera-devel] ipa: raspberrypi: alcs: Limit the calculated lambda values","commit_ref":null,"pull_url":null,"state":"superseded","archived":false,"hash":"e0d3d374f04743a0e5f78fbefc7c629dc4b0ab14","submitter":{"id":34,"url":"https://patchwork.libcamera.org/api/1.1/people/34/?format=json","name":"Naushir Patuck","email":"naush@raspberrypi.com"},"delegate":null,"mbox":"https://patchwork.libcamera.org/patch/15613/mbox/","series":[{"id":3027,"url":"https://patchwork.libcamera.org/api/1.1/series/3027/?format=json","web_url":"https://patchwork.libcamera.org/project/libcamera/list/?series=3027","date":"2022-04-04T15:21:52","name":"[libcamera-devel] ipa: raspberrypi: alcs: Limit the calculated lambda values","version":1,"mbox":"https://patchwork.libcamera.org/series/3027/mbox/"}],"comments":"https://patchwork.libcamera.org/api/patches/15613/comments/","check":"pending","checks":"https://patchwork.libcamera.org/api/patches/15613/checks/","tags":{},"headers":{"Return-Path":"","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id 87FFFC0F1B\n\tfor ;\n\tMon, 4 Apr 2022 15:22:00 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id CA08265641;\n\tMon, 4 Apr 2022 17:21:59 +0200 (CEST)","from mail-wm1-x32c.google.com (mail-wm1-x32c.google.com\n\t[IPv6:2a00:1450:4864:20::32c])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id EF32C633A5\n\tfor ;\n\tMon, 4 Apr 2022 17:21:57 +0200 (CEST)","by mail-wm1-x32c.google.com with SMTP id\n\tl62-20020a1c2541000000b0038e4570af2fso5811473wml.5\n\tfor ;\n\tMon, 04 Apr 2022 08:21:57 -0700 (PDT)","from naush-laptop.pitowers.org\n\t([2a00:1098:3142:14:547b:2a10:726e:d9e0])\n\tby smtp.gmail.com with ESMTPSA id\n\tm5-20020a05600c3b0500b0038e7466b048sm2306199wms.0.2022.04.04.08.21.56\n\t(version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);\n\tMon, 04 Apr 2022 08:21:56 -0700 (PDT)"],"DKIM-Signature":["v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org;\n\ts=mail; t=1649085719;\n\tbh=ZgqrF1AQBquXUMn6ga0Nso2265oM1q2Oj/exV89jc5w=;\n\th=To:Date:Subject:List-Id:List-Unsubscribe:List-Archive:List-Post:\n\tList-Help:List-Subscribe:From:Reply-To:From;\n\tb=r/hcOEoA0PNXRaZ11J8ADVJBFcje+2NvXR+A4+Zt3lVZ4WVSI3dPssQXTIQ7T3fca\n\tuw1ttlW65AIrWvSIk0RAYlD2S7VAAKujB5zYcIUFwuzf+SBrbqs0ieCMVcMK2mRzRD\n\tahJ7DSsDz5VAXcxl0ITFbCv0WiTkSAOD/VT28kl5AOTVc2gWEsRiKB/1TwdYmc1pWK\n\ts0lb9P0vqi3Plojv5V8LtdSeME6/iPbd0LblFNbc2ER1iR6UEsO/FT3cNhQxd6FWoS\n\tC5rG4hgnU/81O0zRoESBA9ozTIjT+QimpxJdlrsi5fC3j1Eq2XQChTfnoMDFmlhg/J\n\t4WM+JlkKVXs6g==","v=1; a=rsa-sha256; c=relaxed/relaxed;\n\td=raspberrypi.com; s=google;\n\th=from:to:cc:subject:date:message-id:mime-version\n\t:content-transfer-encoding;\n\tbh=r/SffLrzOIoO6isG5/Rf+dtGXQf6iUgk+is/ai6xqPM=;\n\tb=F3TUPR2aqkingS5jzCFFeK18KsS5R4wc+w2XsvxKBimAqIo9wdZBgJ5wNistX6acRS\n\t8BWVUDIyzvHynvQHwzWb7fnDu/K+s5vSoANfYmgNKPu2XzyGan+zX9VsRk8ZrgMxbQ07\n\taBK4AeteFwDtaWKzI30iW+4q0nIdIFifO0Km8GZOhGvUs2xmQVU1ZtA1dJyPlJsx78RO\n\tvOOUvhuzpnQkTixAkKa0vMgs+HTO04r513qLDrHcYBTmBQbY6/W52+Pd7nMFv0TyWzXB\n\twmpCrFRpaXOE/uY63bV3YRo1D3zPCrJivKTj7YQfPFrzDlEigZOUbF8dZWOQZj/wA2jn\n\t8dvg=="],"Authentication-Results":"lancelot.ideasonboard.com; dkim=pass (2048-bit key; \n\tunprotected) header.d=raspberrypi.com\n\theader.i=@raspberrypi.com\n\theader.b=\"F3TUPR2a\"; dkim-atps=neutral","X-Google-DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed;\n\td=1e100.net; s=20210112;\n\th=x-gm-message-state:from:to:cc:subject:date:message-id:mime-version\n\t:content-transfer-encoding;\n\tbh=r/SffLrzOIoO6isG5/Rf+dtGXQf6iUgk+is/ai6xqPM=;\n\tb=rRCn+6eQM8hMXqWkqLacx97aIt7rSD2LOowxu/WEXfMSFnMNAKEMJ8gHaKGlDtJbc9\n\tqHHbjFDSVIhLHWtpSawCK+RsupQfBXSdrrRsl/E2CAXA/Bkks/BSzcO/LSfo02vln69L\n\tchG9z2k+vaczF/dDFXSK53qoghaEYn0F5TJDzzkG6Om6kAot7p0g9zRo1lJZrlKiRaAK\n\tIR9l7mGbDrjP1THoLRIanQgb8CwwkOTIl4RFHN33XqsN/whjN3KtZb7bRCpEZ+YV+XRr\n\tRch7svYT+e7g2F+OB/ADk6hUO3PYOKIlmnenhIxpEErqjW/j92HrmGWX4ut4+W3BQBrc\n\tWoRQ==","X-Gm-Message-State":"AOAM532A4tGP3c10M0GQtufsmkODFT3VKt0lsBH01c2kYh8g1DjLzl0N\n\tk4Qpx7I5w+sZnx6jW6gwECf6F1o3HpGNKQ==","X-Google-Smtp-Source":"ABdhPJzDq+s+wAs952DwkYz+EP/8+yuYYnXuUiHDwF6L9Sv+jnx+FkxpCijl4IWgoy1ii1H/92fZZA==","X-Received":"by 2002:a05:600c:5023:b0:38d:1261:aac6 with SMTP id\n\tn35-20020a05600c502300b0038d1261aac6mr15145wmr.180.1649085717177; \n\tMon, 04 Apr 2022 08:21:57 -0700 (PDT)","To":"libcamera-devel@lists.libcamera.org","Date":"Mon, 4 Apr 2022 16:21:52 +0100","Message-Id":"<20220404152152.1779553-1-naush@raspberrypi.com>","X-Mailer":"git-send-email 2.25.1","MIME-Version":"1.0","Content-Transfer-Encoding":"8bit","Subject":"[libcamera-devel] [PATCH] ipa: raspberrypi: alcs: Limit the\n\tcalculated lambda values","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","From":"Naushir Patuck via libcamera-devel\n\t","Reply-To":"Naushir Patuck ","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" "},"content":"Under the right circumstances, the alsc calculations could spread the colour\nerrors across the entire image as lambda remains unbound. This would cause the\ncorrected image chroma values to slowly drift to incorrect values.\n\nThis change adds a config parameter (alcs.lambda_bound) that provides an upper\nand lower bound to the lambda value at every stage of the calculation. With this\nchange, we now adjust the lambda values so that the average across the entire\ngrid is 1 instead of normalising to the minimum value.\n\nSigned-off-by: Naushir Patuck \n---\n src/ipa/raspberrypi/controller/rpi/alsc.cpp | 57 +++++++++++++++------\n src/ipa/raspberrypi/controller/rpi/alsc.hpp | 1 +\n 2 files changed, 43 insertions(+), 15 deletions(-)","diff":"diff --git a/src/ipa/raspberrypi/controller/rpi/alsc.cpp b/src/ipa/raspberrypi/controller/rpi/alsc.cpp\nindex be3d1ae476cd..a88fee9f6d94 100644\n--- a/src/ipa/raspberrypi/controller/rpi/alsc.cpp\n+++ b/src/ipa/raspberrypi/controller/rpi/alsc.cpp\n@@ -149,6 +149,7 @@ void Alsc::Read(boost::property_tree::ptree const ¶ms)\n \tread_calibrations(config_.calibrations_Cb, params, \"calibrations_Cb\");\n \tconfig_.default_ct = params.get(\"default_ct\", 4500.0);\n \tconfig_.threshold = params.get(\"threshold\", 1e-3);\n+\tconfig_.lambda_bound = params.get(\"lambda_bound\", 0.05);\n }\n \n static double get_ct(Metadata *metadata, double default_ct);\n@@ -610,30 +611,47 @@ static double compute_lambda_top_end(int i, double const M[XY][4],\n \n // Gauss-Seidel iteration with over-relaxation.\n static double gauss_seidel2_SOR(double const M[XY][4], double omega,\n-\t\t\t\tdouble lambda[XY])\n+\t\t\t\tdouble lambda[XY], double lambda_bound)\n {\n+\tconst double min = 1 - lambda_bound, max = 1 + lambda_bound;\n \tdouble old_lambda[XY];\n \tint i;\n \tfor (i = 0; i < XY; i++)\n \t\told_lambda[i] = lambda[i];\n \tlambda[0] = compute_lambda_bottom_start(0, M, lambda);\n-\tfor (i = 1; i < X; i++)\n+\tlambda[0] = std::clamp(lambda[0], min, max);\n+\tfor (i = 1; i < X; i++) {\n \t\tlambda[i] = compute_lambda_bottom(i, M, lambda);\n-\tfor (; i < XY - X; i++)\n+\t\tlambda[i] = std::clamp(lambda[i], min, max);\n+\t}\n+\tfor (; i < XY - X; i++) {\n \t\tlambda[i] = compute_lambda_interior(i, M, lambda);\n-\tfor (; i < XY - 1; i++)\n+\t\tlambda[i] = std::clamp(lambda[i], min, max);\n+\t}\n+\tfor (; i < XY - 1; i++) {\n \t\tlambda[i] = compute_lambda_top(i, M, lambda);\n+\t\tlambda[i] = std::clamp(lambda[i], min, max);\n+\t}\n \tlambda[i] = compute_lambda_top_end(i, M, lambda);\n+\tlambda[i] = std::clamp(lambda[i], min, max);\n \t// Also solve the system from bottom to top, to help spread the updates\n \t// better.\n \tlambda[i] = compute_lambda_top_end(i, M, lambda);\n-\tfor (i = XY - 2; i >= XY - X; i--)\n+\tlambda[i] = std::clamp(lambda[i], min, max);\n+\tfor (i = XY - 2; i >= XY - X; i--) {\n \t\tlambda[i] = compute_lambda_top(i, M, lambda);\n-\tfor (; i >= X; i--)\n+\t\tlambda[i] = std::clamp(lambda[i], min, max);\n+\t}\n+\tfor (; i >= X; i--) {\n \t\tlambda[i] = compute_lambda_interior(i, M, lambda);\n-\tfor (; i >= 1; i--)\n+\t\tlambda[i] = std::clamp(lambda[i], min, max);\n+\t}\n+\tfor (; i >= 1; i--) {\n \t\tlambda[i] = compute_lambda_bottom(i, M, lambda);\n+\t\tlambda[i] = std::clamp(lambda[i], min, max);\n+\t}\n \tlambda[0] = compute_lambda_bottom_start(0, M, lambda);\n+\tlambda[0] = std::clamp(lambda[0], min, max);\n \tdouble max_diff = 0;\n \tfor (i = 0; i < XY; i++) {\n \t\tlambda[i] = old_lambda[i] + (lambda[i] - old_lambda[i]) * omega;\n@@ -653,15 +671,26 @@ static void normalise(double *ptr, size_t n)\n \t\tptr[i] /= minval;\n }\n \n+// Rescale the values so that the avarage value is 1.\n+static void reaverage(double *ptr, size_t n)\n+{\n+\tdouble sum = 0;\n+\tfor (size_t i = 0; i < n; i++)\n+\t\tsum += ptr[i];\n+\tdouble ratio = 1 / (sum / n);\n+\tfor (size_t i = 0; i < n; i++)\n+\t\tptr[i] *= ratio;\n+}\n+\n static void run_matrix_iterations(double const C[XY], double lambda[XY],\n \t\t\t\t double const W[XY][4], double omega,\n-\t\t\t\t int n_iter, double threshold)\n+\t\t\t\t int n_iter, double threshold, double lambda_bound)\n {\n \tdouble M[XY][4];\n \tconstruct_M(C, W, M);\n \tdouble last_max_diff = std::numeric_limits::max();\n \tfor (int i = 0; i < n_iter; i++) {\n-\t\tdouble max_diff = fabs(gauss_seidel2_SOR(M, omega, lambda));\n+\t\tdouble max_diff = fabs(gauss_seidel2_SOR(M, omega, lambda, lambda_bound));\n \t\tif (max_diff < threshold) {\n \t\t\tLOG(RPiAlsc, Debug)\n \t\t\t\t<< \"Stop after \" << i + 1 << \" iterations\";\n@@ -675,10 +704,8 @@ static void run_matrix_iterations(double const C[XY], double lambda[XY],\n \t\t\t\t<< last_max_diff << \" to \" << max_diff;\n \t\tlast_max_diff = max_diff;\n \t}\n-\t// We're going to normalise the lambdas so the smallest is 1. Not sure\n-\t// this is really necessary as they get renormalised later, but I\n-\t// suppose it does stop these quantities from wandering off...\n-\tnormalise(lambda, XY);\n+\t// We're going to normalise the lambdas so the total average is 1.\n+\treaverage(lambda, XY);\n }\n \n static void add_luminance_rb(double result[XY], double const lambda[XY],\n@@ -737,9 +764,9 @@ void Alsc::doAlsc()\n \tcompute_W(Cb, config_.sigma_Cb, Wb);\n \t// Run Gauss-Seidel iterations over the resulting matrix, for R and B.\n \trun_matrix_iterations(Cr, lambda_r_, Wr, config_.omega, config_.n_iter,\n-\t\t\t config_.threshold);\n+\t\t\t config_.threshold, config_.lambda_bound);\n \trun_matrix_iterations(Cb, lambda_b_, Wb, config_.omega, config_.n_iter,\n-\t\t\t config_.threshold);\n+\t\t\t config_.threshold, config_.lambda_bound);\n \t// Fold the calibrated gains into our final lambda values. (Note that on\n \t// the next run, we re-start with the lambda values that don't have the\n \t// calibration gains included.)\ndiff --git a/src/ipa/raspberrypi/controller/rpi/alsc.hpp b/src/ipa/raspberrypi/controller/rpi/alsc.hpp\nindex 9616b99ea7ca..d1dbe0d1d22d 100644\n--- a/src/ipa/raspberrypi/controller/rpi/alsc.hpp\n+++ b/src/ipa/raspberrypi/controller/rpi/alsc.hpp\n@@ -41,6 +41,7 @@ struct AlscConfig {\n \tstd::vector calibrations_Cb;\n \tdouble default_ct; // colour temperature if no metadata found\n \tdouble threshold; // iteration termination threshold\n+\tdouble lambda_bound; // upper/lower bound for lambda from a value of 1\n };\n \n class Alsc : public Algorithm\n","prefixes":["libcamera-devel"]}