From patchwork Mon Apr 4 15:21:52 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Naushir Patuck X-Patchwork-Id: 15613 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 87FFFC0F1B for ; Mon, 4 Apr 2022 15:22:00 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id CA08265641; Mon, 4 Apr 2022 17:21:59 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1649085719; bh=ZgqrF1AQBquXUMn6ga0Nso2265oM1q2Oj/exV89jc5w=; h=To:Date:Subject:List-Id:List-Unsubscribe:List-Archive:List-Post: List-Help:List-Subscribe:From:Reply-To:From; b=r/hcOEoA0PNXRaZ11J8ADVJBFcje+2NvXR+A4+Zt3lVZ4WVSI3dPssQXTIQ7T3fca uw1ttlW65AIrWvSIk0RAYlD2S7VAAKujB5zYcIUFwuzf+SBrbqs0ieCMVcMK2mRzRD ahJ7DSsDz5VAXcxl0ITFbCv0WiTkSAOD/VT28kl5AOTVc2gWEsRiKB/1TwdYmc1pWK s0lb9P0vqi3Plojv5V8LtdSeME6/iPbd0LblFNbc2ER1iR6UEsO/FT3cNhQxd6FWoS C5rG4hgnU/81O0zRoESBA9ozTIjT+QimpxJdlrsi5fC3j1Eq2XQChTfnoMDFmlhg/J 4WM+JlkKVXs6g== Received: from mail-wm1-x32c.google.com (mail-wm1-x32c.google.com [IPv6:2a00:1450:4864:20::32c]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id EF32C633A5 for ; Mon, 4 Apr 2022 17:21:57 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="F3TUPR2a"; dkim-atps=neutral Received: by mail-wm1-x32c.google.com with SMTP id l62-20020a1c2541000000b0038e4570af2fso5811473wml.5 for ; Mon, 04 Apr 2022 08:21:57 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; h=from:to:cc:subject:date:message-id:mime-version :content-transfer-encoding; bh=r/SffLrzOIoO6isG5/Rf+dtGXQf6iUgk+is/ai6xqPM=; b=F3TUPR2aqkingS5jzCFFeK18KsS5R4wc+w2XsvxKBimAqIo9wdZBgJ5wNistX6acRS 8BWVUDIyzvHynvQHwzWb7fnDu/K+s5vSoANfYmgNKPu2XzyGan+zX9VsRk8ZrgMxbQ07 aBK4AeteFwDtaWKzI30iW+4q0nIdIFifO0Km8GZOhGvUs2xmQVU1ZtA1dJyPlJsx78RO vOOUvhuzpnQkTixAkKa0vMgs+HTO04r513qLDrHcYBTmBQbY6/W52+Pd7nMFv0TyWzXB wmpCrFRpaXOE/uY63bV3YRo1D3zPCrJivKTj7YQfPFrzDlEigZOUbF8dZWOQZj/wA2jn 8dvg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:mime-version :content-transfer-encoding; bh=r/SffLrzOIoO6isG5/Rf+dtGXQf6iUgk+is/ai6xqPM=; b=rRCn+6eQM8hMXqWkqLacx97aIt7rSD2LOowxu/WEXfMSFnMNAKEMJ8gHaKGlDtJbc9 qHHbjFDSVIhLHWtpSawCK+RsupQfBXSdrrRsl/E2CAXA/Bkks/BSzcO/LSfo02vln69L chG9z2k+vaczF/dDFXSK53qoghaEYn0F5TJDzzkG6Om6kAot7p0g9zRo1lJZrlKiRaAK IR9l7mGbDrjP1THoLRIanQgb8CwwkOTIl4RFHN33XqsN/whjN3KtZb7bRCpEZ+YV+XRr Rch7svYT+e7g2F+OB/ADk6hUO3PYOKIlmnenhIxpEErqjW/j92HrmGWX4ut4+W3BQBrc WoRQ== X-Gm-Message-State: AOAM532A4tGP3c10M0GQtufsmkODFT3VKt0lsBH01c2kYh8g1DjLzl0N k4Qpx7I5w+sZnx6jW6gwECf6F1o3HpGNKQ== X-Google-Smtp-Source: ABdhPJzDq+s+wAs952DwkYz+EP/8+yuYYnXuUiHDwF6L9Sv+jnx+FkxpCijl4IWgoy1ii1H/92fZZA== X-Received: by 2002:a05:600c:5023:b0:38d:1261:aac6 with SMTP id n35-20020a05600c502300b0038d1261aac6mr15145wmr.180.1649085717177; Mon, 04 Apr 2022 08:21:57 -0700 (PDT) Received: from naush-laptop.pitowers.org ([2a00:1098:3142:14:547b:2a10:726e:d9e0]) by smtp.gmail.com with ESMTPSA id m5-20020a05600c3b0500b0038e7466b048sm2306199wms.0.2022.04.04.08.21.56 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 04 Apr 2022 08:21:56 -0700 (PDT) To: libcamera-devel@lists.libcamera.org Date: Mon, 4 Apr 2022 16:21:52 +0100 Message-Id: <20220404152152.1779553-1-naush@raspberrypi.com> X-Mailer: git-send-email 2.25.1 MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH] ipa: raspberrypi: alcs: Limit the calculated lambda values X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Naushir Patuck via libcamera-devel From: Naushir Patuck Reply-To: Naushir Patuck Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Under the right circumstances, the alsc calculations could spread the colour errors across the entire image as lambda remains unbound. This would cause the corrected image chroma values to slowly drift to incorrect values. This change adds a config parameter (alcs.lambda_bound) that provides an upper and lower bound to the lambda value at every stage of the calculation. With this change, we now adjust the lambda values so that the average across the entire grid is 1 instead of normalising to the minimum value. Signed-off-by: Naushir Patuck Reviewed-by: David Plowman --- src/ipa/raspberrypi/controller/rpi/alsc.cpp | 57 +++++++++++++++------ src/ipa/raspberrypi/controller/rpi/alsc.hpp | 1 + 2 files changed, 43 insertions(+), 15 deletions(-) diff --git a/src/ipa/raspberrypi/controller/rpi/alsc.cpp b/src/ipa/raspberrypi/controller/rpi/alsc.cpp index be3d1ae476cd..a88fee9f6d94 100644 --- a/src/ipa/raspberrypi/controller/rpi/alsc.cpp +++ b/src/ipa/raspberrypi/controller/rpi/alsc.cpp @@ -149,6 +149,7 @@ void Alsc::Read(boost::property_tree::ptree const ¶ms) read_calibrations(config_.calibrations_Cb, params, "calibrations_Cb"); config_.default_ct = params.get("default_ct", 4500.0); config_.threshold = params.get("threshold", 1e-3); + config_.lambda_bound = params.get("lambda_bound", 0.05); } static double get_ct(Metadata *metadata, double default_ct); @@ -610,30 +611,47 @@ static double compute_lambda_top_end(int i, double const M[XY][4], // Gauss-Seidel iteration with over-relaxation. static double gauss_seidel2_SOR(double const M[XY][4], double omega, - double lambda[XY]) + double lambda[XY], double lambda_bound) { + const double min = 1 - lambda_bound, max = 1 + lambda_bound; double old_lambda[XY]; int i; for (i = 0; i < XY; i++) old_lambda[i] = lambda[i]; lambda[0] = compute_lambda_bottom_start(0, M, lambda); - for (i = 1; i < X; i++) + lambda[0] = std::clamp(lambda[0], min, max); + for (i = 1; i < X; i++) { lambda[i] = compute_lambda_bottom(i, M, lambda); - for (; i < XY - X; i++) + lambda[i] = std::clamp(lambda[i], min, max); + } + for (; i < XY - X; i++) { lambda[i] = compute_lambda_interior(i, M, lambda); - for (; i < XY - 1; i++) + lambda[i] = std::clamp(lambda[i], min, max); + } + for (; i < XY - 1; i++) { lambda[i] = compute_lambda_top(i, M, lambda); + lambda[i] = std::clamp(lambda[i], min, max); + } lambda[i] = compute_lambda_top_end(i, M, lambda); + lambda[i] = std::clamp(lambda[i], min, max); // Also solve the system from bottom to top, to help spread the updates // better. lambda[i] = compute_lambda_top_end(i, M, lambda); - for (i = XY - 2; i >= XY - X; i--) + lambda[i] = std::clamp(lambda[i], min, max); + for (i = XY - 2; i >= XY - X; i--) { lambda[i] = compute_lambda_top(i, M, lambda); - for (; i >= X; i--) + lambda[i] = std::clamp(lambda[i], min, max); + } + for (; i >= X; i--) { lambda[i] = compute_lambda_interior(i, M, lambda); - for (; i >= 1; i--) + lambda[i] = std::clamp(lambda[i], min, max); + } + for (; i >= 1; i--) { lambda[i] = compute_lambda_bottom(i, M, lambda); + lambda[i] = std::clamp(lambda[i], min, max); + } lambda[0] = compute_lambda_bottom_start(0, M, lambda); + lambda[0] = std::clamp(lambda[0], min, max); double max_diff = 0; for (i = 0; i < XY; i++) { lambda[i] = old_lambda[i] + (lambda[i] - old_lambda[i]) * omega; @@ -653,15 +671,26 @@ static void normalise(double *ptr, size_t n) ptr[i] /= minval; } +// Rescale the values so that the avarage value is 1. +static void reaverage(double *ptr, size_t n) +{ + double sum = 0; + for (size_t i = 0; i < n; i++) + sum += ptr[i]; + double ratio = 1 / (sum / n); + for (size_t i = 0; i < n; i++) + ptr[i] *= ratio; +} + static void run_matrix_iterations(double const C[XY], double lambda[XY], double const W[XY][4], double omega, - int n_iter, double threshold) + int n_iter, double threshold, double lambda_bound) { double M[XY][4]; construct_M(C, W, M); double last_max_diff = std::numeric_limits::max(); for (int i = 0; i < n_iter; i++) { - double max_diff = fabs(gauss_seidel2_SOR(M, omega, lambda)); + double max_diff = fabs(gauss_seidel2_SOR(M, omega, lambda, lambda_bound)); if (max_diff < threshold) { LOG(RPiAlsc, Debug) << "Stop after " << i + 1 << " iterations"; @@ -675,10 +704,8 @@ static void run_matrix_iterations(double const C[XY], double lambda[XY], << last_max_diff << " to " << max_diff; last_max_diff = max_diff; } - // We're going to normalise the lambdas so the smallest is 1. Not sure - // this is really necessary as they get renormalised later, but I - // suppose it does stop these quantities from wandering off... - normalise(lambda, XY); + // We're going to normalise the lambdas so the total average is 1. + reaverage(lambda, XY); } static void add_luminance_rb(double result[XY], double const lambda[XY], @@ -737,9 +764,9 @@ void Alsc::doAlsc() compute_W(Cb, config_.sigma_Cb, Wb); // Run Gauss-Seidel iterations over the resulting matrix, for R and B. run_matrix_iterations(Cr, lambda_r_, Wr, config_.omega, config_.n_iter, - config_.threshold); + config_.threshold, config_.lambda_bound); run_matrix_iterations(Cb, lambda_b_, Wb, config_.omega, config_.n_iter, - config_.threshold); + config_.threshold, config_.lambda_bound); // Fold the calibrated gains into our final lambda values. (Note that on // the next run, we re-start with the lambda values that don't have the // calibration gains included.) diff --git a/src/ipa/raspberrypi/controller/rpi/alsc.hpp b/src/ipa/raspberrypi/controller/rpi/alsc.hpp index 9616b99ea7ca..d1dbe0d1d22d 100644 --- a/src/ipa/raspberrypi/controller/rpi/alsc.hpp +++ b/src/ipa/raspberrypi/controller/rpi/alsc.hpp @@ -41,6 +41,7 @@ struct AlscConfig { std::vector calibrations_Cb; double default_ct; // colour temperature if no metadata found double threshold; // iteration termination threshold + double lambda_bound; // upper/lower bound for lambda from a value of 1 }; class Alsc : public Algorithm