From patchwork Fri Oct 6 13:19:58 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Naushir Patuck X-Patchwork-Id: 19120 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 7A9A9C32B8 for ; Fri, 6 Oct 2023 13:20:22 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 29D0D6299D; Fri, 6 Oct 2023 15:20:22 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1696598422; bh=a45WmFNuorh7GWBgUQj37lz3xuS4EXguvzLOZ0lzpJQ=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To: From; b=OheVbbnQhV909xeZYScJu3PdmJmGN2UkFfieFuWmJ6j69fhPoIVcg8DYqDbv+4MYi n2Xddv4OcKZyPlt+NfPJHVpfdUZ+fhfQeDEzT8WpU4GZLzxCT2hwGKfZF9Fj0vRCGy s/R2OHIntWYnk1e859i727LwqOWhG7VGbONMEpnadNLgOYiWGmosuHIqUUvllYCK/U PK27xGqXRLS4eXLWcfvAHXAMy0Zc3iA5gplmSVRz2Q7FEa21G97JQDqFynLaw/gJbw qME+Xn9VOKSbHVDOrOcrH95X+SeA3CQx7BBaTSSJL2A7cS0UGGu12x0UPOsS5HWAh3 Y5vlQj5zSJnsQ== Received: from mail-wr1-x429.google.com (mail-wr1-x429.google.com [IPv6:2a00:1450:4864:20::429]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id CA3F062969 for ; Fri, 6 Oct 2023 15:20:19 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="iCtnyEPI"; dkim-atps=neutral Received: by mail-wr1-x429.google.com with SMTP id ffacd0b85a97d-323168869daso2030826f8f.2 for ; Fri, 06 Oct 2023 06:20:19 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; t=1696598419; x=1697203219; darn=lists.libcamera.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=ZNgXgb+Cnl0XP4aIcikz0avyv8rl0oeOdAohyVFJnWw=; b=iCtnyEPIQaalWY3Afe+ZANs0wdG84IoXA+zUeOeJvzpdnvxvYYid5vO70yHTHIW/K5 esLfZWzUZGB1Jhsp7awVlPjPy2ZIIIrj9pZXpWnmg2OJftFBcnQv4Ix3p4nY11Kp0F2f b7sPeKfjUYRuXWPVcLZ9g7BzZruOmkjBKkmoCWHR4a+OY+B7rir3l/y8/U8ijBZ3LX+p qH5ktKapWx0LqD4fVLE7rpCzuX99jT9KfoYzXsVVUAW7Yqmc2T7yZk8GniM4/4Ss/LVD xYm6rbYa3hwzJe3a55s+fodxGAt7RPCauOAq9zH3uFH4IfTBsmzUPcfZjkejUsEm0yuZ qEFg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1696598419; x=1697203219; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=ZNgXgb+Cnl0XP4aIcikz0avyv8rl0oeOdAohyVFJnWw=; b=kOQjQHzqs34yP+CWlTb1AAnZPBXZpOLR2OyM9qtYcmNvylSr94mTI6wJLebJAq1QSM DAikd7aVgyj9ae+xy0+mSrD1M8vf5q+s0fIWnvn3svu5av8Lom8FsflYefLaqF+7GQIt KzSdQQ0dcd95DoXgCjhsiU8yHYFObgdqPYPO1IaxJxLXLolsHREkqCBi74DlS00ouM3H /JtesU71FbMQHco3KeopJp/SSsSzNHBObw/bsVQPhfoY/g2I3sZuJuFIRNWW2nA5n/Cm giHR8cEB2MkFx3qn+8R49hS5Uos3XWN9qLqwg3WgLWdvSlJ7YWuH8BcagUxm+sAwQHKd +TXQ== X-Gm-Message-State: AOJu0YyW8yGweS31Ib6/Q1IlpCYnI0bfa0gqoBNnldYetvOH4eRxXeji Fxu4diuDxgZdSGLFxX0dDxjTDPMmk5G7CoAzitwitQ== X-Google-Smtp-Source: AGHT+IEm07U4M6JT1Lv5TjDlWESahv+CwGDiy8JItGud+sA1r9f4Uu904VCmA8xoJRA133r35fCEQA== X-Received: by 2002:a5d:4a0b:0:b0:31a:b3aa:d19b with SMTP id m11-20020a5d4a0b000000b0031ab3aad19bmr7065204wrq.23.1696598417872; Fri, 06 Oct 2023 06:20:17 -0700 (PDT) Received: from localhost.localdomain ([93.93.133.154]) by smtp.gmail.com with ESMTPSA id m8-20020a056000180800b00321773bb933sm1639478wrh.77.2023.10.06.06.20.17 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 06 Oct 2023 06:20:17 -0700 (PDT) To: libcamera-devel@lists.libcamera.org Date: Fri, 6 Oct 2023 14:19:58 +0100 Message-Id: <20231006132000.23504-19-naush@raspberrypi.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20231006132000.23504-1-naush@raspberrypi.com> References: <20231006132000.23504-1-naush@raspberrypi.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH 18/20] ipa: rpi: denoise: Support different denoise configurations X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Naushir Patuck via libcamera-devel From: Naushir Patuck Reply-To: Naushir Patuck Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" From: David Plowman Some use cases may require stronger, or different, denosie settings to others. For example, the way frames are accumulated during single exposure HDR means that we may want stronger denoise. This commit adds such support for different configurations that can be defined in the tuning file. Older tuning files, or files where there is only a single configuration, load only the "normal" denoise configuration. Signed-off-by: David Plowman Reviewed-by: Naushir Patuck --- src/ipa/rpi/controller/denoise_algorithm.h | 4 + src/ipa/rpi/controller/rpi/denoise.cpp | 130 ++++++++++++++------- src/ipa/rpi/controller/rpi/denoise.h | 36 +++--- 3 files changed, 113 insertions(+), 57 deletions(-) diff --git a/src/ipa/rpi/controller/denoise_algorithm.h b/src/ipa/rpi/controller/denoise_algorithm.h index 52009ba95661..444cbc259889 100644 --- a/src/ipa/rpi/controller/denoise_algorithm.h +++ b/src/ipa/rpi/controller/denoise_algorithm.h @@ -6,6 +6,8 @@ */ #pragma once +#include + #include "algorithm.h" namespace RPiController { @@ -18,6 +20,8 @@ public: DenoiseAlgorithm(Controller *controller) : Algorithm(controller) {} /* A Denoise algorithm must provide the following: */ virtual void setMode(DenoiseMode mode) = 0; + /* Some platforms may not be able to define this, so supply a default. */ + virtual void setConfig([[maybe_unused]] std::string const &name) {} }; } /* namespace RPiController */ diff --git a/src/ipa/rpi/controller/rpi/denoise.cpp b/src/ipa/rpi/controller/rpi/denoise.cpp index 440ee4425534..154ee604ddd0 100644 --- a/src/ipa/rpi/controller/rpi/denoise.cpp +++ b/src/ipa/rpi/controller/rpi/denoise.cpp @@ -21,6 +21,45 @@ LOG_DEFINE_CATEGORY(RPiDenoise) #define NAME "rpi.denoise" +int DenoiseConfig::read(const libcamera::YamlObject ¶ms) +{ + sdnEnable = params.contains("sdn"); + if (sdnEnable) { + auto &sdnParams = params["sdn"]; + sdnDeviation = sdnParams["deviation"].get(3.2); + sdnStrength = sdnParams["strength"].get(0.25); + sdnDeviation2 = sdnParams["deviation2"].get(sdnDeviation); + sdnDeviationNoTdn = sdnParams["deviation_no_tdn"].get(sdnDeviation); + sdnStrengthNoTdn = sdnParams["strength_no_tdn"].get(sdnStrength); + sdnTdnBackoff = sdnParams["backoff"].get(0.75); + } + + cdnEnable = params.contains("cdn"); + if (cdnEnable) { + auto &cdnParams = params["cdn"]; + cdnDeviation = cdnParams["deviation"].get(120); + cdnStrength = cdnParams["strength"].get(0.2); + } + + tdnEnable = params.contains("tdn"); + if (tdnEnable) { + auto &tdnParams = params["tdn"]; + tdnDeviation = tdnParams["deviation"].get(0.5); + tdnThreshold = tdnParams["threshold"].get(0.75); + } else if (sdnEnable) { + /* + * If SDN is enabled but TDN isn't, overwrite all the SDN settings + * with the "no TDN" versions. This makes it easier to enable or + * disable TDN in the tuning file without editing all the other + * parameters. + */ + sdnDeviation = sdnDeviation2 = sdnDeviationNoTdn; + sdnStrength = sdnStrengthNoTdn; + } + + return 0; +} + Denoise::Denoise(Controller *controller) : DenoiseAlgorithm(controller), mode_(DenoiseMode::ColourHighQuality) { @@ -33,39 +72,26 @@ char const *Denoise::name() const int Denoise::read(const libcamera::YamlObject ¶ms) { - sdnEnable_ = params.contains("sdn"); - if (sdnEnable_) { - auto &sdnParams = params["sdn"]; - sdnDeviation_ = sdnParams["deviation"].get(3.2); - sdnStrength_ = sdnParams["strength"].get(0.25); - sdnDeviation2_ = sdnParams["deviation2"].get(sdnDeviation_); - sdnDeviationNoTdn_ = sdnParams["deviation_no_tdn"].get(sdnDeviation_); - sdnStrengthNoTdn_ = sdnParams["strength_no_tdn"].get(sdnStrength_); - sdnTdnBackoff_ = sdnParams["backoff"].get(0.75); + if (!params.contains("normal")) { + configs_["normal"].read(params); + currentConfig_ = &configs_["normal"]; + + return 0; } - cdnEnable_ = params.contains("cdn"); - if (cdnEnable_) { - auto &cdnParams = params["cdn"]; - cdnDeviation_ = cdnParams["deviation"].get(120); - cdnStrength_ = cdnParams["strength"].get(0.2); + for (const auto &[key, value] : params.asDict()) { + if (configs_[key].read(value)) { + LOG(RPiDenoise, Error) << "Failed to read denoise config " << key; + return -EINVAL; + } } - tdnEnable_ = params.contains("tdn"); - if (tdnEnable_) { - auto &tdnParams = params["tdn"]; - tdnDeviation_ = tdnParams["deviation"].get(0.5); - tdnThreshold_ = tdnParams["threshold"].get(0.75); - } else if (sdnEnable_) { - /* - * If SDN is enabled but TDN isn't, overwrite all the SDN settings - * with the "no TDN" versions. This makes it easier to enable or - * disable TDN in the tuning file without editing all the other - * parameters. - */ - sdnDeviation_ = sdnDeviation2_ = sdnDeviationNoTdn_; - sdnStrength_ = sdnStrengthNoTdn_; + auto it = configs_.find("normal"); + if (it == configs_.end()) { + LOG(RPiDenoise, Error) << "No normal denoise settings found"; + return -EINVAL; } + currentConfig_ = &it->second; return 0; } @@ -78,9 +104,9 @@ void Denoise::switchMode([[maybe_unused]] CameraMode const &cameraMode, [[maybe_unused]] Metadata *metadata) { /* A mode switch effectively resets temporal denoise and it has to start over. */ - currentSdnDeviation_ = sdnDeviationNoTdn_; - currentSdnStrength_ = sdnStrengthNoTdn_; - currentSdnDeviation2_ = sdnDeviationNoTdn_; + currentSdnDeviation_ = currentConfig_->sdnDeviationNoTdn; + currentSdnStrength_ = currentConfig_->sdnStrengthNoTdn; + currentSdnDeviation2_ = currentConfig_->sdnDeviationNoTdn; } void Denoise::prepare(Metadata *imageMetadata) @@ -97,11 +123,11 @@ void Denoise::prepare(Metadata *imageMetadata) if (mode_ == DenoiseMode::Off) return; - if (sdnEnable_) { + if (currentConfig_->sdnEnable) { struct SdnStatus sdn; sdn.noiseConstant = noiseStatus.noiseConstant * currentSdnDeviation_; sdn.noiseSlope = noiseStatus.noiseSlope * currentSdnDeviation_; - sdn.noiseConstant2 = noiseStatus.noiseConstant * sdnDeviation2_; + sdn.noiseConstant2 = noiseStatus.noiseConstant * currentConfig_->sdnDeviation2; sdn.noiseSlope2 = noiseStatus.noiseSlope * currentSdnDeviation2_; sdn.strength = currentSdnStrength_; imageMetadata->set("sdn.status", sdn); @@ -113,17 +139,17 @@ void Denoise::prepare(Metadata *imageMetadata) << " slope2 " << sdn.noiseSlope2; /* For the next frame, we back off the SDN parameters as TDN ramps up. */ - double f = sdnTdnBackoff_; - currentSdnDeviation_ = f * currentSdnDeviation_ + (1 - f) * sdnDeviation_; - currentSdnStrength_ = f * currentSdnStrength_ + (1 - f) * sdnStrength_; - currentSdnDeviation2_ = f * currentSdnDeviation2_ + (1 - f) * sdnDeviation2_; + double f = currentConfig_->sdnTdnBackoff; + currentSdnDeviation_ = f * currentSdnDeviation_ + (1 - f) * currentConfig_->sdnDeviation; + currentSdnStrength_ = f * currentSdnStrength_ + (1 - f) * currentConfig_->sdnStrength; + currentSdnDeviation2_ = f * currentSdnDeviation2_ + (1 - f) * currentConfig_->sdnDeviation2; } - if (tdnEnable_) { + if (currentConfig_->tdnEnable) { struct TdnStatus tdn; - tdn.noiseConstant = noiseStatus.noiseConstant * tdnDeviation_; - tdn.noiseSlope = noiseStatus.noiseSlope * tdnDeviation_; - tdn.threshold = tdnThreshold_; + tdn.noiseConstant = noiseStatus.noiseConstant * currentConfig_->tdnDeviation; + tdn.noiseSlope = noiseStatus.noiseSlope * currentConfig_->tdnDeviation; + tdn.threshold = currentConfig_->tdnThreshold; imageMetadata->set("tdn.status", tdn); LOG(RPiDenoise, Debug) << "programmed tdn threshold " << tdn.threshold @@ -131,10 +157,10 @@ void Denoise::prepare(Metadata *imageMetadata) << " slope " << tdn.noiseSlope; } - if (cdnEnable_ && mode_ != DenoiseMode::ColourOff) { + if (currentConfig_->cdnEnable && mode_ != DenoiseMode::ColourOff) { struct CdnStatus cdn; - cdn.threshold = cdnDeviation_ * noiseStatus.noiseSlope + noiseStatus.noiseConstant; - cdn.strength = cdnStrength_; + cdn.threshold = currentConfig_->cdnDeviation * noiseStatus.noiseSlope + noiseStatus.noiseConstant; + cdn.strength = currentConfig_->cdnStrength; imageMetadata->set("cdn.status", cdn); LOG(RPiDenoise, Debug) << "programmed cdn threshold " << cdn.threshold @@ -148,6 +174,22 @@ void Denoise::setMode(DenoiseMode mode) mode_ = mode; } +void Denoise::setConfig(std::string const &name) +{ + auto it = configs_.find(name); + if (it == configs_.end()) { + /* + * Some platforms may have no need for different denoise settings, so we only issue + * a warning if there clearly are several configurations. + */ + if (configs_.size() > 1) + LOG(RPiDenoise, Warning) << "No denoise config found for " << name; + else + LOG(RPiDenoise, Debug) << "No denoise config found for " << name; + } else + currentConfig_ = &it->second; +} + // Register algorithm with the system. static Algorithm *Create(Controller *controller) { diff --git a/src/ipa/rpi/controller/rpi/denoise.h b/src/ipa/rpi/controller/rpi/denoise.h index 88b37663e569..92ff4f93edf2 100644 --- a/src/ipa/rpi/controller/rpi/denoise.h +++ b/src/ipa/rpi/controller/rpi/denoise.h @@ -6,6 +6,9 @@ */ #pragma once +#include +#include + #include "algorithm.h" #include "denoise_algorithm.h" @@ -13,6 +16,23 @@ namespace RPiController { // Algorithm to calculate correct denoise settings. +struct DenoiseConfig { + double sdnDeviation; + double sdnStrength; + double sdnDeviation2; + double sdnDeviationNoTdn; + double sdnStrengthNoTdn; + double sdnTdnBackoff; + double cdnDeviation; + double cdnStrength; + double tdnDeviation; + double tdnThreshold; + bool tdnEnable; + bool sdnEnable; + bool cdnEnable; + int read(const libcamera::YamlObject ¶ms); +}; + class Denoise : public DenoiseAlgorithm { public: @@ -23,22 +43,12 @@ public: void switchMode(CameraMode const &cameraMode, Metadata *metadata) override; void prepare(Metadata *imageMetadata) override; void setMode(DenoiseMode mode) override; + void setConfig(std::string const &name) override; private: - double sdnDeviation_; - double sdnStrength_; - double sdnDeviation2_; - double sdnDeviationNoTdn_; - double sdnStrengthNoTdn_; - double sdnTdnBackoff_; - double cdnDeviation_; - double cdnStrength_; - double tdnDeviation_; - double tdnThreshold_; + std::map configs_; + DenoiseConfig *currentConfig_; DenoiseMode mode_; - bool tdnEnable_; - bool sdnEnable_; - bool cdnEnable_; /* SDN parameters attenuate over time if TDN is running. */ double currentSdnDeviation_;