From patchwork Fri Oct 13 07:48:39 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Naushir Patuck X-Patchwork-Id: 19142 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 654D7C32C2 for ; Fri, 13 Oct 2023 07:49:10 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id D5AFF6299A; Fri, 13 Oct 2023 09:49:09 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1697183350; bh=a45WmFNuorh7GWBgUQj37lz3xuS4EXguvzLOZ0lzpJQ=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To: From; b=lkIU19GRIQijDt2hbhgjpWnINY+bNbrM9NiGviP3l+3JWF9wbQ3cKI1o2ABVk4n0D +I3H08UDrXc5cw/xOTBhxUzLHdvlY3ro9AuUuaI2hfDQRYb5KIGqyJNZ/Ps//sjQzC /jqwZuwMb1u4fIZT9+nzWZA36qnPnrmK5KnT1mrPD0jeuG62INvyb+hKUtsllXbzyE QMLdE0UsYznR+CO5ZFu1A/A9jWr4KPMiMcQL1sPR5B/sroJ9vDkh1fRzGRdKxR5cpe arSMUzHwziCQIDo3hHk4FyaQNUAwRCdEC5tHBMzxQUScarhPmGNMWxx2qqB1KMe2R0 3Zog0wM2tVzAg== Received: from mail-wm1-x330.google.com (mail-wm1-x330.google.com [IPv6:2a00:1450:4864:20::330]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 43A086299B for ; Fri, 13 Oct 2023 09:49:00 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="PFnT9zpy"; dkim-atps=neutral Received: by mail-wm1-x330.google.com with SMTP id 5b1f17b1804b1-405524e6769so11287685e9.1 for ; Fri, 13 Oct 2023 00:49:00 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; t=1697183339; x=1697788139; darn=lists.libcamera.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=ZNgXgb+Cnl0XP4aIcikz0avyv8rl0oeOdAohyVFJnWw=; b=PFnT9zpy3YpxAPqkHWivjPj61mKqVhbuahcKTH2ihRj9fM6gdBjCvL8wJyc0zOEJoB yva/tcIipXr2M6m8x+Ld9Ied93qVRXy7VlC7v+mzfJq7AyX6kNK01FXg8POHZBAQK/No T50qErT9mVGXaecruOns5fQ5nUVLrg2eW9RFBHwomHOm9QCQB6iAsMy8XVwIe9wRKtDz ZwBy9/IzhSRvvywmCRVnSi5IUmS0+9oCJqoHdngbA2abDZCbDHt21cor8swgHKEVMf0b kagAiz88z+nh4dTHEV91zuyEg0E4ZuEJdexZPw8pMieSbzePrh9fsutddZhABCvT6lfw KmPg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1697183339; x=1697788139; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=ZNgXgb+Cnl0XP4aIcikz0avyv8rl0oeOdAohyVFJnWw=; b=k8gUffokxFaduKxoIhvFLAy/9oMlb55A2KCvJsJEBsWL0Ge79Na24agy7tTWcO3PDw uexzU6CmGpaMcbZrMkcnAsE+ACOjAoAJRt0YgcHOvgQzmAHEh42X63DaqXcyNC0NBuJN zpkEulkD8YwYMscPG45SNBJolHmGiNOx03FbUMk4fk22V3unK0wifZPGd9Ic0bB80ksZ +L6x00hfFxJkTXu0vmEzuLnA7VYf1Djy/Mud49cGD1RUdj9H2KpH4upw2RJ2rkkZa9Gw v1qKpuzw3R1aKFdjhJoENWpLIWn+yF4hAMB5OxY3n+WNMgU8a9x4LmLiQ3OVcQZDELPp 9u1A== X-Gm-Message-State: AOJu0YzRyCePaEzLW1Pmpv59ggUP0vxMJJpr9pHKDjT303LDtWytrPHt /cSNoh07l8VBVVU28pt4ar6827yTvmDPEJyNHxcF9g== X-Google-Smtp-Source: AGHT+IGwNWo953gy74z4vhMlZ2v70DM6FnBG6yJG5+WrIQByv5Dg0jKvf2GqDYkeH1jVVHbtXacDOg== X-Received: by 2002:a7b:c4cc:0:b0:402:f91e:df80 with SMTP id g12-20020a7bc4cc000000b00402f91edf80mr19174916wmk.3.1697183339464; Fri, 13 Oct 2023 00:48:59 -0700 (PDT) Received: from localhost.localdomain ([93.93.133.154]) by smtp.gmail.com with ESMTPSA id m16-20020a7bca50000000b003fee6e170f9sm1791890wml.45.2023.10.13.00.48.58 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 13 Oct 2023 00:48:59 -0700 (PDT) To: libcamera-devel@lists.libcamera.org Date: Fri, 13 Oct 2023 08:48:39 +0100 Message-Id: <20231013074841.16972-19-naush@raspberrypi.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20231013074841.16972-1-naush@raspberrypi.com> References: <20231013074841.16972-1-naush@raspberrypi.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v2 18/20] ipa: rpi: denoise: Support different denoise configurations X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Naushir Patuck via libcamera-devel From: Naushir Patuck Reply-To: Naushir Patuck Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" From: David Plowman Some use cases may require stronger, or different, denosie settings to others. For example, the way frames are accumulated during single exposure HDR means that we may want stronger denoise. This commit adds such support for different configurations that can be defined in the tuning file. Older tuning files, or files where there is only a single configuration, load only the "normal" denoise configuration. Signed-off-by: David Plowman Reviewed-by: Naushir Patuck --- src/ipa/rpi/controller/denoise_algorithm.h | 4 + src/ipa/rpi/controller/rpi/denoise.cpp | 130 ++++++++++++++------- src/ipa/rpi/controller/rpi/denoise.h | 36 +++--- 3 files changed, 113 insertions(+), 57 deletions(-) diff --git a/src/ipa/rpi/controller/denoise_algorithm.h b/src/ipa/rpi/controller/denoise_algorithm.h index 52009ba95661..444cbc259889 100644 --- a/src/ipa/rpi/controller/denoise_algorithm.h +++ b/src/ipa/rpi/controller/denoise_algorithm.h @@ -6,6 +6,8 @@ */ #pragma once +#include + #include "algorithm.h" namespace RPiController { @@ -18,6 +20,8 @@ public: DenoiseAlgorithm(Controller *controller) : Algorithm(controller) {} /* A Denoise algorithm must provide the following: */ virtual void setMode(DenoiseMode mode) = 0; + /* Some platforms may not be able to define this, so supply a default. */ + virtual void setConfig([[maybe_unused]] std::string const &name) {} }; } /* namespace RPiController */ diff --git a/src/ipa/rpi/controller/rpi/denoise.cpp b/src/ipa/rpi/controller/rpi/denoise.cpp index 440ee4425534..154ee604ddd0 100644 --- a/src/ipa/rpi/controller/rpi/denoise.cpp +++ b/src/ipa/rpi/controller/rpi/denoise.cpp @@ -21,6 +21,45 @@ LOG_DEFINE_CATEGORY(RPiDenoise) #define NAME "rpi.denoise" +int DenoiseConfig::read(const libcamera::YamlObject ¶ms) +{ + sdnEnable = params.contains("sdn"); + if (sdnEnable) { + auto &sdnParams = params["sdn"]; + sdnDeviation = sdnParams["deviation"].get(3.2); + sdnStrength = sdnParams["strength"].get(0.25); + sdnDeviation2 = sdnParams["deviation2"].get(sdnDeviation); + sdnDeviationNoTdn = sdnParams["deviation_no_tdn"].get(sdnDeviation); + sdnStrengthNoTdn = sdnParams["strength_no_tdn"].get(sdnStrength); + sdnTdnBackoff = sdnParams["backoff"].get(0.75); + } + + cdnEnable = params.contains("cdn"); + if (cdnEnable) { + auto &cdnParams = params["cdn"]; + cdnDeviation = cdnParams["deviation"].get(120); + cdnStrength = cdnParams["strength"].get(0.2); + } + + tdnEnable = params.contains("tdn"); + if (tdnEnable) { + auto &tdnParams = params["tdn"]; + tdnDeviation = tdnParams["deviation"].get(0.5); + tdnThreshold = tdnParams["threshold"].get(0.75); + } else if (sdnEnable) { + /* + * If SDN is enabled but TDN isn't, overwrite all the SDN settings + * with the "no TDN" versions. This makes it easier to enable or + * disable TDN in the tuning file without editing all the other + * parameters. + */ + sdnDeviation = sdnDeviation2 = sdnDeviationNoTdn; + sdnStrength = sdnStrengthNoTdn; + } + + return 0; +} + Denoise::Denoise(Controller *controller) : DenoiseAlgorithm(controller), mode_(DenoiseMode::ColourHighQuality) { @@ -33,39 +72,26 @@ char const *Denoise::name() const int Denoise::read(const libcamera::YamlObject ¶ms) { - sdnEnable_ = params.contains("sdn"); - if (sdnEnable_) { - auto &sdnParams = params["sdn"]; - sdnDeviation_ = sdnParams["deviation"].get(3.2); - sdnStrength_ = sdnParams["strength"].get(0.25); - sdnDeviation2_ = sdnParams["deviation2"].get(sdnDeviation_); - sdnDeviationNoTdn_ = sdnParams["deviation_no_tdn"].get(sdnDeviation_); - sdnStrengthNoTdn_ = sdnParams["strength_no_tdn"].get(sdnStrength_); - sdnTdnBackoff_ = sdnParams["backoff"].get(0.75); + if (!params.contains("normal")) { + configs_["normal"].read(params); + currentConfig_ = &configs_["normal"]; + + return 0; } - cdnEnable_ = params.contains("cdn"); - if (cdnEnable_) { - auto &cdnParams = params["cdn"]; - cdnDeviation_ = cdnParams["deviation"].get(120); - cdnStrength_ = cdnParams["strength"].get(0.2); + for (const auto &[key, value] : params.asDict()) { + if (configs_[key].read(value)) { + LOG(RPiDenoise, Error) << "Failed to read denoise config " << key; + return -EINVAL; + } } - tdnEnable_ = params.contains("tdn"); - if (tdnEnable_) { - auto &tdnParams = params["tdn"]; - tdnDeviation_ = tdnParams["deviation"].get(0.5); - tdnThreshold_ = tdnParams["threshold"].get(0.75); - } else if (sdnEnable_) { - /* - * If SDN is enabled but TDN isn't, overwrite all the SDN settings - * with the "no TDN" versions. This makes it easier to enable or - * disable TDN in the tuning file without editing all the other - * parameters. - */ - sdnDeviation_ = sdnDeviation2_ = sdnDeviationNoTdn_; - sdnStrength_ = sdnStrengthNoTdn_; + auto it = configs_.find("normal"); + if (it == configs_.end()) { + LOG(RPiDenoise, Error) << "No normal denoise settings found"; + return -EINVAL; } + currentConfig_ = &it->second; return 0; } @@ -78,9 +104,9 @@ void Denoise::switchMode([[maybe_unused]] CameraMode const &cameraMode, [[maybe_unused]] Metadata *metadata) { /* A mode switch effectively resets temporal denoise and it has to start over. */ - currentSdnDeviation_ = sdnDeviationNoTdn_; - currentSdnStrength_ = sdnStrengthNoTdn_; - currentSdnDeviation2_ = sdnDeviationNoTdn_; + currentSdnDeviation_ = currentConfig_->sdnDeviationNoTdn; + currentSdnStrength_ = currentConfig_->sdnStrengthNoTdn; + currentSdnDeviation2_ = currentConfig_->sdnDeviationNoTdn; } void Denoise::prepare(Metadata *imageMetadata) @@ -97,11 +123,11 @@ void Denoise::prepare(Metadata *imageMetadata) if (mode_ == DenoiseMode::Off) return; - if (sdnEnable_) { + if (currentConfig_->sdnEnable) { struct SdnStatus sdn; sdn.noiseConstant = noiseStatus.noiseConstant * currentSdnDeviation_; sdn.noiseSlope = noiseStatus.noiseSlope * currentSdnDeviation_; - sdn.noiseConstant2 = noiseStatus.noiseConstant * sdnDeviation2_; + sdn.noiseConstant2 = noiseStatus.noiseConstant * currentConfig_->sdnDeviation2; sdn.noiseSlope2 = noiseStatus.noiseSlope * currentSdnDeviation2_; sdn.strength = currentSdnStrength_; imageMetadata->set("sdn.status", sdn); @@ -113,17 +139,17 @@ void Denoise::prepare(Metadata *imageMetadata) << " slope2 " << sdn.noiseSlope2; /* For the next frame, we back off the SDN parameters as TDN ramps up. */ - double f = sdnTdnBackoff_; - currentSdnDeviation_ = f * currentSdnDeviation_ + (1 - f) * sdnDeviation_; - currentSdnStrength_ = f * currentSdnStrength_ + (1 - f) * sdnStrength_; - currentSdnDeviation2_ = f * currentSdnDeviation2_ + (1 - f) * sdnDeviation2_; + double f = currentConfig_->sdnTdnBackoff; + currentSdnDeviation_ = f * currentSdnDeviation_ + (1 - f) * currentConfig_->sdnDeviation; + currentSdnStrength_ = f * currentSdnStrength_ + (1 - f) * currentConfig_->sdnStrength; + currentSdnDeviation2_ = f * currentSdnDeviation2_ + (1 - f) * currentConfig_->sdnDeviation2; } - if (tdnEnable_) { + if (currentConfig_->tdnEnable) { struct TdnStatus tdn; - tdn.noiseConstant = noiseStatus.noiseConstant * tdnDeviation_; - tdn.noiseSlope = noiseStatus.noiseSlope * tdnDeviation_; - tdn.threshold = tdnThreshold_; + tdn.noiseConstant = noiseStatus.noiseConstant * currentConfig_->tdnDeviation; + tdn.noiseSlope = noiseStatus.noiseSlope * currentConfig_->tdnDeviation; + tdn.threshold = currentConfig_->tdnThreshold; imageMetadata->set("tdn.status", tdn); LOG(RPiDenoise, Debug) << "programmed tdn threshold " << tdn.threshold @@ -131,10 +157,10 @@ void Denoise::prepare(Metadata *imageMetadata) << " slope " << tdn.noiseSlope; } - if (cdnEnable_ && mode_ != DenoiseMode::ColourOff) { + if (currentConfig_->cdnEnable && mode_ != DenoiseMode::ColourOff) { struct CdnStatus cdn; - cdn.threshold = cdnDeviation_ * noiseStatus.noiseSlope + noiseStatus.noiseConstant; - cdn.strength = cdnStrength_; + cdn.threshold = currentConfig_->cdnDeviation * noiseStatus.noiseSlope + noiseStatus.noiseConstant; + cdn.strength = currentConfig_->cdnStrength; imageMetadata->set("cdn.status", cdn); LOG(RPiDenoise, Debug) << "programmed cdn threshold " << cdn.threshold @@ -148,6 +174,22 @@ void Denoise::setMode(DenoiseMode mode) mode_ = mode; } +void Denoise::setConfig(std::string const &name) +{ + auto it = configs_.find(name); + if (it == configs_.end()) { + /* + * Some platforms may have no need for different denoise settings, so we only issue + * a warning if there clearly are several configurations. + */ + if (configs_.size() > 1) + LOG(RPiDenoise, Warning) << "No denoise config found for " << name; + else + LOG(RPiDenoise, Debug) << "No denoise config found for " << name; + } else + currentConfig_ = &it->second; +} + // Register algorithm with the system. static Algorithm *Create(Controller *controller) { diff --git a/src/ipa/rpi/controller/rpi/denoise.h b/src/ipa/rpi/controller/rpi/denoise.h index 88b37663e569..92ff4f93edf2 100644 --- a/src/ipa/rpi/controller/rpi/denoise.h +++ b/src/ipa/rpi/controller/rpi/denoise.h @@ -6,6 +6,9 @@ */ #pragma once +#include +#include + #include "algorithm.h" #include "denoise_algorithm.h" @@ -13,6 +16,23 @@ namespace RPiController { // Algorithm to calculate correct denoise settings. +struct DenoiseConfig { + double sdnDeviation; + double sdnStrength; + double sdnDeviation2; + double sdnDeviationNoTdn; + double sdnStrengthNoTdn; + double sdnTdnBackoff; + double cdnDeviation; + double cdnStrength; + double tdnDeviation; + double tdnThreshold; + bool tdnEnable; + bool sdnEnable; + bool cdnEnable; + int read(const libcamera::YamlObject ¶ms); +}; + class Denoise : public DenoiseAlgorithm { public: @@ -23,22 +43,12 @@ public: void switchMode(CameraMode const &cameraMode, Metadata *metadata) override; void prepare(Metadata *imageMetadata) override; void setMode(DenoiseMode mode) override; + void setConfig(std::string const &name) override; private: - double sdnDeviation_; - double sdnStrength_; - double sdnDeviation2_; - double sdnDeviationNoTdn_; - double sdnStrengthNoTdn_; - double sdnTdnBackoff_; - double cdnDeviation_; - double cdnStrength_; - double tdnDeviation_; - double tdnThreshold_; + std::map configs_; + DenoiseConfig *currentConfig_; DenoiseMode mode_; - bool tdnEnable_; - bool sdnEnable_; - bool cdnEnable_; /* SDN parameters attenuate over time if TDN is running. */ double currentSdnDeviation_;