From patchwork Fri Mar 31 07:23:32 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Naushir Patuck X-Patchwork-Id: 18497 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 65D66C329C for ; Fri, 31 Mar 2023 07:40:51 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id B17ED62750; Fri, 31 Mar 2023 09:40:49 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1680248449; bh=yXfpIKnj1URDHAgm+r1nqvyD24jjrUtCCqmSkQhTkb4=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To:Cc: From; b=pA6HbC5EoAI2L6wIPaMjRS5i7fioopDWO6Cs3IUTnWpqyRq1H+EvJQhT2aN8opJdJ wM8i3dXMEkjTUNStDGvnF4KDcNH/7GEo5o/vYFalpb896yvooO8GJ+uqdnkoRshBRs VHKjNSn44OY3LY5d6zXLkN4oLakgj1GXaAeJJUWiCPwcnhQEPK25Ll/fZ7i9j2NGii u0pgsRU69QjPWywXENLHsfrywMqxbuwEyprgB3Z69EKrhZdHQbbonuhjKPoPTKOaa5 EZNVhyXC4T0NQghnIDNRE+3TdJt3lpXwtnAITPXio0TQQYbBA8zpIAPClwW4xKWmpn gzNJl1aCpdqHQ== Received: from mail-io1-xd35.google.com (mail-io1-xd35.google.com [IPv6:2607:f8b0:4864:20::d35]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id AB6E9626E3 for ; Fri, 31 Mar 2023 09:40:48 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="GHiiJmci"; dkim-atps=neutral Received: by mail-io1-xd35.google.com with SMTP id e13so9411943ioc.0 for ; Fri, 31 Mar 2023 00:40:48 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; t=1680248447; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=r1xDF00lIPNyXEShIbWt3BBCmUdK67y4koidjVf0DD8=; b=GHiiJmciwIfMRTfaCtIYrnCybf46IQmtovsIuf4R+AhRxXbRzbhm1xbuusc3UZeuVn O5kCS5QEwKAJxtbhyDGKRIEKj7EDs3AAPMuTnO3v+YRvxKk7Wh4zPEHeYGPmuGLjWqAL lryDDFgF3r4v4hfD+ji/tQ8QeOzZQ+lBG5kH7qRnub44cW9zXoM8g4yNvy2E6hYoq4sx Me4FtvOqILoCJVwM6qe/PP862Sz6J6xB+4FpvPx48sKiGqeHlUNY6V0x3w14NCf9lrIO SS5IIBcgxuiMCcKHlJ6U4tllqqxMyN/MKQzhVdFZ4WkP4c0z1/8QPIHj3ptXv6iPLFlG 9x2w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; t=1680248447; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=r1xDF00lIPNyXEShIbWt3BBCmUdK67y4koidjVf0DD8=; b=75z/ZJv1/89m6S/Z+96mtVhc9/VAk4AG4g/h2UTWiqgOOYX8RpuBAk3Ln4VpVafN+z tD7RUH6OPhlReGL4NArqkGvP8O0pJf6+8dmO8+1nG8DZsihNIM2PnVoF3GpC9cHkUAT8 u1SKbjarFWJCmPwTes5df4NwNAB2Qrqm8baDF/PuXvZfxB83bD8I1cSb8cmtntzOqbTP SDUTaFvHsBf0xXdkrVOqMVc0g/Dilly/ZpkOIe0rS4cOHKbwX9T6qh1LlgJ4ZZFdgLk2 wdQAeA0k0FWCeiEm6KYbCWeVQPBaifAjbN+eQNnCrqSvxzqbBv1b1ACXtMofu8lIPILJ 2Oqw== X-Gm-Message-State: AO0yUKUG17bSSsHLmlXr/6oWhTc3Qx8jArjZTiI2nEBzVu/2lrKHBGPz hXzt5Pxt5b+nnC/bG7RRqz2C4GT27+PYEon29fKWEg== X-Google-Smtp-Source: AK7set9lpzviuVgXZHnVWiK+Dvc7IJrazBarYnRhM8NnFVpP3YHciazlEpFxvsWv/Loc0FT3+N2ooQ== X-Received: by 2002:a6b:6101:0:b0:71b:5cd7:fcd9 with SMTP id v1-20020a6b6101000000b0071b5cd7fcd9mr17957551iob.20.1680248447108; Fri, 31 Mar 2023 00:40:47 -0700 (PDT) Received: from localhost.localdomain ([93.93.133.154]) by smtp.gmail.com with ESMTPSA id i17-20020a6bf411000000b0074555814e73sm421943iog.32.2023.03.31.00.40.46 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 31 Mar 2023 00:40:46 -0700 (PDT) To: libcamera-devel@lists.libcamera.org Date: Fri, 31 Mar 2023 08:23:32 +0100 Message-Id: <20230331072341.24851-2-naush@raspberrypi.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230331072341.24851-1-naush@raspberrypi.com> References: <20230331072341.24851-1-naush@raspberrypi.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v3 01/10] ipa: raspberrypi Store the target string in the controller X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Naushir Patuck via libcamera-devel From: Naushir Patuck Reply-To: Naushir Patuck Cc: Jacopo Mondi Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" The target string may be used by algorithms to determine the running hardware target. Store the target string provided by the camera tuning files in the controller state. Add a getTarget() member function to retrieve this string. Validate the correct hardware target ("bcm2835") during the IPA initialisation phase. Signed-off-by: Naushir Patuck Reviewed-by: Jacopo Mondi Reviewed-by: Kieran Bingham --- src/ipa/raspberrypi/controller/algorithm.h | 4 ++++ src/ipa/raspberrypi/controller/controller.cpp | 6 ++++++ src/ipa/raspberrypi/controller/controller.h | 4 ++++ src/ipa/raspberrypi/raspberrypi.cpp | 8 ++++++++ 4 files changed, 22 insertions(+) diff --git a/src/ipa/raspberrypi/controller/algorithm.h b/src/ipa/raspberrypi/controller/algorithm.h index 4f3275987305..7c22fbe4945c 100644 --- a/src/ipa/raspberrypi/controller/algorithm.h +++ b/src/ipa/raspberrypi/controller/algorithm.h @@ -41,6 +41,10 @@ public: { return controller_->getGlobalMetadata(); } + const std::string &getTarget() const + { + return controller_->getTarget(); + } private: Controller *controller_; diff --git a/src/ipa/raspberrypi/controller/controller.cpp b/src/ipa/raspberrypi/controller/controller.cpp index e91567852d00..a6250ee140b0 100644 --- a/src/ipa/raspberrypi/controller/controller.cpp +++ b/src/ipa/raspberrypi/controller/controller.cpp @@ -38,6 +38,7 @@ int Controller::read(char const *filename) std::unique_ptr root = YamlParser::parse(file); double version = (*root)["version"].get(1.0); + target_ = (*root)["target"].get("bcm2835"); if (version < 2.0) { LOG(RPiController, Warning) @@ -142,3 +143,8 @@ Algorithm *Controller::getAlgorithm(std::string const &name) const } return nullptr; } + +const std::string &Controller::getTarget() const +{ + return target_; +} diff --git a/src/ipa/raspberrypi/controller/controller.h b/src/ipa/raspberrypi/controller/controller.h index e6c950c3a509..24e02903d438 100644 --- a/src/ipa/raspberrypi/controller/controller.h +++ b/src/ipa/raspberrypi/controller/controller.h @@ -46,6 +46,7 @@ public: void process(StatisticsPtr stats, Metadata *imageMetadata); Metadata &getGlobalMetadata(); Algorithm *getAlgorithm(std::string const &name) const; + const std::string &getTarget() const; protected: int createAlgorithm(const std::string &name, const libcamera::YamlObject ¶ms); @@ -53,6 +54,9 @@ protected: Metadata globalMetadata_; std::vector algorithms_; bool switchModeCalled_; + +private: + std::string target_; }; } /* namespace RPiController */ diff --git a/src/ipa/raspberrypi/raspberrypi.cpp b/src/ipa/raspberrypi/raspberrypi.cpp index 66a020d022cd..92c65a2c2f6c 100644 --- a/src/ipa/raspberrypi/raspberrypi.cpp +++ b/src/ipa/raspberrypi/raspberrypi.cpp @@ -266,6 +266,14 @@ int IPARPi::init(const IPASettings &settings, bool lensPresent, IPAInitResult *r return ret; } + const std::string &target = controller_.getTarget(); + if (target != "bcm2835") { + LOG(IPARPI, Error) + << "Tuning data file target returned \"" << target << "\"" + << ", expected \"bcm2835\""; + return -EINVAL; + } + lensPresent_ = lensPresent; controller_.initialise(); From patchwork Fri Mar 31 07:23:33 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Naushir Patuck X-Patchwork-Id: 18498 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 58619C329D for ; Fri, 31 Mar 2023 07:40:52 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id AE47962759; Fri, 31 Mar 2023 09:40:51 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1680248451; bh=eJv12V6KwFXPTSxr4TloZ2Rl1o7jM2rYSgPYX4ktmR0=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To: From; b=TcLbVGnIh5OmzjXkNeDT+FSL6vWM3dY3XeeppGVhuFcnUx1QDQwRZyLLy6t7QCC/+ s5b7xiRcBznYo16jgFMy4yV29nqm+emEF6UJbKvqolWrhDd0saG+OKdZBfP7tIct3K AuoE8+v0+Ot1/FhWmNpScs8YLN7LcVN0ZrBb+oESzuh9q9YDHp3HtFFbQLkuAf6t4+ E4hfPpgQM/Ur/jmnkq0hW8FyIlvoO63U+sJGEnJiCJ2VRGpUTZAkOEzAM2Rj0Lmsde ENbJwRmkDTLygHKXzwKKS3V5HrTH5DtzPVhZDeBJVrYpsWHvr9FFeJ9KePfxIFY2io 5Z+IQZ3LO3ffg== Received: from mail-io1-xd2c.google.com (mail-io1-xd2c.google.com [IPv6:2607:f8b0:4864:20::d2c]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id C62AA62754 for ; Fri, 31 Mar 2023 09:40:49 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="nfsDN1tq"; dkim-atps=neutral Received: by mail-io1-xd2c.google.com with SMTP id h187so7668253iof.7 for ; Fri, 31 Mar 2023 00:40:49 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; t=1680248448; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=WfsHCbIkbpjQForzTwd4q36w5B1LLe0FLiY7FZyGvyo=; b=nfsDN1tqHVHyVmLTZVaTJjQtwoWtURIIRg6L6iQVoRQ3VsIBXrB2OtfuFlxrPm3rdw Be81alWGQVfKRSKaFVDZGEPnYAZxAnBPutSS3vVUG5BWxAcqSLUfL13rhzqSo8nP/g11 Izj48Yh3jhJn/GMBO9OyzjzlomRQ9nTw9/HDI8c2fKy9w+6D9Cm/Qg0F4MYSqEpaca37 vAN92BZm9j0a2U+vTTKohcmIDNIkRIPVyt/K9QxH/h3DlszRPGI+YHDX8I7PA5zwUEZs im3d7ysCDt8jrvvt++G3ePYRpjSpDQ3DT5o512SD/fYhnVBz9eVsmTctqUOdmoHyc04l 2RXw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; t=1680248448; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=WfsHCbIkbpjQForzTwd4q36w5B1LLe0FLiY7FZyGvyo=; b=VObWhrYjaIRJSCOahIimFSWw4d8iOIxlc2xjqclCtfCpIAW0HBpl1HD5TyaaHXBM8L PBvegV3ARKSxWcrxWJqMVrJ1absxecjLNPc7d6I1v9ZW2JFG4VG8ydC8hDj8ftxrmPfY +lNFMFNE40cKduXY+7GvOrnWkB9mg/km88k+maG62Kx9jWtQekHzFhElASSt//Fus5PF iV/wms23zTL1ifxctxpSvCx4rCqVkWKUsfzfB6qDquFuelikJv9eXNC4elaovC/GbPj/ Hb7CWc8gSlwdNLdqks/EQyroK+WFW/aQMcsfZj0Jr17r+jUoBslhdZMkbFIMuSNn0++M REgw== X-Gm-Message-State: AO0yUKXNx8BRvQ4G5ohwylBuk2D5K6btl48Yfj3TdqiunmXJek4oKOBX o1ejT08twWvZC2kZafL2Z89S661S0FIeF0bRho98rw== X-Google-Smtp-Source: AK7set99C4JBGL/5dwIPE4XNdh+R7u5RGbmWx1p5KJbAq62b/aBjNamxB1Rgr4gBRJdM+uv8sO/qZQ== X-Received: by 2002:a5e:d612:0:b0:753:121f:72a7 with SMTP id w18-20020a5ed612000000b00753121f72a7mr20020518iom.8.1680248448340; Fri, 31 Mar 2023 00:40:48 -0700 (PDT) Received: from localhost.localdomain ([93.93.133.154]) by smtp.gmail.com with ESMTPSA id i17-20020a6bf411000000b0074555814e73sm421943iog.32.2023.03.31.00.40.47 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 31 Mar 2023 00:40:48 -0700 (PDT) To: libcamera-devel@lists.libcamera.org Date: Fri, 31 Mar 2023 08:23:33 +0100 Message-Id: <20230331072341.24851-3-naush@raspberrypi.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230331072341.24851-1-naush@raspberrypi.com> References: <20230331072341.24851-1-naush@raspberrypi.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v3 02/10] ipa: raspberrypi: Add hardware configuration to the controller X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Naushir Patuck via libcamera-devel From: Naushir Patuck Reply-To: Naushir Patuck Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Add a new Controller::HardwareConfig structure that captures the hardware statistics grid/histogram sizes and pipeline widths. This ensures there is a single centralised places for these parameters. Add a getHardwareConfig() helper function to retrieve these values for a given hardware target. Update the statistics populating routine in the IPA to use the values from this structure instead of the hardcoded numbers. Signed-off-by: Naushir Patuck Reviewed-by: David Plowman Reviewed-by: Kieran Bingham --- src/ipa/raspberrypi/controller/algorithm.h | 4 +++ src/ipa/raspberrypi/controller/controller.cpp | 31 +++++++++++++++++++ src/ipa/raspberrypi/controller/controller.h | 11 +++++++ src/ipa/raspberrypi/raspberrypi.cpp | 21 +++++-------- 4 files changed, 54 insertions(+), 13 deletions(-) diff --git a/src/ipa/raspberrypi/controller/algorithm.h b/src/ipa/raspberrypi/controller/algorithm.h index 7c22fbe4945c..4aa814ebbebd 100644 --- a/src/ipa/raspberrypi/controller/algorithm.h +++ b/src/ipa/raspberrypi/controller/algorithm.h @@ -45,6 +45,10 @@ public: { return controller_->getTarget(); } + const Controller::HardwareConfig &getHardwareConfig() const + { + return controller_->getHardwareConfig(); + } private: Controller *controller_; diff --git a/src/ipa/raspberrypi/controller/controller.cpp b/src/ipa/raspberrypi/controller/controller.cpp index a6250ee140b0..fa1721130783 100644 --- a/src/ipa/raspberrypi/controller/controller.cpp +++ b/src/ipa/raspberrypi/controller/controller.cpp @@ -20,6 +20,25 @@ using namespace libcamera; LOG_DEFINE_CATEGORY(RPiController) +static const std::map HardwareConfigMap = { + { + "bcm2835", + { + /* + * There are only ever 15 AGC regions computed by the firmware + * due to zoning, but the HW defines AGC_REGIONS == 16! + */ + .agcRegions = { 15 , 1 }, + .agcZoneWeights = { 15 , 1 }, + .awbRegions = { 16, 12 }, + .focusRegions = { 4, 3 }, + .numHistogramBins = 128, + .numGammaPoints = 33, + .pipelineWidth = 13 + } + }, +}; + Controller::Controller() : switchModeCalled_(false) { @@ -148,3 +167,15 @@ const std::string &Controller::getTarget() const { return target_; } + +const Controller::HardwareConfig &Controller::getHardwareConfig() const +{ + auto cfg = HardwareConfigMap.find(getTarget()); + + /* + * This really should not happen, the IPA ought to validate the target + * on initialisation. + */ + ASSERT(cfg != HardwareConfigMap.end()); + return cfg->second; +} diff --git a/src/ipa/raspberrypi/controller/controller.h b/src/ipa/raspberrypi/controller/controller.h index 24e02903d438..c6af5cd6c7d4 100644 --- a/src/ipa/raspberrypi/controller/controller.h +++ b/src/ipa/raspberrypi/controller/controller.h @@ -37,6 +37,16 @@ typedef std::unique_ptr AlgorithmPtr; class Controller { public: + struct HardwareConfig { + libcamera::Size agcRegions; + libcamera::Size agcZoneWeights; + libcamera::Size awbRegions; + libcamera::Size focusRegions; + unsigned int numHistogramBins; + unsigned int numGammaPoints; + unsigned int pipelineWidth; + }; + Controller(); ~Controller(); int read(char const *filename); @@ -47,6 +57,7 @@ public: Metadata &getGlobalMetadata(); Algorithm *getAlgorithm(std::string const &name) const; const std::string &getTarget() const; + const HardwareConfig &getHardwareConfig() const; protected: int createAlgorithm(const std::string &name, const libcamera::YamlObject ¶ms); diff --git a/src/ipa/raspberrypi/raspberrypi.cpp b/src/ipa/raspberrypi/raspberrypi.cpp index 92c65a2c2f6c..e24529920a4b 100644 --- a/src/ipa/raspberrypi/raspberrypi.cpp +++ b/src/ipa/raspberrypi/raspberrypi.cpp @@ -1394,20 +1394,19 @@ RPiController::StatisticsPtr IPARPi::fillStatistics(bcm2835_isp_stats *stats) co { using namespace RPiController; + const Controller::HardwareConfig &hw = controller_.getHardwareConfig(); unsigned int i; StatisticsPtr statistics = std::make_unique(Statistics::AgcStatsPos::PreWb, Statistics::ColourStatsPos::PostLsc); /* RGB histograms are not used, so do not populate them. */ - statistics->yHist = RPiController::Histogram(stats->hist[0].g_hist, NUM_HISTOGRAM_BINS); + statistics->yHist = RPiController::Histogram(stats->hist[0].g_hist, + hw.numHistogramBins); - /* - * All region sums are based on a 13-bit pipeline bit-depth. Normalise - * this to 16-bits for the AGC/AWB/ALSC algorithms. - */ - constexpr unsigned int scale = Statistics::NormalisationFactorPow2 - 13; + /* All region sums are based on a 16-bit normalised pipeline bit-depth. */ + unsigned int scale = Statistics::NormalisationFactorPow2 - hw.pipelineWidth; - statistics->awbRegions.init({ DEFAULT_AWB_REGIONS_X, DEFAULT_AWB_REGIONS_Y }); + statistics->awbRegions.init(hw.awbRegions); for (i = 0; i < statistics->awbRegions.numRegions(); i++) statistics->awbRegions.set(i, { { stats->awb_stats[i].r_sum << scale, stats->awb_stats[i].g_sum << scale, @@ -1415,11 +1414,7 @@ RPiController::StatisticsPtr IPARPi::fillStatistics(bcm2835_isp_stats *stats) co stats->awb_stats[i].counted, stats->awb_stats[i].notcounted }); - /* - * There are only ever 15 regions computed by the firmware due to zoning, - * but the HW defines AGC_REGIONS == 16! - */ - statistics->agcRegions.init(15); + statistics->agcRegions.init(hw.agcRegions); for (i = 0; i < statistics->agcRegions.numRegions(); i++) statistics->agcRegions.set(i, { { stats->agc_stats[i].r_sum << scale, stats->agc_stats[i].g_sum << scale, @@ -1427,7 +1422,7 @@ RPiController::StatisticsPtr IPARPi::fillStatistics(bcm2835_isp_stats *stats) co stats->agc_stats[i].counted, stats->awb_stats[i].notcounted }); - statistics->focusRegions.init({ 4, 3 }); + statistics->focusRegions.init(hw.focusRegions); for (i = 0; i < statistics->focusRegions.numRegions(); i++) statistics->focusRegions.set(i, { stats->focus_stats[i].contrast_val[1][1] / 1000, stats->focus_stats[i].contrast_val_num[1][1], From patchwork Fri Mar 31 07:23:34 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Naushir Patuck X-Patchwork-Id: 18499 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id A0D2DC0F2A for ; Fri, 31 Mar 2023 07:40:54 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 36CD162754; Fri, 31 Mar 2023 09:40:54 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1680248454; bh=cVTCM2l5oLJP7ifI5S2YAS8hJyVbZY5fQMRbArULc5Q=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To: From; b=F1SbUF5UCM+nvDgTv8/X8WIOveDvxhygWd0xWG/3M2I7DLP11RPQecKhwyUi/7rqS Oi8SpQoXRqnAriRKB8Exng8hX1xkU1IdKxT3vvD2vK2ChwLzxIwHMUJfwTDkGpi1u6 BfgG79o4djoudKNSQ6sB5yqSIulqD2sEKJqYshb2TaQwI+HJ9zabDtOfxqto7lDwUo MME8u40oPeQpLn0QN35OvO2xcxzjAqqPfT8yM2FbhiKOazbD+yyzznQkSXhOE4rkEV k3RM4iSpliMkGpEFKJoWQMBIucapfe4FlbRUcBiwODmrDTbmnkCpvQhBP1SgN0QUMX wwgEasTw8SOZQ== Received: from mail-io1-xd30.google.com (mail-io1-xd30.google.com [IPv6:2607:f8b0:4864:20::d30]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id BD7D66275E for ; Fri, 31 Mar 2023 09:40:51 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="h0Ae5X2l"; dkim-atps=neutral Received: by mail-io1-xd30.google.com with SMTP id o12so9392025iow.6 for ; Fri, 31 Mar 2023 00:40:51 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; t=1680248450; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=MfhFi/8krPae9p94SSKFeSLOBmJy0+mnYWpAsiMB3oI=; b=h0Ae5X2lbuJNlkQ0dbXduCFvgQWELSIdhraNopw3X/80Y0jIlvoOyKIapbbJRsDegi 8JVnNZpb4vO+UctYSrSoRPbqrVeNaZCv5TWtbamZYCRjEcDBcPtthMd+eBMbA6DDRcSq k5hWk2GiRPRjFRj3C8aLHkKZ1QwVoq9mpif6HITX8kS6aEdiRc0BWg2ov1CL7CBGp5B8 4EYuP901QgCD2TAZlJeVHb1i/QLfCYnEOWK92p1Ti45tG/mZcPRpr3pHpLr5S+Vv1Kwf lw6ADhJpSSgew8Mdlr2b2P0XT5YxeDnU40NN0wSvlMsm8+8xpBQ6J00Svst/As1HxhLF KYrA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; t=1680248450; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=MfhFi/8krPae9p94SSKFeSLOBmJy0+mnYWpAsiMB3oI=; b=CmUC7IcmAFMt4kW+sTfhz898Ju2rc1yuafEHYVros+SKY1MsTjs1KDSpsGOoseGS97 AdNHk4fVAhaGazjO3V7b6x/hE+pQ0ZZ2qhzkH/KiShzq1qW36ec+Q0s42Gf9QiC/B/5i 7fv8DiYMxOeYWSVhjr/NMGe+kK5dhq5j5CCCB6R2MuZQju5/nmhBm2tMTHKO5sqZXYWO 1X5crBlZ8mVxgCAj08ELNj+zWdXuR0Eh/zMVHydoRprh6EpRo5c9kBUqgZXsoZ1llo/O eOjlfQDPzcaT/qtC+AH5e2uDKlzG+FzPwwM9kskl+iDR2ljbqybUvbuUrHsaTpGv08Ps 9Spg== X-Gm-Message-State: AO0yUKUoP41QmUJd95eL5ZaAff5EvxidMhl2wMcjBvtQ9eXXeGP46c4i OVBLkKU1SiUyLZ9VUnzhAO+KL5hyhPZay/xSQuZZ8A== X-Google-Smtp-Source: AK7set9aumS6jLd/3AKC/eoMh3aW+n0tN5u0n8eFL51cJJwPwB3bC+MXoSITYL9aMvU92fIhQmMqjg== X-Received: by 2002:a5e:9412:0:b0:759:7a20:d795 with SMTP id q18-20020a5e9412000000b007597a20d795mr16729985ioj.19.1680248449654; Fri, 31 Mar 2023 00:40:49 -0700 (PDT) Received: from localhost.localdomain ([93.93.133.154]) by smtp.gmail.com with ESMTPSA id i17-20020a6bf411000000b0074555814e73sm421943iog.32.2023.03.31.00.40.48 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 31 Mar 2023 00:40:49 -0700 (PDT) To: libcamera-devel@lists.libcamera.org Date: Fri, 31 Mar 2023 08:23:34 +0100 Message-Id: <20230331072341.24851-4-naush@raspberrypi.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230331072341.24851-1-naush@raspberrypi.com> References: <20230331072341.24851-1-naush@raspberrypi.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v3 03/10] ipa: raspberrypi: Generalise the ALSC algorithm X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Naushir Patuck via libcamera-devel From: Naushir Patuck Reply-To: Naushir Patuck Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Remove any hard-coded assumptions about the target hardware platform from the ALSC algorithm. Instead, use the "target" string provided by the camera tuning config and generalised statistics structures to determing parameters such as grid and region sizes. The ALSC calculations use run-time allocated arrays/vectors on every frame. Allocating these might add a non-trivial run-time penalty. Replace these dynamic allocations with a set of reusable pre-allocated vectors during the init phase. Signed-off-by: Naushir Patuck Signed-off-by: David Plowman Reviewed-by: David Plowman Reviewed-by: Kieran Bingham --- src/ipa/raspberrypi/controller/alsc_status.h | 13 +- src/ipa/raspberrypi/controller/rpi/alsc.cpp | 341 +++++++++++-------- src/ipa/raspberrypi/controller/rpi/alsc.h | 29 +- src/ipa/raspberrypi/raspberrypi.cpp | 9 +- 4 files changed, 224 insertions(+), 168 deletions(-) diff --git a/src/ipa/raspberrypi/controller/alsc_status.h b/src/ipa/raspberrypi/controller/alsc_status.h index e5aa7e37c330..49a9f4a0cb5a 100644 --- a/src/ipa/raspberrypi/controller/alsc_status.h +++ b/src/ipa/raspberrypi/controller/alsc_status.h @@ -6,16 +6,17 @@ */ #pragma once +#include + /* * The ALSC algorithm should post the following structure into the image's * "alsc.status" metadata. */ -constexpr unsigned int AlscCellsX = 16; -constexpr unsigned int AlscCellsY = 12; - struct AlscStatus { - double r[AlscCellsY][AlscCellsX]; - double g[AlscCellsY][AlscCellsX]; - double b[AlscCellsY][AlscCellsX]; + std::vector r; + std::vector g; + std::vector b; + unsigned int rows; + unsigned int cols; }; diff --git a/src/ipa/raspberrypi/controller/rpi/alsc.cpp b/src/ipa/raspberrypi/controller/rpi/alsc.cpp index eb4e2f9496e1..51fe5d73f52d 100644 --- a/src/ipa/raspberrypi/controller/rpi/alsc.cpp +++ b/src/ipa/raspberrypi/controller/rpi/alsc.cpp @@ -5,6 +5,7 @@ * alsc.cpp - ALSC (auto lens shading correction) control algorithm */ +#include #include #include #include @@ -24,9 +25,6 @@ LOG_DEFINE_CATEGORY(RPiAlsc) #define NAME "rpi.alsc" -static const int X = AlscCellsX; -static const int Y = AlscCellsY; -static const int XY = X * Y; static const double InsufficientData = -1.0; Alsc::Alsc(Controller *controller) @@ -51,8 +49,11 @@ char const *Alsc::name() const return NAME; } -static int generateLut(double *lut, const libcamera::YamlObject ¶ms) +static int generateLut(std::vector &lut, const libcamera::YamlObject ¶ms, + const Size &size) { + /* These must be signed ints for the co-ordinate calculations below. */ + int X = size.width, Y = size.height; double cstrength = params["corner_strength"].get(2.0); if (cstrength <= 1.0) { LOG(RPiAlsc, Error) << "corner_strength must be > 1.0"; @@ -81,9 +82,9 @@ static int generateLut(double *lut, const libcamera::YamlObject ¶ms) return 0; } -static int readLut(double *lut, const libcamera::YamlObject ¶ms) +static int readLut(std::vector &lut, const libcamera::YamlObject ¶ms, const Size &size) { - if (params.size() != XY) { + if (params.size() != size.width * size.height) { LOG(RPiAlsc, Error) << "Invalid number of entries in LSC table"; return -EINVAL; } @@ -101,7 +102,7 @@ static int readLut(double *lut, const libcamera::YamlObject ¶ms) static int readCalibrations(std::vector &calibrations, const libcamera::YamlObject ¶ms, - std::string const &name) + std::string const &name, const Size &size) { if (params.contains(name)) { double lastCt = 0; @@ -119,7 +120,7 @@ static int readCalibrations(std::vector &calibrations, calibration.ct = lastCt = ct; const libcamera::YamlObject &table = p["table"]; - if (table.size() != XY) { + if (table.size() != size.width * size.height) { LOG(RPiAlsc, Error) << "Incorrect number of values for ct " << ct << " in " << name; @@ -127,6 +128,7 @@ static int readCalibrations(std::vector &calibrations, } int num = 0; + calibration.table.resize(size.width * size.height); for (const auto &elem : table.asList()) { value = elem.get(); if (!value) @@ -134,7 +136,7 @@ static int readCalibrations(std::vector &calibrations, calibration.table[num++] = *value; } - calibrations.push_back(calibration); + calibrations.push_back(std::move(calibration)); LOG(RPiAlsc, Debug) << "Read " << name << " calibration for ct " << ct; } @@ -144,6 +146,7 @@ static int readCalibrations(std::vector &calibrations, int Alsc::read(const libcamera::YamlObject ¶ms) { + config_.tableSize = getHardwareConfig().awbRegions; config_.framePeriod = params["frame_period"].get(12); config_.startupFrames = params["startup_frames"].get(10); config_.speed = params["speed"].get(0.05); @@ -153,28 +156,29 @@ int Alsc::read(const libcamera::YamlObject ¶ms) config_.minCount = params["min_count"].get(10.0); config_.minG = params["min_G"].get(50); config_.omega = params["omega"].get(1.3); - config_.nIter = params["n_iter"].get(X + Y); + config_.nIter = params["n_iter"].get(config_.tableSize.width + config_.tableSize.height); config_.luminanceStrength = params["luminance_strength"].get(1.0); - for (int i = 0; i < XY; i++) - config_.luminanceLut[i] = 1.0; + config_.luminanceLut.resize(config_.tableSize.width * config_.tableSize.height, 1.0); int ret = 0; if (params.contains("corner_strength")) - ret = generateLut(config_.luminanceLut, params); + ret = generateLut(config_.luminanceLut, params, config_.tableSize); else if (params.contains("luminance_lut")) - ret = readLut(config_.luminanceLut, params["luminance_lut"]); + ret = readLut(config_.luminanceLut, params["luminance_lut"], config_.tableSize); else LOG(RPiAlsc, Warning) << "no luminance table - assume unity everywhere"; if (ret) return ret; - ret = readCalibrations(config_.calibrationsCr, params, "calibrations_Cr"); + ret = readCalibrations(config_.calibrationsCr, params, "calibrations_Cr", + config_.tableSize); if (ret) return ret; - ret = readCalibrations(config_.calibrationsCb, params, "calibrations_Cb"); + ret = readCalibrations(config_.calibrationsCb, params, "calibrations_Cb", + config_.tableSize); if (ret) return ret; @@ -187,13 +191,16 @@ int Alsc::read(const libcamera::YamlObject ¶ms) static double getCt(Metadata *metadata, double defaultCt); static void getCalTable(double ct, std::vector const &calibrations, - double calTable[XY]); -static void resampleCalTable(double const calTableIn[XY], CameraMode const &cameraMode, - double calTableOut[XY]); -static void compensateLambdasForCal(double const calTable[XY], double const oldLambdas[XY], - double newLambdas[XY]); -static void addLuminanceToTables(double results[3][Y][X], double const lambdaR[XY], double lambdaG, - double const lambdaB[XY], double const luminanceLut[XY], + std::vector &calTable); +static void resampleCalTable(const std::vector &calTableIn, CameraMode const &cameraMode, + const Size &size, std::vector &calTableOut); +static void compensateLambdasForCal(const std::vector &calTable, + const std::vector &oldLambdas, + std::vector &newLambdas); +static void addLuminanceToTables(std::array, 3> &results, + const std::vector &lambdaR, double lambdaG, + const std::vector &lambdaB, + const std::vector &luminanceLut, double luminanceStrength); void Alsc::initialise() @@ -201,7 +208,28 @@ void Alsc::initialise() frameCount2_ = frameCount_ = framePhase_ = 0; firstTime_ = true; ct_ = config_.defaultCt; + + const size_t XY = config_.tableSize.width * config_.tableSize.height; + + for (auto &r : syncResults_) + r.resize(XY); + for (auto &r : prevSyncResults_) + r.resize(XY); + for (auto &r : asyncResults_) + r.resize(XY); + + luminanceTable_.resize(XY); + asyncLambdaR_.resize(XY); + asyncLambdaB_.resize(XY); /* The lambdas are initialised in the SwitchMode. */ + lambdaR_.resize(XY); + lambdaB_.resize(XY); + + /* Temporaries for the computations, but sensible to allocate this up-front! */ + for (auto &c : tmpC_) + c.resize(XY); + for (auto &m : tmpM_) + m.resize(XY); } void Alsc::waitForAysncThread() @@ -262,7 +290,7 @@ void Alsc::switchMode(CameraMode const &cameraMode, * We must resample the luminance table like we do the others, but it's * fixed so we can simply do it up front here. */ - resampleCalTable(config_.luminanceLut, cameraMode_, luminanceTable_); + resampleCalTable(config_.luminanceLut, cameraMode_, config_.tableSize, luminanceTable_); if (resetTables) { /* @@ -272,18 +300,18 @@ void Alsc::switchMode(CameraMode const &cameraMode, * the lambdas, but the rest of this code then echoes the code in * doAlsc, without the adaptive algorithm. */ - for (int i = 0; i < XY; i++) - lambdaR_[i] = lambdaB_[i] = 1.0; - double calTableR[XY], calTableB[XY], calTableTmp[XY]; + std::fill(lambdaR_.begin(), lambdaR_.end(), 1.0); + std::fill(lambdaB_.begin(), lambdaB_.end(), 1.0); + std::vector &calTableR = tmpC_[0], &calTableB = tmpC_[1], &calTableTmp = tmpC_[2]; getCalTable(ct_, config_.calibrationsCr, calTableTmp); - resampleCalTable(calTableTmp, cameraMode_, calTableR); + resampleCalTable(calTableTmp, cameraMode_, config_.tableSize, calTableR); getCalTable(ct_, config_.calibrationsCb, calTableTmp); - resampleCalTable(calTableTmp, cameraMode_, calTableB); + resampleCalTable(calTableTmp, cameraMode_, config_.tableSize, calTableB); compensateLambdasForCal(calTableR, lambdaR_, asyncLambdaR_); compensateLambdasForCal(calTableB, lambdaB_, asyncLambdaB_); addLuminanceToTables(syncResults_, asyncLambdaR_, 1.0, asyncLambdaB_, luminanceTable_, config_.luminanceStrength); - memcpy(prevSyncResults_, syncResults_, sizeof(prevSyncResults_)); + prevSyncResults_ = syncResults_; framePhase_ = config_.framePeriod; /* run the algo again asap */ firstTime_ = false; } @@ -294,7 +322,7 @@ void Alsc::fetchAsyncResults() LOG(RPiAlsc, Debug) << "Fetch ALSC results"; asyncFinished_ = false; asyncStarted_ = false; - memcpy(syncResults_, asyncResults_, sizeof(syncResults_)); + syncResults_ = asyncResults_; } double getCt(Metadata *metadata, double defaultCt) @@ -316,9 +344,9 @@ static void copyStats(RgbyRegions ®ions, StatisticsPtr &stats, if (!regions.numRegions()) regions.init(stats->awbRegions.size()); - double *rTable = (double *)status.r; - double *gTable = (double *)status.g; - double *bTable = (double *)status.b; + const std::vector &rTable = status.r; + const std::vector &gTable = status.g; + const std::vector &bTable = status.b; for (unsigned int i = 0; i < stats->awbRegions.numRegions(); i++) { auto r = stats->awbRegions.get(i); r.val.rSum = static_cast(r.val.rSum / rTable[i]); @@ -344,12 +372,9 @@ void Alsc::restartAsync(StatisticsPtr &stats, Metadata *imageMetadata) if (imageMetadata->get("alsc.status", alscStatus) != 0) { LOG(RPiAlsc, Warning) << "No ALSC status found for applied gains!"; - for (int y = 0; y < Y; y++) - for (int x = 0; x < X; x++) { - alscStatus.r[y][x] = 1.0; - alscStatus.g[y][x] = 1.0; - alscStatus.b[y][x] = 1.0; - } + alscStatus.r.resize(config_.tableSize.width * config_.tableSize.height, 1.0); + alscStatus.g.resize(config_.tableSize.width * config_.tableSize.height, 1.0); + alscStatus.b.resize(config_.tableSize.width * config_.tableSize.height, 1.0); } copyStats(statistics_, stats, alscStatus); framePhase_ = 0; @@ -380,15 +405,15 @@ void Alsc::prepare(Metadata *imageMetadata) fetchAsyncResults(); } /* Apply IIR filter to results and program into the pipeline. */ - double *ptr = (double *)syncResults_, - *pptr = (double *)prevSyncResults_; - for (unsigned int i = 0; i < sizeof(syncResults_) / sizeof(double); i++) - pptr[i] = speed * ptr[i] + (1.0 - speed) * pptr[i]; + for (unsigned int j = 0; j < syncResults_.size(); j++) { + for (unsigned int i = 0; i < syncResults_[j].size(); i++) + prevSyncResults_[j][i] = speed * syncResults_[j][i] + (1.0 - speed) * prevSyncResults_[j][i]; + } /* Put output values into status metadata. */ AlscStatus status; - memcpy(status.r, prevSyncResults_[0], sizeof(status.r)); - memcpy(status.g, prevSyncResults_[1], sizeof(status.g)); - memcpy(status.b, prevSyncResults_[2], sizeof(status.b)); + status.r = prevSyncResults_[0]; + status.g = prevSyncResults_[1]; + status.b = prevSyncResults_[2]; imageMetadata->set("alsc.status", status); } @@ -432,18 +457,17 @@ void Alsc::asyncFunc() } void getCalTable(double ct, std::vector const &calibrations, - double calTable[XY]) + std::vector &calTable) { if (calibrations.empty()) { - for (int i = 0; i < XY; i++) - calTable[i] = 1.0; + std::fill(calTable.begin(), calTable.end(), 1.0); LOG(RPiAlsc, Debug) << "no calibrations found"; } else if (ct <= calibrations.front().ct) { - memcpy(calTable, calibrations.front().table, XY * sizeof(double)); + calTable = calibrations.front().table; LOG(RPiAlsc, Debug) << "using calibration for " << calibrations.front().ct; } else if (ct >= calibrations.back().ct) { - memcpy(calTable, calibrations.back().table, XY * sizeof(double)); + calTable = calibrations.back().table; LOG(RPiAlsc, Debug) << "using calibration for " << calibrations.back().ct; } else { @@ -454,7 +478,7 @@ void getCalTable(double ct, std::vector const &calibrations, LOG(RPiAlsc, Debug) << "ct is " << ct << ", interpolating between " << ct0 << " and " << ct1; - for (int i = 0; i < XY; i++) + for (unsigned int i = 0; i < calTable.size(); i++) calTable[i] = (calibrations[idx].table[i] * (ct1 - ct) + calibrations[idx + 1].table[i] * (ct - ct0)) / @@ -462,9 +486,13 @@ void getCalTable(double ct, std::vector const &calibrations, } } -void resampleCalTable(double const calTableIn[XY], - CameraMode const &cameraMode, double calTableOut[XY]) +void resampleCalTable(const std::vector &calTableIn, + CameraMode const &cameraMode, const Size &size, + std::vector &calTableOut) { + int X = size.width; + int Y = size.height; + /* * Precalculate and cache the x sampling locations and phases to save * recomputing them on every row. @@ -501,23 +529,24 @@ void resampleCalTable(double const calTableIn[XY], yLo = Y - 1 - yLo; yHi = Y - 1 - yHi; } - double const *rowAbove = calTableIn + X * yLo; - double const *rowBelow = calTableIn + X * yHi; + double const *rowAbove = calTableIn.data() + X * yLo; + double const *rowBelow = calTableIn.data() + X * yHi; + double *out = calTableOut.data() + X * j; for (int i = 0; i < X; i++) { double above = rowAbove[xLo[i]] * (1 - xf[i]) + rowAbove[xHi[i]] * xf[i]; double below = rowBelow[xLo[i]] * (1 - xf[i]) + rowBelow[xHi[i]] * xf[i]; - *(calTableOut++) = above * (1 - yf) + below * yf; + *(out++) = above * (1 - yf) + below * yf; } } } /* Calculate chrominance statistics (R/G and B/G) for each region. */ -static void calculateCrCb(const RgbyRegions &awbRegion, double cr[XY], - double cb[XY], uint32_t minCount, uint16_t minG) +static void calculateCrCb(const RgbyRegions &awbRegion, std::vector &cr, + std::vector &cb, uint32_t minCount, uint16_t minG) { - for (int i = 0; i < XY; i++) { + for (unsigned int i = 0; i < cr.size(); i++) { auto s = awbRegion.get(i); if (s.counted <= minCount || s.val.gSum / s.counted <= minG) { @@ -530,33 +559,34 @@ static void calculateCrCb(const RgbyRegions &awbRegion, double cr[XY], } } -static void applyCalTable(double const calTable[XY], double C[XY]) +static void applyCalTable(const std::vector &calTable, std::vector &C) { - for (int i = 0; i < XY; i++) + for (unsigned int i = 0; i < C.size(); i++) if (C[i] != InsufficientData) C[i] *= calTable[i]; } -void compensateLambdasForCal(double const calTable[XY], - double const oldLambdas[XY], - double newLambdas[XY]) +void compensateLambdasForCal(const std::vector &calTable, + const std::vector &oldLambdas, + std::vector &newLambdas) { double minNewLambda = std::numeric_limits::max(); - for (int i = 0; i < XY; i++) { + for (unsigned int i = 0; i < newLambdas.size(); i++) { newLambdas[i] = oldLambdas[i] * calTable[i]; minNewLambda = std::min(minNewLambda, newLambdas[i]); } - for (int i = 0; i < XY; i++) + for (unsigned int i = 0; i < newLambdas.size(); i++) newLambdas[i] /= minNewLambda; } -[[maybe_unused]] static void printCalTable(double const C[XY]) +[[maybe_unused]] static void printCalTable(const std::vector &C, + const Size &size) { printf("table: [\n"); - for (int j = 0; j < Y; j++) { - for (int i = 0; i < X; i++) { - printf("%5.3f", 1.0 / C[j * X + i]); - if (i != X - 1 || j != Y - 1) + for (unsigned int j = 0; j < size.height; j++) { + for (unsigned int i = 0; i < size.width; i++) { + printf("%5.3f", 1.0 / C[j * size.width + i]); + if (i != size.width - 1 || j != size.height - 1) printf(","); } printf("\n"); @@ -577,9 +607,13 @@ static double computeWeight(double Ci, double Cj, double sigma) } /* Compute all weights. */ -static void computeW(double const C[XY], double sigma, double W[XY][4]) +static void computeW(const std::vector &C, double sigma, + std::vector> &W, const Size &size) { - for (int i = 0; i < XY; i++) { + size_t XY = size.width * size.height; + size_t X = size.width; + + for (unsigned int i = 0; i < XY; i++) { /* Start with neighbour above and go clockwise. */ W[i][0] = i >= X ? computeWeight(C[i], C[i - X], sigma) : 0; W[i][1] = i % X < X - 1 ? computeWeight(C[i], C[i + 1], sigma) : 0; @@ -589,11 +623,16 @@ static void computeW(double const C[XY], double sigma, double W[XY][4]) } /* Compute M, the large but sparse matrix such that M * lambdas = 0. */ -static void constructM(double const C[XY], double const W[XY][4], - double M[XY][4]) +static void constructM(const std::vector &C, + const std::vector> &W, + std::vector> &M, + const Size &size) { + size_t XY = size.width * size.height; + size_t X = size.width; + double epsilon = 0.001; - for (int i = 0; i < XY; i++) { + for (unsigned int i = 0; i < XY; i++) { /* * Note how, if C[i] == INSUFFICIENT_DATA, the weights will all * be zero so the equation is still set up correctly. @@ -614,79 +653,80 @@ static void constructM(double const C[XY], double const W[XY][4], * left/right neighbours are zero down the left/right edges, so we don't need * need to test the i value to exclude them. */ -static double computeLambdaBottom(int i, double const M[XY][4], - double lambda[XY]) +static double computeLambdaBottom(int i, const std::vector> &M, + std::vector &lambda, const Size &size) { - return M[i][1] * lambda[i + 1] + M[i][2] * lambda[i + X] + + return M[i][1] * lambda[i + 1] + M[i][2] * lambda[i + size.width] + M[i][3] * lambda[i - 1]; } -static double computeLambdaBottomStart(int i, double const M[XY][4], - double lambda[XY]) +static double computeLambdaBottomStart(int i, const std::vector> &M, + std::vector &lambda, const Size &size) { - return M[i][1] * lambda[i + 1] + M[i][2] * lambda[i + X]; + return M[i][1] * lambda[i + 1] + M[i][2] * lambda[i + size.width]; } -static double computeLambdaInterior(int i, double const M[XY][4], - double lambda[XY]) +static double computeLambdaInterior(int i, const std::vector> &M, + std::vector &lambda, const Size &size) { - return M[i][0] * lambda[i - X] + M[i][1] * lambda[i + 1] + - M[i][2] * lambda[i + X] + M[i][3] * lambda[i - 1]; + return M[i][0] * lambda[i - size.width] + M[i][1] * lambda[i + 1] + + M[i][2] * lambda[i + size.width] + M[i][3] * lambda[i - 1]; } -static double computeLambdaTop(int i, double const M[XY][4], - double lambda[XY]) +static double computeLambdaTop(int i, const std::vector> &M, + std::vector &lambda, const Size &size) { - return M[i][0] * lambda[i - X] + M[i][1] * lambda[i + 1] + + return M[i][0] * lambda[i - size.width] + M[i][1] * lambda[i + 1] + M[i][3] * lambda[i - 1]; } -static double computeLambdaTopEnd(int i, double const M[XY][4], - double lambda[XY]) +static double computeLambdaTopEnd(int i, const std::vector> &M, + std::vector &lambda, const Size &size) { - return M[i][0] * lambda[i - X] + M[i][3] * lambda[i - 1]; + return M[i][0] * lambda[i - size.width] + M[i][3] * lambda[i - 1]; } /* Gauss-Seidel iteration with over-relaxation. */ -static double gaussSeidel2Sor(double const M[XY][4], double omega, - double lambda[XY], double lambdaBound) +static double gaussSeidel2Sor(const std::vector> &M, double omega, + std::vector &lambda, double lambdaBound, + const Size &size) { + int XY = size.width * size.height; + int X = size.width; const double min = 1 - lambdaBound, max = 1 + lambdaBound; - double oldLambda[XY]; + std::vector oldLambda = lambda; int i; - for (i = 0; i < XY; i++) - oldLambda[i] = lambda[i]; - lambda[0] = computeLambdaBottomStart(0, M, lambda); + lambda[0] = computeLambdaBottomStart(0, M, lambda, size); lambda[0] = std::clamp(lambda[0], min, max); for (i = 1; i < X; i++) { - lambda[i] = computeLambdaBottom(i, M, lambda); + lambda[i] = computeLambdaBottom(i, M, lambda, size); lambda[i] = std::clamp(lambda[i], min, max); } for (; i < XY - X; i++) { - lambda[i] = computeLambdaInterior(i, M, lambda); + lambda[i] = computeLambdaInterior(i, M, lambda, size); lambda[i] = std::clamp(lambda[i], min, max); } for (; i < XY - 1; i++) { - lambda[i] = computeLambdaTop(i, M, lambda); + lambda[i] = computeLambdaTop(i, M, lambda, size); lambda[i] = std::clamp(lambda[i], min, max); } - lambda[i] = computeLambdaTopEnd(i, M, lambda); + lambda[i] = computeLambdaTopEnd(i, M, lambda, size); lambda[i] = std::clamp(lambda[i], min, max); /* * Also solve the system from bottom to top, to help spread the updates * better. */ - lambda[i] = computeLambdaTopEnd(i, M, lambda); + lambda[i] = computeLambdaTopEnd(i, M, lambda, size); lambda[i] = std::clamp(lambda[i], min, max); for (i = XY - 2; i >= XY - X; i--) { - lambda[i] = computeLambdaTop(i, M, lambda); + lambda[i] = computeLambdaTop(i, M, lambda, size); lambda[i] = std::clamp(lambda[i], min, max); } for (; i >= X; i--) { - lambda[i] = computeLambdaInterior(i, M, lambda); + lambda[i] = computeLambdaInterior(i, M, lambda, size); lambda[i] = std::clamp(lambda[i], min, max); } for (; i >= 1; i--) { - lambda[i] = computeLambdaBottom(i, M, lambda); + lambda[i] = computeLambdaBottom(i, M, lambda, size); lambda[i] = std::clamp(lambda[i], min, max); } - lambda[0] = computeLambdaBottomStart(0, M, lambda); + lambda[0] = computeLambdaBottomStart(0, M, lambda, size); lambda[0] = std::clamp(lambda[0], min, max); double maxDiff = 0; for (i = 0; i < XY; i++) { @@ -698,33 +738,33 @@ static double gaussSeidel2Sor(double const M[XY][4], double omega, } /* Normalise the values so that the smallest value is 1. */ -static void normalise(double *ptr, size_t n) +static void normalise(std::vector &results) { - double minval = ptr[0]; - for (size_t i = 1; i < n; i++) - minval = std::min(minval, ptr[i]); - for (size_t i = 0; i < n; i++) - ptr[i] /= minval; + double minval = *std::min_element(results.begin(), results.end()); + std::for_each(results.begin(), results.end(), + [minval](double val) { return val / minval; }); } /* Rescale the values so that the average value is 1. */ -static void reaverage(Span data) +static void reaverage(std::vector &data) { double sum = std::accumulate(data.begin(), data.end(), 0.0); double ratio = 1 / (sum / data.size()); - for (double &d : data) - d *= ratio; + std::for_each(data.begin(), data.end(), + [ratio](double val) { return val * ratio; }); } -static void runMatrixIterations(double const C[XY], double lambda[XY], - double const W[XY][4], double omega, - int nIter, double threshold, double lambdaBound) +static void runMatrixIterations(const std::vector &C, + std::vector &lambda, + const std::vector> &W, + std::vector> &M, double omega, + unsigned int nIter, double threshold, double lambdaBound, + const Size &size) { - double M[XY][4]; - constructM(C, W, M); + constructM(C, W, M, size); double lastMaxDiff = std::numeric_limits::max(); - for (int i = 0; i < nIter; i++) { - double maxDiff = fabs(gaussSeidel2Sor(M, omega, lambda, lambdaBound)); + for (unsigned int i = 0; i < nIter; i++) { + double maxDiff = fabs(gaussSeidel2Sor(M, omega, lambda, lambdaBound, size)); if (maxDiff < threshold) { LOG(RPiAlsc, Debug) << "Stop after " << i + 1 << " iterations"; @@ -741,39 +781,44 @@ static void runMatrixIterations(double const C[XY], double lambda[XY], lastMaxDiff = maxDiff; } /* We're going to normalise the lambdas so the total average is 1. */ - reaverage({ lambda, XY }); + reaverage(lambda); } -static void addLuminanceRb(double result[XY], double const lambda[XY], - double const luminanceLut[XY], +static void addLuminanceRb(std::vector &result, const std::vector &lambda, + const std::vector &luminanceLut, double luminanceStrength) { - for (int i = 0; i < XY; i++) + for (unsigned int i = 0; i < result.size(); i++) result[i] = lambda[i] * ((luminanceLut[i] - 1) * luminanceStrength + 1); } -static void addLuminanceG(double result[XY], double lambda, - double const luminanceLut[XY], +static void addLuminanceG(std::vector &result, double lambda, + const std::vector &luminanceLut, double luminanceStrength) { - for (int i = 0; i < XY; i++) + for (unsigned int i = 0; i < result.size(); i++) result[i] = lambda * ((luminanceLut[i] - 1) * luminanceStrength + 1); } -void addLuminanceToTables(double results[3][Y][X], double const lambdaR[XY], - double lambdaG, double const lambdaB[XY], - double const luminanceLut[XY], +void addLuminanceToTables(std::array, 3> &results, + const std::vector &lambdaR, + double lambdaG, const std::vector &lambdaB, + const std::vector &luminanceLut, double luminanceStrength) { - addLuminanceRb((double *)results[0], lambdaR, luminanceLut, luminanceStrength); - addLuminanceG((double *)results[1], lambdaG, luminanceLut, luminanceStrength); - addLuminanceRb((double *)results[2], lambdaB, luminanceLut, luminanceStrength); - normalise((double *)results, 3 * XY); + addLuminanceRb(results[0], lambdaR, luminanceLut, luminanceStrength); + addLuminanceG(results[1], lambdaG, luminanceLut, luminanceStrength); + addLuminanceRb(results[2], lambdaB, luminanceLut, luminanceStrength); + for (auto &r : results) + normalise(r); } void Alsc::doAlsc() { - double cr[XY], cb[XY], wr[XY][4], wb[XY][4], calTableR[XY], calTableB[XY], calTableTmp[XY]; + std::vector &cr = tmpC_[0], &cb = tmpC_[1], &calTableR = tmpC_[2], + &calTableB = tmpC_[3], &calTableTmp = tmpC_[4]; + std::vector> &wr = tmpM_[0], &wb = tmpM_[1], &M = tmpM_[2]; + /* * Calculate our R/B ("Cr"/"Cb") colour statistics, and assess which are * usable. @@ -784,9 +829,9 @@ void Alsc::doAlsc() * case the camera mode is not full-frame. */ getCalTable(ct_, config_.calibrationsCr, calTableTmp); - resampleCalTable(calTableTmp, cameraMode_, calTableR); + resampleCalTable(calTableTmp, cameraMode_, config_.tableSize, calTableR); getCalTable(ct_, config_.calibrationsCb, calTableTmp); - resampleCalTable(calTableTmp, cameraMode_, calTableB); + resampleCalTable(calTableTmp, cameraMode_, config_.tableSize, calTableB); /* * You could print out the cal tables for this image here, if you're * tuning the algorithm... @@ -796,13 +841,13 @@ void Alsc::doAlsc() applyCalTable(calTableR, cr); applyCalTable(calTableB, cb); /* Compute weights between zones. */ - computeW(cr, config_.sigmaCr, wr); - computeW(cb, config_.sigmaCb, wb); + computeW(cr, config_.sigmaCr, wr, config_.tableSize); + computeW(cb, config_.sigmaCb, wb, config_.tableSize); /* Run Gauss-Seidel iterations over the resulting matrix, for R and B. */ - runMatrixIterations(cr, lambdaR_, wr, config_.omega, config_.nIter, - config_.threshold, config_.lambdaBound); - runMatrixIterations(cb, lambdaB_, wb, config_.omega, config_.nIter, - config_.threshold, config_.lambdaBound); + runMatrixIterations(cr, lambdaR_, wr, M, config_.omega, config_.nIter, + config_.threshold, config_.lambdaBound, config_.tableSize); + runMatrixIterations(cb, lambdaB_, wb, M, config_.omega, config_.nIter, + config_.threshold, config_.lambdaBound, config_.tableSize); /* * Fold the calibrated gains into our final lambda values. (Note that on * the next run, we re-start with the lambda values that don't have the diff --git a/src/ipa/raspberrypi/controller/rpi/alsc.h b/src/ipa/raspberrypi/controller/rpi/alsc.h index 9167c9ffa2e3..85e998db40e9 100644 --- a/src/ipa/raspberrypi/controller/rpi/alsc.h +++ b/src/ipa/raspberrypi/controller/rpi/alsc.h @@ -6,9 +6,13 @@ */ #pragma once +#include #include #include #include +#include + +#include #include "../algorithm.h" #include "../alsc_status.h" @@ -20,7 +24,7 @@ namespace RPiController { struct AlscCalibration { double ct; - double table[AlscCellsX * AlscCellsY]; + std::vector table; }; struct AlscConfig { @@ -36,13 +40,14 @@ struct AlscConfig { uint16_t minG; double omega; uint32_t nIter; - double luminanceLut[AlscCellsX * AlscCellsY]; + std::vector luminanceLut; double luminanceStrength; std::vector calibrationsCr; std::vector calibrationsCb; double defaultCt; /* colour temperature if no metadata found */ double threshold; /* iteration termination threshold */ double lambdaBound; /* upper/lower bound for lambda from a value of 1 */ + libcamera::Size tableSize; }; class Alsc : public Algorithm @@ -62,7 +67,7 @@ private: AlscConfig config_; bool firstTime_; CameraMode cameraMode_; - double luminanceTable_[AlscCellsX * AlscCellsY]; + std::vector luminanceTable_; std::thread asyncThread_; void asyncFunc(); /* asynchronous thread function */ std::mutex mutex_; @@ -88,8 +93,8 @@ private: int frameCount_; /* counts up to startupFrames for Process function */ int frameCount2_; - double syncResults_[3][AlscCellsY][AlscCellsX]; - double prevSyncResults_[3][AlscCellsY][AlscCellsX]; + std::array, 3> syncResults_; + std::array, 3> prevSyncResults_; void waitForAysncThread(); /* * The following are for the asynchronous thread to use, though the main @@ -100,12 +105,16 @@ private: void fetchAsyncResults(); double ct_; RgbyRegions statistics_; - double asyncResults_[3][AlscCellsY][AlscCellsX]; - double asyncLambdaR_[AlscCellsX * AlscCellsY]; - double asyncLambdaB_[AlscCellsX * AlscCellsY]; + std::array, 3> asyncResults_; + std::vector asyncLambdaR_; + std::vector asyncLambdaB_; void doAlsc(); - double lambdaR_[AlscCellsX * AlscCellsY]; - double lambdaB_[AlscCellsX * AlscCellsY]; + std::vector lambdaR_; + std::vector lambdaB_; + + /* Temporaries for the computations */ + std::array, 5> tmpC_; + std::array>, 3> tmpM_; }; } /* namespace RPiController */ diff --git a/src/ipa/raspberrypi/raspberrypi.cpp b/src/ipa/raspberrypi/raspberrypi.cpp index e24529920a4b..c74558f1dc0c 100644 --- a/src/ipa/raspberrypi/raspberrypi.cpp +++ b/src/ipa/raspberrypi/raspberrypi.cpp @@ -14,6 +14,7 @@ #include #include #include +#include #include @@ -174,7 +175,7 @@ private: void applyDPC(const struct DpcStatus *dpcStatus, ControlList &ctrls); void applyLS(const struct AlscStatus *lsStatus, ControlList &ctrls); void applyAF(const struct AfStatus *afStatus, ControlList &lensCtrls); - void resampleTable(uint16_t dest[], double const src[12][16], int destW, int destH); + void resampleTable(uint16_t dest[], const std::vector &src, int destW, int destH); std::map buffers_; @@ -1769,7 +1770,7 @@ void IPARPi::applyAF(const struct AfStatus *afStatus, ControlList &lensCtrls) * Resamples a 16x12 table with central sampling to destW x destH with corner * sampling. */ -void IPARPi::resampleTable(uint16_t dest[], double const src[12][16], +void IPARPi::resampleTable(uint16_t dest[], const std::vector &src, int destW, int destH) { /* @@ -1794,8 +1795,8 @@ void IPARPi::resampleTable(uint16_t dest[], double const src[12][16], double yf = y - yLo; int yHi = yLo < 11 ? yLo + 1 : 11; yLo = yLo > 0 ? yLo : 0; - double const *rowAbove = src[yLo]; - double const *rowBelow = src[yHi]; + double const *rowAbove = src.data() + yLo * 16; + double const *rowBelow = src.data() + yHi * 16; for (int i = 0; i < destW; i++) { double above = rowAbove[xLo[i]] * (1 - xf[i]) + rowAbove[xHi[i]] * xf[i]; double below = rowBelow[xLo[i]] * (1 - xf[i]) + rowBelow[xHi[i]] * xf[i]; From patchwork Fri Mar 31 07:23:35 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Naushir Patuck X-Patchwork-Id: 18500 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 1D8E7C329C for ; Fri, 31 Mar 2023 07:40:55 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id C41766276C; Fri, 31 Mar 2023 09:40:54 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1680248454; bh=Xp3l2Cn+AaoAMI7S8X+v1R7g9YRdikS8Td9pAFvi98M=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To:Cc: From; b=A6KNSWUWd+342GudSnFgp6KNjtQ4FtBhKqnPs6A+RPssC2kINugAuzuLn+9rhXImO Q8OEYBf7lM50SwBchSoCPUs0mIHqk0jWKTCvReYXpusKS9aCfl8DrQrqPLDLnS2Dx6 Q6OS0fu8Ex95+q9eOfQyZOcxeEteckWjCyQCvZw0mZPoIKugLwDhtX8Ac0GtjyORCW CsqEtMtQJuip54jsEF+UJIYhZx4iXIos9jPRMoOIEotOb6U5/q5xK1aktnOTiVcT4S EmMMQgxmQaRD4+6eFnCfYiOeHIpNOhU8sPGaFewD+0DcRToX1XdyhdfsNK61F5ZKPk wMwPWX7MA0F1A== Received: from mail-il1-x129.google.com (mail-il1-x129.google.com [IPv6:2607:f8b0:4864:20::129]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id DE8136275E for ; Fri, 31 Mar 2023 09:40:52 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="M93eLi1t"; dkim-atps=neutral Received: by mail-il1-x129.google.com with SMTP id h14so9764537ilj.0 for ; Fri, 31 Mar 2023 00:40:52 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; t=1680248451; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=3JdbajTs00HICJy9McNfdYmKn2OuE3fq0x4d14FeTTQ=; b=M93eLi1t3ntW300+lJEZmf6+nRofYkx0dkrEUn32GA7NEFO3w81d5nn8ebMF2L1hgM HoXHhwA9Y3lbdQ6Z9IhO1JWo3GMjd+8P26PH6tLeSKeFKTW9XBwsIV6Ae1yr0+H8s0az wFzVo04nVfIwt6/UKTRRs5KK2vAnfCpcxSa6zDfgBKgZt7n5DCAcjj6omOT374J+m+aG Kiv6UO8kN9i/q1LdYFjJJLGOzs+JqoPhZDBqnxDTHknIpTJ3Pq6VG04QhOlkRvGvYfoi ujKROhVE6ZKiDm2tMYkyBnm7R2Z4xJIbeIgDPSzPhokcEKJiudRqTS2kD8v6/hBxZ8OD TuXg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; t=1680248451; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=3JdbajTs00HICJy9McNfdYmKn2OuE3fq0x4d14FeTTQ=; b=yf7+TbAOEkGFfkxUZSPuANfDkUDwZjZ33BjHGA+rgENV5tp+TbZ/arBZ+ODL+wlDU7 JybUb2SbG//slXMharyFWNTx1SlkJqIIIomFIRqJ9KweRaqZ/Yzw+vpUxAZyAHHCFvpM TquuYj7dUzxxNoIroKdn6gkv6r0Wtc9Pmgs8sZHhK3YeTj/KmxEQKH/DDw1iTtsK1pJE Q5wyFHpfSZMKGhClGjftYkluk5Hjjqcv5IWpadvx2s8S3DuKPyOXAoWXXuXJRwqqjR1s y9rWIImGuoBQXLvIfpLunuXhOQhFwdww1Cjl9JwHIaJxMqq5hWoqjmpXCeDR262gcmvU TrfQ== X-Gm-Message-State: AAQBX9drT4rgpQiHbEhpMBAIle3FSEpMkMxX2S6QJ4ZIDkr4aLo1memp ziVUVEAlWAxXccgWN8AKyLH72GNTFzXz/EGx45SdYA== X-Google-Smtp-Source: AKy350ZNJEHrk/cb6K+DPPPhnpH4Qi8rH6848a7gGlEN4rcnW5MZ8fbewtl9fSR1RurzBfO+cad3uA== X-Received: by 2002:a92:c80c:0:b0:325:c3e9:f8b5 with SMTP id v12-20020a92c80c000000b00325c3e9f8b5mr19344919iln.12.1680248451035; Fri, 31 Mar 2023 00:40:51 -0700 (PDT) Received: from localhost.localdomain ([93.93.133.154]) by smtp.gmail.com with ESMTPSA id i17-20020a6bf411000000b0074555814e73sm421943iog.32.2023.03.31.00.40.49 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 31 Mar 2023 00:40:50 -0700 (PDT) To: libcamera-devel@lists.libcamera.org Date: Fri, 31 Mar 2023 08:23:35 +0100 Message-Id: <20230331072341.24851-5-naush@raspberrypi.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230331072341.24851-1-naush@raspberrypi.com> References: <20230331072341.24851-1-naush@raspberrypi.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v3 04/10] ipa: raspberrypi: alsc: Replace std::vectors by Array2D class X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Naushir Patuck via libcamera-devel From: Naushir Patuck Reply-To: Naushir Patuck Cc: Jacopo Mondi Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" From: David Plowman The Array2D class is a very thin wrapper round std::vector that can be used almost identically in the code, but it carries its 2D size with it so that we aren't passing it around all the time. All the std::vectors that were X * Y in size (X and Y being the ALSC grid size) have been replaced. The sparse matrices that are XY * 4 in size have not been as they are somewhat different, are used differently, require more code changes, and actually make things more confusing if everything looks like an Array2D but are not the same. There should be no change in algorithm behaviour at all. Signed-off-by: David Plowman Reviewed-by: Naushir Patuck Reviewed-by: Jacopo Mondi --- src/ipa/raspberrypi/controller/rpi/alsc.cpp | 218 ++++++++++---------- src/ipa/raspberrypi/controller/rpi/alsc.h | 68 +++++- 2 files changed, 164 insertions(+), 122 deletions(-) diff --git a/src/ipa/raspberrypi/controller/rpi/alsc.cpp b/src/ipa/raspberrypi/controller/rpi/alsc.cpp index 51fe5d73f52d..524c48093590 100644 --- a/src/ipa/raspberrypi/controller/rpi/alsc.cpp +++ b/src/ipa/raspberrypi/controller/rpi/alsc.cpp @@ -49,11 +49,10 @@ char const *Alsc::name() const return NAME; } -static int generateLut(std::vector &lut, const libcamera::YamlObject ¶ms, - const Size &size) +static int generateLut(Array2D &lut, const libcamera::YamlObject ¶ms) { /* These must be signed ints for the co-ordinate calculations below. */ - int X = size.width, Y = size.height; + int X = lut.dimensions().width, Y = lut.dimensions().height; double cstrength = params["corner_strength"].get(2.0); if (cstrength <= 1.0) { LOG(RPiAlsc, Error) << "corner_strength must be > 1.0"; @@ -82,9 +81,9 @@ static int generateLut(std::vector &lut, const libcamera::YamlObject &pa return 0; } -static int readLut(std::vector &lut, const libcamera::YamlObject ¶ms, const Size &size) +static int readLut(Array2D &lut, const libcamera::YamlObject ¶ms) { - if (params.size() != size.width * size.height) { + if (params.size() != lut.size()) { LOG(RPiAlsc, Error) << "Invalid number of entries in LSC table"; return -EINVAL; } @@ -128,7 +127,7 @@ static int readCalibrations(std::vector &calibrations, } int num = 0; - calibration.table.resize(size.width * size.height); + calibration.table.resize(size); for (const auto &elem : table.asList()) { value = elem.get(); if (!value) @@ -160,13 +159,13 @@ int Alsc::read(const libcamera::YamlObject ¶ms) config_.luminanceStrength = params["luminance_strength"].get(1.0); - config_.luminanceLut.resize(config_.tableSize.width * config_.tableSize.height, 1.0); + config_.luminanceLut.resize(config_.tableSize, 1.0); int ret = 0; if (params.contains("corner_strength")) - ret = generateLut(config_.luminanceLut, params, config_.tableSize); + ret = generateLut(config_.luminanceLut, params); else if (params.contains("luminance_lut")) - ret = readLut(config_.luminanceLut, params["luminance_lut"], config_.tableSize); + ret = readLut(config_.luminanceLut, params["luminance_lut"]); else LOG(RPiAlsc, Warning) << "no luminance table - assume unity everywhere"; @@ -191,16 +190,16 @@ int Alsc::read(const libcamera::YamlObject ¶ms) static double getCt(Metadata *metadata, double defaultCt); static void getCalTable(double ct, std::vector const &calibrations, - std::vector &calTable); -static void resampleCalTable(const std::vector &calTableIn, CameraMode const &cameraMode, - const Size &size, std::vector &calTableOut); -static void compensateLambdasForCal(const std::vector &calTable, - const std::vector &oldLambdas, - std::vector &newLambdas); -static void addLuminanceToTables(std::array, 3> &results, - const std::vector &lambdaR, double lambdaG, - const std::vector &lambdaB, - const std::vector &luminanceLut, + Array2D &calTable); +static void resampleCalTable(const Array2D &calTableIn, CameraMode const &cameraMode, + Array2D &calTableOut); +static void compensateLambdasForCal(const Array2D &calTable, + const Array2D &oldLambdas, + Array2D &newLambdas); +static void addLuminanceToTables(std::array, 3> &results, + const Array2D &lambdaR, double lambdaG, + const Array2D &lambdaB, + const Array2D &luminanceLut, double luminanceStrength); void Alsc::initialise() @@ -212,22 +211,22 @@ void Alsc::initialise() const size_t XY = config_.tableSize.width * config_.tableSize.height; for (auto &r : syncResults_) - r.resize(XY); + r.resize(config_.tableSize); for (auto &r : prevSyncResults_) - r.resize(XY); + r.resize(config_.tableSize); for (auto &r : asyncResults_) - r.resize(XY); + r.resize(config_.tableSize); - luminanceTable_.resize(XY); - asyncLambdaR_.resize(XY); - asyncLambdaB_.resize(XY); + luminanceTable_.resize(config_.tableSize); + asyncLambdaR_.resize(config_.tableSize); + asyncLambdaB_.resize(config_.tableSize); /* The lambdas are initialised in the SwitchMode. */ - lambdaR_.resize(XY); - lambdaB_.resize(XY); + lambdaR_.resize(config_.tableSize); + lambdaB_.resize(config_.tableSize); /* Temporaries for the computations, but sensible to allocate this up-front! */ for (auto &c : tmpC_) - c.resize(XY); + c.resize(config_.tableSize); for (auto &m : tmpM_) m.resize(XY); } @@ -290,7 +289,7 @@ void Alsc::switchMode(CameraMode const &cameraMode, * We must resample the luminance table like we do the others, but it's * fixed so we can simply do it up front here. */ - resampleCalTable(config_.luminanceLut, cameraMode_, config_.tableSize, luminanceTable_); + resampleCalTable(config_.luminanceLut, cameraMode_, luminanceTable_); if (resetTables) { /* @@ -302,11 +301,11 @@ void Alsc::switchMode(CameraMode const &cameraMode, */ std::fill(lambdaR_.begin(), lambdaR_.end(), 1.0); std::fill(lambdaB_.begin(), lambdaB_.end(), 1.0); - std::vector &calTableR = tmpC_[0], &calTableB = tmpC_[1], &calTableTmp = tmpC_[2]; + Array2D &calTableR = tmpC_[0], &calTableB = tmpC_[1], &calTableTmp = tmpC_[2]; getCalTable(ct_, config_.calibrationsCr, calTableTmp); - resampleCalTable(calTableTmp, cameraMode_, config_.tableSize, calTableR); + resampleCalTable(calTableTmp, cameraMode_, calTableR); getCalTable(ct_, config_.calibrationsCb, calTableTmp); - resampleCalTable(calTableTmp, cameraMode_, config_.tableSize, calTableB); + resampleCalTable(calTableTmp, cameraMode_, calTableB); compensateLambdasForCal(calTableR, lambdaR_, asyncLambdaR_); compensateLambdasForCal(calTableB, lambdaB_, asyncLambdaB_); addLuminanceToTables(syncResults_, asyncLambdaR_, 1.0, asyncLambdaB_, @@ -411,9 +410,9 @@ void Alsc::prepare(Metadata *imageMetadata) } /* Put output values into status metadata. */ AlscStatus status; - status.r = prevSyncResults_[0]; - status.g = prevSyncResults_[1]; - status.b = prevSyncResults_[2]; + status.r = prevSyncResults_[0].data(); + status.g = prevSyncResults_[1].data(); + status.b = prevSyncResults_[2].data(); imageMetadata->set("alsc.status", status); } @@ -457,7 +456,7 @@ void Alsc::asyncFunc() } void getCalTable(double ct, std::vector const &calibrations, - std::vector &calTable) + Array2D &calTable) { if (calibrations.empty()) { std::fill(calTable.begin(), calTable.end(), 1.0); @@ -486,12 +485,12 @@ void getCalTable(double ct, std::vector const &calibrations, } } -void resampleCalTable(const std::vector &calTableIn, - CameraMode const &cameraMode, const Size &size, - std::vector &calTableOut) +void resampleCalTable(const Array2D &calTableIn, + CameraMode const &cameraMode, + Array2D &calTableOut) { - int X = size.width; - int Y = size.height; + int X = calTableIn.dimensions().width; + int Y = calTableIn.dimensions().height; /* * Precalculate and cache the x sampling locations and phases to save @@ -529,9 +528,9 @@ void resampleCalTable(const std::vector &calTableIn, yLo = Y - 1 - yLo; yHi = Y - 1 - yHi; } - double const *rowAbove = calTableIn.data() + X * yLo; - double const *rowBelow = calTableIn.data() + X * yHi; - double *out = calTableOut.data() + X * j; + double const *rowAbove = calTableIn.ptr() + X * yLo; + double const *rowBelow = calTableIn.ptr() + X * yHi; + double *out = calTableOut.ptr() + X * j; for (int i = 0; i < X; i++) { double above = rowAbove[xLo[i]] * (1 - xf[i]) + rowAbove[xHi[i]] * xf[i]; @@ -543,8 +542,8 @@ void resampleCalTable(const std::vector &calTableIn, } /* Calculate chrominance statistics (R/G and B/G) for each region. */ -static void calculateCrCb(const RgbyRegions &awbRegion, std::vector &cr, - std::vector &cb, uint32_t minCount, uint16_t minG) +static void calculateCrCb(const RgbyRegions &awbRegion, Array2D &cr, + Array2D &cb, uint32_t minCount, uint16_t minG) { for (unsigned int i = 0; i < cr.size(); i++) { auto s = awbRegion.get(i); @@ -559,16 +558,16 @@ static void calculateCrCb(const RgbyRegions &awbRegion, std::vector &cr, } } -static void applyCalTable(const std::vector &calTable, std::vector &C) +static void applyCalTable(const Array2D &calTable, Array2D &C) { for (unsigned int i = 0; i < C.size(); i++) if (C[i] != InsufficientData) C[i] *= calTable[i]; } -void compensateLambdasForCal(const std::vector &calTable, - const std::vector &oldLambdas, - std::vector &newLambdas) +void compensateLambdasForCal(const Array2D &calTable, + const Array2D &oldLambdas, + Array2D &newLambdas) { double minNewLambda = std::numeric_limits::max(); for (unsigned int i = 0; i < newLambdas.size(); i++) { @@ -579,9 +578,9 @@ void compensateLambdasForCal(const std::vector &calTable, newLambdas[i] /= minNewLambda; } -[[maybe_unused]] static void printCalTable(const std::vector &C, - const Size &size) +[[maybe_unused]] static void printCalTable(const Array2D &C) { + const Size &size = C.dimensions(); printf("table: [\n"); for (unsigned int j = 0; j < size.height; j++) { for (unsigned int i = 0; i < size.width; i++) { @@ -607,11 +606,11 @@ static double computeWeight(double Ci, double Cj, double sigma) } /* Compute all weights. */ -static void computeW(const std::vector &C, double sigma, - std::vector> &W, const Size &size) +static void computeW(const Array2D &C, double sigma, + std::vector> &W) { - size_t XY = size.width * size.height; - size_t X = size.width; + size_t XY = C.size(); + size_t X = C.dimensions().width; for (unsigned int i = 0; i < XY; i++) { /* Start with neighbour above and go clockwise. */ @@ -623,13 +622,12 @@ static void computeW(const std::vector &C, double sigma, } /* Compute M, the large but sparse matrix such that M * lambdas = 0. */ -static void constructM(const std::vector &C, +static void constructM(const Array2D &C, const std::vector> &W, - std::vector> &M, - const Size &size) + std::vector> &M) { - size_t XY = size.width * size.height; - size_t X = size.width; + size_t XY = C.size(); + size_t X = C.dimensions().width; double epsilon = 0.001; for (unsigned int i = 0; i < XY; i++) { @@ -654,79 +652,78 @@ static void constructM(const std::vector &C, * need to test the i value to exclude them. */ static double computeLambdaBottom(int i, const std::vector> &M, - std::vector &lambda, const Size &size) + Array2D &lambda) { - return M[i][1] * lambda[i + 1] + M[i][2] * lambda[i + size.width] + + return M[i][1] * lambda[i + 1] + M[i][2] * lambda[i + lambda.dimensions().width] + M[i][3] * lambda[i - 1]; } static double computeLambdaBottomStart(int i, const std::vector> &M, - std::vector &lambda, const Size &size) + Array2D &lambda) { - return M[i][1] * lambda[i + 1] + M[i][2] * lambda[i + size.width]; + return M[i][1] * lambda[i + 1] + M[i][2] * lambda[i + lambda.dimensions().width]; } static double computeLambdaInterior(int i, const std::vector> &M, - std::vector &lambda, const Size &size) + Array2D &lambda) { - return M[i][0] * lambda[i - size.width] + M[i][1] * lambda[i + 1] + - M[i][2] * lambda[i + size.width] + M[i][3] * lambda[i - 1]; + return M[i][0] * lambda[i - lambda.dimensions().width] + M[i][1] * lambda[i + 1] + + M[i][2] * lambda[i + lambda.dimensions().width] + M[i][3] * lambda[i - 1]; } static double computeLambdaTop(int i, const std::vector> &M, - std::vector &lambda, const Size &size) + Array2D &lambda) { - return M[i][0] * lambda[i - size.width] + M[i][1] * lambda[i + 1] + + return M[i][0] * lambda[i - lambda.dimensions().width] + M[i][1] * lambda[i + 1] + M[i][3] * lambda[i - 1]; } static double computeLambdaTopEnd(int i, const std::vector> &M, - std::vector &lambda, const Size &size) + Array2D &lambda) { - return M[i][0] * lambda[i - size.width] + M[i][3] * lambda[i - 1]; + return M[i][0] * lambda[i - lambda.dimensions().width] + M[i][3] * lambda[i - 1]; } /* Gauss-Seidel iteration with over-relaxation. */ static double gaussSeidel2Sor(const std::vector> &M, double omega, - std::vector &lambda, double lambdaBound, - const Size &size) + Array2D &lambda, double lambdaBound) { - int XY = size.width * size.height; - int X = size.width; + int XY = lambda.size(); + int X = lambda.dimensions().width; const double min = 1 - lambdaBound, max = 1 + lambdaBound; - std::vector oldLambda = lambda; + Array2D oldLambda = lambda; int i; - lambda[0] = computeLambdaBottomStart(0, M, lambda, size); + lambda[0] = computeLambdaBottomStart(0, M, lambda); lambda[0] = std::clamp(lambda[0], min, max); for (i = 1; i < X; i++) { - lambda[i] = computeLambdaBottom(i, M, lambda, size); + lambda[i] = computeLambdaBottom(i, M, lambda); lambda[i] = std::clamp(lambda[i], min, max); } for (; i < XY - X; i++) { - lambda[i] = computeLambdaInterior(i, M, lambda, size); + lambda[i] = computeLambdaInterior(i, M, lambda); lambda[i] = std::clamp(lambda[i], min, max); } for (; i < XY - 1; i++) { - lambda[i] = computeLambdaTop(i, M, lambda, size); + lambda[i] = computeLambdaTop(i, M, lambda); lambda[i] = std::clamp(lambda[i], min, max); } - lambda[i] = computeLambdaTopEnd(i, M, lambda, size); + lambda[i] = computeLambdaTopEnd(i, M, lambda); lambda[i] = std::clamp(lambda[i], min, max); /* * Also solve the system from bottom to top, to help spread the updates * better. */ - lambda[i] = computeLambdaTopEnd(i, M, lambda, size); + lambda[i] = computeLambdaTopEnd(i, M, lambda); lambda[i] = std::clamp(lambda[i], min, max); for (i = XY - 2; i >= XY - X; i--) { - lambda[i] = computeLambdaTop(i, M, lambda, size); + lambda[i] = computeLambdaTop(i, M, lambda); lambda[i] = std::clamp(lambda[i], min, max); } for (; i >= X; i--) { - lambda[i] = computeLambdaInterior(i, M, lambda, size); + lambda[i] = computeLambdaInterior(i, M, lambda); lambda[i] = std::clamp(lambda[i], min, max); } for (; i >= 1; i--) { - lambda[i] = computeLambdaBottom(i, M, lambda, size); + lambda[i] = computeLambdaBottom(i, M, lambda); lambda[i] = std::clamp(lambda[i], min, max); } - lambda[0] = computeLambdaBottomStart(0, M, lambda, size); + lambda[0] = computeLambdaBottomStart(0, M, lambda); lambda[0] = std::clamp(lambda[0], min, max); double maxDiff = 0; for (i = 0; i < XY; i++) { @@ -738,7 +735,7 @@ static double gaussSeidel2Sor(const std::vector> &M, doubl } /* Normalise the values so that the smallest value is 1. */ -static void normalise(std::vector &results) +static void normalise(Array2D &results) { double minval = *std::min_element(results.begin(), results.end()); std::for_each(results.begin(), results.end(), @@ -746,7 +743,7 @@ static void normalise(std::vector &results) } /* Rescale the values so that the average value is 1. */ -static void reaverage(std::vector &data) +static void reaverage(Array2D &data) { double sum = std::accumulate(data.begin(), data.end(), 0.0); double ratio = 1 / (sum / data.size()); @@ -754,17 +751,16 @@ static void reaverage(std::vector &data) [ratio](double val) { return val * ratio; }); } -static void runMatrixIterations(const std::vector &C, - std::vector &lambda, +static void runMatrixIterations(const Array2D &C, + Array2D &lambda, const std::vector> &W, std::vector> &M, double omega, - unsigned int nIter, double threshold, double lambdaBound, - const Size &size) + unsigned int nIter, double threshold, double lambdaBound) { - constructM(C, W, M, size); + constructM(C, W, M); double lastMaxDiff = std::numeric_limits::max(); for (unsigned int i = 0; i < nIter; i++) { - double maxDiff = fabs(gaussSeidel2Sor(M, omega, lambda, lambdaBound, size)); + double maxDiff = fabs(gaussSeidel2Sor(M, omega, lambda, lambdaBound)); if (maxDiff < threshold) { LOG(RPiAlsc, Debug) << "Stop after " << i + 1 << " iterations"; @@ -784,26 +780,26 @@ static void runMatrixIterations(const std::vector &C, reaverage(lambda); } -static void addLuminanceRb(std::vector &result, const std::vector &lambda, - const std::vector &luminanceLut, +static void addLuminanceRb(Array2D &result, const Array2D &lambda, + const Array2D &luminanceLut, double luminanceStrength) { for (unsigned int i = 0; i < result.size(); i++) result[i] = lambda[i] * ((luminanceLut[i] - 1) * luminanceStrength + 1); } -static void addLuminanceG(std::vector &result, double lambda, - const std::vector &luminanceLut, +static void addLuminanceG(Array2D &result, double lambda, + const Array2D &luminanceLut, double luminanceStrength) { for (unsigned int i = 0; i < result.size(); i++) result[i] = lambda * ((luminanceLut[i] - 1) * luminanceStrength + 1); } -void addLuminanceToTables(std::array, 3> &results, - const std::vector &lambdaR, - double lambdaG, const std::vector &lambdaB, - const std::vector &luminanceLut, +void addLuminanceToTables(std::array, 3> &results, + const Array2D &lambdaR, + double lambdaG, const Array2D &lambdaB, + const Array2D &luminanceLut, double luminanceStrength) { addLuminanceRb(results[0], lambdaR, luminanceLut, luminanceStrength); @@ -815,8 +811,8 @@ void addLuminanceToTables(std::array, 3> &results, void Alsc::doAlsc() { - std::vector &cr = tmpC_[0], &cb = tmpC_[1], &calTableR = tmpC_[2], - &calTableB = tmpC_[3], &calTableTmp = tmpC_[4]; + Array2D &cr = tmpC_[0], &cb = tmpC_[1], &calTableR = tmpC_[2], + &calTableB = tmpC_[3], &calTableTmp = tmpC_[4]; std::vector> &wr = tmpM_[0], &wb = tmpM_[1], &M = tmpM_[2]; /* @@ -829,9 +825,9 @@ void Alsc::doAlsc() * case the camera mode is not full-frame. */ getCalTable(ct_, config_.calibrationsCr, calTableTmp); - resampleCalTable(calTableTmp, cameraMode_, config_.tableSize, calTableR); + resampleCalTable(calTableTmp, cameraMode_, calTableR); getCalTable(ct_, config_.calibrationsCb, calTableTmp); - resampleCalTable(calTableTmp, cameraMode_, config_.tableSize, calTableB); + resampleCalTable(calTableTmp, cameraMode_, calTableB); /* * You could print out the cal tables for this image here, if you're * tuning the algorithm... @@ -841,13 +837,13 @@ void Alsc::doAlsc() applyCalTable(calTableR, cr); applyCalTable(calTableB, cb); /* Compute weights between zones. */ - computeW(cr, config_.sigmaCr, wr, config_.tableSize); - computeW(cb, config_.sigmaCb, wb, config_.tableSize); + computeW(cr, config_.sigmaCr, wr); + computeW(cb, config_.sigmaCb, wb); /* Run Gauss-Seidel iterations over the resulting matrix, for R and B. */ runMatrixIterations(cr, lambdaR_, wr, M, config_.omega, config_.nIter, - config_.threshold, config_.lambdaBound, config_.tableSize); + config_.threshold, config_.lambdaBound); runMatrixIterations(cb, lambdaB_, wb, M, config_.omega, config_.nIter, - config_.threshold, config_.lambdaBound, config_.tableSize); + config_.threshold, config_.lambdaBound); /* * Fold the calibrated gains into our final lambda values. (Note that on * the next run, we re-start with the lambda values that don't have the diff --git a/src/ipa/raspberrypi/controller/rpi/alsc.h b/src/ipa/raspberrypi/controller/rpi/alsc.h index 85e998db40e9..1ab61299c4cd 100644 --- a/src/ipa/raspberrypi/controller/rpi/alsc.h +++ b/src/ipa/raspberrypi/controller/rpi/alsc.h @@ -22,9 +22,55 @@ namespace RPiController { /* Algorithm to generate automagic LSC (Lens Shading Correction) tables. */ +/* + * The Array2D class is a very thin wrapper round std::vector so that it can + * be used in exactly the same way in the code but carries its correct width + * and height ("dimensions") with it. + */ + +template +class Array2D +{ +public: + using Size = libcamera::Size; + + const Size &dimensions() const { return dimensions_; } + + size_t size() const { return data_.size(); } + + const std::vector &data() const { return data_; } + + void resize(const Size &dims) + { + dimensions_ = dims; + data_.resize(dims.width * dims.height); + } + + void resize(const Size &dims, const T &value) + { + resize(dims); + std::fill(data_.begin(), data_.end(), value); + } + + T &operator[](int index) { return data_[index]; } + + const T &operator[](int index) const { return data_[index]; } + + T *ptr() { return data_.data(); } + + const T *ptr() const { return data_.data(); } + + auto begin() { return data_.begin(); } + auto end() { return data_.end(); } + +private: + Size dimensions_; + std::vector data_; +}; + struct AlscCalibration { double ct; - std::vector table; + Array2D table; }; struct AlscConfig { @@ -40,7 +86,7 @@ struct AlscConfig { uint16_t minG; double omega; uint32_t nIter; - std::vector luminanceLut; + Array2D luminanceLut; double luminanceStrength; std::vector calibrationsCr; std::vector calibrationsCb; @@ -67,7 +113,7 @@ private: AlscConfig config_; bool firstTime_; CameraMode cameraMode_; - std::vector luminanceTable_; + Array2D luminanceTable_; std::thread asyncThread_; void asyncFunc(); /* asynchronous thread function */ std::mutex mutex_; @@ -93,8 +139,8 @@ private: int frameCount_; /* counts up to startupFrames for Process function */ int frameCount2_; - std::array, 3> syncResults_; - std::array, 3> prevSyncResults_; + std::array, 3> syncResults_; + std::array, 3> prevSyncResults_; void waitForAysncThread(); /* * The following are for the asynchronous thread to use, though the main @@ -105,15 +151,15 @@ private: void fetchAsyncResults(); double ct_; RgbyRegions statistics_; - std::array, 3> asyncResults_; - std::vector asyncLambdaR_; - std::vector asyncLambdaB_; + std::array, 3> asyncResults_; + Array2D asyncLambdaR_; + Array2D asyncLambdaB_; void doAlsc(); - std::vector lambdaR_; - std::vector lambdaB_; + Array2D lambdaR_; + Array2D lambdaB_; /* Temporaries for the computations */ - std::array, 5> tmpC_; + std::array, 5> tmpC_; std::array>, 3> tmpM_; }; From patchwork Fri Mar 31 07:23:36 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Naushir Patuck X-Patchwork-Id: 18501 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id A5256C329E for ; Fri, 31 Mar 2023 07:40:55 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 57ACC6278A; Fri, 31 Mar 2023 09:40:55 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1680248455; bh=klPCji71t9ff50/NTw25Wr7qoGXqjM2oZwi1ptHCVKI=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To:Cc: From; b=TK3iyqJnQ6e9qTJDUAnf1KQSJEU7mN2uiLVayevDURw/SbBZzqTpO+6CLF4QX8lwx 9awhTfkUXcuTAsKT5sANK7JEpENtzIYNbwGCDy/0OFUgNrd7SLIGx+Zq2MRIGf2xR6 zpm8G605Uieyi9IArpsuZ5fbkhTF6YN8+4Eq5cVxwlf1dPReigdrMtOPzKB2tbvGap 6vdkOSj8pjc8YBRERexF/LxvcGOaVvLjmZvIAAWI8IMdI227qVf2okc2c/TcI0pKUt 76eYfCz18glD9y60/0EKseYa5jWjxtVgdWS6K2Sddt+nW780iEPQN0r6qcEp3/1Cqx qGdSjq5NEzIhg== Received: from mail-io1-xd2c.google.com (mail-io1-xd2c.google.com [IPv6:2607:f8b0:4864:20::d2c]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 109C762754 for ; Fri, 31 Mar 2023 09:40:53 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="M7OOqDGK"; dkim-atps=neutral Received: by mail-io1-xd2c.google.com with SMTP id h187so7668305iof.7 for ; Fri, 31 Mar 2023 00:40:53 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; t=1680248452; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=548zcSLJYSR6puTSQTcbbtm0W6Y5GofHbCgq+Ac6wj0=; b=M7OOqDGKYqMQFtZEiafYztQE9d0F+GX8c3YlHV4+G3ApUf3uFJHx42DXE/+ze2LgcK XqngZ/rjWu3QypjMP3CuE38pvY3rOA4nkT2c1ENvKospTcMSFeMtEfbB444gILoZLt9I 3lQITFKmb7exElJYXIqFIos5oosds75gX86r5nLSSp1rB1hJqalzPuk82Djf5FGqJQrY 6B6USeBGMPmqn2Qdp4ENzJwDRdtyd4VGvpayuVyj0zTZ7jyQxQhCOnTIQg7hEBxqha9z ci6vJbnmwaj6FjNlfCQYKOcHhxBz4EeCuermjPuz4qUUzWKUTmfMU85jQtexN/aw2woK 5BIw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; t=1680248452; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=548zcSLJYSR6puTSQTcbbtm0W6Y5GofHbCgq+Ac6wj0=; b=EZpPu5gfc9IafHo/t/F+9OlJGxNsxLqIeM4Il1IdiY4uhH8tu4Etkr5hr0XdjOmXMv 27nLNWjupRhWWZT69275SsDQy4wz59ep4FfaZ1YZVnpX++WaZ3FityKkrU2oqRm46sZI xzmCxNBIApAwJc/J2KEfGvp5Ax5Jy1tuikGKXnE2oN3fwGSftXNK61H6VAGrTuP9jOZc yvuq4BeOYMNnf7g7N/HSIZXFpFU7G/d0ymeXViPCnN4Azf/T1Y61jV7o8aw3S0hSQpxM mQaHuOjfg9cuM7pYgs0wqPFLwylm4Ydg8xSEr4lJl1M3Wh/0f5wqgN6ZXIaGvbWVzQFC GBgw== X-Gm-Message-State: AO0yUKX+0sBQDH/XzupXfemhEzWCHaJMlfMgRwzbC924EV9GU+R0SlYm JH+It5Nd3H8z+ncLF9QkoVAb6bHqRgABQ0+7kLDg8Q== X-Google-Smtp-Source: AK7set+r1qYivsl9wCo7KlfMqxO+4NEO/Egi3ao/192LhMH3d2DUZNo5aSneg9N6bpAb2AufIzhffw== X-Received: by 2002:a5e:c001:0:b0:752:8aee:6d91 with SMTP id u1-20020a5ec001000000b007528aee6d91mr18664332iol.12.1680248452407; Fri, 31 Mar 2023 00:40:52 -0700 (PDT) Received: from localhost.localdomain ([93.93.133.154]) by smtp.gmail.com with ESMTPSA id i17-20020a6bf411000000b0074555814e73sm421943iog.32.2023.03.31.00.40.51 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 31 Mar 2023 00:40:52 -0700 (PDT) To: libcamera-devel@lists.libcamera.org Date: Fri, 31 Mar 2023 08:23:36 +0100 Message-Id: <20230331072341.24851-6-naush@raspberrypi.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230331072341.24851-1-naush@raspberrypi.com> References: <20230331072341.24851-1-naush@raspberrypi.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v3 05/10] ipa: raspberrypi: alsc: Use a better type name for sparse arrays X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Naushir Patuck via libcamera-devel From: Naushir Patuck Reply-To: Naushir Patuck Cc: Jacopo Mondi Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" From: David Plowman The algorithm uses the data type std::vector> to represent the large sparse matrices that are XY (X, Y being the ALSC grid size) high but with only 4 non-zero elements on each row. Replace this slightly long type name by SparseArray. No functional changes. Signed-off-by: David Plowman Reviewed-by: Naushir Patuck Reviewed-by: Jacopo Mondi --- src/ipa/raspberrypi/controller/rpi/alsc.cpp | 24 ++++++++++----------- src/ipa/raspberrypi/controller/rpi/alsc.h | 10 ++++++++- 2 files changed, 21 insertions(+), 13 deletions(-) diff --git a/src/ipa/raspberrypi/controller/rpi/alsc.cpp b/src/ipa/raspberrypi/controller/rpi/alsc.cpp index 524c48093590..3a2e8fe00ca6 100644 --- a/src/ipa/raspberrypi/controller/rpi/alsc.cpp +++ b/src/ipa/raspberrypi/controller/rpi/alsc.cpp @@ -607,7 +607,7 @@ static double computeWeight(double Ci, double Cj, double sigma) /* Compute all weights. */ static void computeW(const Array2D &C, double sigma, - std::vector> &W) + SparseArray &W) { size_t XY = C.size(); size_t X = C.dimensions().width; @@ -623,8 +623,8 @@ static void computeW(const Array2D &C, double sigma, /* Compute M, the large but sparse matrix such that M * lambdas = 0. */ static void constructM(const Array2D &C, - const std::vector> &W, - std::vector> &M) + const SparseArray &W, + SparseArray &M) { size_t XY = C.size(); size_t X = C.dimensions().width; @@ -651,37 +651,37 @@ static void constructM(const Array2D &C, * left/right neighbours are zero down the left/right edges, so we don't need * need to test the i value to exclude them. */ -static double computeLambdaBottom(int i, const std::vector> &M, +static double computeLambdaBottom(int i, const SparseArray &M, Array2D &lambda) { return M[i][1] * lambda[i + 1] + M[i][2] * lambda[i + lambda.dimensions().width] + M[i][3] * lambda[i - 1]; } -static double computeLambdaBottomStart(int i, const std::vector> &M, +static double computeLambdaBottomStart(int i, const SparseArray &M, Array2D &lambda) { return M[i][1] * lambda[i + 1] + M[i][2] * lambda[i + lambda.dimensions().width]; } -static double computeLambdaInterior(int i, const std::vector> &M, +static double computeLambdaInterior(int i, const SparseArray &M, Array2D &lambda) { return M[i][0] * lambda[i - lambda.dimensions().width] + M[i][1] * lambda[i + 1] + M[i][2] * lambda[i + lambda.dimensions().width] + M[i][3] * lambda[i - 1]; } -static double computeLambdaTop(int i, const std::vector> &M, +static double computeLambdaTop(int i, const SparseArray &M, Array2D &lambda) { return M[i][0] * lambda[i - lambda.dimensions().width] + M[i][1] * lambda[i + 1] + M[i][3] * lambda[i - 1]; } -static double computeLambdaTopEnd(int i, const std::vector> &M, +static double computeLambdaTopEnd(int i, const SparseArray &M, Array2D &lambda) { return M[i][0] * lambda[i - lambda.dimensions().width] + M[i][3] * lambda[i - 1]; } /* Gauss-Seidel iteration with over-relaxation. */ -static double gaussSeidel2Sor(const std::vector> &M, double omega, +static double gaussSeidel2Sor(const SparseArray &M, double omega, Array2D &lambda, double lambdaBound) { int XY = lambda.size(); @@ -753,8 +753,8 @@ static void reaverage(Array2D &data) static void runMatrixIterations(const Array2D &C, Array2D &lambda, - const std::vector> &W, - std::vector> &M, double omega, + const SparseArray &W, + SparseArray &M, double omega, unsigned int nIter, double threshold, double lambdaBound) { constructM(C, W, M); @@ -813,7 +813,7 @@ void Alsc::doAlsc() { Array2D &cr = tmpC_[0], &cb = tmpC_[1], &calTableR = tmpC_[2], &calTableB = tmpC_[3], &calTableTmp = tmpC_[4]; - std::vector> &wr = tmpM_[0], &wb = tmpM_[1], &M = tmpM_[2]; + SparseArray &wr = tmpM_[0], &wb = tmpM_[1], &M = tmpM_[2]; /* * Calculate our R/B ("Cr"/"Cb") colour statistics, and assess which are diff --git a/src/ipa/raspberrypi/controller/rpi/alsc.h b/src/ipa/raspberrypi/controller/rpi/alsc.h index 1ab61299c4cd..0b6d9478073c 100644 --- a/src/ipa/raspberrypi/controller/rpi/alsc.h +++ b/src/ipa/raspberrypi/controller/rpi/alsc.h @@ -68,6 +68,14 @@ private: std::vector data_; }; +/* + * We'll use the term SparseArray for the large sparse matrices that are + * XY tall but have only 4 non-zero elements on each row. + */ + +template +using SparseArray = std::vector>; + struct AlscCalibration { double ct; Array2D table; @@ -160,7 +168,7 @@ private: /* Temporaries for the computations */ std::array, 5> tmpC_; - std::array>, 3> tmpM_; + std::array, 3> tmpM_; }; } /* namespace RPiController */ From patchwork Fri Mar 31 07:23:37 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Naushir Patuck X-Patchwork-Id: 18502 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 2582FC329D for ; Fri, 31 Mar 2023 07:40:59 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id B3BA46279A; Fri, 31 Mar 2023 09:40:58 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1680248458; bh=+GFENiSMhQ3HqhhlTZZ9jxVl3H1/G1XEnMd7DpUDotY=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To:Cc: From; b=hTZpMMZyqG9cz7cNeVpCyNWKQZi8jceeWEUNPtlS0S4NmAsQw/3Ie6xPH+H11n4Sv +kO0Hfnx8ZZtSPVsWtHueswrPpx85I4UjjvEW3rOjtPXSPtA3FolVv2QF6VzvfJ1oc CfYXHQewSrs1IDfU+sngdUpMHV5S7SK7VpxGxcFIVAQibzPv/U2P4w6O+JnGM+jKsm yHVEJA5JhryQTAccqDVreJ/1c/scj6FlRF7qXs6YGH//WByWDYQ9KfhsFTNownnOek 97ZpdxUEPNle0xDY/ZCvlK8oWTmdM5B0T6OVnotMkvaWdrXHbBIJdhF3mgVSkpFDBn NdJm9yZXxbHUQ== Received: from mail-io1-xd2e.google.com (mail-io1-xd2e.google.com [IPv6:2607:f8b0:4864:20::d2e]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 504206277E for ; Fri, 31 Mar 2023 09:40:55 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="qIEZ7HLr"; dkim-atps=neutral Received: by mail-io1-xd2e.google.com with SMTP id bl9so9384549iob.8 for ; Fri, 31 Mar 2023 00:40:55 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; t=1680248454; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=CBhJt0BIC9Lqwq4l4PQJqQPOKidr71Zuik9quGaeQXo=; b=qIEZ7HLrdACAYDmlLPtVpqIWy7Rea+VFIgZYUwgZGF8H0ZbEOR9dsjuHoZcO7tM3Zp mGMUca72yl+xnq0Q9P6r0RegBC2LEtEh3Bg6C0od99ibUgV/ePCkWdpICnYuOKeGfdxW EdLsRy5qMZ6vuOvkQEn18S7oTa8y1Iqsc4TmZjnvQ4wvSBaLMKSs8GyJEJN9TKnN+U7F 0v1UbCAXz0g31NSdofsLf+fpW3uHMybFYM5upRlrZJNh4e60YBjh4yKuVho2WQGtKhn2 /pHEBmjfD/gc2nGZkSZABOnnV+QBOgPXAlRAeutRH6pk6yUE3aikoDOqc6z5iYVrhqUc x5mA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; t=1680248454; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=CBhJt0BIC9Lqwq4l4PQJqQPOKidr71Zuik9quGaeQXo=; b=dbmqGMi14J7E0LSqRcRtF4mfKPc/cvej4Lsz4VfKkCYdAZnHRdJzR6EbsRa5hyTa55 AMubmvkH3/we0WqYlxLZes0JjSuf+ufZvhnHuSCdmkINMzvcUWN23dhFhbCQwuXydBC8 NgoxU63wJMfhrcqoDAwAnALjM+HzBcwdwNt9GCpYQucoj9t7pY4cc31ldkKErVVyde2n zE0o2fCEYqoENh23IFGql4sm+Gat9wSKRH3fSrlTr3ZHXdmz6BBzTwYKjqZWAiBfONXX pevw8AEwcynnwYi+fQq6841toN8AmvwdU55ZzpI2zX4ubJGlO4X0jS3QCsWopvII8Y5N FNzQ== X-Gm-Message-State: AO0yUKWbklBHWgoQ0hOQ1fL6Kw9yNto2ojir1ziNG93DmDutoWUEM1k/ /kfwYHs6SWadQI21HVQkWjM5b75uMqVtcKD8tJI/gw== X-Google-Smtp-Source: AK7set+at/3/A6n30NpBjD8Y5w9jFiWISNNVPjiHpLBFcTBUJO5gNRwaLwu684OJYw2GMqyu3+FQOQ== X-Received: by 2002:a5e:8b49:0:b0:758:1db1:ca99 with SMTP id z9-20020a5e8b49000000b007581db1ca99mr18057445iom.19.1680248453684; Fri, 31 Mar 2023 00:40:53 -0700 (PDT) Received: from localhost.localdomain ([93.93.133.154]) by smtp.gmail.com with ESMTPSA id i17-20020a6bf411000000b0074555814e73sm421943iog.32.2023.03.31.00.40.52 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 31 Mar 2023 00:40:53 -0700 (PDT) To: libcamera-devel@lists.libcamera.org Date: Fri, 31 Mar 2023 08:23:37 +0100 Message-Id: <20230331072341.24851-7-naush@raspberrypi.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230331072341.24851-1-naush@raspberrypi.com> References: <20230331072341.24851-1-naush@raspberrypi.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v3 06/10] ipa: raspberrypi: Generalise the contrast algorithm X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Naushir Patuck via libcamera-devel From: Naushir Patuck Reply-To: Naushir Patuck Cc: Jacopo Mondi Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Generalise the contrast algorithm code by removing any hard-coded assumptions about the target hardware platform. Instead, the algorithm code creates a generic Pwl that gets returned to the IPA, where it gets converted to the bcm2835 hardware specific lookup table. As a drive-by, remove an unused mutex. Signed-off-by: Naushir Patuck Reviewed-by: David Plowman Reviewed-by: Jacopo Mondi --- .../raspberrypi/controller/contrast_status.h | 11 ++----- .../raspberrypi/controller/rpi/contrast.cpp | 30 ++++--------------- src/ipa/raspberrypi/controller/rpi/contrast.h | 1 - src/ipa/raspberrypi/raspberrypi.cpp | 15 +++++++--- 4 files changed, 20 insertions(+), 37 deletions(-) diff --git a/src/ipa/raspberrypi/controller/contrast_status.h b/src/ipa/raspberrypi/controller/contrast_status.h index ef2a7c680fc2..fb9fe4bace71 100644 --- a/src/ipa/raspberrypi/controller/contrast_status.h +++ b/src/ipa/raspberrypi/controller/contrast_status.h @@ -6,20 +6,15 @@ */ #pragma once +#include "pwl.h" + /* * The "contrast" algorithm creates a gamma curve, optionally doing a little bit * of contrast stretching based on the AGC histogram. */ -constexpr unsigned int ContrastNumPoints = 33; - -struct ContrastPoint { - uint16_t x; - uint16_t y; -}; - struct ContrastStatus { - struct ContrastPoint points[ContrastNumPoints]; + RPiController::Pwl gammaCurve; double brightness; double contrast; }; diff --git a/src/ipa/raspberrypi/controller/rpi/contrast.cpp b/src/ipa/raspberrypi/controller/rpi/contrast.cpp index a4f8c4f04fc4..bee1eadd3657 100644 --- a/src/ipa/raspberrypi/controller/rpi/contrast.cpp +++ b/src/ipa/raspberrypi/controller/rpi/contrast.cpp @@ -65,34 +65,19 @@ void Contrast::setContrast(double contrast) contrast_ = contrast; } -static void fillInStatus(ContrastStatus &status, double brightness, - double contrast, Pwl &gammaCurve) -{ - status.brightness = brightness; - status.contrast = contrast; - for (unsigned int i = 0; i < ContrastNumPoints - 1; i++) { - int x = i < 16 ? i * 1024 - : (i < 24 ? (i - 16) * 2048 + 16384 - : (i - 24) * 4096 + 32768); - status.points[i].x = x; - status.points[i].y = std::min(65535.0, gammaCurve.eval(x)); - } - status.points[ContrastNumPoints - 1].x = 65535; - status.points[ContrastNumPoints - 1].y = 65535; -} - void Contrast::initialise() { /* * Fill in some default values as Prepare will run before Process gets * called. */ - fillInStatus(status_, brightness_, contrast_, config_.gammaCurve); + status_.brightness = brightness_; + status_.contrast = contrast_; + status_.gammaCurve = config_.gammaCurve; } void Contrast::prepare(Metadata *imageMetadata) { - std::unique_lock lock(mutex_); imageMetadata->set("contrast.status", status_); } @@ -183,12 +168,9 @@ void Contrast::process(StatisticsPtr &stats, * And fill in the status for output. Use more points towards the bottom * of the curve. */ - ContrastStatus status; - fillInStatus(status, brightness_, contrast_, gammaCurve); - { - std::unique_lock lock(mutex_); - status_ = status; - } + status_.brightness = brightness_; + status_.contrast = contrast_; + status_.gammaCurve = std::move(gammaCurve); } /* Register algorithm with the system. */ diff --git a/src/ipa/raspberrypi/controller/rpi/contrast.h b/src/ipa/raspberrypi/controller/rpi/contrast.h index c68adbc9e463..9c81277a0450 100644 --- a/src/ipa/raspberrypi/controller/rpi/contrast.h +++ b/src/ipa/raspberrypi/controller/rpi/contrast.h @@ -46,7 +46,6 @@ private: double brightness_; double contrast_; ContrastStatus status_; - std::mutex mutex_; }; } /* namespace RPiController */ diff --git a/src/ipa/raspberrypi/raspberrypi.cpp b/src/ipa/raspberrypi/raspberrypi.cpp index c74558f1dc0c..e2662938c9d3 100644 --- a/src/ipa/raspberrypi/raspberrypi.cpp +++ b/src/ipa/raspberrypi/raspberrypi.cpp @@ -1593,14 +1593,21 @@ void IPARPi::applyCCM(const struct CcmStatus *ccmStatus, ControlList &ctrls) void IPARPi::applyGamma(const struct ContrastStatus *contrastStatus, ControlList &ctrls) { + const unsigned int numGammaPoints = controller_.getHardwareConfig().numGammaPoints; struct bcm2835_isp_gamma gamma; - gamma.enabled = 1; - for (unsigned int i = 0; i < ContrastNumPoints; i++) { - gamma.x[i] = contrastStatus->points[i].x; - gamma.y[i] = contrastStatus->points[i].y; + for (unsigned int i = 0; i < numGammaPoints - 1; i++) { + int x = i < 16 ? i * 1024 + : (i < 24 ? (i - 16) * 2048 + 16384 + : (i - 24) * 4096 + 32768); + gamma.x[i] = x; + gamma.y[i] = std::min(65535, contrastStatus->gammaCurve.eval(x)); } + gamma.x[numGammaPoints - 1] = 65535; + gamma.y[numGammaPoints - 1] = 65535; + gamma.enabled = 1; + ControlValue c(Span{ reinterpret_cast(&gamma), sizeof(gamma) }); ctrls.set(V4L2_CID_USER_BCM2835_ISP_GAMMA, c); From patchwork Fri Mar 31 07:23:38 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Naushir Patuck X-Patchwork-Id: 18503 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id AE437C0F2A for ; Fri, 31 Mar 2023 07:40:59 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 666C5627A9; Fri, 31 Mar 2023 09:40:59 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1680248459; bh=aRIc5B+sTvytGlTffgEhxKt6BpAyCyw6+uqLU6iJiI8=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To:Cc: From; b=ASJMENd2Us13D6WNRD/flt85Q5LlVRWju0bw06xS3O0uDLdiXDhvAEzLtVZZrC3IK 6U4aff3bHDJ3HV4HOtzDdlqxdTCKY5ClhKaBPuCxJTLkGxowKTlTAoGQoc3K8cFIgp sXaxregnGLPhp7YgBvBs9RsslUScNfls18KyBkue7igELVmhXQ4J7OdiA8AG/5el5S OE2nZMCPFuiShZrMENP4Rn9ngYo98f+o7ZYvqQr0st3tKFvKCyE5siY93EX5BjE5jp 4SYOQX2FQFQ35Yi+1XrvUMXqZ0dssaP3pvGXKb6hJZuM4jMk38oob93urR4KZ63sZU KU3TjCDsbPKJA== Received: from mail-io1-xd2c.google.com (mail-io1-xd2c.google.com [IPv6:2607:f8b0:4864:20::d2c]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 84D6062769 for ; Fri, 31 Mar 2023 09:40:56 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="UCP/xSWf"; dkim-atps=neutral Received: by mail-io1-xd2c.google.com with SMTP id o10so1231212iou.5 for ; Fri, 31 Mar 2023 00:40:56 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; t=1680248455; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=quqiaExFAgBviDMcf8Asfps18bCBmhdSNFsVml+XS2A=; b=UCP/xSWfbrbAC9mFZ65jEjzT3b5Q0PRWpBfFNWWEBTYvc5ufKV1pLHOVA9INugRdyC lLYrMCs5hCByYTjjUBgn34fP9BmOQpwNmoCJkzqDJXNONrfAFeVhlYnd3zgeIJysN2lm uodClJ62zVaxsLwb4ZPbEYD8UmPwvxce8A3Lu+v9WPZWjXArSvd5+KwLtrQSFKzpbqN+ IRb/qRbtfMfwu9lvAq77GlHbgX8smr/lZYduJ/snuavlafnOUK+6k6HoitCU8QmgSCah 8WltIqfiAS3HhlDVmzsQotELQvet9fvaZLP5G8+nUHr97EFIpSnV0AVhDkw3JT77rAqD P1wg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; t=1680248455; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=quqiaExFAgBviDMcf8Asfps18bCBmhdSNFsVml+XS2A=; b=Dzp/OpUo6fAUZSLeDC6chkeHafmm37j4MhtxRG0TcvjzA/6vE2LZozz+DdbibxhYBw hs/viM10BXXNrfMGQnFost3FSn4ORjdD8GQ5FAS6AZ0bp7GyZDUp4klWv/Kyk5TKGXiL IkpT0Kd/k3/TflgRyYXrAvUJmCfa6hoq/BQGWUsiJ99hDIAqQyxHpXjEwUm6f74w3HKI 3/eVi/rvHr/qJpNBuVHRkrYp065b6lPfGkECqFBiTxXid+BJY5QICNrGU6gR7X3Y8p2i S5bqWYwf8R+YyXQFMoGyo8X55R3w0NCuSq/8xqEYweFDl2Fe0FKdF9s++pkrRXwMEcXI TDZg== X-Gm-Message-State: AAQBX9dyPszE2811nc1FbDCU3M+nZbJQU+qiy5Y8i7O/rsu2T4EmFrJS BKM/3rVUUS8af7s/dJvGnp58yNZybtbYzoqp6e2EvQ== X-Google-Smtp-Source: AKy350YgAvyT+X1lyT6RoIpFou1ecFSKAwdWV69HlqV6+FVErvnxfjrjBzJk/8pqFRyvjNd4WMbjEQ== X-Received: by 2002:a05:6602:3806:b0:758:7246:c0f2 with SMTP id bb6-20020a056602380600b007587246c0f2mr5436867iob.1.1680248454855; Fri, 31 Mar 2023 00:40:54 -0700 (PDT) Received: from localhost.localdomain ([93.93.133.154]) by smtp.gmail.com with ESMTPSA id i17-20020a6bf411000000b0074555814e73sm421943iog.32.2023.03.31.00.40.53 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 31 Mar 2023 00:40:54 -0700 (PDT) To: libcamera-devel@lists.libcamera.org Date: Fri, 31 Mar 2023 08:23:38 +0100 Message-Id: <20230331072341.24851-8-naush@raspberrypi.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230331072341.24851-1-naush@raspberrypi.com> References: <20230331072341.24851-1-naush@raspberrypi.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v3 07/10] ipa: raspberrypi: Make RegionStats::get() always return a Region struct X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Naushir Patuck via libcamera-devel From: Naushir Patuck Reply-To: Naushir Patuck Cc: Jacopo Mondi Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Update the overloaded RegionStats::get() and RegionStats::getFloating() member functions to return a Region struct for consistency. Signed-off-by: Naushir Patuck Reviewed-by: David Plowman Reviewed-by: Jacopo Mondi --- src/ipa/raspberrypi/controller/region_stats.h | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/src/ipa/raspberrypi/controller/region_stats.h b/src/ipa/raspberrypi/controller/region_stats.h index 9aaf3a58a6f7..a8860dc8dba2 100644 --- a/src/ipa/raspberrypi/controller/region_stats.h +++ b/src/ipa/raspberrypi/controller/region_stats.h @@ -86,12 +86,12 @@ public: return get_(index); } - const T &get(const libcamera::Point &pos) const + const Region &get(const libcamera::Point &pos) const { return get(pos.y * size_.width + pos.x); } - const T &getFloating(unsigned int index) const + const Region &getFloating(unsigned int index) const { if (index >= numFloatingRegions()) return default_; From patchwork Fri Mar 31 07:23:39 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Naushir Patuck X-Patchwork-Id: 18504 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 3F1EBC329F for ; Fri, 31 Mar 2023 07:41:01 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id D425E6277E; Fri, 31 Mar 2023 09:41:00 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1680248460; bh=zxN8L9BcQbJI34o+Lrtm03FoFSjDEr3NlDyqe82KZiA=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To:Cc: From; b=d+dapaJMfZh7k5P9/Fhj+jY/DfwM7Vghid6eUlrnk167fl7PbGDiFDeHCOf+tu1+c cWMhgPfDNOVE/qxrda1MQwE+QJbVoi/z8QpU2O66mUit9iiv1ScW0m34s0DAUJojTk prub6DaeiyQQPihaiH7NaC6LowKjULzT/7UHkROO6lvhz/IrVT81Wml9Et7eDgx8AG bB20eVQf+otBjnry+ATCuCr7KDFLbUrF1cpMRRaCqXWBCO4Y0JFU32ekqVllnhlRMz ZNQjq7i23wARCWRMPQoR4NSM1ORP7Dn2NgVPApu0yapbSvifauIr6guZWjUq/n18aQ CGiTvATS8K9SQ== Received: from mail-io1-xd30.google.com (mail-io1-xd30.google.com [IPv6:2607:f8b0:4864:20::d30]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 9E0C66279C for ; Fri, 31 Mar 2023 09:40:57 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="RbeHncS/"; dkim-atps=neutral Received: by mail-io1-xd30.google.com with SMTP id x3so1879023iov.3 for ; Fri, 31 Mar 2023 00:40:57 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; t=1680248456; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=rrS46SBM06NUoF4wxPBdfbbZvRNbGZyJPXft+5T02xE=; b=RbeHncS/3pL4Xg6CPucUhgn/ZCNxWhXsFdcBTvDxsMkd6u9Nvzr+BthAKX7uEEsKe6 ObeP3AvM/+6fKoedkWu3R5N6R0ZH8oxKGr+6uZfW+bRgNF7HlWrjGqeDzT6w6zRg0BFO y+l9Gjp5dshk9cZgv/Y2A7f0Tp64AbluPzqmIOWq8n77gacCdgANZ+eTAtWnUUjxKXs+ 60IhIhZx0t0jc32vpTNVVCawmOpfenZ0Iwf9HLuJArjcDK7pDeVjBnTf9FQc8BoDm6Qn UxDZQjOj3DDYw0guIBEZMBfiz5HM+FSJxxIUkjNEVc7dNFHsgzwJtnXfRDMCWwYc9znu 8PVA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; t=1680248456; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=rrS46SBM06NUoF4wxPBdfbbZvRNbGZyJPXft+5T02xE=; b=kbNX5OYxL4C7kkdEzrsWGKxZRMuwXrMl7yPTLCyfcW4wUI8wR3N6FTL9CevSgAdWEc v+tDvznlHHXPid4sNQmBPCj6hHXJM6t2En5J2p5fHw2aBPzlhrGg8r1D8DukpGuTZP4o L439wEdQ2jLwrYOcecYhXNQVMcZpkJRKxFXP8rRZRdMfHrEtHfUbbBcZ3QI7SlLuiDMl jDXhRY/J0qOmpg1KDwnw9W08Z4f1T0H4B+uOMQL/52CNYhC4qUEcVIq+KYU+W07vNnOl OnmI6qBVzeGhGT7UjD25ajbhlzXJEDj0VR1f6zI8p56JEXTszHw2v9yRwmm8ljFNTiZn gDMQ== X-Gm-Message-State: AO0yUKW1sHv+hwkb06e+9eq4LeJZkk1h5QKNXwoYKxiCCHZJ2gv/hq1x ru3kxAWUKmU8PaaeI3dricVwcVEB0kdIEtDdC6sleg== X-Google-Smtp-Source: AK7set//g8Bwy9l51n1GAGQDDEjq4zIeOAFKiiy5pomWgHFjNONN/Ddtzt1Fj/CjxJ1V04Mtz3A+YQ== X-Received: by 2002:a5e:8412:0:b0:758:a647:d21e with SMTP id h18-20020a5e8412000000b00758a647d21emr17587818ioj.21.1680248456097; Fri, 31 Mar 2023 00:40:56 -0700 (PDT) Received: from localhost.localdomain ([93.93.133.154]) by smtp.gmail.com with ESMTPSA id i17-20020a6bf411000000b0074555814e73sm421943iog.32.2023.03.31.00.40.55 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 31 Mar 2023 00:40:55 -0700 (PDT) To: libcamera-devel@lists.libcamera.org Date: Fri, 31 Mar 2023 08:23:39 +0100 Message-Id: <20230331072341.24851-9-naush@raspberrypi.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230331072341.24851-1-naush@raspberrypi.com> References: <20230331072341.24851-1-naush@raspberrypi.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v3 08/10] ipa: raspberrypi: Generalise the agc algorithm X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Naushir Patuck via libcamera-devel From: Naushir Patuck Reply-To: Naushir Patuck Cc: Jacopo Mondi Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Remove any hard-coded assumptions about the target hardware platform from the AGC algorithm. Instead, use the "target" string provided by the camera tuning config and generalised statistics structures to determing parameters such as grid and region sizes. This change replaces all hard-coded arrays with equivalent std::vector types. Signed-off-by: Naushir Patuck Reviewed-by: Jacopo Mondi Reviewed-by: Kieran Bingham --- src/ipa/raspberrypi/controller/rpi/agc.cpp | 20 +++++++++++++------- src/ipa/raspberrypi/controller/rpi/agc.h | 9 +-------- 2 files changed, 14 insertions(+), 15 deletions(-) diff --git a/src/ipa/raspberrypi/controller/rpi/agc.cpp b/src/ipa/raspberrypi/controller/rpi/agc.cpp index f438cab8840a..e6fb7b8dbeb3 100644 --- a/src/ipa/raspberrypi/controller/rpi/agc.cpp +++ b/src/ipa/raspberrypi/controller/rpi/agc.cpp @@ -31,17 +31,12 @@ LOG_DEFINE_CATEGORY(RPiAgc) int AgcMeteringMode::read(const libcamera::YamlObject ¶ms) { const YamlObject &yamlWeights = params["weights"]; - if (yamlWeights.size() != AgcStatsSize) { - LOG(RPiAgc, Error) << "AgcMeteringMode: Incorrect number of weights"; - return -EINVAL; - } - unsigned int num = 0; for (const auto &p : yamlWeights.asList()) { auto value = p.get(); if (!value) return -EINVAL; - weights[num++] = *value; + weights.push_back(*value); } return 0; @@ -248,6 +243,14 @@ int Agc::read(const libcamera::YamlObject ¶ms) if (ret) return ret; + const Size &size = getHardwareConfig().agcZoneWeights; + for (auto const &modes : config_.meteringModes) { + if (modes.second.weights.size() != size.width * size.height) { + LOG(RPiAgc, Error) << "AgcMeteringMode: Incorrect number of weights"; + return -EINVAL; + } + } + /* * Set the config's defaults (which are the first ones it read) as our * current modes, until someone changes them. (they're all known to @@ -585,9 +588,12 @@ void Agc::fetchAwbStatus(Metadata *imageMetadata) } static double computeInitialY(StatisticsPtr &stats, AwbStatus const &awb, - double weights[], double gain) + std::vector &weights, double gain) { constexpr uint64_t maxVal = 1 << Statistics::NormalisationFactorPow2; + + ASSERT(weights.size() == stats->agcRegions.numRegions()); + /* * Note how the calculation below means that equal weights give you * "average" metering (i.e. all pixels equally important). diff --git a/src/ipa/raspberrypi/controller/rpi/agc.h b/src/ipa/raspberrypi/controller/rpi/agc.h index 4fa9b8de5fb2..4e5f272fac78 100644 --- a/src/ipa/raspberrypi/controller/rpi/agc.h +++ b/src/ipa/raspberrypi/controller/rpi/agc.h @@ -17,17 +17,10 @@ /* This is our implementation of AGC. */ -/* - * This is the number actually set up by the firmware, not the maximum possible - * number (which is 16). - */ - -constexpr unsigned int AgcStatsSize = 15; - namespace RPiController { struct AgcMeteringMode { - double weights[AgcStatsSize]; + std::vector weights; int read(const libcamera::YamlObject ¶ms); }; From patchwork Fri Mar 31 07:23:40 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Naushir Patuck X-Patchwork-Id: 18505 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id D1910C32A0 for ; Fri, 31 Mar 2023 07:41:01 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 6D8E6627AD; Fri, 31 Mar 2023 09:41:01 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1680248461; bh=ncrGweLxWea2+Bw4CITlo9GjIg49BlTlYgt3b8PJ76M=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To:Cc: From; b=FR+0RW81V65ApsWzvTzX8nfJfhBAxyDZQGtyI8zuxg91tS/nBCFkfgNEwKgTIVHx5 BgykJIt5b7rJjD3dNxDL/X2ExOYoV/Hcsu8jT17lk5oFwogyREQQlND2ErUfKM8bSC MuYi8H3mhrkUXKjbSPsvrL3o5PH+/ai63+qb502KeKOY4DYuy2+de/eBtC+Plgf4wx drWxXJpj7EIXWXBuO9XF12Hq3XlMYPf2fkbicIm0m+5DrSgpuJgornlBP+fGOcSiuM Ludc5ejOgkOeHTMWCBCZMAV1styDN1H4Wf5QVtcvDFitm7+Bj3rHlzuZk+r6GlvzXr maWh+Ux6Zkenw== Received: from mail-io1-xd35.google.com (mail-io1-xd35.google.com [IPv6:2607:f8b0:4864:20::d35]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 2295F6277E for ; Fri, 31 Mar 2023 09:40:58 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="ldTwBigs"; dkim-atps=neutral Received: by mail-io1-xd35.google.com with SMTP id e13so9412100ioc.0 for ; Fri, 31 Mar 2023 00:40:58 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; t=1680248457; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=f5oCRAO5fd/BFOMWOXt8X7wYbXnymYbh0UJoNzVZLyA=; b=ldTwBigs0cuBqjE/5VWqW32D118Xa6F/sBkf8EmKDyzGrr8Lp+zDJLQ7uehXLLUdeQ WXY6qv6qVcKJAvhHWNcVOFr6cg6mdmjxSPahfZswwp0Ny6hE09/NeHUDUwoIgdju4iHC Fd628YL5lQ32Q80rl+4zkYBYCLb5ZBfXW5yTp1s6jOhTsvylrMo97YEkeXF31qlUyhes qP7G98uHUHrT147p3InOcpx8QjsPIVglt/hPGy5kmMTM3hg8WNlYtYTzAdfVZRxYZv8q UPhGwPla4ZSmHDczREIY6mQDMYe6x5T6gdqujwUEEgTTFcqnpkV/gEDUkqMNtv7YxB8M rbHw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; t=1680248457; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=f5oCRAO5fd/BFOMWOXt8X7wYbXnymYbh0UJoNzVZLyA=; b=WcCiItQiOWFDxplEIWsrwZ1JnE+FuZWase/41awnQqlxNEfoZu/Pgr9o1nTOoADIW7 uGtwev3p+ZthSgk3/GsUr8kYoSW668/3F51c9Wc2PxADu9cd7OqUBQiOH89wvxhNYFRj ALllDN46KM3K5Ku8xSJevItTwM+7Y/PBgqEB35NrIuXqlIRyaz8zLxvoesL8yylF2L1j Ie7CiDZGTS2+rGUAZd9LZJLWo96r9BMKmyH9yb0t9AohiIobs/2iiZU0VKzwDI2nwFJQ eIIWCuZjXx4BCcpKS3IcnWSnbblsdGYV+eKUocB8aYRD3BV26yp4WqBRMdPkynqcd3M8 VW4Q== X-Gm-Message-State: AO0yUKWhygGvkJG7dRqpJ+W2s4iE5a7dQ+mGso5CYgXB4CMDZL3KoDlx uCvOHqdgFaa4UimlIxiViJq3BboC9p4qY/bNhC5toA== X-Google-Smtp-Source: AK7set+ZiSiLt+SJjK5hm6Nv4WfpAEfQjVfid8d9mHqBxwQg2LMj6X+zpJCbOnPK8hjDmpsTwaveBA== X-Received: by 2002:a6b:e503:0:b0:715:3123:9514 with SMTP id y3-20020a6be503000000b0071531239514mr17946539ioc.3.1680248457350; Fri, 31 Mar 2023 00:40:57 -0700 (PDT) Received: from localhost.localdomain ([93.93.133.154]) by smtp.gmail.com with ESMTPSA id i17-20020a6bf411000000b0074555814e73sm421943iog.32.2023.03.31.00.40.56 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 31 Mar 2023 00:40:57 -0700 (PDT) To: libcamera-devel@lists.libcamera.org Date: Fri, 31 Mar 2023 08:23:40 +0100 Message-Id: <20230331072341.24851-10-naush@raspberrypi.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230331072341.24851-1-naush@raspberrypi.com> References: <20230331072341.24851-1-naush@raspberrypi.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v3 09/10] ipa: raspberrypi: Generalise the autofocus algorithm X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Naushir Patuck via libcamera-devel From: Naushir Patuck Reply-To: Naushir Patuck Cc: Jacopo Mondi , Nick Hollinghurst Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" From: Nick Hollinghurst Remove any hard-coded assumptions about the target hardware platform from the autofocus algorithm. Instead, use the "target" string provided by the camera tuning config and generalised statistics structures to determing parameters such as grid and region sizes. Additionally, PDAF statistics are represented by a generalised region statistics structure to be device agnostic. These changes also require the autofocus algorithm to initialise region weights on the first frame's prepare()/process() call rather than during initialisation. Signed-off-by: Nick Hollinghurst Signed-off-by: Naushir Patuck Tested-by: Naushir Patuck Reviewed-by: Naushir Patuck Reviewed-by: Jacopo Mondi --- src/ipa/raspberrypi/cam_helper_imx708.cpp | 23 ++- src/ipa/raspberrypi/controller/pdaf_data.h | 21 +-- src/ipa/raspberrypi/controller/rpi/af.cpp | 176 +++++++++++---------- src/ipa/raspberrypi/controller/rpi/af.h | 28 ++-- 4 files changed, 133 insertions(+), 115 deletions(-) diff --git a/src/ipa/raspberrypi/cam_helper_imx708.cpp b/src/ipa/raspberrypi/cam_helper_imx708.cpp index 1f213d3c0833..641ba18f4b9d 100644 --- a/src/ipa/raspberrypi/cam_helper_imx708.cpp +++ b/src/ipa/raspberrypi/cam_helper_imx708.cpp @@ -69,11 +69,14 @@ private: /* Largest long exposure scale factor given as a left shift on the frame length. */ static constexpr int longExposureShiftMax = 7; + static constexpr int pdafStatsRows = 12; + static constexpr int pdafStatsCols = 16; + void populateMetadata(const MdParser::RegisterMap ®isters, Metadata &metadata) const override; static bool parsePdafData(const uint8_t *ptr, size_t len, unsigned bpp, - PdafData &pdaf); + PdafRegions &pdaf); bool parseAEHist(const uint8_t *ptr, size_t len, unsigned bpp); void putAGCStatistics(StatisticsPtr stats); @@ -120,11 +123,11 @@ void CamHelperImx708::prepare(libcamera::Span buffer, Metadata &m size_t bytesPerLine = (mode_.width * mode_.bitdepth) >> 3; if (buffer.size() > 2 * bytesPerLine) { - PdafData pdaf; + PdafRegions pdaf; if (parsePdafData(&buffer[2 * bytesPerLine], buffer.size() - 2 * bytesPerLine, mode_.bitdepth, pdaf)) - metadata.set("pdaf.data", pdaf); + metadata.set("pdaf.regions", pdaf); } /* Parse AE-HIST data where present */ @@ -239,7 +242,7 @@ void CamHelperImx708::populateMetadata(const MdParser::RegisterMap ®isters, } bool CamHelperImx708::parsePdafData(const uint8_t *ptr, size_t len, - unsigned bpp, PdafData &pdaf) + unsigned bpp, PdafRegions &pdaf) { size_t step = bpp >> 1; /* bytes per PDAF grid entry */ @@ -248,13 +251,17 @@ bool CamHelperImx708::parsePdafData(const uint8_t *ptr, size_t len, return false; } + pdaf.init({ pdafStatsCols, pdafStatsRows }); + ptr += 2 * step; - for (unsigned i = 0; i < PDAF_DATA_ROWS; ++i) { - for (unsigned j = 0; j < PDAF_DATA_COLS; ++j) { + for (unsigned i = 0; i < pdafStatsRows; ++i) { + for (unsigned j = 0; j < pdafStatsCols; ++j) { unsigned c = (ptr[0] << 3) | (ptr[1] >> 5); int p = (((ptr[1] & 0x0F) - (ptr[1] & 0x10)) << 6) | (ptr[2] >> 2); - pdaf.conf[i][j] = c; - pdaf.phase[i][j] = c ? p : 0; + PdafData pdafData; + pdafData.conf = c; + pdafData.phase = c ? p : 0; + pdaf.set(libcamera::Point(j, i), { pdafData, 1, 0 }); ptr += step; } } diff --git a/src/ipa/raspberrypi/controller/pdaf_data.h b/src/ipa/raspberrypi/controller/pdaf_data.h index 03c00d72c0e8..ae6ab996ded0 100644 --- a/src/ipa/raspberrypi/controller/pdaf_data.h +++ b/src/ipa/raspberrypi/controller/pdaf_data.h @@ -2,20 +2,23 @@ /* * Copyright (C) 2022, Raspberry Pi Ltd * - * pdaf_data.h - PDAF Metadata; for now this is - * largely based on IMX708's PDAF "Type 1" output. + * pdaf_data.h - PDAF Metadata */ #pragma once #include -#define PDAF_DATA_ROWS 12 -#define PDAF_DATA_COLS 16 +#include "region_stats.h" -struct PdafData { - /* Confidence values, in raster order, in arbitrary units */ - uint16_t conf[PDAF_DATA_ROWS][PDAF_DATA_COLS]; +namespace RPiController { - /* Phase error, in raster order, in s11 Q4 format (S.6.4) */ - int16_t phase[PDAF_DATA_ROWS][PDAF_DATA_COLS]; +struct PdafData { + /* Confidence, in arbitrary units */ + uint16_t conf; + /* Phase error, in s16 Q4 format (S.11.4) */ + int16_t phase; }; + +using PdafRegions = RegionStats; + +}; // namespace RPiController diff --git a/src/ipa/raspberrypi/controller/rpi/af.cpp b/src/ipa/raspberrypi/controller/rpi/af.cpp index a623651875f2..ed0c8a94d064 100644 --- a/src/ipa/raspberrypi/controller/rpi/af.cpp +++ b/src/ipa/raspberrypi/controller/rpi/af.cpp @@ -174,9 +174,8 @@ Af::Af(Controller *controller) statsRegion_(0, 0, 0, 0), windows_(), useWindows_(false), - phaseWeights_{}, - contrastWeights_{}, - sumWeights_(0), + phaseWeights_(), + contrastWeights_(), scanState_(ScanState::Idle), initted_(false), ftarget_(-1.0), @@ -190,7 +189,15 @@ Af::Af(Controller *controller) scanData_(), reportState_(AfState::Idle) { - scanData_.reserve(24); + /* + * Reserve space for data, to reduce memory fragmentation. It's too early + * to query the size of the PDAF (from camera) and Contrast (from ISP) + * statistics, but these are plausible upper bounds. + */ + phaseWeights_.w.reserve(16 * 12); + contrastWeights_.w.reserve(getHardwareConfig().focusRegions.width * + getHardwareConfig().focusRegions.height); + scanData_.reserve(32); } Af::~Af() @@ -226,7 +233,7 @@ void Af::switchMode(CameraMode const &cameraMode, [[maybe_unused]] Metadata *met << statsRegion_.y << ',' << statsRegion_.width << ',' << statsRegion_.height; - computeWeights(); + invalidateWeights(); if (scanState_ >= ScanState::Coarse && scanState_ < ScanState::Settle) { /* @@ -239,111 +246,99 @@ void Af::switchMode(CameraMode const &cameraMode, [[maybe_unused]] Metadata *met skipCount_ = cfg_.skipFrames; } -void Af::computeWeights() +void Af::computeWeights(RegionWeights *wgts, unsigned rows, unsigned cols) { - constexpr int MaxCellWeight = 240 / (int)MaxWindows; + wgts->rows = rows; + wgts->cols = cols; + wgts->sum = 0; + wgts->w.resize(rows * cols); + std::fill(wgts->w.begin(), wgts->w.end(), 0); - sumWeights_ = 0; - for (int i = 0; i < PDAF_DATA_ROWS; ++i) - std::fill(phaseWeights_[i], phaseWeights_[i] + PDAF_DATA_COLS, 0); - - if (useWindows_ && - statsRegion_.width >= PDAF_DATA_COLS && statsRegion_.height >= PDAF_DATA_ROWS) { + if (rows > 0 && cols > 0 && useWindows_ && + statsRegion_.height >= rows && statsRegion_.width >= cols) { /* * Here we just merge all of the given windows, weighted by area. * \todo Perhaps a better approach might be to find the phase in each * window and choose either the closest or the highest-confidence one? - * - * Using mostly "int" arithmetic, because Rectangle has signed x, y + * Ensure weights sum to less than (1<<16). 46080 is a "round number" + * below 65536, for better rounding when window size is a simple + * fraction of image dimensions. */ - int cellH = (int)(statsRegion_.height / PDAF_DATA_ROWS); - int cellW = (int)(statsRegion_.width / PDAF_DATA_COLS); - int cellA = cellH * cellW; + const unsigned maxCellWeight = 46080u / (MaxWindows * rows * cols); + const unsigned cellH = statsRegion_.height / rows; + const unsigned cellW = statsRegion_.width / cols; + const unsigned cellA = cellH * cellW; for (auto &w : windows_) { - for (int i = 0; i < PDAF_DATA_ROWS; ++i) { - int y0 = std::max(statsRegion_.y + cellH * i, w.y); - int y1 = std::min(statsRegion_.y + cellH * (i + 1), w.y + (int)(w.height)); + for (unsigned r = 0; r < rows; ++r) { + int y0 = std::max(statsRegion_.y + (int)(cellH * r), w.y); + int y1 = std::min(statsRegion_.y + (int)(cellH * (r + 1)), + w.y + (int)(w.height)); if (y0 >= y1) continue; y1 -= y0; - for (int j = 0; j < PDAF_DATA_COLS; ++j) { - int x0 = std::max(statsRegion_.x + cellW * j, w.x); - int x1 = std::min(statsRegion_.x + cellW * (j + 1), w.x + (int)(w.width)); + for (unsigned c = 0; c < cols; ++c) { + int x0 = std::max(statsRegion_.x + (int)(cellW * c), w.x); + int x1 = std::min(statsRegion_.x + (int)(cellW * (c + 1)), + w.x + (int)(w.width)); if (x0 >= x1) continue; - int a = y1 * (x1 - x0); - a = (MaxCellWeight * a + cellA - 1) / cellA; - phaseWeights_[i][j] += a; - sumWeights_ += a; + unsigned a = y1 * (x1 - x0); + a = (maxCellWeight * a + cellA - 1) / cellA; + wgts->w[r * cols + c] += a; + wgts->sum += a; } } } } - if (sumWeights_ == 0) { - /* - * Default AF window is the middle 1/2 width of the middle 1/3 height - * since this maps nicely to both PDAF (16x12) and Focus (4x3) grids. - */ - for (int i = PDAF_DATA_ROWS / 3; i < 2 * PDAF_DATA_ROWS / 3; ++i) { - for (int j = PDAF_DATA_COLS / 4; j < 3 * PDAF_DATA_COLS / 4; ++j) { - phaseWeights_[i][j] = MaxCellWeight; - sumWeights_ += MaxCellWeight; + if (wgts->sum == 0) { + /* Default AF window is the middle 1/2 width of the middle 1/3 height */ + for (unsigned r = rows / 3; r < rows - rows / 3; ++r) { + for (unsigned c = cols / 4; c < cols - cols / 4; ++c) { + wgts->w[r * cols + c] = 1; + wgts->sum += 1; } } } +} - /* Scale from PDAF to Focus Statistics grid (which has fixed size 4x3) */ - constexpr int FocusStatsRows = 3; - constexpr int FocusStatsCols = 4; - static_assert(FOCUS_REGIONS == FocusStatsRows * FocusStatsCols); - static_assert(PDAF_DATA_ROWS % FocusStatsRows == 0); - static_assert(PDAF_DATA_COLS % FocusStatsCols == 0); - constexpr int YFactor = PDAF_DATA_ROWS / FocusStatsRows; - constexpr int XFactor = PDAF_DATA_COLS / FocusStatsCols; - - LOG(RPiAf, Debug) << "Recomputed weights:"; - for (int i = 0; i < FocusStatsRows; ++i) { - for (int j = 0; j < FocusStatsCols; ++j) { - unsigned w = 0; - for (int y = 0; y < YFactor; ++y) - for (int x = 0; x < XFactor; ++x) - w += phaseWeights_[YFactor * i + y][XFactor * j + x]; - contrastWeights_[FocusStatsCols * i + j] = w; - } - LOG(RPiAf, Debug) << " " - << contrastWeights_[FocusStatsCols * i + 0] << " " - << contrastWeights_[FocusStatsCols * i + 1] << " " - << contrastWeights_[FocusStatsCols * i + 2] << " " - << contrastWeights_[FocusStatsCols * i + 3]; - } +void Af::invalidateWeights() +{ + phaseWeights_.sum = 0; + contrastWeights_.sum = 0; } -bool Af::getPhase(PdafData const &data, double &phase, double &conf) const +bool Af::getPhase(PdafRegions const ®ions, double &phase, double &conf) { + libcamera::Size size = regions.size(); + if (size.height != phaseWeights_.rows || size.width != phaseWeights_.cols || + phaseWeights_.sum == 0) { + LOG(RPiAf, Debug) << "Recompute Phase weights " << size.width << 'x' << size.height; + computeWeights(&phaseWeights_, size.height, size.width); + } + uint32_t sumWc = 0; int64_t sumWcp = 0; - - for (unsigned i = 0; i < PDAF_DATA_ROWS; ++i) { - for (unsigned j = 0; j < PDAF_DATA_COLS; ++j) { - if (phaseWeights_[i][j]) { - uint32_t c = data.conf[i][j]; - if (c >= cfg_.confThresh) { - if (c > cfg_.confClip) - c = cfg_.confClip; - c -= (cfg_.confThresh >> 2); - sumWc += phaseWeights_[i][j] * c; - c -= (cfg_.confThresh >> 2); - sumWcp += phaseWeights_[i][j] * data.phase[i][j] * (int64_t)c; - } + for (unsigned i = 0; i < regions.numRegions(); ++i) { + unsigned w = phaseWeights_.w[i]; + if (w) { + const PdafData &data = regions.get(i).val; + unsigned c = data.conf; + if (c >= cfg_.confThresh) { + if (c > cfg_.confClip) + c = cfg_.confClip; + c -= (cfg_.confThresh >> 2); + sumWc += w * c; + c -= (cfg_.confThresh >> 2); + sumWcp += (int64_t)(w * c) * (int64_t)data.phase; } } } - if (0 < sumWeights_ && sumWeights_ <= sumWc) { + if (0 < phaseWeights_.sum && phaseWeights_.sum <= sumWc) { phase = (double)sumWcp / (double)sumWc; - conf = (double)sumWc / (double)sumWeights_; + conf = (double)sumWc / (double)phaseWeights_.sum; return true; } else { phase = 0.0; @@ -352,14 +347,21 @@ bool Af::getPhase(PdafData const &data, double &phase, double &conf) const } } -double Af::getContrast(const FocusRegions &focusStats) const +double Af::getContrast(const FocusRegions &focusStats) { - uint32_t sumWc = 0; + libcamera::Size size = focusStats.size(); + if (size.height != contrastWeights_.rows || + size.width != contrastWeights_.cols || contrastWeights_.sum == 0) { + LOG(RPiAf, Debug) << "Recompute Contrast weights " + << size.width << 'x' << size.height; + computeWeights(&contrastWeights_, size.height, size.width); + } + uint64_t sumWc = 0; for (unsigned i = 0; i < focusStats.numRegions(); ++i) - sumWc += contrastWeights_[i] * focusStats.get(i).val; + sumWc += contrastWeights_.w[i] * focusStats.get(i).val; - return (sumWeights_ == 0) ? 0.0 : (double)sumWc / (double)sumWeights_; + return (contrastWeights_.sum > 0) ? ((double)sumWc / (double)contrastWeights_.sum) : 0.0; } void Af::doPDAF(double phase, double conf) @@ -623,14 +625,14 @@ void Af::prepare(Metadata *imageMetadata) if (initted_) { /* Get PDAF from the embedded metadata, and run AF algorithm core */ - PdafData data; + PdafRegions regions; double phase = 0.0, conf = 0.0; double oldFt = ftarget_; double oldFs = fsmooth_; ScanState oldSs = scanState_; uint32_t oldSt = stepCount_; - if (imageMetadata->get("pdaf.data", data) == 0) - getPhase(data, phase, conf); + if (imageMetadata->get("pdaf.regions", regions) == 0) + getPhase(regions, phase, conf); doAF(prevContrast_, phase, conf); updateLensPosition(); LOG(RPiAf, Debug) << std::fixed << std::setprecision(2) @@ -691,7 +693,7 @@ void Af::setMetering(bool mode) { if (useWindows_ != mode) { useWindows_ = mode; - computeWeights(); + invalidateWeights(); } } @@ -708,7 +710,9 @@ void Af::setWindows(libcamera::Span const &wins) if (windows_.size() >= MaxWindows) break; } - computeWeights(); + + if (useWindows_) + invalidateWeights(); } bool Af::setLensPosition(double dioptres, int *hwpos) diff --git a/src/ipa/raspberrypi/controller/rpi/af.h b/src/ipa/raspberrypi/controller/rpi/af.h index 7959371baf64..b479feb88c39 100644 --- a/src/ipa/raspberrypi/controller/rpi/af.h +++ b/src/ipa/raspberrypi/controller/rpi/af.h @@ -11,12 +11,6 @@ #include "../pdaf_data.h" #include "../pwl.h" -/* - * \todo FOCUS_REGIONS is taken from bcm2835-isp.h, but should be made as a - * generic RegionStats structure. - */ -#define FOCUS_REGIONS 12 - /* * This algorithm implements a hybrid of CDAF and PDAF, favouring PDAF. * @@ -121,9 +115,20 @@ private: double conf; }; - void computeWeights(); - bool getPhase(PdafData const &data, double &phase, double &conf) const; - double getContrast(const FocusRegions &focusStats) const; + struct RegionWeights { + unsigned rows; + unsigned cols; + uint32_t sum; + std::vector w; + + RegionWeights() + : rows(0), cols(0), sum(0), w() {} + }; + + void computeWeights(RegionWeights *wgts, unsigned rows, unsigned cols); + void invalidateWeights(); + bool getPhase(PdafRegions const ®ions, double &phase, double &conf); + double getContrast(const FocusRegions &focusStats); void doPDAF(double phase, double conf); bool earlyTerminationByPhase(double phase); double findPeak(unsigned index) const; @@ -143,9 +148,8 @@ private: libcamera::Rectangle statsRegion_; std::vector windows_; bool useWindows_; - uint8_t phaseWeights_[PDAF_DATA_ROWS][PDAF_DATA_COLS]; - uint16_t contrastWeights_[FOCUS_REGIONS]; - uint32_t sumWeights_; + RegionWeights phaseWeights_; + RegionWeights contrastWeights_; /* Working state. */ ScanState scanState_; From patchwork Fri Mar 31 07:23:41 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Naushir Patuck X-Patchwork-Id: 18506 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id F1457C32A1 for ; Fri, 31 Mar 2023 07:41:02 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 5AEC4627B0; Fri, 31 Mar 2023 09:41:02 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1680248462; bh=Gv6rqzTKA+31wd9768EnOreD8gtbkn7EyWRda1EDJMk=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To: From; b=wHBFS6Nojgs4chzCU+PcHA8TOuT/m1zXC9mfhol3QzOcG7+NY2wFL09PGiipoeNId c5mu1T+3QfSdkDDXNbuWT58F6qZ89h6BEMAsa9gZg38HB8mj+gMVboMaAiRlwj4+wd GtBHxn2Nvek8/K7SkWsODSYbd2TS6wGmX0RzbYPwme0ykxU6KDiqEfIESeVRsbDIlI aE/7qu4+aRjGDlw94x94zW+itSBWONlIMP0j1fegdEmfhfHd74nElzAa4vgjObM9W7 MOpjICSlKolC4Snq6h6908qNpRp18ZDutvokwANYor3G3ylHqwI3qoviGPi5/V/n82 cmx5BP9b1qLQA== Received: from mail-io1-xd2c.google.com (mail-io1-xd2c.google.com [IPv6:2607:f8b0:4864:20::d2c]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 0E4A16276B for ; Fri, 31 Mar 2023 09:40:59 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="CaWTvZY7"; dkim-atps=neutral Received: by mail-io1-xd2c.google.com with SMTP id o10so1231259iou.5 for ; Fri, 31 Mar 2023 00:40:59 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; t=1680248458; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=UFZiwfMADtRs4OW/isIommDAqrPPblljCYn15JQKc4o=; b=CaWTvZY76PwYFhw1Fexrh4zOmVn0mdeiBjsx/SCGb5jtqdxSGw+6nbl1sIlrnOXwso 8BYEVzAdBw0O9fsiKuHYPX4IV7WgloGWrWeFDOG8SsCN7q9O414R6yq0sAANiP4NgL7X gPimLXzNi4PxshYyXRbSHSCRF/9IULkW2P9ALi1NM4MNgPjbihQgIyyUIdwbEuomQI11 VJT/esJ9DjfSomKq0b89Hcfq8NV+Ak2gCQILxf5876ONUyaQczljNuVzPsaOrSL6mrQw 4fqDRSBTbzj5QstABrMYYul5RhHdBy56itB+WO4FHXwmBgqTeNxlgWsnwo4+UUpIrsOk 28mw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; t=1680248458; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=UFZiwfMADtRs4OW/isIommDAqrPPblljCYn15JQKc4o=; b=ER3xgn3oXFUPqL/6487cN6YdPUo1Zua8dM3s8uh9K6gp/JQVQ8dyBHBgJ+1YvBqa3G ZARgAPFG8KtZTAvJBWFBXu6xcGtYDPnioBWkBMCvA9eLn87h3WZ9skY3Z8Rg8i4b6LBZ V/xM2lLioZRE761YIgAcXWNfKFTAMnJIsDZbsOBKURQFZJgH0GpvXEUq6SrJiP+Ngalw U5lhFqvLYBC9OrgdKlTZJbwMnXPuXF6yVTSLExat1FneDW7gP/N3Z62X/f5T4ITJSTN9 bUmmq8AkJ+figLS1IYmuYJzANPHEYFhSfamPBLDEwytJ5VkGnRQfdfD5DZqfHQ1ZkXg5 T/gQ== X-Gm-Message-State: AAQBX9cJgFAxerP3tOHmKYeUGgzKOc+I84sGS+k2AzYdTjS1EePIG1GI fo/y1TNq3w1YEMFyrlBDe+sei4ahrOER+Ncrfq0i7Q== X-Google-Smtp-Source: AKy350ZzstqJf9assJZ8bm8eUTOkPeUst9spAI43KMIEPIUUBTnlGeVldxlFME3LZGomUMXuYdpDGA== X-Received: by 2002:a6b:ce16:0:b0:75d:b1:b718 with SMTP id p22-20020a6bce16000000b0075d00b1b718mr3093279iob.9.1680248458377; Fri, 31 Mar 2023 00:40:58 -0700 (PDT) Received: from localhost.localdomain ([93.93.133.154]) by smtp.gmail.com with ESMTPSA id i17-20020a6bf411000000b0074555814e73sm421943iog.32.2023.03.31.00.40.57 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 31 Mar 2023 00:40:58 -0700 (PDT) To: libcamera-devel@lists.libcamera.org Date: Fri, 31 Mar 2023 08:23:41 +0100 Message-Id: <20230331072341.24851-11-naush@raspberrypi.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230331072341.24851-1-naush@raspberrypi.com> References: <20230331072341.24851-1-naush@raspberrypi.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v3 10/10] ipa: raspberrypi: Generalise the focus reporting code X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Naushir Patuck via libcamera-devel From: Naushir Patuck Reply-To: Naushir Patuck Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Use the generalised focus statistics structure to compute the centre window focus FoM value. This avoids needed to hard-code a specific grid size. Remove the focus reporting algorithm as the functionality is duplicated by this bit of IPA code. Remove focus_status.h as it is no longer needed. Signed-off-by: Naushir Patuck Reviewed-by: Jacopo Mondi Reviewed-by: David Plowman --- src/ipa/raspberrypi/controller/focus_status.h | 20 -------- src/ipa/raspberrypi/controller/rpi/af.h | 1 - src/ipa/raspberrypi/controller/rpi/focus.cpp | 49 ------------------- src/ipa/raspberrypi/data/imx290.json | 3 -- src/ipa/raspberrypi/data/imx296.json | 3 -- src/ipa/raspberrypi/data/imx296_mono.json | 3 -- src/ipa/raspberrypi/data/imx477.json | 3 -- src/ipa/raspberrypi/data/imx477_noir.json | 3 -- .../raspberrypi/data/imx477_scientific.json | 3 -- src/ipa/raspberrypi/data/imx477_v1.json | 3 -- src/ipa/raspberrypi/data/imx708.json | 3 -- src/ipa/raspberrypi/data/imx708_noir.json | 3 -- src/ipa/raspberrypi/data/imx708_wide.json | 3 -- .../raspberrypi/data/imx708_wide_noir.json | 3 -- src/ipa/raspberrypi/meson.build | 1 - src/ipa/raspberrypi/raspberrypi.cpp | 32 +++++++++--- 16 files changed, 24 insertions(+), 112 deletions(-) delete mode 100644 src/ipa/raspberrypi/controller/focus_status.h delete mode 100644 src/ipa/raspberrypi/controller/rpi/focus.cpp diff --git a/src/ipa/raspberrypi/controller/focus_status.h b/src/ipa/raspberrypi/controller/focus_status.h deleted file mode 100644 index 8b74e59840c1..000000000000 --- a/src/ipa/raspberrypi/controller/focus_status.h +++ /dev/null @@ -1,20 +0,0 @@ -/* SPDX-License-Identifier: BSD-2-Clause */ -/* - * Copyright (C) 2020, Raspberry Pi Ltd - * - * focus_status.h - focus measurement status - */ -#pragma once - -#include - -/* - * The focus algorithm should post the following structure into the image's - * "focus.status" metadata. Recall that it's only reporting focus (contrast) - * measurements, it's not driving any kind of auto-focus algorithm! - */ - -struct FocusStatus { - unsigned int num; - uint32_t focusMeasures[FOCUS_REGIONS]; -}; diff --git a/src/ipa/raspberrypi/controller/rpi/af.h b/src/ipa/raspberrypi/controller/rpi/af.h index b479feb88c39..6d2bae671a22 100644 --- a/src/ipa/raspberrypi/controller/rpi/af.h +++ b/src/ipa/raspberrypi/controller/rpi/af.h @@ -28,7 +28,6 @@ * "nuisance" scans. During each interval where PDAF is not working, only * ONE scan will be performed; CAF cannot track objects using CDAF alone. * - * This algorithm is unrelated to "rpi.focus" which merely reports CDAF FoM. */ namespace RPiController { diff --git a/src/ipa/raspberrypi/controller/rpi/focus.cpp b/src/ipa/raspberrypi/controller/rpi/focus.cpp deleted file mode 100644 index ea3cc00e42c3..000000000000 --- a/src/ipa/raspberrypi/controller/rpi/focus.cpp +++ /dev/null @@ -1,49 +0,0 @@ -/* SPDX-License-Identifier: BSD-2-Clause */ -/* - * Copyright (C) 2020, Raspberry Pi Ltd - * - * focus.cpp - focus algorithm - */ -#include - -#include - -#include "../focus_status.h" -#include "focus.h" - -using namespace RPiController; -using namespace libcamera; - -LOG_DEFINE_CATEGORY(RPiFocus) - -#define NAME "rpi.focus" - -Focus::Focus(Controller *controller) - : Algorithm(controller) -{ -} - -char const *Focus::name() const -{ - return NAME; -} - -void Focus::process(StatisticsPtr &stats, Metadata *imageMetadata) -{ - FocusStatus status; - for (unsigned int i = 0; i < stats->focusRegions.numRegions(); i++) - status.focusMeasures[i] = stats->focusRegions.get(i).val; - status.num = stats->focusRegions.numRegions(); - imageMetadata->set("focus.status", status); - - LOG(RPiFocus, Debug) - << "Focus contrast measure: " - << (status.focusMeasures[5] + status.focusMeasures[6]) / 10; -} - -/* Register algorithm with the system. */ -static Algorithm *create(Controller *controller) -{ - return new Focus(controller); -} -static RegisterAlgorithm reg(NAME, &create); diff --git a/src/ipa/raspberrypi/data/imx290.json b/src/ipa/raspberrypi/data/imx290.json index bfb9c6093d26..ace68d0ebf1c 100644 --- a/src/ipa/raspberrypi/data/imx290.json +++ b/src/ipa/raspberrypi/data/imx290.json @@ -195,9 +195,6 @@ } ] } - }, - { - "rpi.focus": { } } ] } \ No newline at end of file diff --git a/src/ipa/raspberrypi/data/imx296.json b/src/ipa/raspberrypi/data/imx296.json index 346f5b658957..ae8722c46a85 100644 --- a/src/ipa/raspberrypi/data/imx296.json +++ b/src/ipa/raspberrypi/data/imx296.json @@ -532,9 +532,6 @@ "strength": 1.0, "limit": 0.18 } - }, - { - "rpi.focus": { } } ] } diff --git a/src/ipa/raspberrypi/data/imx296_mono.json b/src/ipa/raspberrypi/data/imx296_mono.json index e9fa30c6a08d..30965b4b8d61 100644 --- a/src/ipa/raspberrypi/data/imx296_mono.json +++ b/src/ipa/raspberrypi/data/imx296_mono.json @@ -228,9 +228,6 @@ "strength": 1.0, "limit": 0.18 } - }, - { - "rpi.focus": { } } ] } diff --git a/src/ipa/raspberrypi/data/imx477.json b/src/ipa/raspberrypi/data/imx477.json index bfc0774ffdfc..daffc268ff43 100644 --- a/src/ipa/raspberrypi/data/imx477.json +++ b/src/ipa/raspberrypi/data/imx477.json @@ -513,9 +513,6 @@ }, { "rpi.sharpen": { } - }, - { - "rpi.focus": { } } ] } \ No newline at end of file diff --git a/src/ipa/raspberrypi/data/imx477_noir.json b/src/ipa/raspberrypi/data/imx477_noir.json index dadd97bc3d78..52d7f072d0ce 100644 --- a/src/ipa/raspberrypi/data/imx477_noir.json +++ b/src/ipa/raspberrypi/data/imx477_noir.json @@ -424,9 +424,6 @@ }, { "rpi.sharpen": { } - }, - { - "rpi.focus": { } } ] } \ No newline at end of file diff --git a/src/ipa/raspberrypi/data/imx477_scientific.json b/src/ipa/raspberrypi/data/imx477_scientific.json index 17c4ed0a5e74..26c692fdbab9 100644 --- a/src/ipa/raspberrypi/data/imx477_scientific.json +++ b/src/ipa/raspberrypi/data/imx477_scientific.json @@ -474,9 +474,6 @@ }, { "rpi.sharpen": { } - }, - { - "rpi.focus": { } } ] } \ No newline at end of file diff --git a/src/ipa/raspberrypi/data/imx477_v1.json b/src/ipa/raspberrypi/data/imx477_v1.json index 5bcaac67069b..d64020091efa 100644 --- a/src/ipa/raspberrypi/data/imx477_v1.json +++ b/src/ipa/raspberrypi/data/imx477_v1.json @@ -511,9 +511,6 @@ }, { "rpi.sharpen": { } - }, - { - "rpi.focus": { } } ] } \ No newline at end of file diff --git a/src/ipa/raspberrypi/data/imx708.json b/src/ipa/raspberrypi/data/imx708.json index c38b2d4cf256..b9830a3bf692 100644 --- a/src/ipa/raspberrypi/data/imx708.json +++ b/src/ipa/raspberrypi/data/imx708.json @@ -512,9 +512,6 @@ { "rpi.sharpen": { } }, - { - "rpi.focus": { } - }, { "rpi.af": { diff --git a/src/ipa/raspberrypi/data/imx708_noir.json b/src/ipa/raspberrypi/data/imx708_noir.json index 082274e34c53..075f70355cec 100644 --- a/src/ipa/raspberrypi/data/imx708_noir.json +++ b/src/ipa/raspberrypi/data/imx708_noir.json @@ -512,9 +512,6 @@ { "rpi.sharpen": { } }, - { - "rpi.focus": { } - }, { "rpi.af": { diff --git a/src/ipa/raspberrypi/data/imx708_wide.json b/src/ipa/raspberrypi/data/imx708_wide.json index cdc61436995d..b772efee3b96 100644 --- a/src/ipa/raspberrypi/data/imx708_wide.json +++ b/src/ipa/raspberrypi/data/imx708_wide.json @@ -403,9 +403,6 @@ { "rpi.sharpen": { } }, - { - "rpi.focus": { } - }, { "rpi.af": { diff --git a/src/ipa/raspberrypi/data/imx708_wide_noir.json b/src/ipa/raspberrypi/data/imx708_wide_noir.json index 8a7f59910833..c5f6b53dca7a 100644 --- a/src/ipa/raspberrypi/data/imx708_wide_noir.json +++ b/src/ipa/raspberrypi/data/imx708_wide_noir.json @@ -403,9 +403,6 @@ { "rpi.sharpen": { } }, - { - "rpi.focus": { } - }, { "rpi.af": { diff --git a/src/ipa/raspberrypi/meson.build b/src/ipa/raspberrypi/meson.build index 9230e17bca22..de78cbd80f9c 100644 --- a/src/ipa/raspberrypi/meson.build +++ b/src/ipa/raspberrypi/meson.build @@ -33,7 +33,6 @@ rpi_ipa_sources = files([ 'controller/rpi/awb.cpp', 'controller/rpi/sharpen.cpp', 'controller/rpi/black_level.cpp', - 'controller/rpi/focus.cpp', 'controller/rpi/geq.cpp', 'controller/rpi/noise.cpp', 'controller/rpi/lux.cpp', diff --git a/src/ipa/raspberrypi/raspberrypi.cpp b/src/ipa/raspberrypi/raspberrypi.cpp index e2662938c9d3..9f3f3ac3b15d 100644 --- a/src/ipa/raspberrypi/raspberrypi.cpp +++ b/src/ipa/raspberrypi/raspberrypi.cpp @@ -50,7 +50,6 @@ #include "denoise_algorithm.h" #include "denoise_status.h" #include "dpc_status.h" -#include "focus_status.h" #include "geq_status.h" #include "lux_status.h" #include "metadata.h" @@ -642,14 +641,28 @@ void IPARPi::reportMetadata(unsigned int ipaContext) static_cast(blackLevelStatus->blackLevelG), static_cast(blackLevelStatus->blackLevelB) }); - FocusStatus *focusStatus = rpiMetadata.getLocked("focus.status"); - if (focusStatus && focusStatus->num == 12) { + RPiController::FocusRegions *focusStatus = + rpiMetadata.getLocked("focus.status"); + if (focusStatus) { /* - * We get a 4x3 grid of regions by default. Calculate the average - * FoM over the central two positions to give an overall scene FoM. - * This can change later if it is not deemed suitable. + * Calculate the average FoM over the central (symmetric) positions + * to give an overall scene FoM. This can change later if it is + * not deemed suitable. */ - int32_t focusFoM = (focusStatus->focusMeasures[5] + focusStatus->focusMeasures[6]) / 2; + libcamera::Size size = focusStatus->size(); + unsigned rows = size.height; + unsigned cols = size.width; + + uint64_t sum = 0; + unsigned int numRegions = 0; + for (unsigned r = rows / 3; r < rows - rows / 3; ++r) { + for (unsigned c = cols / 4; c < cols - cols / 4; ++c) { + sum += focusStatus->get({ (int)c, (int)r }).val; + numRegions++; + } + } + + uint32_t focusFoM = (sum / numRegions) >> 16; libcameraMetadata_.set(controls::FocusFoM, focusFoM); } @@ -1428,7 +1441,6 @@ RPiController::StatisticsPtr IPARPi::fillStatistics(bcm2835_isp_stats *stats) co statistics->focusRegions.set(i, { stats->focus_stats[i].contrast_val[1][1] / 1000, stats->focus_stats[i].contrast_val_num[1][1], stats->focus_stats[i].contrast_val_num[1][0] }); - return statistics; } @@ -1445,6 +1457,10 @@ void IPARPi::processStats(unsigned int bufferId, unsigned int ipaContext) Span mem = it->second.planes()[0]; bcm2835_isp_stats *stats = reinterpret_cast(mem.data()); RPiController::StatisticsPtr statistics = fillStatistics(stats); + + /* Save the focus stats in the metadata structure to report out later. */ + rpiMetadata_[ipaContext].set("focus.status", statistics->focusRegions); + helper_->process(statistics, rpiMetadata); controller_.process(statistics, &rpiMetadata);