From patchwork Wed Mar 22 13:06:03 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Naushir Patuck X-Patchwork-Id: 18428 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 151AEC3260 for ; Wed, 22 Mar 2023 13:06:22 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 715CF62712; Wed, 22 Mar 2023 14:06:20 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1679490380; bh=NIKBqVYG6vvjAOK1iHAq4BUqgpfey+pFNIS1WFEVp10=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To: From; b=g7qWtwF6CpRXeVOV+20oxBxMdjZR2roOmvc0JCzxQ/ve87PCn7VGLuxeiBelIonAh lGVd917DaiMZZad6pS2T0sJ2RbC3hyRj9JTwbMc+PUCaVbbHAFQpQ6FrYu60XbwQa7 Lke/1+pBWAn1NxqerTX7FVjufQSbXkju/CWwaOJflls01zZUn3nYT9YEno3SiKAU5O jPLE8neolj1e9Ms3NUurHq5itDG7qerkXmy9frn2NqAoRxVUFOA2aGxuoaPxm1zSSU CMu2+Q2VUiAYcvBrkw9ydidtggHOh/WzYTU0V7RJH7veZHmfMNTxChdr7XKfCXoctV sDbDkOs0lXyEA== Received: from mail-io1-xd32.google.com (mail-io1-xd32.google.com [IPv6:2607:f8b0:4864:20::d32]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 98A3861ECE for ; Wed, 22 Mar 2023 14:06:18 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="LyhreHbA"; dkim-atps=neutral Received: by mail-io1-xd32.google.com with SMTP id s4so8385897ioj.11 for ; Wed, 22 Mar 2023 06:06:18 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; t=1679490377; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=Mg0Olb+1u+WM6x4FwKzljBeNwsaSSj8LW3QZkB2AKhw=; b=LyhreHbArJKyGb3bkyzwkNE4TXOFOpPbbXv+MwlxT6fezfmruxmPTgFKDp0C9N3iHQ z6avAR5h7kYOZLiJgFFU2MXrmwNd1C/ToAR1RDB6nW7Nt0jvT/Yj3bLHMLUJZOygOO1z tWveFDI8Bey7XqSHDCMqpnV0d3tRSJyd3vX1MTDa9UHwR8ndKiL7wYmLSGWWQiqI0Pi/ QIJGCR7IaS5F4uyLQmaEK3hEYtBCsZv94a+EV7V2nomgr3lTLoTjWkwRNHz/48QUEd70 YpnikqXXbtWW8rlVIUL/AffQUrsFbRI5WiY3tYS2WZLOLKhf7JSecKl4EMYU5AShcoS6 J50A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; t=1679490377; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=Mg0Olb+1u+WM6x4FwKzljBeNwsaSSj8LW3QZkB2AKhw=; b=wXmcfzyVZ9Db2AvQdj30dJ84sX/ipNb0VZbT5Pcj4LLvqA9D2fEKh6odz6micixiOK SLclXHls+CpWaXQnok0ET3iSwLXvDlmmyy4vx/tW+yH9A4rDmzec0LWRc7sFKlnQm7Xf 68oGZobJl7CTBuoNY5Mz0QLpz1h3nkFj9YF2pVMFCkJGd+HFWFRjw9bfLICjRf3fppAU yKiu0Y4sZluI2Pjvq4HZUbEos2sNYlhoF4SiSZa8KohKH83q3AQbhgofJY8wj13wWFqm jqIjZlLzaqG0/7DiZADhSC8a5tvCDybeFlhgP63wvtB1wjoCwaNASqPSa4rhSzSz3cop YS3A== X-Gm-Message-State: AO0yUKWo5bOyLUGDLYIOR9Cgb6OnB+bStkemSSIw7XpKsY2EFC4BXAeL M8r6OkAzug6oXjX+Nj/MJs0+YfqE5BxWrS8lkqVYaA== X-Google-Smtp-Source: AK7set8/g4ZpNMLIOiK5Jo1Oy0tBL6jgFFrhO67ZgvkwQDxi67QNeRhNlGVj8N4RTkVqDPskqkTBNg== X-Received: by 2002:a5e:aa0d:0:b0:74c:aa8f:a83e with SMTP id s13-20020a5eaa0d000000b0074caa8fa83emr4179450ioe.6.1679490377220; Wed, 22 Mar 2023 06:06:17 -0700 (PDT) Received: from localhost.localdomain ([93.93.133.154]) by smtp.gmail.com with ESMTPSA id l6-20020a026646000000b003c488204c6fsm4927960jaf.76.2023.03.22.06.06.16 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 22 Mar 2023 06:06:17 -0700 (PDT) To: libcamera-devel@lists.libcamera.org Date: Wed, 22 Mar 2023 13:06:03 +0000 Message-Id: <20230322130612.5208-2-naush@raspberrypi.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230322130612.5208-1-naush@raspberrypi.com> References: <20230322130612.5208-1-naush@raspberrypi.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v1 01/10] ipa: raspberrypi Store the target string in the controller X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Naushir Patuck via libcamera-devel From: Naushir Patuck Reply-To: Naushir Patuck Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" The target string may be used by algorithms to determine the running hardware target. Store the target string provided by the camera tuning files in the controller state. Add a getTarget() member function to retrieve this string. Validate the correct hardware target ("bcm2835") during the IPA initialisation phase. Signed-off-by: Naushir Patuck Reviewed-by: David Plowman Reviewed-by: Jacopo Mondi --- src/ipa/raspberrypi/controller/algorithm.h | 4 ++++ src/ipa/raspberrypi/controller/controller.cpp | 6 ++++++ src/ipa/raspberrypi/controller/controller.h | 4 ++++ src/ipa/raspberrypi/raspberrypi.cpp | 8 ++++++++ 4 files changed, 22 insertions(+) diff --git a/src/ipa/raspberrypi/controller/algorithm.h b/src/ipa/raspberrypi/controller/algorithm.h index 4f3275987305..7c22fbe4945c 100644 --- a/src/ipa/raspberrypi/controller/algorithm.h +++ b/src/ipa/raspberrypi/controller/algorithm.h @@ -41,6 +41,10 @@ public: { return controller_->getGlobalMetadata(); } + const std::string &getTarget() const + { + return controller_->getTarget(); + } private: Controller *controller_; diff --git a/src/ipa/raspberrypi/controller/controller.cpp b/src/ipa/raspberrypi/controller/controller.cpp index e91567852d00..a6250ee140b0 100644 --- a/src/ipa/raspberrypi/controller/controller.cpp +++ b/src/ipa/raspberrypi/controller/controller.cpp @@ -38,6 +38,7 @@ int Controller::read(char const *filename) std::unique_ptr root = YamlParser::parse(file); double version = (*root)["version"].get(1.0); + target_ = (*root)["target"].get("bcm2835"); if (version < 2.0) { LOG(RPiController, Warning) @@ -142,3 +143,8 @@ Algorithm *Controller::getAlgorithm(std::string const &name) const } return nullptr; } + +const std::string &Controller::getTarget() const +{ + return target_; +} diff --git a/src/ipa/raspberrypi/controller/controller.h b/src/ipa/raspberrypi/controller/controller.h index e6c950c3a509..24e02903d438 100644 --- a/src/ipa/raspberrypi/controller/controller.h +++ b/src/ipa/raspberrypi/controller/controller.h @@ -46,6 +46,7 @@ public: void process(StatisticsPtr stats, Metadata *imageMetadata); Metadata &getGlobalMetadata(); Algorithm *getAlgorithm(std::string const &name) const; + const std::string &getTarget() const; protected: int createAlgorithm(const std::string &name, const libcamera::YamlObject ¶ms); @@ -53,6 +54,9 @@ protected: Metadata globalMetadata_; std::vector algorithms_; bool switchModeCalled_; + +private: + std::string target_; }; } /* namespace RPiController */ diff --git a/src/ipa/raspberrypi/raspberrypi.cpp b/src/ipa/raspberrypi/raspberrypi.cpp index 1375795568e2..86359538cf67 100644 --- a/src/ipa/raspberrypi/raspberrypi.cpp +++ b/src/ipa/raspberrypi/raspberrypi.cpp @@ -269,6 +269,14 @@ int IPARPi::init(const IPASettings &settings, bool lensPresent, IPAInitResult *r return ret; } + const std::string &target = controller_.getTarget(); + if (target != "bcm2835") { + LOG(IPARPI, Error) + << "Tuning data file target returned \"" << target << "\"" + << ", expected \"bcm2835\""; + return -EINVAL; + } + lensPresent_ = lensPresent; controller_.initialise(); From patchwork Wed Mar 22 13:06:04 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Naushir Patuck X-Patchwork-Id: 18429 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 594F6C329C for ; Wed, 22 Mar 2023 13:06:24 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id A0B9E626E6; Wed, 22 Mar 2023 14:06:23 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1679490383; bh=J1uasa5/tJAHTXqgL1/mM0u5Q3ArfZnJ87UFIogFh88=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To: From; b=eB6wyYSwASaqkoh5yo+6u5o7Xy9uDevIoi+dvET34BGZre92BbZgrM2G9t3OTvPX9 3QKhDKX7E2vPLKzuLoWuB7alIVT9l2Fh8Et0jLmf2qQMwOn/gKQXZE7SVn0vXevGtc OyJIGU5uSKC5Oq72EF1F0Ieb6kkzaFAvJPyLngCM5w5syxWQ79NT0oI7Efa6BQRBF/ EhNTlP36RcQvRiG4igksdmcJsMfRSZXYwlWneKZcjq7APkdGaPB4d4/9LjgQMhMQEX Vru2AAo+1KzXdz/vDaz4lTBlDWtBN8l/jFyZXUCvrO0PCPyYAOZXj4W8KJH0L130Xj VSlFCL31vwHAw== Received: from mail-io1-xd2e.google.com (mail-io1-xd2e.google.com [IPv6:2607:f8b0:4864:20::d2e]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 991FE61ECE for ; Wed, 22 Mar 2023 14:06:20 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="Gg35Co84"; dkim-atps=neutral Received: by mail-io1-xd2e.google.com with SMTP id m22so8406911ioy.4 for ; Wed, 22 Mar 2023 06:06:20 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; t=1679490379; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=xf8pOLOBSpsB8lB4jAOqwlWtCYuXLJMzQzDn0iSf5F4=; b=Gg35Co84B+VcmfBaHoNjKgzVITDHGRKcaHlvsNqQjIt6XSTov1FZmbDO2749Lj72ze j4mRbL8ikyHrlIxbHhaUkt83l46SaVOJVJWGQMx6VpiGbJxh6ANnIS0HQxW35d5ZAG/h Lqk/Lo8cvH3IkAQeHfpgj0VJq8gQGaz/2RMYmbN1Nl1sy8mfsAFAo2xe9fb0Z74BAXhj fWA6jdki/1imqLQL4RE+uQDcH/hKjy4fD6cn7JKvsEBit/BOGvGU/f85v2/BNzfSdD3U raTnOcBrNTD3WbGoT0SEtGgBH/HyqGq+UMuIcOnfEnd++CvLwLGkU+dm1mfv0TcQw8hs dRrw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; t=1679490379; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=xf8pOLOBSpsB8lB4jAOqwlWtCYuXLJMzQzDn0iSf5F4=; b=tabPjjIN9MXcwyljU8QUJ+j7kV09OvXtlSohrH+3DZpshIOBg7ux1FpBU5+ThhMj6q kjynq+mIf2coyLcRJk04FpZTZ0133ILxazUU3WHbd0RVTq4ixvcavZINTl324dBm1VX1 U8PAVs0R0N9wsG8Kc7xe1kaL34vlFRu+jckRVVRSOciLLW5Pg9xOtbYjWxJlz0aiN9DR 0JSkwG0AzkDsEeOXLcCP5o0gturTiXUgEMIcDov3LxLhdXtUBxNYsVJFGtI62Q78M99l Xr9LC487PJ/LQnhIW9j4dEbhaIFW+3y8t5h2Y6m3/2JCedBjFVNdfcCPY7qrKdxq/d0a FMEA== X-Gm-Message-State: AO0yUKXkpUfTuxhxKsV48iWIFo6dJTpW55h0pI8vCvPsFgvf23zdf9LM 71Lf4l9nJZjSCNeEB+g5HCIkSQWodrZk8S/tAWhs/Q== X-Google-Smtp-Source: AK7set/BYwWUWicJqbLAYhwHGgTtVwF9Ow62denmkmsP9LljcShdzMNlWsvUYTquF/jUdYeSj2ShJQ== X-Received: by 2002:a05:6602:1616:b0:752:6f1b:63a2 with SMTP id x22-20020a056602161600b007526f1b63a2mr2043043iow.3.1679490378288; Wed, 22 Mar 2023 06:06:18 -0700 (PDT) Received: from localhost.localdomain ([93.93.133.154]) by smtp.gmail.com with ESMTPSA id l6-20020a026646000000b003c488204c6fsm4927960jaf.76.2023.03.22.06.06.17 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 22 Mar 2023 06:06:18 -0700 (PDT) To: libcamera-devel@lists.libcamera.org Date: Wed, 22 Mar 2023 13:06:04 +0000 Message-Id: <20230322130612.5208-3-naush@raspberrypi.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230322130612.5208-1-naush@raspberrypi.com> References: <20230322130612.5208-1-naush@raspberrypi.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v1 02/10] ipa: raspberrypi: Add hardware configuration to the controller X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Naushir Patuck via libcamera-devel From: Naushir Patuck Reply-To: Naushir Patuck Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Add a new Controller::HardwareConfig structure that captures the hardware statistics grid/histogram sizes and pipeline widths. This ensures there is a single centralised places for these parameters. Add a getHardwareConfig() helper function to retrieve these values for a given hardware target. Update the statistics populating routine in the IPA to use the values from this structure instead of the hardcoded numbers. Signed-off-by: Naushir Patuck Reviewed-by: David Plowman --- src/ipa/raspberrypi/controller/algorithm.h | 4 ++ src/ipa/raspberrypi/controller/controller.cpp | 39 +++++++++++++++++++ src/ipa/raspberrypi/controller/controller.h | 11 ++++++ src/ipa/raspberrypi/raspberrypi.cpp | 21 ++++------ 4 files changed, 62 insertions(+), 13 deletions(-) diff --git a/src/ipa/raspberrypi/controller/algorithm.h b/src/ipa/raspberrypi/controller/algorithm.h index 7c22fbe4945c..4aa814ebbebd 100644 --- a/src/ipa/raspberrypi/controller/algorithm.h +++ b/src/ipa/raspberrypi/controller/algorithm.h @@ -45,6 +45,10 @@ public: { return controller_->getTarget(); } + const Controller::HardwareConfig &getHardwareConfig() const + { + return controller_->getHardwareConfig(); + } private: Controller *controller_; diff --git a/src/ipa/raspberrypi/controller/controller.cpp b/src/ipa/raspberrypi/controller/controller.cpp index a6250ee140b0..2c7517aec6b4 100644 --- a/src/ipa/raspberrypi/controller/controller.cpp +++ b/src/ipa/raspberrypi/controller/controller.cpp @@ -20,6 +20,31 @@ using namespace libcamera; LOG_DEFINE_CATEGORY(RPiController) +static const std::map HardwareConfigMap = { + { + "bcm2835", + { + /* + * There are only ever 15 AGC regions computed by the firmware + * due to zoning, but the HW defines AGC_REGIONS == 16! + */ + .agcRegions = { 15 , 1 }, + .agcZoneWeights = { 15 , 1 }, + .awbRegions = { 16, 12 }, + .focusRegions = { 4, 3 }, + .numHistogramBins = 128, + .numGammaPoints = 33, + .pipelineWidth = 13 + } + }, + /* For error handling. */ + { + "error", + { + } + } +}; + Controller::Controller() : switchModeCalled_(false) { @@ -148,3 +173,17 @@ const std::string &Controller::getTarget() const { return target_; } + +const Controller::HardwareConfig &Controller::getHardwareConfig() const +{ + auto cfg = HardwareConfigMap.find(getTarget()); + + /* + * This really should not happen, the IPA ought to validate the target + * on initialisation. + */ + if (cfg == HardwareConfigMap.end()) + return HardwareConfigMap.at("error"); + + return cfg->second; +} diff --git a/src/ipa/raspberrypi/controller/controller.h b/src/ipa/raspberrypi/controller/controller.h index 24e02903d438..c6af5cd6c7d4 100644 --- a/src/ipa/raspberrypi/controller/controller.h +++ b/src/ipa/raspberrypi/controller/controller.h @@ -37,6 +37,16 @@ typedef std::unique_ptr AlgorithmPtr; class Controller { public: + struct HardwareConfig { + libcamera::Size agcRegions; + libcamera::Size agcZoneWeights; + libcamera::Size awbRegions; + libcamera::Size focusRegions; + unsigned int numHistogramBins; + unsigned int numGammaPoints; + unsigned int pipelineWidth; + }; + Controller(); ~Controller(); int read(char const *filename); @@ -47,6 +57,7 @@ public: Metadata &getGlobalMetadata(); Algorithm *getAlgorithm(std::string const &name) const; const std::string &getTarget() const; + const HardwareConfig &getHardwareConfig() const; protected: int createAlgorithm(const std::string &name, const libcamera::YamlObject ¶ms); diff --git a/src/ipa/raspberrypi/raspberrypi.cpp b/src/ipa/raspberrypi/raspberrypi.cpp index 86359538cf67..b64cb96e2dde 100644 --- a/src/ipa/raspberrypi/raspberrypi.cpp +++ b/src/ipa/raspberrypi/raspberrypi.cpp @@ -1392,20 +1392,19 @@ RPiController::StatisticsPtr IPARPi::fillStatistics(bcm2835_isp_stats *stats) co { using namespace RPiController; + const Controller::HardwareConfig &hw = controller_.getHardwareConfig(); unsigned int i; StatisticsPtr statistics = std::make_unique(Statistics::AgcStatsPos::PreWb, Statistics::ColourStatsPos::PostLsc); /* RGB histograms are not used, so do not populate them. */ - statistics->yHist = RPiController::Histogram(stats->hist[0].g_hist, NUM_HISTOGRAM_BINS); + statistics->yHist = RPiController::Histogram(stats->hist[0].g_hist, + hw.numHistogramBins); - /* - * All region sums are based on a 13-bit pipeline bit-depth. Normalise - * this to 16-bits for the AGC/AWB/ALSC algorithms. - */ - constexpr unsigned int scale = Statistics::NormalisationFactorPow2 - 13; + /* All region sums are based on a 16-bit normalised pipeline bit-depth. */ + unsigned int scale = Statistics::NormalisationFactorPow2 - hw.pipelineWidth; - statistics->awbRegions.init({ DEFAULT_AWB_REGIONS_X, DEFAULT_AWB_REGIONS_Y }); + statistics->awbRegions.init(hw.awbRegions); for (i = 0; i < statistics->awbRegions.numRegions(); i++) statistics->awbRegions.set(i, { { stats->awb_stats[i].r_sum << scale, stats->awb_stats[i].g_sum << scale, @@ -1413,11 +1412,7 @@ RPiController::StatisticsPtr IPARPi::fillStatistics(bcm2835_isp_stats *stats) co stats->awb_stats[i].counted, stats->awb_stats[i].notcounted }); - /* - * There are only ever 15 regions computed by the firmware due to zoning, - * but the HW defines AGC_REGIONS == 16! - */ - statistics->agcRegions.init(15); + statistics->agcRegions.init(hw.agcRegions); for (i = 0; i < statistics->agcRegions.numRegions(); i++) statistics->agcRegions.set(i, { { stats->agc_stats[i].r_sum << scale, stats->agc_stats[i].g_sum << scale, @@ -1425,7 +1420,7 @@ RPiController::StatisticsPtr IPARPi::fillStatistics(bcm2835_isp_stats *stats) co stats->agc_stats[i].counted, stats->awb_stats[i].notcounted }); - statistics->focusRegions.init({ 4, 3 }); + statistics->focusRegions.init(hw.focusRegions); for (i = 0; i < statistics->focusRegions.numRegions(); i++) statistics->focusRegions.set(i, { stats->focus_stats[i].contrast_val[1][1] / 1000, stats->focus_stats[i].contrast_val_num[1][1], From patchwork Wed Mar 22 13:06:05 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Naushir Patuck X-Patchwork-Id: 18430 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id A892FC329D for ; Wed, 22 Mar 2023 13:06:24 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 4553862719; Wed, 22 Mar 2023 14:06:24 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1679490384; bh=mPek/zScnAmKzPCIA0OSpmdJ46HnVIBuWjdDOsuGbWY=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To: From; b=mf89I1zF3PZEBdJk5RLYgkuEoRLn9HNzadm2zrRfFYlyARyH3y2Z4+xw8lG3XXfyW 4Dm5kbpDwZy9dXnr1dLJ75B86P8KGezPQCg0JJ2YS2szekxMidpzPLN5CKicJyuc6J npQsmZyvxqFEN8gKbV9GXt0Lqx/LBRxmZ3FP6QvhVRE8PeRVuCsODihPPegRgukSG0 38W2du6hm+ESeGn7ffZInT/9M+wBUymDev47G4xFCMf7PXlVeI6iyrnIr62bWtam/t wD8h6Oi3pctwKBEOeCyxJnPAC2wM11lhh0+ak086/O7qdaUNwtvJj7UM8DBNxvpTHo DVuE6ZjGEnScQ== Received: from mail-io1-xd2b.google.com (mail-io1-xd2b.google.com [IPv6:2607:f8b0:4864:20::d2b]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 4E7B7626E6 for ; Wed, 22 Mar 2023 14:06:21 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="gfLu5rUK"; dkim-atps=neutral Received: by mail-io1-xd2b.google.com with SMTP id d14so5520308ion.9 for ; Wed, 22 Mar 2023 06:06:21 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; t=1679490380; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=9gtWlJjnVtYPCba1bV1a8ZlAMg1AKXhIDUnb9+Ws+Y8=; b=gfLu5rUKUbXi9PFKbYCEKUlf9R3HPpcqVwmrlvQyDiZuetSwnfaNBVoWWrGC+XDXah jM8yGJkqvYfLAPx+K63MoMme0SmKeo+Efxg/xkEvNtkgcPq/e0aU0Uduug3uN8l8bGPQ 7zP2pQDIJxzATSluODUBMhdqm9NWsiCA9EGEsf9uP9UwC6HtgNFaSZ47CGwYamUBNHgt ui7UAid77whMFiAlcJKaxpveIj0SWf9soBOmSz6ko4eqff+JFx8JRjuoRXbDc5ru4ACl BMlduv94iDMhhe1Ra0sssuEt1DoTlz/Vbw2HmuiN0dHiTSthi02kXVKea+gzTSwQUphb mxVQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; t=1679490381; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=9gtWlJjnVtYPCba1bV1a8ZlAMg1AKXhIDUnb9+Ws+Y8=; b=M9TV8Su9Mh8M3KY1a53o3MHJzSponl3tutKRbMhF5WqaN5nN8efpk5oWAY1BVJuub5 yaR5o3h8faFyoyp4Ob2hBXZ/Uz73Ixka6baHn1b6lY/u6n+a6bjkgS9P2eMl+1uPXEC2 7Dyp8OJbeAXQ61oGqoSQjLi/zk9u+MoyrJ7aDect8apR73q/wTzxRa3aZvXpLVYcDBBg /qyUM4jteUhGrcFu2PaciSPKeRZllldD9ZR99C2AZp8FWLDWfsdHlaHohyUzRLIxdKyn bqSBcOshicMgHHnWcNs7dwxMWx+qPEBaVXz7IVllWGQLgQkwaXPxrCCNTqqD3sPseasZ BY0g== X-Gm-Message-State: AO0yUKWgajvrOIbQIlvYvTDRLjboNEH2SJcVLbRU9+qHBacXTyO9OF2P tjNpSEvQYL2zg5eg4JMfNculbtzAc6pRZ2ouMx9R0g== X-Google-Smtp-Source: AK7set+kYSzpatqQKSSb1bf+hFJdId2tEFqpsNryf690vvdUjU5d8DgLGHb4EvCWQRCvWtCSiOT7Rw== X-Received: by 2002:a5e:de45:0:b0:753:2862:a2c0 with SMTP id e5-20020a5ede45000000b007532862a2c0mr4090564ioq.21.1679490380094; Wed, 22 Mar 2023 06:06:20 -0700 (PDT) Received: from localhost.localdomain ([93.93.133.154]) by smtp.gmail.com with ESMTPSA id l6-20020a026646000000b003c488204c6fsm4927960jaf.76.2023.03.22.06.06.19 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 22 Mar 2023 06:06:19 -0700 (PDT) To: libcamera-devel@lists.libcamera.org Date: Wed, 22 Mar 2023 13:06:05 +0000 Message-Id: <20230322130612.5208-4-naush@raspberrypi.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230322130612.5208-1-naush@raspberrypi.com> References: <20230322130612.5208-1-naush@raspberrypi.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v1 03/10] ipa: raspberrypi: Generalise the ALSC algorithm X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Naushir Patuck via libcamera-devel From: Naushir Patuck Reply-To: Naushir Patuck Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Remove any hard-coded assumptions about the target hardware platform from the ALSC algorithm. Instead, use the "target" string provided by the camera tuning config and generalised statistics structures to determing parameters such as grid and region sizes. The ALSC calculations use run-time allocated arrays/vectors on every frame. Allocating these might add a non-trivial run-time penalty. Replace these dynamic allocations with a set of reusable pre-allocated vectors during the init phase. Signed-off-by: Naushir Patuck Signed-off-by: David Plowman --- src/ipa/raspberrypi/controller/alsc_status.h | 13 +- src/ipa/raspberrypi/controller/rpi/alsc.cpp | 341 +++++++++++-------- src/ipa/raspberrypi/controller/rpi/alsc.h | 29 +- src/ipa/raspberrypi/raspberrypi.cpp | 9 +- 4 files changed, 224 insertions(+), 168 deletions(-) diff --git a/src/ipa/raspberrypi/controller/alsc_status.h b/src/ipa/raspberrypi/controller/alsc_status.h index e5aa7e37c330..49a9f4a0cb5a 100644 --- a/src/ipa/raspberrypi/controller/alsc_status.h +++ b/src/ipa/raspberrypi/controller/alsc_status.h @@ -6,16 +6,17 @@ */ #pragma once +#include + /* * The ALSC algorithm should post the following structure into the image's * "alsc.status" metadata. */ -constexpr unsigned int AlscCellsX = 16; -constexpr unsigned int AlscCellsY = 12; - struct AlscStatus { - double r[AlscCellsY][AlscCellsX]; - double g[AlscCellsY][AlscCellsX]; - double b[AlscCellsY][AlscCellsX]; + std::vector r; + std::vector g; + std::vector b; + unsigned int rows; + unsigned int cols; }; diff --git a/src/ipa/raspberrypi/controller/rpi/alsc.cpp b/src/ipa/raspberrypi/controller/rpi/alsc.cpp index eb4e2f9496e1..51fe5d73f52d 100644 --- a/src/ipa/raspberrypi/controller/rpi/alsc.cpp +++ b/src/ipa/raspberrypi/controller/rpi/alsc.cpp @@ -5,6 +5,7 @@ * alsc.cpp - ALSC (auto lens shading correction) control algorithm */ +#include #include #include #include @@ -24,9 +25,6 @@ LOG_DEFINE_CATEGORY(RPiAlsc) #define NAME "rpi.alsc" -static const int X = AlscCellsX; -static const int Y = AlscCellsY; -static const int XY = X * Y; static const double InsufficientData = -1.0; Alsc::Alsc(Controller *controller) @@ -51,8 +49,11 @@ char const *Alsc::name() const return NAME; } -static int generateLut(double *lut, const libcamera::YamlObject ¶ms) +static int generateLut(std::vector &lut, const libcamera::YamlObject ¶ms, + const Size &size) { + /* These must be signed ints for the co-ordinate calculations below. */ + int X = size.width, Y = size.height; double cstrength = params["corner_strength"].get(2.0); if (cstrength <= 1.0) { LOG(RPiAlsc, Error) << "corner_strength must be > 1.0"; @@ -81,9 +82,9 @@ static int generateLut(double *lut, const libcamera::YamlObject ¶ms) return 0; } -static int readLut(double *lut, const libcamera::YamlObject ¶ms) +static int readLut(std::vector &lut, const libcamera::YamlObject ¶ms, const Size &size) { - if (params.size() != XY) { + if (params.size() != size.width * size.height) { LOG(RPiAlsc, Error) << "Invalid number of entries in LSC table"; return -EINVAL; } @@ -101,7 +102,7 @@ static int readLut(double *lut, const libcamera::YamlObject ¶ms) static int readCalibrations(std::vector &calibrations, const libcamera::YamlObject ¶ms, - std::string const &name) + std::string const &name, const Size &size) { if (params.contains(name)) { double lastCt = 0; @@ -119,7 +120,7 @@ static int readCalibrations(std::vector &calibrations, calibration.ct = lastCt = ct; const libcamera::YamlObject &table = p["table"]; - if (table.size() != XY) { + if (table.size() != size.width * size.height) { LOG(RPiAlsc, Error) << "Incorrect number of values for ct " << ct << " in " << name; @@ -127,6 +128,7 @@ static int readCalibrations(std::vector &calibrations, } int num = 0; + calibration.table.resize(size.width * size.height); for (const auto &elem : table.asList()) { value = elem.get(); if (!value) @@ -134,7 +136,7 @@ static int readCalibrations(std::vector &calibrations, calibration.table[num++] = *value; } - calibrations.push_back(calibration); + calibrations.push_back(std::move(calibration)); LOG(RPiAlsc, Debug) << "Read " << name << " calibration for ct " << ct; } @@ -144,6 +146,7 @@ static int readCalibrations(std::vector &calibrations, int Alsc::read(const libcamera::YamlObject ¶ms) { + config_.tableSize = getHardwareConfig().awbRegions; config_.framePeriod = params["frame_period"].get(12); config_.startupFrames = params["startup_frames"].get(10); config_.speed = params["speed"].get(0.05); @@ -153,28 +156,29 @@ int Alsc::read(const libcamera::YamlObject ¶ms) config_.minCount = params["min_count"].get(10.0); config_.minG = params["min_G"].get(50); config_.omega = params["omega"].get(1.3); - config_.nIter = params["n_iter"].get(X + Y); + config_.nIter = params["n_iter"].get(config_.tableSize.width + config_.tableSize.height); config_.luminanceStrength = params["luminance_strength"].get(1.0); - for (int i = 0; i < XY; i++) - config_.luminanceLut[i] = 1.0; + config_.luminanceLut.resize(config_.tableSize.width * config_.tableSize.height, 1.0); int ret = 0; if (params.contains("corner_strength")) - ret = generateLut(config_.luminanceLut, params); + ret = generateLut(config_.luminanceLut, params, config_.tableSize); else if (params.contains("luminance_lut")) - ret = readLut(config_.luminanceLut, params["luminance_lut"]); + ret = readLut(config_.luminanceLut, params["luminance_lut"], config_.tableSize); else LOG(RPiAlsc, Warning) << "no luminance table - assume unity everywhere"; if (ret) return ret; - ret = readCalibrations(config_.calibrationsCr, params, "calibrations_Cr"); + ret = readCalibrations(config_.calibrationsCr, params, "calibrations_Cr", + config_.tableSize); if (ret) return ret; - ret = readCalibrations(config_.calibrationsCb, params, "calibrations_Cb"); + ret = readCalibrations(config_.calibrationsCb, params, "calibrations_Cb", + config_.tableSize); if (ret) return ret; @@ -187,13 +191,16 @@ int Alsc::read(const libcamera::YamlObject ¶ms) static double getCt(Metadata *metadata, double defaultCt); static void getCalTable(double ct, std::vector const &calibrations, - double calTable[XY]); -static void resampleCalTable(double const calTableIn[XY], CameraMode const &cameraMode, - double calTableOut[XY]); -static void compensateLambdasForCal(double const calTable[XY], double const oldLambdas[XY], - double newLambdas[XY]); -static void addLuminanceToTables(double results[3][Y][X], double const lambdaR[XY], double lambdaG, - double const lambdaB[XY], double const luminanceLut[XY], + std::vector &calTable); +static void resampleCalTable(const std::vector &calTableIn, CameraMode const &cameraMode, + const Size &size, std::vector &calTableOut); +static void compensateLambdasForCal(const std::vector &calTable, + const std::vector &oldLambdas, + std::vector &newLambdas); +static void addLuminanceToTables(std::array, 3> &results, + const std::vector &lambdaR, double lambdaG, + const std::vector &lambdaB, + const std::vector &luminanceLut, double luminanceStrength); void Alsc::initialise() @@ -201,7 +208,28 @@ void Alsc::initialise() frameCount2_ = frameCount_ = framePhase_ = 0; firstTime_ = true; ct_ = config_.defaultCt; + + const size_t XY = config_.tableSize.width * config_.tableSize.height; + + for (auto &r : syncResults_) + r.resize(XY); + for (auto &r : prevSyncResults_) + r.resize(XY); + for (auto &r : asyncResults_) + r.resize(XY); + + luminanceTable_.resize(XY); + asyncLambdaR_.resize(XY); + asyncLambdaB_.resize(XY); /* The lambdas are initialised in the SwitchMode. */ + lambdaR_.resize(XY); + lambdaB_.resize(XY); + + /* Temporaries for the computations, but sensible to allocate this up-front! */ + for (auto &c : tmpC_) + c.resize(XY); + for (auto &m : tmpM_) + m.resize(XY); } void Alsc::waitForAysncThread() @@ -262,7 +290,7 @@ void Alsc::switchMode(CameraMode const &cameraMode, * We must resample the luminance table like we do the others, but it's * fixed so we can simply do it up front here. */ - resampleCalTable(config_.luminanceLut, cameraMode_, luminanceTable_); + resampleCalTable(config_.luminanceLut, cameraMode_, config_.tableSize, luminanceTable_); if (resetTables) { /* @@ -272,18 +300,18 @@ void Alsc::switchMode(CameraMode const &cameraMode, * the lambdas, but the rest of this code then echoes the code in * doAlsc, without the adaptive algorithm. */ - for (int i = 0; i < XY; i++) - lambdaR_[i] = lambdaB_[i] = 1.0; - double calTableR[XY], calTableB[XY], calTableTmp[XY]; + std::fill(lambdaR_.begin(), lambdaR_.end(), 1.0); + std::fill(lambdaB_.begin(), lambdaB_.end(), 1.0); + std::vector &calTableR = tmpC_[0], &calTableB = tmpC_[1], &calTableTmp = tmpC_[2]; getCalTable(ct_, config_.calibrationsCr, calTableTmp); - resampleCalTable(calTableTmp, cameraMode_, calTableR); + resampleCalTable(calTableTmp, cameraMode_, config_.tableSize, calTableR); getCalTable(ct_, config_.calibrationsCb, calTableTmp); - resampleCalTable(calTableTmp, cameraMode_, calTableB); + resampleCalTable(calTableTmp, cameraMode_, config_.tableSize, calTableB); compensateLambdasForCal(calTableR, lambdaR_, asyncLambdaR_); compensateLambdasForCal(calTableB, lambdaB_, asyncLambdaB_); addLuminanceToTables(syncResults_, asyncLambdaR_, 1.0, asyncLambdaB_, luminanceTable_, config_.luminanceStrength); - memcpy(prevSyncResults_, syncResults_, sizeof(prevSyncResults_)); + prevSyncResults_ = syncResults_; framePhase_ = config_.framePeriod; /* run the algo again asap */ firstTime_ = false; } @@ -294,7 +322,7 @@ void Alsc::fetchAsyncResults() LOG(RPiAlsc, Debug) << "Fetch ALSC results"; asyncFinished_ = false; asyncStarted_ = false; - memcpy(syncResults_, asyncResults_, sizeof(syncResults_)); + syncResults_ = asyncResults_; } double getCt(Metadata *metadata, double defaultCt) @@ -316,9 +344,9 @@ static void copyStats(RgbyRegions ®ions, StatisticsPtr &stats, if (!regions.numRegions()) regions.init(stats->awbRegions.size()); - double *rTable = (double *)status.r; - double *gTable = (double *)status.g; - double *bTable = (double *)status.b; + const std::vector &rTable = status.r; + const std::vector &gTable = status.g; + const std::vector &bTable = status.b; for (unsigned int i = 0; i < stats->awbRegions.numRegions(); i++) { auto r = stats->awbRegions.get(i); r.val.rSum = static_cast(r.val.rSum / rTable[i]); @@ -344,12 +372,9 @@ void Alsc::restartAsync(StatisticsPtr &stats, Metadata *imageMetadata) if (imageMetadata->get("alsc.status", alscStatus) != 0) { LOG(RPiAlsc, Warning) << "No ALSC status found for applied gains!"; - for (int y = 0; y < Y; y++) - for (int x = 0; x < X; x++) { - alscStatus.r[y][x] = 1.0; - alscStatus.g[y][x] = 1.0; - alscStatus.b[y][x] = 1.0; - } + alscStatus.r.resize(config_.tableSize.width * config_.tableSize.height, 1.0); + alscStatus.g.resize(config_.tableSize.width * config_.tableSize.height, 1.0); + alscStatus.b.resize(config_.tableSize.width * config_.tableSize.height, 1.0); } copyStats(statistics_, stats, alscStatus); framePhase_ = 0; @@ -380,15 +405,15 @@ void Alsc::prepare(Metadata *imageMetadata) fetchAsyncResults(); } /* Apply IIR filter to results and program into the pipeline. */ - double *ptr = (double *)syncResults_, - *pptr = (double *)prevSyncResults_; - for (unsigned int i = 0; i < sizeof(syncResults_) / sizeof(double); i++) - pptr[i] = speed * ptr[i] + (1.0 - speed) * pptr[i]; + for (unsigned int j = 0; j < syncResults_.size(); j++) { + for (unsigned int i = 0; i < syncResults_[j].size(); i++) + prevSyncResults_[j][i] = speed * syncResults_[j][i] + (1.0 - speed) * prevSyncResults_[j][i]; + } /* Put output values into status metadata. */ AlscStatus status; - memcpy(status.r, prevSyncResults_[0], sizeof(status.r)); - memcpy(status.g, prevSyncResults_[1], sizeof(status.g)); - memcpy(status.b, prevSyncResults_[2], sizeof(status.b)); + status.r = prevSyncResults_[0]; + status.g = prevSyncResults_[1]; + status.b = prevSyncResults_[2]; imageMetadata->set("alsc.status", status); } @@ -432,18 +457,17 @@ void Alsc::asyncFunc() } void getCalTable(double ct, std::vector const &calibrations, - double calTable[XY]) + std::vector &calTable) { if (calibrations.empty()) { - for (int i = 0; i < XY; i++) - calTable[i] = 1.0; + std::fill(calTable.begin(), calTable.end(), 1.0); LOG(RPiAlsc, Debug) << "no calibrations found"; } else if (ct <= calibrations.front().ct) { - memcpy(calTable, calibrations.front().table, XY * sizeof(double)); + calTable = calibrations.front().table; LOG(RPiAlsc, Debug) << "using calibration for " << calibrations.front().ct; } else if (ct >= calibrations.back().ct) { - memcpy(calTable, calibrations.back().table, XY * sizeof(double)); + calTable = calibrations.back().table; LOG(RPiAlsc, Debug) << "using calibration for " << calibrations.back().ct; } else { @@ -454,7 +478,7 @@ void getCalTable(double ct, std::vector const &calibrations, LOG(RPiAlsc, Debug) << "ct is " << ct << ", interpolating between " << ct0 << " and " << ct1; - for (int i = 0; i < XY; i++) + for (unsigned int i = 0; i < calTable.size(); i++) calTable[i] = (calibrations[idx].table[i] * (ct1 - ct) + calibrations[idx + 1].table[i] * (ct - ct0)) / @@ -462,9 +486,13 @@ void getCalTable(double ct, std::vector const &calibrations, } } -void resampleCalTable(double const calTableIn[XY], - CameraMode const &cameraMode, double calTableOut[XY]) +void resampleCalTable(const std::vector &calTableIn, + CameraMode const &cameraMode, const Size &size, + std::vector &calTableOut) { + int X = size.width; + int Y = size.height; + /* * Precalculate and cache the x sampling locations and phases to save * recomputing them on every row. @@ -501,23 +529,24 @@ void resampleCalTable(double const calTableIn[XY], yLo = Y - 1 - yLo; yHi = Y - 1 - yHi; } - double const *rowAbove = calTableIn + X * yLo; - double const *rowBelow = calTableIn + X * yHi; + double const *rowAbove = calTableIn.data() + X * yLo; + double const *rowBelow = calTableIn.data() + X * yHi; + double *out = calTableOut.data() + X * j; for (int i = 0; i < X; i++) { double above = rowAbove[xLo[i]] * (1 - xf[i]) + rowAbove[xHi[i]] * xf[i]; double below = rowBelow[xLo[i]] * (1 - xf[i]) + rowBelow[xHi[i]] * xf[i]; - *(calTableOut++) = above * (1 - yf) + below * yf; + *(out++) = above * (1 - yf) + below * yf; } } } /* Calculate chrominance statistics (R/G and B/G) for each region. */ -static void calculateCrCb(const RgbyRegions &awbRegion, double cr[XY], - double cb[XY], uint32_t minCount, uint16_t minG) +static void calculateCrCb(const RgbyRegions &awbRegion, std::vector &cr, + std::vector &cb, uint32_t minCount, uint16_t minG) { - for (int i = 0; i < XY; i++) { + for (unsigned int i = 0; i < cr.size(); i++) { auto s = awbRegion.get(i); if (s.counted <= minCount || s.val.gSum / s.counted <= minG) { @@ -530,33 +559,34 @@ static void calculateCrCb(const RgbyRegions &awbRegion, double cr[XY], } } -static void applyCalTable(double const calTable[XY], double C[XY]) +static void applyCalTable(const std::vector &calTable, std::vector &C) { - for (int i = 0; i < XY; i++) + for (unsigned int i = 0; i < C.size(); i++) if (C[i] != InsufficientData) C[i] *= calTable[i]; } -void compensateLambdasForCal(double const calTable[XY], - double const oldLambdas[XY], - double newLambdas[XY]) +void compensateLambdasForCal(const std::vector &calTable, + const std::vector &oldLambdas, + std::vector &newLambdas) { double minNewLambda = std::numeric_limits::max(); - for (int i = 0; i < XY; i++) { + for (unsigned int i = 0; i < newLambdas.size(); i++) { newLambdas[i] = oldLambdas[i] * calTable[i]; minNewLambda = std::min(minNewLambda, newLambdas[i]); } - for (int i = 0; i < XY; i++) + for (unsigned int i = 0; i < newLambdas.size(); i++) newLambdas[i] /= minNewLambda; } -[[maybe_unused]] static void printCalTable(double const C[XY]) +[[maybe_unused]] static void printCalTable(const std::vector &C, + const Size &size) { printf("table: [\n"); - for (int j = 0; j < Y; j++) { - for (int i = 0; i < X; i++) { - printf("%5.3f", 1.0 / C[j * X + i]); - if (i != X - 1 || j != Y - 1) + for (unsigned int j = 0; j < size.height; j++) { + for (unsigned int i = 0; i < size.width; i++) { + printf("%5.3f", 1.0 / C[j * size.width + i]); + if (i != size.width - 1 || j != size.height - 1) printf(","); } printf("\n"); @@ -577,9 +607,13 @@ static double computeWeight(double Ci, double Cj, double sigma) } /* Compute all weights. */ -static void computeW(double const C[XY], double sigma, double W[XY][4]) +static void computeW(const std::vector &C, double sigma, + std::vector> &W, const Size &size) { - for (int i = 0; i < XY; i++) { + size_t XY = size.width * size.height; + size_t X = size.width; + + for (unsigned int i = 0; i < XY; i++) { /* Start with neighbour above and go clockwise. */ W[i][0] = i >= X ? computeWeight(C[i], C[i - X], sigma) : 0; W[i][1] = i % X < X - 1 ? computeWeight(C[i], C[i + 1], sigma) : 0; @@ -589,11 +623,16 @@ static void computeW(double const C[XY], double sigma, double W[XY][4]) } /* Compute M, the large but sparse matrix such that M * lambdas = 0. */ -static void constructM(double const C[XY], double const W[XY][4], - double M[XY][4]) +static void constructM(const std::vector &C, + const std::vector> &W, + std::vector> &M, + const Size &size) { + size_t XY = size.width * size.height; + size_t X = size.width; + double epsilon = 0.001; - for (int i = 0; i < XY; i++) { + for (unsigned int i = 0; i < XY; i++) { /* * Note how, if C[i] == INSUFFICIENT_DATA, the weights will all * be zero so the equation is still set up correctly. @@ -614,79 +653,80 @@ static void constructM(double const C[XY], double const W[XY][4], * left/right neighbours are zero down the left/right edges, so we don't need * need to test the i value to exclude them. */ -static double computeLambdaBottom(int i, double const M[XY][4], - double lambda[XY]) +static double computeLambdaBottom(int i, const std::vector> &M, + std::vector &lambda, const Size &size) { - return M[i][1] * lambda[i + 1] + M[i][2] * lambda[i + X] + + return M[i][1] * lambda[i + 1] + M[i][2] * lambda[i + size.width] + M[i][3] * lambda[i - 1]; } -static double computeLambdaBottomStart(int i, double const M[XY][4], - double lambda[XY]) +static double computeLambdaBottomStart(int i, const std::vector> &M, + std::vector &lambda, const Size &size) { - return M[i][1] * lambda[i + 1] + M[i][2] * lambda[i + X]; + return M[i][1] * lambda[i + 1] + M[i][2] * lambda[i + size.width]; } -static double computeLambdaInterior(int i, double const M[XY][4], - double lambda[XY]) +static double computeLambdaInterior(int i, const std::vector> &M, + std::vector &lambda, const Size &size) { - return M[i][0] * lambda[i - X] + M[i][1] * lambda[i + 1] + - M[i][2] * lambda[i + X] + M[i][3] * lambda[i - 1]; + return M[i][0] * lambda[i - size.width] + M[i][1] * lambda[i + 1] + + M[i][2] * lambda[i + size.width] + M[i][3] * lambda[i - 1]; } -static double computeLambdaTop(int i, double const M[XY][4], - double lambda[XY]) +static double computeLambdaTop(int i, const std::vector> &M, + std::vector &lambda, const Size &size) { - return M[i][0] * lambda[i - X] + M[i][1] * lambda[i + 1] + + return M[i][0] * lambda[i - size.width] + M[i][1] * lambda[i + 1] + M[i][3] * lambda[i - 1]; } -static double computeLambdaTopEnd(int i, double const M[XY][4], - double lambda[XY]) +static double computeLambdaTopEnd(int i, const std::vector> &M, + std::vector &lambda, const Size &size) { - return M[i][0] * lambda[i - X] + M[i][3] * lambda[i - 1]; + return M[i][0] * lambda[i - size.width] + M[i][3] * lambda[i - 1]; } /* Gauss-Seidel iteration with over-relaxation. */ -static double gaussSeidel2Sor(double const M[XY][4], double omega, - double lambda[XY], double lambdaBound) +static double gaussSeidel2Sor(const std::vector> &M, double omega, + std::vector &lambda, double lambdaBound, + const Size &size) { + int XY = size.width * size.height; + int X = size.width; const double min = 1 - lambdaBound, max = 1 + lambdaBound; - double oldLambda[XY]; + std::vector oldLambda = lambda; int i; - for (i = 0; i < XY; i++) - oldLambda[i] = lambda[i]; - lambda[0] = computeLambdaBottomStart(0, M, lambda); + lambda[0] = computeLambdaBottomStart(0, M, lambda, size); lambda[0] = std::clamp(lambda[0], min, max); for (i = 1; i < X; i++) { - lambda[i] = computeLambdaBottom(i, M, lambda); + lambda[i] = computeLambdaBottom(i, M, lambda, size); lambda[i] = std::clamp(lambda[i], min, max); } for (; i < XY - X; i++) { - lambda[i] = computeLambdaInterior(i, M, lambda); + lambda[i] = computeLambdaInterior(i, M, lambda, size); lambda[i] = std::clamp(lambda[i], min, max); } for (; i < XY - 1; i++) { - lambda[i] = computeLambdaTop(i, M, lambda); + lambda[i] = computeLambdaTop(i, M, lambda, size); lambda[i] = std::clamp(lambda[i], min, max); } - lambda[i] = computeLambdaTopEnd(i, M, lambda); + lambda[i] = computeLambdaTopEnd(i, M, lambda, size); lambda[i] = std::clamp(lambda[i], min, max); /* * Also solve the system from bottom to top, to help spread the updates * better. */ - lambda[i] = computeLambdaTopEnd(i, M, lambda); + lambda[i] = computeLambdaTopEnd(i, M, lambda, size); lambda[i] = std::clamp(lambda[i], min, max); for (i = XY - 2; i >= XY - X; i--) { - lambda[i] = computeLambdaTop(i, M, lambda); + lambda[i] = computeLambdaTop(i, M, lambda, size); lambda[i] = std::clamp(lambda[i], min, max); } for (; i >= X; i--) { - lambda[i] = computeLambdaInterior(i, M, lambda); + lambda[i] = computeLambdaInterior(i, M, lambda, size); lambda[i] = std::clamp(lambda[i], min, max); } for (; i >= 1; i--) { - lambda[i] = computeLambdaBottom(i, M, lambda); + lambda[i] = computeLambdaBottom(i, M, lambda, size); lambda[i] = std::clamp(lambda[i], min, max); } - lambda[0] = computeLambdaBottomStart(0, M, lambda); + lambda[0] = computeLambdaBottomStart(0, M, lambda, size); lambda[0] = std::clamp(lambda[0], min, max); double maxDiff = 0; for (i = 0; i < XY; i++) { @@ -698,33 +738,33 @@ static double gaussSeidel2Sor(double const M[XY][4], double omega, } /* Normalise the values so that the smallest value is 1. */ -static void normalise(double *ptr, size_t n) +static void normalise(std::vector &results) { - double minval = ptr[0]; - for (size_t i = 1; i < n; i++) - minval = std::min(minval, ptr[i]); - for (size_t i = 0; i < n; i++) - ptr[i] /= minval; + double minval = *std::min_element(results.begin(), results.end()); + std::for_each(results.begin(), results.end(), + [minval](double val) { return val / minval; }); } /* Rescale the values so that the average value is 1. */ -static void reaverage(Span data) +static void reaverage(std::vector &data) { double sum = std::accumulate(data.begin(), data.end(), 0.0); double ratio = 1 / (sum / data.size()); - for (double &d : data) - d *= ratio; + std::for_each(data.begin(), data.end(), + [ratio](double val) { return val * ratio; }); } -static void runMatrixIterations(double const C[XY], double lambda[XY], - double const W[XY][4], double omega, - int nIter, double threshold, double lambdaBound) +static void runMatrixIterations(const std::vector &C, + std::vector &lambda, + const std::vector> &W, + std::vector> &M, double omega, + unsigned int nIter, double threshold, double lambdaBound, + const Size &size) { - double M[XY][4]; - constructM(C, W, M); + constructM(C, W, M, size); double lastMaxDiff = std::numeric_limits::max(); - for (int i = 0; i < nIter; i++) { - double maxDiff = fabs(gaussSeidel2Sor(M, omega, lambda, lambdaBound)); + for (unsigned int i = 0; i < nIter; i++) { + double maxDiff = fabs(gaussSeidel2Sor(M, omega, lambda, lambdaBound, size)); if (maxDiff < threshold) { LOG(RPiAlsc, Debug) << "Stop after " << i + 1 << " iterations"; @@ -741,39 +781,44 @@ static void runMatrixIterations(double const C[XY], double lambda[XY], lastMaxDiff = maxDiff; } /* We're going to normalise the lambdas so the total average is 1. */ - reaverage({ lambda, XY }); + reaverage(lambda); } -static void addLuminanceRb(double result[XY], double const lambda[XY], - double const luminanceLut[XY], +static void addLuminanceRb(std::vector &result, const std::vector &lambda, + const std::vector &luminanceLut, double luminanceStrength) { - for (int i = 0; i < XY; i++) + for (unsigned int i = 0; i < result.size(); i++) result[i] = lambda[i] * ((luminanceLut[i] - 1) * luminanceStrength + 1); } -static void addLuminanceG(double result[XY], double lambda, - double const luminanceLut[XY], +static void addLuminanceG(std::vector &result, double lambda, + const std::vector &luminanceLut, double luminanceStrength) { - for (int i = 0; i < XY; i++) + for (unsigned int i = 0; i < result.size(); i++) result[i] = lambda * ((luminanceLut[i] - 1) * luminanceStrength + 1); } -void addLuminanceToTables(double results[3][Y][X], double const lambdaR[XY], - double lambdaG, double const lambdaB[XY], - double const luminanceLut[XY], +void addLuminanceToTables(std::array, 3> &results, + const std::vector &lambdaR, + double lambdaG, const std::vector &lambdaB, + const std::vector &luminanceLut, double luminanceStrength) { - addLuminanceRb((double *)results[0], lambdaR, luminanceLut, luminanceStrength); - addLuminanceG((double *)results[1], lambdaG, luminanceLut, luminanceStrength); - addLuminanceRb((double *)results[2], lambdaB, luminanceLut, luminanceStrength); - normalise((double *)results, 3 * XY); + addLuminanceRb(results[0], lambdaR, luminanceLut, luminanceStrength); + addLuminanceG(results[1], lambdaG, luminanceLut, luminanceStrength); + addLuminanceRb(results[2], lambdaB, luminanceLut, luminanceStrength); + for (auto &r : results) + normalise(r); } void Alsc::doAlsc() { - double cr[XY], cb[XY], wr[XY][4], wb[XY][4], calTableR[XY], calTableB[XY], calTableTmp[XY]; + std::vector &cr = tmpC_[0], &cb = tmpC_[1], &calTableR = tmpC_[2], + &calTableB = tmpC_[3], &calTableTmp = tmpC_[4]; + std::vector> &wr = tmpM_[0], &wb = tmpM_[1], &M = tmpM_[2]; + /* * Calculate our R/B ("Cr"/"Cb") colour statistics, and assess which are * usable. @@ -784,9 +829,9 @@ void Alsc::doAlsc() * case the camera mode is not full-frame. */ getCalTable(ct_, config_.calibrationsCr, calTableTmp); - resampleCalTable(calTableTmp, cameraMode_, calTableR); + resampleCalTable(calTableTmp, cameraMode_, config_.tableSize, calTableR); getCalTable(ct_, config_.calibrationsCb, calTableTmp); - resampleCalTable(calTableTmp, cameraMode_, calTableB); + resampleCalTable(calTableTmp, cameraMode_, config_.tableSize, calTableB); /* * You could print out the cal tables for this image here, if you're * tuning the algorithm... @@ -796,13 +841,13 @@ void Alsc::doAlsc() applyCalTable(calTableR, cr); applyCalTable(calTableB, cb); /* Compute weights between zones. */ - computeW(cr, config_.sigmaCr, wr); - computeW(cb, config_.sigmaCb, wb); + computeW(cr, config_.sigmaCr, wr, config_.tableSize); + computeW(cb, config_.sigmaCb, wb, config_.tableSize); /* Run Gauss-Seidel iterations over the resulting matrix, for R and B. */ - runMatrixIterations(cr, lambdaR_, wr, config_.omega, config_.nIter, - config_.threshold, config_.lambdaBound); - runMatrixIterations(cb, lambdaB_, wb, config_.omega, config_.nIter, - config_.threshold, config_.lambdaBound); + runMatrixIterations(cr, lambdaR_, wr, M, config_.omega, config_.nIter, + config_.threshold, config_.lambdaBound, config_.tableSize); + runMatrixIterations(cb, lambdaB_, wb, M, config_.omega, config_.nIter, + config_.threshold, config_.lambdaBound, config_.tableSize); /* * Fold the calibrated gains into our final lambda values. (Note that on * the next run, we re-start with the lambda values that don't have the diff --git a/src/ipa/raspberrypi/controller/rpi/alsc.h b/src/ipa/raspberrypi/controller/rpi/alsc.h index 9167c9ffa2e3..85e998db40e9 100644 --- a/src/ipa/raspberrypi/controller/rpi/alsc.h +++ b/src/ipa/raspberrypi/controller/rpi/alsc.h @@ -6,9 +6,13 @@ */ #pragma once +#include #include #include #include +#include + +#include #include "../algorithm.h" #include "../alsc_status.h" @@ -20,7 +24,7 @@ namespace RPiController { struct AlscCalibration { double ct; - double table[AlscCellsX * AlscCellsY]; + std::vector table; }; struct AlscConfig { @@ -36,13 +40,14 @@ struct AlscConfig { uint16_t minG; double omega; uint32_t nIter; - double luminanceLut[AlscCellsX * AlscCellsY]; + std::vector luminanceLut; double luminanceStrength; std::vector calibrationsCr; std::vector calibrationsCb; double defaultCt; /* colour temperature if no metadata found */ double threshold; /* iteration termination threshold */ double lambdaBound; /* upper/lower bound for lambda from a value of 1 */ + libcamera::Size tableSize; }; class Alsc : public Algorithm @@ -62,7 +67,7 @@ private: AlscConfig config_; bool firstTime_; CameraMode cameraMode_; - double luminanceTable_[AlscCellsX * AlscCellsY]; + std::vector luminanceTable_; std::thread asyncThread_; void asyncFunc(); /* asynchronous thread function */ std::mutex mutex_; @@ -88,8 +93,8 @@ private: int frameCount_; /* counts up to startupFrames for Process function */ int frameCount2_; - double syncResults_[3][AlscCellsY][AlscCellsX]; - double prevSyncResults_[3][AlscCellsY][AlscCellsX]; + std::array, 3> syncResults_; + std::array, 3> prevSyncResults_; void waitForAysncThread(); /* * The following are for the asynchronous thread to use, though the main @@ -100,12 +105,16 @@ private: void fetchAsyncResults(); double ct_; RgbyRegions statistics_; - double asyncResults_[3][AlscCellsY][AlscCellsX]; - double asyncLambdaR_[AlscCellsX * AlscCellsY]; - double asyncLambdaB_[AlscCellsX * AlscCellsY]; + std::array, 3> asyncResults_; + std::vector asyncLambdaR_; + std::vector asyncLambdaB_; void doAlsc(); - double lambdaR_[AlscCellsX * AlscCellsY]; - double lambdaB_[AlscCellsX * AlscCellsY]; + std::vector lambdaR_; + std::vector lambdaB_; + + /* Temporaries for the computations */ + std::array, 5> tmpC_; + std::array>, 3> tmpM_; }; } /* namespace RPiController */ diff --git a/src/ipa/raspberrypi/raspberrypi.cpp b/src/ipa/raspberrypi/raspberrypi.cpp index b64cb96e2dde..0fa79bb4af41 100644 --- a/src/ipa/raspberrypi/raspberrypi.cpp +++ b/src/ipa/raspberrypi/raspberrypi.cpp @@ -13,6 +13,7 @@ #include #include #include +#include #include #include @@ -174,7 +175,7 @@ private: void applyDPC(const struct DpcStatus *dpcStatus, ControlList &ctrls); void applyLS(const struct AlscStatus *lsStatus, ControlList &ctrls); void applyAF(const struct AfStatus *afStatus, ControlList &lensCtrls); - void resampleTable(uint16_t dest[], double const src[12][16], int destW, int destH); + void resampleTable(uint16_t dest[], const std::vector &src, int destW, int destH); std::map buffers_; @@ -1768,7 +1769,7 @@ void IPARPi::applyAF(const struct AfStatus *afStatus, ControlList &lensCtrls) * Resamples a 16x12 table with central sampling to destW x destH with corner * sampling. */ -void IPARPi::resampleTable(uint16_t dest[], double const src[12][16], +void IPARPi::resampleTable(uint16_t dest[], const std::vector &src, int destW, int destH) { /* @@ -1793,8 +1794,8 @@ void IPARPi::resampleTable(uint16_t dest[], double const src[12][16], double yf = y - yLo; int yHi = yLo < 11 ? yLo + 1 : 11; yLo = yLo > 0 ? yLo : 0; - double const *rowAbove = src[yLo]; - double const *rowBelow = src[yHi]; + double const *rowAbove = src.data() + yLo * 16; + double const *rowBelow = src.data() + yHi * 16; for (int i = 0; i < destW; i++) { double above = rowAbove[xLo[i]] * (1 - xf[i]) + rowAbove[xHi[i]] * xf[i]; double below = rowBelow[xLo[i]] * (1 - xf[i]) + rowBelow[xHi[i]] * xf[i]; From patchwork Wed Mar 22 13:06:06 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Naushir Patuck X-Patchwork-Id: 18431 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id CB014C329E for ; Wed, 22 Mar 2023 13:06:27 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 5EEB2626DB; Wed, 22 Mar 2023 14:06:26 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1679490387; bh=rNrkcTPDfE3KsH9O4xyE4AhKqe1wa9fkLBFqXbG7IEY=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To: From; b=pxliA1Eha7Jet0LUxuEVW/j+AmSO2mPYeFJNU435c9OaiD3F95cc78t+McuF0w7lp OZgje/l2R6c/gXx0bn2eig6WBoE7xEkKx1Z6hmnadfV/MCc+1vsg2PrtNlxfMesxO1 SrnQBcF0kVbpRvpy4HgDxL9wUJwC+8qgsVPhNXG+nt1iZMv9BNidXUQQ9+J6aSkRqZ Go62lawMITKwFPJlbE/5ZIVYHykW1SMbX51sfGQgX/JQEb6n8ZjkYeUwh/6NHVFcte Gdhk44uUrV388MbEeKzO7SnlhdjKe1pdfdOjyoh+vZzl+tCCbPHVebTE5FmpiSvXuz oBbLKpdE69mRg== Received: from mail-il1-x12c.google.com (mail-il1-x12c.google.com [IPv6:2607:f8b0:4864:20::12c]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 29B3061ECE for ; Wed, 22 Mar 2023 14:06:23 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="GxFqPkqv"; dkim-atps=neutral Received: by mail-il1-x12c.google.com with SMTP id i19so9795173ila.10 for ; Wed, 22 Mar 2023 06:06:23 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; t=1679490382; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=lYPxFFzmW22DpXx5M/MtFvVmxoXfX9plIiPB97/sXz8=; b=GxFqPkqvuLV0/7ptj99UMS66L63jIYvm7CcO7qzXmx4GPPF3ieMMl7RXZuFYEbEZ66 5gIQrRkhM0e94C5GXOPdPz8ZpphJwJuyfWRu0vYZ/WKDdbhE1M2A6U7FEQ6cyOt0Es2G fyzCQcFYKDptRaCuAlbwgB9efWYd+nrp+7ZY94MPanOAL58yH7wSgqslhTyPRDaEbW+n Qm43DAurcgV4dexy1/q/+LPoh0Psp3gEiWanl+ZsGebfw3VjvVrHJHWKAEGY/651JYMc fyH2SlBzyOvjQ7HRARIQlZ7ibxJHbpj66hd8J7elA8jNw7ZBW1M1Lf6gmI5qLqOy+AWM u6yg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; t=1679490382; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=lYPxFFzmW22DpXx5M/MtFvVmxoXfX9plIiPB97/sXz8=; b=trS8Gt+1Meodumpz1YlK6rTbjKTjbK55L3WUyfjKHKM69j2F/EI12OrDC2l8gQtJxC HTLyLE988TZHw5WvoGqr5MolYAyRoovzJU5qv43XKJGNxFiI5C6cvb27zZVNKZVwqp2L //o7jnZaM2eSoWl9fMLmYdZHyhahOxCZqA0D1V+5plbDq5C0YwcmR1s+ChRDSbZ/UVda kEJDbUJrmAJQxU/UXYz3Z4azsbkDkeU1ya/biXSyC6okGPnkKvqXcWbMG798MgHcVo10 PgKf8Hu9XQlRhV1fMTCcJFKgdtXfEg7fYI9L19cNsbyzH7DVlDZp5rM3FuNWJVmIM8r0 KDnQ== X-Gm-Message-State: AO0yUKU5qD3WhxtEq/hLjXNF/umysRI+ze1djxp+15bDCYqb5xzAScvE 3IA5iwfVwtZ2mIf4DPyFOJ0XvGme4NYqEyguwiTxcg== X-Google-Smtp-Source: AK7set9D4Z8Vmra0y0makg/pG4JgJbcOtdhKZQC+yVYr5SpjuDHwBwqSfo6piOKem36xavuR9B3isg== X-Received: by 2002:a92:c605:0:b0:325:af23:ea4d with SMTP id p5-20020a92c605000000b00325af23ea4dmr3295811ilm.12.1679490381455; Wed, 22 Mar 2023 06:06:21 -0700 (PDT) Received: from localhost.localdomain ([93.93.133.154]) by smtp.gmail.com with ESMTPSA id l6-20020a026646000000b003c488204c6fsm4927960jaf.76.2023.03.22.06.06.20 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 22 Mar 2023 06:06:21 -0700 (PDT) To: libcamera-devel@lists.libcamera.org Date: Wed, 22 Mar 2023 13:06:06 +0000 Message-Id: <20230322130612.5208-5-naush@raspberrypi.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230322130612.5208-1-naush@raspberrypi.com> References: <20230322130612.5208-1-naush@raspberrypi.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v1 04/10] ipa: raspberrypi: alsc: Replace std::vectors by Array2D class X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Naushir Patuck via libcamera-devel From: Naushir Patuck Reply-To: Naushir Patuck Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" From: David Plowman The Array2D class is a very thin wrapper round std::vector that can be used almost identically in the code, but it carries its 2D size with it so that we aren't passing it around all the time. All the std::vectors that were X * Y in size (X and Y being the ALSC grid size) have been replaced. The sparse matrices that are XY * 4 in size have not been as they are somewhat different, are used differently, require more code changes, and actually make things more confusing if everything looks like an Array2D but are not the same. There should be no change in algorithm behaviour at all. Signed-off-by: David Plowman Reviewed-by: Naushir Patuck Reviewed-by: Jacopo Mondi --- src/ipa/raspberrypi/controller/rpi/alsc.cpp | 218 ++++++++++---------- src/ipa/raspberrypi/controller/rpi/alsc.h | 68 +++++- 2 files changed, 164 insertions(+), 122 deletions(-) diff --git a/src/ipa/raspberrypi/controller/rpi/alsc.cpp b/src/ipa/raspberrypi/controller/rpi/alsc.cpp index 51fe5d73f52d..524c48093590 100644 --- a/src/ipa/raspberrypi/controller/rpi/alsc.cpp +++ b/src/ipa/raspberrypi/controller/rpi/alsc.cpp @@ -49,11 +49,10 @@ char const *Alsc::name() const return NAME; } -static int generateLut(std::vector &lut, const libcamera::YamlObject ¶ms, - const Size &size) +static int generateLut(Array2D &lut, const libcamera::YamlObject ¶ms) { /* These must be signed ints for the co-ordinate calculations below. */ - int X = size.width, Y = size.height; + int X = lut.dimensions().width, Y = lut.dimensions().height; double cstrength = params["corner_strength"].get(2.0); if (cstrength <= 1.0) { LOG(RPiAlsc, Error) << "corner_strength must be > 1.0"; @@ -82,9 +81,9 @@ static int generateLut(std::vector &lut, const libcamera::YamlObject &pa return 0; } -static int readLut(std::vector &lut, const libcamera::YamlObject ¶ms, const Size &size) +static int readLut(Array2D &lut, const libcamera::YamlObject ¶ms) { - if (params.size() != size.width * size.height) { + if (params.size() != lut.size()) { LOG(RPiAlsc, Error) << "Invalid number of entries in LSC table"; return -EINVAL; } @@ -128,7 +127,7 @@ static int readCalibrations(std::vector &calibrations, } int num = 0; - calibration.table.resize(size.width * size.height); + calibration.table.resize(size); for (const auto &elem : table.asList()) { value = elem.get(); if (!value) @@ -160,13 +159,13 @@ int Alsc::read(const libcamera::YamlObject ¶ms) config_.luminanceStrength = params["luminance_strength"].get(1.0); - config_.luminanceLut.resize(config_.tableSize.width * config_.tableSize.height, 1.0); + config_.luminanceLut.resize(config_.tableSize, 1.0); int ret = 0; if (params.contains("corner_strength")) - ret = generateLut(config_.luminanceLut, params, config_.tableSize); + ret = generateLut(config_.luminanceLut, params); else if (params.contains("luminance_lut")) - ret = readLut(config_.luminanceLut, params["luminance_lut"], config_.tableSize); + ret = readLut(config_.luminanceLut, params["luminance_lut"]); else LOG(RPiAlsc, Warning) << "no luminance table - assume unity everywhere"; @@ -191,16 +190,16 @@ int Alsc::read(const libcamera::YamlObject ¶ms) static double getCt(Metadata *metadata, double defaultCt); static void getCalTable(double ct, std::vector const &calibrations, - std::vector &calTable); -static void resampleCalTable(const std::vector &calTableIn, CameraMode const &cameraMode, - const Size &size, std::vector &calTableOut); -static void compensateLambdasForCal(const std::vector &calTable, - const std::vector &oldLambdas, - std::vector &newLambdas); -static void addLuminanceToTables(std::array, 3> &results, - const std::vector &lambdaR, double lambdaG, - const std::vector &lambdaB, - const std::vector &luminanceLut, + Array2D &calTable); +static void resampleCalTable(const Array2D &calTableIn, CameraMode const &cameraMode, + Array2D &calTableOut); +static void compensateLambdasForCal(const Array2D &calTable, + const Array2D &oldLambdas, + Array2D &newLambdas); +static void addLuminanceToTables(std::array, 3> &results, + const Array2D &lambdaR, double lambdaG, + const Array2D &lambdaB, + const Array2D &luminanceLut, double luminanceStrength); void Alsc::initialise() @@ -212,22 +211,22 @@ void Alsc::initialise() const size_t XY = config_.tableSize.width * config_.tableSize.height; for (auto &r : syncResults_) - r.resize(XY); + r.resize(config_.tableSize); for (auto &r : prevSyncResults_) - r.resize(XY); + r.resize(config_.tableSize); for (auto &r : asyncResults_) - r.resize(XY); + r.resize(config_.tableSize); - luminanceTable_.resize(XY); - asyncLambdaR_.resize(XY); - asyncLambdaB_.resize(XY); + luminanceTable_.resize(config_.tableSize); + asyncLambdaR_.resize(config_.tableSize); + asyncLambdaB_.resize(config_.tableSize); /* The lambdas are initialised in the SwitchMode. */ - lambdaR_.resize(XY); - lambdaB_.resize(XY); + lambdaR_.resize(config_.tableSize); + lambdaB_.resize(config_.tableSize); /* Temporaries for the computations, but sensible to allocate this up-front! */ for (auto &c : tmpC_) - c.resize(XY); + c.resize(config_.tableSize); for (auto &m : tmpM_) m.resize(XY); } @@ -290,7 +289,7 @@ void Alsc::switchMode(CameraMode const &cameraMode, * We must resample the luminance table like we do the others, but it's * fixed so we can simply do it up front here. */ - resampleCalTable(config_.luminanceLut, cameraMode_, config_.tableSize, luminanceTable_); + resampleCalTable(config_.luminanceLut, cameraMode_, luminanceTable_); if (resetTables) { /* @@ -302,11 +301,11 @@ void Alsc::switchMode(CameraMode const &cameraMode, */ std::fill(lambdaR_.begin(), lambdaR_.end(), 1.0); std::fill(lambdaB_.begin(), lambdaB_.end(), 1.0); - std::vector &calTableR = tmpC_[0], &calTableB = tmpC_[1], &calTableTmp = tmpC_[2]; + Array2D &calTableR = tmpC_[0], &calTableB = tmpC_[1], &calTableTmp = tmpC_[2]; getCalTable(ct_, config_.calibrationsCr, calTableTmp); - resampleCalTable(calTableTmp, cameraMode_, config_.tableSize, calTableR); + resampleCalTable(calTableTmp, cameraMode_, calTableR); getCalTable(ct_, config_.calibrationsCb, calTableTmp); - resampleCalTable(calTableTmp, cameraMode_, config_.tableSize, calTableB); + resampleCalTable(calTableTmp, cameraMode_, calTableB); compensateLambdasForCal(calTableR, lambdaR_, asyncLambdaR_); compensateLambdasForCal(calTableB, lambdaB_, asyncLambdaB_); addLuminanceToTables(syncResults_, asyncLambdaR_, 1.0, asyncLambdaB_, @@ -411,9 +410,9 @@ void Alsc::prepare(Metadata *imageMetadata) } /* Put output values into status metadata. */ AlscStatus status; - status.r = prevSyncResults_[0]; - status.g = prevSyncResults_[1]; - status.b = prevSyncResults_[2]; + status.r = prevSyncResults_[0].data(); + status.g = prevSyncResults_[1].data(); + status.b = prevSyncResults_[2].data(); imageMetadata->set("alsc.status", status); } @@ -457,7 +456,7 @@ void Alsc::asyncFunc() } void getCalTable(double ct, std::vector const &calibrations, - std::vector &calTable) + Array2D &calTable) { if (calibrations.empty()) { std::fill(calTable.begin(), calTable.end(), 1.0); @@ -486,12 +485,12 @@ void getCalTable(double ct, std::vector const &calibrations, } } -void resampleCalTable(const std::vector &calTableIn, - CameraMode const &cameraMode, const Size &size, - std::vector &calTableOut) +void resampleCalTable(const Array2D &calTableIn, + CameraMode const &cameraMode, + Array2D &calTableOut) { - int X = size.width; - int Y = size.height; + int X = calTableIn.dimensions().width; + int Y = calTableIn.dimensions().height; /* * Precalculate and cache the x sampling locations and phases to save @@ -529,9 +528,9 @@ void resampleCalTable(const std::vector &calTableIn, yLo = Y - 1 - yLo; yHi = Y - 1 - yHi; } - double const *rowAbove = calTableIn.data() + X * yLo; - double const *rowBelow = calTableIn.data() + X * yHi; - double *out = calTableOut.data() + X * j; + double const *rowAbove = calTableIn.ptr() + X * yLo; + double const *rowBelow = calTableIn.ptr() + X * yHi; + double *out = calTableOut.ptr() + X * j; for (int i = 0; i < X; i++) { double above = rowAbove[xLo[i]] * (1 - xf[i]) + rowAbove[xHi[i]] * xf[i]; @@ -543,8 +542,8 @@ void resampleCalTable(const std::vector &calTableIn, } /* Calculate chrominance statistics (R/G and B/G) for each region. */ -static void calculateCrCb(const RgbyRegions &awbRegion, std::vector &cr, - std::vector &cb, uint32_t minCount, uint16_t minG) +static void calculateCrCb(const RgbyRegions &awbRegion, Array2D &cr, + Array2D &cb, uint32_t minCount, uint16_t minG) { for (unsigned int i = 0; i < cr.size(); i++) { auto s = awbRegion.get(i); @@ -559,16 +558,16 @@ static void calculateCrCb(const RgbyRegions &awbRegion, std::vector &cr, } } -static void applyCalTable(const std::vector &calTable, std::vector &C) +static void applyCalTable(const Array2D &calTable, Array2D &C) { for (unsigned int i = 0; i < C.size(); i++) if (C[i] != InsufficientData) C[i] *= calTable[i]; } -void compensateLambdasForCal(const std::vector &calTable, - const std::vector &oldLambdas, - std::vector &newLambdas) +void compensateLambdasForCal(const Array2D &calTable, + const Array2D &oldLambdas, + Array2D &newLambdas) { double minNewLambda = std::numeric_limits::max(); for (unsigned int i = 0; i < newLambdas.size(); i++) { @@ -579,9 +578,9 @@ void compensateLambdasForCal(const std::vector &calTable, newLambdas[i] /= minNewLambda; } -[[maybe_unused]] static void printCalTable(const std::vector &C, - const Size &size) +[[maybe_unused]] static void printCalTable(const Array2D &C) { + const Size &size = C.dimensions(); printf("table: [\n"); for (unsigned int j = 0; j < size.height; j++) { for (unsigned int i = 0; i < size.width; i++) { @@ -607,11 +606,11 @@ static double computeWeight(double Ci, double Cj, double sigma) } /* Compute all weights. */ -static void computeW(const std::vector &C, double sigma, - std::vector> &W, const Size &size) +static void computeW(const Array2D &C, double sigma, + std::vector> &W) { - size_t XY = size.width * size.height; - size_t X = size.width; + size_t XY = C.size(); + size_t X = C.dimensions().width; for (unsigned int i = 0; i < XY; i++) { /* Start with neighbour above and go clockwise. */ @@ -623,13 +622,12 @@ static void computeW(const std::vector &C, double sigma, } /* Compute M, the large but sparse matrix such that M * lambdas = 0. */ -static void constructM(const std::vector &C, +static void constructM(const Array2D &C, const std::vector> &W, - std::vector> &M, - const Size &size) + std::vector> &M) { - size_t XY = size.width * size.height; - size_t X = size.width; + size_t XY = C.size(); + size_t X = C.dimensions().width; double epsilon = 0.001; for (unsigned int i = 0; i < XY; i++) { @@ -654,79 +652,78 @@ static void constructM(const std::vector &C, * need to test the i value to exclude them. */ static double computeLambdaBottom(int i, const std::vector> &M, - std::vector &lambda, const Size &size) + Array2D &lambda) { - return M[i][1] * lambda[i + 1] + M[i][2] * lambda[i + size.width] + + return M[i][1] * lambda[i + 1] + M[i][2] * lambda[i + lambda.dimensions().width] + M[i][3] * lambda[i - 1]; } static double computeLambdaBottomStart(int i, const std::vector> &M, - std::vector &lambda, const Size &size) + Array2D &lambda) { - return M[i][1] * lambda[i + 1] + M[i][2] * lambda[i + size.width]; + return M[i][1] * lambda[i + 1] + M[i][2] * lambda[i + lambda.dimensions().width]; } static double computeLambdaInterior(int i, const std::vector> &M, - std::vector &lambda, const Size &size) + Array2D &lambda) { - return M[i][0] * lambda[i - size.width] + M[i][1] * lambda[i + 1] + - M[i][2] * lambda[i + size.width] + M[i][3] * lambda[i - 1]; + return M[i][0] * lambda[i - lambda.dimensions().width] + M[i][1] * lambda[i + 1] + + M[i][2] * lambda[i + lambda.dimensions().width] + M[i][3] * lambda[i - 1]; } static double computeLambdaTop(int i, const std::vector> &M, - std::vector &lambda, const Size &size) + Array2D &lambda) { - return M[i][0] * lambda[i - size.width] + M[i][1] * lambda[i + 1] + + return M[i][0] * lambda[i - lambda.dimensions().width] + M[i][1] * lambda[i + 1] + M[i][3] * lambda[i - 1]; } static double computeLambdaTopEnd(int i, const std::vector> &M, - std::vector &lambda, const Size &size) + Array2D &lambda) { - return M[i][0] * lambda[i - size.width] + M[i][3] * lambda[i - 1]; + return M[i][0] * lambda[i - lambda.dimensions().width] + M[i][3] * lambda[i - 1]; } /* Gauss-Seidel iteration with over-relaxation. */ static double gaussSeidel2Sor(const std::vector> &M, double omega, - std::vector &lambda, double lambdaBound, - const Size &size) + Array2D &lambda, double lambdaBound) { - int XY = size.width * size.height; - int X = size.width; + int XY = lambda.size(); + int X = lambda.dimensions().width; const double min = 1 - lambdaBound, max = 1 + lambdaBound; - std::vector oldLambda = lambda; + Array2D oldLambda = lambda; int i; - lambda[0] = computeLambdaBottomStart(0, M, lambda, size); + lambda[0] = computeLambdaBottomStart(0, M, lambda); lambda[0] = std::clamp(lambda[0], min, max); for (i = 1; i < X; i++) { - lambda[i] = computeLambdaBottom(i, M, lambda, size); + lambda[i] = computeLambdaBottom(i, M, lambda); lambda[i] = std::clamp(lambda[i], min, max); } for (; i < XY - X; i++) { - lambda[i] = computeLambdaInterior(i, M, lambda, size); + lambda[i] = computeLambdaInterior(i, M, lambda); lambda[i] = std::clamp(lambda[i], min, max); } for (; i < XY - 1; i++) { - lambda[i] = computeLambdaTop(i, M, lambda, size); + lambda[i] = computeLambdaTop(i, M, lambda); lambda[i] = std::clamp(lambda[i], min, max); } - lambda[i] = computeLambdaTopEnd(i, M, lambda, size); + lambda[i] = computeLambdaTopEnd(i, M, lambda); lambda[i] = std::clamp(lambda[i], min, max); /* * Also solve the system from bottom to top, to help spread the updates * better. */ - lambda[i] = computeLambdaTopEnd(i, M, lambda, size); + lambda[i] = computeLambdaTopEnd(i, M, lambda); lambda[i] = std::clamp(lambda[i], min, max); for (i = XY - 2; i >= XY - X; i--) { - lambda[i] = computeLambdaTop(i, M, lambda, size); + lambda[i] = computeLambdaTop(i, M, lambda); lambda[i] = std::clamp(lambda[i], min, max); } for (; i >= X; i--) { - lambda[i] = computeLambdaInterior(i, M, lambda, size); + lambda[i] = computeLambdaInterior(i, M, lambda); lambda[i] = std::clamp(lambda[i], min, max); } for (; i >= 1; i--) { - lambda[i] = computeLambdaBottom(i, M, lambda, size); + lambda[i] = computeLambdaBottom(i, M, lambda); lambda[i] = std::clamp(lambda[i], min, max); } - lambda[0] = computeLambdaBottomStart(0, M, lambda, size); + lambda[0] = computeLambdaBottomStart(0, M, lambda); lambda[0] = std::clamp(lambda[0], min, max); double maxDiff = 0; for (i = 0; i < XY; i++) { @@ -738,7 +735,7 @@ static double gaussSeidel2Sor(const std::vector> &M, doubl } /* Normalise the values so that the smallest value is 1. */ -static void normalise(std::vector &results) +static void normalise(Array2D &results) { double minval = *std::min_element(results.begin(), results.end()); std::for_each(results.begin(), results.end(), @@ -746,7 +743,7 @@ static void normalise(std::vector &results) } /* Rescale the values so that the average value is 1. */ -static void reaverage(std::vector &data) +static void reaverage(Array2D &data) { double sum = std::accumulate(data.begin(), data.end(), 0.0); double ratio = 1 / (sum / data.size()); @@ -754,17 +751,16 @@ static void reaverage(std::vector &data) [ratio](double val) { return val * ratio; }); } -static void runMatrixIterations(const std::vector &C, - std::vector &lambda, +static void runMatrixIterations(const Array2D &C, + Array2D &lambda, const std::vector> &W, std::vector> &M, double omega, - unsigned int nIter, double threshold, double lambdaBound, - const Size &size) + unsigned int nIter, double threshold, double lambdaBound) { - constructM(C, W, M, size); + constructM(C, W, M); double lastMaxDiff = std::numeric_limits::max(); for (unsigned int i = 0; i < nIter; i++) { - double maxDiff = fabs(gaussSeidel2Sor(M, omega, lambda, lambdaBound, size)); + double maxDiff = fabs(gaussSeidel2Sor(M, omega, lambda, lambdaBound)); if (maxDiff < threshold) { LOG(RPiAlsc, Debug) << "Stop after " << i + 1 << " iterations"; @@ -784,26 +780,26 @@ static void runMatrixIterations(const std::vector &C, reaverage(lambda); } -static void addLuminanceRb(std::vector &result, const std::vector &lambda, - const std::vector &luminanceLut, +static void addLuminanceRb(Array2D &result, const Array2D &lambda, + const Array2D &luminanceLut, double luminanceStrength) { for (unsigned int i = 0; i < result.size(); i++) result[i] = lambda[i] * ((luminanceLut[i] - 1) * luminanceStrength + 1); } -static void addLuminanceG(std::vector &result, double lambda, - const std::vector &luminanceLut, +static void addLuminanceG(Array2D &result, double lambda, + const Array2D &luminanceLut, double luminanceStrength) { for (unsigned int i = 0; i < result.size(); i++) result[i] = lambda * ((luminanceLut[i] - 1) * luminanceStrength + 1); } -void addLuminanceToTables(std::array, 3> &results, - const std::vector &lambdaR, - double lambdaG, const std::vector &lambdaB, - const std::vector &luminanceLut, +void addLuminanceToTables(std::array, 3> &results, + const Array2D &lambdaR, + double lambdaG, const Array2D &lambdaB, + const Array2D &luminanceLut, double luminanceStrength) { addLuminanceRb(results[0], lambdaR, luminanceLut, luminanceStrength); @@ -815,8 +811,8 @@ void addLuminanceToTables(std::array, 3> &results, void Alsc::doAlsc() { - std::vector &cr = tmpC_[0], &cb = tmpC_[1], &calTableR = tmpC_[2], - &calTableB = tmpC_[3], &calTableTmp = tmpC_[4]; + Array2D &cr = tmpC_[0], &cb = tmpC_[1], &calTableR = tmpC_[2], + &calTableB = tmpC_[3], &calTableTmp = tmpC_[4]; std::vector> &wr = tmpM_[0], &wb = tmpM_[1], &M = tmpM_[2]; /* @@ -829,9 +825,9 @@ void Alsc::doAlsc() * case the camera mode is not full-frame. */ getCalTable(ct_, config_.calibrationsCr, calTableTmp); - resampleCalTable(calTableTmp, cameraMode_, config_.tableSize, calTableR); + resampleCalTable(calTableTmp, cameraMode_, calTableR); getCalTable(ct_, config_.calibrationsCb, calTableTmp); - resampleCalTable(calTableTmp, cameraMode_, config_.tableSize, calTableB); + resampleCalTable(calTableTmp, cameraMode_, calTableB); /* * You could print out the cal tables for this image here, if you're * tuning the algorithm... @@ -841,13 +837,13 @@ void Alsc::doAlsc() applyCalTable(calTableR, cr); applyCalTable(calTableB, cb); /* Compute weights between zones. */ - computeW(cr, config_.sigmaCr, wr, config_.tableSize); - computeW(cb, config_.sigmaCb, wb, config_.tableSize); + computeW(cr, config_.sigmaCr, wr); + computeW(cb, config_.sigmaCb, wb); /* Run Gauss-Seidel iterations over the resulting matrix, for R and B. */ runMatrixIterations(cr, lambdaR_, wr, M, config_.omega, config_.nIter, - config_.threshold, config_.lambdaBound, config_.tableSize); + config_.threshold, config_.lambdaBound); runMatrixIterations(cb, lambdaB_, wb, M, config_.omega, config_.nIter, - config_.threshold, config_.lambdaBound, config_.tableSize); + config_.threshold, config_.lambdaBound); /* * Fold the calibrated gains into our final lambda values. (Note that on * the next run, we re-start with the lambda values that don't have the diff --git a/src/ipa/raspberrypi/controller/rpi/alsc.h b/src/ipa/raspberrypi/controller/rpi/alsc.h index 85e998db40e9..1ab61299c4cd 100644 --- a/src/ipa/raspberrypi/controller/rpi/alsc.h +++ b/src/ipa/raspberrypi/controller/rpi/alsc.h @@ -22,9 +22,55 @@ namespace RPiController { /* Algorithm to generate automagic LSC (Lens Shading Correction) tables. */ +/* + * The Array2D class is a very thin wrapper round std::vector so that it can + * be used in exactly the same way in the code but carries its correct width + * and height ("dimensions") with it. + */ + +template +class Array2D +{ +public: + using Size = libcamera::Size; + + const Size &dimensions() const { return dimensions_; } + + size_t size() const { return data_.size(); } + + const std::vector &data() const { return data_; } + + void resize(const Size &dims) + { + dimensions_ = dims; + data_.resize(dims.width * dims.height); + } + + void resize(const Size &dims, const T &value) + { + resize(dims); + std::fill(data_.begin(), data_.end(), value); + } + + T &operator[](int index) { return data_[index]; } + + const T &operator[](int index) const { return data_[index]; } + + T *ptr() { return data_.data(); } + + const T *ptr() const { return data_.data(); } + + auto begin() { return data_.begin(); } + auto end() { return data_.end(); } + +private: + Size dimensions_; + std::vector data_; +}; + struct AlscCalibration { double ct; - std::vector table; + Array2D table; }; struct AlscConfig { @@ -40,7 +86,7 @@ struct AlscConfig { uint16_t minG; double omega; uint32_t nIter; - std::vector luminanceLut; + Array2D luminanceLut; double luminanceStrength; std::vector calibrationsCr; std::vector calibrationsCb; @@ -67,7 +113,7 @@ private: AlscConfig config_; bool firstTime_; CameraMode cameraMode_; - std::vector luminanceTable_; + Array2D luminanceTable_; std::thread asyncThread_; void asyncFunc(); /* asynchronous thread function */ std::mutex mutex_; @@ -93,8 +139,8 @@ private: int frameCount_; /* counts up to startupFrames for Process function */ int frameCount2_; - std::array, 3> syncResults_; - std::array, 3> prevSyncResults_; + std::array, 3> syncResults_; + std::array, 3> prevSyncResults_; void waitForAysncThread(); /* * The following are for the asynchronous thread to use, though the main @@ -105,15 +151,15 @@ private: void fetchAsyncResults(); double ct_; RgbyRegions statistics_; - std::array, 3> asyncResults_; - std::vector asyncLambdaR_; - std::vector asyncLambdaB_; + std::array, 3> asyncResults_; + Array2D asyncLambdaR_; + Array2D asyncLambdaB_; void doAlsc(); - std::vector lambdaR_; - std::vector lambdaB_; + Array2D lambdaR_; + Array2D lambdaB_; /* Temporaries for the computations */ - std::array, 5> tmpC_; + std::array, 5> tmpC_; std::array>, 3> tmpM_; }; From patchwork Wed Mar 22 13:06:07 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Naushir Patuck X-Patchwork-Id: 18432 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 361E3C329F for ; Wed, 22 Mar 2023 13:06:28 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id C1D4662716; Wed, 22 Mar 2023 14:06:27 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1679490387; bh=GzdIKbckX0bRqpgpUVf0MR/k60yQLQR4p9/YtYtkHlU=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To: From; b=gVcAPgT6Hx4EC+zS/Wd2/W7N/u9dxhkrKDaq72vIHH3twX38TGSQbHuOqtL20T2y1 o9gXV0/48sfMwfGrXkggBbnk6ep11TLYDV2u1U2JxA+Sn/230o077+R40mjHVBMZuy PK496yvYqsvm1dBOLvMsnN4xjA3n/SXZyK7KRHCOlAa18DSasdDfGbt/h6yPsVJRMz +SzpqGW/rSUCaQ7A4uyqEyP0MLWiGfDMGqljpqlp77O0t303YG2r4ORxOp3wo9BxUh u4JxYddmIrAWOGx1pWWvhVXbYuLZfQhmoK/VzP6UGo70y9M/P7TdpYz2qSwaz2eyUd muhNjAuWjFRmQ== Received: from mail-io1-xd2e.google.com (mail-io1-xd2e.google.com [IPv6:2607:f8b0:4864:20::d2e]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 7D283626DB for ; Wed, 22 Mar 2023 14:06:23 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="g883W8OV"; dkim-atps=neutral Received: by mail-io1-xd2e.google.com with SMTP id m22so8407009ioy.4 for ; Wed, 22 Mar 2023 06:06:23 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; t=1679490383; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=eKdjahHZlI08PdsXsAVyS9pD8aARi8WXwJYPUcC+57Y=; b=g883W8OVh5Uh0SgWdoS33FtJgYBeXlZcAYc0ghroYnhrp3BgJyO5rjJS1S3P/hd8AB +fSAyKyz+JjBhTLJet59YbnsAWtus5bFZ/792OD42C6eH/kYoq2/EzrWZHCJ5KOZf2Hl AOJ2jlEUa0MOuNEfRhSb5u/s2oRBWppI42j/t3/VnCvthT7pJp6f7vTb4qeTB2ry1+XJ +XgW7/rhnaVUOHljKPFySCD1atu6NBhpv9EhGlExFuw8+iAibWroRae8HK8JamgnKLVW 5rN/TdaZKJEH18VwZi7y+G0pfkvLSWEceewWoqBdd5W4ECYZQsY2QLaHOebJTz7aqu8q 837Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; t=1679490383; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=eKdjahHZlI08PdsXsAVyS9pD8aARi8WXwJYPUcC+57Y=; b=rBXvA1rvZgDwT+M0orN6nANwNwKE94VVsw/7D4ejmSJ96mnPFBRBjPQtPiVP7VvFAz 0EXzs/h0TJ2n8deBDY8zEr/6JPu95RjBP+gTUYN8EFJCveVWurcHB4FDmrt20Vom9Nt9 jyOaZr79mnfUZQr2hS87fjqXq5PJIBVh4YXllyVGWMcfEAaJ+zjZm8IymioJi0jFtKxb 6iX1JqdSriE+m8l9eB3t3ededwuvD8DLR1zPGV4OtOm6GJ+noLsmPF7HTc1hMUf6FKZa 3PQIodi/7eGWG0UOOoe6T5Y/fuIh1isbIvIFY7pZtHMFnDcVQOtT3vKHMRfzLpYqjBD5 6Ttw== X-Gm-Message-State: AO0yUKWMC7BRZ+TLD1xdl4Vcvya59EWOPEdw/N2HrGOOZUP9nA6W0QaZ IuehfvIZ9gWy+9Ry+rFM1xu5o8NJ4XyPxBMwfpjA6w== X-Google-Smtp-Source: AK7set8WvT6o8fUxeO1cxO3mUZqgX7LK0o9ySCl/98BoHKkpQc99O96ajEww5K6jYtXtXjeNm1GXEg== X-Received: by 2002:a5e:de46:0:b0:752:ee32:322d with SMTP id e6-20020a5ede46000000b00752ee32322dmr4002575ioq.18.1679490382932; Wed, 22 Mar 2023 06:06:22 -0700 (PDT) Received: from localhost.localdomain ([93.93.133.154]) by smtp.gmail.com with ESMTPSA id l6-20020a026646000000b003c488204c6fsm4927960jaf.76.2023.03.22.06.06.21 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 22 Mar 2023 06:06:22 -0700 (PDT) To: libcamera-devel@lists.libcamera.org Date: Wed, 22 Mar 2023 13:06:07 +0000 Message-Id: <20230322130612.5208-6-naush@raspberrypi.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230322130612.5208-1-naush@raspberrypi.com> References: <20230322130612.5208-1-naush@raspberrypi.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v1 05/10] ipa: raspberrypi: alsc: Use a better type name for sparse arrays X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Naushir Patuck via libcamera-devel From: Naushir Patuck Reply-To: Naushir Patuck Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" From: David Plowman The algorithm uses the data type std::vector> to represent the large sparse matrices that are XY (X, Y being the ALSC grid size) high but with only 4 non-zero elements on each row. Replace this slightly long type name by SparseArray. No functional changes. Signed-off-by: David Plowman Reviewed-by: Naushir Patuck Reviewed-by: Jacopo Mondi --- src/ipa/raspberrypi/controller/rpi/alsc.cpp | 24 ++++++++++----------- src/ipa/raspberrypi/controller/rpi/alsc.h | 10 ++++++++- 2 files changed, 21 insertions(+), 13 deletions(-) diff --git a/src/ipa/raspberrypi/controller/rpi/alsc.cpp b/src/ipa/raspberrypi/controller/rpi/alsc.cpp index 524c48093590..3a2e8fe00ca6 100644 --- a/src/ipa/raspberrypi/controller/rpi/alsc.cpp +++ b/src/ipa/raspberrypi/controller/rpi/alsc.cpp @@ -607,7 +607,7 @@ static double computeWeight(double Ci, double Cj, double sigma) /* Compute all weights. */ static void computeW(const Array2D &C, double sigma, - std::vector> &W) + SparseArray &W) { size_t XY = C.size(); size_t X = C.dimensions().width; @@ -623,8 +623,8 @@ static void computeW(const Array2D &C, double sigma, /* Compute M, the large but sparse matrix such that M * lambdas = 0. */ static void constructM(const Array2D &C, - const std::vector> &W, - std::vector> &M) + const SparseArray &W, + SparseArray &M) { size_t XY = C.size(); size_t X = C.dimensions().width; @@ -651,37 +651,37 @@ static void constructM(const Array2D &C, * left/right neighbours are zero down the left/right edges, so we don't need * need to test the i value to exclude them. */ -static double computeLambdaBottom(int i, const std::vector> &M, +static double computeLambdaBottom(int i, const SparseArray &M, Array2D &lambda) { return M[i][1] * lambda[i + 1] + M[i][2] * lambda[i + lambda.dimensions().width] + M[i][3] * lambda[i - 1]; } -static double computeLambdaBottomStart(int i, const std::vector> &M, +static double computeLambdaBottomStart(int i, const SparseArray &M, Array2D &lambda) { return M[i][1] * lambda[i + 1] + M[i][2] * lambda[i + lambda.dimensions().width]; } -static double computeLambdaInterior(int i, const std::vector> &M, +static double computeLambdaInterior(int i, const SparseArray &M, Array2D &lambda) { return M[i][0] * lambda[i - lambda.dimensions().width] + M[i][1] * lambda[i + 1] + M[i][2] * lambda[i + lambda.dimensions().width] + M[i][3] * lambda[i - 1]; } -static double computeLambdaTop(int i, const std::vector> &M, +static double computeLambdaTop(int i, const SparseArray &M, Array2D &lambda) { return M[i][0] * lambda[i - lambda.dimensions().width] + M[i][1] * lambda[i + 1] + M[i][3] * lambda[i - 1]; } -static double computeLambdaTopEnd(int i, const std::vector> &M, +static double computeLambdaTopEnd(int i, const SparseArray &M, Array2D &lambda) { return M[i][0] * lambda[i - lambda.dimensions().width] + M[i][3] * lambda[i - 1]; } /* Gauss-Seidel iteration with over-relaxation. */ -static double gaussSeidel2Sor(const std::vector> &M, double omega, +static double gaussSeidel2Sor(const SparseArray &M, double omega, Array2D &lambda, double lambdaBound) { int XY = lambda.size(); @@ -753,8 +753,8 @@ static void reaverage(Array2D &data) static void runMatrixIterations(const Array2D &C, Array2D &lambda, - const std::vector> &W, - std::vector> &M, double omega, + const SparseArray &W, + SparseArray &M, double omega, unsigned int nIter, double threshold, double lambdaBound) { constructM(C, W, M); @@ -813,7 +813,7 @@ void Alsc::doAlsc() { Array2D &cr = tmpC_[0], &cb = tmpC_[1], &calTableR = tmpC_[2], &calTableB = tmpC_[3], &calTableTmp = tmpC_[4]; - std::vector> &wr = tmpM_[0], &wb = tmpM_[1], &M = tmpM_[2]; + SparseArray &wr = tmpM_[0], &wb = tmpM_[1], &M = tmpM_[2]; /* * Calculate our R/B ("Cr"/"Cb") colour statistics, and assess which are diff --git a/src/ipa/raspberrypi/controller/rpi/alsc.h b/src/ipa/raspberrypi/controller/rpi/alsc.h index 1ab61299c4cd..0b6d9478073c 100644 --- a/src/ipa/raspberrypi/controller/rpi/alsc.h +++ b/src/ipa/raspberrypi/controller/rpi/alsc.h @@ -68,6 +68,14 @@ private: std::vector data_; }; +/* + * We'll use the term SparseArray for the large sparse matrices that are + * XY tall but have only 4 non-zero elements on each row. + */ + +template +using SparseArray = std::vector>; + struct AlscCalibration { double ct; Array2D table; @@ -160,7 +168,7 @@ private: /* Temporaries for the computations */ std::array, 5> tmpC_; - std::array>, 3> tmpM_; + std::array, 3> tmpM_; }; } /* namespace RPiController */ From patchwork Wed Mar 22 13:06:08 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Naushir Patuck X-Patchwork-Id: 18433 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 5B29DC0F1B for ; Wed, 22 Mar 2023 13:06:29 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id BA2E762724; Wed, 22 Mar 2023 14:06:28 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1679490389; bh=9A/rwptP+p9jk3E0TaYU1mMV8Yim4LEVhI7SIMYQAvA=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To: From; b=Q3iLK/oLXkjQ2e8AVm8dEq7at38NzgnzFAYSsCOAnE3epxGzvPW6ZWaZIn26TX1NC tUiZKXThZpjEhpL+SxVVT3Po9M4/bxWmf+0y+CL4CXkhAg6/1udi2lDEU7+vFsiyT5 9hnRKiCLY9LWrEWV61ZCQ791HHute5/CUm+ACtKbJP5VkWbHJnfNP6v+iYVlVSp9hb HY17XyxMHNZlLqYvAE0lBarm5ntcpUyEtzDIHmaZJoCx06oAIAXsM6ZRpunjyhXhug xxdom8Fgg7rpnRXxcaEO+OLXVlXcOXJ3IZhgKvVjRE04/NeBaAYO/QbTLVJ9ZcQeWC KSz0Q3Ub+lzBw== Received: from mail-il1-x132.google.com (mail-il1-x132.google.com [IPv6:2607:f8b0:4864:20::132]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 7040E62716 for ; Wed, 22 Mar 2023 14:06:25 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="JYsPqXNI"; dkim-atps=neutral Received: by mail-il1-x132.google.com with SMTP id j6so9804706ilr.7 for ; Wed, 22 Mar 2023 06:06:25 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; t=1679490384; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=yoxd2YPF+mGleci8HMNu54DMJ+86Zo6lhHkuT0jgQAk=; b=JYsPqXNIpAGYp99P9jBi9Vnf1yKSII4hZyJ4yedHgFFCVk9Wn9soVCXdZafSxgNhIQ OaPGaBmpKlt6bmL9qLpGCRhtOOO7rSPqv4ycaksAgWTvY3cESUzTLy3H8KRgIyi3GAQk A0yHL2eUUAILOQdczfWtnrOoAQYZwVk385jJpb2wTo+3ix2310e0//9Z/opq+btCOBRQ vz6hJ1NBp41/CmfoMTVNqYgNc/py1wtiplyauQjIiZbg1pn/dlSe9nSJbOiZ6qq3YAGc NQM2FncyiukrqpKSetwfTsrKoS7Zi6HWIUXRRJIOUHwaj2HcWD08h4GweaQfo4rVekis ILnA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; t=1679490384; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=yoxd2YPF+mGleci8HMNu54DMJ+86Zo6lhHkuT0jgQAk=; b=hOPm51RIehbSGVcjwyL1EzIQ3SuqoZcpxIDW1eTMaSJV16PXChvXQiu2fesTYgIq9C kzhGz3FfnB8ByQVquOF1BNFOSq6rheZDw7CFmPSGrPvurVCvonlQUgQgyMn4EHM97x1G 0ULpPUy19ykexe/EMe/QseLPm5hbzLzo3huROpAclywvR804PaZE1fcoBZZnUCUN4h5Y JAz4UzJXLakIufXsyHZfIyz2YCCmxPmfTDqt7VD+q9FmAgRpkbbYE2G3/mR3RkUnbnPo ASwogL6rWzOAXCnUMT4gYDL3W3uCUE9MzP5yvbpSJwGFizBUmswehsilsNiNwLFhOmNY 8+QA== X-Gm-Message-State: AO0yUKXR/WHAwVgnY7pegHSPgTw80mqgLZKBxSQSTvT9UNJ/6ZB77VSQ Hmb5JuW7qtzesd7g6RIamG8QlYbK/ZFA/RkPe+TtRQ== X-Google-Smtp-Source: AK7set/oen4OaDsZ/oc9QFKjKAFDFYcSXXjiDCiYv7GOMSZenutodUrV1V6g43K9bGdewMWha4sduw== X-Received: by 2002:a92:dac9:0:b0:325:bc3f:f757 with SMTP id o9-20020a92dac9000000b00325bc3ff757mr1588557ilq.0.1679490383992; Wed, 22 Mar 2023 06:06:23 -0700 (PDT) Received: from localhost.localdomain ([93.93.133.154]) by smtp.gmail.com with ESMTPSA id l6-20020a026646000000b003c488204c6fsm4927960jaf.76.2023.03.22.06.06.23 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 22 Mar 2023 06:06:23 -0700 (PDT) To: libcamera-devel@lists.libcamera.org Date: Wed, 22 Mar 2023 13:06:08 +0000 Message-Id: <20230322130612.5208-7-naush@raspberrypi.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230322130612.5208-1-naush@raspberrypi.com> References: <20230322130612.5208-1-naush@raspberrypi.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v1 06/10] ipa: raspberrypi: Generalise the contrast algorithm X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Naushir Patuck via libcamera-devel From: Naushir Patuck Reply-To: Naushir Patuck Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Generalise the contrast algorithm code by removing any hard-coded assumptions about the target hardware platform. Instead, the algorithm code creates a generic Pwl that gets returned to the IPA, where it gets converted to the bcm2835 hardware specific lookup table. As a drive-by, remove an unused mutex. Signed-off-by: Naushir Patuck Reviewed-by: David Plowman Reviewed-by: Jacopo Mondi --- .../raspberrypi/controller/contrast_status.h | 11 ++----- .../raspberrypi/controller/rpi/contrast.cpp | 30 ++++--------------- src/ipa/raspberrypi/controller/rpi/contrast.h | 1 - src/ipa/raspberrypi/raspberrypi.cpp | 15 +++++++--- 4 files changed, 20 insertions(+), 37 deletions(-) diff --git a/src/ipa/raspberrypi/controller/contrast_status.h b/src/ipa/raspberrypi/controller/contrast_status.h index ef2a7c680fc2..fb9fe4bace71 100644 --- a/src/ipa/raspberrypi/controller/contrast_status.h +++ b/src/ipa/raspberrypi/controller/contrast_status.h @@ -6,20 +6,15 @@ */ #pragma once +#include "pwl.h" + /* * The "contrast" algorithm creates a gamma curve, optionally doing a little bit * of contrast stretching based on the AGC histogram. */ -constexpr unsigned int ContrastNumPoints = 33; - -struct ContrastPoint { - uint16_t x; - uint16_t y; -}; - struct ContrastStatus { - struct ContrastPoint points[ContrastNumPoints]; + RPiController::Pwl gammaCurve; double brightness; double contrast; }; diff --git a/src/ipa/raspberrypi/controller/rpi/contrast.cpp b/src/ipa/raspberrypi/controller/rpi/contrast.cpp index a4f8c4f04fc4..bee1eadd3657 100644 --- a/src/ipa/raspberrypi/controller/rpi/contrast.cpp +++ b/src/ipa/raspberrypi/controller/rpi/contrast.cpp @@ -65,34 +65,19 @@ void Contrast::setContrast(double contrast) contrast_ = contrast; } -static void fillInStatus(ContrastStatus &status, double brightness, - double contrast, Pwl &gammaCurve) -{ - status.brightness = brightness; - status.contrast = contrast; - for (unsigned int i = 0; i < ContrastNumPoints - 1; i++) { - int x = i < 16 ? i * 1024 - : (i < 24 ? (i - 16) * 2048 + 16384 - : (i - 24) * 4096 + 32768); - status.points[i].x = x; - status.points[i].y = std::min(65535.0, gammaCurve.eval(x)); - } - status.points[ContrastNumPoints - 1].x = 65535; - status.points[ContrastNumPoints - 1].y = 65535; -} - void Contrast::initialise() { /* * Fill in some default values as Prepare will run before Process gets * called. */ - fillInStatus(status_, brightness_, contrast_, config_.gammaCurve); + status_.brightness = brightness_; + status_.contrast = contrast_; + status_.gammaCurve = config_.gammaCurve; } void Contrast::prepare(Metadata *imageMetadata) { - std::unique_lock lock(mutex_); imageMetadata->set("contrast.status", status_); } @@ -183,12 +168,9 @@ void Contrast::process(StatisticsPtr &stats, * And fill in the status for output. Use more points towards the bottom * of the curve. */ - ContrastStatus status; - fillInStatus(status, brightness_, contrast_, gammaCurve); - { - std::unique_lock lock(mutex_); - status_ = status; - } + status_.brightness = brightness_; + status_.contrast = contrast_; + status_.gammaCurve = std::move(gammaCurve); } /* Register algorithm with the system. */ diff --git a/src/ipa/raspberrypi/controller/rpi/contrast.h b/src/ipa/raspberrypi/controller/rpi/contrast.h index c68adbc9e463..9c81277a0450 100644 --- a/src/ipa/raspberrypi/controller/rpi/contrast.h +++ b/src/ipa/raspberrypi/controller/rpi/contrast.h @@ -46,7 +46,6 @@ private: double brightness_; double contrast_; ContrastStatus status_; - std::mutex mutex_; }; } /* namespace RPiController */ diff --git a/src/ipa/raspberrypi/raspberrypi.cpp b/src/ipa/raspberrypi/raspberrypi.cpp index 0fa79bb4af41..b6e5465ca32a 100644 --- a/src/ipa/raspberrypi/raspberrypi.cpp +++ b/src/ipa/raspberrypi/raspberrypi.cpp @@ -1592,14 +1592,21 @@ void IPARPi::applyCCM(const struct CcmStatus *ccmStatus, ControlList &ctrls) void IPARPi::applyGamma(const struct ContrastStatus *contrastStatus, ControlList &ctrls) { + const unsigned int numGammaPoints = controller_.getHardwareConfig().numGammaPoints; struct bcm2835_isp_gamma gamma; - gamma.enabled = 1; - for (unsigned int i = 0; i < ContrastNumPoints; i++) { - gamma.x[i] = contrastStatus->points[i].x; - gamma.y[i] = contrastStatus->points[i].y; + for (unsigned int i = 0; i < numGammaPoints - 1; i++) { + int x = i < 16 ? i * 1024 + : (i < 24 ? (i - 16) * 2048 + 16384 + : (i - 24) * 4096 + 32768); + gamma.x[i] = x; + gamma.y[i] = std::min(65535, contrastStatus->gammaCurve.eval(x)); } + gamma.x[numGammaPoints - 1] = 65535; + gamma.y[numGammaPoints - 1] = 65535; + gamma.enabled = 1; + ControlValue c(Span{ reinterpret_cast(&gamma), sizeof(gamma) }); ctrls.set(V4L2_CID_USER_BCM2835_ISP_GAMMA, c); From patchwork Wed Mar 22 13:06:09 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Naushir Patuck X-Patchwork-Id: 18434 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id E352CC32A0 for ; Wed, 22 Mar 2023 13:06:29 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 8051362710; Wed, 22 Mar 2023 14:06:29 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1679490389; bh=YMeB7Yt2BHmOWFHNjvg7Zwd4fmpx4B1nhACxM1E/89M=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To: From; b=zhdWyyIYpW8jEwJUR1ec33uXEABulbJ0T5Z3tRs4ai65xtehmk4CGyBdrmimkUXwC OoeVpMOCPtZ0Wkb/tZp99kDdlZwFNjharvetHt1enfkGk3EMEu9xQhvcvBXsTP1Rzn MJ8i9cERQIdPRuk8sVGrqQH0FEUACONuf4RcZkT8iyjxI+YU44uNexP6k7S9FVmeyP xJhBm8fDdURNz/LeeoR2LTnuTNM2CQX/2VWKyFlbXi5vrxcM91L+tqMPQUx3y54Yjr dJZCMA1EAwh6ep1hzri7rM0u6Pn0A/owMSHygXk26P62FolV4c8vGXKOTIdI0pYIrl myGNCEI+vAkeQ== Received: from mail-io1-xd35.google.com (mail-io1-xd35.google.com [IPv6:2607:f8b0:4864:20::d35]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 7631C62722 for ; Wed, 22 Mar 2023 14:06:26 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="SyCfU+4M"; dkim-atps=neutral Received: by mail-io1-xd35.google.com with SMTP id t129so8396943iof.12 for ; Wed, 22 Mar 2023 06:06:26 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; t=1679490385; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=2D3sopp01tCN1h+VaYdiNoKyN5GcBEkpOHldOdbSyLk=; b=SyCfU+4M0wa9n8zfhgQL+cY36Zt7a0UrWjT/srQIXktpiQOWevik88a3KeyNrGMk/H IKStUo0hPOxjpje4E/MkkHjZvy8vBhaETm0nW7ht72bF6bT6QKIedrXeauUV4Z+T1ylr zsQQaZLFRJWZXU+6zl8wlk6kg2LIBAzX15hPf4tfb/FZ3fItvTZ0ABzviW9L9ztbuNM7 RwPGDKzGvuJnvswouNujzjEckKGq1mdDCsygTsu61XdpTW+gEjBjnoUwyRdl7r8KrP60 hYnNR447coY/OH8aZSkNiZ6Ts6Dh7hzKFnEzxTnmgYc9U0sYFGuc7xSPLpF6OHlRrYro TABw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; t=1679490385; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=2D3sopp01tCN1h+VaYdiNoKyN5GcBEkpOHldOdbSyLk=; b=3W8gfMzwn8lDhmFkrcYvP90zJLhbOK5AHO5162jrLGbJFePnEi7T/Aqi6jVcTdhsXV tZBk2H9ywWip2UMcwF2E4w8EqVzdC7LUWjb4++mzMHD4BDMfCHtTYCqR9u7FHlRaRBhk 8QKJ/FO75w3T6fmlxj8xECvS79mBmVRbULeu4TVAsbzp1xO5WJzreV9TJjDax+Q6DSQR nsFcvN3qt5D5kssug0mPWw2V6LNuw7cyn/FbSrSPDDNIAsfDsiwKmVVJ89Zzw5Kzqi+i SNHvNtn/5liNKjr8VP5DCIoZobOoPr0kp/lLNk0iXGEzY0vGioDytSlFRMdm1n+tT7KL d3sQ== X-Gm-Message-State: AO0yUKXg/vMjJHaiYDWf0iiUEalUwthP+PgbMUjETUOU1tgo5yjh2eM4 V4q3AO5DZxFkL2y6HVC8g6r+JT+LQuG5oDRqy17gTQ== X-Google-Smtp-Source: AK7set9AoERnoUHAtrSN3TGtIdEJFpOgeB8ylyX9D4nUxtAkK6DOmKOD8+zVA7mxONDHHjiQL5KAyw== X-Received: by 2002:a6b:b4c5:0:b0:74c:aa8f:1f4c with SMTP id d188-20020a6bb4c5000000b0074caa8f1f4cmr1220412iof.8.1679490385012; Wed, 22 Mar 2023 06:06:25 -0700 (PDT) Received: from localhost.localdomain ([93.93.133.154]) by smtp.gmail.com with ESMTPSA id l6-20020a026646000000b003c488204c6fsm4927960jaf.76.2023.03.22.06.06.24 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 22 Mar 2023 06:06:24 -0700 (PDT) To: libcamera-devel@lists.libcamera.org Date: Wed, 22 Mar 2023 13:06:09 +0000 Message-Id: <20230322130612.5208-8-naush@raspberrypi.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230322130612.5208-1-naush@raspberrypi.com> References: <20230322130612.5208-1-naush@raspberrypi.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v1 07/10] ipa: raspberrypi: Make RegionStats::get() always return a Region struct X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Naushir Patuck via libcamera-devel From: Naushir Patuck Reply-To: Naushir Patuck Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Update the overloaded RegionStats::get() and RegionStats::getFloating() member functions to return a Region struct for consistency. Signed-off-by: Naushir Patuck Reviewed-by: David Plowman Reviewed-by: Jacopo Mondi --- src/ipa/raspberrypi/controller/region_stats.h | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/src/ipa/raspberrypi/controller/region_stats.h b/src/ipa/raspberrypi/controller/region_stats.h index 9aaf3a58a6f7..a8860dc8dba2 100644 --- a/src/ipa/raspberrypi/controller/region_stats.h +++ b/src/ipa/raspberrypi/controller/region_stats.h @@ -86,12 +86,12 @@ public: return get_(index); } - const T &get(const libcamera::Point &pos) const + const Region &get(const libcamera::Point &pos) const { return get(pos.y * size_.width + pos.x); } - const T &getFloating(unsigned int index) const + const Region &getFloating(unsigned int index) const { if (index >= numFloatingRegions()) return default_; From patchwork Wed Mar 22 13:06:10 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Naushir Patuck X-Patchwork-Id: 18435 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 4CDDBC32A1 for ; Wed, 22 Mar 2023 13:06:31 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id C8EF76273D; Wed, 22 Mar 2023 14:06:30 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1679490390; bh=vly+N8kD070ySftpvX91l+mQboT8yWAuvyihR8zCM34=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To: From; b=aV59BNqEI8iuNUQnhZIv1NY0sSsjA/OTOii03Ge+WT+YrVav/1PZhJAlRJzTcGw0D OJElfp/sDLhkjpXlclBqgl+DNbn0T/c6RSU6T3sjsfrmWZW7AK1cbAn19hSiUwvhXY C/on0DqaKYB9TnhP8eqVTeaotl8627HSH8RHzdt/kqNckCcAwKALPT5Vsc4lnO0IR4 mqqjPsCqM+lf3NK+w5v+g0HPezz/vPNWE1nQONfcDGJGUa9FobRkBjSwF9cHOt4Eyj RQos0exCsB1wCTWhtV1tfUodFnVWXptvmml0kXdjcLyUUyWEwxwLYtUW8ST1GAZZfs iVF4TDkANh0eg== Received: from mail-io1-xd32.google.com (mail-io1-xd32.google.com [IPv6:2607:f8b0:4864:20::d32]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id CCA0B62716 for ; Wed, 22 Mar 2023 14:06:26 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="pd246ll/"; dkim-atps=neutral Received: by mail-io1-xd32.google.com with SMTP id s4so8386058ioj.11 for ; Wed, 22 Mar 2023 06:06:26 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; t=1679490386; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=VTK7RKvARCpPhRDMEiuWMbT7meHOFDF62AOEWuoNos8=; b=pd246ll/74pNccmnICdWx6Jm4G0Y0+Irarb3wx3Sze12V7FSCnk1mHr8z+xG6QSaEX qPIW3QqOZXJQJwgBOI/+kXGVV/PfD40qly+jUjj/z1VOAio2wbgEa1xB4/7R12Idc6zD u5dxiYT7Fd3y117eOoOqwPYILkMnM7Bs/upnxWixfEiv8Zb2RajjUIiMi8sFCvKZyrqF xvj1TV5QW6rwrCpDPnH+YOkPcqqyttqNAai79KlE96tjGxt2Hew62agqGxoKrAlheQK6 3y7AVyvt8PUiZ+PCH7kKKGZ8bJp2pYUhFHIU3UcP51U93snhfqOEFwMchQn0sbbdjzR+ AZMA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; t=1679490386; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=VTK7RKvARCpPhRDMEiuWMbT7meHOFDF62AOEWuoNos8=; b=XnBeoi2nBoCo3vzWfaYvjQsSvlg3/jqqxIwgOsYPw8fGOMr0kxSIL8yNnFTC/Y/uxF 0WzVQeuIebSsiU8wjIHYoVuPAJ4ca4P7tRlBwb9p/I8dMArxdFaJcL5kgu7S8ExdmfDA FhyXV3BmFMFR7PDNSFwxRloeaGVUg46u+sqFQCjBre+v4sg+OPdrMpWEecVpw3FFdAQv ZQNiQqHhG47B3jnrEo7AZj5tLg7OyaECJe634XCKMhusf4wIJfH7ufmfOP1o0OI1fJUn Jzcc3m4FGbCq0NlUVUAdFCwUyAcTfoHvXqsqIgwSETxMdOpfIW4RELXaiTTuM9WJXeC8 myFA== X-Gm-Message-State: AO0yUKWx8IrId0zG/4PH4yTEYfj82aVFcv4a++S5U26AqACrHyi0Onp2 e+eq65VDzGAsuvoulkzGzRhxdQZC4XcEUVbX4cWsbA== X-Google-Smtp-Source: AK7set+cjpe2x/LRyuaMf/dDNdqDoD3bC5YsNufoGxaKAceILUGhko88mBp9gnE7n55sI7G8xbCsjw== X-Received: by 2002:a5e:db02:0:b0:746:1c75:233a with SMTP id q2-20020a5edb02000000b007461c75233amr4547901iop.20.1679490386244; Wed, 22 Mar 2023 06:06:26 -0700 (PDT) Received: from localhost.localdomain ([93.93.133.154]) by smtp.gmail.com with ESMTPSA id l6-20020a026646000000b003c488204c6fsm4927960jaf.76.2023.03.22.06.06.25 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 22 Mar 2023 06:06:25 -0700 (PDT) To: libcamera-devel@lists.libcamera.org Date: Wed, 22 Mar 2023 13:06:10 +0000 Message-Id: <20230322130612.5208-9-naush@raspberrypi.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230322130612.5208-1-naush@raspberrypi.com> References: <20230322130612.5208-1-naush@raspberrypi.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v1 08/10] ipa: raspberrypi: Generalise the focus algorithm X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Naushir Patuck via libcamera-devel From: Naushir Patuck Reply-To: Naushir Patuck Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Generalise the focus reporting algorithm code by removing any hard-coded assumptions about the target hardware platform. Instead, the algorithm code uses the generalised statistics structure to determine focus region sizes. Remove focus_status.h as it is no longer needed with this change. Signed-off-by: Naushir Patuck Reviewed-by: Jacopo Mondi --- src/ipa/raspberrypi/controller/focus_status.h | 20 ------------------- src/ipa/raspberrypi/controller/rpi/focus.cpp | 12 ++--------- src/ipa/raspberrypi/raspberrypi.cpp | 11 ++++++---- 3 files changed, 9 insertions(+), 34 deletions(-) delete mode 100644 src/ipa/raspberrypi/controller/focus_status.h diff --git a/src/ipa/raspberrypi/controller/focus_status.h b/src/ipa/raspberrypi/controller/focus_status.h deleted file mode 100644 index 8b74e59840c1..000000000000 --- a/src/ipa/raspberrypi/controller/focus_status.h +++ /dev/null @@ -1,20 +0,0 @@ -/* SPDX-License-Identifier: BSD-2-Clause */ -/* - * Copyright (C) 2020, Raspberry Pi Ltd - * - * focus_status.h - focus measurement status - */ -#pragma once - -#include - -/* - * The focus algorithm should post the following structure into the image's - * "focus.status" metadata. Recall that it's only reporting focus (contrast) - * measurements, it's not driving any kind of auto-focus algorithm! - */ - -struct FocusStatus { - unsigned int num; - uint32_t focusMeasures[FOCUS_REGIONS]; -}; diff --git a/src/ipa/raspberrypi/controller/rpi/focus.cpp b/src/ipa/raspberrypi/controller/rpi/focus.cpp index ea3cc00e42c3..683d460ad6f7 100644 --- a/src/ipa/raspberrypi/controller/rpi/focus.cpp +++ b/src/ipa/raspberrypi/controller/rpi/focus.cpp @@ -8,7 +8,6 @@ #include -#include "../focus_status.h" #include "focus.h" using namespace RPiController; @@ -30,15 +29,8 @@ char const *Focus::name() const void Focus::process(StatisticsPtr &stats, Metadata *imageMetadata) { - FocusStatus status; - for (unsigned int i = 0; i < stats->focusRegions.numRegions(); i++) - status.focusMeasures[i] = stats->focusRegions.get(i).val; - status.num = stats->focusRegions.numRegions(); - imageMetadata->set("focus.status", status); - - LOG(RPiFocus, Debug) - << "Focus contrast measure: " - << (status.focusMeasures[5] + status.focusMeasures[6]) / 10; + /* Pass the stats directly to the IPA to report out to the application */ + imageMetadata->set("focus.status", stats->focusRegions); } /* Register algorithm with the system. */ diff --git a/src/ipa/raspberrypi/raspberrypi.cpp b/src/ipa/raspberrypi/raspberrypi.cpp index b6e5465ca32a..181512de74ad 100644 --- a/src/ipa/raspberrypi/raspberrypi.cpp +++ b/src/ipa/raspberrypi/raspberrypi.cpp @@ -50,7 +50,6 @@ #include "denoise_algorithm.h" #include "denoise_status.h" #include "dpc_status.h" -#include "focus_status.h" #include "geq_status.h" #include "lux_status.h" #include "metadata.h" @@ -640,14 +639,18 @@ void IPARPi::reportMetadata(unsigned int ipaContext) static_cast(blackLevelStatus->blackLevelG), static_cast(blackLevelStatus->blackLevelB) }); - FocusStatus *focusStatus = rpiMetadata.getLocked("focus.status"); - if (focusStatus && focusStatus->num == 12) { + RPiController::FocusRegions *focusStatus = + rpiMetadata.getLocked("focus.status"); + if (focusStatus) { + libcamera::Size size = focusStatus->size(); /* * We get a 4x3 grid of regions by default. Calculate the average * FoM over the central two positions to give an overall scene FoM. * This can change later if it is not deemed suitable. */ - int32_t focusFoM = (focusStatus->focusMeasures[5] + focusStatus->focusMeasures[6]) / 2; + ASSERT(size == Size(4, 3)); + int32_t focusFoM = (focusStatus->get({ 1, 1 }).val + + focusStatus->get({ 2, 1 }).val) / 2; libcameraMetadata_.set(controls::FocusFoM, focusFoM); } From patchwork Wed Mar 22 13:06:11 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Naushir Patuck X-Patchwork-Id: 18436 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 3D9C1C32A2 for ; Wed, 22 Mar 2023 13:06:34 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 9BCA56270D; Wed, 22 Mar 2023 14:06:33 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1679490393; bh=icouwjstIJlplcREzPnVduaDDhW/ubniFTTqrI/y6FU=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To: From; b=xLV22dgMXQNFzxvoSsch1uRDjhg4Sg5YfCATW3JIi4S9UqlsXFfKz5x4LbvCGfPnh g3tGQygmnD8lWlb/JPBi5I7Vuo7Wq9ZqHxRNJtnQwqAIlP4lSDnfLnB0edWBcQmQDL OcfpUTr1RrEHe21Vn+kp+8l8H7MLJM0g8Ws3Jr0dvJVExjlNPTzjgeNj2pBMh/c874 FqLcwMT2X1Q7fu6VoA98c2CG5Q3jGeKP2/Z8XQG7sCpxlPOFrNQR/TXj8fP3oV8PlD GTSIK4BlcQBajK3uJoLvOIXD1+5ozDeNEnTrzxcwySXlLy/9YK5UAMFEvh77nh9C1e bj9J91VgeS2Xw== Received: from mail-il1-x12a.google.com (mail-il1-x12a.google.com [IPv6:2607:f8b0:4864:20::12a]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id B28A762742 for ; Wed, 22 Mar 2023 14:06:28 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="LRRacJ87"; dkim-atps=neutral Received: by mail-il1-x12a.google.com with SMTP id s1so3865698ild.6 for ; Wed, 22 Mar 2023 06:06:28 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; t=1679490387; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=GM7wpbXpS06YcWhjewbppISe52faOHMFSk9/O0DNzEQ=; b=LRRacJ87oZGnJkjkesd+Ht/3B3SNA27IZgTMEbB+of/ASQoNt7YOki2JRslxNrDSyT LD6cYpqXp0dvz1wVptEJboQQc3NBT7OApqCel4OUivYZ5c4R9ACwJ0Va4i89ww2/q4cG nx0C9j2+F/dnRLL9JmqrVDea23p+J7RNCgFdN6yKYfUj3CAjLLIv4KNdlHwv5pTQP56K 4fE2cN8O3nokyTFfCOjpxUHC3MqPZzPQLOTo3ZDfUy4yQxSSS8btl1ZV/W4kFt70x1w0 BpuknBzR9geu6ExjhQFf4Z8o8sW3+Rskm00NKs+valOHMEbq/SPaPh7mZgcnKWV2mlR7 VL/g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; t=1679490387; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=GM7wpbXpS06YcWhjewbppISe52faOHMFSk9/O0DNzEQ=; b=b3+KLZUnw+cunhU7obXWL/Swmjxbk+yM5Fprnf6J4/5SPGq0B9UUc+pJWgY3RubceC CnVPNoLyyH5aUZ8uDIq/8KJIw7EE236sqYIvH52SVTk3L9E+pATz5oH8YnaJxNtIbdOx vMAPnPBrYv0zoXVApUXf6R4PDkrxZYyvfrl/DfsqdIlN7xc2IXZ+uAOlq8U+GCL/KASa TyaQD0iKPrLsn3HY63RQ2RvpVHbx9W9tI9I3FLkHHYzoavlsciAqGMWT99P+nhRPSWlf cGLEMj5q+237hh9YCpLFr+cNotSTk168PCZsxM5Jkqxc9rY0J3YGDgAHhToLJoo/8hTR K6tw== X-Gm-Message-State: AO0yUKVvMELPjiO0OZtwV/OYXaP7Qw8KrpVLMu03YPURSQ0ak2wP31xA yV2KyssA+tf8kWvrD3ssk4U9QhlVzp5vKAlp9q9fwg== X-Google-Smtp-Source: AK7set8PLqXmH0UGWOwh5b25oKKhBUnJz48E4VC3aauokboJ7eB6nS2E70NvbIckivVwVoDCTVt6Dw== X-Received: by 2002:a92:c80c:0:b0:313:fea2:be5c with SMTP id v12-20020a92c80c000000b00313fea2be5cmr4201055iln.11.1679490387186; Wed, 22 Mar 2023 06:06:27 -0700 (PDT) Received: from localhost.localdomain ([93.93.133.154]) by smtp.gmail.com with ESMTPSA id l6-20020a026646000000b003c488204c6fsm4927960jaf.76.2023.03.22.06.06.26 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 22 Mar 2023 06:06:26 -0700 (PDT) To: libcamera-devel@lists.libcamera.org Date: Wed, 22 Mar 2023 13:06:11 +0000 Message-Id: <20230322130612.5208-10-naush@raspberrypi.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230322130612.5208-1-naush@raspberrypi.com> References: <20230322130612.5208-1-naush@raspberrypi.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v1 09/10] ipa: raspberrypi: Generalise the agc algorithm X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Naushir Patuck via libcamera-devel From: Naushir Patuck Reply-To: Naushir Patuck Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Remove any hard-coded assumptions about the target hardware platform from the AGC algorithm. Instead, use the "target" string provided by the camera tuning config and generalised statistics structures to determing parameters such as grid and region sizes. This change replaces all hard-coded arrays with equivalent std::vector types. Signed-off-by: Naushir Patuck Reviewed-by: David Plowman Reviewed-by: Jacopo Mondi --- src/ipa/raspberrypi/controller/rpi/agc.cpp | 19 ++++++++++++------- src/ipa/raspberrypi/controller/rpi/agc.h | 9 +-------- 2 files changed, 13 insertions(+), 15 deletions(-) diff --git a/src/ipa/raspberrypi/controller/rpi/agc.cpp b/src/ipa/raspberrypi/controller/rpi/agc.cpp index 4ea0dd41e66c..3e4a8149b9ae 100644 --- a/src/ipa/raspberrypi/controller/rpi/agc.cpp +++ b/src/ipa/raspberrypi/controller/rpi/agc.cpp @@ -31,17 +31,12 @@ LOG_DEFINE_CATEGORY(RPiAgc) int AgcMeteringMode::read(const libcamera::YamlObject ¶ms) { const YamlObject &yamlWeights = params["weights"]; - if (yamlWeights.size() != AgcStatsSize) { - LOG(RPiAgc, Error) << "AgcMeteringMode: Incorrect number of weights"; - return -EINVAL; - } - unsigned int num = 0; for (const auto &p : yamlWeights.asList()) { auto value = p.get(); if (!value) return -EINVAL; - weights[num++] = *value; + weights.push_back(*value); } return 0; @@ -249,6 +244,14 @@ int Agc::read(const libcamera::YamlObject ¶ms) if (ret) return ret; + const Size &size = getHardwareConfig().agcZoneWeights; + for (auto const &modes : config_.meteringModes) { + if (modes.second.weights.size() != size.width * size.height) { + LOG(RPiAgc, Error) << "AgcMeteringMode: Incorrect number of weights"; + return -EINVAL; + } + } + /* * Set the config's defaults (which are the first ones it read) as our * current modes, until someone changes them. (they're all known to @@ -582,9 +585,11 @@ void Agc::fetchAwbStatus(Metadata *imageMetadata) } static double computeInitialY(StatisticsPtr &stats, AwbStatus const &awb, - double weights[], double gain) + std::vector &weights, double gain) { constexpr uint64_t maxVal = 1 << Statistics::NormalisationFactorPow2; + + ASSERT(weights.size() == stats->agcRegions.numRegions()); /* * Note how the calculation below means that equal weights give you * "average" metering (i.e. all pixels equally important). diff --git a/src/ipa/raspberrypi/controller/rpi/agc.h b/src/ipa/raspberrypi/controller/rpi/agc.h index f04896ca25ad..d11a49c9cd85 100644 --- a/src/ipa/raspberrypi/controller/rpi/agc.h +++ b/src/ipa/raspberrypi/controller/rpi/agc.h @@ -17,17 +17,10 @@ /* This is our implementation of AGC. */ -/* - * This is the number actually set up by the firmware, not the maximum possible - * number (which is 16). - */ - -constexpr unsigned int AgcStatsSize = 15; - namespace RPiController { struct AgcMeteringMode { - double weights[AgcStatsSize]; + std::vector weights; int read(const libcamera::YamlObject ¶ms); }; From patchwork Wed Mar 22 13:06:12 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Naushir Patuck X-Patchwork-Id: 18437 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 09238C32A3 for ; Wed, 22 Mar 2023 13:06:35 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id EEF4A62712; Wed, 22 Mar 2023 14:06:34 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1679490395; bh=GJMOQsM2qNHNfPLEkuukBe56ZHxjjuewJDu4faUllL0=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To:Cc: From; b=nRmmxgGNuszz8wPYQPO+G2kX4qFqzUx8xnCWMH7pauXtBHzUVjUChw2M7d76ycVDO 9PF34Goi8a2FXbhFdy4u3GhlPm5w97ghasVTh9BVTozOFAVwr5LThB6M3uC3IZE9zv 8v4zNdrLZaYdi/iQY3VmH3zRWJr/km3tImMe/Z0yJCpznxOls+OYTdVRSZ4lflhb2P BvHJb+HFSIwgfbJIuZX/wReIyJV6CtfWgpPnHn6ZmP89YePlx56PyXcbjJoy16Gnzn rpK484TNZl+Tw4O0tO3+79bHFJy/v1qjcMyw/frvE9M98LLwIL5TWAob4QZUduLWBE grac2NwdPv1XQ== Received: from mail-io1-xd34.google.com (mail-io1-xd34.google.com [IPv6:2607:f8b0:4864:20::d34]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id DDEF6626E5 for ; Wed, 22 Mar 2023 14:06:29 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="S039sO3c"; dkim-atps=neutral Received: by mail-io1-xd34.google.com with SMTP id bf15so8390195iob.7 for ; Wed, 22 Mar 2023 06:06:29 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; t=1679490388; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=Nq0ULfiQemL8gPeO7f/KCl9WXEmlKDB3Q+oiImcogL4=; b=S039sO3cuGMDsp6PSBPxl2mHzIASkYagldTlKO8ut8O21pNpljnO39WiC9MvDj2hpb RSuN+XUdnbVKKAaD48SfHHjOk9s897/g26w0MODjy1CauIY8WQj2PCAgMtDBum6IR6EW 4F627BXCOs2bVTaA2MQ0m98BOzscCffavONb9K7dlMUH+ztWMl6I2vLcgJMTRKr7M8VM n8kDiX1WFTVdjtT8QngU0wuatgDwOHfrvvNH31fhLJ3Mm8XgD4tQiVKThqyGp49d3nKB fI8Ulhkx18zIt7fFOwb+XhjesLJNQ/aT2Y79WKZ3gStLcAfGYpNVV+XbFK15IVAtlG++ y3rQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; t=1679490388; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=Nq0ULfiQemL8gPeO7f/KCl9WXEmlKDB3Q+oiImcogL4=; b=T6p00azpgumzIxxoTKydIlIRHTmbA+Q6Lrm9Qr0PHulbseuAho6C2jQZ3cuiTuqmap CxPcgdxuaWdZMkokBATOnmQ48qwxal2Tr52Hv6USIa9IcHHGWFB7dMwZBqFZ7i0fKMte 8C8eK4bGmHIMxw2mDnKWxJ3O1YJ3l7RMAbVpVjvu7O4OxssXvRCZltFqSpJfz29K1n4h 4LnfjUWIhTqFuxlJoN/AQIZvHilLTR+DtjfNnkiBBZLEeOCSYMTICgrTDuFyhH6pFoYZ pGTYS2UGJpgP95TT7sFptIK9Jq9F5i1sMrakHSkQ9MS8kIaPaoU+p84sqAnXG8lRGkeu Cn+A== X-Gm-Message-State: AO0yUKUEFUS5CVC+NGRs/RxG2Oj9o/vcy4JMKyr1ozQiIkp45GGYJ69g QFdiGux1FiwTVQCc1cQs9CaHGDbcxo4HMn48gMYgiA== X-Google-Smtp-Source: AK7set+BSLhWOuEu9IaxRzoEL/UgwHGtcb9kOJDNxFT8FKeYj11iLoEyncKLpSRXvoIiNHasSUsqyA== X-Received: by 2002:a6b:db07:0:b0:74c:8dc7:aa1 with SMTP id t7-20020a6bdb07000000b0074c8dc70aa1mr4322570ioc.17.1679490388233; Wed, 22 Mar 2023 06:06:28 -0700 (PDT) Received: from localhost.localdomain ([93.93.133.154]) by smtp.gmail.com with ESMTPSA id l6-20020a026646000000b003c488204c6fsm4927960jaf.76.2023.03.22.06.06.27 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 22 Mar 2023 06:06:28 -0700 (PDT) To: libcamera-devel@lists.libcamera.org Date: Wed, 22 Mar 2023 13:06:12 +0000 Message-Id: <20230322130612.5208-11-naush@raspberrypi.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230322130612.5208-1-naush@raspberrypi.com> References: <20230322130612.5208-1-naush@raspberrypi.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v1 10/10] ipa: raspberrypi: Generalise the autofocus algorithm X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Naushir Patuck via libcamera-devel From: Naushir Patuck Reply-To: Naushir Patuck Cc: Nick Hollinghurst Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" From: Nick Hollinghurst Remove any hard-coded assumptions about the target hardware platform from the autofocus algorithm. Instead, use the "target" string provided by the camera tuning config and generalised statistics structures to determing parameters such as grid and region sizes. Additionally, PDAF statistics are represented by a generalised region statistics structure to be device agnostic. These changes also require the autofocus algorithm to initialise region weights on the first frame's prepare()/process() call rather than during initialisation. Signed-off-by: Nick Hollinghurst Signed-off-by: Naushir Patuck Tested-by: Naushir Patuck --- src/ipa/raspberrypi/cam_helper_imx708.cpp | 23 ++- src/ipa/raspberrypi/controller/pdaf_data.h | 21 +-- src/ipa/raspberrypi/controller/rpi/af.cpp | 161 ++++++++++----------- src/ipa/raspberrypi/controller/rpi/af.h | 28 ++-- 4 files changed, 119 insertions(+), 114 deletions(-) diff --git a/src/ipa/raspberrypi/cam_helper_imx708.cpp b/src/ipa/raspberrypi/cam_helper_imx708.cpp index 1f213d3c0833..641ba18f4b9d 100644 --- a/src/ipa/raspberrypi/cam_helper_imx708.cpp +++ b/src/ipa/raspberrypi/cam_helper_imx708.cpp @@ -69,11 +69,14 @@ private: /* Largest long exposure scale factor given as a left shift on the frame length. */ static constexpr int longExposureShiftMax = 7; + static constexpr int pdafStatsRows = 12; + static constexpr int pdafStatsCols = 16; + void populateMetadata(const MdParser::RegisterMap ®isters, Metadata &metadata) const override; static bool parsePdafData(const uint8_t *ptr, size_t len, unsigned bpp, - PdafData &pdaf); + PdafRegions &pdaf); bool parseAEHist(const uint8_t *ptr, size_t len, unsigned bpp); void putAGCStatistics(StatisticsPtr stats); @@ -120,11 +123,11 @@ void CamHelperImx708::prepare(libcamera::Span buffer, Metadata &m size_t bytesPerLine = (mode_.width * mode_.bitdepth) >> 3; if (buffer.size() > 2 * bytesPerLine) { - PdafData pdaf; + PdafRegions pdaf; if (parsePdafData(&buffer[2 * bytesPerLine], buffer.size() - 2 * bytesPerLine, mode_.bitdepth, pdaf)) - metadata.set("pdaf.data", pdaf); + metadata.set("pdaf.regions", pdaf); } /* Parse AE-HIST data where present */ @@ -239,7 +242,7 @@ void CamHelperImx708::populateMetadata(const MdParser::RegisterMap ®isters, } bool CamHelperImx708::parsePdafData(const uint8_t *ptr, size_t len, - unsigned bpp, PdafData &pdaf) + unsigned bpp, PdafRegions &pdaf) { size_t step = bpp >> 1; /* bytes per PDAF grid entry */ @@ -248,13 +251,17 @@ bool CamHelperImx708::parsePdafData(const uint8_t *ptr, size_t len, return false; } + pdaf.init({ pdafStatsCols, pdafStatsRows }); + ptr += 2 * step; - for (unsigned i = 0; i < PDAF_DATA_ROWS; ++i) { - for (unsigned j = 0; j < PDAF_DATA_COLS; ++j) { + for (unsigned i = 0; i < pdafStatsRows; ++i) { + for (unsigned j = 0; j < pdafStatsCols; ++j) { unsigned c = (ptr[0] << 3) | (ptr[1] >> 5); int p = (((ptr[1] & 0x0F) - (ptr[1] & 0x10)) << 6) | (ptr[2] >> 2); - pdaf.conf[i][j] = c; - pdaf.phase[i][j] = c ? p : 0; + PdafData pdafData; + pdafData.conf = c; + pdafData.phase = c ? p : 0; + pdaf.set(libcamera::Point(j, i), { pdafData, 1, 0 }); ptr += step; } } diff --git a/src/ipa/raspberrypi/controller/pdaf_data.h b/src/ipa/raspberrypi/controller/pdaf_data.h index 03c00d72c0e8..ae6ab996ded0 100644 --- a/src/ipa/raspberrypi/controller/pdaf_data.h +++ b/src/ipa/raspberrypi/controller/pdaf_data.h @@ -2,20 +2,23 @@ /* * Copyright (C) 2022, Raspberry Pi Ltd * - * pdaf_data.h - PDAF Metadata; for now this is - * largely based on IMX708's PDAF "Type 1" output. + * pdaf_data.h - PDAF Metadata */ #pragma once #include -#define PDAF_DATA_ROWS 12 -#define PDAF_DATA_COLS 16 +#include "region_stats.h" -struct PdafData { - /* Confidence values, in raster order, in arbitrary units */ - uint16_t conf[PDAF_DATA_ROWS][PDAF_DATA_COLS]; +namespace RPiController { - /* Phase error, in raster order, in s11 Q4 format (S.6.4) */ - int16_t phase[PDAF_DATA_ROWS][PDAF_DATA_COLS]; +struct PdafData { + /* Confidence, in arbitrary units */ + uint16_t conf; + /* Phase error, in s16 Q4 format (S.11.4) */ + int16_t phase; }; + +using PdafRegions = RegionStats; + +}; // namespace RPiController diff --git a/src/ipa/raspberrypi/controller/rpi/af.cpp b/src/ipa/raspberrypi/controller/rpi/af.cpp index a623651875f2..3d2dad974d8b 100644 --- a/src/ipa/raspberrypi/controller/rpi/af.cpp +++ b/src/ipa/raspberrypi/controller/rpi/af.cpp @@ -174,9 +174,8 @@ Af::Af(Controller *controller) statsRegion_(0, 0, 0, 0), windows_(), useWindows_(false), - phaseWeights_{}, - contrastWeights_{}, - sumWeights_(0), + phaseWeights_(), + contrastWeights_(), scanState_(ScanState::Idle), initted_(false), ftarget_(-1.0), @@ -190,6 +189,8 @@ Af::Af(Controller *controller) scanData_(), reportState_(AfState::Idle) { + phaseWeights_.w.reserve(16 * 12); + contrastWeights_.w.reserve(4 * 3); scanData_.reserve(24); } @@ -226,7 +227,7 @@ void Af::switchMode(CameraMode const &cameraMode, [[maybe_unused]] Metadata *met << statsRegion_.y << ',' << statsRegion_.width << ',' << statsRegion_.height; - computeWeights(); + invalidateWeights(); if (scanState_ >= ScanState::Coarse && scanState_ < ScanState::Settle) { /* @@ -239,111 +240,94 @@ void Af::switchMode(CameraMode const &cameraMode, [[maybe_unused]] Metadata *met skipCount_ = cfg_.skipFrames; } -void Af::computeWeights() +void Af::computeWeights(RegionWeights *wgts, unsigned rows, unsigned cols) { - constexpr int MaxCellWeight = 240 / (int)MaxWindows; + wgts->rows = rows; + wgts->cols = cols; + wgts->sum = 0; + wgts->w.resize(rows * cols); + std::fill(wgts->w.begin(), wgts->w.end(), 0); - sumWeights_ = 0; - for (int i = 0; i < PDAF_DATA_ROWS; ++i) - std::fill(phaseWeights_[i], phaseWeights_[i] + PDAF_DATA_COLS, 0); - - if (useWindows_ && - statsRegion_.width >= PDAF_DATA_COLS && statsRegion_.height >= PDAF_DATA_ROWS) { + if (rows > 0 && cols > 0 && useWindows_ && + statsRegion_.height >= rows && statsRegion_.width >= cols) { /* * Here we just merge all of the given windows, weighted by area. * \todo Perhaps a better approach might be to find the phase in each * window and choose either the closest or the highest-confidence one? - * - * Using mostly "int" arithmetic, because Rectangle has signed x, y */ - int cellH = (int)(statsRegion_.height / PDAF_DATA_ROWS); - int cellW = (int)(statsRegion_.width / PDAF_DATA_COLS); - int cellA = cellH * cellW; + const unsigned maxCellWeight = 46080u / (MaxWindows * rows * cols); + const unsigned cellH = statsRegion_.height / rows; + const unsigned cellW = statsRegion_.width / cols; + const unsigned cellA = cellH * cellW; for (auto &w : windows_) { - for (int i = 0; i < PDAF_DATA_ROWS; ++i) { - int y0 = std::max(statsRegion_.y + cellH * i, w.y); - int y1 = std::min(statsRegion_.y + cellH * (i + 1), w.y + (int)(w.height)); + for (unsigned r = 0; r < rows; ++r) { + int y0 = std::max(statsRegion_.y + (int)(cellH * r), w.y); + int y1 = std::min(statsRegion_.y + (int)(cellH * (r + 1)), w.y + (int)(w.height)); if (y0 >= y1) continue; y1 -= y0; - for (int j = 0; j < PDAF_DATA_COLS; ++j) { - int x0 = std::max(statsRegion_.x + cellW * j, w.x); - int x1 = std::min(statsRegion_.x + cellW * (j + 1), w.x + (int)(w.width)); + for (unsigned c = 0; c < cols; ++c) { + int x0 = std::max(statsRegion_.x + (int)(cellW * c), w.x); + int x1 = std::min(statsRegion_.x + (int)(cellW * (c + 1)), w.x + (int)(w.width)); if (x0 >= x1) continue; - int a = y1 * (x1 - x0); - a = (MaxCellWeight * a + cellA - 1) / cellA; - phaseWeights_[i][j] += a; - sumWeights_ += a; + unsigned a = y1 * (x1 - x0); + a = (maxCellWeight * a + cellA - 1) / cellA; + wgts->w[r * cols + c] += a; + wgts->sum += a; } } } } - if (sumWeights_ == 0) { - /* - * Default AF window is the middle 1/2 width of the middle 1/3 height - * since this maps nicely to both PDAF (16x12) and Focus (4x3) grids. - */ - for (int i = PDAF_DATA_ROWS / 3; i < 2 * PDAF_DATA_ROWS / 3; ++i) { - for (int j = PDAF_DATA_COLS / 4; j < 3 * PDAF_DATA_COLS / 4; ++j) { - phaseWeights_[i][j] = MaxCellWeight; - sumWeights_ += MaxCellWeight; + if (wgts->sum == 0) { + /* Default AF window is the middle 1/2 width of the middle 1/3 height */ + for (unsigned r = rows / 3; r < rows - rows / 3; ++r) { + for (unsigned c = cols / 4; c < cols - cols / 4; ++c) { + wgts->w[r * cols + c] = 1; + wgts->sum += 1; } } } +} - /* Scale from PDAF to Focus Statistics grid (which has fixed size 4x3) */ - constexpr int FocusStatsRows = 3; - constexpr int FocusStatsCols = 4; - static_assert(FOCUS_REGIONS == FocusStatsRows * FocusStatsCols); - static_assert(PDAF_DATA_ROWS % FocusStatsRows == 0); - static_assert(PDAF_DATA_COLS % FocusStatsCols == 0); - constexpr int YFactor = PDAF_DATA_ROWS / FocusStatsRows; - constexpr int XFactor = PDAF_DATA_COLS / FocusStatsCols; - - LOG(RPiAf, Debug) << "Recomputed weights:"; - for (int i = 0; i < FocusStatsRows; ++i) { - for (int j = 0; j < FocusStatsCols; ++j) { - unsigned w = 0; - for (int y = 0; y < YFactor; ++y) - for (int x = 0; x < XFactor; ++x) - w += phaseWeights_[YFactor * i + y][XFactor * j + x]; - contrastWeights_[FocusStatsCols * i + j] = w; - } - LOG(RPiAf, Debug) << " " - << contrastWeights_[FocusStatsCols * i + 0] << " " - << contrastWeights_[FocusStatsCols * i + 1] << " " - << contrastWeights_[FocusStatsCols * i + 2] << " " - << contrastWeights_[FocusStatsCols * i + 3]; - } +void Af::invalidateWeights() +{ + phaseWeights_.sum = 0; + contrastWeights_.sum = 0; } -bool Af::getPhase(PdafData const &data, double &phase, double &conf) const +bool Af::getPhase(PdafRegions const ®ions, double &phase, double &conf) { + libcamera::Size size = regions.size(); + if (size.height != phaseWeights_.rows || size.width != phaseWeights_.cols || + phaseWeights_.sum == 0) { + LOG(RPiAf, Debug) << "Recompute Phase weights " << size.width << 'x' << size.height; + computeWeights(&phaseWeights_, size.height, size.width); + } + uint32_t sumWc = 0; int64_t sumWcp = 0; - - for (unsigned i = 0; i < PDAF_DATA_ROWS; ++i) { - for (unsigned j = 0; j < PDAF_DATA_COLS; ++j) { - if (phaseWeights_[i][j]) { - uint32_t c = data.conf[i][j]; - if (c >= cfg_.confThresh) { - if (c > cfg_.confClip) - c = cfg_.confClip; - c -= (cfg_.confThresh >> 2); - sumWc += phaseWeights_[i][j] * c; - c -= (cfg_.confThresh >> 2); - sumWcp += phaseWeights_[i][j] * data.phase[i][j] * (int64_t)c; - } + for (unsigned i = 0; i < regions.numRegions(); ++i) { + unsigned w = phaseWeights_.w[i]; + if (w) { + const PdafData &data = regions.get(i).val; + unsigned c = data.conf; + if (c >= cfg_.confThresh) { + if (c > cfg_.confClip) + c = cfg_.confClip; + c -= (cfg_.confThresh >> 2); + sumWc += w * c; + c -= (cfg_.confThresh >> 2); + sumWcp += (int64_t)(w * c) * (int64_t)data.phase; } } } - if (0 < sumWeights_ && sumWeights_ <= sumWc) { + if (0 < phaseWeights_.sum && phaseWeights_.sum <= sumWc) { phase = (double)sumWcp / (double)sumWc; - conf = (double)sumWc / (double)sumWeights_; + conf = (double)sumWc / (double)phaseWeights_.sum; return true; } else { phase = 0.0; @@ -352,14 +336,19 @@ bool Af::getPhase(PdafData const &data, double &phase, double &conf) const } } -double Af::getContrast(const FocusRegions &focusStats) const +double Af::getContrast(const FocusRegions &focusStats) { - uint32_t sumWc = 0; + libcamera::Size size = focusStats.size(); + if (size.height != contrastWeights_.rows || size.width != contrastWeights_.cols || contrastWeights_.sum == 0) { + LOG(RPiAf, Debug) << "Recompute Contrast weights " << size.width << 'x' << size.height; + computeWeights(&contrastWeights_, size.height, size.width); + } + uint64_t sumWc = 0; for (unsigned i = 0; i < focusStats.numRegions(); ++i) - sumWc += contrastWeights_[i] * focusStats.get(i).val; + sumWc += contrastWeights_.w[i] * focusStats.get(i).val; - return (sumWeights_ == 0) ? 0.0 : (double)sumWc / (double)sumWeights_; + return (contrastWeights_.sum > 0) ? ((double)sumWc / (double)contrastWeights_.sum) : 0.0; } void Af::doPDAF(double phase, double conf) @@ -623,14 +612,14 @@ void Af::prepare(Metadata *imageMetadata) if (initted_) { /* Get PDAF from the embedded metadata, and run AF algorithm core */ - PdafData data; + PdafRegions regions; double phase = 0.0, conf = 0.0; double oldFt = ftarget_; double oldFs = fsmooth_; ScanState oldSs = scanState_; uint32_t oldSt = stepCount_; - if (imageMetadata->get("pdaf.data", data) == 0) - getPhase(data, phase, conf); + if (imageMetadata->get("pdaf.regions", regions) == 0) + getPhase(regions, phase, conf); doAF(prevContrast_, phase, conf); updateLensPosition(); LOG(RPiAf, Debug) << std::fixed << std::setprecision(2) @@ -691,7 +680,7 @@ void Af::setMetering(bool mode) { if (useWindows_ != mode) { useWindows_ = mode; - computeWeights(); + invalidateWeights(); } } @@ -708,7 +697,9 @@ void Af::setWindows(libcamera::Span const &wins) if (windows_.size() >= MaxWindows) break; } - computeWeights(); + + if (useWindows_) + invalidateWeights(); } bool Af::setLensPosition(double dioptres, int *hwpos) diff --git a/src/ipa/raspberrypi/controller/rpi/af.h b/src/ipa/raspberrypi/controller/rpi/af.h index 7959371baf64..b479feb88c39 100644 --- a/src/ipa/raspberrypi/controller/rpi/af.h +++ b/src/ipa/raspberrypi/controller/rpi/af.h @@ -11,12 +11,6 @@ #include "../pdaf_data.h" #include "../pwl.h" -/* - * \todo FOCUS_REGIONS is taken from bcm2835-isp.h, but should be made as a - * generic RegionStats structure. - */ -#define FOCUS_REGIONS 12 - /* * This algorithm implements a hybrid of CDAF and PDAF, favouring PDAF. * @@ -121,9 +115,20 @@ private: double conf; }; - void computeWeights(); - bool getPhase(PdafData const &data, double &phase, double &conf) const; - double getContrast(const FocusRegions &focusStats) const; + struct RegionWeights { + unsigned rows; + unsigned cols; + uint32_t sum; + std::vector w; + + RegionWeights() + : rows(0), cols(0), sum(0), w() {} + }; + + void computeWeights(RegionWeights *wgts, unsigned rows, unsigned cols); + void invalidateWeights(); + bool getPhase(PdafRegions const ®ions, double &phase, double &conf); + double getContrast(const FocusRegions &focusStats); void doPDAF(double phase, double conf); bool earlyTerminationByPhase(double phase); double findPeak(unsigned index) const; @@ -143,9 +148,8 @@ private: libcamera::Rectangle statsRegion_; std::vector windows_; bool useWindows_; - uint8_t phaseWeights_[PDAF_DATA_ROWS][PDAF_DATA_COLS]; - uint16_t contrastWeights_[FOCUS_REGIONS]; - uint32_t sumWeights_; + RegionWeights phaseWeights_; + RegionWeights contrastWeights_; /* Working state. */ ScanState scanState_;