From patchwork Mon Mar 27 12:20:21 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Naushir Patuck X-Patchwork-Id: 18470 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 20378C326C for ; Mon, 27 Mar 2023 12:20:45 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 7390462726; Mon, 27 Mar 2023 14:20:43 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1679919643; bh=PtM2d3RdN/izHmZQwStx7IjYq7zepzd3IeF7YNSRGBs=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To:Cc: From; b=pOw7ggfpgavG8x/QhhujwZMFwS7Gb+F+N2lW8fvKYrJzbGOypktuN8Cd7yj+XeFt5 s+gFnUYuJYNgHmX+9SdtbeoD7Q96nTm1P7/1QSjho1JtfLp0pxKhmykm4NdChJg7Qt YZHUUi+Ylo+akjaul5hoQ2GgckLSu96rTMNi9yIMwTRyb6jOKqZTWrC8WZ9/EdmJQs gNSp5KHhPwMkmrex7cnHuGKFecKXiSGRwFwdAIVaJbhLGA7KlupD8CxGTHEhxJQtD1 oEU/N0t44U+LJm3osVlQ2qQ4u2rZVjCfRK233tm/tNioXrJtXBohGqVyYYbwNC1f/z urELZgOElV9Jg== Received: from mail-io1-xd33.google.com (mail-io1-xd33.google.com [IPv6:2607:f8b0:4864:20::d33]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 6F2346271C for ; Mon, 27 Mar 2023 14:20:42 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="mSrXaNy/"; dkim-atps=neutral Received: by mail-io1-xd33.google.com with SMTP id e13so3813580ioc.0 for ; Mon, 27 Mar 2023 05:20:42 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; t=1679919641; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=PJ25gprEnpGANBs1RoR9sTHxbPydBDJjgb2I5S0jxek=; b=mSrXaNy/f3ourPOokwBLfyFyk4a2DGIv4lCfsUg2Crj73/4uoyrFEfiNiEUQIkdWkF MEWvAK6X/jC+JkpQ3K78ZTQfEgRJqOBZ5UZDfzH+AczIdY6wYr7hf0JirsGaSgO1fgRD AjQMNn2ZhlLmS3Rq5t/byEW5XnbFcFaRR9pHnx7sqdtj0yWrcee1D7zxiHJ10+1DXJ7R I2o+6pJVTG0CPvXbb87LkY/cgNapdrdXRLPTIkOstSO7IjVHm0RvTk/Drt1stQmR3KR/ gL5sVAgXabUiBTBnClNBo6lumHv3rF9/kw2ljU77n6xwkYWTJLzBLZEjMTEg3wrfgVA4 xH6w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; t=1679919641; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=PJ25gprEnpGANBs1RoR9sTHxbPydBDJjgb2I5S0jxek=; b=FNA1JJLq8YAvio7EQs/iTk4U5jV+L2E+U3cvBH541KpDG7P6te9hROGVKJcvURoTOp 02enZs+26iKietzba9v/Vn9xUQMEaKxDTD+y7HPNhXNo135W1nEuG1ImWnN3hWJ+7pRW D6bsK0gI1Gm7Qs8W/ChSnf3exxjQ1gfVv9NzD7E+UIhShdUy6fN85TDkTJ3Ut8NxJIDe nVy5HI/AG/GeP3lH65XevgzKkt3oYqkFbHQZnWJifU1UCh7/qe2i8RjUm+CD4tXRA7eL 3AQx4gOr5SRfJDWdbxEGJy6Y9ayUJFjUa7NM8ub1FCYD8X0g5GkxKTSRhufdeNJNr7ji vHeQ== X-Gm-Message-State: AO0yUKX4kNMxGW47M94tHaXNwdI1Vqw2f2oatMl2vGk/BBdPXQvwwbHM Aq6+2Mn0yqMK9YsN/jMZLhRHl/+df4qtLzaiha6dnA== X-Google-Smtp-Source: AK7set+4Uikiu4a/rBgXilsoKWB8zVTMbhiPOGPeLBAWCO4XIsyt+y2sl/MKN1Dg8SeATb4YRJlA3A== X-Received: by 2002:a05:6602:408b:b0:758:9e69:13f1 with SMTP id bl11-20020a056602408b00b007589e6913f1mr12124881iob.9.1679919640966; Mon, 27 Mar 2023 05:20:40 -0700 (PDT) Received: from localhost.localdomain ([93.93.133.154]) by smtp.gmail.com with ESMTPSA id r7-20020a5edb47000000b00758ff97aaedsm3239026iop.16.2023.03.27.05.20.40 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 27 Mar 2023 05:20:40 -0700 (PDT) To: libcamera-devel@lists.libcamera.org Date: Mon, 27 Mar 2023 13:20:21 +0100 Message-Id: <20230327122030.11756-2-naush@raspberrypi.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230327122030.11756-1-naush@raspberrypi.com> References: <20230327122030.11756-1-naush@raspberrypi.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v2 01/10] ipa: raspberrypi Store the target string in the controller X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Naushir Patuck via libcamera-devel From: Naushir Patuck Reply-To: Naushir Patuck Cc: Jacopo Mondi Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" The target string may be used by algorithms to determine the running hardware target. Store the target string provided by the camera tuning files in the controller state. Add a getTarget() member function to retrieve this string. Validate the correct hardware target ("bcm2835") during the IPA initialisation phase. Signed-off-by: Naushir Patuck Reviewed-by: Jacopo Mondi Reviewed-by: Kieran Bingham --- src/ipa/raspberrypi/controller/algorithm.h | 4 ++++ src/ipa/raspberrypi/controller/controller.cpp | 6 ++++++ src/ipa/raspberrypi/controller/controller.h | 4 ++++ src/ipa/raspberrypi/raspberrypi.cpp | 8 ++++++++ 4 files changed, 22 insertions(+) diff --git a/src/ipa/raspberrypi/controller/algorithm.h b/src/ipa/raspberrypi/controller/algorithm.h index 4f3275987305..7c22fbe4945c 100644 --- a/src/ipa/raspberrypi/controller/algorithm.h +++ b/src/ipa/raspberrypi/controller/algorithm.h @@ -41,6 +41,10 @@ public: { return controller_->getGlobalMetadata(); } + const std::string &getTarget() const + { + return controller_->getTarget(); + } private: Controller *controller_; diff --git a/src/ipa/raspberrypi/controller/controller.cpp b/src/ipa/raspberrypi/controller/controller.cpp index e91567852d00..a6250ee140b0 100644 --- a/src/ipa/raspberrypi/controller/controller.cpp +++ b/src/ipa/raspberrypi/controller/controller.cpp @@ -38,6 +38,7 @@ int Controller::read(char const *filename) std::unique_ptr root = YamlParser::parse(file); double version = (*root)["version"].get(1.0); + target_ = (*root)["target"].get("bcm2835"); if (version < 2.0) { LOG(RPiController, Warning) @@ -142,3 +143,8 @@ Algorithm *Controller::getAlgorithm(std::string const &name) const } return nullptr; } + +const std::string &Controller::getTarget() const +{ + return target_; +} diff --git a/src/ipa/raspberrypi/controller/controller.h b/src/ipa/raspberrypi/controller/controller.h index e6c950c3a509..24e02903d438 100644 --- a/src/ipa/raspberrypi/controller/controller.h +++ b/src/ipa/raspberrypi/controller/controller.h @@ -46,6 +46,7 @@ public: void process(StatisticsPtr stats, Metadata *imageMetadata); Metadata &getGlobalMetadata(); Algorithm *getAlgorithm(std::string const &name) const; + const std::string &getTarget() const; protected: int createAlgorithm(const std::string &name, const libcamera::YamlObject ¶ms); @@ -53,6 +54,9 @@ protected: Metadata globalMetadata_; std::vector algorithms_; bool switchModeCalled_; + +private: + std::string target_; }; } /* namespace RPiController */ diff --git a/src/ipa/raspberrypi/raspberrypi.cpp b/src/ipa/raspberrypi/raspberrypi.cpp index 1375795568e2..86359538cf67 100644 --- a/src/ipa/raspberrypi/raspberrypi.cpp +++ b/src/ipa/raspberrypi/raspberrypi.cpp @@ -269,6 +269,14 @@ int IPARPi::init(const IPASettings &settings, bool lensPresent, IPAInitResult *r return ret; } + const std::string &target = controller_.getTarget(); + if (target != "bcm2835") { + LOG(IPARPI, Error) + << "Tuning data file target returned \"" << target << "\"" + << ", expected \"bcm2835\""; + return -EINVAL; + } + lensPresent_ = lensPresent; controller_.initialise(); From patchwork Mon Mar 27 12:20:22 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Naushir Patuck X-Patchwork-Id: 18471 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 598D2BD160 for ; Mon, 27 Mar 2023 12:20:46 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 156B662744; Mon, 27 Mar 2023 14:20:46 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1679919646; bh=p8bjwYcPHre60o2C+WXibtxyErkFZE3S7agTpN+m80M=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To: From; b=1J16qOH79mlFamkgG4cpg/m8nNvWo6/HTZfHK/mOAbYTBrYb0NkajKFPdPGwajZaG l98GvRG+AsiEpixn51SdO4FtoWTktT2Q00IFBRVHo2bRU+8aNNyr92z6e7jb7f8Cjs yvCOXUD018anwfsSppMcdBpR04xaX9prmtiRegQtiH7icrZOKYIhKtnHJqlMJV4nXh ZoTqhILwavN6NhuCow4kSiOUdL+6df80+5WS59kgFrbQcUSFxzFQTKH/pUyH0+uDaU CBwiVheTKa1Mws51SXd/aHthR/DjlDrGjhysqDLaP6wtOnCZIR8JWiKrzKXc2LNFKZ I8fCTx+Z1OROg== Received: from mail-io1-xd2b.google.com (mail-io1-xd2b.google.com [IPv6:2607:f8b0:4864:20::d2b]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id C000862740 for ; Mon, 27 Mar 2023 14:20:43 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="KxeYuUtz"; dkim-atps=neutral Received: by mail-io1-xd2b.google.com with SMTP id k17so3803090iob.1 for ; Mon, 27 Mar 2023 05:20:43 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; t=1679919642; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=budr3U+RGAt0DckKXPoFcDC9ZFRMq47CvRp3MWMf5AA=; b=KxeYuUtzVgyF6J1Lo9lIyxmxSfYKUCGQh05EG7Qf16CjEkKTt5iVNlxcFl52S/w/Qy 8QL51NdGQD/Ra6U6PSU/3tkACYH3RO86rmgpWYtvaT/OkNJPC80UN2NpKpNBwN0OlQp6 mbOVqGTx5I4Q9pO7/p9kOPy+RKHlQg8A3G7PMw/gafLDoNznoYfB3SR9lj+P2bNvjxMl D7Y5cXNHNe4b4EXYjWJqtK9PiqyhwGm7GTRxdmgzkjSkj77M7j7lWznsmCEQx25tQQM/ rx7JZlU1PLMg7Dl7eI4472MLfn2adk5zN6HXuBt6Jc4ar6DJb2fcgLh9SeH833Cz4n0G 3Zbg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; t=1679919642; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=budr3U+RGAt0DckKXPoFcDC9ZFRMq47CvRp3MWMf5AA=; b=r5iWsgPERR2fSGKg3I9C+M9vV14g+5Qxvwcftd5oW3EM59d11XKev2G3vuXhL8KTjw 5ZO0Twey/XSvUz+IDvoAlSuvqfsrXY9P+X+7q5d9bWVj4c7XpG+OIa8FElqVgMq1UTaC ZusWXDHrRM74g/7EVzUH7Cm/siKQVD0REWNQafGa5P6aYWT+8AN0TdqiwntEEqT+qNku LvSnjOzvRtqtHRdThUNp4QxaeXpOeRi0CPosDqwInAIVIymMOQKse7cEAgscpdfThjiB 3pZWAYjYk9zRnu1yaVxX4aC8MX5j05SEwDYLQ0FZZeFO9QeRdIJAQHwjKSySbm2dqwhO 3wog== X-Gm-Message-State: AO0yUKVrM96pLcb3dqPkFAmlAyRxKI9cWnMNtp9CvLWce90vcIhbpViX hS5knarksy1aCxKHbMbgRE62/ypZSSJtth3hP3Fpsw== X-Google-Smtp-Source: AK7set8O3NSZzJMD7ctnUOuif0Ol+ZZxLCGhZxijB5a9GZkwVFLc7qSJTamFgHw+fdHAm8rYhHARtQ== X-Received: by 2002:a6b:e50f:0:b0:758:10a7:6813 with SMTP id y15-20020a6be50f000000b0075810a76813mr8803664ioc.16.1679919642083; Mon, 27 Mar 2023 05:20:42 -0700 (PDT) Received: from localhost.localdomain ([93.93.133.154]) by smtp.gmail.com with ESMTPSA id r7-20020a5edb47000000b00758ff97aaedsm3239026iop.16.2023.03.27.05.20.41 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 27 Mar 2023 05:20:41 -0700 (PDT) To: libcamera-devel@lists.libcamera.org Date: Mon, 27 Mar 2023 13:20:22 +0100 Message-Id: <20230327122030.11756-3-naush@raspberrypi.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230327122030.11756-1-naush@raspberrypi.com> References: <20230327122030.11756-1-naush@raspberrypi.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v2 02/10] ipa: raspberrypi: Add hardware configuration to the controller X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Naushir Patuck via libcamera-devel From: Naushir Patuck Reply-To: Naushir Patuck Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Add a new Controller::HardwareConfig structure that captures the hardware statistics grid/histogram sizes and pipeline widths. This ensures there is a single centralised places for these parameters. Add a getHardwareConfig() helper function to retrieve these values for a given hardware target. Update the statistics populating routine in the IPA to use the values from this structure instead of the hardcoded numbers. Signed-off-by: Naushir Patuck Reviewed-by: David Plowman Reviewed-by: Kieran Bingham --- src/ipa/raspberrypi/controller/algorithm.h | 4 +++ src/ipa/raspberrypi/controller/controller.cpp | 31 +++++++++++++++++++ src/ipa/raspberrypi/controller/controller.h | 11 +++++++ src/ipa/raspberrypi/raspberrypi.cpp | 21 +++++-------- 4 files changed, 54 insertions(+), 13 deletions(-) diff --git a/src/ipa/raspberrypi/controller/algorithm.h b/src/ipa/raspberrypi/controller/algorithm.h index 7c22fbe4945c..4aa814ebbebd 100644 --- a/src/ipa/raspberrypi/controller/algorithm.h +++ b/src/ipa/raspberrypi/controller/algorithm.h @@ -45,6 +45,10 @@ public: { return controller_->getTarget(); } + const Controller::HardwareConfig &getHardwareConfig() const + { + return controller_->getHardwareConfig(); + } private: Controller *controller_; diff --git a/src/ipa/raspberrypi/controller/controller.cpp b/src/ipa/raspberrypi/controller/controller.cpp index a6250ee140b0..fa1721130783 100644 --- a/src/ipa/raspberrypi/controller/controller.cpp +++ b/src/ipa/raspberrypi/controller/controller.cpp @@ -20,6 +20,25 @@ using namespace libcamera; LOG_DEFINE_CATEGORY(RPiController) +static const std::map HardwareConfigMap = { + { + "bcm2835", + { + /* + * There are only ever 15 AGC regions computed by the firmware + * due to zoning, but the HW defines AGC_REGIONS == 16! + */ + .agcRegions = { 15 , 1 }, + .agcZoneWeights = { 15 , 1 }, + .awbRegions = { 16, 12 }, + .focusRegions = { 4, 3 }, + .numHistogramBins = 128, + .numGammaPoints = 33, + .pipelineWidth = 13 + } + }, +}; + Controller::Controller() : switchModeCalled_(false) { @@ -148,3 +167,15 @@ const std::string &Controller::getTarget() const { return target_; } + +const Controller::HardwareConfig &Controller::getHardwareConfig() const +{ + auto cfg = HardwareConfigMap.find(getTarget()); + + /* + * This really should not happen, the IPA ought to validate the target + * on initialisation. + */ + ASSERT(cfg != HardwareConfigMap.end()); + return cfg->second; +} diff --git a/src/ipa/raspberrypi/controller/controller.h b/src/ipa/raspberrypi/controller/controller.h index 24e02903d438..c6af5cd6c7d4 100644 --- a/src/ipa/raspberrypi/controller/controller.h +++ b/src/ipa/raspberrypi/controller/controller.h @@ -37,6 +37,16 @@ typedef std::unique_ptr AlgorithmPtr; class Controller { public: + struct HardwareConfig { + libcamera::Size agcRegions; + libcamera::Size agcZoneWeights; + libcamera::Size awbRegions; + libcamera::Size focusRegions; + unsigned int numHistogramBins; + unsigned int numGammaPoints; + unsigned int pipelineWidth; + }; + Controller(); ~Controller(); int read(char const *filename); @@ -47,6 +57,7 @@ public: Metadata &getGlobalMetadata(); Algorithm *getAlgorithm(std::string const &name) const; const std::string &getTarget() const; + const HardwareConfig &getHardwareConfig() const; protected: int createAlgorithm(const std::string &name, const libcamera::YamlObject ¶ms); diff --git a/src/ipa/raspberrypi/raspberrypi.cpp b/src/ipa/raspberrypi/raspberrypi.cpp index 86359538cf67..b64cb96e2dde 100644 --- a/src/ipa/raspberrypi/raspberrypi.cpp +++ b/src/ipa/raspberrypi/raspberrypi.cpp @@ -1392,20 +1392,19 @@ RPiController::StatisticsPtr IPARPi::fillStatistics(bcm2835_isp_stats *stats) co { using namespace RPiController; + const Controller::HardwareConfig &hw = controller_.getHardwareConfig(); unsigned int i; StatisticsPtr statistics = std::make_unique(Statistics::AgcStatsPos::PreWb, Statistics::ColourStatsPos::PostLsc); /* RGB histograms are not used, so do not populate them. */ - statistics->yHist = RPiController::Histogram(stats->hist[0].g_hist, NUM_HISTOGRAM_BINS); + statistics->yHist = RPiController::Histogram(stats->hist[0].g_hist, + hw.numHistogramBins); - /* - * All region sums are based on a 13-bit pipeline bit-depth. Normalise - * this to 16-bits for the AGC/AWB/ALSC algorithms. - */ - constexpr unsigned int scale = Statistics::NormalisationFactorPow2 - 13; + /* All region sums are based on a 16-bit normalised pipeline bit-depth. */ + unsigned int scale = Statistics::NormalisationFactorPow2 - hw.pipelineWidth; - statistics->awbRegions.init({ DEFAULT_AWB_REGIONS_X, DEFAULT_AWB_REGIONS_Y }); + statistics->awbRegions.init(hw.awbRegions); for (i = 0; i < statistics->awbRegions.numRegions(); i++) statistics->awbRegions.set(i, { { stats->awb_stats[i].r_sum << scale, stats->awb_stats[i].g_sum << scale, @@ -1413,11 +1412,7 @@ RPiController::StatisticsPtr IPARPi::fillStatistics(bcm2835_isp_stats *stats) co stats->awb_stats[i].counted, stats->awb_stats[i].notcounted }); - /* - * There are only ever 15 regions computed by the firmware due to zoning, - * but the HW defines AGC_REGIONS == 16! - */ - statistics->agcRegions.init(15); + statistics->agcRegions.init(hw.agcRegions); for (i = 0; i < statistics->agcRegions.numRegions(); i++) statistics->agcRegions.set(i, { { stats->agc_stats[i].r_sum << scale, stats->agc_stats[i].g_sum << scale, @@ -1425,7 +1420,7 @@ RPiController::StatisticsPtr IPARPi::fillStatistics(bcm2835_isp_stats *stats) co stats->agc_stats[i].counted, stats->awb_stats[i].notcounted }); - statistics->focusRegions.init({ 4, 3 }); + statistics->focusRegions.init(hw.focusRegions); for (i = 0; i < statistics->focusRegions.numRegions(); i++) statistics->focusRegions.set(i, { stats->focus_stats[i].contrast_val[1][1] / 1000, stats->focus_stats[i].contrast_val_num[1][1], From patchwork Mon Mar 27 12:20:23 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Naushir Patuck X-Patchwork-Id: 18472 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id E8AA3C329C for ; Mon, 27 Mar 2023 12:20:46 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 9E3676274E; Mon, 27 Mar 2023 14:20:46 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1679919646; bh=e6Gybp5jflFnxPv4XeyHtmXfuE7aSjvYz6G1G+G/8ws=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To: From; b=GHe7ogxvXXbPWypJZ0SvZBKeiqtSGjrObEWTXb+C5a3hDljTdgsboHkk7DS/29tkM mPRRtWnk220HiQefBpvaX33clhN0l+udaubi+CobD3c1r3MSkMVQyDeX5UHPGvIMUc pWy25mlnrK45dSqwVc66Wze3Xw3g4Hxp7gwAgA4TP0rdqLWyXcKwPi61Z9DFhk6KY/ Sb6jxYyZrKf+nHwRP8UbVUIVwrUdrZhM007LPBF9AUTOFL8IDMdWvD8MZgOb7bVNJy y4dk79LNh00dKoHWJAi78A75GoR2n/MAqTMGainme7BgqB3+ddONpdLzOIL6akUzq9 zrTs3DsWCLM/A== Received: from mail-io1-xd2c.google.com (mail-io1-xd2c.google.com [IPv6:2607:f8b0:4864:20::d2c]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 51F6862734 for ; Mon, 27 Mar 2023 14:20:45 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="F+ES8kb3"; dkim-atps=neutral Received: by mail-io1-xd2c.google.com with SMTP id y85so3776606iof.13 for ; Mon, 27 Mar 2023 05:20:45 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; t=1679919644; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=lxk+09OLb8NVievLsl0v8vF6rW59QElmPt8Tvx8omoY=; b=F+ES8kb3XfDaJR9zfbQy9HOR/iwIQiAvhV/7WkSClQ6KO7eMhktZtr8N0ETzIO3vTX pmviL4ExLr+UWNv+R5dqJ5K9NaLYz7i+YSuf+pgWlbwEAR5PRE/h6n+fANdcVmFDJUjE K1ph1uEeVj3VMP2RlbkBk1UwiPYrrkvyMPo+Mhgr6sxOVHdQMyftZhZCpzgRLIxTFaV6 H5r3xy0hubi+Fk5VmrrvaUEoJXdZkkggObhscFwGzbFV415uUJy5CZnAa3uOUqvVmP9w 4Ploob4/C4kqIMuTvRnah2ZYzUKzvv0Z8SsJqZ9B6heofbED1shUTKnXTK2SyMcNVJxq MzBQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; t=1679919644; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=lxk+09OLb8NVievLsl0v8vF6rW59QElmPt8Tvx8omoY=; b=gueJQaOe00Fijk1HhDdNJVm/JDadEtHm87zvuxzaC+dJVpmA390jEY2oMKD4Vm/jNU usDB72hxJ5t0IcE1VuFyarlCnqwn8/usnoe010aTgUVQatiX/SxfO7GfFg14Pv56CJ2a F+3HTeYaUV40auSj83ynrrPBJKlY8jvO7V4i1PzPUbk7iIyOnZQFdbI5Z4LufXGa7zcf s/nsGx+IiaLBOLqNMGRqx5dY3nkRXgf/if7bwtBaNBYrpoVRh4M9A3QIEKsydGPq9H3j vusL8MDpFLG9bzR2FPQ2UNvBZxmsE3ASu0ChTHfa+11CCQSJBlmrv1D7DGbtUiUP+jNr B16w== X-Gm-Message-State: AO0yUKXHjJf6Vxq1fuqk9CxnZvfpEffMPj66Eg3q2doBa4BafdsfN/jR e7/5344nIwwlhXw1cwWsqscijSv+ORrMOR/PJkS/qA== X-Google-Smtp-Source: AK7set9t0r84Q4CwmcNbgKXLmOxAevYqIJa69pCqXYIuT8Czkwfk7Fv8Ek+X6X+z2BCQoYGrorXBlg== X-Received: by 2002:a6b:5b14:0:b0:74c:91c7:8794 with SMTP id v20-20020a6b5b14000000b0074c91c78794mr8786814ioh.21.1679919643331; Mon, 27 Mar 2023 05:20:43 -0700 (PDT) Received: from localhost.localdomain ([93.93.133.154]) by smtp.gmail.com with ESMTPSA id r7-20020a5edb47000000b00758ff97aaedsm3239026iop.16.2023.03.27.05.20.42 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 27 Mar 2023 05:20:43 -0700 (PDT) To: libcamera-devel@lists.libcamera.org Date: Mon, 27 Mar 2023 13:20:23 +0100 Message-Id: <20230327122030.11756-4-naush@raspberrypi.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230327122030.11756-1-naush@raspberrypi.com> References: <20230327122030.11756-1-naush@raspberrypi.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v2 03/10] ipa: raspberrypi: Generalise the ALSC algorithm X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Naushir Patuck via libcamera-devel From: Naushir Patuck Reply-To: Naushir Patuck Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Remove any hard-coded assumptions about the target hardware platform from the ALSC algorithm. Instead, use the "target" string provided by the camera tuning config and generalised statistics structures to determing parameters such as grid and region sizes. The ALSC calculations use run-time allocated arrays/vectors on every frame. Allocating these might add a non-trivial run-time penalty. Replace these dynamic allocations with a set of reusable pre-allocated vectors during the init phase. Signed-off-by: Naushir Patuck Signed-off-by: David Plowman Reviewed-off-by: David Plowman Reviewed-by: Kieran Bingham --- src/ipa/raspberrypi/controller/alsc_status.h | 13 +- src/ipa/raspberrypi/controller/rpi/alsc.cpp | 341 +++++++++++-------- src/ipa/raspberrypi/controller/rpi/alsc.h | 29 +- src/ipa/raspberrypi/raspberrypi.cpp | 9 +- 4 files changed, 224 insertions(+), 168 deletions(-) diff --git a/src/ipa/raspberrypi/controller/alsc_status.h b/src/ipa/raspberrypi/controller/alsc_status.h index e5aa7e37c330..49a9f4a0cb5a 100644 --- a/src/ipa/raspberrypi/controller/alsc_status.h +++ b/src/ipa/raspberrypi/controller/alsc_status.h @@ -6,16 +6,17 @@ */ #pragma once +#include + /* * The ALSC algorithm should post the following structure into the image's * "alsc.status" metadata. */ -constexpr unsigned int AlscCellsX = 16; -constexpr unsigned int AlscCellsY = 12; - struct AlscStatus { - double r[AlscCellsY][AlscCellsX]; - double g[AlscCellsY][AlscCellsX]; - double b[AlscCellsY][AlscCellsX]; + std::vector r; + std::vector g; + std::vector b; + unsigned int rows; + unsigned int cols; }; diff --git a/src/ipa/raspberrypi/controller/rpi/alsc.cpp b/src/ipa/raspberrypi/controller/rpi/alsc.cpp index eb4e2f9496e1..51fe5d73f52d 100644 --- a/src/ipa/raspberrypi/controller/rpi/alsc.cpp +++ b/src/ipa/raspberrypi/controller/rpi/alsc.cpp @@ -5,6 +5,7 @@ * alsc.cpp - ALSC (auto lens shading correction) control algorithm */ +#include #include #include #include @@ -24,9 +25,6 @@ LOG_DEFINE_CATEGORY(RPiAlsc) #define NAME "rpi.alsc" -static const int X = AlscCellsX; -static const int Y = AlscCellsY; -static const int XY = X * Y; static const double InsufficientData = -1.0; Alsc::Alsc(Controller *controller) @@ -51,8 +49,11 @@ char const *Alsc::name() const return NAME; } -static int generateLut(double *lut, const libcamera::YamlObject ¶ms) +static int generateLut(std::vector &lut, const libcamera::YamlObject ¶ms, + const Size &size) { + /* These must be signed ints for the co-ordinate calculations below. */ + int X = size.width, Y = size.height; double cstrength = params["corner_strength"].get(2.0); if (cstrength <= 1.0) { LOG(RPiAlsc, Error) << "corner_strength must be > 1.0"; @@ -81,9 +82,9 @@ static int generateLut(double *lut, const libcamera::YamlObject ¶ms) return 0; } -static int readLut(double *lut, const libcamera::YamlObject ¶ms) +static int readLut(std::vector &lut, const libcamera::YamlObject ¶ms, const Size &size) { - if (params.size() != XY) { + if (params.size() != size.width * size.height) { LOG(RPiAlsc, Error) << "Invalid number of entries in LSC table"; return -EINVAL; } @@ -101,7 +102,7 @@ static int readLut(double *lut, const libcamera::YamlObject ¶ms) static int readCalibrations(std::vector &calibrations, const libcamera::YamlObject ¶ms, - std::string const &name) + std::string const &name, const Size &size) { if (params.contains(name)) { double lastCt = 0; @@ -119,7 +120,7 @@ static int readCalibrations(std::vector &calibrations, calibration.ct = lastCt = ct; const libcamera::YamlObject &table = p["table"]; - if (table.size() != XY) { + if (table.size() != size.width * size.height) { LOG(RPiAlsc, Error) << "Incorrect number of values for ct " << ct << " in " << name; @@ -127,6 +128,7 @@ static int readCalibrations(std::vector &calibrations, } int num = 0; + calibration.table.resize(size.width * size.height); for (const auto &elem : table.asList()) { value = elem.get(); if (!value) @@ -134,7 +136,7 @@ static int readCalibrations(std::vector &calibrations, calibration.table[num++] = *value; } - calibrations.push_back(calibration); + calibrations.push_back(std::move(calibration)); LOG(RPiAlsc, Debug) << "Read " << name << " calibration for ct " << ct; } @@ -144,6 +146,7 @@ static int readCalibrations(std::vector &calibrations, int Alsc::read(const libcamera::YamlObject ¶ms) { + config_.tableSize = getHardwareConfig().awbRegions; config_.framePeriod = params["frame_period"].get(12); config_.startupFrames = params["startup_frames"].get(10); config_.speed = params["speed"].get(0.05); @@ -153,28 +156,29 @@ int Alsc::read(const libcamera::YamlObject ¶ms) config_.minCount = params["min_count"].get(10.0); config_.minG = params["min_G"].get(50); config_.omega = params["omega"].get(1.3); - config_.nIter = params["n_iter"].get(X + Y); + config_.nIter = params["n_iter"].get(config_.tableSize.width + config_.tableSize.height); config_.luminanceStrength = params["luminance_strength"].get(1.0); - for (int i = 0; i < XY; i++) - config_.luminanceLut[i] = 1.0; + config_.luminanceLut.resize(config_.tableSize.width * config_.tableSize.height, 1.0); int ret = 0; if (params.contains("corner_strength")) - ret = generateLut(config_.luminanceLut, params); + ret = generateLut(config_.luminanceLut, params, config_.tableSize); else if (params.contains("luminance_lut")) - ret = readLut(config_.luminanceLut, params["luminance_lut"]); + ret = readLut(config_.luminanceLut, params["luminance_lut"], config_.tableSize); else LOG(RPiAlsc, Warning) << "no luminance table - assume unity everywhere"; if (ret) return ret; - ret = readCalibrations(config_.calibrationsCr, params, "calibrations_Cr"); + ret = readCalibrations(config_.calibrationsCr, params, "calibrations_Cr", + config_.tableSize); if (ret) return ret; - ret = readCalibrations(config_.calibrationsCb, params, "calibrations_Cb"); + ret = readCalibrations(config_.calibrationsCb, params, "calibrations_Cb", + config_.tableSize); if (ret) return ret; @@ -187,13 +191,16 @@ int Alsc::read(const libcamera::YamlObject ¶ms) static double getCt(Metadata *metadata, double defaultCt); static void getCalTable(double ct, std::vector const &calibrations, - double calTable[XY]); -static void resampleCalTable(double const calTableIn[XY], CameraMode const &cameraMode, - double calTableOut[XY]); -static void compensateLambdasForCal(double const calTable[XY], double const oldLambdas[XY], - double newLambdas[XY]); -static void addLuminanceToTables(double results[3][Y][X], double const lambdaR[XY], double lambdaG, - double const lambdaB[XY], double const luminanceLut[XY], + std::vector &calTable); +static void resampleCalTable(const std::vector &calTableIn, CameraMode const &cameraMode, + const Size &size, std::vector &calTableOut); +static void compensateLambdasForCal(const std::vector &calTable, + const std::vector &oldLambdas, + std::vector &newLambdas); +static void addLuminanceToTables(std::array, 3> &results, + const std::vector &lambdaR, double lambdaG, + const std::vector &lambdaB, + const std::vector &luminanceLut, double luminanceStrength); void Alsc::initialise() @@ -201,7 +208,28 @@ void Alsc::initialise() frameCount2_ = frameCount_ = framePhase_ = 0; firstTime_ = true; ct_ = config_.defaultCt; + + const size_t XY = config_.tableSize.width * config_.tableSize.height; + + for (auto &r : syncResults_) + r.resize(XY); + for (auto &r : prevSyncResults_) + r.resize(XY); + for (auto &r : asyncResults_) + r.resize(XY); + + luminanceTable_.resize(XY); + asyncLambdaR_.resize(XY); + asyncLambdaB_.resize(XY); /* The lambdas are initialised in the SwitchMode. */ + lambdaR_.resize(XY); + lambdaB_.resize(XY); + + /* Temporaries for the computations, but sensible to allocate this up-front! */ + for (auto &c : tmpC_) + c.resize(XY); + for (auto &m : tmpM_) + m.resize(XY); } void Alsc::waitForAysncThread() @@ -262,7 +290,7 @@ void Alsc::switchMode(CameraMode const &cameraMode, * We must resample the luminance table like we do the others, but it's * fixed so we can simply do it up front here. */ - resampleCalTable(config_.luminanceLut, cameraMode_, luminanceTable_); + resampleCalTable(config_.luminanceLut, cameraMode_, config_.tableSize, luminanceTable_); if (resetTables) { /* @@ -272,18 +300,18 @@ void Alsc::switchMode(CameraMode const &cameraMode, * the lambdas, but the rest of this code then echoes the code in * doAlsc, without the adaptive algorithm. */ - for (int i = 0; i < XY; i++) - lambdaR_[i] = lambdaB_[i] = 1.0; - double calTableR[XY], calTableB[XY], calTableTmp[XY]; + std::fill(lambdaR_.begin(), lambdaR_.end(), 1.0); + std::fill(lambdaB_.begin(), lambdaB_.end(), 1.0); + std::vector &calTableR = tmpC_[0], &calTableB = tmpC_[1], &calTableTmp = tmpC_[2]; getCalTable(ct_, config_.calibrationsCr, calTableTmp); - resampleCalTable(calTableTmp, cameraMode_, calTableR); + resampleCalTable(calTableTmp, cameraMode_, config_.tableSize, calTableR); getCalTable(ct_, config_.calibrationsCb, calTableTmp); - resampleCalTable(calTableTmp, cameraMode_, calTableB); + resampleCalTable(calTableTmp, cameraMode_, config_.tableSize, calTableB); compensateLambdasForCal(calTableR, lambdaR_, asyncLambdaR_); compensateLambdasForCal(calTableB, lambdaB_, asyncLambdaB_); addLuminanceToTables(syncResults_, asyncLambdaR_, 1.0, asyncLambdaB_, luminanceTable_, config_.luminanceStrength); - memcpy(prevSyncResults_, syncResults_, sizeof(prevSyncResults_)); + prevSyncResults_ = syncResults_; framePhase_ = config_.framePeriod; /* run the algo again asap */ firstTime_ = false; } @@ -294,7 +322,7 @@ void Alsc::fetchAsyncResults() LOG(RPiAlsc, Debug) << "Fetch ALSC results"; asyncFinished_ = false; asyncStarted_ = false; - memcpy(syncResults_, asyncResults_, sizeof(syncResults_)); + syncResults_ = asyncResults_; } double getCt(Metadata *metadata, double defaultCt) @@ -316,9 +344,9 @@ static void copyStats(RgbyRegions ®ions, StatisticsPtr &stats, if (!regions.numRegions()) regions.init(stats->awbRegions.size()); - double *rTable = (double *)status.r; - double *gTable = (double *)status.g; - double *bTable = (double *)status.b; + const std::vector &rTable = status.r; + const std::vector &gTable = status.g; + const std::vector &bTable = status.b; for (unsigned int i = 0; i < stats->awbRegions.numRegions(); i++) { auto r = stats->awbRegions.get(i); r.val.rSum = static_cast(r.val.rSum / rTable[i]); @@ -344,12 +372,9 @@ void Alsc::restartAsync(StatisticsPtr &stats, Metadata *imageMetadata) if (imageMetadata->get("alsc.status", alscStatus) != 0) { LOG(RPiAlsc, Warning) << "No ALSC status found for applied gains!"; - for (int y = 0; y < Y; y++) - for (int x = 0; x < X; x++) { - alscStatus.r[y][x] = 1.0; - alscStatus.g[y][x] = 1.0; - alscStatus.b[y][x] = 1.0; - } + alscStatus.r.resize(config_.tableSize.width * config_.tableSize.height, 1.0); + alscStatus.g.resize(config_.tableSize.width * config_.tableSize.height, 1.0); + alscStatus.b.resize(config_.tableSize.width * config_.tableSize.height, 1.0); } copyStats(statistics_, stats, alscStatus); framePhase_ = 0; @@ -380,15 +405,15 @@ void Alsc::prepare(Metadata *imageMetadata) fetchAsyncResults(); } /* Apply IIR filter to results and program into the pipeline. */ - double *ptr = (double *)syncResults_, - *pptr = (double *)prevSyncResults_; - for (unsigned int i = 0; i < sizeof(syncResults_) / sizeof(double); i++) - pptr[i] = speed * ptr[i] + (1.0 - speed) * pptr[i]; + for (unsigned int j = 0; j < syncResults_.size(); j++) { + for (unsigned int i = 0; i < syncResults_[j].size(); i++) + prevSyncResults_[j][i] = speed * syncResults_[j][i] + (1.0 - speed) * prevSyncResults_[j][i]; + } /* Put output values into status metadata. */ AlscStatus status; - memcpy(status.r, prevSyncResults_[0], sizeof(status.r)); - memcpy(status.g, prevSyncResults_[1], sizeof(status.g)); - memcpy(status.b, prevSyncResults_[2], sizeof(status.b)); + status.r = prevSyncResults_[0]; + status.g = prevSyncResults_[1]; + status.b = prevSyncResults_[2]; imageMetadata->set("alsc.status", status); } @@ -432,18 +457,17 @@ void Alsc::asyncFunc() } void getCalTable(double ct, std::vector const &calibrations, - double calTable[XY]) + std::vector &calTable) { if (calibrations.empty()) { - for (int i = 0; i < XY; i++) - calTable[i] = 1.0; + std::fill(calTable.begin(), calTable.end(), 1.0); LOG(RPiAlsc, Debug) << "no calibrations found"; } else if (ct <= calibrations.front().ct) { - memcpy(calTable, calibrations.front().table, XY * sizeof(double)); + calTable = calibrations.front().table; LOG(RPiAlsc, Debug) << "using calibration for " << calibrations.front().ct; } else if (ct >= calibrations.back().ct) { - memcpy(calTable, calibrations.back().table, XY * sizeof(double)); + calTable = calibrations.back().table; LOG(RPiAlsc, Debug) << "using calibration for " << calibrations.back().ct; } else { @@ -454,7 +478,7 @@ void getCalTable(double ct, std::vector const &calibrations, LOG(RPiAlsc, Debug) << "ct is " << ct << ", interpolating between " << ct0 << " and " << ct1; - for (int i = 0; i < XY; i++) + for (unsigned int i = 0; i < calTable.size(); i++) calTable[i] = (calibrations[idx].table[i] * (ct1 - ct) + calibrations[idx + 1].table[i] * (ct - ct0)) / @@ -462,9 +486,13 @@ void getCalTable(double ct, std::vector const &calibrations, } } -void resampleCalTable(double const calTableIn[XY], - CameraMode const &cameraMode, double calTableOut[XY]) +void resampleCalTable(const std::vector &calTableIn, + CameraMode const &cameraMode, const Size &size, + std::vector &calTableOut) { + int X = size.width; + int Y = size.height; + /* * Precalculate and cache the x sampling locations and phases to save * recomputing them on every row. @@ -501,23 +529,24 @@ void resampleCalTable(double const calTableIn[XY], yLo = Y - 1 - yLo; yHi = Y - 1 - yHi; } - double const *rowAbove = calTableIn + X * yLo; - double const *rowBelow = calTableIn + X * yHi; + double const *rowAbove = calTableIn.data() + X * yLo; + double const *rowBelow = calTableIn.data() + X * yHi; + double *out = calTableOut.data() + X * j; for (int i = 0; i < X; i++) { double above = rowAbove[xLo[i]] * (1 - xf[i]) + rowAbove[xHi[i]] * xf[i]; double below = rowBelow[xLo[i]] * (1 - xf[i]) + rowBelow[xHi[i]] * xf[i]; - *(calTableOut++) = above * (1 - yf) + below * yf; + *(out++) = above * (1 - yf) + below * yf; } } } /* Calculate chrominance statistics (R/G and B/G) for each region. */ -static void calculateCrCb(const RgbyRegions &awbRegion, double cr[XY], - double cb[XY], uint32_t minCount, uint16_t minG) +static void calculateCrCb(const RgbyRegions &awbRegion, std::vector &cr, + std::vector &cb, uint32_t minCount, uint16_t minG) { - for (int i = 0; i < XY; i++) { + for (unsigned int i = 0; i < cr.size(); i++) { auto s = awbRegion.get(i); if (s.counted <= minCount || s.val.gSum / s.counted <= minG) { @@ -530,33 +559,34 @@ static void calculateCrCb(const RgbyRegions &awbRegion, double cr[XY], } } -static void applyCalTable(double const calTable[XY], double C[XY]) +static void applyCalTable(const std::vector &calTable, std::vector &C) { - for (int i = 0; i < XY; i++) + for (unsigned int i = 0; i < C.size(); i++) if (C[i] != InsufficientData) C[i] *= calTable[i]; } -void compensateLambdasForCal(double const calTable[XY], - double const oldLambdas[XY], - double newLambdas[XY]) +void compensateLambdasForCal(const std::vector &calTable, + const std::vector &oldLambdas, + std::vector &newLambdas) { double minNewLambda = std::numeric_limits::max(); - for (int i = 0; i < XY; i++) { + for (unsigned int i = 0; i < newLambdas.size(); i++) { newLambdas[i] = oldLambdas[i] * calTable[i]; minNewLambda = std::min(minNewLambda, newLambdas[i]); } - for (int i = 0; i < XY; i++) + for (unsigned int i = 0; i < newLambdas.size(); i++) newLambdas[i] /= minNewLambda; } -[[maybe_unused]] static void printCalTable(double const C[XY]) +[[maybe_unused]] static void printCalTable(const std::vector &C, + const Size &size) { printf("table: [\n"); - for (int j = 0; j < Y; j++) { - for (int i = 0; i < X; i++) { - printf("%5.3f", 1.0 / C[j * X + i]); - if (i != X - 1 || j != Y - 1) + for (unsigned int j = 0; j < size.height; j++) { + for (unsigned int i = 0; i < size.width; i++) { + printf("%5.3f", 1.0 / C[j * size.width + i]); + if (i != size.width - 1 || j != size.height - 1) printf(","); } printf("\n"); @@ -577,9 +607,13 @@ static double computeWeight(double Ci, double Cj, double sigma) } /* Compute all weights. */ -static void computeW(double const C[XY], double sigma, double W[XY][4]) +static void computeW(const std::vector &C, double sigma, + std::vector> &W, const Size &size) { - for (int i = 0; i < XY; i++) { + size_t XY = size.width * size.height; + size_t X = size.width; + + for (unsigned int i = 0; i < XY; i++) { /* Start with neighbour above and go clockwise. */ W[i][0] = i >= X ? computeWeight(C[i], C[i - X], sigma) : 0; W[i][1] = i % X < X - 1 ? computeWeight(C[i], C[i + 1], sigma) : 0; @@ -589,11 +623,16 @@ static void computeW(double const C[XY], double sigma, double W[XY][4]) } /* Compute M, the large but sparse matrix such that M * lambdas = 0. */ -static void constructM(double const C[XY], double const W[XY][4], - double M[XY][4]) +static void constructM(const std::vector &C, + const std::vector> &W, + std::vector> &M, + const Size &size) { + size_t XY = size.width * size.height; + size_t X = size.width; + double epsilon = 0.001; - for (int i = 0; i < XY; i++) { + for (unsigned int i = 0; i < XY; i++) { /* * Note how, if C[i] == INSUFFICIENT_DATA, the weights will all * be zero so the equation is still set up correctly. @@ -614,79 +653,80 @@ static void constructM(double const C[XY], double const W[XY][4], * left/right neighbours are zero down the left/right edges, so we don't need * need to test the i value to exclude them. */ -static double computeLambdaBottom(int i, double const M[XY][4], - double lambda[XY]) +static double computeLambdaBottom(int i, const std::vector> &M, + std::vector &lambda, const Size &size) { - return M[i][1] * lambda[i + 1] + M[i][2] * lambda[i + X] + + return M[i][1] * lambda[i + 1] + M[i][2] * lambda[i + size.width] + M[i][3] * lambda[i - 1]; } -static double computeLambdaBottomStart(int i, double const M[XY][4], - double lambda[XY]) +static double computeLambdaBottomStart(int i, const std::vector> &M, + std::vector &lambda, const Size &size) { - return M[i][1] * lambda[i + 1] + M[i][2] * lambda[i + X]; + return M[i][1] * lambda[i + 1] + M[i][2] * lambda[i + size.width]; } -static double computeLambdaInterior(int i, double const M[XY][4], - double lambda[XY]) +static double computeLambdaInterior(int i, const std::vector> &M, + std::vector &lambda, const Size &size) { - return M[i][0] * lambda[i - X] + M[i][1] * lambda[i + 1] + - M[i][2] * lambda[i + X] + M[i][3] * lambda[i - 1]; + return M[i][0] * lambda[i - size.width] + M[i][1] * lambda[i + 1] + + M[i][2] * lambda[i + size.width] + M[i][3] * lambda[i - 1]; } -static double computeLambdaTop(int i, double const M[XY][4], - double lambda[XY]) +static double computeLambdaTop(int i, const std::vector> &M, + std::vector &lambda, const Size &size) { - return M[i][0] * lambda[i - X] + M[i][1] * lambda[i + 1] + + return M[i][0] * lambda[i - size.width] + M[i][1] * lambda[i + 1] + M[i][3] * lambda[i - 1]; } -static double computeLambdaTopEnd(int i, double const M[XY][4], - double lambda[XY]) +static double computeLambdaTopEnd(int i, const std::vector> &M, + std::vector &lambda, const Size &size) { - return M[i][0] * lambda[i - X] + M[i][3] * lambda[i - 1]; + return M[i][0] * lambda[i - size.width] + M[i][3] * lambda[i - 1]; } /* Gauss-Seidel iteration with over-relaxation. */ -static double gaussSeidel2Sor(double const M[XY][4], double omega, - double lambda[XY], double lambdaBound) +static double gaussSeidel2Sor(const std::vector> &M, double omega, + std::vector &lambda, double lambdaBound, + const Size &size) { + int XY = size.width * size.height; + int X = size.width; const double min = 1 - lambdaBound, max = 1 + lambdaBound; - double oldLambda[XY]; + std::vector oldLambda = lambda; int i; - for (i = 0; i < XY; i++) - oldLambda[i] = lambda[i]; - lambda[0] = computeLambdaBottomStart(0, M, lambda); + lambda[0] = computeLambdaBottomStart(0, M, lambda, size); lambda[0] = std::clamp(lambda[0], min, max); for (i = 1; i < X; i++) { - lambda[i] = computeLambdaBottom(i, M, lambda); + lambda[i] = computeLambdaBottom(i, M, lambda, size); lambda[i] = std::clamp(lambda[i], min, max); } for (; i < XY - X; i++) { - lambda[i] = computeLambdaInterior(i, M, lambda); + lambda[i] = computeLambdaInterior(i, M, lambda, size); lambda[i] = std::clamp(lambda[i], min, max); } for (; i < XY - 1; i++) { - lambda[i] = computeLambdaTop(i, M, lambda); + lambda[i] = computeLambdaTop(i, M, lambda, size); lambda[i] = std::clamp(lambda[i], min, max); } - lambda[i] = computeLambdaTopEnd(i, M, lambda); + lambda[i] = computeLambdaTopEnd(i, M, lambda, size); lambda[i] = std::clamp(lambda[i], min, max); /* * Also solve the system from bottom to top, to help spread the updates * better. */ - lambda[i] = computeLambdaTopEnd(i, M, lambda); + lambda[i] = computeLambdaTopEnd(i, M, lambda, size); lambda[i] = std::clamp(lambda[i], min, max); for (i = XY - 2; i >= XY - X; i--) { - lambda[i] = computeLambdaTop(i, M, lambda); + lambda[i] = computeLambdaTop(i, M, lambda, size); lambda[i] = std::clamp(lambda[i], min, max); } for (; i >= X; i--) { - lambda[i] = computeLambdaInterior(i, M, lambda); + lambda[i] = computeLambdaInterior(i, M, lambda, size); lambda[i] = std::clamp(lambda[i], min, max); } for (; i >= 1; i--) { - lambda[i] = computeLambdaBottom(i, M, lambda); + lambda[i] = computeLambdaBottom(i, M, lambda, size); lambda[i] = std::clamp(lambda[i], min, max); } - lambda[0] = computeLambdaBottomStart(0, M, lambda); + lambda[0] = computeLambdaBottomStart(0, M, lambda, size); lambda[0] = std::clamp(lambda[0], min, max); double maxDiff = 0; for (i = 0; i < XY; i++) { @@ -698,33 +738,33 @@ static double gaussSeidel2Sor(double const M[XY][4], double omega, } /* Normalise the values so that the smallest value is 1. */ -static void normalise(double *ptr, size_t n) +static void normalise(std::vector &results) { - double minval = ptr[0]; - for (size_t i = 1; i < n; i++) - minval = std::min(minval, ptr[i]); - for (size_t i = 0; i < n; i++) - ptr[i] /= minval; + double minval = *std::min_element(results.begin(), results.end()); + std::for_each(results.begin(), results.end(), + [minval](double val) { return val / minval; }); } /* Rescale the values so that the average value is 1. */ -static void reaverage(Span data) +static void reaverage(std::vector &data) { double sum = std::accumulate(data.begin(), data.end(), 0.0); double ratio = 1 / (sum / data.size()); - for (double &d : data) - d *= ratio; + std::for_each(data.begin(), data.end(), + [ratio](double val) { return val * ratio; }); } -static void runMatrixIterations(double const C[XY], double lambda[XY], - double const W[XY][4], double omega, - int nIter, double threshold, double lambdaBound) +static void runMatrixIterations(const std::vector &C, + std::vector &lambda, + const std::vector> &W, + std::vector> &M, double omega, + unsigned int nIter, double threshold, double lambdaBound, + const Size &size) { - double M[XY][4]; - constructM(C, W, M); + constructM(C, W, M, size); double lastMaxDiff = std::numeric_limits::max(); - for (int i = 0; i < nIter; i++) { - double maxDiff = fabs(gaussSeidel2Sor(M, omega, lambda, lambdaBound)); + for (unsigned int i = 0; i < nIter; i++) { + double maxDiff = fabs(gaussSeidel2Sor(M, omega, lambda, lambdaBound, size)); if (maxDiff < threshold) { LOG(RPiAlsc, Debug) << "Stop after " << i + 1 << " iterations"; @@ -741,39 +781,44 @@ static void runMatrixIterations(double const C[XY], double lambda[XY], lastMaxDiff = maxDiff; } /* We're going to normalise the lambdas so the total average is 1. */ - reaverage({ lambda, XY }); + reaverage(lambda); } -static void addLuminanceRb(double result[XY], double const lambda[XY], - double const luminanceLut[XY], +static void addLuminanceRb(std::vector &result, const std::vector &lambda, + const std::vector &luminanceLut, double luminanceStrength) { - for (int i = 0; i < XY; i++) + for (unsigned int i = 0; i < result.size(); i++) result[i] = lambda[i] * ((luminanceLut[i] - 1) * luminanceStrength + 1); } -static void addLuminanceG(double result[XY], double lambda, - double const luminanceLut[XY], +static void addLuminanceG(std::vector &result, double lambda, + const std::vector &luminanceLut, double luminanceStrength) { - for (int i = 0; i < XY; i++) + for (unsigned int i = 0; i < result.size(); i++) result[i] = lambda * ((luminanceLut[i] - 1) * luminanceStrength + 1); } -void addLuminanceToTables(double results[3][Y][X], double const lambdaR[XY], - double lambdaG, double const lambdaB[XY], - double const luminanceLut[XY], +void addLuminanceToTables(std::array, 3> &results, + const std::vector &lambdaR, + double lambdaG, const std::vector &lambdaB, + const std::vector &luminanceLut, double luminanceStrength) { - addLuminanceRb((double *)results[0], lambdaR, luminanceLut, luminanceStrength); - addLuminanceG((double *)results[1], lambdaG, luminanceLut, luminanceStrength); - addLuminanceRb((double *)results[2], lambdaB, luminanceLut, luminanceStrength); - normalise((double *)results, 3 * XY); + addLuminanceRb(results[0], lambdaR, luminanceLut, luminanceStrength); + addLuminanceG(results[1], lambdaG, luminanceLut, luminanceStrength); + addLuminanceRb(results[2], lambdaB, luminanceLut, luminanceStrength); + for (auto &r : results) + normalise(r); } void Alsc::doAlsc() { - double cr[XY], cb[XY], wr[XY][4], wb[XY][4], calTableR[XY], calTableB[XY], calTableTmp[XY]; + std::vector &cr = tmpC_[0], &cb = tmpC_[1], &calTableR = tmpC_[2], + &calTableB = tmpC_[3], &calTableTmp = tmpC_[4]; + std::vector> &wr = tmpM_[0], &wb = tmpM_[1], &M = tmpM_[2]; + /* * Calculate our R/B ("Cr"/"Cb") colour statistics, and assess which are * usable. @@ -784,9 +829,9 @@ void Alsc::doAlsc() * case the camera mode is not full-frame. */ getCalTable(ct_, config_.calibrationsCr, calTableTmp); - resampleCalTable(calTableTmp, cameraMode_, calTableR); + resampleCalTable(calTableTmp, cameraMode_, config_.tableSize, calTableR); getCalTable(ct_, config_.calibrationsCb, calTableTmp); - resampleCalTable(calTableTmp, cameraMode_, calTableB); + resampleCalTable(calTableTmp, cameraMode_, config_.tableSize, calTableB); /* * You could print out the cal tables for this image here, if you're * tuning the algorithm... @@ -796,13 +841,13 @@ void Alsc::doAlsc() applyCalTable(calTableR, cr); applyCalTable(calTableB, cb); /* Compute weights between zones. */ - computeW(cr, config_.sigmaCr, wr); - computeW(cb, config_.sigmaCb, wb); + computeW(cr, config_.sigmaCr, wr, config_.tableSize); + computeW(cb, config_.sigmaCb, wb, config_.tableSize); /* Run Gauss-Seidel iterations over the resulting matrix, for R and B. */ - runMatrixIterations(cr, lambdaR_, wr, config_.omega, config_.nIter, - config_.threshold, config_.lambdaBound); - runMatrixIterations(cb, lambdaB_, wb, config_.omega, config_.nIter, - config_.threshold, config_.lambdaBound); + runMatrixIterations(cr, lambdaR_, wr, M, config_.omega, config_.nIter, + config_.threshold, config_.lambdaBound, config_.tableSize); + runMatrixIterations(cb, lambdaB_, wb, M, config_.omega, config_.nIter, + config_.threshold, config_.lambdaBound, config_.tableSize); /* * Fold the calibrated gains into our final lambda values. (Note that on * the next run, we re-start with the lambda values that don't have the diff --git a/src/ipa/raspberrypi/controller/rpi/alsc.h b/src/ipa/raspberrypi/controller/rpi/alsc.h index 9167c9ffa2e3..85e998db40e9 100644 --- a/src/ipa/raspberrypi/controller/rpi/alsc.h +++ b/src/ipa/raspberrypi/controller/rpi/alsc.h @@ -6,9 +6,13 @@ */ #pragma once +#include #include #include #include +#include + +#include #include "../algorithm.h" #include "../alsc_status.h" @@ -20,7 +24,7 @@ namespace RPiController { struct AlscCalibration { double ct; - double table[AlscCellsX * AlscCellsY]; + std::vector table; }; struct AlscConfig { @@ -36,13 +40,14 @@ struct AlscConfig { uint16_t minG; double omega; uint32_t nIter; - double luminanceLut[AlscCellsX * AlscCellsY]; + std::vector luminanceLut; double luminanceStrength; std::vector calibrationsCr; std::vector calibrationsCb; double defaultCt; /* colour temperature if no metadata found */ double threshold; /* iteration termination threshold */ double lambdaBound; /* upper/lower bound for lambda from a value of 1 */ + libcamera::Size tableSize; }; class Alsc : public Algorithm @@ -62,7 +67,7 @@ private: AlscConfig config_; bool firstTime_; CameraMode cameraMode_; - double luminanceTable_[AlscCellsX * AlscCellsY]; + std::vector luminanceTable_; std::thread asyncThread_; void asyncFunc(); /* asynchronous thread function */ std::mutex mutex_; @@ -88,8 +93,8 @@ private: int frameCount_; /* counts up to startupFrames for Process function */ int frameCount2_; - double syncResults_[3][AlscCellsY][AlscCellsX]; - double prevSyncResults_[3][AlscCellsY][AlscCellsX]; + std::array, 3> syncResults_; + std::array, 3> prevSyncResults_; void waitForAysncThread(); /* * The following are for the asynchronous thread to use, though the main @@ -100,12 +105,16 @@ private: void fetchAsyncResults(); double ct_; RgbyRegions statistics_; - double asyncResults_[3][AlscCellsY][AlscCellsX]; - double asyncLambdaR_[AlscCellsX * AlscCellsY]; - double asyncLambdaB_[AlscCellsX * AlscCellsY]; + std::array, 3> asyncResults_; + std::vector asyncLambdaR_; + std::vector asyncLambdaB_; void doAlsc(); - double lambdaR_[AlscCellsX * AlscCellsY]; - double lambdaB_[AlscCellsX * AlscCellsY]; + std::vector lambdaR_; + std::vector lambdaB_; + + /* Temporaries for the computations */ + std::array, 5> tmpC_; + std::array>, 3> tmpM_; }; } /* namespace RPiController */ diff --git a/src/ipa/raspberrypi/raspberrypi.cpp b/src/ipa/raspberrypi/raspberrypi.cpp index b64cb96e2dde..257d862e07b6 100644 --- a/src/ipa/raspberrypi/raspberrypi.cpp +++ b/src/ipa/raspberrypi/raspberrypi.cpp @@ -14,6 +14,7 @@ #include #include #include +#include #include @@ -174,7 +175,7 @@ private: void applyDPC(const struct DpcStatus *dpcStatus, ControlList &ctrls); void applyLS(const struct AlscStatus *lsStatus, ControlList &ctrls); void applyAF(const struct AfStatus *afStatus, ControlList &lensCtrls); - void resampleTable(uint16_t dest[], double const src[12][16], int destW, int destH); + void resampleTable(uint16_t dest[], const std::vector &src, int destW, int destH); std::map buffers_; @@ -1768,7 +1769,7 @@ void IPARPi::applyAF(const struct AfStatus *afStatus, ControlList &lensCtrls) * Resamples a 16x12 table with central sampling to destW x destH with corner * sampling. */ -void IPARPi::resampleTable(uint16_t dest[], double const src[12][16], +void IPARPi::resampleTable(uint16_t dest[], const std::vector &src, int destW, int destH) { /* @@ -1793,8 +1794,8 @@ void IPARPi::resampleTable(uint16_t dest[], double const src[12][16], double yf = y - yLo; int yHi = yLo < 11 ? yLo + 1 : 11; yLo = yLo > 0 ? yLo : 0; - double const *rowAbove = src[yLo]; - double const *rowBelow = src[yHi]; + double const *rowAbove = src.data() + yLo * 16; + double const *rowBelow = src.data() + yHi * 16; for (int i = 0; i < destW; i++) { double above = rowAbove[xLo[i]] * (1 - xf[i]) + rowAbove[xHi[i]] * xf[i]; double below = rowBelow[xLo[i]] * (1 - xf[i]) + rowBelow[xHi[i]] * xf[i]; From patchwork Mon Mar 27 12:20:24 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Naushir Patuck X-Patchwork-Id: 18473 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id ED7C7BD160 for ; Mon, 27 Mar 2023 12:20:50 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 6213C62720; Mon, 27 Mar 2023 14:20:50 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1679919650; bh=Xp3l2Cn+AaoAMI7S8X+v1R7g9YRdikS8Td9pAFvi98M=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To:Cc: From; b=j3Uo14wzns7JaxhCdqTGnP9UBGl2f5bGQRVwhy7/0NxoRq+oFwtN7xvp3vvU+ecGv y3sFQxtS9acVhA0GgXmPnKtni5X3hBncq5qgEtp+6R7revpJVMxF+5himCUIbPR/Ui Zvc0jMeCpOCrUgeVnd2kIU9Q92x8qvWFQsp7WQBl6eHBS+TAg5asFZb/Gx0+unOPdL LY6LAIeK3hFTWaqOCKG37SCWyw78yCPsE7IrHfUiRbAWNNCIr+GZ+7L8hWcg58OHRL N3o8gHEs12cwmtGQvnJsiLGAmTd996gYETZbGkjHqWM63kH3V5xrUEt/XLotvVT1z2 51wL1ayXEokuA== Received: from mail-io1-xd32.google.com (mail-io1-xd32.google.com [IPv6:2607:f8b0:4864:20::d32]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 4034C62747 for ; Mon, 27 Mar 2023 14:20:46 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="sX0WDjEu"; dkim-atps=neutral Received: by mail-io1-xd32.google.com with SMTP id d22so2765225iow.12 for ; Mon, 27 Mar 2023 05:20:46 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; t=1679919645; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=3JdbajTs00HICJy9McNfdYmKn2OuE3fq0x4d14FeTTQ=; b=sX0WDjEu2bVwskROw2P+rOCxE8Px2y8SYBZouyQHJbRM30iWzkXnN5CGS2qhMO67ev k5xUFHNy0zzgNkg2DkTJWjIgdJskJdfSZd+lyaseDn0al3TwRAKTgzRulqa3DXKfFlY5 SDGUMfYVkmvyEnzK7Pb9JQPksPjfGGGKas/9I1ubPD+UDzJGxpvo8u+gH3XjJF4sC/aV dlzjIhyrahFNDMmGP4wakYQoSfXtbOFSsCWr8UuHSuvLx2VYxVIg1rAgynM8/tLSustO zw27q4ywwP+phVtVLWXwfCdlQdTJ6XEdBlrjAY+WAJtHodJlwQFHkm/Exyabfye3eeuS Ne6w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; t=1679919645; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=3JdbajTs00HICJy9McNfdYmKn2OuE3fq0x4d14FeTTQ=; b=d/O8YIjG7CpJhXlTmG5PGLO5+pd65R1truzoj76ivzTyWA1iYqJBjqT5oSx0ZxyAFd YKCik18ClcCOzR6yEh6JP5/JD7/cRU5GoAaBSp0yg3w+mQsh8a7hg/xI9J6rSzOKsVNM dJdVx1iSm/UHRqSgv2ymfS4kENagivf5dp4YF81DOZBl3G+dvWWPd5Cvj+selfdHPtIT Vu/GIlpoJWALQTCimvt3FQa1wJMSMOWDRZdJBF1vxx+b9npe8nsqs4TWUODbc/xRhhf+ TPlR+ywgIQe5kUQDky9wEh17mAry0Mfrj1x//JPPUHIUZxJGKiykaUMssk2bYzpeqJcM Tf1g== X-Gm-Message-State: AAQBX9df/gJ4bISvgelEy5fWd6cO/VVQoTrRUl3KPAlOSVgy7XxodRjf xSqfFJryBWH64+CnPqcDXS/PQ4UGOgJubTixYAUR7A== X-Google-Smtp-Source: AKy350b2GcxlegagGHlG2REtj3yrSVv/wzWqnwhbnVEYQFm2GV3GYZ3TGRRNjmoAgzVb8Z7ol/7jbw== X-Received: by 2002:a5d:8b54:0:b0:75c:5a15:9679 with SMTP id c20-20020a5d8b54000000b0075c5a159679mr1331739iot.6.1679919644578; Mon, 27 Mar 2023 05:20:44 -0700 (PDT) Received: from localhost.localdomain ([93.93.133.154]) by smtp.gmail.com with ESMTPSA id r7-20020a5edb47000000b00758ff97aaedsm3239026iop.16.2023.03.27.05.20.43 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 27 Mar 2023 05:20:44 -0700 (PDT) To: libcamera-devel@lists.libcamera.org Date: Mon, 27 Mar 2023 13:20:24 +0100 Message-Id: <20230327122030.11756-5-naush@raspberrypi.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230327122030.11756-1-naush@raspberrypi.com> References: <20230327122030.11756-1-naush@raspberrypi.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v2 04/10] ipa: raspberrypi: alsc: Replace std::vectors by Array2D class X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Naushir Patuck via libcamera-devel From: Naushir Patuck Reply-To: Naushir Patuck Cc: Jacopo Mondi Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" From: David Plowman The Array2D class is a very thin wrapper round std::vector that can be used almost identically in the code, but it carries its 2D size with it so that we aren't passing it around all the time. All the std::vectors that were X * Y in size (X and Y being the ALSC grid size) have been replaced. The sparse matrices that are XY * 4 in size have not been as they are somewhat different, are used differently, require more code changes, and actually make things more confusing if everything looks like an Array2D but are not the same. There should be no change in algorithm behaviour at all. Signed-off-by: David Plowman Reviewed-by: Naushir Patuck Reviewed-by: Jacopo Mondi --- src/ipa/raspberrypi/controller/rpi/alsc.cpp | 218 ++++++++++---------- src/ipa/raspberrypi/controller/rpi/alsc.h | 68 +++++- 2 files changed, 164 insertions(+), 122 deletions(-) diff --git a/src/ipa/raspberrypi/controller/rpi/alsc.cpp b/src/ipa/raspberrypi/controller/rpi/alsc.cpp index 51fe5d73f52d..524c48093590 100644 --- a/src/ipa/raspberrypi/controller/rpi/alsc.cpp +++ b/src/ipa/raspberrypi/controller/rpi/alsc.cpp @@ -49,11 +49,10 @@ char const *Alsc::name() const return NAME; } -static int generateLut(std::vector &lut, const libcamera::YamlObject ¶ms, - const Size &size) +static int generateLut(Array2D &lut, const libcamera::YamlObject ¶ms) { /* These must be signed ints for the co-ordinate calculations below. */ - int X = size.width, Y = size.height; + int X = lut.dimensions().width, Y = lut.dimensions().height; double cstrength = params["corner_strength"].get(2.0); if (cstrength <= 1.0) { LOG(RPiAlsc, Error) << "corner_strength must be > 1.0"; @@ -82,9 +81,9 @@ static int generateLut(std::vector &lut, const libcamera::YamlObject &pa return 0; } -static int readLut(std::vector &lut, const libcamera::YamlObject ¶ms, const Size &size) +static int readLut(Array2D &lut, const libcamera::YamlObject ¶ms) { - if (params.size() != size.width * size.height) { + if (params.size() != lut.size()) { LOG(RPiAlsc, Error) << "Invalid number of entries in LSC table"; return -EINVAL; } @@ -128,7 +127,7 @@ static int readCalibrations(std::vector &calibrations, } int num = 0; - calibration.table.resize(size.width * size.height); + calibration.table.resize(size); for (const auto &elem : table.asList()) { value = elem.get(); if (!value) @@ -160,13 +159,13 @@ int Alsc::read(const libcamera::YamlObject ¶ms) config_.luminanceStrength = params["luminance_strength"].get(1.0); - config_.luminanceLut.resize(config_.tableSize.width * config_.tableSize.height, 1.0); + config_.luminanceLut.resize(config_.tableSize, 1.0); int ret = 0; if (params.contains("corner_strength")) - ret = generateLut(config_.luminanceLut, params, config_.tableSize); + ret = generateLut(config_.luminanceLut, params); else if (params.contains("luminance_lut")) - ret = readLut(config_.luminanceLut, params["luminance_lut"], config_.tableSize); + ret = readLut(config_.luminanceLut, params["luminance_lut"]); else LOG(RPiAlsc, Warning) << "no luminance table - assume unity everywhere"; @@ -191,16 +190,16 @@ int Alsc::read(const libcamera::YamlObject ¶ms) static double getCt(Metadata *metadata, double defaultCt); static void getCalTable(double ct, std::vector const &calibrations, - std::vector &calTable); -static void resampleCalTable(const std::vector &calTableIn, CameraMode const &cameraMode, - const Size &size, std::vector &calTableOut); -static void compensateLambdasForCal(const std::vector &calTable, - const std::vector &oldLambdas, - std::vector &newLambdas); -static void addLuminanceToTables(std::array, 3> &results, - const std::vector &lambdaR, double lambdaG, - const std::vector &lambdaB, - const std::vector &luminanceLut, + Array2D &calTable); +static void resampleCalTable(const Array2D &calTableIn, CameraMode const &cameraMode, + Array2D &calTableOut); +static void compensateLambdasForCal(const Array2D &calTable, + const Array2D &oldLambdas, + Array2D &newLambdas); +static void addLuminanceToTables(std::array, 3> &results, + const Array2D &lambdaR, double lambdaG, + const Array2D &lambdaB, + const Array2D &luminanceLut, double luminanceStrength); void Alsc::initialise() @@ -212,22 +211,22 @@ void Alsc::initialise() const size_t XY = config_.tableSize.width * config_.tableSize.height; for (auto &r : syncResults_) - r.resize(XY); + r.resize(config_.tableSize); for (auto &r : prevSyncResults_) - r.resize(XY); + r.resize(config_.tableSize); for (auto &r : asyncResults_) - r.resize(XY); + r.resize(config_.tableSize); - luminanceTable_.resize(XY); - asyncLambdaR_.resize(XY); - asyncLambdaB_.resize(XY); + luminanceTable_.resize(config_.tableSize); + asyncLambdaR_.resize(config_.tableSize); + asyncLambdaB_.resize(config_.tableSize); /* The lambdas are initialised in the SwitchMode. */ - lambdaR_.resize(XY); - lambdaB_.resize(XY); + lambdaR_.resize(config_.tableSize); + lambdaB_.resize(config_.tableSize); /* Temporaries for the computations, but sensible to allocate this up-front! */ for (auto &c : tmpC_) - c.resize(XY); + c.resize(config_.tableSize); for (auto &m : tmpM_) m.resize(XY); } @@ -290,7 +289,7 @@ void Alsc::switchMode(CameraMode const &cameraMode, * We must resample the luminance table like we do the others, but it's * fixed so we can simply do it up front here. */ - resampleCalTable(config_.luminanceLut, cameraMode_, config_.tableSize, luminanceTable_); + resampleCalTable(config_.luminanceLut, cameraMode_, luminanceTable_); if (resetTables) { /* @@ -302,11 +301,11 @@ void Alsc::switchMode(CameraMode const &cameraMode, */ std::fill(lambdaR_.begin(), lambdaR_.end(), 1.0); std::fill(lambdaB_.begin(), lambdaB_.end(), 1.0); - std::vector &calTableR = tmpC_[0], &calTableB = tmpC_[1], &calTableTmp = tmpC_[2]; + Array2D &calTableR = tmpC_[0], &calTableB = tmpC_[1], &calTableTmp = tmpC_[2]; getCalTable(ct_, config_.calibrationsCr, calTableTmp); - resampleCalTable(calTableTmp, cameraMode_, config_.tableSize, calTableR); + resampleCalTable(calTableTmp, cameraMode_, calTableR); getCalTable(ct_, config_.calibrationsCb, calTableTmp); - resampleCalTable(calTableTmp, cameraMode_, config_.tableSize, calTableB); + resampleCalTable(calTableTmp, cameraMode_, calTableB); compensateLambdasForCal(calTableR, lambdaR_, asyncLambdaR_); compensateLambdasForCal(calTableB, lambdaB_, asyncLambdaB_); addLuminanceToTables(syncResults_, asyncLambdaR_, 1.0, asyncLambdaB_, @@ -411,9 +410,9 @@ void Alsc::prepare(Metadata *imageMetadata) } /* Put output values into status metadata. */ AlscStatus status; - status.r = prevSyncResults_[0]; - status.g = prevSyncResults_[1]; - status.b = prevSyncResults_[2]; + status.r = prevSyncResults_[0].data(); + status.g = prevSyncResults_[1].data(); + status.b = prevSyncResults_[2].data(); imageMetadata->set("alsc.status", status); } @@ -457,7 +456,7 @@ void Alsc::asyncFunc() } void getCalTable(double ct, std::vector const &calibrations, - std::vector &calTable) + Array2D &calTable) { if (calibrations.empty()) { std::fill(calTable.begin(), calTable.end(), 1.0); @@ -486,12 +485,12 @@ void getCalTable(double ct, std::vector const &calibrations, } } -void resampleCalTable(const std::vector &calTableIn, - CameraMode const &cameraMode, const Size &size, - std::vector &calTableOut) +void resampleCalTable(const Array2D &calTableIn, + CameraMode const &cameraMode, + Array2D &calTableOut) { - int X = size.width; - int Y = size.height; + int X = calTableIn.dimensions().width; + int Y = calTableIn.dimensions().height; /* * Precalculate and cache the x sampling locations and phases to save @@ -529,9 +528,9 @@ void resampleCalTable(const std::vector &calTableIn, yLo = Y - 1 - yLo; yHi = Y - 1 - yHi; } - double const *rowAbove = calTableIn.data() + X * yLo; - double const *rowBelow = calTableIn.data() + X * yHi; - double *out = calTableOut.data() + X * j; + double const *rowAbove = calTableIn.ptr() + X * yLo; + double const *rowBelow = calTableIn.ptr() + X * yHi; + double *out = calTableOut.ptr() + X * j; for (int i = 0; i < X; i++) { double above = rowAbove[xLo[i]] * (1 - xf[i]) + rowAbove[xHi[i]] * xf[i]; @@ -543,8 +542,8 @@ void resampleCalTable(const std::vector &calTableIn, } /* Calculate chrominance statistics (R/G and B/G) for each region. */ -static void calculateCrCb(const RgbyRegions &awbRegion, std::vector &cr, - std::vector &cb, uint32_t minCount, uint16_t minG) +static void calculateCrCb(const RgbyRegions &awbRegion, Array2D &cr, + Array2D &cb, uint32_t minCount, uint16_t minG) { for (unsigned int i = 0; i < cr.size(); i++) { auto s = awbRegion.get(i); @@ -559,16 +558,16 @@ static void calculateCrCb(const RgbyRegions &awbRegion, std::vector &cr, } } -static void applyCalTable(const std::vector &calTable, std::vector &C) +static void applyCalTable(const Array2D &calTable, Array2D &C) { for (unsigned int i = 0; i < C.size(); i++) if (C[i] != InsufficientData) C[i] *= calTable[i]; } -void compensateLambdasForCal(const std::vector &calTable, - const std::vector &oldLambdas, - std::vector &newLambdas) +void compensateLambdasForCal(const Array2D &calTable, + const Array2D &oldLambdas, + Array2D &newLambdas) { double minNewLambda = std::numeric_limits::max(); for (unsigned int i = 0; i < newLambdas.size(); i++) { @@ -579,9 +578,9 @@ void compensateLambdasForCal(const std::vector &calTable, newLambdas[i] /= minNewLambda; } -[[maybe_unused]] static void printCalTable(const std::vector &C, - const Size &size) +[[maybe_unused]] static void printCalTable(const Array2D &C) { + const Size &size = C.dimensions(); printf("table: [\n"); for (unsigned int j = 0; j < size.height; j++) { for (unsigned int i = 0; i < size.width; i++) { @@ -607,11 +606,11 @@ static double computeWeight(double Ci, double Cj, double sigma) } /* Compute all weights. */ -static void computeW(const std::vector &C, double sigma, - std::vector> &W, const Size &size) +static void computeW(const Array2D &C, double sigma, + std::vector> &W) { - size_t XY = size.width * size.height; - size_t X = size.width; + size_t XY = C.size(); + size_t X = C.dimensions().width; for (unsigned int i = 0; i < XY; i++) { /* Start with neighbour above and go clockwise. */ @@ -623,13 +622,12 @@ static void computeW(const std::vector &C, double sigma, } /* Compute M, the large but sparse matrix such that M * lambdas = 0. */ -static void constructM(const std::vector &C, +static void constructM(const Array2D &C, const std::vector> &W, - std::vector> &M, - const Size &size) + std::vector> &M) { - size_t XY = size.width * size.height; - size_t X = size.width; + size_t XY = C.size(); + size_t X = C.dimensions().width; double epsilon = 0.001; for (unsigned int i = 0; i < XY; i++) { @@ -654,79 +652,78 @@ static void constructM(const std::vector &C, * need to test the i value to exclude them. */ static double computeLambdaBottom(int i, const std::vector> &M, - std::vector &lambda, const Size &size) + Array2D &lambda) { - return M[i][1] * lambda[i + 1] + M[i][2] * lambda[i + size.width] + + return M[i][1] * lambda[i + 1] + M[i][2] * lambda[i + lambda.dimensions().width] + M[i][3] * lambda[i - 1]; } static double computeLambdaBottomStart(int i, const std::vector> &M, - std::vector &lambda, const Size &size) + Array2D &lambda) { - return M[i][1] * lambda[i + 1] + M[i][2] * lambda[i + size.width]; + return M[i][1] * lambda[i + 1] + M[i][2] * lambda[i + lambda.dimensions().width]; } static double computeLambdaInterior(int i, const std::vector> &M, - std::vector &lambda, const Size &size) + Array2D &lambda) { - return M[i][0] * lambda[i - size.width] + M[i][1] * lambda[i + 1] + - M[i][2] * lambda[i + size.width] + M[i][3] * lambda[i - 1]; + return M[i][0] * lambda[i - lambda.dimensions().width] + M[i][1] * lambda[i + 1] + + M[i][2] * lambda[i + lambda.dimensions().width] + M[i][3] * lambda[i - 1]; } static double computeLambdaTop(int i, const std::vector> &M, - std::vector &lambda, const Size &size) + Array2D &lambda) { - return M[i][0] * lambda[i - size.width] + M[i][1] * lambda[i + 1] + + return M[i][0] * lambda[i - lambda.dimensions().width] + M[i][1] * lambda[i + 1] + M[i][3] * lambda[i - 1]; } static double computeLambdaTopEnd(int i, const std::vector> &M, - std::vector &lambda, const Size &size) + Array2D &lambda) { - return M[i][0] * lambda[i - size.width] + M[i][3] * lambda[i - 1]; + return M[i][0] * lambda[i - lambda.dimensions().width] + M[i][3] * lambda[i - 1]; } /* Gauss-Seidel iteration with over-relaxation. */ static double gaussSeidel2Sor(const std::vector> &M, double omega, - std::vector &lambda, double lambdaBound, - const Size &size) + Array2D &lambda, double lambdaBound) { - int XY = size.width * size.height; - int X = size.width; + int XY = lambda.size(); + int X = lambda.dimensions().width; const double min = 1 - lambdaBound, max = 1 + lambdaBound; - std::vector oldLambda = lambda; + Array2D oldLambda = lambda; int i; - lambda[0] = computeLambdaBottomStart(0, M, lambda, size); + lambda[0] = computeLambdaBottomStart(0, M, lambda); lambda[0] = std::clamp(lambda[0], min, max); for (i = 1; i < X; i++) { - lambda[i] = computeLambdaBottom(i, M, lambda, size); + lambda[i] = computeLambdaBottom(i, M, lambda); lambda[i] = std::clamp(lambda[i], min, max); } for (; i < XY - X; i++) { - lambda[i] = computeLambdaInterior(i, M, lambda, size); + lambda[i] = computeLambdaInterior(i, M, lambda); lambda[i] = std::clamp(lambda[i], min, max); } for (; i < XY - 1; i++) { - lambda[i] = computeLambdaTop(i, M, lambda, size); + lambda[i] = computeLambdaTop(i, M, lambda); lambda[i] = std::clamp(lambda[i], min, max); } - lambda[i] = computeLambdaTopEnd(i, M, lambda, size); + lambda[i] = computeLambdaTopEnd(i, M, lambda); lambda[i] = std::clamp(lambda[i], min, max); /* * Also solve the system from bottom to top, to help spread the updates * better. */ - lambda[i] = computeLambdaTopEnd(i, M, lambda, size); + lambda[i] = computeLambdaTopEnd(i, M, lambda); lambda[i] = std::clamp(lambda[i], min, max); for (i = XY - 2; i >= XY - X; i--) { - lambda[i] = computeLambdaTop(i, M, lambda, size); + lambda[i] = computeLambdaTop(i, M, lambda); lambda[i] = std::clamp(lambda[i], min, max); } for (; i >= X; i--) { - lambda[i] = computeLambdaInterior(i, M, lambda, size); + lambda[i] = computeLambdaInterior(i, M, lambda); lambda[i] = std::clamp(lambda[i], min, max); } for (; i >= 1; i--) { - lambda[i] = computeLambdaBottom(i, M, lambda, size); + lambda[i] = computeLambdaBottom(i, M, lambda); lambda[i] = std::clamp(lambda[i], min, max); } - lambda[0] = computeLambdaBottomStart(0, M, lambda, size); + lambda[0] = computeLambdaBottomStart(0, M, lambda); lambda[0] = std::clamp(lambda[0], min, max); double maxDiff = 0; for (i = 0; i < XY; i++) { @@ -738,7 +735,7 @@ static double gaussSeidel2Sor(const std::vector> &M, doubl } /* Normalise the values so that the smallest value is 1. */ -static void normalise(std::vector &results) +static void normalise(Array2D &results) { double minval = *std::min_element(results.begin(), results.end()); std::for_each(results.begin(), results.end(), @@ -746,7 +743,7 @@ static void normalise(std::vector &results) } /* Rescale the values so that the average value is 1. */ -static void reaverage(std::vector &data) +static void reaverage(Array2D &data) { double sum = std::accumulate(data.begin(), data.end(), 0.0); double ratio = 1 / (sum / data.size()); @@ -754,17 +751,16 @@ static void reaverage(std::vector &data) [ratio](double val) { return val * ratio; }); } -static void runMatrixIterations(const std::vector &C, - std::vector &lambda, +static void runMatrixIterations(const Array2D &C, + Array2D &lambda, const std::vector> &W, std::vector> &M, double omega, - unsigned int nIter, double threshold, double lambdaBound, - const Size &size) + unsigned int nIter, double threshold, double lambdaBound) { - constructM(C, W, M, size); + constructM(C, W, M); double lastMaxDiff = std::numeric_limits::max(); for (unsigned int i = 0; i < nIter; i++) { - double maxDiff = fabs(gaussSeidel2Sor(M, omega, lambda, lambdaBound, size)); + double maxDiff = fabs(gaussSeidel2Sor(M, omega, lambda, lambdaBound)); if (maxDiff < threshold) { LOG(RPiAlsc, Debug) << "Stop after " << i + 1 << " iterations"; @@ -784,26 +780,26 @@ static void runMatrixIterations(const std::vector &C, reaverage(lambda); } -static void addLuminanceRb(std::vector &result, const std::vector &lambda, - const std::vector &luminanceLut, +static void addLuminanceRb(Array2D &result, const Array2D &lambda, + const Array2D &luminanceLut, double luminanceStrength) { for (unsigned int i = 0; i < result.size(); i++) result[i] = lambda[i] * ((luminanceLut[i] - 1) * luminanceStrength + 1); } -static void addLuminanceG(std::vector &result, double lambda, - const std::vector &luminanceLut, +static void addLuminanceG(Array2D &result, double lambda, + const Array2D &luminanceLut, double luminanceStrength) { for (unsigned int i = 0; i < result.size(); i++) result[i] = lambda * ((luminanceLut[i] - 1) * luminanceStrength + 1); } -void addLuminanceToTables(std::array, 3> &results, - const std::vector &lambdaR, - double lambdaG, const std::vector &lambdaB, - const std::vector &luminanceLut, +void addLuminanceToTables(std::array, 3> &results, + const Array2D &lambdaR, + double lambdaG, const Array2D &lambdaB, + const Array2D &luminanceLut, double luminanceStrength) { addLuminanceRb(results[0], lambdaR, luminanceLut, luminanceStrength); @@ -815,8 +811,8 @@ void addLuminanceToTables(std::array, 3> &results, void Alsc::doAlsc() { - std::vector &cr = tmpC_[0], &cb = tmpC_[1], &calTableR = tmpC_[2], - &calTableB = tmpC_[3], &calTableTmp = tmpC_[4]; + Array2D &cr = tmpC_[0], &cb = tmpC_[1], &calTableR = tmpC_[2], + &calTableB = tmpC_[3], &calTableTmp = tmpC_[4]; std::vector> &wr = tmpM_[0], &wb = tmpM_[1], &M = tmpM_[2]; /* @@ -829,9 +825,9 @@ void Alsc::doAlsc() * case the camera mode is not full-frame. */ getCalTable(ct_, config_.calibrationsCr, calTableTmp); - resampleCalTable(calTableTmp, cameraMode_, config_.tableSize, calTableR); + resampleCalTable(calTableTmp, cameraMode_, calTableR); getCalTable(ct_, config_.calibrationsCb, calTableTmp); - resampleCalTable(calTableTmp, cameraMode_, config_.tableSize, calTableB); + resampleCalTable(calTableTmp, cameraMode_, calTableB); /* * You could print out the cal tables for this image here, if you're * tuning the algorithm... @@ -841,13 +837,13 @@ void Alsc::doAlsc() applyCalTable(calTableR, cr); applyCalTable(calTableB, cb); /* Compute weights between zones. */ - computeW(cr, config_.sigmaCr, wr, config_.tableSize); - computeW(cb, config_.sigmaCb, wb, config_.tableSize); + computeW(cr, config_.sigmaCr, wr); + computeW(cb, config_.sigmaCb, wb); /* Run Gauss-Seidel iterations over the resulting matrix, for R and B. */ runMatrixIterations(cr, lambdaR_, wr, M, config_.omega, config_.nIter, - config_.threshold, config_.lambdaBound, config_.tableSize); + config_.threshold, config_.lambdaBound); runMatrixIterations(cb, lambdaB_, wb, M, config_.omega, config_.nIter, - config_.threshold, config_.lambdaBound, config_.tableSize); + config_.threshold, config_.lambdaBound); /* * Fold the calibrated gains into our final lambda values. (Note that on * the next run, we re-start with the lambda values that don't have the diff --git a/src/ipa/raspberrypi/controller/rpi/alsc.h b/src/ipa/raspberrypi/controller/rpi/alsc.h index 85e998db40e9..1ab61299c4cd 100644 --- a/src/ipa/raspberrypi/controller/rpi/alsc.h +++ b/src/ipa/raspberrypi/controller/rpi/alsc.h @@ -22,9 +22,55 @@ namespace RPiController { /* Algorithm to generate automagic LSC (Lens Shading Correction) tables. */ +/* + * The Array2D class is a very thin wrapper round std::vector so that it can + * be used in exactly the same way in the code but carries its correct width + * and height ("dimensions") with it. + */ + +template +class Array2D +{ +public: + using Size = libcamera::Size; + + const Size &dimensions() const { return dimensions_; } + + size_t size() const { return data_.size(); } + + const std::vector &data() const { return data_; } + + void resize(const Size &dims) + { + dimensions_ = dims; + data_.resize(dims.width * dims.height); + } + + void resize(const Size &dims, const T &value) + { + resize(dims); + std::fill(data_.begin(), data_.end(), value); + } + + T &operator[](int index) { return data_[index]; } + + const T &operator[](int index) const { return data_[index]; } + + T *ptr() { return data_.data(); } + + const T *ptr() const { return data_.data(); } + + auto begin() { return data_.begin(); } + auto end() { return data_.end(); } + +private: + Size dimensions_; + std::vector data_; +}; + struct AlscCalibration { double ct; - std::vector table; + Array2D table; }; struct AlscConfig { @@ -40,7 +86,7 @@ struct AlscConfig { uint16_t minG; double omega; uint32_t nIter; - std::vector luminanceLut; + Array2D luminanceLut; double luminanceStrength; std::vector calibrationsCr; std::vector calibrationsCb; @@ -67,7 +113,7 @@ private: AlscConfig config_; bool firstTime_; CameraMode cameraMode_; - std::vector luminanceTable_; + Array2D luminanceTable_; std::thread asyncThread_; void asyncFunc(); /* asynchronous thread function */ std::mutex mutex_; @@ -93,8 +139,8 @@ private: int frameCount_; /* counts up to startupFrames for Process function */ int frameCount2_; - std::array, 3> syncResults_; - std::array, 3> prevSyncResults_; + std::array, 3> syncResults_; + std::array, 3> prevSyncResults_; void waitForAysncThread(); /* * The following are for the asynchronous thread to use, though the main @@ -105,15 +151,15 @@ private: void fetchAsyncResults(); double ct_; RgbyRegions statistics_; - std::array, 3> asyncResults_; - std::vector asyncLambdaR_; - std::vector asyncLambdaB_; + std::array, 3> asyncResults_; + Array2D asyncLambdaR_; + Array2D asyncLambdaB_; void doAlsc(); - std::vector lambdaR_; - std::vector lambdaB_; + Array2D lambdaR_; + Array2D lambdaB_; /* Temporaries for the computations */ - std::array, 5> tmpC_; + std::array, 5> tmpC_; std::array>, 3> tmpM_; }; From patchwork Mon Mar 27 12:20:25 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Naushir Patuck X-Patchwork-Id: 18474 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id EE1FDC326C for ; Mon, 27 Mar 2023 12:20:51 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id A116562767; Mon, 27 Mar 2023 14:20:51 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1679919651; bh=klPCji71t9ff50/NTw25Wr7qoGXqjM2oZwi1ptHCVKI=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To:Cc: From; b=mtCRhjttbE6t4Hriduc8rqn+NZLzL9l4LIVPnQqqCJY2MpkyfIoh7DVmB0kqKd2Oz UA0pJK4O9wI8d8/lfVvSGXmxN8ccqQAin4eygsxkzhzzvvCULrApnNIcrQLm+niX/b QAcPPXzQ3I73C9qMorWFnJZntc86gSF5Eu+BWFNl67h1aRZd/dr43jl7gqbtbUk4Ex kO5H2ZUc/r5bJP1qnWXnZ8ovqypVs1QTV393dxUcGL+4xZMSst7iUQMIgttLaDgrAa mDfmV4IrIbUZHci+S0Ttk7NPpiYEc//LUHH4t5uPR2k9C1RsKZP+puVgjeefoys1j0 qIWmEnGKzACAA== Received: from mail-il1-x12a.google.com (mail-il1-x12a.google.com [IPv6:2607:f8b0:4864:20::12a]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 3E81962747 for ; Mon, 27 Mar 2023 14:20:47 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="cuCWr7qb"; dkim-atps=neutral Received: by mail-il1-x12a.google.com with SMTP id h11so4512717ild.11 for ; Mon, 27 Mar 2023 05:20:47 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; t=1679919646; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=548zcSLJYSR6puTSQTcbbtm0W6Y5GofHbCgq+Ac6wj0=; b=cuCWr7qbrh2dk9ysm9/4Yi+HViFsEDnCNDAuJvvnohN6QwdQaOeSnHUq7yVs6oK83m 3CMCYufg5ipouPZIpEMn2JjLRBjOimaI9ckytp1muPK9Obm+d8umozH2SgJNSFLYx++q pJn/815uxj016I7wDjLxqNh574tz0wcXnjqvjp8I5OL4QKTaMQhozuUmQuV2jpSmRLo6 WUjTkDuMk7sjvYPUKQGZ3MkajOECj0EY/eLhIn+twgE7Do0NA0lfhKRG3MiL+d+2nEdB YyJf59reUh70lIstYluOdgfrTINa0UoVLaR4zPlX/2DzOfz8mfBPi8mlIULaBKvCaty8 RJ+A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; t=1679919646; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=548zcSLJYSR6puTSQTcbbtm0W6Y5GofHbCgq+Ac6wj0=; b=2qN0OV5vJe+IiNOrSlmZj9t2kHI34hjA6p8iyDyrAzIB6r98dcPBMJ1Q+44udLY3Gu kLWf/uBXcfKol7LZlOiDxbngXv9f0BTHRCZjkHeHN28f8l9hWefnpbf8rRuCr2xVxcxD Gk9uuWBgk5K4hXQ7JHWon9zi4hm5+AxnQ/uU9twCpSEur9wciLa24PsEKSJepg4EtEML vLSDd+XXg1o5yLJc7PF0+mIbckQD9Qjn7lcpo+0gUOXwgMhfIcxhHkyiRPocvqCBgpDF ng6khmDg8/GnR3XGxp5kkFS7h3WpxmkCg1pXo5ui50EZj1t8+Ai589kHhBR6W0TKkXT5 pwZQ== X-Gm-Message-State: AAQBX9d+GtSjUBQrt+Ks9E/pdqWgAzzDHoJPOkCkcX+dJ/evBA3j8tZa tsohcdFZLAPw60OkyZE+KKZyp4RYoc6+JORJRDmQ0Q== X-Google-Smtp-Source: AKy350YkYUUcK7HiNDPSGBkShSBbq4CzNDLwmgCdHU3wK+UFUQTeGh5hxYCObXxE7HVXI8lapTqB8g== X-Received: by 2002:a92:512:0:b0:316:b0bb:3d99 with SMTP id q18-20020a920512000000b00316b0bb3d99mr7950014ile.22.1679919645815; Mon, 27 Mar 2023 05:20:45 -0700 (PDT) Received: from localhost.localdomain ([93.93.133.154]) by smtp.gmail.com with ESMTPSA id r7-20020a5edb47000000b00758ff97aaedsm3239026iop.16.2023.03.27.05.20.44 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 27 Mar 2023 05:20:45 -0700 (PDT) To: libcamera-devel@lists.libcamera.org Date: Mon, 27 Mar 2023 13:20:25 +0100 Message-Id: <20230327122030.11756-6-naush@raspberrypi.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230327122030.11756-1-naush@raspberrypi.com> References: <20230327122030.11756-1-naush@raspberrypi.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v2 05/10] ipa: raspberrypi: alsc: Use a better type name for sparse arrays X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Naushir Patuck via libcamera-devel From: Naushir Patuck Reply-To: Naushir Patuck Cc: Jacopo Mondi Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" From: David Plowman The algorithm uses the data type std::vector> to represent the large sparse matrices that are XY (X, Y being the ALSC grid size) high but with only 4 non-zero elements on each row. Replace this slightly long type name by SparseArray. No functional changes. Signed-off-by: David Plowman Reviewed-by: Naushir Patuck Reviewed-by: Jacopo Mondi --- src/ipa/raspberrypi/controller/rpi/alsc.cpp | 24 ++++++++++----------- src/ipa/raspberrypi/controller/rpi/alsc.h | 10 ++++++++- 2 files changed, 21 insertions(+), 13 deletions(-) diff --git a/src/ipa/raspberrypi/controller/rpi/alsc.cpp b/src/ipa/raspberrypi/controller/rpi/alsc.cpp index 524c48093590..3a2e8fe00ca6 100644 --- a/src/ipa/raspberrypi/controller/rpi/alsc.cpp +++ b/src/ipa/raspberrypi/controller/rpi/alsc.cpp @@ -607,7 +607,7 @@ static double computeWeight(double Ci, double Cj, double sigma) /* Compute all weights. */ static void computeW(const Array2D &C, double sigma, - std::vector> &W) + SparseArray &W) { size_t XY = C.size(); size_t X = C.dimensions().width; @@ -623,8 +623,8 @@ static void computeW(const Array2D &C, double sigma, /* Compute M, the large but sparse matrix such that M * lambdas = 0. */ static void constructM(const Array2D &C, - const std::vector> &W, - std::vector> &M) + const SparseArray &W, + SparseArray &M) { size_t XY = C.size(); size_t X = C.dimensions().width; @@ -651,37 +651,37 @@ static void constructM(const Array2D &C, * left/right neighbours are zero down the left/right edges, so we don't need * need to test the i value to exclude them. */ -static double computeLambdaBottom(int i, const std::vector> &M, +static double computeLambdaBottom(int i, const SparseArray &M, Array2D &lambda) { return M[i][1] * lambda[i + 1] + M[i][2] * lambda[i + lambda.dimensions().width] + M[i][3] * lambda[i - 1]; } -static double computeLambdaBottomStart(int i, const std::vector> &M, +static double computeLambdaBottomStart(int i, const SparseArray &M, Array2D &lambda) { return M[i][1] * lambda[i + 1] + M[i][2] * lambda[i + lambda.dimensions().width]; } -static double computeLambdaInterior(int i, const std::vector> &M, +static double computeLambdaInterior(int i, const SparseArray &M, Array2D &lambda) { return M[i][0] * lambda[i - lambda.dimensions().width] + M[i][1] * lambda[i + 1] + M[i][2] * lambda[i + lambda.dimensions().width] + M[i][3] * lambda[i - 1]; } -static double computeLambdaTop(int i, const std::vector> &M, +static double computeLambdaTop(int i, const SparseArray &M, Array2D &lambda) { return M[i][0] * lambda[i - lambda.dimensions().width] + M[i][1] * lambda[i + 1] + M[i][3] * lambda[i - 1]; } -static double computeLambdaTopEnd(int i, const std::vector> &M, +static double computeLambdaTopEnd(int i, const SparseArray &M, Array2D &lambda) { return M[i][0] * lambda[i - lambda.dimensions().width] + M[i][3] * lambda[i - 1]; } /* Gauss-Seidel iteration with over-relaxation. */ -static double gaussSeidel2Sor(const std::vector> &M, double omega, +static double gaussSeidel2Sor(const SparseArray &M, double omega, Array2D &lambda, double lambdaBound) { int XY = lambda.size(); @@ -753,8 +753,8 @@ static void reaverage(Array2D &data) static void runMatrixIterations(const Array2D &C, Array2D &lambda, - const std::vector> &W, - std::vector> &M, double omega, + const SparseArray &W, + SparseArray &M, double omega, unsigned int nIter, double threshold, double lambdaBound) { constructM(C, W, M); @@ -813,7 +813,7 @@ void Alsc::doAlsc() { Array2D &cr = tmpC_[0], &cb = tmpC_[1], &calTableR = tmpC_[2], &calTableB = tmpC_[3], &calTableTmp = tmpC_[4]; - std::vector> &wr = tmpM_[0], &wb = tmpM_[1], &M = tmpM_[2]; + SparseArray &wr = tmpM_[0], &wb = tmpM_[1], &M = tmpM_[2]; /* * Calculate our R/B ("Cr"/"Cb") colour statistics, and assess which are diff --git a/src/ipa/raspberrypi/controller/rpi/alsc.h b/src/ipa/raspberrypi/controller/rpi/alsc.h index 1ab61299c4cd..0b6d9478073c 100644 --- a/src/ipa/raspberrypi/controller/rpi/alsc.h +++ b/src/ipa/raspberrypi/controller/rpi/alsc.h @@ -68,6 +68,14 @@ private: std::vector data_; }; +/* + * We'll use the term SparseArray for the large sparse matrices that are + * XY tall but have only 4 non-zero elements on each row. + */ + +template +using SparseArray = std::vector>; + struct AlscCalibration { double ct; Array2D table; @@ -160,7 +168,7 @@ private: /* Temporaries for the computations */ std::array, 5> tmpC_; - std::array>, 3> tmpM_; + std::array, 3> tmpM_; }; } /* namespace RPiController */ From patchwork Mon Mar 27 12:20:26 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Naushir Patuck X-Patchwork-Id: 18475 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 92F86C329D for ; Mon, 27 Mar 2023 12:20:52 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 2D30162754; Mon, 27 Mar 2023 14:20:52 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1679919652; bh=XJZ+b37RC6DAoNkN9LySXWzskZlFzHIPkUne8vcW67I=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To:Cc: From; b=lD+PI0F6CZZ0LG6+dCzDlZnoTumBlN+PWjnHUN2z59Bkqglgd2iSha2J93FOxTu7v UkLes9+uEaNAt62v9jeDFCIhIdqiuGYflh36T9wWy1VzCCNgz+nL1Cd2surSDDQaFe atN1ZEZWGzqIFNW/NSIFJbP/2GOHvJ6LmPWNXGvM1BMtP0VaPgwJ9NThop9dS7cByt B1eD51htCoCNqztDWNJZDitOMC2QLiPP/nqRHqC4LEUKXF6kK3jf16fyzLV2jLBCgH uBo2emCUG899bUwdF8N72Te5V72iUW2jiXa1xje+mw8QguthHKcYI5UgPz/0a/1bcN rtnNy0enBWbGQ== Received: from mail-il1-x12a.google.com (mail-il1-x12a.google.com [IPv6:2607:f8b0:4864:20::12a]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id A2AC562720 for ; Mon, 27 Mar 2023 14:20:47 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="V43C2Lus"; dkim-atps=neutral Received: by mail-il1-x12a.google.com with SMTP id h11so4512733ild.11 for ; Mon, 27 Mar 2023 05:20:47 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; t=1679919647; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=FKREP6jJa308Taqx9N+sEobA15WmVJReEHkapGWA6lw=; b=V43C2LusEWI/Ci8v5eK6nkNsOWY/tnnwnl0wWIJ7XJne/uQibdIQG74uAmOE1DFF4+ 0IhZimiO2eF6tHsca8BcYjVj29437wykbr2MxcFAhyfoDgdFz/xp9aEAmKHDlbIGbaav EapabiebyTGxmj1X/PvnsyYp2iINppPUgei98x1yuERApBrT3ohoA8J49qBlsMV0vaBa eQpnK/SLtbr9AoJCdQtK6cl3zkTCWoKXn5DtfkIMZVuol++veg7gyWNEqpU8QJlPrlyL IytfJ73mjcW2L4Q2fuemPfN0T06X8AvNEHybzktdrTqeX+K6PXnNXVQ/Ae21OPX7dLNo JAiQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; t=1679919647; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=FKREP6jJa308Taqx9N+sEobA15WmVJReEHkapGWA6lw=; b=P612gSLouQJiQAPQ/8vnaGzMgpKvI+aXPXoOi6JOYbJ+CeFOrCgxWOgJvy/PQ1efX6 zEgyRqxNzm7tk7WmX+ln3CWX0cyVMz91+yANnCHEe6A1vJAdy7rYvBcRhBetiCHKLQA0 yGMuB4jLyivtc838nX+Jn22+h60DHJUBxF/m8yum+mmXYbcjxsVxg7dCoz18PVQRdRyX W3IlCoE8ce6gIj5QtmZRqEa6mhDnk/l9611zahZmTt8KH1znNFpGPneg9fAyMKl//1KT nD6Ty0JnGi84nT2a9BlrHwjd2Q40l9G7ENBeDQVoLh8FggSEf+s0e7ObrDVniip0SaDW DTsQ== X-Gm-Message-State: AAQBX9cuyqXXz67/qeCPzM4zzGFzIc/h0deVQUOf6kUCm02RXAz+exYC MfmqIaO+2yagMONuXUnbA3xgEh2Z5Keh8yZfJNjHMg== X-Google-Smtp-Source: AKy350YvtuZSR5nTWWQFOvLSKWdSSZ7lWa3+cNOMUNy15Bh0uqjAVjiAFS1fYBBVqxFrAz8OAH2lxw== X-Received: by 2002:a92:da11:0:b0:325:b002:89b4 with SMTP id z17-20020a92da11000000b00325b00289b4mr9008042ilm.25.1679919646985; Mon, 27 Mar 2023 05:20:46 -0700 (PDT) Received: from localhost.localdomain ([93.93.133.154]) by smtp.gmail.com with ESMTPSA id r7-20020a5edb47000000b00758ff97aaedsm3239026iop.16.2023.03.27.05.20.46 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 27 Mar 2023 05:20:46 -0700 (PDT) To: libcamera-devel@lists.libcamera.org Date: Mon, 27 Mar 2023 13:20:26 +0100 Message-Id: <20230327122030.11756-7-naush@raspberrypi.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230327122030.11756-1-naush@raspberrypi.com> References: <20230327122030.11756-1-naush@raspberrypi.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v2 06/10] ipa: raspberrypi: Generalise the contrast algorithm X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Naushir Patuck via libcamera-devel From: Naushir Patuck Reply-To: Naushir Patuck Cc: Jacopo Mondi Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Generalise the contrast algorithm code by removing any hard-coded assumptions about the target hardware platform. Instead, the algorithm code creates a generic Pwl that gets returned to the IPA, where it gets converted to the bcm2835 hardware specific lookup table. As a drive-by, remove an unused mutex. Signed-off-by: Naushir Patuck Reviewed-by: David Plowman Reviewed-by: Jacopo Mondi --- .../raspberrypi/controller/contrast_status.h | 11 ++----- .../raspberrypi/controller/rpi/contrast.cpp | 30 ++++--------------- src/ipa/raspberrypi/controller/rpi/contrast.h | 1 - src/ipa/raspberrypi/raspberrypi.cpp | 15 +++++++--- 4 files changed, 20 insertions(+), 37 deletions(-) diff --git a/src/ipa/raspberrypi/controller/contrast_status.h b/src/ipa/raspberrypi/controller/contrast_status.h index ef2a7c680fc2..fb9fe4bace71 100644 --- a/src/ipa/raspberrypi/controller/contrast_status.h +++ b/src/ipa/raspberrypi/controller/contrast_status.h @@ -6,20 +6,15 @@ */ #pragma once +#include "pwl.h" + /* * The "contrast" algorithm creates a gamma curve, optionally doing a little bit * of contrast stretching based on the AGC histogram. */ -constexpr unsigned int ContrastNumPoints = 33; - -struct ContrastPoint { - uint16_t x; - uint16_t y; -}; - struct ContrastStatus { - struct ContrastPoint points[ContrastNumPoints]; + RPiController::Pwl gammaCurve; double brightness; double contrast; }; diff --git a/src/ipa/raspberrypi/controller/rpi/contrast.cpp b/src/ipa/raspberrypi/controller/rpi/contrast.cpp index a4f8c4f04fc4..bee1eadd3657 100644 --- a/src/ipa/raspberrypi/controller/rpi/contrast.cpp +++ b/src/ipa/raspberrypi/controller/rpi/contrast.cpp @@ -65,34 +65,19 @@ void Contrast::setContrast(double contrast) contrast_ = contrast; } -static void fillInStatus(ContrastStatus &status, double brightness, - double contrast, Pwl &gammaCurve) -{ - status.brightness = brightness; - status.contrast = contrast; - for (unsigned int i = 0; i < ContrastNumPoints - 1; i++) { - int x = i < 16 ? i * 1024 - : (i < 24 ? (i - 16) * 2048 + 16384 - : (i - 24) * 4096 + 32768); - status.points[i].x = x; - status.points[i].y = std::min(65535.0, gammaCurve.eval(x)); - } - status.points[ContrastNumPoints - 1].x = 65535; - status.points[ContrastNumPoints - 1].y = 65535; -} - void Contrast::initialise() { /* * Fill in some default values as Prepare will run before Process gets * called. */ - fillInStatus(status_, brightness_, contrast_, config_.gammaCurve); + status_.brightness = brightness_; + status_.contrast = contrast_; + status_.gammaCurve = config_.gammaCurve; } void Contrast::prepare(Metadata *imageMetadata) { - std::unique_lock lock(mutex_); imageMetadata->set("contrast.status", status_); } @@ -183,12 +168,9 @@ void Contrast::process(StatisticsPtr &stats, * And fill in the status for output. Use more points towards the bottom * of the curve. */ - ContrastStatus status; - fillInStatus(status, brightness_, contrast_, gammaCurve); - { - std::unique_lock lock(mutex_); - status_ = status; - } + status_.brightness = brightness_; + status_.contrast = contrast_; + status_.gammaCurve = std::move(gammaCurve); } /* Register algorithm with the system. */ diff --git a/src/ipa/raspberrypi/controller/rpi/contrast.h b/src/ipa/raspberrypi/controller/rpi/contrast.h index c68adbc9e463..9c81277a0450 100644 --- a/src/ipa/raspberrypi/controller/rpi/contrast.h +++ b/src/ipa/raspberrypi/controller/rpi/contrast.h @@ -46,7 +46,6 @@ private: double brightness_; double contrast_; ContrastStatus status_; - std::mutex mutex_; }; } /* namespace RPiController */ diff --git a/src/ipa/raspberrypi/raspberrypi.cpp b/src/ipa/raspberrypi/raspberrypi.cpp index 257d862e07b6..d813b1b7ca61 100644 --- a/src/ipa/raspberrypi/raspberrypi.cpp +++ b/src/ipa/raspberrypi/raspberrypi.cpp @@ -1592,14 +1592,21 @@ void IPARPi::applyCCM(const struct CcmStatus *ccmStatus, ControlList &ctrls) void IPARPi::applyGamma(const struct ContrastStatus *contrastStatus, ControlList &ctrls) { + const unsigned int numGammaPoints = controller_.getHardwareConfig().numGammaPoints; struct bcm2835_isp_gamma gamma; - gamma.enabled = 1; - for (unsigned int i = 0; i < ContrastNumPoints; i++) { - gamma.x[i] = contrastStatus->points[i].x; - gamma.y[i] = contrastStatus->points[i].y; + for (unsigned int i = 0; i < numGammaPoints - 1; i++) { + int x = i < 16 ? i * 1024 + : (i < 24 ? (i - 16) * 2048 + 16384 + : (i - 24) * 4096 + 32768); + gamma.x[i] = x; + gamma.y[i] = std::min(65535, contrastStatus->gammaCurve.eval(x)); } + gamma.x[numGammaPoints - 1] = 65535; + gamma.y[numGammaPoints - 1] = 65535; + gamma.enabled = 1; + ControlValue c(Span{ reinterpret_cast(&gamma), sizeof(gamma) }); ctrls.set(V4L2_CID_USER_BCM2835_ISP_GAMMA, c); From patchwork Mon Mar 27 12:20:27 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Naushir Patuck X-Patchwork-Id: 18476 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 5B16FC329E for ; Mon, 27 Mar 2023 12:20:53 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id CB3C66276C; Mon, 27 Mar 2023 14:20:52 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1679919652; bh=aRIc5B+sTvytGlTffgEhxKt6BpAyCyw6+uqLU6iJiI8=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To:Cc: From; b=aecVvSP1fy0q90dGkWRZqgowL08tb3zfrATDhv6LyGEX1CItXHkLdZVUpgxQxqyjM ImiJMaM77Uhe9/wMSRqr/+oktW+TUu448c1O1OO0pxE6+tiEgYbmFwIbbMJiCsFC0f kteZ1YX6zrMWaQBsN3GTPB+QAc5HcJhiQUL7FfKxtvv8lAJqpo1mTMumB0jM2tL59/ QZ0h+o/0mzLvyDqDznfc1MUMXLzdVrpETMIqHWWoWKs+UQXK9H+c0SwlV2cKZWdmmt 4CKaBmGzNG1YGBcdcXO8NiTYkZpNPteSFIwnedCmW5FfRZpkmNC3LoNvdAgKbcXNdh Cd4nIlYbxM5Gg== Received: from mail-io1-xd30.google.com (mail-io1-xd30.google.com [IPv6:2607:f8b0:4864:20::d30]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id BB8B262747 for ; Mon, 27 Mar 2023 14:20:49 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="TOC0g+Cx"; dkim-atps=neutral Received: by mail-io1-xd30.google.com with SMTP id d14so3784938ion.9 for ; Mon, 27 Mar 2023 05:20:49 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; t=1679919648; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=quqiaExFAgBviDMcf8Asfps18bCBmhdSNFsVml+XS2A=; b=TOC0g+Cx+txP667AvOXek9H2l0zNgVOXocOqKnqk+/aGVgR+A1lRJykNJ5aFkB76Yt 6vZq8vEC9Coufs+xOJ/UPV007HNEN8q2aY1Pq37p9tLs1mOMmvBDUfmq8wMN0XPhGjmC KPLt+mJbB43RrtY7mCu7VHh6Do8kB5+oNw0N+9UMVikylgaxEBBAESq57P6T2horyVU7 pqv2jJ1/XHUtLaoA4wIehyY8PfkbC8GG42SAlpQcyRt6YhpztERZlE1lPTs/Z0gF0R5w thuokgaBasIfq1GaZ/i6WaFXZyzugquY+WtBegB+1gmherni8S9uogNN9Xpc9mNiYN6G smbw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; t=1679919648; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=quqiaExFAgBviDMcf8Asfps18bCBmhdSNFsVml+XS2A=; b=5VR9XWFbcUyks+qEaQmta5+wz++f//4oWBk+B6VRip2MEuTzRMi5Xju2sP8ukqJsiP bMn2TlgmMHaHFaDjFyNbt8DoxGwrnHxYypT8mOiakhkrVUFWOCAruDIQTlthgmIGdwd4 pVH34uZPWcxM/tOSyl54wOSxEi2elWbjqtg4AdVDJdsFVtQMEV0lzB8/4G+w5uVPM9oJ e1qcJCES+3u7pxPiMxySAfILkTtkJR7jqXWXYvwqopXNd3ISKG9zZVFPQneRZHM4t7q8 G4/1w7b0oUxnPhS+awZq+RR8X2nz9tjkX81ZPFUPToCD1vl+4C4dZ3/hbYv+EmY1TWJI IEGQ== X-Gm-Message-State: AO0yUKXAyn6TCbEoKDACQRb9UihtB6ZsPRpZTXR1H7ZatVNCBW/XM30o zaAq/JQZoYuFuhtDiyoK0scWY7iJzVduDVcmB6OxoQ== X-Google-Smtp-Source: AK7set/TOoRFyhzghmmSKw1JTHBYKblB+YgAfN+a3qpPxG6C0bdn7KVz68/Aw1OWVU0JC6ZQoI34Wg== X-Received: by 2002:a6b:a1a:0:b0:759:1e9e:6daa with SMTP id z26-20020a6b0a1a000000b007591e9e6daamr9091771ioi.10.1679919648352; Mon, 27 Mar 2023 05:20:48 -0700 (PDT) Received: from localhost.localdomain ([93.93.133.154]) by smtp.gmail.com with ESMTPSA id r7-20020a5edb47000000b00758ff97aaedsm3239026iop.16.2023.03.27.05.20.47 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 27 Mar 2023 05:20:47 -0700 (PDT) To: libcamera-devel@lists.libcamera.org Date: Mon, 27 Mar 2023 13:20:27 +0100 Message-Id: <20230327122030.11756-8-naush@raspberrypi.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230327122030.11756-1-naush@raspberrypi.com> References: <20230327122030.11756-1-naush@raspberrypi.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v2 07/10] ipa: raspberrypi: Make RegionStats::get() always return a Region struct X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Naushir Patuck via libcamera-devel From: Naushir Patuck Reply-To: Naushir Patuck Cc: Jacopo Mondi Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Update the overloaded RegionStats::get() and RegionStats::getFloating() member functions to return a Region struct for consistency. Signed-off-by: Naushir Patuck Reviewed-by: David Plowman Reviewed-by: Jacopo Mondi --- src/ipa/raspberrypi/controller/region_stats.h | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/src/ipa/raspberrypi/controller/region_stats.h b/src/ipa/raspberrypi/controller/region_stats.h index 9aaf3a58a6f7..a8860dc8dba2 100644 --- a/src/ipa/raspberrypi/controller/region_stats.h +++ b/src/ipa/raspberrypi/controller/region_stats.h @@ -86,12 +86,12 @@ public: return get_(index); } - const T &get(const libcamera::Point &pos) const + const Region &get(const libcamera::Point &pos) const { return get(pos.y * size_.width + pos.x); } - const T &getFloating(unsigned int index) const + const Region &getFloating(unsigned int index) const { if (index >= numFloatingRegions()) return default_; From patchwork Mon Mar 27 12:20:28 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Naushir Patuck X-Patchwork-Id: 18477 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 3028AC329C for ; Mon, 27 Mar 2023 12:20:54 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id D67F66276E; Mon, 27 Mar 2023 14:20:53 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1679919653; bh=ikDcOL4X/wpeiEwyev6qGchPa+D+ecIaiLLxK93NTDI=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To:Cc: From; b=sGTK48dfDLaLXEEjQglwNhSo+ivZ63VIVhFSZ5t2Sh/zV3ZEwYq18ZMXgKZ41DJ0+ etDPZhylMiV17O5jf0f/wS8uJxLAz5Zpo4jPn1gwVTnzRwoN/+rIZb2JRnmYFdrGQA Zqd8sTSyCOH9V0oR9DjVsv6+LdBYNEvoQJ4cJQkGt49/xFux9ZcTW/iVoJnRn70UI1 9itESjgjR2Wiw8Ai05ZEi9nHgdaRm4UthvtqJVjfMb0aD5dlCXR4wdvuiXoIBXldcg mB/ir6wK3+v0IjZ2HVXVeiSj/b9YlKhkSoTQS7ub5DV1sYFdAXzDfcx/lfynD2wZD7 kBiQ1kB4A0Rlg== Received: from mail-il1-x12a.google.com (mail-il1-x12a.google.com [IPv6:2607:f8b0:4864:20::12a]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id F2B9A62758 for ; Mon, 27 Mar 2023 14:20:49 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="ZS6sZh7T"; dkim-atps=neutral Received: by mail-il1-x12a.google.com with SMTP id h11so4512798ild.11 for ; Mon, 27 Mar 2023 05:20:49 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; t=1679919649; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=jFO9Xj1fKv8w1hxje+taVYe/vtFu0agkjxfPZsxmJQQ=; b=ZS6sZh7Trtz78CXj32JqqZy1CHyYojL6fNy1IoyWNykhnKjtx4RGYxebk+uTqF/5s0 AZO4C1C1RNNRcuxqwUf1nezN24gX16Nx5SSAzx/cQKiz1v12/Gz5V2ZafRgbM9/AKZRy SEiSpvGy8Eh/f55/No0JKJeNxepoOV9+3hxMgsB9SxT06K1fRtvQ74k/Jy73ml+uWgJ8 xmLOaVTF74yrTcb6pnyi36+/XKD3cC8B/jMOjEcHLQoihhsEaqxE2XeqSZQmn+3HOEcc PUsTfVdodESPKu7plLFio7PEy6QLS+7zv+q1xnSa3zVOLPOF/mvnOylF9UZ3foA/d5FQ 6l0Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; t=1679919649; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=jFO9Xj1fKv8w1hxje+taVYe/vtFu0agkjxfPZsxmJQQ=; b=UWBaj9bOF0Dtv/G2nu5YSjBOpK6iCnSohNEHZXzhN3bcEcq/sDKROxkxtiWpx3eSKb JxENYWnfTj6cJAWqFJfWLzly3GLBWO5FRgY0EIMaqWoFEFJgn5KLWimS61Cks/x+znDn wLLFR2fDHv8DDqvEMuQmy69REA4RdpdDwaqxUrpYqtMSY5VnmbkJueII6dO+pUVe+gXv tmvN4oXdX3CnlUNktJmi0GVP5QVt3cohHBCaSYj/d8CaWr3C+y7Yn/6B1cpb3BVxaeBw LaViJHZgXCbbaCeYJgJOyccezoiPDLpNleqSxDK8AcqccRa4f6laiR72qSu1yZoPUJD8 uJVw== X-Gm-Message-State: AAQBX9cpNqnei3Gti8qtX18UPs9e55MTbKMbwSq9wVH6NY4L7Kp5VMyK ZvGM5cj2V0lBQ7aJXx0F6PZ2uTTTXXDOvFflx8SaCQ== X-Google-Smtp-Source: AKy350bpm90pMeoJsaLV9ynHorlNTwk7/PJxoajuKhuWZksJf7Mu9O7ve/wSByUCm3LuQs22NYfvvw== X-Received: by 2002:a05:6e02:811:b0:315:365d:5358 with SMTP id u17-20020a056e02081100b00315365d5358mr9665395ilm.10.1679919649425; Mon, 27 Mar 2023 05:20:49 -0700 (PDT) Received: from localhost.localdomain ([93.93.133.154]) by smtp.gmail.com with ESMTPSA id r7-20020a5edb47000000b00758ff97aaedsm3239026iop.16.2023.03.27.05.20.48 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 27 Mar 2023 05:20:49 -0700 (PDT) To: libcamera-devel@lists.libcamera.org Date: Mon, 27 Mar 2023 13:20:28 +0100 Message-Id: <20230327122030.11756-9-naush@raspberrypi.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230327122030.11756-1-naush@raspberrypi.com> References: <20230327122030.11756-1-naush@raspberrypi.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v2 08/10] ipa: raspberrypi: Generalise the agc algorithm X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Naushir Patuck via libcamera-devel From: Naushir Patuck Reply-To: Naushir Patuck Cc: Jacopo Mondi Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Remove any hard-coded assumptions about the target hardware platform from the AGC algorithm. Instead, use the "target" string provided by the camera tuning config and generalised statistics structures to determing parameters such as grid and region sizes. This change replaces all hard-coded arrays with equivalent std::vector types. Signed-off-by: Naushir Patuck Reviewed-by: Jacopo Mondi Reviewed-by: Kieran Bingham --- src/ipa/raspberrypi/controller/rpi/agc.cpp | 20 +++++++++++++------- src/ipa/raspberrypi/controller/rpi/agc.h | 9 +-------- 2 files changed, 14 insertions(+), 15 deletions(-) diff --git a/src/ipa/raspberrypi/controller/rpi/agc.cpp b/src/ipa/raspberrypi/controller/rpi/agc.cpp index 4ea0dd41e66c..fb96bb556e84 100644 --- a/src/ipa/raspberrypi/controller/rpi/agc.cpp +++ b/src/ipa/raspberrypi/controller/rpi/agc.cpp @@ -31,17 +31,12 @@ LOG_DEFINE_CATEGORY(RPiAgc) int AgcMeteringMode::read(const libcamera::YamlObject ¶ms) { const YamlObject &yamlWeights = params["weights"]; - if (yamlWeights.size() != AgcStatsSize) { - LOG(RPiAgc, Error) << "AgcMeteringMode: Incorrect number of weights"; - return -EINVAL; - } - unsigned int num = 0; for (const auto &p : yamlWeights.asList()) { auto value = p.get(); if (!value) return -EINVAL; - weights[num++] = *value; + weights.push_back(*value); } return 0; @@ -249,6 +244,14 @@ int Agc::read(const libcamera::YamlObject ¶ms) if (ret) return ret; + const Size &size = getHardwareConfig().agcZoneWeights; + for (auto const &modes : config_.meteringModes) { + if (modes.second.weights.size() != size.width * size.height) { + LOG(RPiAgc, Error) << "AgcMeteringMode: Incorrect number of weights"; + return -EINVAL; + } + } + /* * Set the config's defaults (which are the first ones it read) as our * current modes, until someone changes them. (they're all known to @@ -582,9 +585,12 @@ void Agc::fetchAwbStatus(Metadata *imageMetadata) } static double computeInitialY(StatisticsPtr &stats, AwbStatus const &awb, - double weights[], double gain) + std::vector &weights, double gain) { constexpr uint64_t maxVal = 1 << Statistics::NormalisationFactorPow2; + + ASSERT(weights.size() == stats->agcRegions.numRegions()); + /* * Note how the calculation below means that equal weights give you * "average" metering (i.e. all pixels equally important). diff --git a/src/ipa/raspberrypi/controller/rpi/agc.h b/src/ipa/raspberrypi/controller/rpi/agc.h index f04896ca25ad..d11a49c9cd85 100644 --- a/src/ipa/raspberrypi/controller/rpi/agc.h +++ b/src/ipa/raspberrypi/controller/rpi/agc.h @@ -17,17 +17,10 @@ /* This is our implementation of AGC. */ -/* - * This is the number actually set up by the firmware, not the maximum possible - * number (which is 16). - */ - -constexpr unsigned int AgcStatsSize = 15; - namespace RPiController { struct AgcMeteringMode { - double weights[AgcStatsSize]; + std::vector weights; int read(const libcamera::YamlObject ¶ms); }; From patchwork Mon Mar 27 12:20:29 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Naushir Patuck X-Patchwork-Id: 18478 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 8FCF3C329F for ; Mon, 27 Mar 2023 12:20:55 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 4783D62752; Mon, 27 Mar 2023 14:20:55 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1679919655; bh=yr0ktkwDyN3/Tu/8D/ANe+ofHGDWo4sO1gP6XkDzX0Q=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To:Cc: From; b=r4Irz/ox6EDLKqNFZC3M9napDfYeuCBgqR7Zp006bTk8s+T4B6UgbAMTl5Au9eI3X 23SKBujuqwwU24jDX2MoC/VVAVxwJK0SkNGd8osEkrbCIN6RE0KSM7mjmFCzlxIPgT Aft3NeaEe8ZCxGbfPDA5yGYctrhx0sjymfMF6ImS3HwMcTPStvisN+0L5M5yO7UBsG VnIC7GBKO5/JcJethMzekMeAAVD9loeMF2gmI+GQhTS8CnYeCmWLc6HA6fKtBW8X9h cBlIqfNeI5Z9BwJWlkr3Kp9D98b0PnwbcswBmvUaOEVx5Xee9rCClFglO+a/XQXs+Y qsUOu7aUlyopg== Received: from mail-io1-xd30.google.com (mail-io1-xd30.google.com [IPv6:2607:f8b0:4864:20::d30]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 5F92762763 for ; Mon, 27 Mar 2023 14:20:51 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="qJAYLiUt"; dkim-atps=neutral Received: by mail-io1-xd30.google.com with SMTP id d14so3784981ion.9 for ; Mon, 27 Mar 2023 05:20:51 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; t=1679919651; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=F00/QMQd7UwuEW037ja5uCEEW7iXoeIOE6r3lrYMnjg=; b=qJAYLiUtzuKwbluPYRZ2Q51iLVRwnh6ejp9r6En1jTwdRxUDXz7vj4isz/qsjnJNBM 3Zt8G9t5vm0qvqsAS9IdbTVN0LxkTVFBxJzOoqkBGLqOSQVHekWVcxZTZp/jw+iqI+cw 8stpDZvUdEcERWHIRc6/qoYGyqJZNiNrjpJPTHZblVnWwgEXiN+i/++2Ev6eOuL+YMMV OtjYDYWZyj6yTt24Jj2uPQkhMAUrtk1DqqcB1tH7EtMjemNdRIfaBLaCRIQpf6mnpf8d FG/u4u63IvvFFzz6Z83j/mtN/++2Nty5GyjZxpfeFEv3gQ6qTsQJ9pcfkO+Xom4l74jV No4A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; t=1679919651; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=F00/QMQd7UwuEW037ja5uCEEW7iXoeIOE6r3lrYMnjg=; b=KMFyVTX+Mh9eGER0g7JRwlFqNrcD8traPnxQWIj6w+nWYkE3HTmSA7v1oZR6H42xRh 9/QTxmhEfSU9cIysZOHVpLGVk+N6is3zToRmw2cmeVINdMg1lTyDkL36ztMGIEdtW03X 9XzT37dPCKePIc6UAC9sh8GZd84UWC/ELyy8yyeKNO1/YT/mPvWElFwPqjv1s0n0e5S/ eYtrMU8pTx8Ej5Qw7gBEoKzudOu0GOPXCght+u9XoVNsHugI3J7DMcX/tKpKRi7iU058 r12XvyIdim9FKzVEXA34UiSd8DxQpXaPoJltS2qYWOSWnvTyny4UtSASmP/9oS1szqEh V0hQ== X-Gm-Message-State: AO0yUKWDNsVOlqjB+wKZTWcA2AMgXtJCkk53esWAFV2glp6OfSvbPan6 /kq8uNBmtn36vnOKaw5XypNIcF8TyWGw8VoRKGiUrg== X-Google-Smtp-Source: AK7set8m2QwvI0YscrB6EHp+2DHTC+xNtDUfgLp69wMcVEcFCO/xsKsdclTNfNEB6skv4DM8k199eg== X-Received: by 2002:a6b:f117:0:b0:758:dba1:471c with SMTP id e23-20020a6bf117000000b00758dba1471cmr8876800iog.16.1679919650581; Mon, 27 Mar 2023 05:20:50 -0700 (PDT) Received: from localhost.localdomain ([93.93.133.154]) by smtp.gmail.com with ESMTPSA id r7-20020a5edb47000000b00758ff97aaedsm3239026iop.16.2023.03.27.05.20.49 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 27 Mar 2023 05:20:50 -0700 (PDT) To: libcamera-devel@lists.libcamera.org Date: Mon, 27 Mar 2023 13:20:29 +0100 Message-Id: <20230327122030.11756-10-naush@raspberrypi.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230327122030.11756-1-naush@raspberrypi.com> References: <20230327122030.11756-1-naush@raspberrypi.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v2 09/10] ipa: raspberrypi: Generalise the autofocus algorithm X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Naushir Patuck via libcamera-devel From: Naushir Patuck Reply-To: Naushir Patuck Cc: Nick Hollinghurst Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" From: Nick Hollinghurst Remove any hard-coded assumptions about the target hardware platform from the autofocus algorithm. Instead, use the "target" string provided by the camera tuning config and generalised statistics structures to determing parameters such as grid and region sizes. Additionally, PDAF statistics are represented by a generalised region statistics structure to be device agnostic. These changes also require the autofocus algorithm to initialise region weights on the first frame's prepare()/process() call rather than during initialisation. Signed-off-by: Nick Hollinghurst Signed-off-by: Naushir Patuck Tested-by: Naushir Patuck Reviewed-by: Jacopo Mondi --- src/ipa/raspberrypi/cam_helper_imx708.cpp | 23 ++- src/ipa/raspberrypi/controller/pdaf_data.h | 21 +-- src/ipa/raspberrypi/controller/rpi/af.cpp | 176 +++++++++++---------- src/ipa/raspberrypi/controller/rpi/af.h | 28 ++-- 4 files changed, 133 insertions(+), 115 deletions(-) diff --git a/src/ipa/raspberrypi/cam_helper_imx708.cpp b/src/ipa/raspberrypi/cam_helper_imx708.cpp index 1f213d3c0833..641ba18f4b9d 100644 --- a/src/ipa/raspberrypi/cam_helper_imx708.cpp +++ b/src/ipa/raspberrypi/cam_helper_imx708.cpp @@ -69,11 +69,14 @@ private: /* Largest long exposure scale factor given as a left shift on the frame length. */ static constexpr int longExposureShiftMax = 7; + static constexpr int pdafStatsRows = 12; + static constexpr int pdafStatsCols = 16; + void populateMetadata(const MdParser::RegisterMap ®isters, Metadata &metadata) const override; static bool parsePdafData(const uint8_t *ptr, size_t len, unsigned bpp, - PdafData &pdaf); + PdafRegions &pdaf); bool parseAEHist(const uint8_t *ptr, size_t len, unsigned bpp); void putAGCStatistics(StatisticsPtr stats); @@ -120,11 +123,11 @@ void CamHelperImx708::prepare(libcamera::Span buffer, Metadata &m size_t bytesPerLine = (mode_.width * mode_.bitdepth) >> 3; if (buffer.size() > 2 * bytesPerLine) { - PdafData pdaf; + PdafRegions pdaf; if (parsePdafData(&buffer[2 * bytesPerLine], buffer.size() - 2 * bytesPerLine, mode_.bitdepth, pdaf)) - metadata.set("pdaf.data", pdaf); + metadata.set("pdaf.regions", pdaf); } /* Parse AE-HIST data where present */ @@ -239,7 +242,7 @@ void CamHelperImx708::populateMetadata(const MdParser::RegisterMap ®isters, } bool CamHelperImx708::parsePdafData(const uint8_t *ptr, size_t len, - unsigned bpp, PdafData &pdaf) + unsigned bpp, PdafRegions &pdaf) { size_t step = bpp >> 1; /* bytes per PDAF grid entry */ @@ -248,13 +251,17 @@ bool CamHelperImx708::parsePdafData(const uint8_t *ptr, size_t len, return false; } + pdaf.init({ pdafStatsCols, pdafStatsRows }); + ptr += 2 * step; - for (unsigned i = 0; i < PDAF_DATA_ROWS; ++i) { - for (unsigned j = 0; j < PDAF_DATA_COLS; ++j) { + for (unsigned i = 0; i < pdafStatsRows; ++i) { + for (unsigned j = 0; j < pdafStatsCols; ++j) { unsigned c = (ptr[0] << 3) | (ptr[1] >> 5); int p = (((ptr[1] & 0x0F) - (ptr[1] & 0x10)) << 6) | (ptr[2] >> 2); - pdaf.conf[i][j] = c; - pdaf.phase[i][j] = c ? p : 0; + PdafData pdafData; + pdafData.conf = c; + pdafData.phase = c ? p : 0; + pdaf.set(libcamera::Point(j, i), { pdafData, 1, 0 }); ptr += step; } } diff --git a/src/ipa/raspberrypi/controller/pdaf_data.h b/src/ipa/raspberrypi/controller/pdaf_data.h index 03c00d72c0e8..ae6ab996ded0 100644 --- a/src/ipa/raspberrypi/controller/pdaf_data.h +++ b/src/ipa/raspberrypi/controller/pdaf_data.h @@ -2,20 +2,23 @@ /* * Copyright (C) 2022, Raspberry Pi Ltd * - * pdaf_data.h - PDAF Metadata; for now this is - * largely based on IMX708's PDAF "Type 1" output. + * pdaf_data.h - PDAF Metadata */ #pragma once #include -#define PDAF_DATA_ROWS 12 -#define PDAF_DATA_COLS 16 +#include "region_stats.h" -struct PdafData { - /* Confidence values, in raster order, in arbitrary units */ - uint16_t conf[PDAF_DATA_ROWS][PDAF_DATA_COLS]; +namespace RPiController { - /* Phase error, in raster order, in s11 Q4 format (S.6.4) */ - int16_t phase[PDAF_DATA_ROWS][PDAF_DATA_COLS]; +struct PdafData { + /* Confidence, in arbitrary units */ + uint16_t conf; + /* Phase error, in s16 Q4 format (S.11.4) */ + int16_t phase; }; + +using PdafRegions = RegionStats; + +}; // namespace RPiController diff --git a/src/ipa/raspberrypi/controller/rpi/af.cpp b/src/ipa/raspberrypi/controller/rpi/af.cpp index a623651875f2..ed0c8a94d064 100644 --- a/src/ipa/raspberrypi/controller/rpi/af.cpp +++ b/src/ipa/raspberrypi/controller/rpi/af.cpp @@ -174,9 +174,8 @@ Af::Af(Controller *controller) statsRegion_(0, 0, 0, 0), windows_(), useWindows_(false), - phaseWeights_{}, - contrastWeights_{}, - sumWeights_(0), + phaseWeights_(), + contrastWeights_(), scanState_(ScanState::Idle), initted_(false), ftarget_(-1.0), @@ -190,7 +189,15 @@ Af::Af(Controller *controller) scanData_(), reportState_(AfState::Idle) { - scanData_.reserve(24); + /* + * Reserve space for data, to reduce memory fragmentation. It's too early + * to query the size of the PDAF (from camera) and Contrast (from ISP) + * statistics, but these are plausible upper bounds. + */ + phaseWeights_.w.reserve(16 * 12); + contrastWeights_.w.reserve(getHardwareConfig().focusRegions.width * + getHardwareConfig().focusRegions.height); + scanData_.reserve(32); } Af::~Af() @@ -226,7 +233,7 @@ void Af::switchMode(CameraMode const &cameraMode, [[maybe_unused]] Metadata *met << statsRegion_.y << ',' << statsRegion_.width << ',' << statsRegion_.height; - computeWeights(); + invalidateWeights(); if (scanState_ >= ScanState::Coarse && scanState_ < ScanState::Settle) { /* @@ -239,111 +246,99 @@ void Af::switchMode(CameraMode const &cameraMode, [[maybe_unused]] Metadata *met skipCount_ = cfg_.skipFrames; } -void Af::computeWeights() +void Af::computeWeights(RegionWeights *wgts, unsigned rows, unsigned cols) { - constexpr int MaxCellWeight = 240 / (int)MaxWindows; + wgts->rows = rows; + wgts->cols = cols; + wgts->sum = 0; + wgts->w.resize(rows * cols); + std::fill(wgts->w.begin(), wgts->w.end(), 0); - sumWeights_ = 0; - for (int i = 0; i < PDAF_DATA_ROWS; ++i) - std::fill(phaseWeights_[i], phaseWeights_[i] + PDAF_DATA_COLS, 0); - - if (useWindows_ && - statsRegion_.width >= PDAF_DATA_COLS && statsRegion_.height >= PDAF_DATA_ROWS) { + if (rows > 0 && cols > 0 && useWindows_ && + statsRegion_.height >= rows && statsRegion_.width >= cols) { /* * Here we just merge all of the given windows, weighted by area. * \todo Perhaps a better approach might be to find the phase in each * window and choose either the closest or the highest-confidence one? - * - * Using mostly "int" arithmetic, because Rectangle has signed x, y + * Ensure weights sum to less than (1<<16). 46080 is a "round number" + * below 65536, for better rounding when window size is a simple + * fraction of image dimensions. */ - int cellH = (int)(statsRegion_.height / PDAF_DATA_ROWS); - int cellW = (int)(statsRegion_.width / PDAF_DATA_COLS); - int cellA = cellH * cellW; + const unsigned maxCellWeight = 46080u / (MaxWindows * rows * cols); + const unsigned cellH = statsRegion_.height / rows; + const unsigned cellW = statsRegion_.width / cols; + const unsigned cellA = cellH * cellW; for (auto &w : windows_) { - for (int i = 0; i < PDAF_DATA_ROWS; ++i) { - int y0 = std::max(statsRegion_.y + cellH * i, w.y); - int y1 = std::min(statsRegion_.y + cellH * (i + 1), w.y + (int)(w.height)); + for (unsigned r = 0; r < rows; ++r) { + int y0 = std::max(statsRegion_.y + (int)(cellH * r), w.y); + int y1 = std::min(statsRegion_.y + (int)(cellH * (r + 1)), + w.y + (int)(w.height)); if (y0 >= y1) continue; y1 -= y0; - for (int j = 0; j < PDAF_DATA_COLS; ++j) { - int x0 = std::max(statsRegion_.x + cellW * j, w.x); - int x1 = std::min(statsRegion_.x + cellW * (j + 1), w.x + (int)(w.width)); + for (unsigned c = 0; c < cols; ++c) { + int x0 = std::max(statsRegion_.x + (int)(cellW * c), w.x); + int x1 = std::min(statsRegion_.x + (int)(cellW * (c + 1)), + w.x + (int)(w.width)); if (x0 >= x1) continue; - int a = y1 * (x1 - x0); - a = (MaxCellWeight * a + cellA - 1) / cellA; - phaseWeights_[i][j] += a; - sumWeights_ += a; + unsigned a = y1 * (x1 - x0); + a = (maxCellWeight * a + cellA - 1) / cellA; + wgts->w[r * cols + c] += a; + wgts->sum += a; } } } } - if (sumWeights_ == 0) { - /* - * Default AF window is the middle 1/2 width of the middle 1/3 height - * since this maps nicely to both PDAF (16x12) and Focus (4x3) grids. - */ - for (int i = PDAF_DATA_ROWS / 3; i < 2 * PDAF_DATA_ROWS / 3; ++i) { - for (int j = PDAF_DATA_COLS / 4; j < 3 * PDAF_DATA_COLS / 4; ++j) { - phaseWeights_[i][j] = MaxCellWeight; - sumWeights_ += MaxCellWeight; + if (wgts->sum == 0) { + /* Default AF window is the middle 1/2 width of the middle 1/3 height */ + for (unsigned r = rows / 3; r < rows - rows / 3; ++r) { + for (unsigned c = cols / 4; c < cols - cols / 4; ++c) { + wgts->w[r * cols + c] = 1; + wgts->sum += 1; } } } +} - /* Scale from PDAF to Focus Statistics grid (which has fixed size 4x3) */ - constexpr int FocusStatsRows = 3; - constexpr int FocusStatsCols = 4; - static_assert(FOCUS_REGIONS == FocusStatsRows * FocusStatsCols); - static_assert(PDAF_DATA_ROWS % FocusStatsRows == 0); - static_assert(PDAF_DATA_COLS % FocusStatsCols == 0); - constexpr int YFactor = PDAF_DATA_ROWS / FocusStatsRows; - constexpr int XFactor = PDAF_DATA_COLS / FocusStatsCols; - - LOG(RPiAf, Debug) << "Recomputed weights:"; - for (int i = 0; i < FocusStatsRows; ++i) { - for (int j = 0; j < FocusStatsCols; ++j) { - unsigned w = 0; - for (int y = 0; y < YFactor; ++y) - for (int x = 0; x < XFactor; ++x) - w += phaseWeights_[YFactor * i + y][XFactor * j + x]; - contrastWeights_[FocusStatsCols * i + j] = w; - } - LOG(RPiAf, Debug) << " " - << contrastWeights_[FocusStatsCols * i + 0] << " " - << contrastWeights_[FocusStatsCols * i + 1] << " " - << contrastWeights_[FocusStatsCols * i + 2] << " " - << contrastWeights_[FocusStatsCols * i + 3]; - } +void Af::invalidateWeights() +{ + phaseWeights_.sum = 0; + contrastWeights_.sum = 0; } -bool Af::getPhase(PdafData const &data, double &phase, double &conf) const +bool Af::getPhase(PdafRegions const ®ions, double &phase, double &conf) { + libcamera::Size size = regions.size(); + if (size.height != phaseWeights_.rows || size.width != phaseWeights_.cols || + phaseWeights_.sum == 0) { + LOG(RPiAf, Debug) << "Recompute Phase weights " << size.width << 'x' << size.height; + computeWeights(&phaseWeights_, size.height, size.width); + } + uint32_t sumWc = 0; int64_t sumWcp = 0; - - for (unsigned i = 0; i < PDAF_DATA_ROWS; ++i) { - for (unsigned j = 0; j < PDAF_DATA_COLS; ++j) { - if (phaseWeights_[i][j]) { - uint32_t c = data.conf[i][j]; - if (c >= cfg_.confThresh) { - if (c > cfg_.confClip) - c = cfg_.confClip; - c -= (cfg_.confThresh >> 2); - sumWc += phaseWeights_[i][j] * c; - c -= (cfg_.confThresh >> 2); - sumWcp += phaseWeights_[i][j] * data.phase[i][j] * (int64_t)c; - } + for (unsigned i = 0; i < regions.numRegions(); ++i) { + unsigned w = phaseWeights_.w[i]; + if (w) { + const PdafData &data = regions.get(i).val; + unsigned c = data.conf; + if (c >= cfg_.confThresh) { + if (c > cfg_.confClip) + c = cfg_.confClip; + c -= (cfg_.confThresh >> 2); + sumWc += w * c; + c -= (cfg_.confThresh >> 2); + sumWcp += (int64_t)(w * c) * (int64_t)data.phase; } } } - if (0 < sumWeights_ && sumWeights_ <= sumWc) { + if (0 < phaseWeights_.sum && phaseWeights_.sum <= sumWc) { phase = (double)sumWcp / (double)sumWc; - conf = (double)sumWc / (double)sumWeights_; + conf = (double)sumWc / (double)phaseWeights_.sum; return true; } else { phase = 0.0; @@ -352,14 +347,21 @@ bool Af::getPhase(PdafData const &data, double &phase, double &conf) const } } -double Af::getContrast(const FocusRegions &focusStats) const +double Af::getContrast(const FocusRegions &focusStats) { - uint32_t sumWc = 0; + libcamera::Size size = focusStats.size(); + if (size.height != contrastWeights_.rows || + size.width != contrastWeights_.cols || contrastWeights_.sum == 0) { + LOG(RPiAf, Debug) << "Recompute Contrast weights " + << size.width << 'x' << size.height; + computeWeights(&contrastWeights_, size.height, size.width); + } + uint64_t sumWc = 0; for (unsigned i = 0; i < focusStats.numRegions(); ++i) - sumWc += contrastWeights_[i] * focusStats.get(i).val; + sumWc += contrastWeights_.w[i] * focusStats.get(i).val; - return (sumWeights_ == 0) ? 0.0 : (double)sumWc / (double)sumWeights_; + return (contrastWeights_.sum > 0) ? ((double)sumWc / (double)contrastWeights_.sum) : 0.0; } void Af::doPDAF(double phase, double conf) @@ -623,14 +625,14 @@ void Af::prepare(Metadata *imageMetadata) if (initted_) { /* Get PDAF from the embedded metadata, and run AF algorithm core */ - PdafData data; + PdafRegions regions; double phase = 0.0, conf = 0.0; double oldFt = ftarget_; double oldFs = fsmooth_; ScanState oldSs = scanState_; uint32_t oldSt = stepCount_; - if (imageMetadata->get("pdaf.data", data) == 0) - getPhase(data, phase, conf); + if (imageMetadata->get("pdaf.regions", regions) == 0) + getPhase(regions, phase, conf); doAF(prevContrast_, phase, conf); updateLensPosition(); LOG(RPiAf, Debug) << std::fixed << std::setprecision(2) @@ -691,7 +693,7 @@ void Af::setMetering(bool mode) { if (useWindows_ != mode) { useWindows_ = mode; - computeWeights(); + invalidateWeights(); } } @@ -708,7 +710,9 @@ void Af::setWindows(libcamera::Span const &wins) if (windows_.size() >= MaxWindows) break; } - computeWeights(); + + if (useWindows_) + invalidateWeights(); } bool Af::setLensPosition(double dioptres, int *hwpos) diff --git a/src/ipa/raspberrypi/controller/rpi/af.h b/src/ipa/raspberrypi/controller/rpi/af.h index 7959371baf64..b479feb88c39 100644 --- a/src/ipa/raspberrypi/controller/rpi/af.h +++ b/src/ipa/raspberrypi/controller/rpi/af.h @@ -11,12 +11,6 @@ #include "../pdaf_data.h" #include "../pwl.h" -/* - * \todo FOCUS_REGIONS is taken from bcm2835-isp.h, but should be made as a - * generic RegionStats structure. - */ -#define FOCUS_REGIONS 12 - /* * This algorithm implements a hybrid of CDAF and PDAF, favouring PDAF. * @@ -121,9 +115,20 @@ private: double conf; }; - void computeWeights(); - bool getPhase(PdafData const &data, double &phase, double &conf) const; - double getContrast(const FocusRegions &focusStats) const; + struct RegionWeights { + unsigned rows; + unsigned cols; + uint32_t sum; + std::vector w; + + RegionWeights() + : rows(0), cols(0), sum(0), w() {} + }; + + void computeWeights(RegionWeights *wgts, unsigned rows, unsigned cols); + void invalidateWeights(); + bool getPhase(PdafRegions const ®ions, double &phase, double &conf); + double getContrast(const FocusRegions &focusStats); void doPDAF(double phase, double conf); bool earlyTerminationByPhase(double phase); double findPeak(unsigned index) const; @@ -143,9 +148,8 @@ private: libcamera::Rectangle statsRegion_; std::vector windows_; bool useWindows_; - uint8_t phaseWeights_[PDAF_DATA_ROWS][PDAF_DATA_COLS]; - uint16_t contrastWeights_[FOCUS_REGIONS]; - uint32_t sumWeights_; + RegionWeights phaseWeights_; + RegionWeights contrastWeights_; /* Working state. */ ScanState scanState_; From patchwork Mon Mar 27 12:20:30 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Naushir Patuck X-Patchwork-Id: 18479 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 33779C32A0 for ; Mon, 27 Mar 2023 12:20:56 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id B75046278B; Mon, 27 Mar 2023 14:20:55 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1679919655; bh=I+GJeNia0yCq893YG/WPqw5ionNfPpft/qbqOyAt4ew=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To: From; b=g7vz9Ak86zExOVDQbpC9xJFM9ba3Vpwqk89u3d5ihbxNio3CeXleFmy0o7G6wswR2 h71XuRfRspZJPow0sCtyO9sC1lQJqCeBt5MCpaEmCmNZFkY/5oqcW7oLNhFLNuYNP2 zZ+2AwdVpgrPI34iDFp7xsNnRIGqSpVDyd7keuwNk46UaxHkfYrYNV4opa4XWMy3ib vkRZhMAgVoW+Cw5hM9OtSAgpwkxN0IUPF0mcHTemq7ICf5u59GaTGsHBNjEWgaVLKV x+xybbT869BJ20+OnYAx+89A5S1ju5bYw50o9Jnl+Xb3FEuVDxEbnC9Ff9Q3WmmIT6 Y5O9V+p0ltQbA== Received: from mail-io1-xd30.google.com (mail-io1-xd30.google.com [IPv6:2607:f8b0:4864:20::d30]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 5B78A6276A for ; Mon, 27 Mar 2023 14:20:52 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="P75iVobl"; dkim-atps=neutral Received: by mail-io1-xd30.google.com with SMTP id d14so3784998ion.9 for ; Mon, 27 Mar 2023 05:20:52 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; t=1679919652; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=gCzt2lkm41Vsp+5UNwKr7yvk6OMuYLjlo17eIpDSEa8=; b=P75iVoblhdiGehWMQ9j4cIUDqhkr/Kosn+7YFxV1vC6putKODLIHMEXjPE6zWw5564 gYA0IfrBlYlgTOtSqEYET3FJF6tnIBNde5VRLWKE1qJ7eY0ISz8+EADvce8YouQfIYli Ek0ntQjoD4rMz/heCzniZmm/bHMyrKCtJKTRSgJiPFQ05MkcBY1yv+DnshZoMzmiR3ML c+3MPjiGUuyfJi0l/lmKTJ899XMS/2PxBLlBZne3Z7wi9pSgxEgAmhcMXWUThj1w6k9K kJMeFxNvI21om8Y/Y2Egm+h7ov7n6UrC8piGfoY8+Yv3L02hhKRFfh/iIeykdbZisk9m uI6w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; t=1679919652; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=gCzt2lkm41Vsp+5UNwKr7yvk6OMuYLjlo17eIpDSEa8=; b=LTV5RaDA3aH/p1acsAuYH2ValbWpM46NmbC6S62zUIDjN+Xd09L/qMcGELS/ZTl4l8 Aw2d78OJTzpQYCcjDwDSRGFe8I5huFi8LE640APvtIynzmURwZVMn9CczFIezBjlP4He RbDQQxfPijmbTrkixXVtjYlcaf93C7cm2gE3KwydEgRQqPlBrKc1HFVTiKnCPC78XBNt eidTcOsz+8uGQOc5FuO1WAEhmgIXd4cF7N7vMrB6ETU0aMHG3F7xy6W+q18dX8MJImcv jcQIfmxfEBKZHS242YYMPG+mfJqsUoamwpSQeFUOQElBGvzjvSwmXnEXoxNDU4gUicxw aSSQ== X-Gm-Message-State: AO0yUKVFjbpnaZm6HmbSK1R3kLy8ovGZlxFcD+se+6sHiFxWgt+Ttyof TltqnFrX4JVpLw3ZywCbFYBlmTliwQ6M4HaN0VJHLQ== X-Google-Smtp-Source: AK7set/scNFAwKnbx7lLY9se+Sy9BXdj238sT+f02zzNL/sMn5Lw1z5F0cJmRuFPLRb/rMLg5qGfeg== X-Received: by 2002:a5d:9d8d:0:b0:753:2226:952f with SMTP id ay13-20020a5d9d8d000000b007532226952fmr13274248iob.0.1679919651660; Mon, 27 Mar 2023 05:20:51 -0700 (PDT) Received: from localhost.localdomain ([93.93.133.154]) by smtp.gmail.com with ESMTPSA id r7-20020a5edb47000000b00758ff97aaedsm3239026iop.16.2023.03.27.05.20.50 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 27 Mar 2023 05:20:51 -0700 (PDT) To: libcamera-devel@lists.libcamera.org Date: Mon, 27 Mar 2023 13:20:30 +0100 Message-Id: <20230327122030.11756-11-naush@raspberrypi.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230327122030.11756-1-naush@raspberrypi.com> References: <20230327122030.11756-1-naush@raspberrypi.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v2 10/10] ipa: raspberrypi: Remove the focus reporting algorithm X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Naushir Patuck via libcamera-devel From: Naushir Patuck Reply-To: Naushir Patuck Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Update the IPA focus reporting code to use the generalised statistics Region structure. Remove the focus reporting algorithm as the functionality is duplicated this bit of IPA code. Remove focus_status.h as it is no longer needed. Signed-off-by: Naushir Patuck --- src/ipa/raspberrypi/controller/focus_status.h | 20 -------- src/ipa/raspberrypi/controller/rpi/af.h | 1 - src/ipa/raspberrypi/controller/rpi/focus.cpp | 49 ------------------- src/ipa/raspberrypi/data/imx290.json | 3 -- src/ipa/raspberrypi/data/imx296.json | 3 -- src/ipa/raspberrypi/data/imx296_mono.json | 3 -- src/ipa/raspberrypi/data/imx477.json | 3 -- src/ipa/raspberrypi/data/imx477_noir.json | 3 -- .../raspberrypi/data/imx477_scientific.json | 3 -- src/ipa/raspberrypi/data/imx477_v1.json | 3 -- src/ipa/raspberrypi/data/imx708.json | 3 -- src/ipa/raspberrypi/data/imx708_noir.json | 3 -- src/ipa/raspberrypi/data/imx708_wide.json | 3 -- .../raspberrypi/data/imx708_wide_noir.json | 3 -- src/ipa/raspberrypi/meson.build | 1 - src/ipa/raspberrypi/raspberrypi.cpp | 16 ++++-- 16 files changed, 11 insertions(+), 109 deletions(-) delete mode 100644 src/ipa/raspberrypi/controller/focus_status.h delete mode 100644 src/ipa/raspberrypi/controller/rpi/focus.cpp diff --git a/src/ipa/raspberrypi/controller/focus_status.h b/src/ipa/raspberrypi/controller/focus_status.h deleted file mode 100644 index 8b74e59840c1..000000000000 --- a/src/ipa/raspberrypi/controller/focus_status.h +++ /dev/null @@ -1,20 +0,0 @@ -/* SPDX-License-Identifier: BSD-2-Clause */ -/* - * Copyright (C) 2020, Raspberry Pi Ltd - * - * focus_status.h - focus measurement status - */ -#pragma once - -#include - -/* - * The focus algorithm should post the following structure into the image's - * "focus.status" metadata. Recall that it's only reporting focus (contrast) - * measurements, it's not driving any kind of auto-focus algorithm! - */ - -struct FocusStatus { - unsigned int num; - uint32_t focusMeasures[FOCUS_REGIONS]; -}; diff --git a/src/ipa/raspberrypi/controller/rpi/af.h b/src/ipa/raspberrypi/controller/rpi/af.h index b479feb88c39..6d2bae671a22 100644 --- a/src/ipa/raspberrypi/controller/rpi/af.h +++ b/src/ipa/raspberrypi/controller/rpi/af.h @@ -28,7 +28,6 @@ * "nuisance" scans. During each interval where PDAF is not working, only * ONE scan will be performed; CAF cannot track objects using CDAF alone. * - * This algorithm is unrelated to "rpi.focus" which merely reports CDAF FoM. */ namespace RPiController { diff --git a/src/ipa/raspberrypi/controller/rpi/focus.cpp b/src/ipa/raspberrypi/controller/rpi/focus.cpp deleted file mode 100644 index ea3cc00e42c3..000000000000 --- a/src/ipa/raspberrypi/controller/rpi/focus.cpp +++ /dev/null @@ -1,49 +0,0 @@ -/* SPDX-License-Identifier: BSD-2-Clause */ -/* - * Copyright (C) 2020, Raspberry Pi Ltd - * - * focus.cpp - focus algorithm - */ -#include - -#include - -#include "../focus_status.h" -#include "focus.h" - -using namespace RPiController; -using namespace libcamera; - -LOG_DEFINE_CATEGORY(RPiFocus) - -#define NAME "rpi.focus" - -Focus::Focus(Controller *controller) - : Algorithm(controller) -{ -} - -char const *Focus::name() const -{ - return NAME; -} - -void Focus::process(StatisticsPtr &stats, Metadata *imageMetadata) -{ - FocusStatus status; - for (unsigned int i = 0; i < stats->focusRegions.numRegions(); i++) - status.focusMeasures[i] = stats->focusRegions.get(i).val; - status.num = stats->focusRegions.numRegions(); - imageMetadata->set("focus.status", status); - - LOG(RPiFocus, Debug) - << "Focus contrast measure: " - << (status.focusMeasures[5] + status.focusMeasures[6]) / 10; -} - -/* Register algorithm with the system. */ -static Algorithm *create(Controller *controller) -{ - return new Focus(controller); -} -static RegisterAlgorithm reg(NAME, &create); diff --git a/src/ipa/raspberrypi/data/imx290.json b/src/ipa/raspberrypi/data/imx290.json index bfb9c6093d26..ace68d0ebf1c 100644 --- a/src/ipa/raspberrypi/data/imx290.json +++ b/src/ipa/raspberrypi/data/imx290.json @@ -195,9 +195,6 @@ } ] } - }, - { - "rpi.focus": { } } ] } \ No newline at end of file diff --git a/src/ipa/raspberrypi/data/imx296.json b/src/ipa/raspberrypi/data/imx296.json index 346f5b658957..ae8722c46a85 100644 --- a/src/ipa/raspberrypi/data/imx296.json +++ b/src/ipa/raspberrypi/data/imx296.json @@ -532,9 +532,6 @@ "strength": 1.0, "limit": 0.18 } - }, - { - "rpi.focus": { } } ] } diff --git a/src/ipa/raspberrypi/data/imx296_mono.json b/src/ipa/raspberrypi/data/imx296_mono.json index e9fa30c6a08d..30965b4b8d61 100644 --- a/src/ipa/raspberrypi/data/imx296_mono.json +++ b/src/ipa/raspberrypi/data/imx296_mono.json @@ -228,9 +228,6 @@ "strength": 1.0, "limit": 0.18 } - }, - { - "rpi.focus": { } } ] } diff --git a/src/ipa/raspberrypi/data/imx477.json b/src/ipa/raspberrypi/data/imx477.json index bfc0774ffdfc..daffc268ff43 100644 --- a/src/ipa/raspberrypi/data/imx477.json +++ b/src/ipa/raspberrypi/data/imx477.json @@ -513,9 +513,6 @@ }, { "rpi.sharpen": { } - }, - { - "rpi.focus": { } } ] } \ No newline at end of file diff --git a/src/ipa/raspberrypi/data/imx477_noir.json b/src/ipa/raspberrypi/data/imx477_noir.json index dadd97bc3d78..52d7f072d0ce 100644 --- a/src/ipa/raspberrypi/data/imx477_noir.json +++ b/src/ipa/raspberrypi/data/imx477_noir.json @@ -424,9 +424,6 @@ }, { "rpi.sharpen": { } - }, - { - "rpi.focus": { } } ] } \ No newline at end of file diff --git a/src/ipa/raspberrypi/data/imx477_scientific.json b/src/ipa/raspberrypi/data/imx477_scientific.json index 17c4ed0a5e74..26c692fdbab9 100644 --- a/src/ipa/raspberrypi/data/imx477_scientific.json +++ b/src/ipa/raspberrypi/data/imx477_scientific.json @@ -474,9 +474,6 @@ }, { "rpi.sharpen": { } - }, - { - "rpi.focus": { } } ] } \ No newline at end of file diff --git a/src/ipa/raspberrypi/data/imx477_v1.json b/src/ipa/raspberrypi/data/imx477_v1.json index 5bcaac67069b..d64020091efa 100644 --- a/src/ipa/raspberrypi/data/imx477_v1.json +++ b/src/ipa/raspberrypi/data/imx477_v1.json @@ -511,9 +511,6 @@ }, { "rpi.sharpen": { } - }, - { - "rpi.focus": { } } ] } \ No newline at end of file diff --git a/src/ipa/raspberrypi/data/imx708.json b/src/ipa/raspberrypi/data/imx708.json index c38b2d4cf256..b9830a3bf692 100644 --- a/src/ipa/raspberrypi/data/imx708.json +++ b/src/ipa/raspberrypi/data/imx708.json @@ -512,9 +512,6 @@ { "rpi.sharpen": { } }, - { - "rpi.focus": { } - }, { "rpi.af": { diff --git a/src/ipa/raspberrypi/data/imx708_noir.json b/src/ipa/raspberrypi/data/imx708_noir.json index 082274e34c53..075f70355cec 100644 --- a/src/ipa/raspberrypi/data/imx708_noir.json +++ b/src/ipa/raspberrypi/data/imx708_noir.json @@ -512,9 +512,6 @@ { "rpi.sharpen": { } }, - { - "rpi.focus": { } - }, { "rpi.af": { diff --git a/src/ipa/raspberrypi/data/imx708_wide.json b/src/ipa/raspberrypi/data/imx708_wide.json index cdc61436995d..b772efee3b96 100644 --- a/src/ipa/raspberrypi/data/imx708_wide.json +++ b/src/ipa/raspberrypi/data/imx708_wide.json @@ -403,9 +403,6 @@ { "rpi.sharpen": { } }, - { - "rpi.focus": { } - }, { "rpi.af": { diff --git a/src/ipa/raspberrypi/data/imx708_wide_noir.json b/src/ipa/raspberrypi/data/imx708_wide_noir.json index 8a7f59910833..c5f6b53dca7a 100644 --- a/src/ipa/raspberrypi/data/imx708_wide_noir.json +++ b/src/ipa/raspberrypi/data/imx708_wide_noir.json @@ -403,9 +403,6 @@ { "rpi.sharpen": { } }, - { - "rpi.focus": { } - }, { "rpi.af": { diff --git a/src/ipa/raspberrypi/meson.build b/src/ipa/raspberrypi/meson.build index 9230e17bca22..de78cbd80f9c 100644 --- a/src/ipa/raspberrypi/meson.build +++ b/src/ipa/raspberrypi/meson.build @@ -33,7 +33,6 @@ rpi_ipa_sources = files([ 'controller/rpi/awb.cpp', 'controller/rpi/sharpen.cpp', 'controller/rpi/black_level.cpp', - 'controller/rpi/focus.cpp', 'controller/rpi/geq.cpp', 'controller/rpi/noise.cpp', 'controller/rpi/lux.cpp', diff --git a/src/ipa/raspberrypi/raspberrypi.cpp b/src/ipa/raspberrypi/raspberrypi.cpp index d813b1b7ca61..c7b9b7007ce9 100644 --- a/src/ipa/raspberrypi/raspberrypi.cpp +++ b/src/ipa/raspberrypi/raspberrypi.cpp @@ -50,7 +50,6 @@ #include "denoise_algorithm.h" #include "denoise_status.h" #include "dpc_status.h" -#include "focus_status.h" #include "geq_status.h" #include "lux_status.h" #include "metadata.h" @@ -640,14 +639,18 @@ void IPARPi::reportMetadata(unsigned int ipaContext) static_cast(blackLevelStatus->blackLevelG), static_cast(blackLevelStatus->blackLevelB) }); - FocusStatus *focusStatus = rpiMetadata.getLocked("focus.status"); - if (focusStatus && focusStatus->num == 12) { + RPiController::FocusRegions *focusStatus = + rpiMetadata.getLocked("focus.status"); + if (focusStatus) { + libcamera::Size size = focusStatus->size(); /* * We get a 4x3 grid of regions by default. Calculate the average * FoM over the central two positions to give an overall scene FoM. * This can change later if it is not deemed suitable. */ - int32_t focusFoM = (focusStatus->focusMeasures[5] + focusStatus->focusMeasures[6]) / 2; + ASSERT(size == Size(4, 3)); + int32_t focusFoM = (focusStatus->get({ 1, 1 }).val + + focusStatus->get({ 2, 1 }).val) / 2; libcameraMetadata_.set(controls::FocusFoM, focusFoM); } @@ -1426,7 +1429,6 @@ RPiController::StatisticsPtr IPARPi::fillStatistics(bcm2835_isp_stats *stats) co statistics->focusRegions.set(i, { stats->focus_stats[i].contrast_val[1][1] / 1000, stats->focus_stats[i].contrast_val_num[1][1], stats->focus_stats[i].contrast_val_num[1][0] }); - return statistics; } @@ -1443,6 +1445,10 @@ void IPARPi::processStats(unsigned int bufferId, unsigned int ipaContext) Span mem = it->second.planes()[0]; bcm2835_isp_stats *stats = reinterpret_cast(mem.data()); RPiController::StatisticsPtr statistics = fillStatistics(stats); + + /* Save the focus stats in the metadata structure to report out later. */ + rpiMetadata_[ipaContext].set("focus.status", statistics->focusRegions); + helper_->process(statistics, rpiMetadata); controller_.process(statistics, &rpiMetadata);