From patchwork Tue Nov 22 11:22:20 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Naushir Patuck X-Patchwork-Id: 17830 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 5E1A7BE08B for ; Tue, 22 Nov 2022 11:22:30 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 11081603CE; Tue, 22 Nov 2022 12:22:29 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1669116149; bh=YmRgADhS3s4kSwmcHgSScFX3tgmguW1qArHN7RPr/5I=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To: From; b=uW5DALLZOwCBfTk4Dknuq0AonVD3UIuCFFHhcwTJLYisg9zHN+yfq0GOZXds/AkHG OtYBJ2T+RjKrXQBmwkKvkc6uhfWfXbo9/nkv3T90O3Dz/bKSfDgESHufE+gvbUJC84 9NBtbthgE45RCCAt0zGsasbklMq2O811lCtz4nlzgRAlAiVi8Z2djfBIBEMRVvESn/ CY+6j6CpNaA8tHMQl1Z+GN7dS0+UecU2O3HqlQM6D38UiJI9U3AYgvc+MDjMB9y35h PUrswJnodIQDHOumRFczvH9orUldofnoRjAlhuoTSvx9gwLVLJ7d1RSKqby15Hef7N j5+MPwiH55Xdg== Received: from mail-wr1-x42b.google.com (mail-wr1-x42b.google.com [IPv6:2a00:1450:4864:20::42b]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 3551561F2B for ; Tue, 22 Nov 2022 12:22:28 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="phmvlgWa"; dkim-atps=neutral Received: by mail-wr1-x42b.google.com with SMTP id e11so11304781wru.8 for ; Tue, 22 Nov 2022 03:22:28 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=84I5pgUTPV442TRgUl4HrW7AgpbIpJKmlWRibV0h/I4=; b=phmvlgWah2hV1vjsEKp5ZfZlNGpLiaawYE1R9o4/MHB0p7E4+p22zWwVfrnfaLkmM4 PO8sHCdvg3n7aN50wk81AHSIJ4ShxhKFPrKlR63W+9ceuqzHILHupoKpO4DQo5PsYxxn 6P/MhoW4NLWBlrf4avbOHB3VzwvgHZXdKO6qR+CvrMDujYe31OuSUS1uByVYTQ7EwoMj 1rVftyaXqbF+0PUStO33EYHutT9Ijw4k3Vq4fkfmQ6IfpNBKWc5XIuoSAoOOUH4/f94N 403RepUHpEfzuhnoRlLOcOLB+goEG8dO+2zZz2vsxzJtzU23+lpfBrfZJIdubZVRMUyY fRdg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=84I5pgUTPV442TRgUl4HrW7AgpbIpJKmlWRibV0h/I4=; b=TICtatDBd4jc3jw48Uiyy+mMLPy+gRRquanLO9QhiOz1gZM2BRl1weykGDoSg+I7EC eG8xUlNsRb0mv17YMmCDhVJUo5mqT499b9a9Tu5btqJa8TOdKGqkqFr4JolMYyHKZjFt 4s87z6w2RD6c3fdQimond7qbwOVL/01zok7lIAg9ZYUFpXjXd9KpwRc1WWjRLh976UET Z4+tgwDRHXL8RnimjUBTK0koJDVDVBT1vLgb1cwG/yoyU4kqTKaD3osXfr4WuCWXFm4Z Jzfpl6j74jryTFKWDRF9uYQGBOS9NMpGyNE8nAkYuqSS2Ps9SgQLTdOjWUjx1JgV4b6+ 6z4A== X-Gm-Message-State: ANoB5pnQHKjC8CHaBWmnuZAQQWhMkLd24IRBGbBApGsxNdqNXiSTBARM dXiSYend9bmR0JWQBKh7kiULLlBSK4AWZQ== X-Google-Smtp-Source: AA0mqf6WeZtqpmhakrTBt5+bLFCcjeN3XcRd+2fXR4daE2NtufFNs3t6ZCEN0DGGSJOp3Efg7qz17w== X-Received: by 2002:adf:dc8b:0:b0:22e:2b7:f215 with SMTP id r11-20020adfdc8b000000b0022e02b7f215mr6731651wrj.512.1669116147541; Tue, 22 Nov 2022 03:22:27 -0800 (PST) Received: from naush-laptop.localdomain ([93.93.133.154]) by smtp.gmail.com with ESMTPSA id q125-20020a1c4383000000b003c6cd82596esm21400388wma.43.2022.11.22.03.22.26 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 22 Nov 2022 03:22:27 -0800 (PST) To: libcamera-devel@lists.libcamera.org Date: Tue, 22 Nov 2022 11:22:20 +0000 Message-Id: <20221122112224.31691-2-naush@raspberrypi.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20221122112224.31691-1-naush@raspberrypi.com> References: <20221122112224.31691-1-naush@raspberrypi.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v1 1/5] ipa: raspberrypi: Generalise statistics X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Naushir Patuck via libcamera-devel From: Naushir Patuck Reply-To: Naushir Patuck Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" At present, the controller algorithms access the bcm2835_isp_stats structure, which is hardware specific. It would be desirable to abstract out the statistics structure to remove hardware specific headers from the algorithms source files. Define a new templated RegionStats class that encompasses region based statistics generated by the ISP. For the VC4 ISP, this can be used to hold RGB sums and focus FoM values. Define a new Statistics structure that holds all the VC4 ISP statistics output. This includes AGC hostograms, AGC/AWB region sums and focus FoM regions. Signed-off-by: Naushir Patuck Reviewed-by: David Plowman --- src/ipa/raspberrypi/controller/region_stats.h | 109 ++++++++++++++++++ src/ipa/raspberrypi/statistics.h | 70 +++++++++++ 2 files changed, 179 insertions(+) create mode 100644 src/ipa/raspberrypi/controller/region_stats.h create mode 100644 src/ipa/raspberrypi/statistics.h diff --git a/src/ipa/raspberrypi/controller/region_stats.h b/src/ipa/raspberrypi/controller/region_stats.h new file mode 100644 index 000000000000..8a3453617654 --- /dev/null +++ b/src/ipa/raspberrypi/controller/region_stats.h @@ -0,0 +1,109 @@ +/* SPDX-License-Identifier: BSD-2-Clause */ +/* + * Copyright (C) 2022, Raspberry Pi Ltd + * + * region_stats.h - Raspberry Pi region based statistics container + */ +#pragma once + +#include +#include +#include + +namespace RPiController { + +template +class RegionStats +{ +public: + RegionStats() + : numRows_(0), numCols_(0), numFloating_(0) + { + } + + void init(unsigned int numRows, unsigned int numCols = 1, unsigned int numFloating = 0) + { + numRows_ = numRows; + numCols_ = numCols; + numFloating_ = numFloating; + regions_.resize(numRows_ * numCols_ + numFloating_); + } + + unsigned int numRegions() const + { + return numRows_ * numCols_; + } + + unsigned int numFloatingRegions() const + { + return numFloating_; + } + + void set(unsigned int index, const T &value, uint32_t counted, uint32_t uncounted) + { + if (index >= numRegions()) + return; + set_(index, value, counted, uncounted); + } + + void set(unsigned int row, unsigned int col, const T &value, uint32_t counted, uint32_t uncounted) + { + set(row * numCols_ + col, value, counted, uncounted); + } + + void setFloating(unsigned int index, const T &value, uint32_t counted, uint32_t uncounted) + { + if (index >= numFloatingRegions()) + return; + set(numRegions() + index, value, counted, uncounted); + } + + const T &get(unsigned int index, uint32_t &counted, uint32_t &uncounted) const + { + counted = uncounted = 0; + if (index >= numRegions()) + return default_; + return get_(index, counted, uncounted); + } + + const T &get(unsigned int row, unsigned int col, uint32_t &counted, uint32_t &uncounted) const + { + return get(row * numCols_ + col, counted, uncounted); + } + + const T &getFloating(unsigned int index, uint32_t &counted, uint32_t &uncounted) const + { + counted = uncounted = 0; + if (index >= numFloatingRegions()) + return default_; + return get_(numRegions() + index, counted, uncounted); + } + +private: + void set_(unsigned int index, const T &value, uint32_t counted, uint32_t uncounted) + { + regions_[index] = { value, counted, uncounted }; + } + + const T &get_(unsigned int index, uint32_t &counted, uint32_t &uncounted) const + { + counted = regions_[index].counted; + uncounted = regions_[index].uncounted; + return regions_[index].value; + } + + unsigned int numRows_; + unsigned int numCols_; + unsigned int numFloating_; + + struct Region { + T value; + uint32_t counted; + uint32_t uncounted; + }; + + std::vector regions_; + T default_; +}; + +} /* namespace RPiController */ diff --git a/src/ipa/raspberrypi/statistics.h b/src/ipa/raspberrypi/statistics.h new file mode 100644 index 000000000000..a762bf3d41aa --- /dev/null +++ b/src/ipa/raspberrypi/statistics.h @@ -0,0 +1,70 @@ +/* SPDX-License-Identifier: BSD-2-Clause */ +/* + * Copyright (C) 2022, Raspberry Pi Ltd + * + * statistics.h - Raspberry Pi generic statistics structure + */ +#pragma once + +#include +#include +#include + +#include "histogram.h" +#include "region_stats.h" + +namespace RPiController { + +struct RgbySums { + RgbySums(uint64_t _rSum = 0, uint64_t _gSum = 0, uint64_t _bSum = 0, uint64_t _ySum = 0) + : rSum(_rSum), gSum(_gSum), bSum(_bSum), ySum(_ySum) + { + } + uint64_t rSum; + uint64_t gSum; + uint64_t bSum; + uint64_t ySum; +}; + +using RgbyRegions = RegionStats; +using FocusRegions = RegionStats; + +struct Statistics { + /* + * Positioning of the AGC statistics gathering in the pipeline: + * Pre-WB correction or post-WB correction. + * Assume this is post-LSC. + */ + enum class AgcStatsPos { PreWb, PostWb }; + const AgcStatsPos agcStatsPos; + + /* + * Positioning of the AWB/ALSC statistics gathering in the pipeline: + * Pre-LSC or post-LSC. + */ + enum class ColourStatsPos { PreLsc, PostLsc }; + const ColourStatsPos colourStatsPos; + + Statistics(AgcStatsPos a, ColourStatsPos c) + : agcStatsPos(a), colourStatsPos(c) + { + } + + /* Histogram statistics. Not all histograms may be populated! */ + Histogram rHist; + Histogram gHist; + Histogram bHist; + Histogram yHist; + + /* Row sums for flicker avoidance. */ + std::vector rowSums; + + /* Region based colour sums. */ + RgbyRegions agcRegions; + RgbyRegions awbRegions; + + /* Region based focus FoM. */ + FocusRegions focusRegions; +}; + +} /* namespace RPiController */ From patchwork Tue Nov 22 11:22:21 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Naushir Patuck X-Patchwork-Id: 17831 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id BBA6FBE08B for ; Tue, 22 Nov 2022 11:22:31 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 87E4F63323; Tue, 22 Nov 2022 12:22:31 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1669116151; bh=nV2KMfIX/aSz7EeOXcUnYblpXLNHbIzztPfK6uf2CyA=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To: From; b=J+OthZMdfLCv/Ra4t+o6K7yO/tY7CI1jAwLECJ8kUoS2ZfVIlFIp63945H2oN0N5I 53wYpQGTCUMxSqkSpNR90skiecm7tgj+QbCILB/gd1DLyA82BHQHyihp5hoNkXObiP FY760gqx8fah+9yvDLIbmaxUaI0RbRHsCKpBBho3ZKAUgaD0VG5/WrCRiJkultn5Tt SWlFYumeWh+zjUJ14imKAU6gcJpQyl23G+gOYOlgpe8nnuikIKIHrJEl1fJ40/r/MI bYM7lapkDm3/vqDCBRWCPhM0h42npwO5Un0DhxDx3jSofO4e0J6vgklYKoJ3iKMIPT KT/Dc+JD3uPiA== Received: from mail-wm1-x32d.google.com (mail-wm1-x32d.google.com [IPv6:2a00:1450:4864:20::32d]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id E5AC163319 for ; Tue, 22 Nov 2022 12:22:28 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="Bf9Gm9p8"; dkim-atps=neutral Received: by mail-wm1-x32d.google.com with SMTP id m7-20020a05600c090700b003cf8a105d9eso11035610wmp.5 for ; Tue, 22 Nov 2022 03:22:28 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=85uGoaw0AEnYQ+Egl5/eIy+7JdznUeKgQUblC7ayBfM=; b=Bf9Gm9p8FUWIZIF+zGlZVNtER4kOJIpWEE0qfwO8coczc1IpnjZiAfxfr8dCw9OO+f qWHW72qgYz/0bahZhyLnlwIg0iuSmT2EBnBDeFhP6Zk6gJhpA+htJGf1An7ErBSYCMjU Hzdw3E+bsLer4vtwtV0gJVZDbrgECPb2PUNVpitWK1pcKR07eIVl+lQwiou2KvYlQlW+ AEFh+el5vVhNQVGOJdue+fgfzQDNlI++Lw3KdGdTovHM7xejxBqIvj2loGXRi5pzsFMK HKyAjNiJVrrtkAWrjyaMzHVkqOj5tDDjXhDHv8aSgkPykGKLMkWLYm/klt/14+1cmY4N FJIA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=85uGoaw0AEnYQ+Egl5/eIy+7JdznUeKgQUblC7ayBfM=; b=mR0pxvBZ10ZaxtaO/9LQSnMBz1ofzNPxExhJrbieagq6GW28laPmFyAB3M/lhN42wR q8JxXzcFgPA2wA1+MJygc+YOtTNaSAs1vDeZWaos5R1zMzWWfxt2+/6bFEd7IQ31cAk0 DHQfePtW26COB9mSQSX7Vm6hzGRXQexja4+du/KSENN1fOz0wumnoHqdF0m2XetE8a5J J6UKjEYkKmp06/VziosjXcDiNEd/rtBAv08N/Mx/8/iC9VvCEqvD97Raax+yehD1p33/ 7CAE7LEVgflbxiLZa2q/Hk0OTsn8NRk5iR0yA42y4PhTPeFlaCYHY9kssKeTfUxFKVxd bTWQ== X-Gm-Message-State: ANoB5pkHq3DTi4Vg6GMQFDwN18XxXqS4R4/QpNl15qnzKYhq/x59WzRL 9hsg6xkfQkeJ4CPjmu/mdGahvqSj31ttWQ== X-Google-Smtp-Source: AA0mqf6umRHLstW42DmbbtClXlQV1RmODsHn0Gw4pvXwPQplNZigQ89EVSg6f9j17nF1G4M9zq3BTA== X-Received: by 2002:a05:600c:3d10:b0:3cf:8a44:e1eb with SMTP id bh16-20020a05600c3d1000b003cf8a44e1ebmr6730969wmb.189.1669116148338; Tue, 22 Nov 2022 03:22:28 -0800 (PST) Received: from naush-laptop.localdomain ([93.93.133.154]) by smtp.gmail.com with ESMTPSA id q125-20020a1c4383000000b003c6cd82596esm21400388wma.43.2022.11.22.03.22.27 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 22 Nov 2022 03:22:27 -0800 (PST) To: libcamera-devel@lists.libcamera.org Date: Tue, 22 Nov 2022 11:22:21 +0000 Message-Id: <20221122112224.31691-3-naush@raspberrypi.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20221122112224.31691-1-naush@raspberrypi.com> References: <20221122112224.31691-1-naush@raspberrypi.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v1 2/5] ipa: raspberrypi: histogram: Add a constructor for an empty histogram X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Naushir Patuck via libcamera-devel From: Naushir Patuck Reply-To: Naushir Patuck Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Add a default constructor to the RPiController::Histogram class that creates an empty histogram. Since this is a cumulative histogram, push a value of 0 into the first (and only) bin to signify this. Signed-off-by: Naushir Patuck Reviewed-by: David Plowman --- src/ipa/raspberrypi/controller/histogram.h | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/src/ipa/raspberrypi/controller/histogram.h b/src/ipa/raspberrypi/controller/histogram.h index 66a68b087964..6b3e3a9e68e4 100644 --- a/src/ipa/raspberrypi/controller/histogram.h +++ b/src/ipa/raspberrypi/controller/histogram.h @@ -20,6 +20,11 @@ namespace RPiController { class Histogram { public: + Histogram() + { + cumulative_.push_back(0); + } + template Histogram(T *histogram, int num) { assert(num); From patchwork Tue Nov 22 11:22:22 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Naushir Patuck X-Patchwork-Id: 17832 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 7577DBE08B for ; Tue, 22 Nov 2022 11:22:32 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 0535063319; Tue, 22 Nov 2022 12:22:32 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1669116152; bh=ITa9tEwEKy2zHLDsWR78Jt6uw0jVRF4OguWTlKQnrZU=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To: From; b=jRb/K13yap7DpPELnTVpZ0rJL3X1XLhil4GquMLZ7lLnBdpJUNaprnboLxH+ELhp6 62JDXAXtcbiHkeLlfwcm0sdCRKN9g2cpKq40n3kDeRmKUnfXmpfSXeKjQYmNOQYKrE enSoBF6Y37ie70Cc+VFSDo0LquakAIx2Ikn2z5y9o4dSfSWrTpE04h7kRdgSL+9fq9 stap7QvRmLNfkYtx/iRv/g6elvjkDKZZQn08ZaEMk62JA94W3z2I2JU/DrLw/IbKka j2YgQGxTFsKFMEeos9yMIDN0cKUkA7Kpky9jxwo6RFTKQFzukdizgXTpTDtXio4G26 5dNK5va9l/ZCQ== Received: from mail-wm1-x32f.google.com (mail-wm1-x32f.google.com [IPv6:2a00:1450:4864:20::32f]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 823526331D for ; Tue, 22 Nov 2022 12:22:29 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="rOiaQKp0"; dkim-atps=neutral Received: by mail-wm1-x32f.google.com with SMTP id p13-20020a05600c468d00b003cf8859ed1bso11059905wmo.1 for ; Tue, 22 Nov 2022 03:22:29 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=iiHzRQDeANfYrEuD7IDZlbSTxQgE8EZsA8eKtigFok4=; b=rOiaQKp05GdVNO7PWtguf25TO8J+sryTDtB1b2F/SLZ0KTo4ltNzI9kDa9VMibZkj8 zB7AhqX0KECmW1GqM9GdG1QErY1wGOar4Vqpl2bxLsgJw/4F2EcTcwHoFE/pKqBJRPGc cNeFJSI78nekpcmLa+N33InuMg5eHX8ubysGEbq+0fho4BRgihk8jwTBMRza6Sba6XmN REl/IURH5cW0anwS+mdWOlqxADBBtxZUv+Jd0bJXhS4XECHWLhz8MIzQSifgZzU9qmy+ 1Gq2p4PGHB8xvj58aiOh/gvfMJNEamaRT2vsmtcaM6O7bku0Vn5/ba4eVz3HF3jumrqR ZG4w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=iiHzRQDeANfYrEuD7IDZlbSTxQgE8EZsA8eKtigFok4=; b=r4tWLouZKiXxY9cZ0A7VlN+OzUQKoGXuMsEVkckKxX8QqAb50ANQBs6T66DAwMaqeb qxFe215N0UHZK9J7+Kl1Wc8+JJorrDBO/VhNGNwTiDTqRapIV7jRihqw14icvewPX/xX +vC3RKINTiUt9xKCsC+nH8edJznAIHVRbOnLqwnxZXscJ19ChIgMakhoscO0dzJmgfgV AM1J26KAyY9pbOfrGEjzeh7Pcske45NOivghQT1spc+CFWpuvBRix1TxOidnGHk4Q0Mw tcVB1dqwVJ9iElWzqu4x9CvdaHKNcJtthyn8Qu/C1JTOTXDbl42dibi2XBaynQK6CtFZ UFuQ== X-Gm-Message-State: ANoB5pnNe+ZBrIy7tUOUrJrcY8WruvCdnywlJKbt7xAb6vW7R0o1brkm Nwb6kOychriB/i1HY/H8JhtkHyrp3R4z1w== X-Google-Smtp-Source: AA0mqf7gE9xIW4GdnDrQKu5R2fNHDvAxLXMd8Lb3lNWTbnvJ3IFyisR1DOovlp8CEVzbJvg+ut6fIA== X-Received: by 2002:a05:600c:1f1b:b0:3cf:35c8:289f with SMTP id bd27-20020a05600c1f1b00b003cf35c8289fmr7112407wmb.153.1669116148965; Tue, 22 Nov 2022 03:22:28 -0800 (PST) Received: from naush-laptop.localdomain ([93.93.133.154]) by smtp.gmail.com with ESMTPSA id q125-20020a1c4383000000b003c6cd82596esm21400388wma.43.2022.11.22.03.22.28 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 22 Nov 2022 03:22:28 -0800 (PST) To: libcamera-devel@lists.libcamera.org Date: Tue, 22 Nov 2022 11:22:22 +0000 Message-Id: <20221122112224.31691-4-naush@raspberrypi.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20221122112224.31691-1-naush@raspberrypi.com> References: <20221122112224.31691-1-naush@raspberrypi.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v1 3/5] ipa: raspberrypi: awb: Delay release of the statistics buffer X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Naushir Patuck via libcamera-devel From: Naushir Patuck Reply-To: Naushir Patuck Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Release the statistics buffer after running the through the AWB calculations. Only the "counted" statistics are copied out to a local structure, so keeping the statistics buffer allows the algorithm to see the "uncounted" statistics as well. This is currently handled by hard-coding the total number of statistics regions regions based on the structure definition in the bcm2835_isp_stats structure. Signed-off-by: Naushir Patuck Reviewed-by: David Plowman --- src/ipa/raspberrypi/controller/rpi/awb.cpp | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/src/ipa/raspberrypi/controller/rpi/awb.cpp b/src/ipa/raspberrypi/controller/rpi/awb.cpp index 8d8ddf0913f7..861022014896 100644 --- a/src/ipa/raspberrypi/controller/rpi/awb.cpp +++ b/src/ipa/raspberrypi/controller/rpi/awb.cpp @@ -432,11 +432,6 @@ void Awb::prepareStats() */ generateStats(zones_, statistics_->awb_stats, config_.minPixels, config_.minG); - /* - * we're done with these; we may as well relinquish our hold on the - * pointer. - */ - statistics_.reset(); /* * apply sensitivities, so values appear to come from our "canonical" * sensor. @@ -733,6 +728,11 @@ void Awb::doAwb() << " with gains r " << asyncResults_.gainR << " and b " << asyncResults_.gainB; } + /* + * we're done with these; we may as well relinquish our hold on the + * pointer. + */ + statistics_.reset(); } /* Register algorithm with the system. */ From patchwork Tue Nov 22 11:22:23 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Naushir Patuck X-Patchwork-Id: 17833 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 8E99ABE08B for ; Tue, 22 Nov 2022 11:22:34 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 59E996331F; Tue, 22 Nov 2022 12:22:34 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1669116154; bh=nyXzcX26JAwa29RqzhMsvVCfAG7X9PASyh1wVHLsIkU=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To: From; b=2QEpllM04zkjf94ZQ1ytiOrsCWF98qaqdKLyhOxRyAgV9/Fi7/Hm4QYK+5TAL+pKw FeAH+0t+0m+YZElNkkJJsve5peR5qXghFwXGRO7DfbuyX4Xob5YLBSuP2/iTzkTcI1 qBhl9QI6cLnGIeT1hZsj7Fxpg/ysHb5Np7bg63vsuKSGujXrqvyphI3+R/td59EM/O EpHF9cDCCsiTaBu/csNdarKQ4aU8LvA57yyjwBXSGDtVTaEYiTx66fZr0e9Y/v0ekX tDzzWaS5P8/4p1zVoT02jetDb3nRVsmuwNW443SXY8z7nYqp5zhJEAXiuT/+T1HWle J+Jjkbh7ZRd/A== Received: from mail-wr1-x42f.google.com (mail-wr1-x42f.google.com [IPv6:2a00:1450:4864:20::42f]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 6F8D0632F8 for ; Tue, 22 Nov 2022 12:22:30 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="sdgG/XD7"; dkim-atps=neutral Received: by mail-wr1-x42f.google.com with SMTP id g12so24180951wrs.10 for ; Tue, 22 Nov 2022 03:22:30 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=Ofpu7pav/pZ0qu53mhmcFk8VAs550SOzAo26PUhiDFc=; b=sdgG/XD7bqseqYkZZwYmMKbmf+lXuwru1Uvmc7KygbgAtdnGyDbYASpZ7kmg5reno9 Acq2NtflRP5OUKY0dqA5LSjjCBcASIHxB/eIEpHxc+4KbjX3jjfwmEwX8oEBfK9wUYy9 B/bzodTKDtHQcbabDoOvoCI4HrlScatzm0rDNZ1Vorn0b856y/pAgGEH79BHgr5UqUNR jgqQc6GGgifvWJXBf3WOyg07gdgCCMncP5vLn7EEzfmCEPor+Jq1vSXfi7Yd+d0DCzVs GjBXRjH7bdbu+yusYEiTVCe5cI8thR3/ceiRpgc31vvyx/CIJUZw8OwbaUnINRE6V99v cLqA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=Ofpu7pav/pZ0qu53mhmcFk8VAs550SOzAo26PUhiDFc=; b=pfZP5o5Y2F08LdMbjm2VzcTkegQ8U9cAud0VbIrEFQTmQHIEGlFpPXaG+GTEAfSv4P 6FmI7WwB/bnYHyTuFzpfD/YfxW1GbaqiVJCre+hoH1byiPvMeGRPmVy+b1JSCmnNEo5X kQG8Q9uBiKU+HIwdMTcymv7FiwrRTnL32vn9yfvsfU1qosnqQqr3i6ZCIfbzO++29NZ/ XHhRjkoyvI697T9TddI7JvpqyHbj37OrZudPfgjaI+B5pBoUGvnezRqh6cJ96vcGe/ah v/0VtsBURJyt4vtScjSCo6ZloKJYwNrn4TLpT7QCRvpOyUBlG4h4go20vjOZ4uX3j7bL URbA== X-Gm-Message-State: ANoB5plG+a9HmIj/kxVQScHJ+xMhplLeJq4nTq5yODVmC+M4VuNRMBZj L2v6G9UbKW+kjsdPFXYUTHqPBWopDH80cQ== X-Google-Smtp-Source: AA0mqf5UjPoaADx2LOURKwD+L9eO6Cvk4/2N8S64u4wH0lbMqvyreeRtngW+jN0I1k/fiOMnv3NAbA== X-Received: by 2002:a05:6000:510:b0:22e:3ca6:d4ab with SMTP id a16-20020a056000051000b0022e3ca6d4abmr14528642wrf.658.1669116149652; Tue, 22 Nov 2022 03:22:29 -0800 (PST) Received: from naush-laptop.localdomain ([93.93.133.154]) by smtp.gmail.com with ESMTPSA id q125-20020a1c4383000000b003c6cd82596esm21400388wma.43.2022.11.22.03.22.29 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 22 Nov 2022 03:22:29 -0800 (PST) To: libcamera-devel@lists.libcamera.org Date: Tue, 22 Nov 2022 11:22:23 +0000 Message-Id: <20221122112224.31691-5-naush@raspberrypi.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20221122112224.31691-1-naush@raspberrypi.com> References: <20221122112224.31691-1-naush@raspberrypi.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v1 4/5] ipa: raspberrypi: Use the generic statistics structure in the algorithms X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Naushir Patuck via libcamera-devel From: Naushir Patuck Reply-To: Naushir Patuck Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Repurpose the StatisticsPtr type from being a shared_ptr to shared_ptr. This removes any hardware specific header files and structures from the algorithms source code. Add a new function in the Raspberry Pi IPA to populate the generic statistics structure from the values provided by the hardware in the bcm2835_isp_stats structure. Update the Lux, AWB, AGC, ALCS, Contrast, and Focus algorithms to use the generic statistics structure appropriately in their calculations. Additionally, remove references to any hardware specific headers and defines in these source files. Signed-off-by: Naushir Patuck --- src/ipa/raspberrypi/controller/controller.h | 4 +- src/ipa/raspberrypi/controller/rpi/agc.cpp | 26 ++++++------- src/ipa/raspberrypi/controller/rpi/agc.h | 2 +- src/ipa/raspberrypi/controller/rpi/alsc.cpp | 33 +++++++++------- src/ipa/raspberrypi/controller/rpi/alsc.h | 3 +- src/ipa/raspberrypi/controller/rpi/awb.cpp | 20 +++++----- src/ipa/raspberrypi/controller/rpi/awb.h | 1 + .../raspberrypi/controller/rpi/contrast.cpp | 8 ++-- src/ipa/raspberrypi/controller/rpi/focus.cpp | 7 ++-- src/ipa/raspberrypi/controller/rpi/lux.cpp | 13 +------ src/ipa/raspberrypi/raspberrypi.cpp | 39 ++++++++++++++++++- src/ipa/raspberrypi/statistics.h | 2 + 12 files changed, 95 insertions(+), 63 deletions(-) diff --git a/src/ipa/raspberrypi/controller/controller.h b/src/ipa/raspberrypi/controller/controller.h index 3e1e051703b3..e6c950c3a509 100644 --- a/src/ipa/raspberrypi/controller/controller.h +++ b/src/ipa/raspberrypi/controller/controller.h @@ -15,19 +15,17 @@ #include #include -#include - #include "libcamera/internal/yaml_parser.h" #include "camera_mode.h" #include "device_status.h" #include "metadata.h" +#include "statistics.h" namespace RPiController { class Algorithm; typedef std::unique_ptr AlgorithmPtr; -typedef std::shared_ptr StatisticsPtr; /* * The Controller holds a pointer to some global_metadata, which is how diff --git a/src/ipa/raspberrypi/controller/rpi/agc.cpp b/src/ipa/raspberrypi/controller/rpi/agc.cpp index bd54a639d637..79c83e0a9eae 100644 --- a/src/ipa/raspberrypi/controller/rpi/agc.cpp +++ b/src/ipa/raspberrypi/controller/rpi/agc.cpp @@ -9,8 +9,6 @@ #include #include -#include - #include #include "../awb_status.h" @@ -450,7 +448,7 @@ void Agc::process(StatisticsPtr &stats, Metadata *imageMetadata) fetchCurrentExposure(imageMetadata); /* Compute the total gain we require relative to the current exposure. */ double gain, targetY; - computeGain(stats.get(), imageMetadata, gain, targetY); + computeGain(stats, imageMetadata, gain, targetY); /* Now compute the target (final) exposure which we think we want. */ computeTargetExposure(gain); /* @@ -584,20 +582,20 @@ void Agc::fetchAwbStatus(Metadata *imageMetadata) LOG(RPiAgc, Debug) << "No AWB status found"; } -static double computeInitialY(bcm2835_isp_stats *stats, AwbStatus const &awb, +static double computeInitialY(StatisticsPtr &stats, AwbStatus const &awb, double weights[], double gain) { - bcm2835_isp_stats_region *regions = stats->agc_stats; /* * Note how the calculation below means that equal weights give you * "average" metering (i.e. all pixels equally important). */ double rSum = 0, gSum = 0, bSum = 0, pixelSum = 0; - for (unsigned int i = 0; i < AgcStatsSize; i++) { - double counted = regions[i].counted; - double rAcc = std::min(regions[i].r_sum * gain, ((1 << PipelineBits) - 1) * counted); - double gAcc = std::min(regions[i].g_sum * gain, ((1 << PipelineBits) - 1) * counted); - double bAcc = std::min(regions[i].b_sum * gain, ((1 << PipelineBits) - 1) * counted); + for (unsigned int i = 0; i < stats->agcRegions.numRegions(); i++) { + uint32_t counted, uncounted; + auto s = stats->agcRegions.get(i, counted, uncounted); + double rAcc = std::min(s.rSum * gain, ((1 << PipelineBits) - 1) * counted); + double gAcc = std::min(s.gSum * gain, ((1 << PipelineBits) - 1) * counted); + double bAcc = std::min(s.bSum * gain, ((1 << PipelineBits) - 1) * counted); rSum += rAcc * weights[i]; gSum += gAcc * weights[i]; bSum += bAcc * weights[i]; @@ -623,23 +621,23 @@ static double computeInitialY(bcm2835_isp_stats *stats, AwbStatus const &awb, static constexpr double EvGainYTargetLimit = 0.9; -static double constraintComputeGain(AgcConstraint &c, Histogram &h, double lux, +static double constraintComputeGain(AgcConstraint &c, const Histogram &h, double lux, double evGain, double &targetY) { targetY = c.yTarget.eval(c.yTarget.domain().clip(lux)); targetY = std::min(EvGainYTargetLimit, targetY * evGain); double iqm = h.interQuantileMean(c.qLo, c.qHi); - return (targetY * NUM_HISTOGRAM_BINS) / iqm; + return (targetY * h.bins()) / iqm; } -void Agc::computeGain(bcm2835_isp_stats *statistics, Metadata *imageMetadata, +void Agc::computeGain(StatisticsPtr &statistics, Metadata *imageMetadata, double &gain, double &targetY) { struct LuxStatus lux = {}; lux.lux = 400; /* default lux level to 400 in case no metadata found */ if (imageMetadata->get("lux.status", lux) != 0) LOG(RPiAgc, Warning) << "No lux level found"; - Histogram h(statistics->hist[0].g_hist, NUM_HISTOGRAM_BINS); + const Histogram &h = statistics->yHist; double evGain = status_.ev * config_.baseEv; /* * The initial gain and target_Y come from some of the regions. After diff --git a/src/ipa/raspberrypi/controller/rpi/agc.h b/src/ipa/raspberrypi/controller/rpi/agc.h index 6d6b0e5ad857..0ea71e6d1aa0 100644 --- a/src/ipa/raspberrypi/controller/rpi/agc.h +++ b/src/ipa/raspberrypi/controller/rpi/agc.h @@ -98,7 +98,7 @@ private: void housekeepConfig(); void fetchCurrentExposure(Metadata *imageMetadata); void fetchAwbStatus(Metadata *imageMetadata); - void computeGain(bcm2835_isp_stats *statistics, Metadata *imageMetadata, + void computeGain(StatisticsPtr &statistics, Metadata *imageMetadata, double &gain, double &targetY); void computeTargetExposure(double gain); bool applyDigitalGain(double gain, double targetY); diff --git a/src/ipa/raspberrypi/controller/rpi/alsc.cpp b/src/ipa/raspberrypi/controller/rpi/alsc.cpp index a4afaf841c41..d8c650e6faa6 100644 --- a/src/ipa/raspberrypi/controller/rpi/alsc.cpp +++ b/src/ipa/raspberrypi/controller/rpi/alsc.cpp @@ -310,18 +310,22 @@ double getCt(Metadata *metadata, double defaultCt) return awbStatus.temperatureK; } -static void copyStats(bcm2835_isp_stats_region regions[XY], StatisticsPtr &stats, +static void copyStats(RgbyRegions ®ions, StatisticsPtr &stats, AlscStatus const &status) { - bcm2835_isp_stats_region *inputRegions = stats->awb_stats; + if (!regions.numRegions()) + regions.init(stats->awbRegions.numRegions()); + double *rTable = (double *)status.r; double *gTable = (double *)status.g; double *bTable = (double *)status.b; - for (int i = 0; i < XY; i++) { - regions[i].r_sum = inputRegions[i].r_sum / rTable[i]; - regions[i].g_sum = inputRegions[i].g_sum / gTable[i]; - regions[i].b_sum = inputRegions[i].b_sum / bTable[i]; - regions[i].counted = inputRegions[i].counted; + for (unsigned int i = 0; i < stats->awbRegions.numRegions(); i++) { + uint32_t counted, uncounted; + auto s = stats->awbRegions.get(i, counted, uncounted); + regions.set(i, { s.rSum / static_cast(rTable[i]), + s.gSum / static_cast(gTable[i]), + s.bSum / static_cast(bTable[i]) }, + counted, uncounted); /* (don't care about the uncounted value) */ } } @@ -512,19 +516,20 @@ void resampleCalTable(double const calTableIn[XY], } /* Calculate chrominance statistics (R/G and B/G) for each region. */ -static_assert(XY == AWB_REGIONS, "ALSC/AWB statistics region mismatch"); -static void calculateCrCb(bcm2835_isp_stats_region *awbRegion, double cr[XY], +static void calculateCrCb(const RgbyRegions &awbRegion, double cr[XY], double cb[XY], uint32_t minCount, uint16_t minG) { for (int i = 0; i < XY; i++) { - bcm2835_isp_stats_region &zone = awbRegion[i]; - if (zone.counted <= minCount || - zone.g_sum / zone.counted <= minG) { + uint32_t counted, uncounted; + auto s = awbRegion.get(i, counted, uncounted); + + if (counted <= minCount || s.gSum / counted <= minG) { cr[i] = cb[i] = InsufficientData; continue; } - cr[i] = zone.r_sum / (double)zone.g_sum; - cb[i] = zone.b_sum / (double)zone.g_sum; + + cr[i] = s.rSum / (double)s.gSum; + cb[i] = s.bSum / (double)s.gSum; } } diff --git a/src/ipa/raspberrypi/controller/rpi/alsc.h b/src/ipa/raspberrypi/controller/rpi/alsc.h index a858ef5a6551..9167c9ffa2e3 100644 --- a/src/ipa/raspberrypi/controller/rpi/alsc.h +++ b/src/ipa/raspberrypi/controller/rpi/alsc.h @@ -12,6 +12,7 @@ #include "../algorithm.h" #include "../alsc_status.h" +#include "../statistics.h" namespace RPiController { @@ -98,7 +99,7 @@ private: /* copy out the results from the async thread so that it can be restarted */ void fetchAsyncResults(); double ct_; - bcm2835_isp_stats_region statistics_[AlscCellsY * AlscCellsX]; + RgbyRegions statistics_; double asyncResults_[3][AlscCellsY][AlscCellsX]; double asyncLambdaR_[AlscCellsX * AlscCellsY]; double asyncLambdaB_[AlscCellsX * AlscCellsY]; diff --git a/src/ipa/raspberrypi/controller/rpi/awb.cpp b/src/ipa/raspberrypi/controller/rpi/awb.cpp index 861022014896..901498382c6b 100644 --- a/src/ipa/raspberrypi/controller/rpi/awb.cpp +++ b/src/ipa/raspberrypi/controller/rpi/awb.cpp @@ -21,9 +21,6 @@ LOG_DEFINE_CATEGORY(RPiAwb) #define NAME "rpi.awb" -static constexpr unsigned int AwbStatsSizeX = DEFAULT_AWB_REGIONS_X; -static constexpr unsigned int AwbStatsSizeY = DEFAULT_AWB_REGIONS_Y; - /* * todo - the locking in this algorithm needs some tidying up as has been done * elsewhere (ALSC and AGC). @@ -406,17 +403,18 @@ void Awb::asyncFunc() } static void generateStats(std::vector &zones, - bcm2835_isp_stats_region *stats, double minPixels, + RgbyRegions &stats, double minPixels, double minG) { - for (unsigned int i = 0; i < AwbStatsSizeX * AwbStatsSizeY; i++) { + for (unsigned int i = 0; i < stats.numRegions(); i++) { Awb::RGB zone; - double counted = stats[i].counted; + uint32_t counted, uncounted; + auto s = stats.get(i, counted, uncounted); if (counted >= minPixels) { - zone.G = stats[i].g_sum / counted; + zone.G = s.gSum / counted; if (zone.G >= minG) { - zone.R = stats[i].r_sum / counted; - zone.B = stats[i].b_sum / counted; + zone.R = s.rSum / counted; + zone.B = s.bSum / counted; zones.push_back(zone); } } @@ -430,7 +428,7 @@ void Awb::prepareStats() * LSC has already been applied to the stats in this pipeline, so stop * any LSC compensation. We also ignore config_.fast in this version. */ - generateStats(zones_, statistics_->awb_stats, config_.minPixels, + generateStats(zones_, statistics_->awbRegions, config_.minPixels, config_.minG); /* * apply sensitivities, so values appear to come from our "canonical" @@ -646,7 +644,7 @@ void Awb::awbBayes() * valid... not entirely sure about this. */ Pwl prior = interpolatePrior(); - prior *= zones_.size() / (double)(AwbStatsSizeX * AwbStatsSizeY); + prior *= zones_.size() / (double)(statistics_->awbRegions.numRegions()); prior.map([](double x, double y) { LOG(RPiAwb, Debug) << "(" << x << "," << y << ")"; }); diff --git a/src/ipa/raspberrypi/controller/rpi/awb.h b/src/ipa/raspberrypi/controller/rpi/awb.h index 30acd89d0969..d81779dd51ff 100644 --- a/src/ipa/raspberrypi/controller/rpi/awb.h +++ b/src/ipa/raspberrypi/controller/rpi/awb.h @@ -13,6 +13,7 @@ #include "../awb_algorithm.h" #include "../pwl.h" #include "../awb_status.h" +#include "../statistics.h" namespace RPiController { diff --git a/src/ipa/raspberrypi/controller/rpi/contrast.cpp b/src/ipa/raspberrypi/controller/rpi/contrast.cpp index 5b37edcbd46a..a4f8c4f04fc4 100644 --- a/src/ipa/raspberrypi/controller/rpi/contrast.cpp +++ b/src/ipa/raspberrypi/controller/rpi/contrast.cpp @@ -106,7 +106,7 @@ Pwl computeStretchCurve(Histogram const &histogram, * bit. */ double histLo = histogram.quantile(config.loHistogram) * - (65536 / NUM_HISTOGRAM_BINS); + (65536 / histogram.bins()); double levelLo = config.loLevel * 65536; LOG(RPiContrast, Debug) << "Move histogram point " << histLo << " to " << levelLo; @@ -119,7 +119,7 @@ Pwl computeStretchCurve(Histogram const &histogram, * Keep the mid-point (median) in the same place, though, to limit the * apparent amount of global brightness shift. */ - double mid = histogram.quantile(0.5) * (65536 / NUM_HISTOGRAM_BINS); + double mid = histogram.quantile(0.5) * (65536 / histogram.bins()); enhance.append(mid, mid); /* @@ -127,7 +127,7 @@ Pwl computeStretchCurve(Histogram const &histogram, * there up. */ double histHi = histogram.quantile(config.hiHistogram) * - (65536 / NUM_HISTOGRAM_BINS); + (65536 / histogram.bins()); double levelHi = config.hiLevel * 65536; LOG(RPiContrast, Debug) << "Move histogram point " << histHi << " to " << levelHi; @@ -158,7 +158,7 @@ Pwl applyManualContrast(Pwl const &gammaCurve, double brightness, void Contrast::process(StatisticsPtr &stats, [[maybe_unused]] Metadata *imageMetadata) { - Histogram histogram(stats->hist[0].g_hist, NUM_HISTOGRAM_BINS); + Histogram &histogram = stats->yHist; /* * We look at the histogram and adjust the gamma curve in the following * ways: 1. Adjust the gamma curve so as to pull the start of the diff --git a/src/ipa/raspberrypi/controller/rpi/focus.cpp b/src/ipa/raspberrypi/controller/rpi/focus.cpp index 8c5029bd0e95..41afbf43f2b7 100644 --- a/src/ipa/raspberrypi/controller/rpi/focus.cpp +++ b/src/ipa/raspberrypi/controller/rpi/focus.cpp @@ -32,9 +32,10 @@ void Focus::process(StatisticsPtr &stats, Metadata *imageMetadata) { FocusStatus status; unsigned int i; - for (i = 0; i < FOCUS_REGIONS; i++) - status.focusMeasures[i] = stats->focus_stats[i].contrast_val[1][1] / 1000; - status.num = i; + uint32_t counted, uncounted; + for (i = 0; i < stats->focusRegions.numRegions(); i++) + status.focusMeasures[i] = stats->focusRegions.get(i, counted, uncounted); + status.num = stats->focusRegions.numRegions(); imageMetadata->set("focus.status", status); LOG(RPiFocus, Debug) diff --git a/src/ipa/raspberrypi/controller/rpi/lux.cpp b/src/ipa/raspberrypi/controller/rpi/lux.cpp index 9759186afacf..a49d402707c7 100644 --- a/src/ipa/raspberrypi/controller/rpi/lux.cpp +++ b/src/ipa/raspberrypi/controller/rpi/lux.cpp @@ -6,8 +6,6 @@ */ #include -#include - #include #include "../device_status.h" @@ -83,20 +81,13 @@ void Lux::process(StatisticsPtr &stats, Metadata *imageMetadata) if (imageMetadata->get("device.status", deviceStatus) == 0) { double currentGain = deviceStatus.analogueGain; double currentAperture = deviceStatus.aperture.value_or(currentAperture_); - uint64_t sum = 0; - uint32_t num = 0; - uint32_t *bin = stats->hist[0].g_hist; - const int numBins = sizeof(stats->hist[0].g_hist) / - sizeof(stats->hist[0].g_hist[0]); - for (int i = 0; i < numBins; i++) - sum += bin[i] * (uint64_t)i, num += bin[i]; /* add .5 to reflect the mid-points of bins */ - double currentY = sum / (double)num + .5; + double currentY = stats->yHist.total() / stats->yHist.bins() + .5; double gainRatio = referenceGain_ / currentGain; double shutterSpeedRatio = referenceShutterSpeed_ / deviceStatus.shutterSpeed; double apertureRatio = referenceAperture_ / currentAperture; - double yRatio = currentY * (65536 / numBins) / referenceY_; + double yRatio = currentY * (65536 / stats->yHist.bins()) / referenceY_; double estimatedLux = shutterSpeedRatio * gainRatio * apertureRatio * apertureRatio * yRatio * referenceLux_; diff --git a/src/ipa/raspberrypi/raspberrypi.cpp b/src/ipa/raspberrypi/raspberrypi.cpp index b74f1ecf738f..8fcfa0b3ea50 100644 --- a/src/ipa/raspberrypi/raspberrypi.cpp +++ b/src/ipa/raspberrypi/raspberrypi.cpp @@ -51,6 +51,7 @@ #include "metadata.h" #include "sharpen_algorithm.h" #include "sharpen_status.h" +#include "statistics.h" namespace libcamera { @@ -136,6 +137,7 @@ private: void prepareISP(const ISPConfig &data); void reportMetadata(); void fillDeviceStatus(const ControlList &sensorControls); + RPiController::StatisticsPtr fillStatistics(bcm2835_isp_stats *stats) const; void processStats(unsigned int bufferId); void applyFrameDurations(Duration minFrameDuration, Duration maxFrameDuration); void applyAGC(const struct AgcStatus *agcStatus, ControlList &ctrls); @@ -1119,6 +1121,41 @@ void IPARPi::fillDeviceStatus(const ControlList &sensorControls) rpiMetadata_.set("device.status", deviceStatus); } +RPiController::StatisticsPtr IPARPi::fillStatistics(bcm2835_isp_stats *stats) const +{ + using namespace RPiController; + + unsigned int i; + StatisticsPtr statistics = + std::make_unique(Statistics::AgcStatsPos::PreWb, Statistics::ColourStatsPos::PostLsc); + + /* RGB histograms are not used, so do not populate them. */ + statistics->yHist = std::move(RPiController::Histogram(stats->hist[0].g_hist, NUM_HISTOGRAM_BINS)); + + statistics->awbRegions.init(DEFAULT_AWB_REGIONS_X, DEFAULT_AWB_REGIONS_Y); + for (i = 0; i < statistics->awbRegions.numRegions(); i++) + statistics->awbRegions.set(i, { stats->awb_stats[i].r_sum, + stats->awb_stats[i].g_sum, + stats->awb_stats[i].b_sum }, + stats->awb_stats[i].counted, stats->awb_stats[i].notcounted); + + /* There are only ever 15 regions computed by the firmware, but the HW defines AGC_REGIONS == 16! */ + statistics->agcRegions.init(15); + for (i = 0; i < statistics->agcRegions.numRegions(); i++) + statistics->agcRegions.set(i, { stats->agc_stats[i].r_sum, + stats->agc_stats[i].g_sum, + stats->agc_stats[i].b_sum }, + stats->agc_stats[i].counted, 0); + + statistics->focusRegions.init(4, 3); + for (i = 0; i < FOCUS_REGIONS; i++) + statistics->focusRegions.set(i, stats->focus_stats[i].contrast_val[1][1] / 1000, + stats->focus_stats[i].contrast_val_num[1][1], + stats->focus_stats[i].contrast_val_num[1][0]); + + return statistics; +} + void IPARPi::processStats(unsigned int bufferId) { auto it = buffers_.find(bufferId); @@ -1129,7 +1166,7 @@ void IPARPi::processStats(unsigned int bufferId) Span mem = it->second.planes()[0]; bcm2835_isp_stats *stats = reinterpret_cast(mem.data()); - RPiController::StatisticsPtr statistics = std::make_shared(*stats); + RPiController::StatisticsPtr statistics = fillStatistics(stats); helper_->process(statistics, rpiMetadata_); controller_.process(statistics, &rpiMetadata_); diff --git a/src/ipa/raspberrypi/statistics.h b/src/ipa/raspberrypi/statistics.h index a762bf3d41aa..affb7272c963 100644 --- a/src/ipa/raspberrypi/statistics.h +++ b/src/ipa/raspberrypi/statistics.h @@ -67,4 +67,6 @@ struct Statistics { FocusRegions focusRegions; }; +using StatisticsPtr = std::shared_ptr; + } /* namespace RPiController */ From patchwork Tue Nov 22 11:22:24 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Naushir Patuck X-Patchwork-Id: 17834 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 09348C3286 for ; Tue, 22 Nov 2022 11:22:35 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id BBF2C632F8; Tue, 22 Nov 2022 12:22:34 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1669116154; bh=o3ITZHbe6aFynDHRpdIvstbdwMQ8TCOwVNf8A98D2/c=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To: From; b=AfDQfV4wNuxZDlXIzOk9v2VFrvhh25eRiAwpPaK7g+Ahkp2P0A5EcZsTmVcW86Ey1 pEG3h0zgNyo3RZ+CyIeDXWcu/eOa21jKTX/MLGUx9ZbtVusvg/OYslv36J9f3KzHvB 7zgl1J2PIdiM9Jk+QeTvFHaZmSI7INorfxqTnqsFW3cC0sLQXFYIu6znPT+X80oU/J 8Kx9+yTF+KHV04/Ii+MsB05AduJsh4XiR5ugakn5FXaGoyMNlyKcH+fA5ktS08IDeO t/2ogvJwebAdGN1X2/Sw4BRdN/n5bpTOdkhNyi4jh2lObsydnSw/JeLSsNXcjW1tkz /DMaqvfoJOEoA== Received: from mail-wm1-x32e.google.com (mail-wm1-x32e.google.com [IPv6:2a00:1450:4864:20::32e]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id E03886331A for ; Tue, 22 Nov 2022 12:22:30 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="qh/ZpMeJ"; dkim-atps=neutral Received: by mail-wm1-x32e.google.com with SMTP id v124-20020a1cac82000000b003cf7a4ea2caso14969149wme.5 for ; Tue, 22 Nov 2022 03:22:30 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=cEBBhUzHrYHZA7FKrwtQWXcsENcHMTeNMYFSTGUqYVw=; b=qh/ZpMeJU1CeRA9khp+YGCOojxzVgYNkSdrKDY/maoxzcAyrp67JiX9THYi/hZ4jRg 1S98yOK4LjFeTjcPI3ARnZq7W7iCHvKJkJ0pqiUpnojUQ55tA6xXzrzaPD3RmXIOFjva 117mL0pjb/nPLDMauiK4+aSASnEkMfn9kEPBA3VOzHA4ASZYS03xAz+C/yOSp0MJhxfW 8VjQlTR3Y8Dx868vdWM0ibuDw6qxv1ASqahl5aW9rDYYlrLukmypV7CsaVCUMNyPprpy 715IrHiNSXDquVdHRNq9GM+ZZwFShxuaeyrqbbvTJIypKCU1gY1DEyoeefzLhcGuw4h4 oQxA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=cEBBhUzHrYHZA7FKrwtQWXcsENcHMTeNMYFSTGUqYVw=; b=FpEx1Ni/wRxeI+pAAhBKQ93mpbiAXBCiLRYrD/tG8j+Zz6hBePLwb92udtRaijklsY WHKI94fghiNddtX/bZg5X0qf8yK+bUtqBcNHuD4vQjZM/INFPrm7oXqiiBhY1oGaJv58 GUgcxwAj6iAqoMlyO8chLWIau4ojTFNyJ/CtnW9Y7YHaEB8JqYPPA6qkA68nmUTVWy8F yQY053DXMdsd7qCkhkpuiY8kRgbSC44ZZ4m9iYFvxRCtS4sYm6k8neW1NHSJDexfuK9E rquWhCEjSLQLCfJ+r2Zf22pK52KP6tmW57f+0nf/6ISE2PGhqKoTwGi0kSmwvn1Sho6u iXTA== X-Gm-Message-State: ANoB5pn+pjjuheVsmVBeE9estYt0hntZsl58K0luLiyP2o1jM7W9WwOt wBPOV9a3PnTQ8Wpexg5fUFUvx8F/xujrOQ== X-Google-Smtp-Source: AA0mqf7R+tmk+7GaU/py51HBlR7+8CzvHxJEWpefABBdOOqdO+cscrTZ0CD5X7ynNzgPOeyyAaqJRA== X-Received: by 2002:a7b:c2aa:0:b0:3cf:5d41:bea1 with SMTP id c10-20020a7bc2aa000000b003cf5d41bea1mr3630138wmk.118.1669116150287; Tue, 22 Nov 2022 03:22:30 -0800 (PST) Received: from naush-laptop.localdomain ([93.93.133.154]) by smtp.gmail.com with ESMTPSA id q125-20020a1c4383000000b003c6cd82596esm21400388wma.43.2022.11.22.03.22.29 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 22 Nov 2022 03:22:29 -0800 (PST) To: libcamera-devel@lists.libcamera.org Date: Tue, 22 Nov 2022 11:22:24 +0000 Message-Id: <20221122112224.31691-6-naush@raspberrypi.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20221122112224.31691-1-naush@raspberrypi.com> References: <20221122112224.31691-1-naush@raspberrypi.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v1 5/5] ipa: raspberrypi: Normalise region sums to 16-bits X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Naushir Patuck via libcamera-devel From: Naushir Patuck Reply-To: Naushir Patuck Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" The VC4 ISP uses a pipeline bit-depth of 13-bits. The AGC algorithm needs to know this bit-depth when computing the Y value for the image. Instead of hardcoding the VC4 bit-depth in the AGC source code, normalise all region sums to 16-bits when filling the Statistics structure. AWB and ALSC are agonistic about pipline depth, so do not need chaning. Signed-off-by: Naushir Patuck Reviewed-by: David Plowman --- src/ipa/raspberrypi/controller/rpi/agc.cpp | 10 ++++------ src/ipa/raspberrypi/raspberrypi.cpp | 21 ++++++++++++++------- 2 files changed, 18 insertions(+), 13 deletions(-) diff --git a/src/ipa/raspberrypi/controller/rpi/agc.cpp b/src/ipa/raspberrypi/controller/rpi/agc.cpp index 79c83e0a9eae..358af8f2544b 100644 --- a/src/ipa/raspberrypi/controller/rpi/agc.cpp +++ b/src/ipa/raspberrypi/controller/rpi/agc.cpp @@ -28,8 +28,6 @@ LOG_DEFINE_CATEGORY(RPiAgc) #define NAME "rpi.agc" -static constexpr unsigned int PipelineBits = 13; /* seems to be a 13-bit pipeline */ - int AgcMeteringMode::read(const libcamera::YamlObject ¶ms) { const YamlObject &yamlWeights = params["weights"]; @@ -593,9 +591,9 @@ static double computeInitialY(StatisticsPtr &stats, AwbStatus const &awb, for (unsigned int i = 0; i < stats->agcRegions.numRegions(); i++) { uint32_t counted, uncounted; auto s = stats->agcRegions.get(i, counted, uncounted); - double rAcc = std::min(s.rSum * gain, ((1 << PipelineBits) - 1) * counted); - double gAcc = std::min(s.gSum * gain, ((1 << PipelineBits) - 1) * counted); - double bAcc = std::min(s.bSum * gain, ((1 << PipelineBits) - 1) * counted); + double rAcc = std::min(s.rSum * gain, ((1 << 16) - 1) * counted); + double gAcc = std::min(s.gSum * gain, ((1 << 16) - 1) * counted); + double bAcc = std::min(s.bSum * gain, ((1 << 16) - 1) * counted); rSum += rAcc * weights[i]; gSum += gAcc * weights[i]; bSum += bAcc * weights[i]; @@ -608,7 +606,7 @@ static double computeInitialY(StatisticsPtr &stats, AwbStatus const &awb, double ySum = rSum * awb.gainR * .299 + gSum * awb.gainG * .587 + bSum * awb.gainB * .114; - return ySum / pixelSum / (1 << PipelineBits); + return ySum / pixelSum / (1 << 16); } /* diff --git a/src/ipa/raspberrypi/raspberrypi.cpp b/src/ipa/raspberrypi/raspberrypi.cpp index 8fcfa0b3ea50..fcecd60bcf6d 100644 --- a/src/ipa/raspberrypi/raspberrypi.cpp +++ b/src/ipa/raspberrypi/raspberrypi.cpp @@ -1132,19 +1132,26 @@ RPiController::StatisticsPtr IPARPi::fillStatistics(bcm2835_isp_stats *stats) co /* RGB histograms are not used, so do not populate them. */ statistics->yHist = std::move(RPiController::Histogram(stats->hist[0].g_hist, NUM_HISTOGRAM_BINS)); + /* + * All region sums are based on a 13-bit pipeline bit-depth. Normalise + * this to 16-bits for the AGC/AWB/ALSC algorithms. + */ statistics->awbRegions.init(DEFAULT_AWB_REGIONS_X, DEFAULT_AWB_REGIONS_Y); for (i = 0; i < statistics->awbRegions.numRegions(); i++) - statistics->awbRegions.set(i, { stats->awb_stats[i].r_sum, - stats->awb_stats[i].g_sum, - stats->awb_stats[i].b_sum }, + statistics->awbRegions.set(i, { stats->awb_stats[i].r_sum << 3, + stats->awb_stats[i].g_sum << 3, + stats->awb_stats[i].b_sum << 3 }, stats->awb_stats[i].counted, stats->awb_stats[i].notcounted); - /* There are only ever 15 regions computed by the firmware, but the HW defines AGC_REGIONS == 16! */ + /* + * There are only ever 15 regions computed by the firmware due to zoning, + * but the HW defines AGC_REGIONS == 16! + */ statistics->agcRegions.init(15); for (i = 0; i < statistics->agcRegions.numRegions(); i++) - statistics->agcRegions.set(i, { stats->agc_stats[i].r_sum, - stats->agc_stats[i].g_sum, - stats->agc_stats[i].b_sum }, + statistics->agcRegions.set(i, { stats->agc_stats[i].r_sum << 3, + stats->agc_stats[i].g_sum << 3, + stats->agc_stats[i].b_sum << 3 }, stats->agc_stats[i].counted, 0); statistics->focusRegions.init(4, 3);