From patchwork Tue Dec 13 11:48:32 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Naushir Patuck X-Patchwork-Id: 17996 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id B8A91C328D for ; Tue, 13 Dec 2022 11:48:51 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id A809963360; Tue, 13 Dec 2022 12:48:50 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1670932130; bh=VUsG/0hDy7HJda57ZvAicpnKqWLEc8nGuOSYSLpEzLQ=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To: From; b=s8SmKqwyG2NuSlwsFyTGcGzfwY2XCfp+gMFxoTo+bOYPmF2Tx7twkX63/N/XljQs3 HWBQdrGY2M9pwl42FhCDSJ0kIrwasRmD+fqSxuKhTfpZP9JHWlYAOr+oHfBNxa3cC1 PC8PizSIuCmsDAt7m2AIHeJSRIpf0c2t8BJM8o+5vJMS0Bz/1IGFhUOqyFfEi6xQTl z5+XWf2BEWSfrbgFyY3j0ZtXjJ27YxQHGgdMlLeVj2KxmTueVFL8yBVOnilox6NTCH 5DBHezje+TReU2TFu52FKAcI7agvwUqgO4aMFOpjnrKg0ya6DjQpASMMLmgCceMuWf nNJ9Dr/IYaHZQ== Received: from mail-wr1-x42f.google.com (mail-wr1-x42f.google.com [IPv6:2a00:1450:4864:20::42f]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 66F3B63354 for ; Tue, 13 Dec 2022 12:48:48 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="j1thFQVn"; dkim-atps=neutral Received: by mail-wr1-x42f.google.com with SMTP id o5so15333346wrm.1 for ; Tue, 13 Dec 2022 03:48:48 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=UzGll8qwMNu1ew3kASktX78GxzRx0n9eVyC6tPz+/7A=; b=j1thFQVnggX0tyOFdDiaD67k8ZG0YCXNMI/LGLjcGMqRTEdQNkKKE9XhUgRplST+q/ tVjavjlb8Y8K8fOsLqKbRSXNk5tZ+vUkC2B/CfoFLiwD/GQP9JUo+T0tKJcGr5xFkEtA czomtR1jLZo3sX96ifn/pIXKKEfGIYPWNe/18bjopHubvyHUHiVLFB4XYZSGmfJqO1L/ Wmj36YxqzHJWnDVwwVE4637frXlg2R1p1eFo6a8huJR5rXD5MLrluEzbd+dLSUZfyg+/ 3YzkTkNBfEZbMRtVKlLq80++xxq7oJXo1IN9NkxKIea40su8UUuKNCC4Jwc7k0Il74CJ I3LA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=UzGll8qwMNu1ew3kASktX78GxzRx0n9eVyC6tPz+/7A=; b=4lQgn9PT1mvKwL5D+joWorLFo5cXnGiWgu4L5Apqj640KB1X355HkHMvMeywSIEDBJ 3V8EOn9fR7G6rYh6TqnQmQUoUEpmubIo7GWDayOcnUi8OLIsvOaZ8tcwrSFrEn4PgtJy bSTZnXmWYPmuy+0nUTU80wtHcXiL+6wumHRvBIoIqsMuCTKy2eHYRarb21+9ncninsdb 0S1FCvwL+6dICjfhri94+YV5ruGkSWOYIk0EZkPbqV1dn5mmmxT8lxQ3Zdku/STrrEbK jGSmc1JRT75BzEKpT5Cd99ZY8zraQTcty2bn45LLIUncIB+S2y3vItvc0M7wxTTRsPog BFpQ== X-Gm-Message-State: ANoB5plpXkqB4nLt76isgJ4m1hGCPY2hFCfa/r2Yo7sFdgABVc0sTzlN NFVqiTCYSsQRpqMLRMtPfw9nUvrPIabPKncO X-Google-Smtp-Source: AA0mqf4kdmOCIXI25SGf8+96JF5RuIQmuhCRCPO4Bh6mpZ/uPEEnrXp6jJBPAa2LNAy8CuE2roraIQ== X-Received: by 2002:adf:dfc2:0:b0:242:5877:1ebc with SMTP id q2-20020adfdfc2000000b0024258771ebcmr12117212wrn.33.1670932127757; Tue, 13 Dec 2022 03:48:47 -0800 (PST) Received: from naush-laptop.localdomain ([93.93.133.154]) by smtp.gmail.com with ESMTPSA id j8-20020a5d4528000000b0024287d9d4a8sm11574541wra.74.2022.12.13.03.48.47 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 13 Dec 2022 03:48:47 -0800 (PST) To: libcamera-devel@lists.libcamera.org Date: Tue, 13 Dec 2022 11:48:32 +0000 Message-Id: <20221213114836.15473-2-naush@raspberrypi.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20221213114836.15473-1-naush@raspberrypi.com> References: <20221213114836.15473-1-naush@raspberrypi.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v3 1/5] ipa: raspberrypi: Generalise statistics X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Naushir Patuck via libcamera-devel From: Naushir Patuck Reply-To: Naushir Patuck Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" At present, the controller algorithms access the bcm2835_isp_stats structure, which is hardware specific. It would be desirable to abstract out the statistics structure to remove hardware specific headers from the algorithms source files. Define a new templated RegionStats class that encompasses region based statistics generated by the ISP. For the VC4 ISP, this can be used to hold RGB sums and focus FoM values. Define a new Statistics structure that holds all the VC4 ISP statistics output. This includes AGC histograms, AGC/AWB region sums and focus FoM regions. Signed-off-by: Naushir Patuck Reviewed-by: David Plowman Reviewed-by: Kieran Bingham --- src/ipa/raspberrypi/controller/region_stats.h | 123 ++++++++++++++++++ src/ipa/raspberrypi/statistics.h | 70 ++++++++++ 2 files changed, 193 insertions(+) create mode 100644 src/ipa/raspberrypi/controller/region_stats.h create mode 100644 src/ipa/raspberrypi/statistics.h diff --git a/src/ipa/raspberrypi/controller/region_stats.h b/src/ipa/raspberrypi/controller/region_stats.h new file mode 100644 index 000000000000..9aaf3a58a6f7 --- /dev/null +++ b/src/ipa/raspberrypi/controller/region_stats.h @@ -0,0 +1,123 @@ +/* SPDX-License-Identifier: BSD-2-Clause */ +/* + * Copyright (C) 2022, Raspberry Pi Ltd + * + * region_stats.h - Raspberry Pi region based statistics container + */ +#pragma once + +#include +#include +#include + +#include + +namespace RPiController { + +template +class RegionStats +{ +public: + struct Region { + T val; + uint32_t counted; + uint32_t uncounted; + }; + + RegionStats() + : size_({}), numFloating_(0), default_({}) + { + } + + void init(const libcamera::Size &size, unsigned int numFloating = 0) + { + size_ = size; + numFloating_ = numFloating; + regions_.clear(); + regions_.resize(size_.width * size_.height + numFloating_); + } + + void init(unsigned int num) + { + size_ = libcamera::Size(num, 1); + numFloating_ = 0; + regions_.clear(); + regions_.resize(num); + } + + unsigned int numRegions() const + { + return size_.width * size_.height; + } + + unsigned int numFloatingRegions() const + { + return numFloating_; + } + + libcamera::Size size() const + { + return size_; + } + + void set(unsigned int index, const Region ®ion) + { + if (index >= numRegions()) + return; + set_(index, region); + } + + void set(const libcamera::Point &pos, const Region ®ion) + { + set(pos.y * size_.width + pos.x, region); + } + + void setFloating(unsigned int index, const Region ®ion) + { + if (index >= numFloatingRegions()) + return; + set(numRegions() + index, region); + } + + const Region &get(unsigned int index) const + { + if (index >= numRegions()) + return default_; + return get_(index); + } + + const T &get(const libcamera::Point &pos) const + { + return get(pos.y * size_.width + pos.x); + } + + const T &getFloating(unsigned int index) const + { + if (index >= numFloatingRegions()) + return default_; + return get_(numRegions() + index); + } + + typename std::vector::iterator begin() { return regions_.begin(); } + typename std::vector::iterator end() { return regions_.end(); } + typename std::vector::const_iterator begin() const { return regions_.begin(); } + typename std::vector::const_iterator end() const { return regions_.end(); } + +private: + void set_(unsigned int index, const Region ®ion) + { + regions_[index] = region; + } + + const Region &get_(unsigned int index) const + { + return regions_[index]; + } + + libcamera::Size size_; + unsigned int numFloating_; + std::vector regions_; + Region default_; +}; + +} /* namespace RPiController */ diff --git a/src/ipa/raspberrypi/statistics.h b/src/ipa/raspberrypi/statistics.h new file mode 100644 index 000000000000..a762bf3d41aa --- /dev/null +++ b/src/ipa/raspberrypi/statistics.h @@ -0,0 +1,70 @@ +/* SPDX-License-Identifier: BSD-2-Clause */ +/* + * Copyright (C) 2022, Raspberry Pi Ltd + * + * statistics.h - Raspberry Pi generic statistics structure + */ +#pragma once + +#include +#include +#include + +#include "histogram.h" +#include "region_stats.h" + +namespace RPiController { + +struct RgbySums { + RgbySums(uint64_t _rSum = 0, uint64_t _gSum = 0, uint64_t _bSum = 0, uint64_t _ySum = 0) + : rSum(_rSum), gSum(_gSum), bSum(_bSum), ySum(_ySum) + { + } + uint64_t rSum; + uint64_t gSum; + uint64_t bSum; + uint64_t ySum; +}; + +using RgbyRegions = RegionStats; +using FocusRegions = RegionStats; + +struct Statistics { + /* + * Positioning of the AGC statistics gathering in the pipeline: + * Pre-WB correction or post-WB correction. + * Assume this is post-LSC. + */ + enum class AgcStatsPos { PreWb, PostWb }; + const AgcStatsPos agcStatsPos; + + /* + * Positioning of the AWB/ALSC statistics gathering in the pipeline: + * Pre-LSC or post-LSC. + */ + enum class ColourStatsPos { PreLsc, PostLsc }; + const ColourStatsPos colourStatsPos; + + Statistics(AgcStatsPos a, ColourStatsPos c) + : agcStatsPos(a), colourStatsPos(c) + { + } + + /* Histogram statistics. Not all histograms may be populated! */ + Histogram rHist; + Histogram gHist; + Histogram bHist; + Histogram yHist; + + /* Row sums for flicker avoidance. */ + std::vector rowSums; + + /* Region based colour sums. */ + RgbyRegions agcRegions; + RgbyRegions awbRegions; + + /* Region based focus FoM. */ + FocusRegions focusRegions; +}; + +} /* namespace RPiController */ From patchwork Tue Dec 13 11:48:33 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Naushir Patuck X-Patchwork-Id: 17997 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 60643BDE6B for ; Tue, 13 Dec 2022 11:48:53 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 0B61663362; Tue, 13 Dec 2022 12:48:53 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1670932133; bh=ma9szNikYEmGctrJ31Ey01v3NafeZYa8iuiY1l8LTwQ=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To: From; b=rcSXn2TzwO1vD9vE9E/6VlSzLAOEYoDNwS1Tt9TwSPG4+wmadMel9lcw7df3Ew/El LVkwyNCX10DEFlw5VoGaRfX03AI+AC8px+eqdwY65BHofsUpGau6lokOJRQ2vZbJSn uB668ZF9EdA31v+yNbVabjPOCOJbY0l+VyF5D8PZs2LGAgY5TCT+jQmKemYKCIpGys JicZP6hHqPeT4gGbJRyfxtBylLktXHDr289FKmy5/o3sUpDtQUIsquA0B2xwNDptIk XISk0IGegnk6L3WVSHpMlTABLbNr4vQ9bSPjcZ1mZrdIQRjEAZT4bgimi5Y8NfcdjZ 3yFSSyjSkeseA== Received: from mail-wr1-x433.google.com (mail-wr1-x433.google.com [IPv6:2a00:1450:4864:20::433]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 64E7063340 for ; Tue, 13 Dec 2022 12:48:49 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="seXBmToy"; dkim-atps=neutral Received: by mail-wr1-x433.google.com with SMTP id h7so15316231wrs.6 for ; Tue, 13 Dec 2022 03:48:49 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=Oz62LSxuhaQPLtvLBoBKpt4oVpu0W2M+xPyY7uBTRX0=; b=seXBmToy2BJzXhxPyODicg1DeEJDr4nGeX78SSQVwYM/DnHHgOipXPlQXgh7sji62S Cd6hAqjusDdwAMqk1urvi/AwD6HB+5O6P5DRLeiO2LqvXcLwigCyXUDJK0VXd3PZLGGI 4+6gyrLXZPJljdYYEKHD/HJXmmygHuqgEkPKhQIRd+kKj0Nkxkhk+CjT5rR8daBJBIBW StPO82p/7mrU1USAlFKN5LHmZWq4phZwkplnb5ortABVSJmyJPjpGIVycy/vGRJ+Kt4e +Lvg0squjvacHM3OpBwnfG4PcO+ZmhWkcWakggX35CHdS05ixQwwmhMnVnjc+o+f3Hft 4fCA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=Oz62LSxuhaQPLtvLBoBKpt4oVpu0W2M+xPyY7uBTRX0=; b=a5+FnXUwVW50kX6/V4D0Qi+83K011Ps0WcMvvffKS6aFgMgMdqi3O+uJ0KEGlbZ+tF BJezflCa++KZxZpPODrMmvRYg6Vutfk8mgdUJPkKt/6tDppWHi0K3Snym/J7QzqGFjif CPvBalVYy2y7yLReN4/ORJJ/3MaV5UNwh1SU/fPsQ6PwdNM8Cy9ytZx51MBujNCAfCv1 rqm6c05QILqKkbOF5GPV9iHCwYDUGOGwoZ1SJBVjJZFAb7tJqdtLptiRccDwviR81LP7 JB1JFHom5CrXzXLavJoOfJM4sKCfWObWS6bUk00PNEbuGFNkLbtdX7O0R31oZPTjiLUF jbVQ== X-Gm-Message-State: ANoB5pmwaeK5t/hlK80vehctmlEzLm52HUq9YUoWQMara7oVOPFdl4c9 qd+Q0zdq/IqBEMolD7xYD7epOGUNQriuGJqj X-Google-Smtp-Source: AA0mqf702/BAWQyzB7P80fCnQQu70XzTWtFXYd6yJadG5KvtIoSS728OZXFXjajKqx12AtP/u8rfpw== X-Received: by 2002:a5d:6045:0:b0:242:24a7:c7f2 with SMTP id j5-20020a5d6045000000b0024224a7c7f2mr11822147wrt.58.1670932128681; Tue, 13 Dec 2022 03:48:48 -0800 (PST) Received: from naush-laptop.localdomain ([93.93.133.154]) by smtp.gmail.com with ESMTPSA id j8-20020a5d4528000000b0024287d9d4a8sm11574541wra.74.2022.12.13.03.48.47 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 13 Dec 2022 03:48:48 -0800 (PST) To: libcamera-devel@lists.libcamera.org Date: Tue, 13 Dec 2022 11:48:33 +0000 Message-Id: <20221213114836.15473-3-naush@raspberrypi.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20221213114836.15473-1-naush@raspberrypi.com> References: <20221213114836.15473-1-naush@raspberrypi.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v3 2/5] ipa: raspberrypi: histogram: Add a constructor for an empty histogram X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Naushir Patuck via libcamera-devel From: Naushir Patuck Reply-To: Naushir Patuck Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Add a default constructor to the RPiController::Histogram class that creates an empty histogram. Since this is a cumulative histogram, push a value of 0 into the first (and only) bin to signify this. Signed-off-by: Naushir Patuck Reviewed-by: David Plowman Reviewed-by: Kieran Bingham --- src/ipa/raspberrypi/controller/histogram.h | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/src/ipa/raspberrypi/controller/histogram.h b/src/ipa/raspberrypi/controller/histogram.h index 66a68b087964..6b3e3a9e68e4 100644 --- a/src/ipa/raspberrypi/controller/histogram.h +++ b/src/ipa/raspberrypi/controller/histogram.h @@ -20,6 +20,11 @@ namespace RPiController { class Histogram { public: + Histogram() + { + cumulative_.push_back(0); + } + template Histogram(T *histogram, int num) { assert(num); From patchwork Tue Dec 13 11:48:34 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Naushir Patuck X-Patchwork-Id: 17998 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id CE3B3C328D for ; Tue, 13 Dec 2022 11:48:53 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 81D326336A; Tue, 13 Dec 2022 12:48:53 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1670932133; bh=WGDI8twxcdHAKJzJ5im7ArFW9wvzj17s+nghLwrJXYo=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To: From; b=UsViOkovCQg4tNZ1oC0vifu5XdE6H0N+vpsOEdb3nq4q1ZViKiBH8sF+fdNEhZiEu mgoHx1drvXHW6Dzcv9i4XPHobeTzzBJrbbDzb3kM+4zFuNY/CIrsW5OoQctYaY9ACx QkkOG+6ZRrvqXD6nnbutLfYyWdhik3eHmTuiihMrzWDZTpI1Yezp/rtuADcEp14PVY fjjnmgBh/zxqfhITnsldrSKDvACgBq1RZdlad/jONF6ZWUjiz/n3SuwNYXs+S4t1kC jyF4r6gA1Emkk5cziLw1SYp03MGDyNM3FoEy1bSUJHiR6ISQDjWdBuA8WI+X332Eee N51+5cDFChVRw== Received: from mail-wr1-x431.google.com (mail-wr1-x431.google.com [IPv6:2a00:1450:4864:20::431]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 85ADE63364 for ; Tue, 13 Dec 2022 12:48:50 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="folMdn4E"; dkim-atps=neutral Received: by mail-wr1-x431.google.com with SMTP id f18so15312831wrj.5 for ; Tue, 13 Dec 2022 03:48:50 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=2xu9NvIf3FFVhTiDKKJMfUfn/nFLA9UTQHonQ3imTOM=; b=folMdn4E0rlqAYw4PoNyIB2dsKvyUxhU+q1XNE8Tid4IIY8/nwFtvVVgVis2IFoMll C+8ExS+KFT3IPqUFEC3BolN7LaHmOOaklJYG1Vkg5iTgjaBhswJ6UK8yJACzV7+2oZyS 3KqculpCWNlJub3LvxhEuoT5HzSm5NpXvj4IviRqwXGjRZLPtqhoF34J3rmyPPJ9V1J/ fFWg9SYwkGNMnGlfIpZFhxfFx5CQ4hPbGLM8OZoiIdTPiliuBhDsUK+y911cy3Jy3RQt GuvWdRUdI8I+EaZGRZCVZetA0N6B1D0HLxRZ6SRyB2Nbl8NpnmqzVr80ctEb+FbFEyBJ Qsyg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=2xu9NvIf3FFVhTiDKKJMfUfn/nFLA9UTQHonQ3imTOM=; b=SHK3O4m9zApSi4IA3OYinVIl3DdAGLYuw6ssIh2S9yNRGaYn74HNsxelnStT5SgVyw m9DcOuQM/uAfy25Sddm0o5uHzZOFU2+1y4WLeHsHIWPOxyaOSbFV5sRbZis2DeD8cXZ2 lZTFlrK9ob3Y7vyePz035FpZqMUtNlmedTS97EZ3pngYaW+0iwFZBsF8BP5wvvnkYhPp kVLLBo6H6Dq/nbGX28Zxn8LVIgimmQ7MkDezEfNNtHLWuiBEQL01iCS27OPXHVyKSYE4 NewnDrh0vmh0g+OIaxwNqcN45D18ek2TNUn6dXYpY9AsucfALNejdUE4MsVA4hLOkb7X D7rg== X-Gm-Message-State: ANoB5plukVHfBmCitt1Djd6owzfCUelybwL5QuWn+WsfmjCSrHpIDu4Z gnIKk/gmJYcjJpp6JWizxhCONGrLK13xN8kD X-Google-Smtp-Source: AA0mqf4NqdZGw3ZyMXJsvBQItsxB+ZtGNOTGSPmVa+iZy3DFd+dvZROsFbp/Pcxb0PsmlAFV1WLbiA== X-Received: by 2002:adf:f10f:0:b0:242:19b3:67 with SMTP id r15-20020adff10f000000b0024219b30067mr12059414wro.37.1670932129588; Tue, 13 Dec 2022 03:48:49 -0800 (PST) Received: from naush-laptop.localdomain ([93.93.133.154]) by smtp.gmail.com with ESMTPSA id j8-20020a5d4528000000b0024287d9d4a8sm11574541wra.74.2022.12.13.03.48.48 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 13 Dec 2022 03:48:48 -0800 (PST) To: libcamera-devel@lists.libcamera.org Date: Tue, 13 Dec 2022 11:48:34 +0000 Message-Id: <20221213114836.15473-4-naush@raspberrypi.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20221213114836.15473-1-naush@raspberrypi.com> References: <20221213114836.15473-1-naush@raspberrypi.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v3 3/5] ipa: raspberrypi: awb: Delay release of the statistics buffer X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Naushir Patuck via libcamera-devel From: Naushir Patuck Reply-To: Naushir Patuck Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Release the statistics buffer after running the through the AWB calculations. Only the "counted" statistics are copied out to a local structure, so keeping the statistics buffer allows the algorithm to see the "uncounted" statistics as well. This is currently handled by hard-coding the total number of statistics regions regions based on the structure definition in the bcm2835_isp_stats structure. Signed-off-by: Naushir Patuck Reviewed-by: David Plowman Reviewed-by: Kieran Bingham --- src/ipa/raspberrypi/controller/rpi/awb.cpp | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/src/ipa/raspberrypi/controller/rpi/awb.cpp b/src/ipa/raspberrypi/controller/rpi/awb.cpp index 4f6af4ba98dc..04d1c8783654 100644 --- a/src/ipa/raspberrypi/controller/rpi/awb.cpp +++ b/src/ipa/raspberrypi/controller/rpi/awb.cpp @@ -427,11 +427,6 @@ void Awb::prepareStats() */ generateStats(zones_, statistics_->awb_stats, config_.minPixels, config_.minG); - /* - * we're done with these; we may as well relinquish our hold on the - * pointer. - */ - statistics_.reset(); /* * apply sensitivities, so values appear to come from our "canonical" * sensor. @@ -728,6 +723,11 @@ void Awb::doAwb() << " with gains r " << asyncResults_.gainR << " and b " << asyncResults_.gainB; } + /* + * we're done with these; we may as well relinquish our hold on the + * pointer. + */ + statistics_.reset(); } /* Register algorithm with the system. */ From patchwork Tue Dec 13 11:48:35 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Naushir Patuck X-Patchwork-Id: 17999 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 63031C328E for ; Tue, 13 Dec 2022 11:48:54 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id D4FCD63371; Tue, 13 Dec 2022 12:48:53 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1670932133; bh=o8PWOqsX2TB940Z9Zj46tNl7Fc2kzvOV4K2rRYslMTQ=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To: From; b=BW5C1P2TdU0FUwEjAMJZhr+VQq3ui+k8QBT/FUTq1DVdTWNq1nLXz7crgCjlgF0wm Ia4vltbnz2D6fRYR2M9E2ORZNIIqctIVrOItSuisNfF1CSrPO3BY9KK3EStpS8s2n8 8X/grT4MbDsk4YrPZdVDyaduo/rxcYYvWpWPknNwTOufw0SuS3wzJxPnhWQvZhy08l DY4alUQJd2nYfN205+dojUAWlb+ZlT12T94moydrEBHa3//rZp4wRF04M8Uf8NUSQN AEW7w+hxzQBerDg6YbXVljwH9JBIyspeNReVqYjNEZd9HE9tGLSVkwb5tDXA37vzdR II28FbFyM5IIw== Received: from mail-wr1-x436.google.com (mail-wr1-x436.google.com [IPv6:2a00:1450:4864:20::436]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 707C563340 for ; Tue, 13 Dec 2022 12:48:51 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="t2BcTUDq"; dkim-atps=neutral Received: by mail-wr1-x436.google.com with SMTP id co23so15326911wrb.4 for ; Tue, 13 Dec 2022 03:48:51 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=v0x8ty6DgF46kagMZdOgouR/aj8d7hgIU3D2SW9xejs=; b=t2BcTUDqyueTTBP6QETjsc0j66TgFRfh+ZvGkgdFfEJWrOUOWJnQCQEViCQrvG97NY 2VH7Z+qpTKJBoiwXS1nwBLYP83c5Qy9R6HJcmLawB8EVNDLSzLIr8BK0el4ugH1On2qj aJ6jBiYxCoXOmEV2sX2hwtgreuMLrr/4MncyR9p/rJaHaXqJzDJc110kxuFqRb3HC1Oj 7/s+GHGsemXAAJGKrIvNqPMqk7xCTxCCXgcwc3gpkW7/R9qIdj0GECyT6RRGTDyivCZF ZTE6nDjRq7q4Xc5hPBzhIY5VdpjM39/GTJg0LeS8HCClbzXXO7cy1ndJWSVT8L4/SXZf M97w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=v0x8ty6DgF46kagMZdOgouR/aj8d7hgIU3D2SW9xejs=; b=Lnrys3UsAX04ekLRMaS0Ml0ZcsZiIaytqsm9BCU/GB4wDBwMQg9BQMv5/CoJZmel0R 06VVMWw70FJyQJTmSPhNJe/BlJtMjI36V6WweFyXHSxq18FsaJtrvv9V7p/kOPhlej4j A0aJC6/tvq0r2TbLZF4yPulFfhlkdNzqPieSrtUgXj/VYVwJhraszTesLwKzw7GTdZIw Rn6GtOJBthyB/oL9kwltKav9dI7xAt7KORmnaSbjFFvvOd4lVOalKfa9NAszv83hL/hl 1mSDCBeNO5ctAVHK5NNqj2rJzG/RiqM7Txz3XyZNCSkfi7H6Loz6HXjraiQ12T7qbFlc h4og== X-Gm-Message-State: ANoB5pkV9oPePU9hB86E4HkngCdiINRvD8CymtU0VZtxpx6QrWJbzij3 bd7Bd7nX7EVZF2o1ufajDfLTO/RXedAcm8ou X-Google-Smtp-Source: AA0mqf6m9oWQpJF8alPa/IwVR0+Rw0DBKDIV2oV/xcEIxUFBScbV8csh5EcpbG0f1LzDKDUxihuUqg== X-Received: by 2002:a5d:5405:0:b0:24a:e56d:e2f with SMTP id g5-20020a5d5405000000b0024ae56d0e2fmr8387033wrv.24.1670932130608; Tue, 13 Dec 2022 03:48:50 -0800 (PST) Received: from naush-laptop.localdomain ([93.93.133.154]) by smtp.gmail.com with ESMTPSA id j8-20020a5d4528000000b0024287d9d4a8sm11574541wra.74.2022.12.13.03.48.49 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 13 Dec 2022 03:48:49 -0800 (PST) To: libcamera-devel@lists.libcamera.org Date: Tue, 13 Dec 2022 11:48:35 +0000 Message-Id: <20221213114836.15473-5-naush@raspberrypi.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20221213114836.15473-1-naush@raspberrypi.com> References: <20221213114836.15473-1-naush@raspberrypi.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v3 4/5] ipa: raspberrypi: Use the generic statistics structure in the algorithms X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Naushir Patuck via libcamera-devel From: Naushir Patuck Reply-To: Naushir Patuck Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Repurpose the StatisticsPtr type from being a shared_ptr to shared_ptr. This removes any hardware specific header files and structures from the algorithms source code. Add a new function in the Raspberry Pi IPA to populate the generic statistics structure from the values provided by the hardware in the bcm2835_isp_stats structure. Update the Lux, AWB, AGC, ALSC, Contrast, and Focus algorithms to use the generic statistics structure appropriately in their calculations. Additionally, remove references to any hardware specific headers and defines in these source files. Signed-off-by: Naushir Patuck Reviewed-by: David Plowman Reviewed-by: Kieran Bingham --- src/ipa/raspberrypi/controller/controller.h | 4 +- src/ipa/raspberrypi/controller/rpi/agc.cpp | 27 +++++------- src/ipa/raspberrypi/controller/rpi/agc.h | 2 +- src/ipa/raspberrypi/controller/rpi/alsc.cpp | 32 +++++++------- src/ipa/raspberrypi/controller/rpi/alsc.h | 3 +- src/ipa/raspberrypi/controller/rpi/awb.cpp | 20 ++++----- src/ipa/raspberrypi/controller/rpi/awb.h | 1 + .../raspberrypi/controller/rpi/contrast.cpp | 8 ++-- src/ipa/raspberrypi/controller/rpi/focus.cpp | 7 ++- src/ipa/raspberrypi/controller/rpi/lux.cpp | 14 +----- src/ipa/raspberrypi/raspberrypi.cpp | 44 ++++++++++++++++++- src/ipa/raspberrypi/statistics.h | 2 + 12 files changed, 96 insertions(+), 68 deletions(-) diff --git a/src/ipa/raspberrypi/controller/controller.h b/src/ipa/raspberrypi/controller/controller.h index 3e1e051703b3..e6c950c3a509 100644 --- a/src/ipa/raspberrypi/controller/controller.h +++ b/src/ipa/raspberrypi/controller/controller.h @@ -15,19 +15,17 @@ #include #include -#include - #include "libcamera/internal/yaml_parser.h" #include "camera_mode.h" #include "device_status.h" #include "metadata.h" +#include "statistics.h" namespace RPiController { class Algorithm; typedef std::unique_ptr AlgorithmPtr; -typedef std::shared_ptr StatisticsPtr; /* * The Controller holds a pointer to some global_metadata, which is how diff --git a/src/ipa/raspberrypi/controller/rpi/agc.cpp b/src/ipa/raspberrypi/controller/rpi/agc.cpp index 46dcc81ae14c..868c30f03d66 100644 --- a/src/ipa/raspberrypi/controller/rpi/agc.cpp +++ b/src/ipa/raspberrypi/controller/rpi/agc.cpp @@ -9,8 +9,6 @@ #include #include -#include - #include #include "../awb_status.h" @@ -451,7 +449,7 @@ void Agc::process(StatisticsPtr &stats, Metadata *imageMetadata) fetchCurrentExposure(imageMetadata); /* Compute the total gain we require relative to the current exposure. */ double gain, targetY; - computeGain(stats.get(), imageMetadata, gain, targetY); + computeGain(stats, imageMetadata, gain, targetY); /* Now compute the target (final) exposure which we think we want. */ computeTargetExposure(gain); /* @@ -585,24 +583,23 @@ void Agc::fetchAwbStatus(Metadata *imageMetadata) LOG(RPiAgc, Debug) << "No AWB status found"; } -static double computeInitialY(bcm2835_isp_stats *stats, AwbStatus const &awb, +static double computeInitialY(StatisticsPtr &stats, AwbStatus const &awb, double weights[], double gain) { - bcm2835_isp_stats_region *regions = stats->agc_stats; /* * Note how the calculation below means that equal weights give you * "average" metering (i.e. all pixels equally important). */ double rSum = 0, gSum = 0, bSum = 0, pixelSum = 0; - for (unsigned int i = 0; i < AgcStatsSize; i++) { - double counted = regions[i].counted; - double rAcc = std::min(regions[i].r_sum * gain, ((1 << PipelineBits) - 1) * counted); - double gAcc = std::min(regions[i].g_sum * gain, ((1 << PipelineBits) - 1) * counted); - double bAcc = std::min(regions[i].b_sum * gain, ((1 << PipelineBits) - 1) * counted); + for (unsigned int i = 0; i < stats->agcRegions.numRegions(); i++) { + auto ®ion = stats->agcRegions.get(i); + double rAcc = std::min(region.val.rSum * gain, ((1 << PipelineBits) - 1) * region.counted); + double gAcc = std::min(region.val.gSum * gain, ((1 << PipelineBits) - 1) * region.counted); + double bAcc = std::min(region.val.bSum * gain, ((1 << PipelineBits) - 1) * region.counted); rSum += rAcc * weights[i]; gSum += gAcc * weights[i]; bSum += bAcc * weights[i]; - pixelSum += counted * weights[i]; + pixelSum += region.counted * weights[i]; } if (pixelSum == 0.0) { LOG(RPiAgc, Warning) << "computeInitialY: pixelSum is zero"; @@ -624,23 +621,23 @@ static double computeInitialY(bcm2835_isp_stats *stats, AwbStatus const &awb, static constexpr double EvGainYTargetLimit = 0.9; -static double constraintComputeGain(AgcConstraint &c, Histogram &h, double lux, +static double constraintComputeGain(AgcConstraint &c, const Histogram &h, double lux, double evGain, double &targetY) { targetY = c.yTarget.eval(c.yTarget.domain().clip(lux)); targetY = std::min(EvGainYTargetLimit, targetY * evGain); double iqm = h.interQuantileMean(c.qLo, c.qHi); - return (targetY * NUM_HISTOGRAM_BINS) / iqm; + return (targetY * h.bins()) / iqm; } -void Agc::computeGain(bcm2835_isp_stats *statistics, Metadata *imageMetadata, +void Agc::computeGain(StatisticsPtr &statistics, Metadata *imageMetadata, double &gain, double &targetY) { struct LuxStatus lux = {}; lux.lux = 400; /* default lux level to 400 in case no metadata found */ if (imageMetadata->get("lux.status", lux) != 0) LOG(RPiAgc, Warning) << "No lux level found"; - Histogram h(statistics->hist[0].g_hist, NUM_HISTOGRAM_BINS); + const Histogram &h = statistics->yHist; double evGain = status_.ev * config_.baseEv; /* * The initial gain and target_Y come from some of the regions. After diff --git a/src/ipa/raspberrypi/controller/rpi/agc.h b/src/ipa/raspberrypi/controller/rpi/agc.h index cf04da1973f1..f04896ca25ad 100644 --- a/src/ipa/raspberrypi/controller/rpi/agc.h +++ b/src/ipa/raspberrypi/controller/rpi/agc.h @@ -96,7 +96,7 @@ private: void housekeepConfig(); void fetchCurrentExposure(Metadata *imageMetadata); void fetchAwbStatus(Metadata *imageMetadata); - void computeGain(bcm2835_isp_stats *statistics, Metadata *imageMetadata, + void computeGain(StatisticsPtr &statistics, Metadata *imageMetadata, double &gain, double &targetY); void computeTargetExposure(double gain); bool applyDigitalGain(double gain, double targetY); diff --git a/src/ipa/raspberrypi/controller/rpi/alsc.cpp b/src/ipa/raspberrypi/controller/rpi/alsc.cpp index a4afaf841c41..eb4e2f9496e1 100644 --- a/src/ipa/raspberrypi/controller/rpi/alsc.cpp +++ b/src/ipa/raspberrypi/controller/rpi/alsc.cpp @@ -310,19 +310,21 @@ double getCt(Metadata *metadata, double defaultCt) return awbStatus.temperatureK; } -static void copyStats(bcm2835_isp_stats_region regions[XY], StatisticsPtr &stats, +static void copyStats(RgbyRegions ®ions, StatisticsPtr &stats, AlscStatus const &status) { - bcm2835_isp_stats_region *inputRegions = stats->awb_stats; + if (!regions.numRegions()) + regions.init(stats->awbRegions.size()); + double *rTable = (double *)status.r; double *gTable = (double *)status.g; double *bTable = (double *)status.b; - for (int i = 0; i < XY; i++) { - regions[i].r_sum = inputRegions[i].r_sum / rTable[i]; - regions[i].g_sum = inputRegions[i].g_sum / gTable[i]; - regions[i].b_sum = inputRegions[i].b_sum / bTable[i]; - regions[i].counted = inputRegions[i].counted; - /* (don't care about the uncounted value) */ + for (unsigned int i = 0; i < stats->awbRegions.numRegions(); i++) { + auto r = stats->awbRegions.get(i); + r.val.rSum = static_cast(r.val.rSum / rTable[i]); + r.val.gSum = static_cast(r.val.gSum / gTable[i]); + r.val.bSum = static_cast(r.val.bSum / bTable[i]); + regions.set(i, r); } } @@ -512,19 +514,19 @@ void resampleCalTable(double const calTableIn[XY], } /* Calculate chrominance statistics (R/G and B/G) for each region. */ -static_assert(XY == AWB_REGIONS, "ALSC/AWB statistics region mismatch"); -static void calculateCrCb(bcm2835_isp_stats_region *awbRegion, double cr[XY], +static void calculateCrCb(const RgbyRegions &awbRegion, double cr[XY], double cb[XY], uint32_t minCount, uint16_t minG) { for (int i = 0; i < XY; i++) { - bcm2835_isp_stats_region &zone = awbRegion[i]; - if (zone.counted <= minCount || - zone.g_sum / zone.counted <= minG) { + auto s = awbRegion.get(i); + + if (s.counted <= minCount || s.val.gSum / s.counted <= minG) { cr[i] = cb[i] = InsufficientData; continue; } - cr[i] = zone.r_sum / (double)zone.g_sum; - cb[i] = zone.b_sum / (double)zone.g_sum; + + cr[i] = s.val.rSum / (double)s.val.gSum; + cb[i] = s.val.bSum / (double)s.val.gSum; } } diff --git a/src/ipa/raspberrypi/controller/rpi/alsc.h b/src/ipa/raspberrypi/controller/rpi/alsc.h index a858ef5a6551..9167c9ffa2e3 100644 --- a/src/ipa/raspberrypi/controller/rpi/alsc.h +++ b/src/ipa/raspberrypi/controller/rpi/alsc.h @@ -12,6 +12,7 @@ #include "../algorithm.h" #include "../alsc_status.h" +#include "../statistics.h" namespace RPiController { @@ -98,7 +99,7 @@ private: /* copy out the results from the async thread so that it can be restarted */ void fetchAsyncResults(); double ct_; - bcm2835_isp_stats_region statistics_[AlscCellsY * AlscCellsX]; + RgbyRegions statistics_; double asyncResults_[3][AlscCellsY][AlscCellsX]; double asyncLambdaR_[AlscCellsX * AlscCellsY]; double asyncLambdaB_[AlscCellsX * AlscCellsY]; diff --git a/src/ipa/raspberrypi/controller/rpi/awb.cpp b/src/ipa/raspberrypi/controller/rpi/awb.cpp index 04d1c8783654..ef3435d66106 100644 --- a/src/ipa/raspberrypi/controller/rpi/awb.cpp +++ b/src/ipa/raspberrypi/controller/rpi/awb.cpp @@ -21,9 +21,6 @@ LOG_DEFINE_CATEGORY(RPiAwb) #define NAME "rpi.awb" -static constexpr unsigned int AwbStatsSizeX = DEFAULT_AWB_REGIONS_X; -static constexpr unsigned int AwbStatsSizeY = DEFAULT_AWB_REGIONS_Y; - /* * todo - the locking in this algorithm needs some tidying up as has been done * elsewhere (ALSC and AGC). @@ -401,17 +398,16 @@ void Awb::asyncFunc() } static void generateStats(std::vector &zones, - bcm2835_isp_stats_region *stats, double minPixels, + RgbyRegions &stats, double minPixels, double minG) { - for (unsigned int i = 0; i < AwbStatsSizeX * AwbStatsSizeY; i++) { + for (auto const ®ion : stats) { Awb::RGB zone; - double counted = stats[i].counted; - if (counted >= minPixels) { - zone.G = stats[i].g_sum / counted; + if (region.counted >= minPixels) { + zone.G = region.val.gSum / region.counted; if (zone.G >= minG) { - zone.R = stats[i].r_sum / counted; - zone.B = stats[i].b_sum / counted; + zone.R = region.val.rSum / region.counted; + zone.B = region.val.bSum / region.counted; zones.push_back(zone); } } @@ -425,7 +421,7 @@ void Awb::prepareStats() * LSC has already been applied to the stats in this pipeline, so stop * any LSC compensation. We also ignore config_.fast in this version. */ - generateStats(zones_, statistics_->awb_stats, config_.minPixels, + generateStats(zones_, statistics_->awbRegions, config_.minPixels, config_.minG); /* * apply sensitivities, so values appear to come from our "canonical" @@ -641,7 +637,7 @@ void Awb::awbBayes() * valid... not entirely sure about this. */ Pwl prior = interpolatePrior(); - prior *= zones_.size() / (double)(AwbStatsSizeX * AwbStatsSizeY); + prior *= zones_.size() / (double)(statistics_->awbRegions.numRegions()); prior.map([](double x, double y) { LOG(RPiAwb, Debug) << "(" << x << "," << y << ")"; }); diff --git a/src/ipa/raspberrypi/controller/rpi/awb.h b/src/ipa/raspberrypi/controller/rpi/awb.h index 2254c3ed2cc4..e7d49cd8036b 100644 --- a/src/ipa/raspberrypi/controller/rpi/awb.h +++ b/src/ipa/raspberrypi/controller/rpi/awb.h @@ -13,6 +13,7 @@ #include "../awb_algorithm.h" #include "../pwl.h" #include "../awb_status.h" +#include "../statistics.h" namespace RPiController { diff --git a/src/ipa/raspberrypi/controller/rpi/contrast.cpp b/src/ipa/raspberrypi/controller/rpi/contrast.cpp index 5b37edcbd46a..a4f8c4f04fc4 100644 --- a/src/ipa/raspberrypi/controller/rpi/contrast.cpp +++ b/src/ipa/raspberrypi/controller/rpi/contrast.cpp @@ -106,7 +106,7 @@ Pwl computeStretchCurve(Histogram const &histogram, * bit. */ double histLo = histogram.quantile(config.loHistogram) * - (65536 / NUM_HISTOGRAM_BINS); + (65536 / histogram.bins()); double levelLo = config.loLevel * 65536; LOG(RPiContrast, Debug) << "Move histogram point " << histLo << " to " << levelLo; @@ -119,7 +119,7 @@ Pwl computeStretchCurve(Histogram const &histogram, * Keep the mid-point (median) in the same place, though, to limit the * apparent amount of global brightness shift. */ - double mid = histogram.quantile(0.5) * (65536 / NUM_HISTOGRAM_BINS); + double mid = histogram.quantile(0.5) * (65536 / histogram.bins()); enhance.append(mid, mid); /* @@ -127,7 +127,7 @@ Pwl computeStretchCurve(Histogram const &histogram, * there up. */ double histHi = histogram.quantile(config.hiHistogram) * - (65536 / NUM_HISTOGRAM_BINS); + (65536 / histogram.bins()); double levelHi = config.hiLevel * 65536; LOG(RPiContrast, Debug) << "Move histogram point " << histHi << " to " << levelHi; @@ -158,7 +158,7 @@ Pwl applyManualContrast(Pwl const &gammaCurve, double brightness, void Contrast::process(StatisticsPtr &stats, [[maybe_unused]] Metadata *imageMetadata) { - Histogram histogram(stats->hist[0].g_hist, NUM_HISTOGRAM_BINS); + Histogram &histogram = stats->yHist; /* * We look at the histogram and adjust the gamma curve in the following * ways: 1. Adjust the gamma curve so as to pull the start of the diff --git a/src/ipa/raspberrypi/controller/rpi/focus.cpp b/src/ipa/raspberrypi/controller/rpi/focus.cpp index 8c5029bd0e95..ea3cc00e42c3 100644 --- a/src/ipa/raspberrypi/controller/rpi/focus.cpp +++ b/src/ipa/raspberrypi/controller/rpi/focus.cpp @@ -31,10 +31,9 @@ char const *Focus::name() const void Focus::process(StatisticsPtr &stats, Metadata *imageMetadata) { FocusStatus status; - unsigned int i; - for (i = 0; i < FOCUS_REGIONS; i++) - status.focusMeasures[i] = stats->focus_stats[i].contrast_val[1][1] / 1000; - status.num = i; + for (unsigned int i = 0; i < stats->focusRegions.numRegions(); i++) + status.focusMeasures[i] = stats->focusRegions.get(i).val; + status.num = stats->focusRegions.numRegions(); imageMetadata->set("focus.status", status); LOG(RPiFocus, Debug) diff --git a/src/ipa/raspberrypi/controller/rpi/lux.cpp b/src/ipa/raspberrypi/controller/rpi/lux.cpp index 9759186afacf..06625f3a5ea3 100644 --- a/src/ipa/raspberrypi/controller/rpi/lux.cpp +++ b/src/ipa/raspberrypi/controller/rpi/lux.cpp @@ -6,8 +6,6 @@ */ #include -#include - #include #include "../device_status.h" @@ -83,20 +81,12 @@ void Lux::process(StatisticsPtr &stats, Metadata *imageMetadata) if (imageMetadata->get("device.status", deviceStatus) == 0) { double currentGain = deviceStatus.analogueGain; double currentAperture = deviceStatus.aperture.value_or(currentAperture_); - uint64_t sum = 0; - uint32_t num = 0; - uint32_t *bin = stats->hist[0].g_hist; - const int numBins = sizeof(stats->hist[0].g_hist) / - sizeof(stats->hist[0].g_hist[0]); - for (int i = 0; i < numBins; i++) - sum += bin[i] * (uint64_t)i, num += bin[i]; - /* add .5 to reflect the mid-points of bins */ - double currentY = sum / (double)num + .5; + double currentY = stats->yHist.interQuantileMean(0, 1); double gainRatio = referenceGain_ / currentGain; double shutterSpeedRatio = referenceShutterSpeed_ / deviceStatus.shutterSpeed; double apertureRatio = referenceAperture_ / currentAperture; - double yRatio = currentY * (65536 / numBins) / referenceY_; + double yRatio = currentY * (65536 / stats->yHist.bins()) / referenceY_; double estimatedLux = shutterSpeedRatio * gainRatio * apertureRatio * apertureRatio * yRatio * referenceLux_; diff --git a/src/ipa/raspberrypi/raspberrypi.cpp b/src/ipa/raspberrypi/raspberrypi.cpp index bead436def3c..cff079bbafb3 100644 --- a/src/ipa/raspberrypi/raspberrypi.cpp +++ b/src/ipa/raspberrypi/raspberrypi.cpp @@ -51,6 +51,7 @@ #include "metadata.h" #include "sharpen_algorithm.h" #include "sharpen_status.h" +#include "statistics.h" namespace libcamera { @@ -139,6 +140,7 @@ private: void prepareISP(const ISPConfig &data); void reportMetadata(unsigned int ipaContext); void fillDeviceStatus(const ControlList &sensorControls, unsigned int ipaContext); + RPiController::StatisticsPtr fillStatistics(bcm2835_isp_stats *stats) const; void processStats(unsigned int bufferId, unsigned int ipaContext); void applyFrameDurations(Duration minFrameDuration, Duration maxFrameDuration); void applyAGC(const struct AgcStatus *agcStatus, ControlList &ctrls); @@ -1141,6 +1143,46 @@ void IPARPi::fillDeviceStatus(const ControlList &sensorControls, unsigned int ip rpiMetadata_[ipaContext].set("device.status", deviceStatus); } +RPiController::StatisticsPtr IPARPi::fillStatistics(bcm2835_isp_stats *stats) const +{ + using namespace RPiController; + + unsigned int i; + StatisticsPtr statistics = + std::make_unique(Statistics::AgcStatsPos::PreWb, Statistics::ColourStatsPos::PostLsc); + + /* RGB histograms are not used, so do not populate them. */ + statistics->yHist = std::move(RPiController::Histogram(stats->hist[0].g_hist, NUM_HISTOGRAM_BINS)); + + statistics->awbRegions.init({ DEFAULT_AWB_REGIONS_X, DEFAULT_AWB_REGIONS_Y }); + for (i = 0; i < statistics->awbRegions.numRegions(); i++) + statistics->awbRegions.set(i, { { stats->awb_stats[i].r_sum, + stats->awb_stats[i].g_sum, + stats->awb_stats[i].b_sum }, + stats->awb_stats[i].counted, + stats->awb_stats[i].notcounted }); + + /* + * There are only ever 15 regions computed by the firmware due to zoning, + * but the HW defines AGC_REGIONS == 16! + */ + statistics->agcRegions.init(15); + for (i = 0; i < statistics->agcRegions.numRegions(); i++) + statistics->agcRegions.set(i, { { stats->agc_stats[i].r_sum, + stats->agc_stats[i].g_sum, + stats->agc_stats[i].b_sum }, + stats->agc_stats[i].counted, + stats->awb_stats[i].notcounted }); + + statistics->focusRegions.init({ 4, 3 }); + for (i = 0; i < statistics->focusRegions.numRegions(); i++) + statistics->focusRegions.set(i, { stats->focus_stats[i].contrast_val[1][1] / 1000, + stats->focus_stats[i].contrast_val_num[1][1], + stats->focus_stats[i].contrast_val_num[1][0] }); + + return statistics; +} + void IPARPi::processStats(unsigned int bufferId, unsigned int ipaContext) { RPiController::Metadata &rpiMetadata = rpiMetadata_[ipaContext]; @@ -1153,7 +1195,7 @@ void IPARPi::processStats(unsigned int bufferId, unsigned int ipaContext) Span mem = it->second.planes()[0]; bcm2835_isp_stats *stats = reinterpret_cast(mem.data()); - RPiController::StatisticsPtr statistics = std::make_shared(*stats); + RPiController::StatisticsPtr statistics = fillStatistics(stats); helper_->process(statistics, rpiMetadata); controller_.process(statistics, &rpiMetadata); diff --git a/src/ipa/raspberrypi/statistics.h b/src/ipa/raspberrypi/statistics.h index a762bf3d41aa..affb7272c963 100644 --- a/src/ipa/raspberrypi/statistics.h +++ b/src/ipa/raspberrypi/statistics.h @@ -67,4 +67,6 @@ struct Statistics { FocusRegions focusRegions; }; +using StatisticsPtr = std::shared_ptr; + } /* namespace RPiController */ From patchwork Tue Dec 13 11:48:36 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Naushir Patuck X-Patchwork-Id: 18000 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id DDD06BDE6B for ; Tue, 13 Dec 2022 11:48:54 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id A4AA26336B; Tue, 13 Dec 2022 12:48:54 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1670932134; bh=WCd5Yg2ao//5cMiTWI7RL+O49Lp+oN5KbKipJ45uZ20=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To: From; b=utLrL6MqrjLxGawt8JEZBTS3EXI0GzeELX3GmY+p+fCckWQoNYfS7Y4tsuaDSsFG2 /DbjMOCDuxi2LmE86JeS+VNSPEe2TAGoFQxAQYUmOHg6URY+x0o7k/T2ow790eyDFv KWLFXXOkTBTDB5USpGImE4ngVMfMui8aelSGhixQJmuzDUpXdBugYbPj+ucCFXpORI 3UHrzUhHlI96mtc5u+UV83TcYQ5vb6BzAoq+DmPGmj54do+hWmPcjPJJBaFCFaylJZ x4ahB61C+Rw81dgZ2YS8SWgAYGw+Zque0ngZ+PATP4jUbshJV+xMbLYfO3fNEsRrZ6 NYMd2zjQL7YOw== Received: from mail-wr1-x430.google.com (mail-wr1-x430.google.com [IPv6:2a00:1450:4864:20::430]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id DB6F163364 for ; Tue, 13 Dec 2022 12:48:51 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="q/fWJoBy"; dkim-atps=neutral Received: by mail-wr1-x430.google.com with SMTP id i7so691759wrv.8 for ; Tue, 13 Dec 2022 03:48:51 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=FaD2be+yF2ADNeJbCXDRJDi2/VL2T516BxJINXKCmzU=; b=q/fWJoBymzUZHc6DKcKhCEf9HsIsm5E5VAUleIx3Dxh2sGxcywexv2niDkVbDPoUIj P27JMh6ajBXZQBfcIwqOgA/SCm7FdaK2L1XgZMSiCpPEwVYoSp0XGKtv18+ZQ/Zy1Z1w pXmCuBossnBlbu4BChpvYsixUWh/9N4aHymhGDFJPvnED3qYrvZhX0CpJyoXdboCOMfB x9Pq9F7TUk6n59Aib2yEw1++DwQ/sjp+qio25ggJvnbCr7SoRZYkhLtTQ4aSF11aFhRE INY9c8gVJx9QMtBS6CIUrT22o4pd+EOkaY2+CET9OR/oDI6FiUKiDxh3IwE63qC8FiWT AVSQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=FaD2be+yF2ADNeJbCXDRJDi2/VL2T516BxJINXKCmzU=; b=5ZIIBQehTwRhKEOm2S1YYfAXZnyupCTMcGcv4Si9+jEsQhEAYhlXOwU+DSiNjiBrZv z5wOZrKpJ1njzq1aGpqdGluS8w5rN3bgg7OSI9MstYvkH+0TsaT/5DRG0ILLQxAn5Jnm f7X9i9sJkwo9acmnGRKytOJ69n98fAce2Dd0TtAq/O4ehbC1hrNBRoXSYVsrJLIqeD58 7SwUIbzLq30GaHmQfDapN/OA6E6hUrwRBpu42y3Gi5zisQ7ds2HRSRF+oOv8HufWY+oy P9moX/nwWIxeWOBJMKkVNOxn+F/A6u7OJCMNmh2ZoocCSgH0hAGI2rjIyis18wzLXvAo b9cQ== X-Gm-Message-State: ANoB5pkhVLJfIV9FMflZCQo58xjHeUqRu9gC5KtTDrSfRXzV4XkV74NU +UuUIhJzz9Wyln39M8T1z3RN6oI79YTBQo4j X-Google-Smtp-Source: AA0mqf75dQdM6gw9cfiZc+Z/779n+ldVMgptmeuTECkh5DrFSJJOIegVLwTe0Pi1pf0z9u3WgJs+3Q== X-Received: by 2002:adf:a193:0:b0:246:e6df:86ea with SMTP id u19-20020adfa193000000b00246e6df86eamr13618219wru.25.1670932131289; Tue, 13 Dec 2022 03:48:51 -0800 (PST) Received: from naush-laptop.localdomain ([93.93.133.154]) by smtp.gmail.com with ESMTPSA id j8-20020a5d4528000000b0024287d9d4a8sm11574541wra.74.2022.12.13.03.48.50 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 13 Dec 2022 03:48:50 -0800 (PST) To: libcamera-devel@lists.libcamera.org Date: Tue, 13 Dec 2022 11:48:36 +0000 Message-Id: <20221213114836.15473-6-naush@raspberrypi.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20221213114836.15473-1-naush@raspberrypi.com> References: <20221213114836.15473-1-naush@raspberrypi.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v3 5/5] ipa: raspberrypi: Normalise region sums to 16-bits X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Naushir Patuck via libcamera-devel From: Naushir Patuck Reply-To: Naushir Patuck Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" The VC4 ISP uses a pipeline bit-depth of 13-bits. The AGC algorithm needs to know this bit-depth when computing the Y value for the image. Instead of hardcoding the VC4 bit-depth in the AGC source code, normalise all region sums to 16-bits when filling the Statistics structure. AWB and ALSC are agnostic about pipeline depth, so do not need changing. Signed-off-by: Naushir Patuck Reviewed-by: David Plowman Reviewed-by: Kieran Bingham --- src/ipa/raspberrypi/controller/rpi/agc.cpp | 10 ++++------ src/ipa/raspberrypi/raspberrypi.cpp | 16 ++++++++++------ 2 files changed, 14 insertions(+), 12 deletions(-) diff --git a/src/ipa/raspberrypi/controller/rpi/agc.cpp b/src/ipa/raspberrypi/controller/rpi/agc.cpp index 868c30f03d66..7ec8292d32ff 100644 --- a/src/ipa/raspberrypi/controller/rpi/agc.cpp +++ b/src/ipa/raspberrypi/controller/rpi/agc.cpp @@ -28,8 +28,6 @@ LOG_DEFINE_CATEGORY(RPiAgc) #define NAME "rpi.agc" -static constexpr unsigned int PipelineBits = 13; /* seems to be a 13-bit pipeline */ - int AgcMeteringMode::read(const libcamera::YamlObject ¶ms) { const YamlObject &yamlWeights = params["weights"]; @@ -593,9 +591,9 @@ static double computeInitialY(StatisticsPtr &stats, AwbStatus const &awb, double rSum = 0, gSum = 0, bSum = 0, pixelSum = 0; for (unsigned int i = 0; i < stats->agcRegions.numRegions(); i++) { auto ®ion = stats->agcRegions.get(i); - double rAcc = std::min(region.val.rSum * gain, ((1 << PipelineBits) - 1) * region.counted); - double gAcc = std::min(region.val.gSum * gain, ((1 << PipelineBits) - 1) * region.counted); - double bAcc = std::min(region.val.bSum * gain, ((1 << PipelineBits) - 1) * region.counted); + double rAcc = std::min(region.val.rSum * gain, ((1 << 16) - 1) * region.counted); + double gAcc = std::min(region.val.gSum * gain, ((1 << 16) - 1) * region.counted); + double bAcc = std::min(region.val.bSum * gain, ((1 << 16) - 1) * region.counted); rSum += rAcc * weights[i]; gSum += gAcc * weights[i]; bSum += bAcc * weights[i]; @@ -608,7 +606,7 @@ static double computeInitialY(StatisticsPtr &stats, AwbStatus const &awb, double ySum = rSum * awb.gainR * .299 + gSum * awb.gainG * .587 + bSum * awb.gainB * .114; - return ySum / pixelSum / (1 << PipelineBits); + return ySum / pixelSum / (1 << 16); } /* diff --git a/src/ipa/raspberrypi/raspberrypi.cpp b/src/ipa/raspberrypi/raspberrypi.cpp index cff079bbafb3..50fdeb4f0478 100644 --- a/src/ipa/raspberrypi/raspberrypi.cpp +++ b/src/ipa/raspberrypi/raspberrypi.cpp @@ -1154,11 +1154,15 @@ RPiController::StatisticsPtr IPARPi::fillStatistics(bcm2835_isp_stats *stats) co /* RGB histograms are not used, so do not populate them. */ statistics->yHist = std::move(RPiController::Histogram(stats->hist[0].g_hist, NUM_HISTOGRAM_BINS)); + /* + * All region sums are based on a 13-bit pipeline bit-depth. Normalise + * this to 16-bits for the AGC/AWB/ALSC algorithms. + */ statistics->awbRegions.init({ DEFAULT_AWB_REGIONS_X, DEFAULT_AWB_REGIONS_Y }); for (i = 0; i < statistics->awbRegions.numRegions(); i++) - statistics->awbRegions.set(i, { { stats->awb_stats[i].r_sum, - stats->awb_stats[i].g_sum, - stats->awb_stats[i].b_sum }, + statistics->awbRegions.set(i, { { stats->awb_stats[i].r_sum << 3, + stats->awb_stats[i].g_sum << 3, + stats->awb_stats[i].b_sum << 3 }, stats->awb_stats[i].counted, stats->awb_stats[i].notcounted }); @@ -1168,9 +1172,9 @@ RPiController::StatisticsPtr IPARPi::fillStatistics(bcm2835_isp_stats *stats) co */ statistics->agcRegions.init(15); for (i = 0; i < statistics->agcRegions.numRegions(); i++) - statistics->agcRegions.set(i, { { stats->agc_stats[i].r_sum, - stats->agc_stats[i].g_sum, - stats->agc_stats[i].b_sum }, + statistics->agcRegions.set(i, { { stats->agc_stats[i].r_sum << 3, + stats->agc_stats[i].g_sum << 3, + stats->agc_stats[i].b_sum << 3 }, stats->agc_stats[i].counted, stats->awb_stats[i].notcounted });