From patchwork Thu Nov 24 10:38:28 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Naushir Patuck X-Patchwork-Id: 17862 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 410D7BDE6B for ; Thu, 24 Nov 2022 10:38:45 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id A36526330A; Thu, 24 Nov 2022 11:38:43 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1669286323; bh=NOrgDIPDWhC3tzOO598pHppsdGT5KpshNbARfH0Eg20=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To: From; b=rAHp4QBB3Ac1GjloCxaP0gGzwzTpzr9tzWQ3BKuELbMhKBy8OVBnKoLakrPWDuZwZ ex/jS8UYO2mwBVkLKwiXpazAPG+82jr8AoEYdILgvq4GHxK4jDTfL2Va5be+Hr/E+9 L4BmxKH0WBKR1H1R2t46+QbSjXuqXy1G0GHrlShtuMix5Hk7Ep4Qa2da5B3Qd+QEKY g2eh6Ik8l9IK+bZtvi+TRNlIUUhkENYzXCgo8LNEaSnrt+Atop4+Gv37nHGIAeDmP4 eJ0gHH0Kraidx767MYQwwIxhqvhRSUTJHyKu9F8pyOTaFL8SVENPwGHRIklZYJhv1x f+pX8jUKHOo8Q== Received: from mail-wr1-x42d.google.com (mail-wr1-x42d.google.com [IPv6:2a00:1450:4864:20::42d]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id C860B6330A for ; Thu, 24 Nov 2022 11:38:40 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="VuD4r0v6"; dkim-atps=neutral Received: by mail-wr1-x42d.google.com with SMTP id g12so1821488wrs.10 for ; Thu, 24 Nov 2022 02:38:40 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=TgY1XqtE04Zqn2QHeFSJfKHeETJOVH8v2DU+dZ7yV4Y=; b=VuD4r0v6GFdSe3zV6tXNyYqOlkICbE5bTdvT4GWPml4+6b3k0I0f7NcU9U84dbxRhq BBiAQ15Gnjf74VT0e0cIu7+dnTAG/Xgyc0ajjOBMrYeGPd3fgzthzQsao3DT1kxyw/Ch RTmlpi+EeWOZr/x1WPWzV1y5hTyq0OvIcADRhWIqr06PAxWG07HQJCcLS3Qd0QwEy6Jp ICNhzgP71bi3xPBfgX0I5pKOwd66Me5mpjfsQIMKnSfFxK9OYb5glVBb8jEPa1zUGbpm a3M4Fn5ZkNPBL4EoR39etm3Gxkqum1nb4gavEmR0wkte0/SsfcqRpE3ahni4VXft/08M wVPw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=TgY1XqtE04Zqn2QHeFSJfKHeETJOVH8v2DU+dZ7yV4Y=; b=VNI7PMlwVCyRS2BdBqYX9DXqyVCqEbdfrpXgdbOaotV+JueNW+5kzgOxZVqUnT2xsJ 5TwDzsMo1Hnhrs41+rSn/pqS++HgJzPYBiCWydWxjKGDnnua3/7mGk6cGVpxi2aEIwLX ZHEg+OTYLfHTppxdJR53pZ12Dw+llU8wfjHn9LEo20FP+WyWOrT9yk+7SDlD/+K9rdw4 UDDMcbyvuSZG4/9YGCoId2+jbJyS3fX3a7C70eFQ4KKzckys5hAZMgxiQxIoZKfLGYz+ l1+rbWFVanHYNQ2I82zi1GphqxwKNU/QoakDYRMcqk/xKlihQRuWizCF40ET9ATy1qnt wIjg== X-Gm-Message-State: ANoB5pnPmTx7y2OuUT59idMKo0Uvl4KmUou4NNDSBuwxOkqiMB6Xg+AU 0L0xspe/H8Si9XIY6SA1R+Z7xUvGi9yarQ== X-Google-Smtp-Source: AA0mqf5O/xuMfFm6nnqCh7Uiq5K/+4fYQ6xg7vrZD2g/nCZ2zzp0tFCInBd9MdAYwD1NEY7ne9LHyw== X-Received: by 2002:a5d:6244:0:b0:236:73b2:f026 with SMTP id m4-20020a5d6244000000b0023673b2f026mr19914801wrv.396.1669286320172; Thu, 24 Nov 2022 02:38:40 -0800 (PST) Received: from naush-laptop.localdomain ([93.93.133.154]) by smtp.gmail.com with ESMTPSA id s16-20020adff810000000b002368a6deaf8sm995103wrp.57.2022.11.24.02.38.39 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 24 Nov 2022 02:38:39 -0800 (PST) To: libcamera-devel@lists.libcamera.org Date: Thu, 24 Nov 2022 10:38:28 +0000 Message-Id: <20221124103832.6172-2-naush@raspberrypi.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20221124103832.6172-1-naush@raspberrypi.com> References: <20221124103832.6172-1-naush@raspberrypi.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v2 1/5] ipa: raspberrypi: Generalise statistics X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Naushir Patuck via libcamera-devel From: Naushir Patuck Reply-To: Naushir Patuck Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" At present, the controller algorithms access the bcm2835_isp_stats structure, which is hardware specific. It would be desirable to abstract out the statistics structure to remove hardware specific headers from the algorithms source files. Define a new templated RegionStats class that encompasses region based statistics generated by the ISP. For the VC4 ISP, this can be used to hold RGB sums and focus FoM values. Define a new Statistics structure that holds all the VC4 ISP statistics output. This includes AGC hostograms, AGC/AWB region sums and focus FoM regions. Signed-off-by: Naushir Patuck Reviewed-by: David Plowman --- src/ipa/raspberrypi/controller/region_stats.h | 123 ++++++++++++++++++ src/ipa/raspberrypi/statistics.h | 70 ++++++++++ 2 files changed, 193 insertions(+) create mode 100644 src/ipa/raspberrypi/controller/region_stats.h create mode 100644 src/ipa/raspberrypi/statistics.h diff --git a/src/ipa/raspberrypi/controller/region_stats.h b/src/ipa/raspberrypi/controller/region_stats.h new file mode 100644 index 000000000000..9aaf3a58a6f7 --- /dev/null +++ b/src/ipa/raspberrypi/controller/region_stats.h @@ -0,0 +1,123 @@ +/* SPDX-License-Identifier: BSD-2-Clause */ +/* + * Copyright (C) 2022, Raspberry Pi Ltd + * + * region_stats.h - Raspberry Pi region based statistics container + */ +#pragma once + +#include +#include +#include + +#include + +namespace RPiController { + +template +class RegionStats +{ +public: + struct Region { + T val; + uint32_t counted; + uint32_t uncounted; + }; + + RegionStats() + : size_({}), numFloating_(0), default_({}) + { + } + + void init(const libcamera::Size &size, unsigned int numFloating = 0) + { + size_ = size; + numFloating_ = numFloating; + regions_.clear(); + regions_.resize(size_.width * size_.height + numFloating_); + } + + void init(unsigned int num) + { + size_ = libcamera::Size(num, 1); + numFloating_ = 0; + regions_.clear(); + regions_.resize(num); + } + + unsigned int numRegions() const + { + return size_.width * size_.height; + } + + unsigned int numFloatingRegions() const + { + return numFloating_; + } + + libcamera::Size size() const + { + return size_; + } + + void set(unsigned int index, const Region ®ion) + { + if (index >= numRegions()) + return; + set_(index, region); + } + + void set(const libcamera::Point &pos, const Region ®ion) + { + set(pos.y * size_.width + pos.x, region); + } + + void setFloating(unsigned int index, const Region ®ion) + { + if (index >= numFloatingRegions()) + return; + set(numRegions() + index, region); + } + + const Region &get(unsigned int index) const + { + if (index >= numRegions()) + return default_; + return get_(index); + } + + const T &get(const libcamera::Point &pos) const + { + return get(pos.y * size_.width + pos.x); + } + + const T &getFloating(unsigned int index) const + { + if (index >= numFloatingRegions()) + return default_; + return get_(numRegions() + index); + } + + typename std::vector::iterator begin() { return regions_.begin(); } + typename std::vector::iterator end() { return regions_.end(); } + typename std::vector::const_iterator begin() const { return regions_.begin(); } + typename std::vector::const_iterator end() const { return regions_.end(); } + +private: + void set_(unsigned int index, const Region ®ion) + { + regions_[index] = region; + } + + const Region &get_(unsigned int index) const + { + return regions_[index]; + } + + libcamera::Size size_; + unsigned int numFloating_; + std::vector regions_; + Region default_; +}; + +} /* namespace RPiController */ diff --git a/src/ipa/raspberrypi/statistics.h b/src/ipa/raspberrypi/statistics.h new file mode 100644 index 000000000000..a762bf3d41aa --- /dev/null +++ b/src/ipa/raspberrypi/statistics.h @@ -0,0 +1,70 @@ +/* SPDX-License-Identifier: BSD-2-Clause */ +/* + * Copyright (C) 2022, Raspberry Pi Ltd + * + * statistics.h - Raspberry Pi generic statistics structure + */ +#pragma once + +#include +#include +#include + +#include "histogram.h" +#include "region_stats.h" + +namespace RPiController { + +struct RgbySums { + RgbySums(uint64_t _rSum = 0, uint64_t _gSum = 0, uint64_t _bSum = 0, uint64_t _ySum = 0) + : rSum(_rSum), gSum(_gSum), bSum(_bSum), ySum(_ySum) + { + } + uint64_t rSum; + uint64_t gSum; + uint64_t bSum; + uint64_t ySum; +}; + +using RgbyRegions = RegionStats; +using FocusRegions = RegionStats; + +struct Statistics { + /* + * Positioning of the AGC statistics gathering in the pipeline: + * Pre-WB correction or post-WB correction. + * Assume this is post-LSC. + */ + enum class AgcStatsPos { PreWb, PostWb }; + const AgcStatsPos agcStatsPos; + + /* + * Positioning of the AWB/ALSC statistics gathering in the pipeline: + * Pre-LSC or post-LSC. + */ + enum class ColourStatsPos { PreLsc, PostLsc }; + const ColourStatsPos colourStatsPos; + + Statistics(AgcStatsPos a, ColourStatsPos c) + : agcStatsPos(a), colourStatsPos(c) + { + } + + /* Histogram statistics. Not all histograms may be populated! */ + Histogram rHist; + Histogram gHist; + Histogram bHist; + Histogram yHist; + + /* Row sums for flicker avoidance. */ + std::vector rowSums; + + /* Region based colour sums. */ + RgbyRegions agcRegions; + RgbyRegions awbRegions; + + /* Region based focus FoM. */ + FocusRegions focusRegions; +}; + +} /* namespace RPiController */ From patchwork Thu Nov 24 10:38:29 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Naushir Patuck X-Patchwork-Id: 17863 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 5ED46BDE6B for ; Thu, 24 Nov 2022 10:38:46 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 2059463321; Thu, 24 Nov 2022 11:38:44 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1669286324; bh=ma9szNikYEmGctrJ31Ey01v3NafeZYa8iuiY1l8LTwQ=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To: From; b=3wvZ/tVkFn7rlHhqoOsm5z9iiUXN9u0tNj7Agmyoi3RmyFmvML5lWYVBgVbDjAM7Q ryqtzYMdl7qpNC6apsulQbDKORPRjrtpz2bAIYIRz69B6pAdPagh/+PIqZT2x2Zkge O9WU2X/eUvld4UA/dbQMLHjbf+K3hCIFHhs8dBMPjqukR/6gpPP6z5cRYlHG0LdsxF nN0c0h7h5AWQ4YFmGjKrOwL4uSYjkS+4C1fYUsZicJcgEkQoaWeProjTBq+5i5wSfJ Wse64w1UsNFcZ/LR/gB6tUij6469lCT9IxndRowbO/Vlff1ac5z18SIl1Fe1mA8VWM gD/dHsuF1Ya8Q== Received: from mail-wr1-x42f.google.com (mail-wr1-x42f.google.com [IPv6:2a00:1450:4864:20::42f]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 9366363316 for ; Thu, 24 Nov 2022 11:38:41 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="F0PlEgh7"; dkim-atps=neutral Received: by mail-wr1-x42f.google.com with SMTP id q7so961182wrr.8 for ; Thu, 24 Nov 2022 02:38:41 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=Oz62LSxuhaQPLtvLBoBKpt4oVpu0W2M+xPyY7uBTRX0=; b=F0PlEgh7jJBm5RP+QgTLgVIQQr6li6qeh8AX6e843o7xvocib8sOx5ANGxvDVUAZg1 X2k96ubQthWl81TTJQTcN1HQfG4+ayfCU2aVeFvXK679KImjgifBigumNi9dkVs1t8S1 6HRwtg6SIcvnpwmnhbSj7OgN8FkWGhT0qqij4SgKl1Phac0wj9/YdyByxP7Kf2LEX5zP bfRLYWvSieKxgCAPYwU/6deEKV9Qt0iZKmZOf+/jtqqOc3fRhl6zsE49PbL7pxKt1EnW 9iKUmolXKRQPiSmi30IdgDx0AQ8durxrTYPYp9vTJJPGGmU0u1f61FsotJFN40JF6W2N pc3A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=Oz62LSxuhaQPLtvLBoBKpt4oVpu0W2M+xPyY7uBTRX0=; b=ulVWP71CIPhmPEQ7ymBbrXDnDEWKOHK0OUffbje8qa/GlV4I5iu56DmKBhOzftB0oA vEg1dERQyTvom9fUFVC/hpr6pKsabvdZMKe0twExyATk2s4tK6uFROw60IXHxLP0o0zf dSVjRMCOuHVdZDOHYoGJrTHGA5zsEKIfAqFwrQF4WMnikg1QrROw3R7zmFQt+Rvp/kka d1xrmhjhJ3wrirvtQefymHo99h9WQBdQCn30DgleQJDQRgxbClHqz4SeQYW9meXkDrI5 JvpP0UVN3NaHeeU0qndu/6yiEK9a4IVqFBayoZP/HeE1KE7tVMD/lAYL+1hUIx0653Wo rtXA== X-Gm-Message-State: ANoB5pmkyikbGhw7kbFlyL3dfSWF2LwdD8IDJGQC0WVTO/FcxqkcveQK pv0UtQ/xu91jNknzlGxXpliED3KTpEj5yA== X-Google-Smtp-Source: AA0mqf43A1Nlhg4Rt+SPFOQakbKLUgOccGZZb+2XUykGEBWMUU/8d5TF+JEkEIc1Oz6udp80cre/AA== X-Received: by 2002:a5d:620e:0:b0:241:e32f:8371 with SMTP id y14-20020a5d620e000000b00241e32f8371mr7003104wru.156.1669286320979; Thu, 24 Nov 2022 02:38:40 -0800 (PST) Received: from naush-laptop.localdomain ([93.93.133.154]) by smtp.gmail.com with ESMTPSA id s16-20020adff810000000b002368a6deaf8sm995103wrp.57.2022.11.24.02.38.40 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 24 Nov 2022 02:38:40 -0800 (PST) To: libcamera-devel@lists.libcamera.org Date: Thu, 24 Nov 2022 10:38:29 +0000 Message-Id: <20221124103832.6172-3-naush@raspberrypi.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20221124103832.6172-1-naush@raspberrypi.com> References: <20221124103832.6172-1-naush@raspberrypi.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v2 2/5] ipa: raspberrypi: histogram: Add a constructor for an empty histogram X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Naushir Patuck via libcamera-devel From: Naushir Patuck Reply-To: Naushir Patuck Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Add a default constructor to the RPiController::Histogram class that creates an empty histogram. Since this is a cumulative histogram, push a value of 0 into the first (and only) bin to signify this. Signed-off-by: Naushir Patuck Reviewed-by: David Plowman --- src/ipa/raspberrypi/controller/histogram.h | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/src/ipa/raspberrypi/controller/histogram.h b/src/ipa/raspberrypi/controller/histogram.h index 66a68b087964..6b3e3a9e68e4 100644 --- a/src/ipa/raspberrypi/controller/histogram.h +++ b/src/ipa/raspberrypi/controller/histogram.h @@ -20,6 +20,11 @@ namespace RPiController { class Histogram { public: + Histogram() + { + cumulative_.push_back(0); + } + template Histogram(T *histogram, int num) { assert(num); From patchwork Thu Nov 24 10:38:30 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Naushir Patuck X-Patchwork-Id: 17864 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id D63F5C3286 for ; Thu, 24 Nov 2022 10:38:46 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id B80D06331B; Thu, 24 Nov 2022 11:38:44 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1669286324; bh=/5KjD6c/IL+fXydsKOzb9YnlaccYxgItMGrGOcqLFgc=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To: From; b=1ObkNSMDWXDqWYMUhl/+mqX/StKj3tjh0SPTgA05ONDaH8ZFDWt0cpUzvxVPk10Hl XLGiFa/feMPO+mTasRSEos9cZB7QwpRLNtwKqn2ZqgCTqeWI11crcg3fMVi6D5wBU/ yAgmq+A9OZ05jdoovG8x8YwEwdo82ykN41d5R/rzjG3McmwNYQoPhUW+CNJpJWNNsd rUcO/mYCcs4AmEmrlu/+id1Z73ofA60uK1/rbzRRqycjR0yCP1m95FlNCAmLYHdwSZ 0JTgL19BPhdZnEFRKwjd21KWYmUY+AYChmnur1+HgKIcIbKRl3R/UmG7Xk/Nn0LRDu gEkjVvsW81X3w== Received: from mail-wr1-x42a.google.com (mail-wr1-x42a.google.com [IPv6:2a00:1450:4864:20::42a]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 478E663316 for ; Thu, 24 Nov 2022 11:38:42 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="A0Eqnhyr"; dkim-atps=neutral Received: by mail-wr1-x42a.google.com with SMTP id n7so1819974wrr.13 for ; Thu, 24 Nov 2022 02:38:42 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=raGM6H8WE6ZOibbytcLb47YD73g9Kdg0YHaASybQJqg=; b=A0EqnhyrEMpdzklSN1K7/Xm6Xc7qluzQEu8NmC7OMJZXBGR/b0x1ZzZWAenRI/2i9N Rpj/96RavfG++RlJfXmevu7LM/OM+lSdtqrvMLrKY/OCTxXLxl0shjm53Cw3VvuJvYTT /aKsnFJGX4DrWeYCsw2titWC4LA2eZzh93eRl0luAsD0+gXj76JG1GNPyYoOu7CverGf +QYgS6+iq2DZQFa+6Z0rKv7oCyvHcVr8h1N/Vntx22ulbX1pCq7ynUPOq0YG2f8DZqrC qWSwtZQ3hrPczCOvnvnSH01En0DlifRVPXBY9u4eIRoFi+3Vi1q5tn9sTNV/Ols3x6JK rCoQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=raGM6H8WE6ZOibbytcLb47YD73g9Kdg0YHaASybQJqg=; b=Rgj2eID3fsVuuNFyk+OovUbCVn0l3cG3LInZIqkcsYcORzx983F7TcJu5/pmq18iXm 1qov2RF29Bn14RLNVeHpY9rnZ1nSAMv0LZ5HR37uChrKKwZ6tSxtJMOiObV2g26Y19vO vUHTf016/S62zCbSuwEO1DVMM7fQf0UiHJ5VCWr1PsCrXhqclYt2PHIaQm9MtTTiN4ap 887INGp/Op8eHcUbkNc3t9EOoCu9c2k74BAkWuF8Lw9UFArs5VzOsy9m2J7Cv9Z45Zvp 0wAt4kMKWYrC+4GXcdom+O5UfXA5jGFuBbzgK79ZmydKKM7shLDxvr3YVHXNS8pLAHpP lxeg== X-Gm-Message-State: ANoB5pkZgh/pCzQYcdb88nOjNPP6Zs4+9udsnL6qVayFvN/1acciV4GR /wwlBiOmYO+7wg138vomMWZEGTk0pVxwKg== X-Google-Smtp-Source: AA0mqf7k/Uvk5SeJ1ceMN5chg0dhgos+H8JvrD2DBdpG+dHP7+OfQB5Jh0I7cQg6voLZK85Hd2n3ww== X-Received: by 2002:a5d:5684:0:b0:236:61bb:c79d with SMTP id f4-20020a5d5684000000b0023661bbc79dmr19347079wrv.632.1669286321700; Thu, 24 Nov 2022 02:38:41 -0800 (PST) Received: from naush-laptop.localdomain ([93.93.133.154]) by smtp.gmail.com with ESMTPSA id s16-20020adff810000000b002368a6deaf8sm995103wrp.57.2022.11.24.02.38.41 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 24 Nov 2022 02:38:41 -0800 (PST) To: libcamera-devel@lists.libcamera.org Date: Thu, 24 Nov 2022 10:38:30 +0000 Message-Id: <20221124103832.6172-4-naush@raspberrypi.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20221124103832.6172-1-naush@raspberrypi.com> References: <20221124103832.6172-1-naush@raspberrypi.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v2 3/5] ipa: raspberrypi: awb: Delay release of the statistics buffer X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Naushir Patuck via libcamera-devel From: Naushir Patuck Reply-To: Naushir Patuck Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Release the statistics buffer after running the through the AWB calculations. Only the "counted" statistics are copied out to a local structure, so keeping the statistics buffer allows the algorithm to see the "uncounted" statistics as well. This is currently handled by hard-coding the total number of statistics regions regions based on the structure definition in the bcm2835_isp_stats structure. Signed-off-by: Naushir Patuck Reviewed-by: David Plowman --- src/ipa/raspberrypi/controller/rpi/awb.cpp | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/src/ipa/raspberrypi/controller/rpi/awb.cpp b/src/ipa/raspberrypi/controller/rpi/awb.cpp index 8d8ddf0913f7..861022014896 100644 --- a/src/ipa/raspberrypi/controller/rpi/awb.cpp +++ b/src/ipa/raspberrypi/controller/rpi/awb.cpp @@ -432,11 +432,6 @@ void Awb::prepareStats() */ generateStats(zones_, statistics_->awb_stats, config_.minPixels, config_.minG); - /* - * we're done with these; we may as well relinquish our hold on the - * pointer. - */ - statistics_.reset(); /* * apply sensitivities, so values appear to come from our "canonical" * sensor. @@ -733,6 +728,11 @@ void Awb::doAwb() << " with gains r " << asyncResults_.gainR << " and b " << asyncResults_.gainB; } + /* + * we're done with these; we may as well relinquish our hold on the + * pointer. + */ + statistics_.reset(); } /* Register algorithm with the system. */ From patchwork Thu Nov 24 10:38:31 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Naushir Patuck X-Patchwork-Id: 17865 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id E60A5BDE6B for ; Thu, 24 Nov 2022 10:38:47 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id A522063325; Thu, 24 Nov 2022 11:38:47 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1669286327; bh=WNBUmKaA8eAmX6ziSmJ7c/VJTmbjFr0YF+ETxqbfW8E=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To: From; b=yslVjMMLcYg6q1Xi7bbtC45aaFnA+jr5uGzcjGUBPYay/9M3ZIHAzP2SZmnJWrYL/ ngmvTlHqZZPvn/nZLzxOPShZ5Cq76AzaFtgkQE/M3YKaNJGME1OyEA4OtTOdfUi3Mo Dkpgz2ubuV5rY8zMDCZBzOpUxD3gI40xpNvdg43U91UVHWrWBFW60NoAFMvAsXkPcf o6QvfFLqsf3y8W6gqR54HskGg9eF1xqmg4Q2MInTnzw5BqQGWVxpONci3nAdXAOGP/ gGYIO7P3dWl+wyB+1SDifRpUHGCbxBFdtXVszmVKwsHyy+XlZ9UK/c2ngznzr0GAk9 wVN4+sUtzvBdw== Received: from mail-wr1-x42f.google.com (mail-wr1-x42f.google.com [IPv6:2a00:1450:4864:20::42f]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 797D263316 for ; Thu, 24 Nov 2022 11:38:43 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="Zhb/Z6di"; dkim-atps=neutral Received: by mail-wr1-x42f.google.com with SMTP id q7so961316wrr.8 for ; Thu, 24 Nov 2022 02:38:43 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=6KA1x2/XLmduJUFV1jLTJqy+rDVS3k4ooI+vQMQQH0A=; b=Zhb/Z6diXwbhYFqSlRAnFrhBA086eIfkkB3aM9QLf56TxbmtvXwHub7a03ZaJNuHAf zIX3MjrZqDd6UT61C7hKYjNHFjOMkQsBhTvaNnaOtfNuU19LodOQjbZUvAy3EhEOjQl9 QSeC0TzZZL8/ge8up9ehrGkOQPS8aR7mVbBEbDkGAOUFO+dD/R6wIJ7QX35IPBuuJOte fnyB6VZKTg3//n+zTMAN0iUvy/YMyz34YdL3xDPQlEqU834hIgy+EyHbCxhAyOSWwNbs ugCCf4pAgFedcD6+21rKKgu5hAaHcujlUX9L4pZAdBWJongpqcIv2LqPpE1xTTQjP3Fl Q7yQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=6KA1x2/XLmduJUFV1jLTJqy+rDVS3k4ooI+vQMQQH0A=; b=YOCZb7OJdpCN9vHgJ31uYyqNsnCfDTSehPVJeSYbf5ExTpX/UYTLsc1P8UfdTgQS0Q zYbzS58GU5OShAiwqei0xsc3uSkW+436lcCjxdq7sFlzGLkfhKNnxVZa2g9B8topeZEP eiBBCmzMWrX7u4REF6EMFxlG8Fly0wVqO/DYWVpIKuLysO9/e1zhFBmjg5J2pDIZ5Dbh XjXYHWvSRTGX+Fny47X7k+UZ7XgYq5xzOovdghO2Gd3DXaFJ3p1WpUxhR6cQM5u3GtkL 5cguD6LOSlWsUadFQHJM8X3MXEKZwUMwCTXSiCk+8GIjkvjcI3iO40/xVjRZ/BJPMLfD mYkQ== X-Gm-Message-State: ANoB5pkjQ7bf6ctnst2lLPsczURRxFfxWm52M8LopebQlJcRCefWT4vV a5uau9s0mmDnF6fjnJZDRY+z4x1UVsavFg== X-Google-Smtp-Source: AA0mqf4fl34p7vv2eIM7eQi5l/NuwBJ9JoBZtLsOPbLS8JNt0ZRe8CGffu/QdGbSsMroN6DhtyvZBA== X-Received: by 2002:adf:ebc5:0:b0:241:6c90:79e9 with SMTP id v5-20020adfebc5000000b002416c9079e9mr4179878wrn.642.1669286322575; Thu, 24 Nov 2022 02:38:42 -0800 (PST) Received: from naush-laptop.localdomain ([93.93.133.154]) by smtp.gmail.com with ESMTPSA id s16-20020adff810000000b002368a6deaf8sm995103wrp.57.2022.11.24.02.38.41 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 24 Nov 2022 02:38:42 -0800 (PST) To: libcamera-devel@lists.libcamera.org Date: Thu, 24 Nov 2022 10:38:31 +0000 Message-Id: <20221124103832.6172-5-naush@raspberrypi.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20221124103832.6172-1-naush@raspberrypi.com> References: <20221124103832.6172-1-naush@raspberrypi.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v2 4/5] ipa: raspberrypi: Use the generic statistics structure in the algorithms X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Naushir Patuck via libcamera-devel From: Naushir Patuck Reply-To: Naushir Patuck Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Repurpose the StatisticsPtr type from being a shared_ptr to shared_ptr. This removes any hardware specific header files and structures from the algorithms source code. Add a new function in the Raspberry Pi IPA to populate the generic statistics structure from the values provided by the hardware in the bcm2835_isp_stats structure. Update the Lux, AWB, AGC, ALSC, Contrast, and Focus algorithms to use the generic statistics structure appropriately in their calculations. Additionally, remove references to any hardware specific headers and defines in these source files. Signed-off-by: Naushir Patuck Reviewed-by: David Plowman --- src/ipa/raspberrypi/controller/controller.h | 4 +- src/ipa/raspberrypi/controller/rpi/agc.cpp | 27 +++++------- src/ipa/raspberrypi/controller/rpi/agc.h | 2 +- src/ipa/raspberrypi/controller/rpi/alsc.cpp | 32 +++++++------- src/ipa/raspberrypi/controller/rpi/alsc.h | 3 +- src/ipa/raspberrypi/controller/rpi/awb.cpp | 20 ++++----- src/ipa/raspberrypi/controller/rpi/awb.h | 1 + .../raspberrypi/controller/rpi/contrast.cpp | 8 ++-- src/ipa/raspberrypi/controller/rpi/focus.cpp | 7 ++- src/ipa/raspberrypi/controller/rpi/lux.cpp | 14 +----- src/ipa/raspberrypi/raspberrypi.cpp | 44 ++++++++++++++++++- src/ipa/raspberrypi/statistics.h | 2 + 12 files changed, 96 insertions(+), 68 deletions(-) diff --git a/src/ipa/raspberrypi/controller/controller.h b/src/ipa/raspberrypi/controller/controller.h index 3e1e051703b3..e6c950c3a509 100644 --- a/src/ipa/raspberrypi/controller/controller.h +++ b/src/ipa/raspberrypi/controller/controller.h @@ -15,19 +15,17 @@ #include #include -#include - #include "libcamera/internal/yaml_parser.h" #include "camera_mode.h" #include "device_status.h" #include "metadata.h" +#include "statistics.h" namespace RPiController { class Algorithm; typedef std::unique_ptr AlgorithmPtr; -typedef std::shared_ptr StatisticsPtr; /* * The Controller holds a pointer to some global_metadata, which is how diff --git a/src/ipa/raspberrypi/controller/rpi/agc.cpp b/src/ipa/raspberrypi/controller/rpi/agc.cpp index bd54a639d637..a7f0e954407d 100644 --- a/src/ipa/raspberrypi/controller/rpi/agc.cpp +++ b/src/ipa/raspberrypi/controller/rpi/agc.cpp @@ -9,8 +9,6 @@ #include #include -#include - #include #include "../awb_status.h" @@ -450,7 +448,7 @@ void Agc::process(StatisticsPtr &stats, Metadata *imageMetadata) fetchCurrentExposure(imageMetadata); /* Compute the total gain we require relative to the current exposure. */ double gain, targetY; - computeGain(stats.get(), imageMetadata, gain, targetY); + computeGain(stats, imageMetadata, gain, targetY); /* Now compute the target (final) exposure which we think we want. */ computeTargetExposure(gain); /* @@ -584,24 +582,23 @@ void Agc::fetchAwbStatus(Metadata *imageMetadata) LOG(RPiAgc, Debug) << "No AWB status found"; } -static double computeInitialY(bcm2835_isp_stats *stats, AwbStatus const &awb, +static double computeInitialY(StatisticsPtr &stats, AwbStatus const &awb, double weights[], double gain) { - bcm2835_isp_stats_region *regions = stats->agc_stats; /* * Note how the calculation below means that equal weights give you * "average" metering (i.e. all pixels equally important). */ double rSum = 0, gSum = 0, bSum = 0, pixelSum = 0; - for (unsigned int i = 0; i < AgcStatsSize; i++) { - double counted = regions[i].counted; - double rAcc = std::min(regions[i].r_sum * gain, ((1 << PipelineBits) - 1) * counted); - double gAcc = std::min(regions[i].g_sum * gain, ((1 << PipelineBits) - 1) * counted); - double bAcc = std::min(regions[i].b_sum * gain, ((1 << PipelineBits) - 1) * counted); + for (unsigned int i = 0; i < stats->agcRegions.numRegions(); i++) { + auto ®ion = stats->agcRegions.get(i); + double rAcc = std::min(region.val.rSum * gain, ((1 << PipelineBits) - 1) * region.counted); + double gAcc = std::min(region.val.gSum * gain, ((1 << PipelineBits) - 1) * region.counted); + double bAcc = std::min(region.val.bSum * gain, ((1 << PipelineBits) - 1) * region.counted); rSum += rAcc * weights[i]; gSum += gAcc * weights[i]; bSum += bAcc * weights[i]; - pixelSum += counted * weights[i]; + pixelSum += region.counted * weights[i]; } if (pixelSum == 0.0) { LOG(RPiAgc, Warning) << "computeInitialY: pixelSum is zero"; @@ -623,23 +620,23 @@ static double computeInitialY(bcm2835_isp_stats *stats, AwbStatus const &awb, static constexpr double EvGainYTargetLimit = 0.9; -static double constraintComputeGain(AgcConstraint &c, Histogram &h, double lux, +static double constraintComputeGain(AgcConstraint &c, const Histogram &h, double lux, double evGain, double &targetY) { targetY = c.yTarget.eval(c.yTarget.domain().clip(lux)); targetY = std::min(EvGainYTargetLimit, targetY * evGain); double iqm = h.interQuantileMean(c.qLo, c.qHi); - return (targetY * NUM_HISTOGRAM_BINS) / iqm; + return (targetY * h.bins()) / iqm; } -void Agc::computeGain(bcm2835_isp_stats *statistics, Metadata *imageMetadata, +void Agc::computeGain(StatisticsPtr &statistics, Metadata *imageMetadata, double &gain, double &targetY) { struct LuxStatus lux = {}; lux.lux = 400; /* default lux level to 400 in case no metadata found */ if (imageMetadata->get("lux.status", lux) != 0) LOG(RPiAgc, Warning) << "No lux level found"; - Histogram h(statistics->hist[0].g_hist, NUM_HISTOGRAM_BINS); + const Histogram &h = statistics->yHist; double evGain = status_.ev * config_.baseEv; /* * The initial gain and target_Y come from some of the regions. After diff --git a/src/ipa/raspberrypi/controller/rpi/agc.h b/src/ipa/raspberrypi/controller/rpi/agc.h index 6d6b0e5ad857..0ea71e6d1aa0 100644 --- a/src/ipa/raspberrypi/controller/rpi/agc.h +++ b/src/ipa/raspberrypi/controller/rpi/agc.h @@ -98,7 +98,7 @@ private: void housekeepConfig(); void fetchCurrentExposure(Metadata *imageMetadata); void fetchAwbStatus(Metadata *imageMetadata); - void computeGain(bcm2835_isp_stats *statistics, Metadata *imageMetadata, + void computeGain(StatisticsPtr &statistics, Metadata *imageMetadata, double &gain, double &targetY); void computeTargetExposure(double gain); bool applyDigitalGain(double gain, double targetY); diff --git a/src/ipa/raspberrypi/controller/rpi/alsc.cpp b/src/ipa/raspberrypi/controller/rpi/alsc.cpp index a4afaf841c41..eb4e2f9496e1 100644 --- a/src/ipa/raspberrypi/controller/rpi/alsc.cpp +++ b/src/ipa/raspberrypi/controller/rpi/alsc.cpp @@ -310,19 +310,21 @@ double getCt(Metadata *metadata, double defaultCt) return awbStatus.temperatureK; } -static void copyStats(bcm2835_isp_stats_region regions[XY], StatisticsPtr &stats, +static void copyStats(RgbyRegions ®ions, StatisticsPtr &stats, AlscStatus const &status) { - bcm2835_isp_stats_region *inputRegions = stats->awb_stats; + if (!regions.numRegions()) + regions.init(stats->awbRegions.size()); + double *rTable = (double *)status.r; double *gTable = (double *)status.g; double *bTable = (double *)status.b; - for (int i = 0; i < XY; i++) { - regions[i].r_sum = inputRegions[i].r_sum / rTable[i]; - regions[i].g_sum = inputRegions[i].g_sum / gTable[i]; - regions[i].b_sum = inputRegions[i].b_sum / bTable[i]; - regions[i].counted = inputRegions[i].counted; - /* (don't care about the uncounted value) */ + for (unsigned int i = 0; i < stats->awbRegions.numRegions(); i++) { + auto r = stats->awbRegions.get(i); + r.val.rSum = static_cast(r.val.rSum / rTable[i]); + r.val.gSum = static_cast(r.val.gSum / gTable[i]); + r.val.bSum = static_cast(r.val.bSum / bTable[i]); + regions.set(i, r); } } @@ -512,19 +514,19 @@ void resampleCalTable(double const calTableIn[XY], } /* Calculate chrominance statistics (R/G and B/G) for each region. */ -static_assert(XY == AWB_REGIONS, "ALSC/AWB statistics region mismatch"); -static void calculateCrCb(bcm2835_isp_stats_region *awbRegion, double cr[XY], +static void calculateCrCb(const RgbyRegions &awbRegion, double cr[XY], double cb[XY], uint32_t minCount, uint16_t minG) { for (int i = 0; i < XY; i++) { - bcm2835_isp_stats_region &zone = awbRegion[i]; - if (zone.counted <= minCount || - zone.g_sum / zone.counted <= minG) { + auto s = awbRegion.get(i); + + if (s.counted <= minCount || s.val.gSum / s.counted <= minG) { cr[i] = cb[i] = InsufficientData; continue; } - cr[i] = zone.r_sum / (double)zone.g_sum; - cb[i] = zone.b_sum / (double)zone.g_sum; + + cr[i] = s.val.rSum / (double)s.val.gSum; + cb[i] = s.val.bSum / (double)s.val.gSum; } } diff --git a/src/ipa/raspberrypi/controller/rpi/alsc.h b/src/ipa/raspberrypi/controller/rpi/alsc.h index a858ef5a6551..9167c9ffa2e3 100644 --- a/src/ipa/raspberrypi/controller/rpi/alsc.h +++ b/src/ipa/raspberrypi/controller/rpi/alsc.h @@ -12,6 +12,7 @@ #include "../algorithm.h" #include "../alsc_status.h" +#include "../statistics.h" namespace RPiController { @@ -98,7 +99,7 @@ private: /* copy out the results from the async thread so that it can be restarted */ void fetchAsyncResults(); double ct_; - bcm2835_isp_stats_region statistics_[AlscCellsY * AlscCellsX]; + RgbyRegions statistics_; double asyncResults_[3][AlscCellsY][AlscCellsX]; double asyncLambdaR_[AlscCellsX * AlscCellsY]; double asyncLambdaB_[AlscCellsX * AlscCellsY]; diff --git a/src/ipa/raspberrypi/controller/rpi/awb.cpp b/src/ipa/raspberrypi/controller/rpi/awb.cpp index 861022014896..c089aedf020c 100644 --- a/src/ipa/raspberrypi/controller/rpi/awb.cpp +++ b/src/ipa/raspberrypi/controller/rpi/awb.cpp @@ -21,9 +21,6 @@ LOG_DEFINE_CATEGORY(RPiAwb) #define NAME "rpi.awb" -static constexpr unsigned int AwbStatsSizeX = DEFAULT_AWB_REGIONS_X; -static constexpr unsigned int AwbStatsSizeY = DEFAULT_AWB_REGIONS_Y; - /* * todo - the locking in this algorithm needs some tidying up as has been done * elsewhere (ALSC and AGC). @@ -406,17 +403,16 @@ void Awb::asyncFunc() } static void generateStats(std::vector &zones, - bcm2835_isp_stats_region *stats, double minPixels, + RgbyRegions &stats, double minPixels, double minG) { - for (unsigned int i = 0; i < AwbStatsSizeX * AwbStatsSizeY; i++) { + for (auto const ®ion : stats) { Awb::RGB zone; - double counted = stats[i].counted; - if (counted >= minPixels) { - zone.G = stats[i].g_sum / counted; + if (region.counted >= minPixels) { + zone.G = region.val.gSum / region.counted; if (zone.G >= minG) { - zone.R = stats[i].r_sum / counted; - zone.B = stats[i].b_sum / counted; + zone.R = region.val.rSum / region.counted; + zone.B = region.val.bSum / region.counted; zones.push_back(zone); } } @@ -430,7 +426,7 @@ void Awb::prepareStats() * LSC has already been applied to the stats in this pipeline, so stop * any LSC compensation. We also ignore config_.fast in this version. */ - generateStats(zones_, statistics_->awb_stats, config_.minPixels, + generateStats(zones_, statistics_->awbRegions, config_.minPixels, config_.minG); /* * apply sensitivities, so values appear to come from our "canonical" @@ -646,7 +642,7 @@ void Awb::awbBayes() * valid... not entirely sure about this. */ Pwl prior = interpolatePrior(); - prior *= zones_.size() / (double)(AwbStatsSizeX * AwbStatsSizeY); + prior *= zones_.size() / (double)(statistics_->awbRegions.numRegions()); prior.map([](double x, double y) { LOG(RPiAwb, Debug) << "(" << x << "," << y << ")"; }); diff --git a/src/ipa/raspberrypi/controller/rpi/awb.h b/src/ipa/raspberrypi/controller/rpi/awb.h index 30acd89d0969..d81779dd51ff 100644 --- a/src/ipa/raspberrypi/controller/rpi/awb.h +++ b/src/ipa/raspberrypi/controller/rpi/awb.h @@ -13,6 +13,7 @@ #include "../awb_algorithm.h" #include "../pwl.h" #include "../awb_status.h" +#include "../statistics.h" namespace RPiController { diff --git a/src/ipa/raspberrypi/controller/rpi/contrast.cpp b/src/ipa/raspberrypi/controller/rpi/contrast.cpp index 5b37edcbd46a..a4f8c4f04fc4 100644 --- a/src/ipa/raspberrypi/controller/rpi/contrast.cpp +++ b/src/ipa/raspberrypi/controller/rpi/contrast.cpp @@ -106,7 +106,7 @@ Pwl computeStretchCurve(Histogram const &histogram, * bit. */ double histLo = histogram.quantile(config.loHistogram) * - (65536 / NUM_HISTOGRAM_BINS); + (65536 / histogram.bins()); double levelLo = config.loLevel * 65536; LOG(RPiContrast, Debug) << "Move histogram point " << histLo << " to " << levelLo; @@ -119,7 +119,7 @@ Pwl computeStretchCurve(Histogram const &histogram, * Keep the mid-point (median) in the same place, though, to limit the * apparent amount of global brightness shift. */ - double mid = histogram.quantile(0.5) * (65536 / NUM_HISTOGRAM_BINS); + double mid = histogram.quantile(0.5) * (65536 / histogram.bins()); enhance.append(mid, mid); /* @@ -127,7 +127,7 @@ Pwl computeStretchCurve(Histogram const &histogram, * there up. */ double histHi = histogram.quantile(config.hiHistogram) * - (65536 / NUM_HISTOGRAM_BINS); + (65536 / histogram.bins()); double levelHi = config.hiLevel * 65536; LOG(RPiContrast, Debug) << "Move histogram point " << histHi << " to " << levelHi; @@ -158,7 +158,7 @@ Pwl applyManualContrast(Pwl const &gammaCurve, double brightness, void Contrast::process(StatisticsPtr &stats, [[maybe_unused]] Metadata *imageMetadata) { - Histogram histogram(stats->hist[0].g_hist, NUM_HISTOGRAM_BINS); + Histogram &histogram = stats->yHist; /* * We look at the histogram and adjust the gamma curve in the following * ways: 1. Adjust the gamma curve so as to pull the start of the diff --git a/src/ipa/raspberrypi/controller/rpi/focus.cpp b/src/ipa/raspberrypi/controller/rpi/focus.cpp index 8c5029bd0e95..ea3cc00e42c3 100644 --- a/src/ipa/raspberrypi/controller/rpi/focus.cpp +++ b/src/ipa/raspberrypi/controller/rpi/focus.cpp @@ -31,10 +31,9 @@ char const *Focus::name() const void Focus::process(StatisticsPtr &stats, Metadata *imageMetadata) { FocusStatus status; - unsigned int i; - for (i = 0; i < FOCUS_REGIONS; i++) - status.focusMeasures[i] = stats->focus_stats[i].contrast_val[1][1] / 1000; - status.num = i; + for (unsigned int i = 0; i < stats->focusRegions.numRegions(); i++) + status.focusMeasures[i] = stats->focusRegions.get(i).val; + status.num = stats->focusRegions.numRegions(); imageMetadata->set("focus.status", status); LOG(RPiFocus, Debug) diff --git a/src/ipa/raspberrypi/controller/rpi/lux.cpp b/src/ipa/raspberrypi/controller/rpi/lux.cpp index 9759186afacf..06625f3a5ea3 100644 --- a/src/ipa/raspberrypi/controller/rpi/lux.cpp +++ b/src/ipa/raspberrypi/controller/rpi/lux.cpp @@ -6,8 +6,6 @@ */ #include -#include - #include #include "../device_status.h" @@ -83,20 +81,12 @@ void Lux::process(StatisticsPtr &stats, Metadata *imageMetadata) if (imageMetadata->get("device.status", deviceStatus) == 0) { double currentGain = deviceStatus.analogueGain; double currentAperture = deviceStatus.aperture.value_or(currentAperture_); - uint64_t sum = 0; - uint32_t num = 0; - uint32_t *bin = stats->hist[0].g_hist; - const int numBins = sizeof(stats->hist[0].g_hist) / - sizeof(stats->hist[0].g_hist[0]); - for (int i = 0; i < numBins; i++) - sum += bin[i] * (uint64_t)i, num += bin[i]; - /* add .5 to reflect the mid-points of bins */ - double currentY = sum / (double)num + .5; + double currentY = stats->yHist.interQuantileMean(0, 1); double gainRatio = referenceGain_ / currentGain; double shutterSpeedRatio = referenceShutterSpeed_ / deviceStatus.shutterSpeed; double apertureRatio = referenceAperture_ / currentAperture; - double yRatio = currentY * (65536 / numBins) / referenceY_; + double yRatio = currentY * (65536 / stats->yHist.bins()) / referenceY_; double estimatedLux = shutterSpeedRatio * gainRatio * apertureRatio * apertureRatio * yRatio * referenceLux_; diff --git a/src/ipa/raspberrypi/raspberrypi.cpp b/src/ipa/raspberrypi/raspberrypi.cpp index b74f1ecf738f..44527357243d 100644 --- a/src/ipa/raspberrypi/raspberrypi.cpp +++ b/src/ipa/raspberrypi/raspberrypi.cpp @@ -51,6 +51,7 @@ #include "metadata.h" #include "sharpen_algorithm.h" #include "sharpen_status.h" +#include "statistics.h" namespace libcamera { @@ -136,6 +137,7 @@ private: void prepareISP(const ISPConfig &data); void reportMetadata(); void fillDeviceStatus(const ControlList &sensorControls); + RPiController::StatisticsPtr fillStatistics(bcm2835_isp_stats *stats) const; void processStats(unsigned int bufferId); void applyFrameDurations(Duration minFrameDuration, Duration maxFrameDuration); void applyAGC(const struct AgcStatus *agcStatus, ControlList &ctrls); @@ -1119,6 +1121,46 @@ void IPARPi::fillDeviceStatus(const ControlList &sensorControls) rpiMetadata_.set("device.status", deviceStatus); } +RPiController::StatisticsPtr IPARPi::fillStatistics(bcm2835_isp_stats *stats) const +{ + using namespace RPiController; + + unsigned int i; + StatisticsPtr statistics = + std::make_unique(Statistics::AgcStatsPos::PreWb, Statistics::ColourStatsPos::PostLsc); + + /* RGB histograms are not used, so do not populate them. */ + statistics->yHist = std::move(RPiController::Histogram(stats->hist[0].g_hist, NUM_HISTOGRAM_BINS)); + + statistics->awbRegions.init({ DEFAULT_AWB_REGIONS_X, DEFAULT_AWB_REGIONS_Y }); + for (i = 0; i < statistics->awbRegions.numRegions(); i++) + statistics->awbRegions.set(i, { { stats->awb_stats[i].r_sum, + stats->awb_stats[i].g_sum, + stats->awb_stats[i].b_sum }, + stats->awb_stats[i].counted, + stats->awb_stats[i].notcounted }); + + /* + * There are only ever 15 regions computed by the firmware due to zoning, + * but the HW defines AGC_REGIONS == 16! + */ + statistics->agcRegions.init(15); + for (i = 0; i < statistics->agcRegions.numRegions(); i++) + statistics->agcRegions.set(i, { { stats->agc_stats[i].r_sum, + stats->agc_stats[i].g_sum, + stats->agc_stats[i].b_sum }, + stats->agc_stats[i].counted, + stats->awb_stats[i].notcounted }); + + statistics->focusRegions.init({ 4, 3 }); + for (i = 0; i < statistics->focusRegions.numRegions(); i++) + statistics->focusRegions.set(i, { stats->focus_stats[i].contrast_val[1][1] / 1000, + stats->focus_stats[i].contrast_val_num[1][1], + stats->focus_stats[i].contrast_val_num[1][0] }); + + return statistics; +} + void IPARPi::processStats(unsigned int bufferId) { auto it = buffers_.find(bufferId); @@ -1129,7 +1171,7 @@ void IPARPi::processStats(unsigned int bufferId) Span mem = it->second.planes()[0]; bcm2835_isp_stats *stats = reinterpret_cast(mem.data()); - RPiController::StatisticsPtr statistics = std::make_shared(*stats); + RPiController::StatisticsPtr statistics = fillStatistics(stats); helper_->process(statistics, rpiMetadata_); controller_.process(statistics, &rpiMetadata_); diff --git a/src/ipa/raspberrypi/statistics.h b/src/ipa/raspberrypi/statistics.h index a762bf3d41aa..affb7272c963 100644 --- a/src/ipa/raspberrypi/statistics.h +++ b/src/ipa/raspberrypi/statistics.h @@ -67,4 +67,6 @@ struct Statistics { FocusRegions focusRegions; }; +using StatisticsPtr = std::shared_ptr; + } /* namespace RPiController */ From patchwork Thu Nov 24 10:38:32 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Naushir Patuck X-Patchwork-Id: 17866 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 98B7BBDE6B for ; Thu, 24 Nov 2022 10:38:48 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 500E66331B; Thu, 24 Nov 2022 11:38:48 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1669286328; bh=sTGi7RgxaxNbxoWSQSrBl7HG/qhIfv5sSuH0VB65uFY=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To: From; b=gfGeBlK/DMF0aBm169KN8rlPTClmlCKAkZatj3jY3TNLJNJsj829UFIJvTNiaDAGb DCGc7Pbgnlr7TdbRP8gcBdJRfwo1VyozQyDFuqT/k4foKCPAcZJNITDBIqaisD+nvE Gmr/r8KY2jFJyEp2bBSb+Pqya/B0+Z67yKI0+XxbYwt/mwAgyc0GHgjNc5jntsnWZ2 tQ0nJTPZEFWlYxr6r6qxDOdfAsHoJG+uwYRSVvf1kYcwGLjXm4Bob3jjV8mxoWJdNi yPUKMWlKMuxanW/Dq3Z/n9bBuuOJdN8FTxhgDDFwgnQhi88sLs+rv6drsQHVADp9v8 52j+zOUyzwE7Q== Received: from mail-wr1-x42f.google.com (mail-wr1-x42f.google.com [IPv6:2a00:1450:4864:20::42f]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id DB78363325 for ; Thu, 24 Nov 2022 11:38:43 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="X8Dbr3vy"; dkim-atps=neutral Received: by mail-wr1-x42f.google.com with SMTP id bs21so1858225wrb.4 for ; Thu, 24 Nov 2022 02:38:43 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=p1mDNy10gWoZUW3LMkb06Ux8rbvmE8YDIyd+biq7Yk0=; b=X8Dbr3vyosce7VXOkrNCZnu3OEldS1CUgJ/ZpBT33DvRaMaNo/ScJDZ2Zd7bzlDo6o MYLeTN7MIw/zwlOKj4vgQ7Cq/Pq5RwczXetTVQsxmm5Dip1ukqJCOVVPhmHcoV/UFu26 Oi3MD/9Mlo6P2Hs12vFyi9LzLtEs+Jn0QpLZ/h0qCHJg3AyNttrIG+Q0gPNaxp9KrWZi 87uWnEEesyjBceAiAXzgLY2Blx/tcyEUZWSkxmehdv5Is1esQ9WlUXkUkagZxv6FmwB5 N1RFYmh25w9kuqsAU8ERNwOjVir65a25/qQGzTZPOEe3uVFf7TdpOZTpkjgv5vhoAeXG uNzw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=p1mDNy10gWoZUW3LMkb06Ux8rbvmE8YDIyd+biq7Yk0=; b=BRMS6OZQ1If7jcBFlzG0bTB8ifrpZk07aqcN7bzNk2kCIxjwqxT/mov979QvTuq1tY TIUtcGyx/3GT3fYZouxLLefJ7uLTX7VgjbC1GDN55OGjsaPWxM0qq73Av+BObQd9jFrT vJEuwQnFugfAThXCPX5Jh5PirSTXJbY0aYFB64gOy/y2k8K/S0PkuBU6o5IbwDg5VGG6 YzEBWdBiuzqfqz01fQZAQOndHIt8s6gShEl4MavY4eG4xcqBsVKCaR3GjiuS7sZmA6Yq 1KyaohrmQsKViTvEMgI1RawTRVQjvW7wTK2qxjUzCI6GDgWDGmxrZZvAVjvViAPS6XNE rotQ== X-Gm-Message-State: ANoB5plbHnKpVNx4KhlYb59sK779Uvc52ToeSzxqsgp+FjLxeAvY8+7q 7kDxC5nIedJWSyzO07RWuEEWsFscNW6CDQ== X-Google-Smtp-Source: AA0mqf494r2jvx462NG0h5HZUl4zV+rzNvbpeJJBemn7q7BlLDmgNBWer05iUfcCZ9ghtwqI6MJQOg== X-Received: by 2002:a5d:68c1:0:b0:236:8a38:4deb with SMTP id p1-20020a5d68c1000000b002368a384debmr19629002wrw.487.1669286323204; Thu, 24 Nov 2022 02:38:43 -0800 (PST) Received: from naush-laptop.localdomain ([93.93.133.154]) by smtp.gmail.com with ESMTPSA id s16-20020adff810000000b002368a6deaf8sm995103wrp.57.2022.11.24.02.38.42 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 24 Nov 2022 02:38:42 -0800 (PST) To: libcamera-devel@lists.libcamera.org Date: Thu, 24 Nov 2022 10:38:32 +0000 Message-Id: <20221124103832.6172-6-naush@raspberrypi.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20221124103832.6172-1-naush@raspberrypi.com> References: <20221124103832.6172-1-naush@raspberrypi.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v2 5/5] ipa: raspberrypi: Normalise region sums to 16-bits X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Naushir Patuck via libcamera-devel From: Naushir Patuck Reply-To: Naushir Patuck Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" The VC4 ISP uses a pipeline bit-depth of 13-bits. The AGC algorithm needs to know this bit-depth when computing the Y value for the image. Instead of hardcoding the VC4 bit-depth in the AGC source code, normalise all region sums to 16-bits when filling the Statistics structure. AWB and ALSC are agnostic about pipeline depth, so do not need changing. Signed-off-by: Naushir Patuck Reviewed-by: David Plowman --- src/ipa/raspberrypi/controller/rpi/agc.cpp | 10 ++++------ src/ipa/raspberrypi/raspberrypi.cpp | 16 ++++++++++------ 2 files changed, 14 insertions(+), 12 deletions(-) diff --git a/src/ipa/raspberrypi/controller/rpi/agc.cpp b/src/ipa/raspberrypi/controller/rpi/agc.cpp index a7f0e954407d..e400e478162f 100644 --- a/src/ipa/raspberrypi/controller/rpi/agc.cpp +++ b/src/ipa/raspberrypi/controller/rpi/agc.cpp @@ -28,8 +28,6 @@ LOG_DEFINE_CATEGORY(RPiAgc) #define NAME "rpi.agc" -static constexpr unsigned int PipelineBits = 13; /* seems to be a 13-bit pipeline */ - int AgcMeteringMode::read(const libcamera::YamlObject ¶ms) { const YamlObject &yamlWeights = params["weights"]; @@ -592,9 +590,9 @@ static double computeInitialY(StatisticsPtr &stats, AwbStatus const &awb, double rSum = 0, gSum = 0, bSum = 0, pixelSum = 0; for (unsigned int i = 0; i < stats->agcRegions.numRegions(); i++) { auto ®ion = stats->agcRegions.get(i); - double rAcc = std::min(region.val.rSum * gain, ((1 << PipelineBits) - 1) * region.counted); - double gAcc = std::min(region.val.gSum * gain, ((1 << PipelineBits) - 1) * region.counted); - double bAcc = std::min(region.val.bSum * gain, ((1 << PipelineBits) - 1) * region.counted); + double rAcc = std::min(region.val.rSum * gain, ((1 << 16) - 1) * region.counted); + double gAcc = std::min(region.val.gSum * gain, ((1 << 16) - 1) * region.counted); + double bAcc = std::min(region.val.bSum * gain, ((1 << 16) - 1) * region.counted); rSum += rAcc * weights[i]; gSum += gAcc * weights[i]; bSum += bAcc * weights[i]; @@ -607,7 +605,7 @@ static double computeInitialY(StatisticsPtr &stats, AwbStatus const &awb, double ySum = rSum * awb.gainR * .299 + gSum * awb.gainG * .587 + bSum * awb.gainB * .114; - return ySum / pixelSum / (1 << PipelineBits); + return ySum / pixelSum / (1 << 16); } /* diff --git a/src/ipa/raspberrypi/raspberrypi.cpp b/src/ipa/raspberrypi/raspberrypi.cpp index 44527357243d..ac1875d203ae 100644 --- a/src/ipa/raspberrypi/raspberrypi.cpp +++ b/src/ipa/raspberrypi/raspberrypi.cpp @@ -1132,11 +1132,15 @@ RPiController::StatisticsPtr IPARPi::fillStatistics(bcm2835_isp_stats *stats) co /* RGB histograms are not used, so do not populate them. */ statistics->yHist = std::move(RPiController::Histogram(stats->hist[0].g_hist, NUM_HISTOGRAM_BINS)); + /* + * All region sums are based on a 13-bit pipeline bit-depth. Normalise + * this to 16-bits for the AGC/AWB/ALSC algorithms. + */ statistics->awbRegions.init({ DEFAULT_AWB_REGIONS_X, DEFAULT_AWB_REGIONS_Y }); for (i = 0; i < statistics->awbRegions.numRegions(); i++) - statistics->awbRegions.set(i, { { stats->awb_stats[i].r_sum, - stats->awb_stats[i].g_sum, - stats->awb_stats[i].b_sum }, + statistics->awbRegions.set(i, { { stats->awb_stats[i].r_sum << 3, + stats->awb_stats[i].g_sum << 3, + stats->awb_stats[i].b_sum << 3 }, stats->awb_stats[i].counted, stats->awb_stats[i].notcounted }); @@ -1146,9 +1150,9 @@ RPiController::StatisticsPtr IPARPi::fillStatistics(bcm2835_isp_stats *stats) co */ statistics->agcRegions.init(15); for (i = 0; i < statistics->agcRegions.numRegions(); i++) - statistics->agcRegions.set(i, { { stats->agc_stats[i].r_sum, - stats->agc_stats[i].g_sum, - stats->agc_stats[i].b_sum }, + statistics->agcRegions.set(i, { { stats->agc_stats[i].r_sum << 3, + stats->agc_stats[i].g_sum << 3, + stats->agc_stats[i].b_sum << 3 }, stats->agc_stats[i].counted, stats->awb_stats[i].notcounted });