From patchwork Thu Feb 9 12:47:35 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Naushir Patuck X-Patchwork-Id: 18273 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 58E4CBEFBE for ; Thu, 9 Feb 2023 12:47:42 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 8EC04625F4; Thu, 9 Feb 2023 13:47:41 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1675946861; bh=AJeTN+oLY+858q0CciWxg1g0ObfnNDN+qE2aBw47C2g=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To:Cc: From; b=cI3R0cK0AmoXlFQMz8KC9ih9gUYQbZq7PEBq2zl2SmlDZ393xuRclgpZ3ENiFudCy +cqIu3tog0vDW5EXWy7dd9SJPndBynMLTDdLRRsBvwKY10QtN33cjG06OtwJlCTckB +rtelcKreFs8ze/siy7h5btqejo5Iv1F0Qm/Q9LanGl05miIMpVbrU6gpEnjqe0maW 8wAmiurgfCgJdB5SmB5tmeuJp5ejCFvgfewc8BZehEShU5jY9m3ocgSzVNCOCMnQhL 92kEbkDe++vkvqPfRnUGCYtgN51X8pbxoQC7AuYKbyKkpyfVxLBuj4OpKyEa/wnIB+ uI/G2c3xRXsyw== Received: from mail-wr1-x434.google.com (mail-wr1-x434.google.com [IPv6:2a00:1450:4864:20::434]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 5ACD0603BD for ; Thu, 9 Feb 2023 13:47:40 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="rLnoSjJm"; dkim-atps=neutral Received: by mail-wr1-x434.google.com with SMTP id bu23so1632518wrb.8 for ; Thu, 09 Feb 2023 04:47:40 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=miGxzBjqJd91MQb5jpUim+HImmAw8bvBwbaNUXosn/w=; b=rLnoSjJmR64QYvxUNWv+O2kgXFpbk85sQoN8ueNyv4YKGcW1Jza9L9NqsEx82ILQCA JYYqqmm6QLebieNw3wT8SyRFTsl5Qj1up+aqr3nh3enOjgDtK8qDGQlWAoTJqfa6O9sV aydokr147YpnWPYSMT8px/7WQKrVLwsixhcvpt2EJTE311xJkU2xsqEr8sfMJ9YLd0JI jBoEU5AO/ZDsC6muFTXEmkzmxjtPSg3O2xICITHaH9dUiKukbFEVRFm121IhakMENtfU bqeoHDcss6NKX6DI7T5WuDL2RtSSBSFPEfmOhG7BkE21SETV2R/weZrS8O/lvdZhxoSv vXdg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=miGxzBjqJd91MQb5jpUim+HImmAw8bvBwbaNUXosn/w=; b=t1XckAiIlwfIqVjkCc8/qgx30wBnoIkZwQUEmzEWjtTfhc9lz8wYVE3RDGBWNnYZA4 njS+5rdII86XmCklYk9219VOYHFfSWQOqxUobcT63odiPhcS6yVaqxzonohIrCzgvbLh EUqgxgwlJuSIQohC7mIuW5tdWrx8ySk5OqgE7hpCBr4rBayXSvOoHIzYocu8uzw/HtlR w2wcbBLPcQCUwjICH8NeKcpR1JPy4SOA82+S2BdQf7uT3YkLN2FFog6RONffIYiKt4f/ DzvgaDWWcVC9Oru4VVJBBSD3clKZ/ZpqeV13HM3xuYPS3uTWx+UwEp2alb+VLXNaPf3t EtNg== X-Gm-Message-State: AO0yUKXttr3rU30o2MDnawjM1O8rQfjBD8TWX/TIuR29g4V1ndBJ/hSh CP/yB5keA7XQmziDuAcVUugqm11ArTeNn+lVWiU= X-Google-Smtp-Source: AK7set9F47enXbfkXTcecVRqhNPyK7xOq5lRViFFYdC/cVuXlNHNH19Zei3vMlpq+V7V2K2BIPs9bA== X-Received: by 2002:adf:f503:0:b0:2c4:27e4:495f with SMTP id q3-20020adff503000000b002c427e4495fmr2079161wro.69.1675946859477; Thu, 09 Feb 2023 04:47:39 -0800 (PST) Received: from localhost.localdomain ([93.93.133.154]) by smtp.gmail.com with ESMTPSA id j5-20020a5d4485000000b002be25db0b7bsm1167325wrq.10.2023.02.09.04.47.38 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 09 Feb 2023 04:47:39 -0800 (PST) To: libcamera-devel@lists.libcamera.org Date: Thu, 9 Feb 2023 12:47:35 +0000 Message-Id: <20230209124735.31170-1-naush@raspberrypi.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20230203091734.22803-5-naush@raspberrypi.com> References: <20230203091734.22803-5-naush@raspberrypi.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH] ipa: raspberrypi: Use the generic statistics structure in the algorithms X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Naushir Patuck via libcamera-devel From: Naushir Patuck Reply-To: Naushir Patuck Cc: Nick Hollinghurst Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Repurpose the StatisticsPtr type from being a shared_ptr to shared_ptr. This removes any hardware specific header files and structures from the algorithms source code. Add a new function in the Raspberry Pi IPA to populate the generic statistics structure from the values provided by the hardware in the bcm2835_isp_stats structure. Update the Lux, AWB, AGC, ALSC, Contrast, and Focus algorithms to use the generic statistics structure appropriately in their calculations. Additionally, remove references to any hardware specific headers and defines in these source files. Signed-off-by: Naushir Patuck Reviewed-by: David Plowman Tested-by: Nick Hollinghurst --- src/ipa/raspberrypi/cam_helper_imx708.cpp | 26 ++++++----- src/ipa/raspberrypi/controller/controller.h | 4 +- src/ipa/raspberrypi/controller/rpi/af.cpp | 10 ++--- src/ipa/raspberrypi/controller/rpi/af.h | 8 +++- src/ipa/raspberrypi/controller/rpi/agc.cpp | 27 +++++------- src/ipa/raspberrypi/controller/rpi/agc.h | 2 +- src/ipa/raspberrypi/controller/rpi/alsc.cpp | 32 +++++++------- src/ipa/raspberrypi/controller/rpi/alsc.h | 3 +- src/ipa/raspberrypi/controller/rpi/awb.cpp | 20 ++++----- src/ipa/raspberrypi/controller/rpi/awb.h | 1 + .../raspberrypi/controller/rpi/contrast.cpp | 8 ++-- src/ipa/raspberrypi/controller/rpi/focus.cpp | 7 ++- src/ipa/raspberrypi/controller/rpi/lux.cpp | 14 +----- src/ipa/raspberrypi/raspberrypi.cpp | 44 ++++++++++++++++++- src/ipa/raspberrypi/statistics.h | 2 + 15 files changed, 121 insertions(+), 87 deletions(-) diff --git a/src/ipa/raspberrypi/cam_helper_imx708.cpp b/src/ipa/raspberrypi/cam_helper_imx708.cpp index 4c43c8e7fb7d..1f213d3c0833 100644 --- a/src/ipa/raspberrypi/cam_helper_imx708.cpp +++ b/src/ipa/raspberrypi/cam_helper_imx708.cpp @@ -78,14 +78,14 @@ private: bool parseAEHist(const uint8_t *ptr, size_t len, unsigned bpp); void putAGCStatistics(StatisticsPtr stats); - uint32_t aeHistLinear_[128]; + Histogram aeHistLinear_; uint32_t aeHistAverage_; bool aeHistValid_; }; CamHelperImx708::CamHelperImx708() : CamHelper(std::make_unique(registerList), frameIntegrationDiff), - aeHistLinear_{ 0 }, aeHistAverage_(0), aeHistValid_(false) + aeHistLinear_{}, aeHistAverage_(0), aeHistValid_(false) { } @@ -264,9 +264,11 @@ bool CamHelperImx708::parsePdafData(const uint8_t *ptr, size_t len, bool CamHelperImx708::parseAEHist(const uint8_t *ptr, size_t len, unsigned bpp) { - static const uint32_t ISP_PIPELINE_BITS = 13; + static constexpr unsigned int PipelineBits = Statistics::NormalisationFactorPow2; + uint64_t count = 0, sum = 0; size_t step = bpp >> 1; /* bytes per histogram bin */ + uint32_t hist[128]; if (len < 144 * step) return false; @@ -280,12 +282,12 @@ bool CamHelperImx708::parseAEHist(const uint8_t *ptr, size_t len, unsigned bpp) if (ptr[3] != 0x55) return false; uint32_t c = (ptr[0] << 14) + (ptr[1] << 6) + (ptr[2] >> 2); - aeHistLinear_[i] = c >> 2; /* pixels to quads */ + hist[i] = c >> 2; /* pixels to quads */ if (i != 0) { count += c; sum += c * - (i * (1u << (ISP_PIPELINE_BITS - 7)) + - (1u << (ISP_PIPELINE_BITS - 8))); + (i * (1u << (PipelineBits - 7)) + + (1u << (PipelineBits - 8))); } ptr += step; } @@ -301,15 +303,16 @@ bool CamHelperImx708::parseAEHist(const uint8_t *ptr, size_t len, unsigned bpp) uint32_t c = (ptr[0] << 14) + (ptr[1] << 6) + (ptr[2] >> 2); count += c; sum += c * - ((3u << ISP_PIPELINE_BITS) >> (17 - i)); + ((3u << PipelineBits) >> (17 - i)); ptr += step; } if ((unsigned)((ptr[0] << 12) + (ptr[1] << 4) + (ptr[2] >> 4)) != - aeHistLinear_[1]) { + hist[1]) { LOG(IPARPI, Error) << "Lin/Log histogram mismatch"; return false; } + aeHistLinear_ = Histogram(hist, 128); aeHistAverage_ = count ? (sum / count) : 0; return count != 0; @@ -329,13 +332,12 @@ void CamHelperImx708::putAGCStatistics(StatisticsPtr stats) * scaled by a fiddle-factor so that a conventional (non-HDR) y_target * of e.g. 0.17 will map to a suitable level for HDR. */ - memcpy(stats->hist[0].g_hist, aeHistLinear_, sizeof(stats->hist[0].g_hist)); + stats->yHist = aeHistLinear_; constexpr unsigned int HdrHeadroomFactor = 4; uint64_t v = HdrHeadroomFactor * aeHistAverage_; - for (int i = 0; i < AGC_REGIONS; i++) { - struct bcm2835_isp_stats_region &r = stats->agc_stats[i]; - r.r_sum = r.b_sum = r.g_sum = r.counted * v; + for (auto ®ion : stats->agcRegions) { + region.val.rSum = region.val.gSum = region.val.bSum = region.counted * v; } } diff --git a/src/ipa/raspberrypi/controller/controller.h b/src/ipa/raspberrypi/controller/controller.h index 3e1e051703b3..e6c950c3a509 100644 --- a/src/ipa/raspberrypi/controller/controller.h +++ b/src/ipa/raspberrypi/controller/controller.h @@ -15,19 +15,17 @@ #include #include -#include - #include "libcamera/internal/yaml_parser.h" #include "camera_mode.h" #include "device_status.h" #include "metadata.h" +#include "statistics.h" namespace RPiController { class Algorithm; typedef std::unique_ptr AlgorithmPtr; -typedef std::shared_ptr StatisticsPtr; /* * The Controller holds a pointer to some global_metadata, which is how diff --git a/src/ipa/raspberrypi/controller/rpi/af.cpp b/src/ipa/raspberrypi/controller/rpi/af.cpp index 2e72f239fb25..a623651875f2 100644 --- a/src/ipa/raspberrypi/controller/rpi/af.cpp +++ b/src/ipa/raspberrypi/controller/rpi/af.cpp @@ -352,14 +352,12 @@ bool Af::getPhase(PdafData const &data, double &phase, double &conf) const } } -double Af::getContrast(struct bcm2835_isp_stats_focus const focus_stats[FOCUS_REGIONS]) const +double Af::getContrast(const FocusRegions &focusStats) const { uint32_t sumWc = 0; - for (unsigned i = 0; i < FOCUS_REGIONS; ++i) { - unsigned w = contrastWeights_[i]; - sumWc += w * (focus_stats[i].contrast_val[1][1] >> 10); - } + for (unsigned i = 0; i < focusStats.numRegions(); ++i) + sumWc += contrastWeights_[i] * focusStats.get(i).val; return (sumWeights_ == 0) ? 0.0 : (double)sumWc / (double)sumWeights_; } @@ -666,7 +664,7 @@ void Af::prepare(Metadata *imageMetadata) void Af::process(StatisticsPtr &stats, [[maybe_unused]] Metadata *imageMetadata) { (void)imageMetadata; - prevContrast_ = getContrast(stats->focus_stats); + prevContrast_ = getContrast(stats->focusRegions); } /* Controls */ diff --git a/src/ipa/raspberrypi/controller/rpi/af.h b/src/ipa/raspberrypi/controller/rpi/af.h index f7baf8979873..7959371baf64 100644 --- a/src/ipa/raspberrypi/controller/rpi/af.h +++ b/src/ipa/raspberrypi/controller/rpi/af.h @@ -11,6 +11,12 @@ #include "../pdaf_data.h" #include "../pwl.h" +/* + * \todo FOCUS_REGIONS is taken from bcm2835-isp.h, but should be made as a + * generic RegionStats structure. + */ +#define FOCUS_REGIONS 12 + /* * This algorithm implements a hybrid of CDAF and PDAF, favouring PDAF. * @@ -117,7 +123,7 @@ private: void computeWeights(); bool getPhase(PdafData const &data, double &phase, double &conf) const; - double getContrast(struct bcm2835_isp_stats_focus const focus_stats[FOCUS_REGIONS]) const; + double getContrast(const FocusRegions &focusStats) const; void doPDAF(double phase, double conf); bool earlyTerminationByPhase(double phase); double findPeak(unsigned index) const; diff --git a/src/ipa/raspberrypi/controller/rpi/agc.cpp b/src/ipa/raspberrypi/controller/rpi/agc.cpp index 46dcc81ae14c..868c30f03d66 100644 --- a/src/ipa/raspberrypi/controller/rpi/agc.cpp +++ b/src/ipa/raspberrypi/controller/rpi/agc.cpp @@ -9,8 +9,6 @@ #include #include -#include - #include #include "../awb_status.h" @@ -451,7 +449,7 @@ void Agc::process(StatisticsPtr &stats, Metadata *imageMetadata) fetchCurrentExposure(imageMetadata); /* Compute the total gain we require relative to the current exposure. */ double gain, targetY; - computeGain(stats.get(), imageMetadata, gain, targetY); + computeGain(stats, imageMetadata, gain, targetY); /* Now compute the target (final) exposure which we think we want. */ computeTargetExposure(gain); /* @@ -585,24 +583,23 @@ void Agc::fetchAwbStatus(Metadata *imageMetadata) LOG(RPiAgc, Debug) << "No AWB status found"; } -static double computeInitialY(bcm2835_isp_stats *stats, AwbStatus const &awb, +static double computeInitialY(StatisticsPtr &stats, AwbStatus const &awb, double weights[], double gain) { - bcm2835_isp_stats_region *regions = stats->agc_stats; /* * Note how the calculation below means that equal weights give you * "average" metering (i.e. all pixels equally important). */ double rSum = 0, gSum = 0, bSum = 0, pixelSum = 0; - for (unsigned int i = 0; i < AgcStatsSize; i++) { - double counted = regions[i].counted; - double rAcc = std::min(regions[i].r_sum * gain, ((1 << PipelineBits) - 1) * counted); - double gAcc = std::min(regions[i].g_sum * gain, ((1 << PipelineBits) - 1) * counted); - double bAcc = std::min(regions[i].b_sum * gain, ((1 << PipelineBits) - 1) * counted); + for (unsigned int i = 0; i < stats->agcRegions.numRegions(); i++) { + auto ®ion = stats->agcRegions.get(i); + double rAcc = std::min(region.val.rSum * gain, ((1 << PipelineBits) - 1) * region.counted); + double gAcc = std::min(region.val.gSum * gain, ((1 << PipelineBits) - 1) * region.counted); + double bAcc = std::min(region.val.bSum * gain, ((1 << PipelineBits) - 1) * region.counted); rSum += rAcc * weights[i]; gSum += gAcc * weights[i]; bSum += bAcc * weights[i]; - pixelSum += counted * weights[i]; + pixelSum += region.counted * weights[i]; } if (pixelSum == 0.0) { LOG(RPiAgc, Warning) << "computeInitialY: pixelSum is zero"; @@ -624,23 +621,23 @@ static double computeInitialY(bcm2835_isp_stats *stats, AwbStatus const &awb, static constexpr double EvGainYTargetLimit = 0.9; -static double constraintComputeGain(AgcConstraint &c, Histogram &h, double lux, +static double constraintComputeGain(AgcConstraint &c, const Histogram &h, double lux, double evGain, double &targetY) { targetY = c.yTarget.eval(c.yTarget.domain().clip(lux)); targetY = std::min(EvGainYTargetLimit, targetY * evGain); double iqm = h.interQuantileMean(c.qLo, c.qHi); - return (targetY * NUM_HISTOGRAM_BINS) / iqm; + return (targetY * h.bins()) / iqm; } -void Agc::computeGain(bcm2835_isp_stats *statistics, Metadata *imageMetadata, +void Agc::computeGain(StatisticsPtr &statistics, Metadata *imageMetadata, double &gain, double &targetY) { struct LuxStatus lux = {}; lux.lux = 400; /* default lux level to 400 in case no metadata found */ if (imageMetadata->get("lux.status", lux) != 0) LOG(RPiAgc, Warning) << "No lux level found"; - Histogram h(statistics->hist[0].g_hist, NUM_HISTOGRAM_BINS); + const Histogram &h = statistics->yHist; double evGain = status_.ev * config_.baseEv; /* * The initial gain and target_Y come from some of the regions. After diff --git a/src/ipa/raspberrypi/controller/rpi/agc.h b/src/ipa/raspberrypi/controller/rpi/agc.h index cf04da1973f1..f04896ca25ad 100644 --- a/src/ipa/raspberrypi/controller/rpi/agc.h +++ b/src/ipa/raspberrypi/controller/rpi/agc.h @@ -96,7 +96,7 @@ private: void housekeepConfig(); void fetchCurrentExposure(Metadata *imageMetadata); void fetchAwbStatus(Metadata *imageMetadata); - void computeGain(bcm2835_isp_stats *statistics, Metadata *imageMetadata, + void computeGain(StatisticsPtr &statistics, Metadata *imageMetadata, double &gain, double &targetY); void computeTargetExposure(double gain); bool applyDigitalGain(double gain, double targetY); diff --git a/src/ipa/raspberrypi/controller/rpi/alsc.cpp b/src/ipa/raspberrypi/controller/rpi/alsc.cpp index a4afaf841c41..eb4e2f9496e1 100644 --- a/src/ipa/raspberrypi/controller/rpi/alsc.cpp +++ b/src/ipa/raspberrypi/controller/rpi/alsc.cpp @@ -310,19 +310,21 @@ double getCt(Metadata *metadata, double defaultCt) return awbStatus.temperatureK; } -static void copyStats(bcm2835_isp_stats_region regions[XY], StatisticsPtr &stats, +static void copyStats(RgbyRegions ®ions, StatisticsPtr &stats, AlscStatus const &status) { - bcm2835_isp_stats_region *inputRegions = stats->awb_stats; + if (!regions.numRegions()) + regions.init(stats->awbRegions.size()); + double *rTable = (double *)status.r; double *gTable = (double *)status.g; double *bTable = (double *)status.b; - for (int i = 0; i < XY; i++) { - regions[i].r_sum = inputRegions[i].r_sum / rTable[i]; - regions[i].g_sum = inputRegions[i].g_sum / gTable[i]; - regions[i].b_sum = inputRegions[i].b_sum / bTable[i]; - regions[i].counted = inputRegions[i].counted; - /* (don't care about the uncounted value) */ + for (unsigned int i = 0; i < stats->awbRegions.numRegions(); i++) { + auto r = stats->awbRegions.get(i); + r.val.rSum = static_cast(r.val.rSum / rTable[i]); + r.val.gSum = static_cast(r.val.gSum / gTable[i]); + r.val.bSum = static_cast(r.val.bSum / bTable[i]); + regions.set(i, r); } } @@ -512,19 +514,19 @@ void resampleCalTable(double const calTableIn[XY], } /* Calculate chrominance statistics (R/G and B/G) for each region. */ -static_assert(XY == AWB_REGIONS, "ALSC/AWB statistics region mismatch"); -static void calculateCrCb(bcm2835_isp_stats_region *awbRegion, double cr[XY], +static void calculateCrCb(const RgbyRegions &awbRegion, double cr[XY], double cb[XY], uint32_t minCount, uint16_t minG) { for (int i = 0; i < XY; i++) { - bcm2835_isp_stats_region &zone = awbRegion[i]; - if (zone.counted <= minCount || - zone.g_sum / zone.counted <= minG) { + auto s = awbRegion.get(i); + + if (s.counted <= minCount || s.val.gSum / s.counted <= minG) { cr[i] = cb[i] = InsufficientData; continue; } - cr[i] = zone.r_sum / (double)zone.g_sum; - cb[i] = zone.b_sum / (double)zone.g_sum; + + cr[i] = s.val.rSum / (double)s.val.gSum; + cb[i] = s.val.bSum / (double)s.val.gSum; } } diff --git a/src/ipa/raspberrypi/controller/rpi/alsc.h b/src/ipa/raspberrypi/controller/rpi/alsc.h index a858ef5a6551..9167c9ffa2e3 100644 --- a/src/ipa/raspberrypi/controller/rpi/alsc.h +++ b/src/ipa/raspberrypi/controller/rpi/alsc.h @@ -12,6 +12,7 @@ #include "../algorithm.h" #include "../alsc_status.h" +#include "../statistics.h" namespace RPiController { @@ -98,7 +99,7 @@ private: /* copy out the results from the async thread so that it can be restarted */ void fetchAsyncResults(); double ct_; - bcm2835_isp_stats_region statistics_[AlscCellsY * AlscCellsX]; + RgbyRegions statistics_; double asyncResults_[3][AlscCellsY][AlscCellsX]; double asyncLambdaR_[AlscCellsX * AlscCellsY]; double asyncLambdaB_[AlscCellsX * AlscCellsY]; diff --git a/src/ipa/raspberrypi/controller/rpi/awb.cpp b/src/ipa/raspberrypi/controller/rpi/awb.cpp index 04d1c8783654..ef3435d66106 100644 --- a/src/ipa/raspberrypi/controller/rpi/awb.cpp +++ b/src/ipa/raspberrypi/controller/rpi/awb.cpp @@ -21,9 +21,6 @@ LOG_DEFINE_CATEGORY(RPiAwb) #define NAME "rpi.awb" -static constexpr unsigned int AwbStatsSizeX = DEFAULT_AWB_REGIONS_X; -static constexpr unsigned int AwbStatsSizeY = DEFAULT_AWB_REGIONS_Y; - /* * todo - the locking in this algorithm needs some tidying up as has been done * elsewhere (ALSC and AGC). @@ -401,17 +398,16 @@ void Awb::asyncFunc() } static void generateStats(std::vector &zones, - bcm2835_isp_stats_region *stats, double minPixels, + RgbyRegions &stats, double minPixels, double minG) { - for (unsigned int i = 0; i < AwbStatsSizeX * AwbStatsSizeY; i++) { + for (auto const ®ion : stats) { Awb::RGB zone; - double counted = stats[i].counted; - if (counted >= minPixels) { - zone.G = stats[i].g_sum / counted; + if (region.counted >= minPixels) { + zone.G = region.val.gSum / region.counted; if (zone.G >= minG) { - zone.R = stats[i].r_sum / counted; - zone.B = stats[i].b_sum / counted; + zone.R = region.val.rSum / region.counted; + zone.B = region.val.bSum / region.counted; zones.push_back(zone); } } @@ -425,7 +421,7 @@ void Awb::prepareStats() * LSC has already been applied to the stats in this pipeline, so stop * any LSC compensation. We also ignore config_.fast in this version. */ - generateStats(zones_, statistics_->awb_stats, config_.minPixels, + generateStats(zones_, statistics_->awbRegions, config_.minPixels, config_.minG); /* * apply sensitivities, so values appear to come from our "canonical" @@ -641,7 +637,7 @@ void Awb::awbBayes() * valid... not entirely sure about this. */ Pwl prior = interpolatePrior(); - prior *= zones_.size() / (double)(AwbStatsSizeX * AwbStatsSizeY); + prior *= zones_.size() / (double)(statistics_->awbRegions.numRegions()); prior.map([](double x, double y) { LOG(RPiAwb, Debug) << "(" << x << "," << y << ")"; }); diff --git a/src/ipa/raspberrypi/controller/rpi/awb.h b/src/ipa/raspberrypi/controller/rpi/awb.h index 2254c3ed2cc4..e7d49cd8036b 100644 --- a/src/ipa/raspberrypi/controller/rpi/awb.h +++ b/src/ipa/raspberrypi/controller/rpi/awb.h @@ -13,6 +13,7 @@ #include "../awb_algorithm.h" #include "../pwl.h" #include "../awb_status.h" +#include "../statistics.h" namespace RPiController { diff --git a/src/ipa/raspberrypi/controller/rpi/contrast.cpp b/src/ipa/raspberrypi/controller/rpi/contrast.cpp index 5b37edcbd46a..a4f8c4f04fc4 100644 --- a/src/ipa/raspberrypi/controller/rpi/contrast.cpp +++ b/src/ipa/raspberrypi/controller/rpi/contrast.cpp @@ -106,7 +106,7 @@ Pwl computeStretchCurve(Histogram const &histogram, * bit. */ double histLo = histogram.quantile(config.loHistogram) * - (65536 / NUM_HISTOGRAM_BINS); + (65536 / histogram.bins()); double levelLo = config.loLevel * 65536; LOG(RPiContrast, Debug) << "Move histogram point " << histLo << " to " << levelLo; @@ -119,7 +119,7 @@ Pwl computeStretchCurve(Histogram const &histogram, * Keep the mid-point (median) in the same place, though, to limit the * apparent amount of global brightness shift. */ - double mid = histogram.quantile(0.5) * (65536 / NUM_HISTOGRAM_BINS); + double mid = histogram.quantile(0.5) * (65536 / histogram.bins()); enhance.append(mid, mid); /* @@ -127,7 +127,7 @@ Pwl computeStretchCurve(Histogram const &histogram, * there up. */ double histHi = histogram.quantile(config.hiHistogram) * - (65536 / NUM_HISTOGRAM_BINS); + (65536 / histogram.bins()); double levelHi = config.hiLevel * 65536; LOG(RPiContrast, Debug) << "Move histogram point " << histHi << " to " << levelHi; @@ -158,7 +158,7 @@ Pwl applyManualContrast(Pwl const &gammaCurve, double brightness, void Contrast::process(StatisticsPtr &stats, [[maybe_unused]] Metadata *imageMetadata) { - Histogram histogram(stats->hist[0].g_hist, NUM_HISTOGRAM_BINS); + Histogram &histogram = stats->yHist; /* * We look at the histogram and adjust the gamma curve in the following * ways: 1. Adjust the gamma curve so as to pull the start of the diff --git a/src/ipa/raspberrypi/controller/rpi/focus.cpp b/src/ipa/raspberrypi/controller/rpi/focus.cpp index 8c5029bd0e95..ea3cc00e42c3 100644 --- a/src/ipa/raspberrypi/controller/rpi/focus.cpp +++ b/src/ipa/raspberrypi/controller/rpi/focus.cpp @@ -31,10 +31,9 @@ char const *Focus::name() const void Focus::process(StatisticsPtr &stats, Metadata *imageMetadata) { FocusStatus status; - unsigned int i; - for (i = 0; i < FOCUS_REGIONS; i++) - status.focusMeasures[i] = stats->focus_stats[i].contrast_val[1][1] / 1000; - status.num = i; + for (unsigned int i = 0; i < stats->focusRegions.numRegions(); i++) + status.focusMeasures[i] = stats->focusRegions.get(i).val; + status.num = stats->focusRegions.numRegions(); imageMetadata->set("focus.status", status); LOG(RPiFocus, Debug) diff --git a/src/ipa/raspberrypi/controller/rpi/lux.cpp b/src/ipa/raspberrypi/controller/rpi/lux.cpp index 9759186afacf..06625f3a5ea3 100644 --- a/src/ipa/raspberrypi/controller/rpi/lux.cpp +++ b/src/ipa/raspberrypi/controller/rpi/lux.cpp @@ -6,8 +6,6 @@ */ #include -#include - #include #include "../device_status.h" @@ -83,20 +81,12 @@ void Lux::process(StatisticsPtr &stats, Metadata *imageMetadata) if (imageMetadata->get("device.status", deviceStatus) == 0) { double currentGain = deviceStatus.analogueGain; double currentAperture = deviceStatus.aperture.value_or(currentAperture_); - uint64_t sum = 0; - uint32_t num = 0; - uint32_t *bin = stats->hist[0].g_hist; - const int numBins = sizeof(stats->hist[0].g_hist) / - sizeof(stats->hist[0].g_hist[0]); - for (int i = 0; i < numBins; i++) - sum += bin[i] * (uint64_t)i, num += bin[i]; - /* add .5 to reflect the mid-points of bins */ - double currentY = sum / (double)num + .5; + double currentY = stats->yHist.interQuantileMean(0, 1); double gainRatio = referenceGain_ / currentGain; double shutterSpeedRatio = referenceShutterSpeed_ / deviceStatus.shutterSpeed; double apertureRatio = referenceAperture_ / currentAperture; - double yRatio = currentY * (65536 / numBins) / referenceY_; + double yRatio = currentY * (65536 / stats->yHist.bins()) / referenceY_; double estimatedLux = shutterSpeedRatio * gainRatio * apertureRatio * apertureRatio * yRatio * referenceLux_; diff --git a/src/ipa/raspberrypi/raspberrypi.cpp b/src/ipa/raspberrypi/raspberrypi.cpp index 5f7397e2ab21..8c0df0ecd60b 100644 --- a/src/ipa/raspberrypi/raspberrypi.cpp +++ b/src/ipa/raspberrypi/raspberrypi.cpp @@ -54,6 +54,7 @@ #include "metadata.h" #include "sharpen_algorithm.h" #include "sharpen_status.h" +#include "statistics.h" namespace libcamera { @@ -152,6 +153,7 @@ private: void prepareISP(const ISPConfig &data); void reportMetadata(unsigned int ipaContext); void fillDeviceStatus(const ControlList &sensorControls, unsigned int ipaContext); + RPiController::StatisticsPtr fillStatistics(bcm2835_isp_stats *stats) const; void processStats(unsigned int bufferId, unsigned int ipaContext); void applyFrameDurations(Duration minFrameDuration, Duration maxFrameDuration); void applyAGC(const struct AgcStatus *agcStatus, ControlList &ctrls); @@ -1364,6 +1366,46 @@ void IPARPi::fillDeviceStatus(const ControlList &sensorControls, unsigned int ip rpiMetadata_[ipaContext].set("device.status", deviceStatus); } +RPiController::StatisticsPtr IPARPi::fillStatistics(bcm2835_isp_stats *stats) const +{ + using namespace RPiController; + + unsigned int i; + StatisticsPtr statistics = + std::make_unique(Statistics::AgcStatsPos::PreWb, Statistics::ColourStatsPos::PostLsc); + + /* RGB histograms are not used, so do not populate them. */ + statistics->yHist = RPiController::Histogram(stats->hist[0].g_hist, NUM_HISTOGRAM_BINS); + + statistics->awbRegions.init({ DEFAULT_AWB_REGIONS_X, DEFAULT_AWB_REGIONS_Y }); + for (i = 0; i < statistics->awbRegions.numRegions(); i++) + statistics->awbRegions.set(i, { { stats->awb_stats[i].r_sum, + stats->awb_stats[i].g_sum, + stats->awb_stats[i].b_sum }, + stats->awb_stats[i].counted, + stats->awb_stats[i].notcounted }); + + /* + * There are only ever 15 regions computed by the firmware due to zoning, + * but the HW defines AGC_REGIONS == 16! + */ + statistics->agcRegions.init(15); + for (i = 0; i < statistics->agcRegions.numRegions(); i++) + statistics->agcRegions.set(i, { { stats->agc_stats[i].r_sum, + stats->agc_stats[i].g_sum, + stats->agc_stats[i].b_sum }, + stats->agc_stats[i].counted, + stats->awb_stats[i].notcounted }); + + statistics->focusRegions.init({ 4, 3 }); + for (i = 0; i < statistics->focusRegions.numRegions(); i++) + statistics->focusRegions.set(i, { stats->focus_stats[i].contrast_val[1][1] / 1000, + stats->focus_stats[i].contrast_val_num[1][1], + stats->focus_stats[i].contrast_val_num[1][0] }); + + return statistics; +} + void IPARPi::processStats(unsigned int bufferId, unsigned int ipaContext) { RPiController::Metadata &rpiMetadata = rpiMetadata_[ipaContext]; @@ -1376,7 +1418,7 @@ void IPARPi::processStats(unsigned int bufferId, unsigned int ipaContext) Span mem = it->second.planes()[0]; bcm2835_isp_stats *stats = reinterpret_cast(mem.data()); - RPiController::StatisticsPtr statistics = std::make_shared(*stats); + RPiController::StatisticsPtr statistics = fillStatistics(stats); helper_->process(statistics, rpiMetadata); controller_.process(statistics, &rpiMetadata); diff --git a/src/ipa/raspberrypi/statistics.h b/src/ipa/raspberrypi/statistics.h index ad486534b216..015d4efc6454 100644 --- a/src/ipa/raspberrypi/statistics.h +++ b/src/ipa/raspberrypi/statistics.h @@ -73,4 +73,6 @@ struct Statistics { FocusRegions focusRegions; }; +using StatisticsPtr = std::shared_ptr; + } /* namespace RPiController */