From patchwork Wed Sep 2 19:15:51 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Martin Neiva de Carvalho X-Patchwork-Id: 28175 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 72F7DC3348 for ; Wed, 2 Sep 2026 20:04:36 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id CE97A685E6; Wed, 2 Sep 2026 22:04:31 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.b="rov4pPMi"; dkim-atps=neutral Received: from mail-ua1-x92a.google.com (mail-ua1-x92a.google.com [IPv6:2607:f8b0:4864:20::92a]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id C68DA685CA for ; Wed, 2 Sep 2026 21:16:06 +0200 (CEST) Received: by mail-ua1-x92a.google.com with SMTP id a1e0cc1a2514c-96723c7151eso469041241.2 for ; Wed, 02 Sep 2026 12:16:06 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20251104; t=1788376565; x=1788981365; darn=lists.libcamera.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to:content-type; bh=eI6DA0Kc7wKjDdliOszoWG5RqrysvXPEhUI0ccoPGOw=; b=rov4pPMisG1M7VRwPmiZqdcqp/ZRbXLc2J2lchD5BHenL2iufFtmugOADvhJYPXXTl im+t6KND/4K0Wa8Ci2DrhMGVwsdhfNYKO1sfdHptKA4NPAjuA4rxtPS9nVNSKQcITGrY j5CCHF4OkCKxwadEAFxsBQ7D9RypRLqmt4Al8JN53+RNev9+Z8NPAGyNRBXMAft+vu05 pZO/zYKGk8cmL96Z56EoCqcUHhGvJCbVUTplVTscaOVeIF7uh070ux8ZMOfci2u3S4cN 1Rczslw1e8iFBnoEDE2/4z0fsz2UWV4uwUnsVSJ/lvOOk4IFEb4wksMkA1DFk4yR3NMG 1F0A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1788376565; x=1788981365; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to:content-type; bh=eI6DA0Kc7wKjDdliOszoWG5RqrysvXPEhUI0ccoPGOw=; b=hdonlywqnBr0Yzw6tdMsd6wRlSS9Rzxu4uIoUFP4A/gDcfeFK6cEiOIZaeiw+fSPno eNbOgibU6TAobKdCqdAoM2l72dpeZNvuWs4rCiOdmfquugonWtbs4psOvvG5U2BCkiEz 9U48B/QCsGdMJPEKMayZ7iHjuAIf5TMN4Z22yG1VdLAU0NFTTP0LNBcgqnUtrUK/KIGv ycK7YgiTe9gJGiqQwS+4CiguumrgXe8zzMk3hq0s3tL75Yg0KNtKtXAlF1qSdeZFTmSY sQJ1wkWYSUSGa2m9HNHYpU/ys2R5PIqqowcXsFzG3OSGmqAB8l+3i/w1MHbhx7qdKDRw SjLQ== X-Gm-Message-State: AFuF++lVJWjxsfSHqOstPJiojMQ5F8NgI6TYBqGEZ5Py+ExtYO6XwTmu 9QkshJr07hjKO6c0gaXqGNYw+5iU8ZSRIo0aC9J5sKe/lNc+bTCTEvUXxyQPEHv7daE= X-Gm-Gg: AYBFou2cjQFtY/FMRY/Ip+YScfkb3PTgRtFpeIt44zZgjngsn3h/bDTDhMLXgW9fB9D OHv9ClkR1JTi2ljWjjjSfYQR2bnhGK+TdsVikUJ+MMPplEnugiRotBEyRIvJCoV/6c4ujwXgvai WjrDXmo1nOZvKbVj6K66BDsmbhG1i1k4qmoEpSk/G+niZhXYzKlSqssuwWzh45bAP00YgG5Cdnh rTJ4gbsD+JBnumlyuMmdZ+nc46gWTm7HargGza1tysS3EDlzoWeRfLFeRgQZnr/eiKnWKrVk3cf myf9u5LXV2thrr9LZ+Vij9jTFpOnbudMVIbhJqcjdZZhCLM3eGTwzomERduqeKilto6Nc50dbDF FNLDrHpacZQhDWHgs+sUgctPIz9ZqqrRcHlbJe8f+ij3rWHj2h6rcmVdqzE26Hj2ClojrxwaXc8 3vT/uCvrviWlrMKIimgVRMSLLbCPCaL4eOWVx1eD0NfvSneNmty9eKX/2o94M= X-Received: by 2002:a05:6102:fa0:b0:785:d308:cc0a with SMTP id ada2fe7eead31-78a1f3220a3mr2554627137.9.1788376565529; Wed, 02 Sep 2026 12:16:05 -0700 (PDT) Received: from laptop ([45.184.68.165]) by smtp.gmail.com with ESMTPSA id ada2fe7eead31-78a1a1feac0sm3003754137.2.2026.09.02.12.16.04 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 02 Sep 2026 12:16:05 -0700 (PDT) From: Martin Neiva de Carvalho To: libcamera-devel@lists.libcamera.org Cc: Martin Neiva de Carvalho Subject: [PATCH 2/5] libcamera: software_isp: Exclude saturated pixels from colour sums Date: Wed, 2 Sep 2026 16:15:51 -0300 Message-ID: <20260902191554.84922-3-martincarvalho@gmail.com> X-Mailer: git-send-email 2.55.0 In-Reply-To: <20260902191554.84922-1-martincarvalho@gmail.com> References: <20260902191554.84922-1-martincarvalho@gmail.com> MIME-Version: 1.0 X-Mailman-Approved-At: Wed, 02 Sep 2026 22:04:26 +0200 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" The grey world estimate takes the sums of the colour channels over the frame and assumes the scene averages to grey. Saturated pixels break that: whatever colour a pixel was, once it clips it reads the same in every channel, so it carries no information about the illuminant. It is however the brightest kind of pixel there is, and the sums are weighted by brightness, so it carries a large share of the vote. Measured on a scene with a sunlit white wall filling part of the frame, 9% of the photosites were clipped and, being at full scale, accounted for roughly a third of the total sums. The estimate was dragged towards neutral and left a visible cyan cast: surfaces that should have been neutral came out with a linear R/G of 0.855 and B/G of 0.942. Leaving the saturated pixels out brings those to 0.944 and 0.986. Keep a separate count of the pixels that did contribute, because the black level the IPA subtracts from the sums is a per-pixel offset and would otherwise be scaled by the wrong number. Signed-off-by: Martin Neiva de Carvalho --- .../internal/software_isp/swisp_stats.h | 8 ++++ src/ipa/softisp/algorithms/awb.cpp | 15 +++++-- src/libcamera/software_isp/swstats_cpu.cpp | 41 ++++++++++++++----- 3 files changed, 50 insertions(+), 14 deletions(-) diff --git a/include/libcamera/internal/software_isp/swisp_stats.h b/include/libcamera/internal/software_isp/swisp_stats.h index d9d0d9be8..77b70a251 100644 --- a/include/libcamera/internal/software_isp/swisp_stats.h +++ b/include/libcamera/internal/software_isp/swisp_stats.h @@ -31,6 +31,14 @@ struct SwIspStats { * \brief Sums of colour channels of all the sampled pixels */ RGB sum_; + /** + * \brief Number of sampled pixels that contributed to sum_ + * + * Saturated pixels are left out of the sums, so this is smaller than + * the number of pixels sampled. It is needed to subtract the black + * level from the sums, which is a per-pixel offset. + */ + uint64_t sumSamples_; /** * \brief Number of bins in the yHistogram */ diff --git a/src/ipa/softisp/algorithms/awb.cpp b/src/ipa/softisp/algorithms/awb.cpp index 55fd326fc..b48441987 100644 --- a/src/ipa/softisp/algorithms/awb.cpp +++ b/src/ipa/softisp/algorithms/awb.cpp @@ -7,7 +7,6 @@ #include "awb.h" -#include #include #include @@ -118,16 +117,24 @@ SoftIspAwbStats Awb::calculateRgbMeans(IPAContext &context, if (!stats->valid) return {}; - const SwIspStats::Histogram &histogram = stats->yHistogram; const uint8_t blackLevel = context.activeState.blc.level; + /* + * Saturated pixels are not counted in the sums, so the number of + * contributing pixels comes from the statistics rather than from the + * histogram, which counts every pixel sampled. + */ + const uint64_t nPixels = stats->sumSamples_; + + /* Nothing but saturation was sampled, so there is nothing to say. */ + if (!nPixels) + return {}; + /* * Black level must be subtracted to get the correct AWB ratios, they * would be off if they were computed from the whole brightness range * rather than from the sensor range. */ - const uint64_t nPixels = std::accumulate( - histogram.begin(), histogram.end(), uint64_t(0)); const uint64_t offset = blackLevel * nPixels; const uint64_t minValid = 1; diff --git a/src/libcamera/software_isp/swstats_cpu.cpp b/src/libcamera/software_isp/swstats_cpu.cpp index 7fb77ce7d..4349adf15 100644 --- a/src/libcamera/software_isp/swstats_cpu.cpp +++ b/src/libcamera/software_isp/swstats_cpu.cpp @@ -167,6 +167,17 @@ SwStatsCpu::SwStatsCpu(const CameraManager &cm) << "Failed to create shared memory for statistics"; } +/* + * Pixels at or above this fraction of full scale are treated as saturated and + * excluded from the colour sums. A saturated pixel reads the same in every + * channel whatever colour it was, so it says nothing about the illuminant - + * but it is the brightest kind of pixel there is, so it carries a lot of + * weight in a sum. A scene with a blown out window or wall can put a third of + * the total into pixels that carry no information, dragging the grey world + * estimate towards neutral and leaving a visible cast on everything else. + */ +static constexpr unsigned int kSaturationThreshold = 248; + static constexpr unsigned int kRedYMul = 77; /* 0.299 * 256 */ static constexpr unsigned int kGreenYMul = 150; /* 0.587 * 256 */ static constexpr unsigned int kBlueYMul = 29; /* 0.114 * 256 */ @@ -177,22 +188,29 @@ static constexpr unsigned int kBlueYMul = 29; /* 0.114 * 256 */ \ uint64_t sumR = 0; \ uint64_t sumG = 0; \ - uint64_t sumB = 0; + uint64_t sumB = 0; \ + uint64_t nSamples = 0; -#define SWSTATS_ACCUMULATE_LINE_STATS(div) \ - sumR += r; \ - sumG += g; \ - sumB += b; \ - \ - yVal = r * kRedYMul; \ - yVal += g * kGreenYMul; \ - yVal += b * kBlueYMul; \ +#define SWSTATS_ACCUMULATE_LINE_STATS(div) \ + if (r < kSaturationThreshold * (div) && \ + g < kSaturationThreshold * (div) && \ + b < kSaturationThreshold * (div)) { \ + sumR += r; \ + sumG += g; \ + sumB += b; \ + nSamples++; \ + } \ + \ + yVal = r * kRedYMul; \ + yVal += g * kGreenYMul; \ + yVal += b * kBlueYMul; \ stats.yHistogram[yVal * SwIspStats::kYHistogramSize / (256 * 256 * (div))]++; #define SWSTATS_FINISH_LINE_STATS() \ stats.sum_.r() += sumR; \ stats.sum_.g() += sumG; \ - stats.sum_.b() += sumB; + stats.sum_.b() += sumB; \ + stats.sumSamples_ += nSamples; void SwStatsCpu::statsBGGR8Line0(const uint8_t *src[], SwIspStats &stats) { @@ -391,6 +409,7 @@ void SwStatsCpu::startFrame(uint32_t frame) for (auto &s : stats_) { s.sum_ = RGB({ 0, 0, 0 }); + s.sumSamples_ = 0; s.yHistogram.fill(0); } } @@ -408,9 +427,11 @@ void SwStatsCpu::finishFrame(uint32_t frame, uint32_t bufferId) if (valid) { sharedStats_->sum_ = RGB({ 0, 0, 0 }); + sharedStats_->sumSamples_ = 0; sharedStats_->yHistogram.fill(0); for (const auto &s : stats_) { sharedStats_->sum_ += s.sum_; + sharedStats_->sumSamples_ += s.sumSamples_; for (unsigned int j = 0; j < SwIspStats::kYHistogramSize; j++) sharedStats_->yHistogram[j] += s.yHistogram[j]; }