From patchwork Wed Sep 2 19:15:50 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Martin Neiva de Carvalho X-Patchwork-Id: 28174 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id F1C99C3344 for ; Wed, 2 Sep 2026 20:04:34 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 6BA9C685E3; Wed, 2 Sep 2026 22:04:30 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.b="TEkV+2Bk"; dkim-atps=neutral Received: from mail-ua1-x930.google.com (mail-ua1-x930.google.com [IPv6:2607:f8b0:4864:20::930]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 0D817685BA for ; Wed, 2 Sep 2026 21:16:05 +0200 (CEST) Received: by mail-ua1-x930.google.com with SMTP id a1e0cc1a2514c-97ca74bd6e0so420341241.1 for ; Wed, 02 Sep 2026 12:16:05 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20251104; t=1788376564; x=1788981364; darn=lists.libcamera.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to:content-type; bh=BzaQLorpqZ6Wd+W6fzUyGU4VxdGo/QG+xwxii7NCHt4=; b=TEkV+2BkfEtgSr7lehdwL5KgISUPKMbIDS60kGu67Utb3YrF0RaTJWdsC7ui5Y51ax RBRm8XfL7+3ULf/T72etI4A8fjHitbXwXGTR8lppaVTnPuj/vhDMrP+VWD5W3x6/oTvI A/REc4qwSpH6nrmrqpK6lE0De89ZdBWuV6PBrvZmf72VIzbSu7KsOix6VzMkLqR64/eM DnmuHd1fcKNl/xpUxzTHJ/rkNX9EFcEhoy3JheUU4Dbfe5enDToi7TDNV3NxVn5TPZeD SqOR8QVg8HaiDdWkgEryA11C3kygbxGEJoqt9SBaZ/WYI/uBmxtww+JxJPBpA2ubJXTl wP4g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1788376564; x=1788981364; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to:content-type; bh=BzaQLorpqZ6Wd+W6fzUyGU4VxdGo/QG+xwxii7NCHt4=; b=jbkTLyHk4yMxBKPC+jftJX92oH0/sWhBD5hJPba8c9mSk5Pvnvpycy3xvuLufeH9F8 pIlNe007qlD4dJSjoP7z5xHM9C9Vojugm/45rKe1Qvnn58Lmyxxlxg4m90AeN8nDxx5c Z54gTbyAQMKoFib8QEOZUo11awK75hGO69+QYCb4X/F+qimSvjAdVLEtQMhxlEvCBxhV CZRM/dwCgb4nn3u+HiJ5VsdyyJqzCjChV5VdiE8GaEaLvN12Lnz47FomapoU20yJR8RX velegV9SJpToGiMQMXR5/zBDgc3yuoa0Iw7DjZU3si7KV7s6UpvBj9n8wPOwDO5VKxLb FaSg== X-Gm-Message-State: AFuF++lS2VtpeXTSMX7mYi9a8Hz8yADzqhOLvH8W5kJZcthnv5gmuFRo DnRdUewP8aiCMeo0mBSdwsvwK6TC94EF6w6Zkg1wlk26BHlqGC+FfzvzKxvcoI/NYs4= X-Gm-Gg: AYBFou1AhXukaxXPymIS6dKCe7Km1y24+VvCEWPXT6aBbpy+UrrYQJk5wKmokTGlzLh QRMO8aB6Xpqy1P8jlr7/LMkfByezSDPS1xPD/m1E9wfyHNIjIizpP+nuOSipYpOx1efkIBXwx0Q AJTOXmPBwuldvFrO9GvoCd1WkMPeLCgN88i+rRvcQrnnBBC1m7n86ryLNyhz1b8qWdt0C/YKuXD 9u/1vCtSSVzQSI0D23K1Xg7sGvWTFj/OqQneev/D3J36t4zRbx3jnJJKqp5Sv5ccz3jQpBaH8Dw 4dkZOud+OV5/5JukmJDjx2ZySOQV9/edNjnPVtDS4CgPNzmmyX++8Xvav6TEf5YPO/MvLDU4p7v MyBzVoeASb7Td7HHZn2OxuSH/Yj3RTxvypiU3FvUFN6/nqovN+SEypBu1CU7y++/nz4K5IKH0cI Rubawzxxqw++u1HBXHkp2Uidtfm/lZVm9D0dcijUPUK5TBAPxBvOIrgGrGCoc= X-Received: by 2002:a05:6102:6d2:b0:77b:2881:580b with SMTP id ada2fe7eead31-78a1f1aff84mr2288362137.4.1788376563702; Wed, 02 Sep 2026 12:16:03 -0700 (PDT) Received: from laptop ([45.184.68.165]) by smtp.gmail.com with ESMTPSA id ada2fe7eead31-78a1a1feac0sm3003754137.2.2026.09.02.12.16.02 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 02 Sep 2026 12:16:03 -0700 (PDT) From: Martin Neiva de Carvalho To: libcamera-devel@lists.libcamera.org Cc: Martin Neiva de Carvalho Subject: [PATCH 1/5] ipa: libipa: camera_sensor_helper: Add OV02C10 Date: Wed, 2 Sep 2026 16:15:50 -0300 Message-ID: <20260902191554.84922-2-martincarvalho@gmail.com> X-Mailer: git-send-email 2.55.0 In-Reply-To: <20260902191554.84922-1-martincarvalho@gmail.com> References: <20260902191554.84922-1-martincarvalho@gmail.com> MIME-Version: 1.0 X-Mailman-Approved-At: Wed, 02 Sep 2026 22:04:26 +0200 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" The OmniVision OV02C10 is used as the front sensor on several Intel IPU6 laptops, where it is driven by the software ISP and so depends on this helper to make sense of its gain control. V4L2_CID_ANALOGUE_GAIN on this sensor has a minimum and a default of 0x10, which is unity gain, so the code is expressed in 1/16th steps. Without a helper the default one-to-one mapping is used, and the AGC reads the smallest gain the sensor will accept as 16x. It then believes the sensor is already amplifying heavily when it is not, and never raises the gain, leaving the image dark in anything but bright light. Verified by measurement on a Samsung Galaxy Book (NP960XGL): with the helper in place the AGC drives the control across its whole range and the resulting image level tracks the requested gain. The black level matches the 0x40 at 10 bits that the kernel driver programs as its BLC target. Signed-off-by: Martin Neiva de Carvalho --- src/ipa/libipa/camera_sensor_helper.cpp | 12 ++++++++++++ 1 file changed, 12 insertions(+) diff --git a/src/ipa/libipa/camera_sensor_helper.cpp b/src/ipa/libipa/camera_sensor_helper.cpp index 82bf255d2..d382d3e4f 100644 --- a/src/ipa/libipa/camera_sensor_helper.cpp +++ b/src/ipa/libipa/camera_sensor_helper.cpp @@ -714,6 +714,18 @@ public: }; REGISTER_CAMERA_SENSOR_HELPER("ov01a10", CameraSensorHelperOv01a10) +class CameraSensorHelperOv02c10 : public CameraSensorHelper +{ +public: + CameraSensorHelperOv02c10() + { + /* From Linux kernel driver: 0x40 at 10bits. */ + blackLevel_ = 4096; + gain_ = AnalogueGainLinear{ 1, 0, 0, 16 }; + } +}; +REGISTER_CAMERA_SENSOR_HELPER("ov02c10", CameraSensorHelperOv02c10) + class CameraSensorHelperOv08d10 : public CameraSensorHelper { public: From patchwork Wed Sep 2 19:15:51 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Martin Neiva de Carvalho X-Patchwork-Id: 28175 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 72F7DC3348 for ; Wed, 2 Sep 2026 20:04:36 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id CE97A685E6; Wed, 2 Sep 2026 22:04:31 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.b="rov4pPMi"; dkim-atps=neutral Received: from mail-ua1-x92a.google.com (mail-ua1-x92a.google.com [IPv6:2607:f8b0:4864:20::92a]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id C68DA685CA for ; Wed, 2 Sep 2026 21:16:06 +0200 (CEST) Received: by mail-ua1-x92a.google.com with SMTP id a1e0cc1a2514c-96723c7151eso469041241.2 for ; Wed, 02 Sep 2026 12:16:06 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20251104; t=1788376565; x=1788981365; darn=lists.libcamera.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to:content-type; bh=eI6DA0Kc7wKjDdliOszoWG5RqrysvXPEhUI0ccoPGOw=; b=rov4pPMisG1M7VRwPmiZqdcqp/ZRbXLc2J2lchD5BHenL2iufFtmugOADvhJYPXXTl im+t6KND/4K0Wa8Ci2DrhMGVwsdhfNYKO1sfdHptKA4NPAjuA4rxtPS9nVNSKQcITGrY j5CCHF4OkCKxwadEAFxsBQ7D9RypRLqmt4Al8JN53+RNev9+Z8NPAGyNRBXMAft+vu05 pZO/zYKGk8cmL96Z56EoCqcUHhGvJCbVUTplVTscaOVeIF7uh070ux8ZMOfci2u3S4cN 1Rczslw1e8iFBnoEDE2/4z0fsz2UWV4uwUnsVSJ/lvOOk4IFEb4wksMkA1DFk4yR3NMG 1F0A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1788376565; x=1788981365; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to:content-type; bh=eI6DA0Kc7wKjDdliOszoWG5RqrysvXPEhUI0ccoPGOw=; b=hdonlywqnBr0Yzw6tdMsd6wRlSS9Rzxu4uIoUFP4A/gDcfeFK6cEiOIZaeiw+fSPno eNbOgibU6TAobKdCqdAoM2l72dpeZNvuWs4rCiOdmfquugonWtbs4psOvvG5U2BCkiEz 9U48B/QCsGdMJPEKMayZ7iHjuAIf5TMN4Z22yG1VdLAU0NFTTP0LNBcgqnUtrUK/KIGv ycK7YgiTe9gJGiqQwS+4CiguumrgXe8zzMk3hq0s3tL75Yg0KNtKtXAlF1qSdeZFTmSY sQJ1wkWYSUSGa2m9HNHYpU/ys2R5PIqqowcXsFzG3OSGmqAB8l+3i/w1MHbhx7qdKDRw SjLQ== X-Gm-Message-State: AFuF++lVJWjxsfSHqOstPJiojMQ5F8NgI6TYBqGEZ5Py+ExtYO6XwTmu 9QkshJr07hjKO6c0gaXqGNYw+5iU8ZSRIo0aC9J5sKe/lNc+bTCTEvUXxyQPEHv7daE= X-Gm-Gg: AYBFou2cjQFtY/FMRY/Ip+YScfkb3PTgRtFpeIt44zZgjngsn3h/bDTDhMLXgW9fB9D OHv9ClkR1JTi2ljWjjjSfYQR2bnhGK+TdsVikUJ+MMPplEnugiRotBEyRIvJCoV/6c4ujwXgvai WjrDXmo1nOZvKbVj6K66BDsmbhG1i1k4qmoEpSk/G+niZhXYzKlSqssuwWzh45bAP00YgG5Cdnh rTJ4gbsD+JBnumlyuMmdZ+nc46gWTm7HargGza1tysS3EDlzoWeRfLFeRgQZnr/eiKnWKrVk3cf myf9u5LXV2thrr9LZ+Vij9jTFpOnbudMVIbhJqcjdZZhCLM3eGTwzomERduqeKilto6Nc50dbDF FNLDrHpacZQhDWHgs+sUgctPIz9ZqqrRcHlbJe8f+ij3rWHj2h6rcmVdqzE26Hj2ClojrxwaXc8 3vT/uCvrviWlrMKIimgVRMSLLbCPCaL4eOWVx1eD0NfvSneNmty9eKX/2o94M= X-Received: by 2002:a05:6102:fa0:b0:785:d308:cc0a with SMTP id ada2fe7eead31-78a1f3220a3mr2554627137.9.1788376565529; Wed, 02 Sep 2026 12:16:05 -0700 (PDT) Received: from laptop ([45.184.68.165]) by smtp.gmail.com with ESMTPSA id ada2fe7eead31-78a1a1feac0sm3003754137.2.2026.09.02.12.16.04 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 02 Sep 2026 12:16:05 -0700 (PDT) From: Martin Neiva de Carvalho To: libcamera-devel@lists.libcamera.org Cc: Martin Neiva de Carvalho Subject: [PATCH 2/5] libcamera: software_isp: Exclude saturated pixels from colour sums Date: Wed, 2 Sep 2026 16:15:51 -0300 Message-ID: <20260902191554.84922-3-martincarvalho@gmail.com> X-Mailer: git-send-email 2.55.0 In-Reply-To: <20260902191554.84922-1-martincarvalho@gmail.com> References: <20260902191554.84922-1-martincarvalho@gmail.com> MIME-Version: 1.0 X-Mailman-Approved-At: Wed, 02 Sep 2026 22:04:26 +0200 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" The grey world estimate takes the sums of the colour channels over the frame and assumes the scene averages to grey. Saturated pixels break that: whatever colour a pixel was, once it clips it reads the same in every channel, so it carries no information about the illuminant. It is however the brightest kind of pixel there is, and the sums are weighted by brightness, so it carries a large share of the vote. Measured on a scene with a sunlit white wall filling part of the frame, 9% of the photosites were clipped and, being at full scale, accounted for roughly a third of the total sums. The estimate was dragged towards neutral and left a visible cyan cast: surfaces that should have been neutral came out with a linear R/G of 0.855 and B/G of 0.942. Leaving the saturated pixels out brings those to 0.944 and 0.986. Keep a separate count of the pixels that did contribute, because the black level the IPA subtracts from the sums is a per-pixel offset and would otherwise be scaled by the wrong number. Signed-off-by: Martin Neiva de Carvalho --- .../internal/software_isp/swisp_stats.h | 8 ++++ src/ipa/softisp/algorithms/awb.cpp | 15 +++++-- src/libcamera/software_isp/swstats_cpu.cpp | 41 ++++++++++++++----- 3 files changed, 50 insertions(+), 14 deletions(-) diff --git a/include/libcamera/internal/software_isp/swisp_stats.h b/include/libcamera/internal/software_isp/swisp_stats.h index d9d0d9be8..77b70a251 100644 --- a/include/libcamera/internal/software_isp/swisp_stats.h +++ b/include/libcamera/internal/software_isp/swisp_stats.h @@ -31,6 +31,14 @@ struct SwIspStats { * \brief Sums of colour channels of all the sampled pixels */ RGB sum_; + /** + * \brief Number of sampled pixels that contributed to sum_ + * + * Saturated pixels are left out of the sums, so this is smaller than + * the number of pixels sampled. It is needed to subtract the black + * level from the sums, which is a per-pixel offset. + */ + uint64_t sumSamples_; /** * \brief Number of bins in the yHistogram */ diff --git a/src/ipa/softisp/algorithms/awb.cpp b/src/ipa/softisp/algorithms/awb.cpp index 55fd326fc..b48441987 100644 --- a/src/ipa/softisp/algorithms/awb.cpp +++ b/src/ipa/softisp/algorithms/awb.cpp @@ -7,7 +7,6 @@ #include "awb.h" -#include #include #include @@ -118,16 +117,24 @@ SoftIspAwbStats Awb::calculateRgbMeans(IPAContext &context, if (!stats->valid) return {}; - const SwIspStats::Histogram &histogram = stats->yHistogram; const uint8_t blackLevel = context.activeState.blc.level; + /* + * Saturated pixels are not counted in the sums, so the number of + * contributing pixels comes from the statistics rather than from the + * histogram, which counts every pixel sampled. + */ + const uint64_t nPixels = stats->sumSamples_; + + /* Nothing but saturation was sampled, so there is nothing to say. */ + if (!nPixels) + return {}; + /* * Black level must be subtracted to get the correct AWB ratios, they * would be off if they were computed from the whole brightness range * rather than from the sensor range. */ - const uint64_t nPixels = std::accumulate( - histogram.begin(), histogram.end(), uint64_t(0)); const uint64_t offset = blackLevel * nPixels; const uint64_t minValid = 1; diff --git a/src/libcamera/software_isp/swstats_cpu.cpp b/src/libcamera/software_isp/swstats_cpu.cpp index 7fb77ce7d..4349adf15 100644 --- a/src/libcamera/software_isp/swstats_cpu.cpp +++ b/src/libcamera/software_isp/swstats_cpu.cpp @@ -167,6 +167,17 @@ SwStatsCpu::SwStatsCpu(const CameraManager &cm) << "Failed to create shared memory for statistics"; } +/* + * Pixels at or above this fraction of full scale are treated as saturated and + * excluded from the colour sums. A saturated pixel reads the same in every + * channel whatever colour it was, so it says nothing about the illuminant - + * but it is the brightest kind of pixel there is, so it carries a lot of + * weight in a sum. A scene with a blown out window or wall can put a third of + * the total into pixels that carry no information, dragging the grey world + * estimate towards neutral and leaving a visible cast on everything else. + */ +static constexpr unsigned int kSaturationThreshold = 248; + static constexpr unsigned int kRedYMul = 77; /* 0.299 * 256 */ static constexpr unsigned int kGreenYMul = 150; /* 0.587 * 256 */ static constexpr unsigned int kBlueYMul = 29; /* 0.114 * 256 */ @@ -177,22 +188,29 @@ static constexpr unsigned int kBlueYMul = 29; /* 0.114 * 256 */ \ uint64_t sumR = 0; \ uint64_t sumG = 0; \ - uint64_t sumB = 0; + uint64_t sumB = 0; \ + uint64_t nSamples = 0; -#define SWSTATS_ACCUMULATE_LINE_STATS(div) \ - sumR += r; \ - sumG += g; \ - sumB += b; \ - \ - yVal = r * kRedYMul; \ - yVal += g * kGreenYMul; \ - yVal += b * kBlueYMul; \ +#define SWSTATS_ACCUMULATE_LINE_STATS(div) \ + if (r < kSaturationThreshold * (div) && \ + g < kSaturationThreshold * (div) && \ + b < kSaturationThreshold * (div)) { \ + sumR += r; \ + sumG += g; \ + sumB += b; \ + nSamples++; \ + } \ + \ + yVal = r * kRedYMul; \ + yVal += g * kGreenYMul; \ + yVal += b * kBlueYMul; \ stats.yHistogram[yVal * SwIspStats::kYHistogramSize / (256 * 256 * (div))]++; #define SWSTATS_FINISH_LINE_STATS() \ stats.sum_.r() += sumR; \ stats.sum_.g() += sumG; \ - stats.sum_.b() += sumB; + stats.sum_.b() += sumB; \ + stats.sumSamples_ += nSamples; void SwStatsCpu::statsBGGR8Line0(const uint8_t *src[], SwIspStats &stats) { @@ -391,6 +409,7 @@ void SwStatsCpu::startFrame(uint32_t frame) for (auto &s : stats_) { s.sum_ = RGB({ 0, 0, 0 }); + s.sumSamples_ = 0; s.yHistogram.fill(0); } } @@ -408,9 +427,11 @@ void SwStatsCpu::finishFrame(uint32_t frame, uint32_t bufferId) if (valid) { sharedStats_->sum_ = RGB({ 0, 0, 0 }); + sharedStats_->sumSamples_ = 0; sharedStats_->yHistogram.fill(0); for (const auto &s : stats_) { sharedStats_->sum_ += s.sum_; + sharedStats_->sumSamples_ += s.sumSamples_; for (unsigned int j = 0; j < SwIspStats::kYHistogramSize; j++) sharedStats_->yHistogram[j] += s.yHistogram[j]; } From patchwork Wed Sep 2 19:15:52 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Martin Neiva de Carvalho X-Patchwork-Id: 28176 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 4A597C3344 for ; Wed, 2 Sep 2026 20:04:37 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id DA524685DF; Wed, 2 Sep 2026 22:04:32 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.b="f0ooWWd/"; dkim-atps=neutral Received: from mail-ua1-x936.google.com (mail-ua1-x936.google.com [IPv6:2607:f8b0:4864:20::936]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 27601685CA for ; Wed, 2 Sep 2026 21:16:09 +0200 (CEST) Received: by mail-ua1-x936.google.com with SMTP id a1e0cc1a2514c-97e76cabd96so429548241.3 for ; Wed, 02 Sep 2026 12:16:09 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20251104; t=1788376568; x=1788981368; darn=lists.libcamera.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to:content-type; bh=7Q0srxNAvkHO6qCwRKFV+Fnzoa0y0n18TSwlPbZ2snE=; b=f0ooWWd/vJO2jbAdlrLO1i4NCOLT5B6JoEAYeYdfhuIM/3Uyfeb7y5medQBrKrtyLs 3SRUFNfUCUg8hFJ9iKdZzj8pqAmqAj/Mg2AVTURJ60W3YPXpdMYBcYPgcJI8zdFBg2KN ULXHkbwpo8AeJtyn8v7rzl8Q7DCSsI3L5AQPwpMeUWyjuOY++KRv8BL+nJtfWXd2Y1+B IOJtrzbYeAot7NL7qK81mDv6IwRP4kuqd2kPVBYVE9fUsxxtNsAWzDp+Pat/imo9fm/T MZfmeRr6Px7DyAIjeZOQXOxGpu3ALuaKumjRBDWqiF4twmTlo/mbd2RpnhchfOg2loy6 X1qA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1788376568; x=1788981368; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to:content-type; bh=7Q0srxNAvkHO6qCwRKFV+Fnzoa0y0n18TSwlPbZ2snE=; b=VxrZ/jAL1IAFi6HNtke/5Fv3QgzFTyE15dvbYw/Q+HTJ+RcvE+k3Y+7fdCAksSPfy2 eoHgsSD4DEzqjgXmF5nffM7QoZM9kq7x/DpElWiIjPcUZtkkdouSSpSBsQwt1fFAga4M MZRX1WiCJAI0UrBcrVe4lgHvlAInBeCk3m1PajoCR0WCEEV1RdCA1gBSL56Wcni5/8S2 SUoVAbEx7lmoMFXNTSVwfSoj/92amjZVsu4VJ1cxRVeJFkIIICq4uL3kHMGL2IFlxaB9 OKH4WUWjqKFxtaTdhX+mQncwBe8MmoICHRzMT+Ol2hPfkn9jmBE87fdKWTdIcywuTO1C KcMw== X-Gm-Message-State: AFuF++maRkXMR223i2jIQlO/g4i9UYju5T1Mw13etxc2/oNqBDDSfkEQ 5JqIqkkU+rkhQiTMStdbpV7YA8Jli6HU5eu5BF1/4YpqrLJfbigwaHUtMqBd+mI++XA= X-Gm-Gg: AYBFou3Adzf107QCtaj9jhcUk+3k//sBmXob99w2cla1HcKKuNe9jZzTrnLRwYKYJJS JU3gAyibbLC6gS+rKB0Y2uWTRV+ikE220aPkQk9Jcgxsg0GaTWS6pUfcarhQqAS7j5W93gwLWmI mIRaaVGoP0qap7zIpNxizvamA20SR15TGyFNAOn1WA0CtxO3nPxizaAO4fAma8kOOIpJ//IA8g3 QoVwP8wGfCfMb2UeTWaf+nk3KDdpb+cMvc1VhmLDnLdRe5jPLrGkRriToinsw3lLvYF+2FytCGD gEdI5XPaaQhv+Jn0acrWN14SuJZjC3vzi6xf3aF7KAlO2Iltq5xhc4g49iJ4aFijWd+5gqQRQ5H Y/yNI9IdbAkhRtrvFwRWbrIa/+2L/z6Q1FYkGSpz03BTvOhhEoK1xunbwZnTxTdzL8jr6jdbTnJ YQxlPJeWcIuSa1jpXeHT7FobY3xnCH4Mk6Lypu9cizS2x5H7FOYmNOWDkDFwE= X-Received: by 2002:a05:6102:4194:b0:779:e75e:63c6 with SMTP id ada2fe7eead31-78a1f39554cmr2160142137.12.1788376567756; Wed, 02 Sep 2026 12:16:07 -0700 (PDT) Received: from laptop ([45.184.68.165]) by smtp.gmail.com with ESMTPSA id ada2fe7eead31-78a1a1feac0sm3003754137.2.2026.09.02.12.16.05 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 02 Sep 2026 12:16:06 -0700 (PDT) From: Martin Neiva de Carvalho To: libcamera-devel@lists.libcamera.org Cc: Martin Neiva de Carvalho Subject: [PATCH 3/5] libcamera: software_isp: Estimate white balance from neutral pixels Date: Wed, 2 Sep 2026 16:15:52 -0300 Message-ID: <20260902191554.84922-4-martincarvalho@gmail.com> X-Mailer: git-send-email 2.55.0 In-Reply-To: <20260902191554.84922-1-martincarvalho@gmail.com> References: <20260902191554.84922-1-martincarvalho@gmail.com> MIME-Version: 1.0 X-Mailman-Approved-At: Wed, 02 Sep 2026 22:04:26 +0200 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Grey world takes the whole frame as evidence of the illuminant. That holds for a scene whose colours average out, and fails for one with a large coloured object in it: the object's colour enters the estimate as though it were the light, and the correction that follows tints everything else the other way. Restrict the estimate to the pixels that the gains already in use bring close to neutral. Those are the ones the grey world assumption actually applies to, and iterating pulls the gains towards the illuminant rather than towards the average scene colour. Measured on a room with a wooden door filling about a fifth of the frame, against three white-painted surfaces as a reference: their combined deviation from neutral went from 0.059 to 0.026. The walls went from a red-to-green ratio of 0.950 to 0.986, with blue-to-green similarly improved. The selection has to happen while the statistics are gathered rather than in the IPA, because by the time the IPA sees them they are already summed, and a sum cannot be un-mixed. The gains therefore lag by one frame, which costs nothing here: the estimator is iterative anyway, and the iteration simply runs across frames instead of within one. Fall back to the whole frame when less than a twentieth of it looks neutral. That covers a genuinely monochrome scene, and it covers the first frames, where the gains have not converged, nothing would qualify, and the estimate could otherwise never get started. Signed-off-by: Martin Neiva de Carvalho --- .../internal/software_isp/swisp_stats.h | 13 +++ .../internal/software_isp/swstats_cpu.h | 5 ++ src/ipa/softisp/algorithms/awb.cpp | 20 ++++- src/libcamera/software_isp/debayer_cpu.cpp | 7 ++ src/libcamera/software_isp/debayer_egl.cpp | 7 ++ src/libcamera/software_isp/swstats_cpu.cpp | 85 +++++++++++++++++-- 6 files changed, 129 insertions(+), 8 deletions(-) diff --git a/include/libcamera/internal/software_isp/swisp_stats.h b/include/libcamera/internal/software_isp/swisp_stats.h index 77b70a251..577825c5e 100644 --- a/include/libcamera/internal/software_isp/swisp_stats.h +++ b/include/libcamera/internal/software_isp/swisp_stats.h @@ -39,6 +39,19 @@ struct SwIspStats { * level from the sums, which is a per-pixel offset. */ uint64_t sumSamples_; + /** + * \brief Sums over only those sampled pixels that already look neutral + * + * Grey world assumes the scene averages to grey. One large coloured + * object breaks that assumption and takes the estimate with it. These + * sums cover only the pixels the current gains already bring close to + * neutral, which is the same assumption applied where it holds. + */ + RGB sumNeutral_; + /** + * \brief Number of sampled pixels that contributed to sumNeutral_ + */ + uint64_t sumNeutralSamples_; /** * \brief Number of bins in the yHistogram */ diff --git a/include/libcamera/internal/software_isp/swstats_cpu.h b/include/libcamera/internal/software_isp/swstats_cpu.h index 551870921..4453a1d4f 100644 --- a/include/libcamera/internal/software_isp/swstats_cpu.h +++ b/include/libcamera/internal/software_isp/swstats_cpu.h @@ -54,6 +54,7 @@ public: int configure(const StreamConfiguration &inputCfg, unsigned int statsBufferCount = 1); void setWindow(const Rectangle &window); + void setNeutralReference(const RGB &gains, const RGB &blackLevel); void startFrame(uint32_t frame); void finishFrame(uint32_t frame, uint32_t bufferId); void processFrame(uint32_t frame, uint32_t bufferId, MappedFrameBuffer &input); @@ -106,6 +107,10 @@ private: processFrameFn processFrame_; + /* Reference the neutral-pixel test is taken against, from the IPA. */ + float neutralGains_[3] = { 1.0f, 1.0f, 1.0f }; + float neutralBlack_ = 0.0f; + /* Variables set by configure(), used every line */ statsProcessFn stats0_; statsProcessFn stats2_; diff --git a/src/ipa/softisp/algorithms/awb.cpp b/src/ipa/softisp/algorithms/awb.cpp index b48441987..65457e2dc 100644 --- a/src/ipa/softisp/algorithms/awb.cpp +++ b/src/ipa/softisp/algorithms/awb.cpp @@ -123,8 +123,24 @@ SoftIspAwbStats Awb::calculateRgbMeans(IPAContext &context, * Saturated pixels are not counted in the sums, so the number of * contributing pixels comes from the statistics rather than from the * histogram, which counts every pixel sampled. + * + * Prefer the pixels that already look neutral under the current gains. + * Grey world takes the whole scene as evidence of the illuminant, so a + * single large coloured object is enough to pull the estimate off it. + * Fall back to the whole frame when too little of it qualifies - a + * genuinely monochrome scene, or the first frames before the gains + * have converged, where nothing would qualify and the estimate could + * never get started. */ - const uint64_t nPixels = stats->sumSamples_; + constexpr double kMinNeutralFraction = 0.05; + + uint64_t nPixels = stats->sumNeutralSamples_; + RGB rawSum = stats->sumNeutral_; + + if (nPixels < stats->sumSamples_ * kMinNeutralFraction) { + nPixels = stats->sumSamples_; + rawSum = stats->sum_; + } /* Nothing but saturation was sampled, so there is nothing to say. */ if (!nPixels) @@ -142,7 +158,7 @@ SoftIspAwbStats Awb::calculateRgbMeans(IPAContext &context, * Make sure the sums are at least minValid, while preventing unsigned * integer underflow. */ - const RGB sum = stats->sum_.max(offset + minValid) - offset; + const RGB sum = rawSum.max(offset + minValid) - offset; RGB rgbMeans = { { static_cast(sum.r()) / nPixels, static_cast(sum.g()) / nPixels, diff --git a/src/libcamera/software_isp/debayer_cpu.cpp b/src/libcamera/software_isp/debayer_cpu.cpp index c6d5d1e18..9949a77ca 100644 --- a/src/libcamera/software_isp/debayer_cpu.cpp +++ b/src/libcamera/software_isp/debayer_cpu.cpp @@ -1052,6 +1052,13 @@ void DebayerCpu::process(uint32_t frame, FrameBuffer *input, FrameBuffer *output updateLookupTables(params); + /* + * The statistics need this frame's gains to tell which pixels already + * look neutral. They cannot come from the IPA afterwards: by then the + * sums are taken, and a sum cannot be un-mixed. + */ + stats_->setNeutralReference(params.gains, params.blackLevel); + /* Copy metadata from the input buffer */ FrameMetadata &metadata = output->_d()->metadata(); metadata.status = input->metadata().status; diff --git a/src/libcamera/software_isp/debayer_egl.cpp b/src/libcamera/software_isp/debayer_egl.cpp index 97aa03793..c042d3864 100644 --- a/src/libcamera/software_isp/debayer_egl.cpp +++ b/src/libcamera/software_isp/debayer_egl.cpp @@ -608,6 +608,13 @@ void DebayerEGL::process(uint32_t frame, FrameBuffer *input, FrameBuffer *output egl_.assertThread(); bench_.startFrame(); + /* + * The statistics need this frame's gains to tell which pixels already + * look neutral. They cannot come from the IPA afterwards: by then the + * sums are taken, and a sum cannot be un-mixed. + */ + stats_->setNeutralReference(params.gains, params.blackLevel); + /* Copy metadata from the input buffer */ FrameMetadata &metadata = output->_d()->metadata(); metadata.status = input->metadata().status; diff --git a/src/libcamera/software_isp/swstats_cpu.cpp b/src/libcamera/software_isp/swstats_cpu.cpp index 4349adf15..c14f1dc6c 100644 --- a/src/libcamera/software_isp/swstats_cpu.cpp +++ b/src/libcamera/software_isp/swstats_cpu.cpp @@ -11,6 +11,8 @@ #include "libcamera/internal/software_isp/swstats_cpu.h" +#include + #include #include @@ -178,6 +180,17 @@ SwStatsCpu::SwStatsCpu(const CameraManager &cm) */ static constexpr unsigned int kSaturationThreshold = 248; +/* + * How far a pixel may sit from neutral, as a fraction of its own level, and + * still count as evidence of what neutral looks like. Loose enough that a + * typical scene keeps most of its pixels, because an estimator that discards + * almost everything is one awkward frame away from having nothing left. + */ +static constexpr float kNeutralTolerance = 0.20f; + +/* Below this level above black the channel ratios are mostly noise. */ +static constexpr float kNeutralMinLevel = 8.0f; + static constexpr unsigned int kRedYMul = 77; /* 0.299 * 256 */ static constexpr unsigned int kGreenYMul = 150; /* 0.587 * 256 */ static constexpr unsigned int kBlueYMul = 29; /* 0.114 * 256 */ @@ -189,7 +202,11 @@ static constexpr unsigned int kBlueYMul = 29; /* 0.114 * 256 */ uint64_t sumR = 0; \ uint64_t sumG = 0; \ uint64_t sumB = 0; \ - uint64_t nSamples = 0; + uint64_t nSamples = 0; \ + uint64_t sumNR = 0; \ + uint64_t sumNG = 0; \ + uint64_t sumNB = 0; \ + uint64_t nNeutral = 0; #define SWSTATS_ACCUMULATE_LINE_STATS(div) \ if (r < kSaturationThreshold * (div) && \ @@ -199,6 +216,7 @@ static constexpr unsigned int kBlueYMul = 29; /* 0.114 * 256 */ sumG += g; \ sumB += b; \ nSamples++; \ + SWSTATS_ACCUMULATE_NEUTRAL(div) \ } \ \ yVal = r * kRedYMul; \ @@ -206,11 +224,42 @@ static constexpr unsigned int kBlueYMul = 29; /* 0.114 * 256 */ yVal += b * kBlueYMul; \ stats.yHistogram[yVal * SwIspStats::kYHistogramSize / (256 * 256 * (div))]++; -#define SWSTATS_FINISH_LINE_STATS() \ - stats.sum_.r() += sumR; \ - stats.sum_.g() += sumG; \ - stats.sum_.b() += sumB; \ - stats.sumSamples_ += nSamples; +/* + * Apply the gains the IPA settled on last frame and ask whether what comes out + * is neutral. It has to happen here rather than in the IPA: by the time the + * IPA sees the statistics they are already summed, and a sum cannot be + * un-mixed. One frame of lag on the gains costs nothing, the estimator being + * iterative by nature - the iteration simply runs across frames. + */ +#define SWSTATS_ACCUMULATE_NEUTRAL(div) \ + { \ + float black = neutralBlack_ * (div); \ + float nr = (r - black) * neutralGains_[0]; \ + float ng = (g - black) * neutralGains_[1]; \ + float nb = (b - black) * neutralGains_[2]; \ + float mean = (nr + ng + nb) / 3.0f; \ + float tol = kNeutralTolerance * mean; \ + \ + if (mean > kNeutralMinLevel * (div) && \ + std::abs(nr - mean) < tol && \ + std::abs(ng - mean) < tol && \ + std::abs(nb - mean) < tol) { \ + sumNR += r; \ + sumNG += g; \ + sumNB += b; \ + nNeutral++; \ + } \ + } + +#define SWSTATS_FINISH_LINE_STATS() \ + stats.sum_.r() += sumR; \ + stats.sum_.g() += sumG; \ + stats.sum_.b() += sumB; \ + stats.sumSamples_ += nSamples; \ + stats.sumNeutral_.r() += sumNR; \ + stats.sumNeutral_.g() += sumNG; \ + stats.sumNeutral_.b() += sumNB; \ + stats.sumNeutralSamples_ += nNeutral; void SwStatsCpu::statsBGGR8Line0(const uint8_t *src[], SwIspStats &stats) { @@ -393,6 +442,23 @@ void SwStatsCpu::statsGBRG12PLine0(const uint8_t *src[], SwIspStats &stats) SWSTATS_FINISH_LINE_STATS() } +/** + * \brief Set what the neutral-pixel test measures against + * \param[in] gains Colour gains the IPA is currently applying + * \param[in] blackLevel Black level, normalised to [0,1] + * + * Called once per frame, before the statistics for that frame are gathered. + */ +void SwStatsCpu::setNeutralReference(const RGB &gains, + const RGB &blackLevel) +{ + for (unsigned int i = 0; i < 3; i++) + neutralGains_[i] = static_cast(gains[i]); + + /* The line statistics work on an 8-bit scale times the format divisor. */ + neutralBlack_ = static_cast(blackLevel[1]) * 255.0f; +} + /** * \brief Reset state to start statistics gathering for a new frame * \param[in] frame The frame number @@ -410,6 +476,8 @@ void SwStatsCpu::startFrame(uint32_t frame) for (auto &s : stats_) { s.sum_ = RGB({ 0, 0, 0 }); s.sumSamples_ = 0; + s.sumNeutral_ = RGB({ 0, 0, 0 }); + s.sumNeutralSamples_ = 0; s.yHistogram.fill(0); } } @@ -428,15 +496,20 @@ void SwStatsCpu::finishFrame(uint32_t frame, uint32_t bufferId) if (valid) { sharedStats_->sum_ = RGB({ 0, 0, 0 }); sharedStats_->sumSamples_ = 0; + sharedStats_->sumNeutral_ = RGB({ 0, 0, 0 }); + sharedStats_->sumNeutralSamples_ = 0; sharedStats_->yHistogram.fill(0); for (const auto &s : stats_) { sharedStats_->sum_ += s.sum_; sharedStats_->sumSamples_ += s.sumSamples_; + sharedStats_->sumNeutral_ += s.sumNeutral_; + sharedStats_->sumNeutralSamples_ += s.sumNeutralSamples_; for (unsigned int j = 0; j < SwIspStats::kYHistogramSize; j++) sharedStats_->yHistogram[j] += s.yHistogram[j]; } sharedStats_->sum_ >>= sumShift_; + sharedStats_->sumNeutral_ >>= sumShift_; } sharedStats_->valid = valid; From patchwork Wed Sep 2 19:15:53 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Martin Neiva de Carvalho X-Patchwork-Id: 28177 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id E7FC5C334A for ; Wed, 2 Sep 2026 20:04:37 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 8E794685CC; Wed, 2 Sep 2026 22:04:33 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.b="SFy9wrzR"; dkim-atps=neutral Received: from mail-vs1-xe2e.google.com (mail-vs1-xe2e.google.com [IPv6:2607:f8b0:4864:20::e2e]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id E7B96685CC for ; Wed, 2 Sep 2026 21:16:10 +0200 (CEST) Received: by mail-vs1-xe2e.google.com with SMTP id ada2fe7eead31-7564b7a9490so173588137.1 for ; Wed, 02 Sep 2026 12:16:10 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20251104; t=1788376570; x=1788981370; darn=lists.libcamera.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to:content-type; bh=/ZWh8i8tdb81/tpHA7u6h+2efRXqZdoe8eToPoqLh4w=; b=SFy9wrzRtVviv3plwU6i12dE57/WZ/aubr8kJxObZ14pdwBtsa6Dv7RjfbxpWmlUI9 CrBMNcdAVvIDHTrMZbfwg/evWukJPoju8gI2rsroUhOqhfodT7lSoEEphroc3YI02nvE jxhu0eHi0Thsw7e8u2of5KbFbu797NTFXtvxu+34EcJCFlpckLUhNILdN18H5f2t3M5l kKaUkGyx3v2hy140whbfK0UM0X598ohwDHlihGHnBMAzO6BlcrZ84iIctrM5oCWk4O/G vDOxPoVipQ7hpMCxh9TySKHY1QkHw7E9Wu82B0QjD/hiceoDsrwWI8KW2Z4QdiN2X4jj UzCQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1788376570; x=1788981370; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to:content-type; bh=/ZWh8i8tdb81/tpHA7u6h+2efRXqZdoe8eToPoqLh4w=; b=IbtXGYby/AiXTkS1QTo9YrrPvFRJXwqFNC5Og4BVu1VuUQPsalcW7RivZMrG4jt4XK uyQJJWxynNp+wL+HS4mFXnpiu88zbsU1xiC2Vz54sYxqoL4X0XDh2pIC0YWAUjy2iYxq AeClllFPgolv7NIPGryx1Z+lNWjT9zhyCVGynZ1N1KKdRZsiBh0dLnKtPyXlOjsPjSFG Wh5Fx3rl5TT/8SFuAwi2i6HjaB7aecDbQ4zUB3qAIhEWc8PxbuzG9ci6HGJV3GZ0HoeX ZmOUuhzMPc5I2INIY39OzfRrXTutNO51IBscGSB3pj7mkVfpAGOoJYnEFVZc11Mmj5vQ 0NfA== X-Gm-Message-State: AFuF++nkzbB/DJtfR5HXF4v5hBOg4dzQdB4np/Ew19yDyx/122Qgtefg aE5EdXPjoe+v3m56jl+EkkgxgJZXwo8SBgpZ+7XHqWYLoFEqX4h+gHueXVv/s4qYs9E= X-Gm-Gg: AYBFou1lK4vb8a3WkigkKu7snzFltci5jgjOJ62A4n910wY0TQjYPRqx5rPcq5bb1Dh 5Xy8MWyD/Sm1wmOaHI7dS51vr1ONgelrmQjs5irwgD6DX6SrOH85U7d+rK+7DZBZdE1nRgESz6b N1deuFECq9TvpmLDcb+yHb2g/O+40LUlMPT9MloFlvbdtjUPuThtt9Ur1FN8hshT7Vbt33M6L2Y owA6ke+gvPetiGxNKqOX1SpqfDeN2RoBemYOWSX4wd0tigBW4xhISWD6W6WYg0IHUPhyrLWLLtE io4ha9elFDG6mKDUH/IHxuv0T5dTbMI2oCo2JLEG9m/polRii0bqqvaZK/uKRsnq/B53Sg1msWy ev3JDlC8aen65Z1ZS35H8lSVNVULLBJ4zKYqjxrBDjB/FHc0jDpxn1KlUMC5O/y81aBal0Ewmkt QD1DeNz6RGf68OrZOdIs4JWeqPcB4wcG2KNZYhEEJxop/EjmlYVKZKgQZ2qos= X-Received: by 2002:a05:6102:5615:b0:786:56ab:910b with SMTP id ada2fe7eead31-78a2ee66256mr387946137.3.1788376569719; Wed, 02 Sep 2026 12:16:09 -0700 (PDT) Received: from laptop ([45.184.68.165]) by smtp.gmail.com with ESMTPSA id ada2fe7eead31-78a1a1feac0sm3003754137.2.2026.09.02.12.16.08 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 02 Sep 2026 12:16:09 -0700 (PDT) From: Martin Neiva de Carvalho To: libcamera-devel@lists.libcamera.org Cc: Martin Neiva de Carvalho Subject: [PATCH 4/5] ipa: softisp: Derive the minimum gain step from the sensor Date: Wed, 2 Sep 2026 16:15:53 -0300 Message-ID: <20260902191554.84922-5-martincarvalho@gmail.com> X-Mailer: git-send-email 2.55.0 In-Reply-To: <20260902191554.84922-1-martincarvalho@gmail.com> References: <20260902191554.84922-1-martincarvalho@gmail.com> MIME-Version: 1.0 X-Mailman-Approved-At: Wed, 02 Sep 2026 22:04:26 +0200 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" The AGC uses againMinStep to tell when a multiplicative correction is too small to reach the next gain code, so that it steps by one code rather than asking for a change that quantises away and stalls the loop. The value is currently taken as one hundredth of the gain range, which ties it to how much gain the sensor offers rather than to how finely that gain can be set. On a sensor with a wide range the two are far apart. With a range of 1 to 248 the step becomes 2.47, so the smallest correction the AGC can make is around 12% at a typical indoor gain - much larger than the correction it is trying to make. Every step then crosses the target and the loop oscillates about it indefinitely, which is visible as the picture pulsing. Take the step from the sensor helper instead, as the gain difference between the two lowest codes. On an OV02C10, whose gain is linear in 1/16 steps, this changes it from 2.47 to 0.0625. Measured over a static scene, with the two settings interleaved so that changing light could not favour either, the frame to frame brightness ripple around the trend drops from 2.80 to 0.38 DN RMS, and its peak to peak excursion from 9.5 to 2.3 DN. Signed-off-by: Martin Neiva de Carvalho --- src/ipa/softisp/softisp.cpp | 22 +++++++++++++++++++--- 1 file changed, 19 insertions(+), 3 deletions(-) diff --git a/src/ipa/softisp/softisp.cpp b/src/ipa/softisp/softisp.cpp index aec04c528..5a20475b1 100644 --- a/src/ipa/softisp/softisp.cpp +++ b/src/ipa/softisp/softisp.cpp @@ -228,10 +228,26 @@ int IPASoftIsp::configure(const IPAConfigInfo &configInfo) context_.configuration.agc.againMin = camHelper_->gain(againMin); context_.configuration.agc.againMax = camHelper_->gain(againMax); context_.configuration.agc.again10 = std::max(context_.configuration.agc.againMin, 1.0); + /* + * The smallest gain change the sensor can actually make. + * + * The AGC needs this to know when a multiplicative correction + * is too small to reach the next gain code, so that it steps + * by one code instead of asking for a change that quantises + * away and stalls the loop. + * + * Taking a fixed fraction of the total range instead ties the + * step to how much gain the sensor offers, which is unrelated + * to how finely it can be set. On a sensor whose range runs + * from 1 to 248 that fraction is 2.47, so the smallest + * correction the AGC can make is 12% at a typical indoor gain, + * far larger than the correction it wants. The loop then + * cannot settle: every step crosses the target and the + * exposure oscillates about it indefinitely. + */ context_.configuration.agc.againMinStep = - (context_.configuration.agc.againMax - - context_.configuration.agc.againMin) / - 100.0; + camHelper_->gain(againMin + 1) - + camHelper_->gain(againMin); if (camHelper_->blackLevel().has_value()) { /* * The black level from camHelper_ is a 16 bit value, software ISP From patchwork Wed Sep 2 19:15:54 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Martin Neiva de Carvalho X-Patchwork-Id: 28178 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 74B7AC3348 for ; Wed, 2 Sep 2026 20:04:38 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 4B0DE685E4; Wed, 2 Sep 2026 22:04:34 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.b="kyCkl/fW"; dkim-atps=neutral Received: from mail-ua1-x934.google.com (mail-ua1-x934.google.com [IPv6:2607:f8b0:4864:20::934]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 180CA685CC for ; Wed, 2 Sep 2026 21:16:13 +0200 (CEST) Received: by mail-ua1-x934.google.com with SMTP id a1e0cc1a2514c-97bf8f6a31eso558862241.1 for ; Wed, 02 Sep 2026 12:16:13 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20251104; t=1788376572; x=1788981372; darn=lists.libcamera.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to:content-type; bh=8kt0mrOlzLzzldA/TB1pVNyPk4PXIcCqxuuviWAZGXc=; b=kyCkl/fWYjUwUQ00TYgp2YEkeAC27YWbX/XBGlkgYTri5D8zSQMJyqbWzK+Noaamf6 ZJdJGm6YOhDB7kE7TEtrMkT0GniIs6FDdDZ9OSkm1uuBqPNoqbs1DhK2jDd0JPUh46de ZfKE3iKHP6EtBAoDz+XOncL0QZ1mioacCwPB8jyRy0BIp2Ta/qsvgHxY36O7wbyxky/j HsIy+UUsJFDbY5qzN6LlEIDWVXQGxX8A4Ubf0qhD5gGNlSOmE2QEy0mXzu5rSWM7kU4s psrIHSbCyA9NFKtzclTZT2iUUzBvAwqki3YSTOntgOiTmNMjNNyh2x4PQuSi4MAAj4vq tnrA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1788376572; x=1788981372; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to:content-type; bh=8kt0mrOlzLzzldA/TB1pVNyPk4PXIcCqxuuviWAZGXc=; b=scRhD5KOCG51s+bqjiAWWO0WBATulHjmCgo95VRHqZYQ0oqbyuxqSXJUjtY5nyzYa9 SHHXUmPjU/IxPr0atccgrRH2JBLKgJNepgX9Qvf7SWAlBO1FUqQKXb8bta/mQLCqnaOe RgENbR0cUh5WGydSM1Jptb/uAFuhoQGIiJJ45pMxvPV52nOrUVTGeM3R6x9IgCKP8Ojg tuZKFvwldV1nYIdtjKCsk1otWO41j8/QAhF64XaFJOwxrskRZWISFWmWkuv2fn+nkW9p MSWP8yC51lxFMDJ+U7bWRSwrU6MaKTviDQSjd3Qb/ZWa+Tn5JhNhnzJ/4Sdux20Nnopn WfUw== X-Gm-Message-State: AFuF++m/BfrJ9t2E637DYgQ7+m3inwUMYPy7GTUgqHfIjC7FngDAuVuR +cSgiuP+BPG9XlqPZ8cJyys/x7Fg7Y9CtRZMdSK/zpetnFOHhNaTAneX3YXFtEX1ftM= X-Gm-Gg: AYBFou0qdAPU0+xGfeNx/N+jVL+g9ZQZuub4QdCyGY0XjcnjoDxcAwWrqsGwMhPefUI NxS3qLpQlHE1CbVrppI7z/qLcO6tzgk6/dtYwQE0vzUNr/3xWSUVfb4KeAPE/FCoqHL+EFfYZJ0 TiziRci4ypzyBCDqwqeYHHUafWwvZBvNCXMVKqlKVWrMFdCp97mF1XE2szE4pMyMuWgtiqYcAIh 6w/MFPC/u6RMkg73QXVrQU/UcIECmwJmk8+EZW1GT16CIydbL9b90NkPu1eSDvzT3nliWR9913T XyfNdcRL2QDoB6GXfNBaEGGaRLxYde6GLAPjz3DxZJmc1riQG9BKX4AW2+U3quHw7JIH04hsGf0 pmCyauPmwjS4shYRaTjuZw1x6EASWga5NkDLtHMernLRv9I7cEDH1ZSlSOn5L5j4ArqF7o5VW+h NgBCJznI7UbNH7Km2/req5AC2uHr3+OHV3fOKLqH3ujWMjtUGUbJkLl6ioZ/0= X-Received: by 2002:a05:6102:524a:b0:77c:dbae:f9d2 with SMTP id ada2fe7eead31-78a1f2df7a6mr2094212137.9.1788376571801; Wed, 02 Sep 2026 12:16:11 -0700 (PDT) Received: from laptop ([45.184.68.165]) by smtp.gmail.com with ESMTPSA id ada2fe7eead31-78a1a1feac0sm3003754137.2.2026.09.02.12.16.10 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 02 Sep 2026 12:16:11 -0700 (PDT) From: Martin Neiva de Carvalho To: libcamera-devel@lists.libcamera.org Cc: Martin Neiva de Carvalho Subject: [PATCH 5/5] libcamera: software_isp: Centre the crop the GPU debayer applies Date: Wed, 2 Sep 2026 16:15:54 -0300 Message-ID: <20260902191554.84922-6-martincarvalho@gmail.com> X-Mailer: git-send-email 2.55.0 In-Reply-To: <20260902191554.84922-1-martincarvalho@gmail.com> References: <20260902191554.84922-1-martincarvalho@gmail.com> MIME-Version: 1.0 X-Mailman-Approved-At: Wed, 02 Sep 2026 22:04:26 +0200 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" A stream whose aspect ratio differs from the sensor's cannot show all of it, and cropping the sides is the right answer - black bars would be worse. But the crop has to be taken from both sides. The viewport is sized from the sensor frame while the buffer being drawn into is the size the application asked for, so that buffer covers only the corner of the viewport running from NDC -1 to where its own width reaches. Translating the quad by -(1 - scale) puts the texture's left and bottom edges exactly at that corner, so everything cropped comes off one side. Measured on an OV02C10 by registering a 640x480 stream against a 1920x1080 one of the same scene - searching for the scale and offset that maximise the normalised correlation between them - the smaller stream showed a 1391x1043 window of the sensor's 1920x1080, starting at (48, 22). Centring the 4:3 window in the 16:9 frame calls for 1440x1080 starting at (240, 0). A subject had to sit a sixth of a frame off centre to appear centred, and the framing moved whenever the application chose a different resolution. Translate to the middle of the region the buffer actually covers instead. That region spans NDC -1 to 2 * out / viewport - 1, so its midpoint is out / viewport - 1. The same measurement then reports 1438x1079 starting at (243, 0), which is the centred window to within the three pixels the registration can resolve. At the native size this shares the eight columns and twelve rows of Bayer border between the two edges rather than taking them all off one, moving the picture by half a border. Nothing is resampled: the viewport and the scale factor are untouched. Signed-off-by: Martin Neiva de Carvalho --- src/libcamera/software_isp/debayer_egl.cpp | 23 +++++++++++++++++++--- 1 file changed, 20 insertions(+), 3 deletions(-) diff --git a/src/libcamera/software_isp/debayer_egl.cpp b/src/libcamera/software_isp/debayer_egl.cpp index c042d3864..0ccae473f 100644 --- a/src/libcamera/software_isp/debayer_egl.cpp +++ b/src/libcamera/software_isp/debayer_egl.cpp @@ -388,16 +388,33 @@ void DebayerEGL::setShaderVariableValues(eGLImage &eglImageIn, const DebayerPara /* * Scale the output size from the native size the algorithm produces for * the input size. Keep the aspect ratio and prefer cropping over black - * bars. + * bars - but crop from both sides, not from one. + * + * The viewport is the sensor frame while the buffer drawn into is the + * requested size, so the buffer covers only the corner of the viewport + * that starts at NDC -1 and ends where its own width reaches. Placing + * the quad at -(1 - scale) puts the texture's left edge exactly at + * that corner, so everything cropped is taken off one side: metered + * by registering a 640x480 stream against a 1920x1080 one of the same + * scene, the frame shown ran from x=48 rather than from the x=240 + * that centring the 4:3 window in a 16:9 sensor calls for. + * + * Centre it on the region the buffer actually covers instead. That + * region spans NDC -1 to 2*out/viewport - 1, whose midpoint is + * out/viewport - 1, and putting the quad's centre there splits the + * crop evenly between the two edges at every size, the native one + * included - where it shares the eight columns of Bayer border + * instead of taking them all off the right. */ GLfloat scale = std::max((GLfloat)outputSize_.width / nativeOutputSize_.width, (GLfloat)outputSize_.height / nativeOutputSize_.height); - GLfloat trans = -(1.0f - scale); + GLfloat transX = (GLfloat)outputSize_.width / width_ - 1.0f; + GLfloat transY = (GLfloat)outputSize_.height / height_ - 1.0f; GLfloat projMatrix[] = { scale, 0, 0, 0, 0, scale, 0, 0, 0, 0, 1, 0, - trans, trans, 0, 1 + transX, transY, 0, 1 }; /* Static const coordinates */ static const GLfloat vcoordinates[4][2] = {