{"id":23003,"url":"https://patchwork.libcamera.org/api/patches/23003/?format=json","web_url":"https://patchwork.libcamera.org/patch/23003/","project":{"id":1,"url":"https://patchwork.libcamera.org/api/projects/1/?format=json","name":"libcamera","link_name":"libcamera","list_id":"libcamera_core","list_email":"libcamera-devel@lists.libcamera.org","web_url":"","scm_url":"","webscm_url":""},"msgid":"<20250324170803.103296-3-stefan.klug@ideasonboard.com>","date":"2025-03-24T17:07:37","name":"[2/5] libipa: histogram: Fix quantile() calculation for fractional results","commit_ref":null,"pull_url":null,"state":"accepted","archived":false,"hash":"5d4becb986887f490566b53ddf6a98cf2016784a","submitter":{"id":184,"url":"https://patchwork.libcamera.org/api/people/184/?format=json","name":"Stefan Klug","email":"stefan.klug@ideasonboard.com"},"delegate":null,"mbox":"https://patchwork.libcamera.org/patch/23003/mbox/","series":[{"id":5077,"url":"https://patchwork.libcamera.org/api/series/5077/?format=json","web_url":"https://patchwork.libcamera.org/project/libcamera/list/?series=5077","date":"2025-03-24T17:07:35","name":"Fix histogram for some (corner) cases","version":1,"mbox":"https://patchwork.libcamera.org/series/5077/mbox/"}],"comments":"https://patchwork.libcamera.org/api/patches/23003/comments/","check":"pending","checks":"https://patchwork.libcamera.org/api/patches/23003/checks/","tags":{},"headers":{"Return-Path":"","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id 2AF6CC3213\n\tfor ;\n\tMon, 24 Mar 2025 17:08:21 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id D266968959;\n\tMon, 24 Mar 2025 18:08:20 +0100 (CET)","from perceval.ideasonboard.com (perceval.ideasonboard.com\n\t[213.167.242.64])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id D394B68945\n\tfor ;\n\tMon, 24 Mar 2025 18:08:18 +0100 (CET)","from ideasonboard.com (unknown\n\t[IPv6:2a00:6020:448c:6c00:854:5bc6:b2a7:e242])\n\tby perceval.ideasonboard.com (Postfix) with ESMTPSA id 50E78A8F;\n\tMon, 24 Mar 2025 18:06:32 +0100 (CET)"],"Authentication-Results":"lancelot.ideasonboard.com; dkim=pass (1024-bit key;\n\tunprotected) header.d=ideasonboard.com header.i=@ideasonboard.com\n\theader.b=\"ehz8lwSq\"; dkim-atps=neutral","DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com;\n\ts=mail; t=1742835992;\n\tbh=rjzPA6Us67ZjTb/h97K3bGEFgnhDfi+yLIZKMoqHl98=;\n\th=From:To:Cc:Subject:Date:In-Reply-To:References:From;\n\tb=ehz8lwSqBj+OUyB8X9cNv07AnpAQtNh7Te3saFBNbFkK27z0u4epv0CSlql/sRbL3\n\tE+hAu54DBE44uXQLCAuyvIKIiWL67SI72xPPTRLwEZOIgs3DuaWa7z//y/OnQvSzAq\n\txvbH2d8aIxN2yWvulEEk/yEW4SdAH8j09ALaxjOg=","From":"Stefan Klug ","To":"libcamera-devel@lists.libcamera.org","Cc":"Stefan Klug ","Subject":"[PATCH 2/5] libipa: histogram: Fix quantile() calculation for\n\tfractional results","Date":"Mon, 24 Mar 2025 18:07:37 +0100","Message-ID":"<20250324170803.103296-3-stefan.klug@ideasonboard.com>","X-Mailer":"git-send-email 2.43.0","In-Reply-To":"<20250324170803.103296-1-stefan.klug@ideasonboard.com>","References":"<20250324170803.103296-1-stefan.klug@ideasonboard.com>","MIME-Version":"1.0","Content-Transfer-Encoding":"8bit","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" "},"content":"The calculation of the frac variable is based solely on integers and\ntherefore results in the fractional part being either 0 or 1.\n\nIn the original code from RaspberryPi this is mitigated by casting the\nnominator to a double. This works for most cases, but fails when q is\nvery small because of the quantization introduced by item being an\ninteger.\n\nFix both issues by doing the full calculation in double.\n\nSigned-off-by: Stefan Klug \n---\n src/ipa/libipa/histogram.cpp | 3 ++-\n 1 file changed, 2 insertions(+), 1 deletion(-)","diff":"diff --git a/src/ipa/libipa/histogram.cpp b/src/ipa/libipa/histogram.cpp\nindex 10e44b54a0cf..c19a4cbbf3cd 100644\n--- a/src/ipa/libipa/histogram.cpp\n+++ b/src/ipa/libipa/histogram.cpp\n@@ -130,7 +130,8 @@ double Histogram::quantile(double q, uint32_t first, uint32_t last) const\n \tif (cumulative_[first + 1] == cumulative_[first])\n \t\tfrac = 0;\n \telse\n-\t\tfrac = (item - cumulative_[first]) / (cumulative_[first + 1] - cumulative_[first]);\n+\t\tfrac = (q * total() - cumulative_[first]) /\n+\t\t (cumulative_[first + 1] - cumulative_[first]);\n \treturn first + frac;\n }\n \n","prefixes":["2/5"]}