{"id":23090,"url":"https://patchwork.libcamera.org/api/patches/23090/?format=json","web_url":"https://patchwork.libcamera.org/patch/23090/","project":{"id":1,"url":"https://patchwork.libcamera.org/api/projects/1/?format=json","name":"libcamera","link_name":"libcamera","list_id":"libcamera_core","list_email":"libcamera-devel@lists.libcamera.org","web_url":"","scm_url":"","webscm_url":""},"msgid":"<20250401123633.58887-4-stefan.klug@ideasonboard.com>","date":"2025-04-01T12:36:11","name":"[v2,3/5] libipa: histogram: Fix quantile() calculation for fractional results","commit_ref":null,"pull_url":null,"state":"accepted","archived":false,"hash":"a4551f17865540283513930979b1347514e123f7","submitter":{"id":184,"url":"https://patchwork.libcamera.org/api/people/184/?format=json","name":"Stefan Klug","email":"stefan.klug@ideasonboard.com"},"delegate":null,"mbox":"https://patchwork.libcamera.org/patch/23090/mbox/","series":[{"id":5099,"url":"https://patchwork.libcamera.org/api/series/5099/?format=json","web_url":"https://patchwork.libcamera.org/project/libcamera/list/?series=5099","date":"2025-04-01T12:36:08","name":"Fix histogram for some (corner) cases","version":2,"mbox":"https://patchwork.libcamera.org/series/5099/mbox/"}],"comments":"https://patchwork.libcamera.org/api/patches/23090/comments/","check":"pending","checks":"https://patchwork.libcamera.org/api/patches/23090/checks/","tags":{},"headers":{"Return-Path":"","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id 846DCC3213\n\tfor ;\n\tTue, 1 Apr 2025 12:36:49 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id 456686898A;\n\tTue, 1 Apr 2025 14:36:49 +0200 (CEST)","from perceval.ideasonboard.com (perceval.ideasonboard.com\n\t[IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id 81CE568987\n\tfor ;\n\tTue, 1 Apr 2025 14:36:47 +0200 (CEST)","from ideasonboard.com (unknown\n\t[IPv6:2a00:6020:448c:6c00:14c7:4fcc:495b:719f])\n\tby perceval.ideasonboard.com (Postfix) with ESMTPSA id 57E128DB;\n\tTue, 1 Apr 2025 14:34:55 +0200 (CEST)"],"Authentication-Results":"lancelot.ideasonboard.com; dkim=pass (1024-bit key;\n\tunprotected) header.d=ideasonboard.com header.i=@ideasonboard.com\n\theader.b=\"ApnMNn2I\"; dkim-atps=neutral","DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com;\n\ts=mail; t=1743510895;\n\tbh=l3JdK+kvaObWpCAA5lqKijrnmQNvjPFPucRyBvKUfMI=;\n\th=From:To:Cc:Subject:Date:In-Reply-To:References:From;\n\tb=ApnMNn2IabMUGMW2FhdyR56T9SG+GgjsmcIqMszJ527Y+5YZlyB5Bz/W8lal+zrwn\n\t5JcpdLpp8kpn3fuUYCOUleJcQXlmGAKCtgvlSj/fr6OQ9jqNqjXRbKPsN9au6Jqe6i\n\tUXeSnzk2S1wScCKuZuQhOYoSRIp0Z8doS+GPFqSk=","From":"Stefan Klug ","To":"libcamera-devel@lists.libcamera.org","Cc":"Stefan Klug ,\n\tKieran Bingham ,\n\tLaurent Pinchart ","Subject":"[PATCH v2 3/5] libipa: histogram: Fix quantile() calculation for\n\tfractional results","Date":"Tue, 1 Apr 2025 14:36:11 +0200","Message-ID":"<20250401123633.58887-4-stefan.klug@ideasonboard.com>","X-Mailer":"git-send-email 2.43.0","In-Reply-To":"<20250401123633.58887-1-stefan.klug@ideasonboard.com>","References":"<20250401123633.58887-1-stefan.klug@ideasonboard.com>","MIME-Version":"1.0","Content-Transfer-Encoding":"8bit","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" "},"content":"The calculation of the frac variable is based solely on integers and\ntherefore results in the fractional part being either 0 or 1.\n\nIn the original code from RaspberryPi this is mitigated by casting the\nnominator to a double. This works for most cases, but fails when q is\nvery small because of the quantization introduced by item being an\ninteger.\n\nFix both issues by doing the full calculation in double and remove the\nshould_fail tag.\n\nSigned-off-by: Stefan Klug \nReviewed-by: Kieran Bingham \nReviewed-by: Laurent Pinchart \n\n---\n\nChanges in v2:\n- Order fix patch after test\n- Remove should_fail tag\n- Break line before operator instead of after\n---\n src/ipa/libipa/histogram.cpp | 3 ++-\n test/ipa/libipa/meson.build | 2 +-\n 2 files changed, 3 insertions(+), 2 deletions(-)","diff":"diff --git a/src/ipa/libipa/histogram.cpp b/src/ipa/libipa/histogram.cpp\nindex 10e44b54a0cf..ea042f0a17b9 100644\n--- a/src/ipa/libipa/histogram.cpp\n+++ b/src/ipa/libipa/histogram.cpp\n@@ -130,7 +130,8 @@ double Histogram::quantile(double q, uint32_t first, uint32_t last) const\n \tif (cumulative_[first + 1] == cumulative_[first])\n \t\tfrac = 0;\n \telse\n-\t\tfrac = (item - cumulative_[first]) / (cumulative_[first + 1] - cumulative_[first]);\n+\t\tfrac = (q * total() - cumulative_[first])\n+\t\t / (cumulative_[first + 1] - cumulative_[first]);\n \treturn first + frac;\n }\n \ndiff --git a/test/ipa/libipa/meson.build b/test/ipa/libipa/meson.build\nindex 83c84bd8c227..8c63ebd8e2f7 100644\n--- a/test/ipa/libipa/meson.build\n+++ b/test/ipa/libipa/meson.build\n@@ -2,7 +2,7 @@\n \n libipa_test = [\n {'name': 'fixedpoint', 'sources': ['fixedpoint.cpp']},\n- {'name': 'histogram', 'sources': ['histogram.cpp'], 'should_fail': true},\n+ {'name': 'histogram', 'sources': ['histogram.cpp']},\n {'name': 'interpolator', 'sources': ['interpolator.cpp']},\n ]\n \n","prefixes":["v2","3/5"]}