From patchwork Fri Sep 18 12:09:31 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Barnab=C3=A1s_P=C5=91cze?= X-Patchwork-Id: 28341 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 29996C3356 for ; Fri, 18 Sep 2026 12:10:07 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 97A7768753; Fri, 18 Sep 2026 14:09:59 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="DI5XSMo1"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id DC9FB6870F for ; Fri, 18 Sep 2026 14:09:53 +0200 (CEST) Received: from pb-laptop.local (185.221.142.0.nat.pool.zt.hu [185.221.142.0]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 145C1417F for ; Fri, 18 Sep 2026 14:08:11 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1789733291; bh=LJcc+7Q0+oJtRKwuFob0FA/DlMoY4x7UjJlByZ1N5mQ=; h=From:To:Subject:Date:In-Reply-To:References:From; b=DI5XSMo1cx/MWvWashFiqJ0C79n4HFDH4svwV65Io3yVMdn+L0bEgFjVqAzdqC9Ms CSZXbY9hV3DzlV9yJ+bDM15LUOHDJQdw+MEPs7Hr86NmNsCh4cKSC89LFuSq6AjZ6S 1da0D9vfb/eAVjlR4jxP8oDLO0Fek0Jtfr9znvXM= From: =?utf-8?q?Barnab=C3=A1s_P=C5=91cze?= To: libcamera-devel@lists.libcamera.org Subject: [PATCH v3 03/21] ipa: libipa: quantized: Use `double` when necessary Date: Fri, 18 Sep 2026 14:09:31 +0200 Message-ID: <20260918120949.191668-4-barnabas.pocze@ideasonboard.com> X-Mailer: git-send-email 2.55.0 In-Reply-To: <20260918120949.191668-1-barnabas.pocze@ideasonboard.com> References: <20260918120949.191668-1-barnabas.pocze@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" If more than 24 bits would be used, use `double` as not all integers above 2^24 can be exactly represented by an IEEE754 binary32. Also add a static assertion that IEEE754 floating point types are used. Signed-off-by: Barnabás Pőcze --- src/ipa/libipa/fixedpoint.h | 17 ++++++++++------- 1 file changed, 10 insertions(+), 7 deletions(-) diff --git a/src/ipa/libipa/fixedpoint.h b/src/ipa/libipa/fixedpoint.h index 66863075c6..0640605bee 100644 --- a/src/ipa/libipa/fixedpoint.h +++ b/src/ipa/libipa/fixedpoint.h @@ -25,19 +25,20 @@ private: static constexpr unsigned int bits = I + F; static_assert(bits <= sizeof(UT) * 8, "FixedPointQTraits: too many bits for type UT"); - /* - * If fixed point storage is required with more than 24 bits, consider - * updating this implementation to use double-precision floating point. - */ - static_assert(bits <= 24, "Floating point precision may be insufficient for more than 24 bits"); + /* IEEE754 binary64 can faithfully represent integers up to 2^53 */ + static_assert(bits <= 53, "Floating point precision may be insufficient"); static constexpr UT bitMask = bits < sizeof(UT) * 8 ? (UT{ 1 } << bits) - 1 : ~UT{ 0 }; public: + /* IEEE754 binary32 can faithfully represent integers up to 2^24 */ using QuantizedType = UT; - using FloatingType = float; + using FloatingType = std::conditional_t; + + static_assert(std::numeric_limits::is_iec559, + "requires IEEE754 floating point"); static constexpr UT qMin = std::is_signed_v ? -(UT{ 1 } << (bits - 1)) @@ -88,7 +89,7 @@ namespace details { template constexpr auto qtype() { - static_assert(Bits <= 32, + static_assert(Bits <= 64, "Unsupported number of bits for quantized type"); if constexpr (Bits <= 8) @@ -97,6 +98,8 @@ constexpr auto qtype() return int16_t(); else if constexpr (Bits <= 32) return int32_t(); + else if constexpr (Bits <= 64) + return int64_t(); } } /* namespace details */