From patchwork Fri Nov 13 16:40:36 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Barnab=C3=A1s_P=C5=91cze?= X-Patchwork-Id: 28890 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id E4370C3304 for ; Fri, 13 Nov 2026 16:41:11 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 9153A6A81E; Fri, 13 Nov 2026 17:41:02 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="ja3ZJAJw"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id CA40D690A0 for ; Fri, 13 Nov 2026 17:40:58 +0100 (CET) Received: from pb-laptop.local (185.221.140.49.nat.pool.zt.hu [185.221.140.49]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 14960269; Fri, 13 Nov 2026 17:38:34 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1794587914; bh=+ARIYmR/6U1+rHhiq14e5JfJWpeBtpP2WyZ+kOlryVg=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=ja3ZJAJwsL8hbUinrhaEfU4VhZC4FCe7CitHudp2wxkJZ68GeVi4AQGqzbsbsBptc Ln2le04vh204QQpiPfPtLU6l3ELbrIbqbl/y5IGj8PbG24pPOEOsXyhCzTD7+oQSW+ Obpwuq1Tgpaf0MxmRKPKVByGCnEclMwLOwfhtHfw= From: =?utf-8?q?Barnab=C3=A1s_P=C5=91cze?= To: libcamera-devel@lists.libcamera.org Cc: Kieran Bingham Subject: [PATCH v4 03/21] ipa: libipa: quantized: Use `double` when necessary Date: Fri, 13 Nov 2026 17:40:36 +0100 Message-ID: <20261113164054.267164-4-barnabas.pocze@ideasonboard.com> X-Mailer: git-send-email 2.56.1 In-Reply-To: <20261113164054.267164-1-barnabas.pocze@ideasonboard.com> References: <20261113164054.267164-1-barnabas.pocze@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" The currently used `float` type (usually IEEE754 binary32) can only represent integers exactly up to 2^24. So automatically use `double` (usually IEEE754 binary64) if more than 24 bits are needed for the fixed-point representation. Also add a static assertion that IEEE754 floating point types are used. But note, that does not guarantee that float and double are binary32 and binary64. Signed-off-by: Barnabás Pőcze Reviewed-by: Kieran Bingham --- src/ipa/libipa/fixedpoint.h | 17 ++++++++++------- 1 file changed, 10 insertions(+), 7 deletions(-) diff --git a/src/ipa/libipa/fixedpoint.h b/src/ipa/libipa/fixedpoint.h index 66863075c6..0640605bee 100644 --- a/src/ipa/libipa/fixedpoint.h +++ b/src/ipa/libipa/fixedpoint.h @@ -25,19 +25,20 @@ private: static constexpr unsigned int bits = I + F; static_assert(bits <= sizeof(UT) * 8, "FixedPointQTraits: too many bits for type UT"); - /* - * If fixed point storage is required with more than 24 bits, consider - * updating this implementation to use double-precision floating point. - */ - static_assert(bits <= 24, "Floating point precision may be insufficient for more than 24 bits"); + /* IEEE754 binary64 can faithfully represent integers up to 2^53 */ + static_assert(bits <= 53, "Floating point precision may be insufficient"); static constexpr UT bitMask = bits < sizeof(UT) * 8 ? (UT{ 1 } << bits) - 1 : ~UT{ 0 }; public: + /* IEEE754 binary32 can faithfully represent integers up to 2^24 */ using QuantizedType = UT; - using FloatingType = float; + using FloatingType = std::conditional_t; + + static_assert(std::numeric_limits::is_iec559, + "requires IEEE754 floating point"); static constexpr UT qMin = std::is_signed_v ? -(UT{ 1 } << (bits - 1)) @@ -88,7 +89,7 @@ namespace details { template constexpr auto qtype() { - static_assert(Bits <= 32, + static_assert(Bits <= 64, "Unsupported number of bits for quantized type"); if constexpr (Bits <= 8) @@ -97,6 +98,8 @@ constexpr auto qtype() return int16_t(); else if constexpr (Bits <= 32) return int32_t(); + else if constexpr (Bits <= 64) + return int64_t(); } } /* namespace details */