From patchwork Wed Nov 18 13:31:03 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Barnab=C3=A1s_P=C5=91cze?= X-Patchwork-Id: 28992 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id B11C3BDE17 for ; Wed, 18 Nov 2026 13:31:42 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 5B6106A8E4; Wed, 18 Nov 2026 14:31:33 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="oDHwoMsl"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id B62BF6A8C7 for ; Wed, 18 Nov 2026 14:31:28 +0100 (CET) Received: from pb-laptop.local (185.221.140.49.nat.pool.zt.hu [185.221.140.49]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 4117A4F1; Wed, 18 Nov 2026 14:29:00 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1795008540; bh=Xn00jw8vKbURcwPy8NXsQf+Li5w69cdBRSFdHIEjvNI=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=oDHwoMslnpDhAu77c5tX/6QQ3mRhtGqgTxuUr+gFZ/mMRX/kfAdEY1cjO3j7q1+yP 9HQ+hdFxZMd3/rq3YX9VgiLpyBed4JkTe/KISeNnaB2/vgP8vgcOfxgmBu9NpwFOeQ pyFSCehFf+8gTgMo79szIb56Q/1Spx1kDUe8CCI4= From: =?utf-8?q?Barnab=C3=A1s_P=C5=91cze?= To: libcamera-devel@lists.libcamera.org Cc: Kieran Bingham , Jacopo Mondi Subject: [PATCH v6 03/23] ipa: libipa: quantized: Use `double` when necessary Date: Wed, 18 Nov 2026 14:31:03 +0100 Message-ID: <20261118133123.1710288-4-barnabas.pocze@ideasonboard.com> X-Mailer: git-send-email 2.56.1 In-Reply-To: <20261118133123.1710288-1-barnabas.pocze@ideasonboard.com> References: <20261118133123.1710288-1-barnabas.pocze@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" The currently used `float` type (usually IEEE754 binary32) can only represent integers exactly up to 2^24. So automatically use `double` (usually IEEE754 binary64) if more than 24 bits are needed for the fixed-point representation. Also add a static assertion that IEEE754 floating point types are used. But note, that does not guarantee that float and double are binary32 and binary64. Signed-off-by: Barnabás Pőcze Reviewed-by: Kieran Bingham Acked-by: Jacopo Mondi --- src/ipa/libipa/fixedpoint.h | 17 ++++++++++------- 1 file changed, 10 insertions(+), 7 deletions(-) diff --git a/src/ipa/libipa/fixedpoint.h b/src/ipa/libipa/fixedpoint.h index 66863075c6..0640605bee 100644 --- a/src/ipa/libipa/fixedpoint.h +++ b/src/ipa/libipa/fixedpoint.h @@ -25,19 +25,20 @@ private: static constexpr unsigned int bits = I + F; static_assert(bits <= sizeof(UT) * 8, "FixedPointQTraits: too many bits for type UT"); - /* - * If fixed point storage is required with more than 24 bits, consider - * updating this implementation to use double-precision floating point. - */ - static_assert(bits <= 24, "Floating point precision may be insufficient for more than 24 bits"); + /* IEEE754 binary64 can faithfully represent integers up to 2^53 */ + static_assert(bits <= 53, "Floating point precision may be insufficient"); static constexpr UT bitMask = bits < sizeof(UT) * 8 ? (UT{ 1 } << bits) - 1 : ~UT{ 0 }; public: + /* IEEE754 binary32 can faithfully represent integers up to 2^24 */ using QuantizedType = UT; - using FloatingType = float; + using FloatingType = std::conditional_t; + + static_assert(std::numeric_limits::is_iec559, + "requires IEEE754 floating point"); static constexpr UT qMin = std::is_signed_v ? -(UT{ 1 } << (bits - 1)) @@ -88,7 +89,7 @@ namespace details { template constexpr auto qtype() { - static_assert(Bits <= 32, + static_assert(Bits <= 64, "Unsupported number of bits for quantized type"); if constexpr (Bits <= 8) @@ -97,6 +98,8 @@ constexpr auto qtype() return int16_t(); else if constexpr (Bits <= 32) return int32_t(); + else if constexpr (Bits <= 64) + return int64_t(); } } /* namespace details */