From patchwork Wed Oct 29 17:24:30 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kieran Bingham X-Patchwork-Id: 24894 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id CEBBEC32CE for ; Wed, 29 Oct 2025 17:25:03 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 258EB608CF; Wed, 29 Oct 2025 18:25:03 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="wC/hCxxK"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 8C39E60856 for ; Wed, 29 Oct 2025 18:24:47 +0100 (CET) Received: from Monstersaurus.hippo-penny.ts.net (cpc89244-aztw30-2-0-cust6594.18-1.cable.virginm.net [86.31.185.195]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 1F5F84E53; Wed, 29 Oct 2025 18:22:58 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1761758578; bh=0DJtx4vem6f5b0jigxFARPnYoLjWn0g5Fx9x6O15FKA=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=wC/hCxxKippFeOO1d5FixAX9FgFuJ5EV4z+PuVcRL95a63t2Laa1BCYLK50PNSXq1 Lrbw4XWtpiRPtBWTWPxko1S9qBRaKYAvwgKeFl6IdfEmqiiGTBC2ZaGP/IuwTMFcR2 OVoLDNj2O+jvvGxGFO4j3MrWs7FMTVf2WHzV0uL0= From: Kieran Bingham To: libcamera devel Cc: Kieran Bingham Subject: [PATCH v2 05/13] ipa: libipa: fixedpoint: Fix unsigned usage Date: Wed, 29 Oct 2025 17:24:30 +0000 Message-ID: <20251029172439.1513907-6-kieran.bingham@ideasonboard.com> X-Mailer: git-send-email 2.50.1 In-Reply-To: <20251029172439.1513907-1-kieran.bingham@ideasonboard.com> References: <20251029172439.1513907-1-kieran.bingham@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" The fixedToFloatingPoint does not support unsigned Q types, and incorrectly sign-extends all values which have the top most bit set in the quantized values. Fix this by ensuring that only signed types perform sign extension, and simplify the calcuation for unsigned types. Convert the storage of the test cases to signed types to correctly represent their intended purpose, to prevent test failures. Signed-off-by: Kieran Bingham --- src/ipa/libipa/fixedpoint.h | 3 +++ test/ipa/libipa/fixedpoint.cpp | 6 +++--- 2 files changed, 6 insertions(+), 3 deletions(-) diff --git a/src/ipa/libipa/fixedpoint.h b/src/ipa/libipa/fixedpoint.h index 9a7674e528d9..557f7f5b3382 100644 --- a/src/ipa/libipa/fixedpoint.h +++ b/src/ipa/libipa/fixedpoint.h @@ -51,6 +51,9 @@ constexpr R fixedToFloatingPoint(T number) static_assert(sizeof(int) >= sizeof(T)); static_assert(I + F <= sizeof(T) * 8); + if constexpr (std::is_unsigned_v) + return static_cast(number) / static_cast(1 << F); + /* * Recreate the upper bits in case of a negative number by shifting the sign * bit from the fixed point to the first bit of the unsigned and then right shifting diff --git a/test/ipa/libipa/fixedpoint.cpp b/test/ipa/libipa/fixedpoint.cpp index c88b53a31f28..6f6ef4ce258e 100644 --- a/test/ipa/libipa/fixedpoint.cpp +++ b/test/ipa/libipa/fixedpoint.cpp @@ -70,7 +70,7 @@ protected: * The second 7.992 test is to test that unused bits don't * affect the result. */ - std::map testCases = { + std::map testCases = { { 7.992, 0x3ff }, { 0.2, 0x01a }, { -0.2, 0x7e6 }, @@ -83,14 +83,14 @@ protected: int ret; for (const auto &testCase : testCases) { - ret = testSingleFixedPoint<4, 7, uint16_t>(testCase.first, + ret = testSingleFixedPoint<4, 7, int16_t>(testCase.first, testCase.second); if (ret != TestPass) return ret; } /* Special case with a superfluous one in the unused bits */ - ret = testFixedToFloat<4, 7, uint16_t, double>(0xbff, 7.992); + ret = testFixedToFloat<4, 7, int16_t, double>(0xbff, 7.992); if (ret != TestPass) return ret;