From patchwork Thu Feb 19 15:05:04 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kieran Bingham X-Patchwork-Id: 26199 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id D62FAC0DA4 for ; Thu, 19 Feb 2026 15:05:19 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 0E1FD62277; Thu, 19 Feb 2026 16:05:14 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="XlIzQ0zs"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 5359C62253 for ; Thu, 19 Feb 2026 16:05:09 +0100 (CET) Received: from ping.linuxembedded.co.uk (cpc89244-aztw30-2-0-cust6594.18-1.cable.virginm.net [86.31.185.195]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 0A55DFCC; Thu, 19 Feb 2026 16:04:16 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1771513456; bh=0w7k+nsaptyC4B22GJj31eSBRHS0E9X8RAxn0D4HPjg=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=XlIzQ0zsILsdyVrlY1Tpz+L4Ul3e2TMChCs+mcjmyMCtEpkFCokUeeBg2gd4xVoeg MKNHjdPvBueSJg7qNnfEwsrhTd42SM8PHayPaflzNiIGuAt8/Dq7Pm9oRGCYHM6PGE 6HVldg0a+lppjktBNTgH8OIEKi9zfn5lsdflxonw= From: Kieran Bingham Date: Thu, 19 Feb 2026 15:05:04 +0000 Subject: [PATCH v8 03/15] ipa: libipa: fixedpoint: Fix unsigned usage MIME-Version: 1.0 Message-Id: <20260219-kbingham-quantizers-v8-3-2b6ff68ead26@ideasonboard.com> References: <20260219-kbingham-quantizers-v8-0-2b6ff68ead26@ideasonboard.com> In-Reply-To: <20260219-kbingham-quantizers-v8-0-2b6ff68ead26@ideasonboard.com> To: libcamera-devel@lists.libcamera.org Cc: Kieran Bingham , Isaac Scott , Paul Elder , Laurent Pinchart X-Mailer: b4 0.14.3 X-Developer-Signature: v=1; a=ed25519-sha256; t=1771513507; l=2457; i=kieran.bingham@ideasonboard.com; s=20260207; h=from:subject:message-id; bh=0w7k+nsaptyC4B22GJj31eSBRHS0E9X8RAxn0D4HPjg=; b=vfIN6Mwmybo7FoTLJ6FwlWO40OhB/O6U4EiPFYj9tpAcjc4U2hwZ++SMh9yr4qIg6/Q4YeW3m nnxxHqQVblqDf+xPNZ94vaAUYB1x4KR0fVeE/UBDh/kP5t4fBTaa+Gp X-Developer-Key: i=kieran.bingham@ideasonboard.com; a=ed25519; pk=FVXKN7YuwHc6UtbRUeTMAmranfsQomA+vnilfglWdaY= X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" The fixedToFloatingPoint does not support unsigned Q types, and incorrectly sign-extends all values which have the top most bit set in the quantized values. Fix this by ensuring that only signed types perform sign extension, and simplify the calculation for unsigned types. Convert the storage of the test cases to signed types to correctly represent their intended purpose, to prevent test failures. Reviewed-by: Isaac Scott Reviewed-by: Paul Elder Reviewed-by: Laurent Pinchart Signed-off-by: Kieran Bingham --- src/ipa/libipa/fixedpoint.h | 3 +++ test/ipa/libipa/fixedpoint.cpp | 6 +++--- 2 files changed, 6 insertions(+), 3 deletions(-) diff --git a/src/ipa/libipa/fixedpoint.h b/src/ipa/libipa/fixedpoint.h index 709cf50f0fcd..aeb9bce3269b 100644 --- a/src/ipa/libipa/fixedpoint.h +++ b/src/ipa/libipa/fixedpoint.h @@ -49,6 +49,9 @@ constexpr R fixedToFloatingPoint(T number) static_assert(sizeof(int) >= sizeof(T)); static_assert(I + F <= sizeof(T) * 8); + if constexpr (std::is_unsigned_v) + return static_cast(number) / static_cast(T{ 1 } << F); + /* * Recreate the upper bits in case of a negative number by shifting the sign * bit from the fixed point to the first bit of the unsigned and then right shifting diff --git a/test/ipa/libipa/fixedpoint.cpp b/test/ipa/libipa/fixedpoint.cpp index 99eb662ddf4e..4b017e86a74f 100644 --- a/test/ipa/libipa/fixedpoint.cpp +++ b/test/ipa/libipa/fixedpoint.cpp @@ -68,7 +68,7 @@ protected: * The second 7.992 test is to test that unused bits don't * affect the result. */ - std::map testCases = { + std::map testCases = { { 7.992, 0x3ff }, { 0.2, 0x01a }, { -0.2, 0x7e6 }, @@ -81,14 +81,14 @@ protected: int ret; for (const auto &testCase : testCases) { - ret = testSingleFixedPoint<4, 7, uint16_t>(testCase.first, + ret = testSingleFixedPoint<4, 7, int16_t>(testCase.first, testCase.second); if (ret != TestPass) return ret; } /* Special case with a superfluous one in the unused bits */ - ret = testFixedToFloat<4, 7, uint16_t, double>(0xbff, 7.992); + ret = testFixedToFloat<4, 7, int16_t, double>(0xbff, 7.992); if (ret != TestPass) return ret;