From patchwork Wed Jan 14 17:39:03 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kieran Bingham X-Patchwork-Id: 25799 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 61106C3274 for ; Wed, 14 Jan 2026 17:39:34 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id B932F61FD8; Wed, 14 Jan 2026 18:39:27 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="FHECP9il"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 5AFE461FBC for ; Wed, 14 Jan 2026 18:39:23 +0100 (CET) Received: from Monstersaurus.infra.iob (cpc89244-aztw30-2-0-cust6594.18-1.cable.virginm.net [86.31.185.195]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 679961E43; Wed, 14 Jan 2026 18:38:56 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1768412336; bh=Oz5+iydAOiXwieOaY0iM+ms3+pAUlSkp5TaVa1wjFpA=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=FHECP9ilrdtlnf0CX8DcnxiemUjGB+XiXwFcXk9LdZg0SJqBnQh7y+epxfQLKVmho gIwmqsWIjhcu+mmqas/3FUUeoI0Nkj2+ln5sXnEytpS+/nZZyJw7xds2ZC72ZbgnOb TV/W7YHp2SQIhsAjJ54oGOblkloELYJ22zeBbVB4= From: Kieran Bingham To: libcamera devel Cc: Kieran Bingham , Isaac Scott Subject: [PATCH v5 03/16] ipa: libipa: fixedpoint: Fix unsigned usage Date: Wed, 14 Jan 2026 17:39:03 +0000 Message-ID: <20260114173918.1744023-4-kieran.bingham@ideasonboard.com> X-Mailer: git-send-email 2.52.0 In-Reply-To: <20260114173918.1744023-1-kieran.bingham@ideasonboard.com> References: <20260114173918.1744023-1-kieran.bingham@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" The fixedToFloatingPoint does not support unsigned Q types, and incorrectly sign-extends all values which have the top most bit set in the quantized values. Fix this by ensuring that only signed types perform sign extension, and simplify the calculation for unsigned types. Convert the storage of the test cases to signed types to correctly represent their intended purpose, to prevent test failures. Reviewed-by: Isaac Scott Signed-off-by: Kieran Bingham --- src/ipa/libipa/fixedpoint.h | 3 +++ test/ipa/libipa/fixedpoint.cpp | 6 +++--- 2 files changed, 6 insertions(+), 3 deletions(-) diff --git a/src/ipa/libipa/fixedpoint.h b/src/ipa/libipa/fixedpoint.h index 709cf50f0fcd..b4a7fa5e0ecd 100644 --- a/src/ipa/libipa/fixedpoint.h +++ b/src/ipa/libipa/fixedpoint.h @@ -49,6 +49,9 @@ constexpr R fixedToFloatingPoint(T number) static_assert(sizeof(int) >= sizeof(T)); static_assert(I + F <= sizeof(T) * 8); + if constexpr (std::is_unsigned_v) + return static_cast(number) / static_cast(1 << F); + /* * Recreate the upper bits in case of a negative number by shifting the sign * bit from the fixed point to the first bit of the unsigned and then right shifting diff --git a/test/ipa/libipa/fixedpoint.cpp b/test/ipa/libipa/fixedpoint.cpp index 99eb662ddf4e..4b017e86a74f 100644 --- a/test/ipa/libipa/fixedpoint.cpp +++ b/test/ipa/libipa/fixedpoint.cpp @@ -68,7 +68,7 @@ protected: * The second 7.992 test is to test that unused bits don't * affect the result. */ - std::map testCases = { + std::map testCases = { { 7.992, 0x3ff }, { 0.2, 0x01a }, { -0.2, 0x7e6 }, @@ -81,14 +81,14 @@ protected: int ret; for (const auto &testCase : testCases) { - ret = testSingleFixedPoint<4, 7, uint16_t>(testCase.first, + ret = testSingleFixedPoint<4, 7, int16_t>(testCase.first, testCase.second); if (ret != TestPass) return ret; } /* Special case with a superfluous one in the unused bits */ - ret = testFixedToFloat<4, 7, uint16_t, double>(0xbff, 7.992); + ret = testFixedToFloat<4, 7, int16_t, double>(0xbff, 7.992); if (ret != TestPass) return ret;