From patchwork Thu Sep 29 19:53:17 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Laurent Pinchart X-Patchwork-Id: 17469 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 02D7DC327E for ; Thu, 29 Sep 2022 19:53:24 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 920F462397; Thu, 29 Sep 2022 21:53:24 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1664481204; bh=0Nx9nM4oNxTHghZxzwhc8DUbEgkHMrD7kCT65jlrVyk=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To: From; b=rh/DDtPkp+9uGMv0VMVm90xGKr1Uj/ZhRe+uyfq4KQMA1YQWraY3V1bCEbD1p587d LouLvxAcoD9rtbBeM681h+DK8fwPPCcEknvadYqRPvDM18O4Cg++YcKKcJCQLyFLtB /td8bGEo8vtKRoTqrWIBf4F1KEW4UKfvn1X58DtJmczDNL8hFV95UbM9n1PmM9iHbc FHdan23Hj2R/luYvTQOIPFXooqvB7xGWvqFimLlSCJFefqckZtmCSOlpwMiX970h0c rIXSFRYSOh7TXgle7RsWmwJcMQoobgjoafi9TSFDToCbUGao108tuwPBtbTwS/ZVaT mzhLuMliCWK2w== Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 78CFB62384 for ; Thu, 29 Sep 2022 21:53:22 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="UP+r1Iqt"; dkim-atps=neutral Received: from pendragon.ideasonboard.com (62-78-145-57.bb.dnainternet.fi [62.78.145.57]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 054EE47C for ; Thu, 29 Sep 2022 21:53:21 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1664481202; bh=0Nx9nM4oNxTHghZxzwhc8DUbEgkHMrD7kCT65jlrVyk=; h=From:To:Subject:Date:In-Reply-To:References:From; b=UP+r1IqtLz8fhROYFPF7+GOu43OHmU51+oarVla+8EhFyASE6P3JULVlWUlKvU5/I uY+q5QNY+eIa4H/+WY7RlS2obPChpciW6sVgwpI+oAdDIRX8GeBJ8GdBwxi3tALs0m c/YKrm1duHcbQLIiNwywBpXERmpNTvEjYU11vwuk= To: libcamera-devel@lists.libcamera.org Date: Thu, 29 Sep 2022 22:53:17 +0300 Message-Id: <20220929195318.13577-2-laurent.pinchart@ideasonboard.com> X-Mailer: git-send-email 2.35.1 In-Reply-To: <20220929195318.13577-1-laurent.pinchart@ideasonboard.com> References: <20220929195318.13577-1-laurent.pinchart@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH 1/2] utils: rkisp1: gen-csc-table: Fix inverted CSC calculation X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Laurent Pinchart via libcamera-devel From: Laurent Pinchart Reply-To: Laurent Pinchart Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Coefficient scaling in the inverted CSC table is wrong, as the luma-related coefficients are not in the first row, but in the first column. The simplest way to fix it is to remove the inversion handling from scale_coeff() and invert the matrix after scaling the coefficients. The multiplication by the fixed point precision must still happen after inversion, and must thus be moved out of scale_coeff(). Fixes: b5c4b6e0339a ("utils: rkisp1: gen-csc-table: Add support for inverting the CSC") Signed-off-by: Laurent Pinchart --- utils/rkisp1/gen-csc-table.py | 22 ++++++++-------------- 1 file changed, 8 insertions(+), 14 deletions(-) diff --git a/utils/rkisp1/gen-csc-table.py b/utils/rkisp1/gen-csc-table.py index 83aea357966a..934ace377e17 100755 --- a/utils/rkisp1/gen-csc-table.py +++ b/utils/rkisp1/gen-csc-table.py @@ -64,9 +64,8 @@ class Quantization(enum.Enum): LIMITED = 1 -def scale_coeff(coeff, quantization, luma, precision, invert): - """Scale a coefficient to the output range dictated by the quantization and - the precision. +def scale_coeff(coeff, quantization, luma): + """Scale a coefficient to the output range dictated by the quantization. Parameters ---------- @@ -76,9 +75,6 @@ def scale_coeff(coeff, quantization, luma, precision, invert): The quantization, either FULL or LIMITED luma : bool True if the coefficient corresponds to a luma value, False otherwise - precision : int - The desired precision for the scaled coefficient as a number of - fractional bits """ # Assume the input range is 8 bits. The output range is set by the @@ -92,10 +88,7 @@ def scale_coeff(coeff, quantization, luma, precision, invert): else: out_range = 240 - 16 - if invert: - in_range, out_range = out_range, in_range - - return coeff * out_range / in_range * (1 << precision) + return coeff * out_range / in_range def round_array(values): @@ -172,20 +165,21 @@ def main(argv): encoding = encodings[args.encoding] quantization = Quantization[args.quantization.upper()] - if args.invert: - encoding = np.linalg.inv(encoding) - # Scale and round the encoding coefficients based on the precision and # quantization range. luma = True scaled_coeffs = [] for line in encoding: - line = [scale_coeff(coeff, quantization, luma, precision.fractional, args.invert) for coeff in line] + line = [scale_coeff(coeff, quantization, luma) for coeff in line] scaled_coeffs.append(line) luma = False + if args.invert: + scaled_coeffs = np.linalg.inv(scaled_coeffs) + rounded_coeffs = [] for line in scaled_coeffs: + line = [coeff * (1 << precision.fractional) for coeff in line] line = round_array(line) # Convert coefficients to the number of bits selected by the precision. From patchwork Thu Sep 29 19:53:18 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Laurent Pinchart X-Patchwork-Id: 17470 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 9DFA8BD16B for ; Thu, 29 Sep 2022 19:53:27 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 6576262392; Thu, 29 Sep 2022 21:53:27 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1664481207; bh=WnZ5HZVnMFyjAWNLklRCkQW6mdS/F6+N78aMDdoMO84=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To: From; b=WDQzcwbA0NHgwmOPjNbDWDDBnK5b0cAyvyJNjTPCBiZ+yT/d3FMx5pOlrduF7n8tT QevWIlYZQ/+PgUf528/ZrzTd3I4WSLTyYuH+5qVQ9uoTea0RV67m3LL+2oD64PYVfc jLhV6npOMb9tWopDX/pgiJkOZLos6PtS1lgBdcY7MZiENQHTZdsn0o9cwW4Ooivts1 eRtXeyoFKqCiafiSuEVOa6jKcycTHm8hvI7qnU1HH0MfSVChfMOSBMUi2v0rmEt5j8 BSBAXb+2agONbWKPdZTaim/cwHiTHjIiXzgHnUfUmHqVI2Nm6UXReVjIJn5Y870XVQ DH2JLAb0hXhVQ== Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id BDB8262384 for ; Thu, 29 Sep 2022 21:53:23 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="fGtzfIAF"; dkim-atps=neutral Received: from pendragon.ideasonboard.com (62-78-145-57.bb.dnainternet.fi [62.78.145.57]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 4FB1847C for ; Thu, 29 Sep 2022 21:53:23 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1664481203; bh=WnZ5HZVnMFyjAWNLklRCkQW6mdS/F6+N78aMDdoMO84=; h=From:To:Subject:Date:In-Reply-To:References:From; b=fGtzfIAFh+bqRQ6pnT71SCjMjc1wLKm6krUby92sDHyTG5aOwCJXIA4DSBTtHZmfh X4hB0+OPO8JMzJlP5nW0MkMX+4fLQ9eE1JMUkX6WsN+i0a0oHqcF05YMzSI26T7xkP Rw40W1VQ18dqZcjzPT8IxFqxURiY59gPzHZ/lGow= To: libcamera-devel@lists.libcamera.org Date: Thu, 29 Sep 2022 22:53:18 +0300 Message-Id: <20220929195318.13577-3-laurent.pinchart@ideasonboard.com> X-Mailer: git-send-email 2.35.1 In-Reply-To: <20220929195318.13577-1-laurent.pinchart@ideasonboard.com> References: <20220929195318.13577-1-laurent.pinchart@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH 2/2] utils: rkisp1: gen-csc-table: Perform simple rounding for inverted CSC X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Laurent Pinchart via libcamera-devel From: Laurent Pinchart Reply-To: Laurent Pinchart Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" The CSC coefficients rounding keeps the sum of all coefficients in the line identical. This was implemented for the RGB to YUV matrix calculation, where the sum of luma coefficients is 1.0 and the sum of chroma coefficients 0.0 (in full range), in order to avoid introducing biases and overflow. This however doesn't make much sense for the YUV to RGB components. Use simple rounding in that case. Signed-off-by: Laurent Pinchart --- utils/rkisp1/gen-csc-table.py | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/utils/rkisp1/gen-csc-table.py b/utils/rkisp1/gen-csc-table.py index 934ace377e17..aca21ded2c8e 100755 --- a/utils/rkisp1/gen-csc-table.py +++ b/utils/rkisp1/gen-csc-table.py @@ -180,7 +180,10 @@ def main(argv): rounded_coeffs = [] for line in scaled_coeffs: line = [coeff * (1 << precision.fractional) for coeff in line] - line = round_array(line) + if args.invert: + line = [round(coeff) for coeff in line] + else: + line = round_array(line) # Convert coefficients to the number of bits selected by the precision. # Negative values will be turned into positive integers using 2's