{"id":17469,"url":"https://patchwork.libcamera.org/api/patches/17469/?format=json","web_url":"https://patchwork.libcamera.org/patch/17469/","project":{"id":1,"url":"https://patchwork.libcamera.org/api/projects/1/?format=json","name":"libcamera","link_name":"libcamera","list_id":"libcamera_core","list_email":"libcamera-devel@lists.libcamera.org","web_url":"","scm_url":"","webscm_url":""},"msgid":"<20220929195318.13577-2-laurent.pinchart@ideasonboard.com>","date":"2022-09-29T19:53:17","name":"[libcamera-devel,1/2] utils: rkisp1: gen-csc-table: Fix inverted CSC calculation","commit_ref":null,"pull_url":null,"state":"superseded","archived":false,"hash":"7f2fa858730967661a2a8de94681fca022dca316","submitter":{"id":2,"url":"https://patchwork.libcamera.org/api/people/2/?format=json","name":"Laurent Pinchart","email":"laurent.pinchart@ideasonboard.com"},"delegate":null,"mbox":"https://patchwork.libcamera.org/patch/17469/mbox/","series":[{"id":3515,"url":"https://patchwork.libcamera.org/api/series/3515/?format=json","web_url":"https://patchwork.libcamera.org/project/libcamera/list/?series=3515","date":"2022-09-29T19:53:16","name":"utils: rkisp1: gen-csc-table: Miscellaneous fixes","version":1,"mbox":"https://patchwork.libcamera.org/series/3515/mbox/"}],"comments":"https://patchwork.libcamera.org/api/patches/17469/comments/","check":"pending","checks":"https://patchwork.libcamera.org/api/patches/17469/checks/","tags":{},"headers":{"Return-Path":"","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id 02D7DC327E\n\tfor ;\n\tThu, 29 Sep 2022 19:53:24 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id 920F462397;\n\tThu, 29 Sep 2022 21:53:24 +0200 (CEST)","from perceval.ideasonboard.com (perceval.ideasonboard.com\n\t[IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id 78CFB62384\n\tfor ;\n\tThu, 29 Sep 2022 21:53:22 +0200 (CEST)","from pendragon.ideasonboard.com (62-78-145-57.bb.dnainternet.fi\n\t[62.78.145.57])\n\tby perceval.ideasonboard.com (Postfix) with ESMTPSA id 054EE47C\n\tfor ;\n\tThu, 29 Sep 2022 21:53:21 +0200 (CEST)"],"DKIM-Signature":["v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org;\n\ts=mail; t=1664481204;\n\tbh=0Nx9nM4oNxTHghZxzwhc8DUbEgkHMrD7kCT65jlrVyk=;\n\th=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe:\n\tList-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To:\n\tFrom;\n\tb=rh/DDtPkp+9uGMv0VMVm90xGKr1Uj/ZhRe+uyfq4KQMA1YQWraY3V1bCEbD1p587d\n\tLouLvxAcoD9rtbBeM681h+DK8fwPPCcEknvadYqRPvDM18O4Cg++YcKKcJCQLyFLtB\n\t/td8bGEo8vtKRoTqrWIBf4F1KEW4UKfvn1X58DtJmczDNL8hFV95UbM9n1PmM9iHbc\n\tFHdan23Hj2R/luYvTQOIPFXooqvB7xGWvqFimLlSCJFefqckZtmCSOlpwMiX970h0c\n\trIXSFRYSOh7TXgle7RsWmwJcMQoobgjoafi9TSFDToCbUGao108tuwPBtbTwS/ZVaT\n\tmzhLuMliCWK2w==","v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com;\n\ts=mail; t=1664481202;\n\tbh=0Nx9nM4oNxTHghZxzwhc8DUbEgkHMrD7kCT65jlrVyk=;\n\th=From:To:Subject:Date:In-Reply-To:References:From;\n\tb=UP+r1IqtLz8fhROYFPF7+GOu43OHmU51+oarVla+8EhFyASE6P3JULVlWUlKvU5/I\n\tuY+q5QNY+eIa4H/+WY7RlS2obPChpciW6sVgwpI+oAdDIRX8GeBJ8GdBwxi3tALs0m\n\tc/YKrm1duHcbQLIiNwywBpXERmpNTvEjYU11vwuk="],"Authentication-Results":"lancelot.ideasonboard.com; dkim=pass (1024-bit key; \n\tunprotected) header.d=ideasonboard.com\n\theader.i=@ideasonboard.com\n\theader.b=\"UP+r1Iqt\"; dkim-atps=neutral","To":"libcamera-devel@lists.libcamera.org","Date":"Thu, 29 Sep 2022 22:53:17 +0300","Message-Id":"<20220929195318.13577-2-laurent.pinchart@ideasonboard.com>","X-Mailer":"git-send-email 2.35.1","In-Reply-To":"<20220929195318.13577-1-laurent.pinchart@ideasonboard.com>","References":"<20220929195318.13577-1-laurent.pinchart@ideasonboard.com>","MIME-Version":"1.0","Content-Transfer-Encoding":"8bit","Subject":"[libcamera-devel] [PATCH 1/2] utils: rkisp1: gen-csc-table: Fix\n\tinverted CSC calculation","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","From":"Laurent Pinchart via libcamera-devel\n\t","Reply-To":"Laurent Pinchart ","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" "},"content":"Coefficient scaling in the inverted CSC table is wrong, as the\nluma-related coefficients are not in the first row, but in the first\ncolumn. The simplest way to fix it is to remove the inversion handling\nfrom scale_coeff() and invert the matrix after scaling the coefficients.\nThe multiplication by the fixed point precision must still happen after\ninversion, and must thus be moved out of scale_coeff().\n\nFixes: b5c4b6e0339a (\"utils: rkisp1: gen-csc-table: Add support for inverting the CSC\")\nSigned-off-by: Laurent Pinchart \n---\n utils/rkisp1/gen-csc-table.py | 22 ++++++++--------------\n 1 file changed, 8 insertions(+), 14 deletions(-)","diff":"diff --git a/utils/rkisp1/gen-csc-table.py b/utils/rkisp1/gen-csc-table.py\nindex 83aea357966a..934ace377e17 100755\n--- a/utils/rkisp1/gen-csc-table.py\n+++ b/utils/rkisp1/gen-csc-table.py\n@@ -64,9 +64,8 @@ class Quantization(enum.Enum):\n LIMITED = 1\n \n \n-def scale_coeff(coeff, quantization, luma, precision, invert):\n- \"\"\"Scale a coefficient to the output range dictated by the quantization and\n- the precision.\n+def scale_coeff(coeff, quantization, luma):\n+ \"\"\"Scale a coefficient to the output range dictated by the quantization.\n \n Parameters\n ----------\n@@ -76,9 +75,6 @@ def scale_coeff(coeff, quantization, luma, precision, invert):\n The quantization, either FULL or LIMITED\n luma : bool\n True if the coefficient corresponds to a luma value, False otherwise\n- precision : int\n- The desired precision for the scaled coefficient as a number of\n- fractional bits\n \"\"\"\n \n # Assume the input range is 8 bits. The output range is set by the\n@@ -92,10 +88,7 @@ def scale_coeff(coeff, quantization, luma, precision, invert):\n else:\n out_range = 240 - 16\n \n- if invert:\n- in_range, out_range = out_range, in_range\n-\n- return coeff * out_range / in_range * (1 << precision)\n+ return coeff * out_range / in_range\n \n \n def round_array(values):\n@@ -172,20 +165,21 @@ def main(argv):\n encoding = encodings[args.encoding]\n quantization = Quantization[args.quantization.upper()]\n \n- if args.invert:\n- encoding = np.linalg.inv(encoding)\n-\n # Scale and round the encoding coefficients based on the precision and\n # quantization range.\n luma = True\n scaled_coeffs = []\n for line in encoding:\n- line = [scale_coeff(coeff, quantization, luma, precision.fractional, args.invert) for coeff in line]\n+ line = [scale_coeff(coeff, quantization, luma) for coeff in line]\n scaled_coeffs.append(line)\n luma = False\n \n+ if args.invert:\n+ scaled_coeffs = np.linalg.inv(scaled_coeffs)\n+\n rounded_coeffs = []\n for line in scaled_coeffs:\n+ line = [coeff * (1 << precision.fractional) for coeff in line]\n line = round_array(line)\n \n # Convert coefficients to the number of bits selected by the precision.\n","prefixes":["libcamera-devel","1/2"]}