[libcamera-devel,2/2] utils: rkisp1: gen-csc-table: Perform simple rounding for inverted CSC
diff mbox series

Message ID 20220929195318.13577-3-laurent.pinchart@ideasonboard.com
State Superseded
Headers show
Series
  • utils: rkisp1: gen-csc-table: Miscellaneous fixes
Related show

Commit Message

Laurent Pinchart Sept. 29, 2022, 7:53 p.m. UTC 
The CSC coefficients rounding keeps the sum of all coefficients in the
line identical. This was implemented for the RGB to YUV matrix
calculation, where the sum of luma coefficients is 1.0 and the sum of
chroma coefficients 0.0 (in full range), in order to avoid introducing
biases and overflow. This however doesn't make much sense for the YUV to
RGB components. Use simple rounding in that case.

Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
---
 utils/rkisp1/gen-csc-table.py | 5 ++++-
 1 file changed, 4 insertions(+), 1 deletion(-)

Patch
diff mbox series 

diff --git a/utils/rkisp1/gen-csc-table.py b/utils/rkisp1/gen-csc-table.py
index 934ace377e17..aca21ded2c8e 100755
--- a/utils/rkisp1/gen-csc-table.py
+++ b/utils/rkisp1/gen-csc-table.py
@@ -180,7 +180,10 @@  def main(argv):
     rounded_coeffs = []
     for line in scaled_coeffs:
         line = [coeff * (1 << precision.fractional) for coeff in line]
-        line = round_array(line)
+        if args.invert:
+            line = [round(coeff) for coeff in line]
+        else:
+            line = round_array(line)
 
         # Convert coefficients to the number of bits selected by the precision.
         # Negative values will be turned into positive integers using 2's