[libcamera-devel] utils: rkisp1: gen-csc-table: Specify coefficients with full precision

17220 diff mbox series

The Rec01, Rec709, Rec2020 and SMTPE 240M standards specify the Cb and
Cr coefficients of the RGB to Y'CbCr conversion matrix using a quotient
of two values. Use the exact same formulas instead of hardcoding the
division results with a lower precision.

Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
---
 utils/rkisp1/gen-csc-table.py | 24 ++++++++++++------------
 1 file changed, 12 insertions(+), 12 deletions(-)


base-commit: 3fad116f89e0d3497567043cbf6d8c49f1c102db

Hi Laurent,

Reviewed-by: Florian Sylvestre <fsylvestre@baylibre.com>

On Mon, 29 Aug 2022 at 10:47, Laurent Pinchart
<laurent.pinchart@ideasonboard.com> wrote:
>
> The Rec01, Rec709, Rec2020 and SMTPE 240M standards specify the Cb and
> Cr coefficients of the RGB to Y'CbCr conversion matrix using a quotient
> of two values. Use the exact same formulas instead of hardcoding the
> division results with a lower precision.
>
> Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
> ---
>  utils/rkisp1/gen-csc-table.py | 24 ++++++++++++------------
>  1 file changed, 12 insertions(+), 12 deletions(-)
>
> diff --git a/utils/rkisp1/gen-csc-table.py b/utils/rkisp1/gen-csc-table.py
> index ca0187ec24c6..2fb702746421 100755
> --- a/utils/rkisp1/gen-csc-table.py
> +++ b/utils/rkisp1/gen-csc-table.py
> @@ -12,24 +12,24 @@ import sys
>
>  encodings = {
>      'rec601': [
> -        [  0.2990,  0.5870,  0.1140 ],
> -        [ -0.1687, -0.3313,  0.5    ],
> -        [  0.5,    -0.4187, -0.0813 ]
> +        [  0.299,          0.587,          0.114         ],
> +        [ -0.299 / 1.772, -0.587 / 1.772,  0.886 / 1.772 ],
> +        [  0.701 / 1.402, -0.587 / 1.402, -0.114 / 1.402 ]
>      ],
>      'rec709': [
> -        [  0.2126,  0.7152,  0.0722 ],
> -        [ -0.1146, -0.3854,  0.5    ],
> -        [  0.5,    -0.4542, -0.0458 ]
> +        [  0.2126,           0.7152,           0.0722          ],
> +        [ -0.2126 / 1.8556, -0.7152 / 1.8556,  0.9278 / 1.8556 ],
> +        [  0.7874 / 1.5748, -0.7152 / 1.5748, -0.0722 / 1.5748 ]
>      ],
>      'rec2020': [
> -        [  0.2627,  0.6780,  0.0593 ],
> -        [ -0.1396, -0.3604,  0.5    ],
> -        [  0.5,    -0.4598, -0.0402 ]
> +        [  0.2627,           0.6780,           0.0593          ],
> +        [ -0.2627 / 1.8814, -0.6780 / 1.8814,  0.9407 / 1.8814 ],
> +        [  0.7373 / 1.4746, -0.6780 / 1.4746, -0.0593 / 1.4746 ],
>      ],
>      'smpte240m': [
> -        [  0.2122,  0.7013,  0.0865 ],
> -        [ -0.1161, -0.3839,  0.5    ],
> -        [  0.5,    -0.4451, -0.0549 ]
> +        [  0.2122,           0.7013,           0.0865          ],
> +        [ -0.2122 / 1.8270, -0.7013 / 1.8270,  0.9135 / 1.8270 ],
> +        [  0.7878 / 1.5756, -0.7013 / 1.5756, -0.0865 / 1.5756 ],
>      ],
>  }
>
>
> base-commit: 3fad116f89e0d3497567043cbf6d8c49f1c102db
> --
> Regards,
>
> Laurent Pinchart
>

Hi Laurent,

Thank you for the patch

On 8/29/22 2:17 PM, Laurent Pinchart via libcamera-devel wrote:
> The Rec01, Rec709, Rec2020 and SMTPE 240M standards specify the Cb and
> Cr coefficients of the RGB to Y'CbCr conversion matrix using a quotient
> of two values. Use the exact same formulas instead of hardcoding the
> division results with a lower precision.
>
> Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>

Reviewed-by: Umang Jain <umang.jain@ideasonboard.com>
> ---
>   utils/rkisp1/gen-csc-table.py | 24 ++++++++++++------------
>   1 file changed, 12 insertions(+), 12 deletions(-)
>
> diff --git a/utils/rkisp1/gen-csc-table.py b/utils/rkisp1/gen-csc-table.py
> index ca0187ec24c6..2fb702746421 100755
> --- a/utils/rkisp1/gen-csc-table.py
> +++ b/utils/rkisp1/gen-csc-table.py
> @@ -12,24 +12,24 @@ import sys
>   
>   encodings = {
>       'rec601': [
> -        [  0.2990,  0.5870,  0.1140 ],
> -        [ -0.1687, -0.3313,  0.5    ],
> -        [  0.5,    -0.4187, -0.0813 ]
> +        [  0.299,          0.587,          0.114         ],
> +        [ -0.299 / 1.772, -0.587 / 1.772,  0.886 / 1.772 ],
> +        [  0.701 / 1.402, -0.587 / 1.402, -0.114 / 1.402 ]
>       ],
>       'rec709': [
> -        [  0.2126,  0.7152,  0.0722 ],
> -        [ -0.1146, -0.3854,  0.5    ],
> -        [  0.5,    -0.4542, -0.0458 ]
> +        [  0.2126,           0.7152,           0.0722          ],
> +        [ -0.2126 / 1.8556, -0.7152 / 1.8556,  0.9278 / 1.8556 ],
> +        [  0.7874 / 1.5748, -0.7152 / 1.5748, -0.0722 / 1.5748 ]
>       ],
>       'rec2020': [
> -        [  0.2627,  0.6780,  0.0593 ],
> -        [ -0.1396, -0.3604,  0.5    ],
> -        [  0.5,    -0.4598, -0.0402 ]
> +        [  0.2627,           0.6780,           0.0593          ],
> +        [ -0.2627 / 1.8814, -0.6780 / 1.8814,  0.9407 / 1.8814 ],
> +        [  0.7373 / 1.4746, -0.6780 / 1.4746, -0.0593 / 1.4746 ],
>       ],
>       'smpte240m': [
> -        [  0.2122,  0.7013,  0.0865 ],
> -        [ -0.1161, -0.3839,  0.5    ],
> -        [  0.5,    -0.4451, -0.0549 ]
> +        [  0.2122,           0.7013,           0.0865          ],
> +        [ -0.2122 / 1.8270, -0.7013 / 1.8270,  0.9135 / 1.8270 ],
> +        [  0.7878 / 1.5756, -0.7013 / 1.5756, -0.0865 / 1.5756 ],
>       ],
>   }
>   
>
> base-commit: 3fad116f89e0d3497567043cbf6d8c49f1c102db