utils: raspberrypi: ctt: Fix integer division error calculating LSC cell size
diff mbox series

Message ID 20250507153103.253107-1-david.plowman@raspberrypi.com
State Accepted
Commit e0405b171e928651ed7be5d7ae47b426c07125ff
Headers show
Series
  • utils: raspberrypi: ctt: Fix integer division error calculating LSC cell size
Related show

Commit Message

David Plowman May 7, 2025, 3:31 p.m. UTC
The cell sizes must be cast to integers as the parameters that
were passed in may be floats.

Bug: https://github.com/raspberrypi/libcamera/issues/260
Signed-off-by: David Plowman <david.plowman@raspberrypi.com>
Fixes: 36ba0e551507 ("utils: raspberrypi: ctt: Fix NaNs in lens shading tables")
---
 utils/raspberrypi/ctt/ctt_alsc.py | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

Comments

Kieran Bingham May 7, 2025, 5:28 p.m. UTC | #1
Quoting David Plowman (2025-05-07 16:31:03)
> The cell sizes must be cast to integers as the parameters that
> were passed in may be floats.
> 
> Bug: https://github.com/raspberrypi/libcamera/issues/260
> Signed-off-by: David Plowman <david.plowman@raspberrypi.com>
> Fixes: 36ba0e551507 ("utils: raspberrypi: ctt: Fix NaNs in lens shading tables")

Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>

> ---
>  utils/raspberrypi/ctt/ctt_alsc.py | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/utils/raspberrypi/ctt/ctt_alsc.py b/utils/raspberrypi/ctt/ctt_alsc.py
> index f4fd09e3..5d8b2ced 100644
> --- a/utils/raspberrypi/ctt/ctt_alsc.py
> +++ b/utils/raspberrypi/ctt/ctt_alsc.py
> @@ -131,7 +131,7 @@ def alsc(Cam, Img, do_alsc_colour, plot=False, grid_size=(16, 12), max_gain=8.0)
>      pixels.
>      """
>      w, h = Img.w/2, Img.h/2
> -    dx, dy = (w - 1) // (grid_w - 1), (h - 1) // (grid_h - 1)
> +    dx, dy = int((w - 1) // (grid_w - 1)), int((h - 1) // (grid_h - 1))
>  
>      """
>      average the green channels into one
> -- 
> 2.34.1
>
Naushir Patuck May 8, 2025, 8:24 a.m. UTC | #2
Hi David,

Thanks for the fix.

On Wed, 7 May 2025 at 16:31, David Plowman
<david.plowman@raspberrypi.com> wrote:
>
> The cell sizes must be cast to integers as the parameters that
> were passed in may be floats.
>
> Bug: https://github.com/raspberrypi/libcamera/issues/260
> Signed-off-by: David Plowman <david.plowman@raspberrypi.com>
> Fixes: 36ba0e551507 ("utils: raspberrypi: ctt: Fix NaNs in lens shading tables")

Reviewed-by: Naushir Patuck <naush@raspberrypi.com>

> ---
>  utils/raspberrypi/ctt/ctt_alsc.py | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
>
> diff --git a/utils/raspberrypi/ctt/ctt_alsc.py b/utils/raspberrypi/ctt/ctt_alsc.py
> index f4fd09e3..5d8b2ced 100644
> --- a/utils/raspberrypi/ctt/ctt_alsc.py
> +++ b/utils/raspberrypi/ctt/ctt_alsc.py
> @@ -131,7 +131,7 @@ def alsc(Cam, Img, do_alsc_colour, plot=False, grid_size=(16, 12), max_gain=8.0)
>      pixels.
>      """
>      w, h = Img.w/2, Img.h/2
> -    dx, dy = (w - 1) // (grid_w - 1), (h - 1) // (grid_h - 1)
> +    dx, dy = int((w - 1) // (grid_w - 1)), int((h - 1) // (grid_h - 1))
>
>      """
>      average the green channels into one
> --
> 2.34.1
>

Patch
diff mbox series

diff --git a/utils/raspberrypi/ctt/ctt_alsc.py b/utils/raspberrypi/ctt/ctt_alsc.py
index f4fd09e3..5d8b2ced 100644
--- a/utils/raspberrypi/ctt/ctt_alsc.py
+++ b/utils/raspberrypi/ctt/ctt_alsc.py
@@ -131,7 +131,7 @@  def alsc(Cam, Img, do_alsc_colour, plot=False, grid_size=(16, 12), max_gain=8.0)
     pixels.
     """
     w, h = Img.w/2, Img.h/2
-    dx, dy = (w - 1) // (grid_w - 1), (h - 1) // (grid_h - 1)
+    dx, dy = int((w - 1) // (grid_w - 1)), int((h - 1) // (grid_h - 1))
 
     """
     average the green channels into one