| Message ID | 20211102124657.1038011-1-naush@raspberrypi.com |
|---|---|
| State | Accepted |
| Commit | 06008c6e81e8d62c3ebf45025448a3137b76d394 |
| Headers | show |
| Series |
|
| Related | show |
Hi Naush Oops, yes we should have spotted that! Thanks! On Tue, 2 Nov 2021 at 12:47, Naushir Patuck <naush@raspberrypi.com> wrote: > The aspect ratio calculation divides two integer values then casts to a > double. > This might reduce precision when scoring for aspect rato differences. > > Fix this by casting the integer to a double before the division. > > Reported-by: Coverity CID=361652 > Signed-off-by: Naushir Patuck <naush@raspberrypi.com> > Reviewed-by: David Plowman <david.plowman@raspberrypi.com> Thanks David > --- > src/libcamera/pipeline/raspberrypi/raspberrypi.cpp | 2 +- > 1 file changed, 1 insertion(+), 1 deletion(-) > > diff --git a/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp > b/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp > index 44760093f7ab..5e1f227398a9 100644 > --- a/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp > +++ b/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp > @@ -141,7 +141,7 @@ V4L2SubdeviceFormat findBestFormat(const SensorFormats > &formatsMap, const Size & > > for (const Size &size : iter.second) { > double reqAr = static_cast<double>(req.width) / > req.height; > - double fmtAr = size.width / size.height; > + double fmtAr = static_cast<double>(size.width) / > size.height; > > /* Score the dimensions for closeness. */ > score = scoreFormat(req.width, size.width); > -- > 2.25.1 > >
Quoting Naushir Patuck (2021-11-02 12:46:57) > The aspect ratio calculation divides two integer values then casts to a double. > This might reduce precision when scoring for aspect rato differences. > > Fix this by casting the integer to a double before the division. > > Reported-by: Coverity CID=361652 > Signed-off-by: Naushir Patuck <naush@raspberrypi.com> Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com> > --- > src/libcamera/pipeline/raspberrypi/raspberrypi.cpp | 2 +- > 1 file changed, 1 insertion(+), 1 deletion(-) > > diff --git a/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp b/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp > index 44760093f7ab..5e1f227398a9 100644 > --- a/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp > +++ b/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp > @@ -141,7 +141,7 @@ V4L2SubdeviceFormat findBestFormat(const SensorFormats &formatsMap, const Size & > > for (const Size &size : iter.second) { > double reqAr = static_cast<double>(req.width) / req.height; > - double fmtAr = size.width / size.height; > + double fmtAr = static_cast<double>(size.width) / size.height; > > /* Score the dimensions for closeness. */ > score = scoreFormat(req.width, size.width); > -- > 2.25.1 >
diff --git a/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp b/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp index 44760093f7ab..5e1f227398a9 100644 --- a/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp +++ b/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp @@ -141,7 +141,7 @@ V4L2SubdeviceFormat findBestFormat(const SensorFormats &formatsMap, const Size & for (const Size &size : iter.second) { double reqAr = static_cast<double>(req.width) / req.height; - double fmtAr = size.width / size.height; + double fmtAr = static_cast<double>(size.width) / size.height; /* Score the dimensions for closeness. */ score = scoreFormat(req.width, size.width);
The aspect ratio calculation divides two integer values then casts to a double. This might reduce precision when scoring for aspect rato differences. Fix this by casting the integer to a double before the division. Reported-by: Coverity CID=361652 Signed-off-by: Naushir Patuck <naush@raspberrypi.com> --- src/libcamera/pipeline/raspberrypi/raspberrypi.cpp | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)