From patchwork Tue Nov 2 12:46:57 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Naushir Patuck X-Patchwork-Id: 14449 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 1F1F5BF415 for ; Tue, 2 Nov 2021 12:47:35 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 4C92660325; Tue, 2 Nov 2021 13:47:34 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="ea0lcWrz"; dkim-atps=neutral Received: from mail-wm1-x331.google.com (mail-wm1-x331.google.com [IPv6:2a00:1450:4864:20::331]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 5A009600B8 for ; Tue, 2 Nov 2021 13:47:33 +0100 (CET) Received: by mail-wm1-x331.google.com with SMTP id c71-20020a1c9a4a000000b0032cdcc8cbafso1992030wme.3 for ; Tue, 02 Nov 2021 05:47:33 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; h=from:to:cc:subject:date:message-id:mime-version :content-transfer-encoding; bh=SYUO1PYnmOPjZcfZJWZ0/h35oIBSqp+gSAIzj1MqJjE=; b=ea0lcWrzYwCi74w5e+oYCZQUl4iRJv3CGeioSFeGwc5ICuM92KHUB4jEIRv8geVYiZ NEF2hUH59sSLnQpfJyser5x5tABtpUBXFyikRQ7u2wNydyJ2VgSuT7xzBvWvtJyDuVEX mydYbqfwDBzKWNOwdWQytQvEhuupL6gJ+FUH78gcNYzsMO9fkbirkW1cgxEUT6jHF9VZ cYwa7RGpI/lhtQMgG/a2EqCIfOpogWPomVmwcIhlMJOCKFPZpI+Idu4d3n285td20UAe V0bCiJsgz1NNRGTPvx7ZLEi1AV3CI3O+3gE2u/X6RZUqFR8XU337Jpypx0DkShEJx7OX 6urw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:mime-version :content-transfer-encoding; bh=SYUO1PYnmOPjZcfZJWZ0/h35oIBSqp+gSAIzj1MqJjE=; b=gWYYDaFox+6CQraOoEVkt8hOfdIdtCBTGe1MwUxlVrMJNr9XS8EW0krOaVqEW38yNR 4XDxXyJejlvSBPld5/ZwNB4QpiqBCke+ASAicw+5FP/dMf9UiQWTB83GHuo9vM4131bM EECBGRI+RHSYzEcDkKOdrS3moR6QaqST7+B+lqa+czjUkXdE7y4pa0QeW+e9Uy3eG/yN 6Pm8IZPxRcXsGAJFHJ93FDOfm1l+3SUuz+RBBqFAoJoNmdm7SEUn+1CbznTewGOyDRLR iOMz4apN6ortwhzPhIKnS2rNE8mrAs5GWuCb0hQ6IOYmsTozb8GSq+ehjGhbTg1d2+V+ 9Srw== X-Gm-Message-State: AOAM5335SuDPtSdcZFqaxadzrp7LXuhlWWQKAJqD8TuDljf8DlwBU5Yz 2Z7z9fheP3aVcl3RCHq5xBvpFb3EF4Q+tpVR X-Google-Smtp-Source: ABdhPJynn+snuhJE+nyrwi6Oz+9Nb57bN3Z13W6DvzsVNaIbvp1oNTpqay5e7nr/Wti1EYWw6BLz9A== X-Received: by 2002:a05:600c:1d0e:: with SMTP id l14mr6656522wms.64.1635857252562; Tue, 02 Nov 2021 05:47:32 -0700 (PDT) Received: from naush-laptop.pitowers.org ([2a00:1098:3142:14:34fe:b3d0:b6e2:78dc]) by smtp.gmail.com with ESMTPSA id u10sm21495242wrs.5.2021.11.02.05.47.31 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 02 Nov 2021 05:47:31 -0700 (PDT) From: Naushir Patuck To: libcamera-devel@lists.libcamera.org Date: Tue, 2 Nov 2021 12:46:57 +0000 Message-Id: <20211102124657.1038011-1-naush@raspberrypi.com> X-Mailer: git-send-email 2.25.1 MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH] pipeline: raspberrypi: Fix rounding issue in findBestFormat() X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" The aspect ratio calculation divides two integer values then casts to a double. This might reduce precision when scoring for aspect rato differences. Fix this by casting the integer to a double before the division. Reported-by: Coverity CID=361652 Signed-off-by: Naushir Patuck Reviewed-by: David Plowman Reviewed-by: Kieran Bingham --- src/libcamera/pipeline/raspberrypi/raspberrypi.cpp | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp b/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp index 44760093f7ab..5e1f227398a9 100644 --- a/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp +++ b/src/libcamera/pipeline/raspberrypi/raspberrypi.cpp @@ -141,7 +141,7 @@ V4L2SubdeviceFormat findBestFormat(const SensorFormats &formatsMap, const Size & for (const Size &size : iter.second) { double reqAr = static_cast(req.width) / req.height; - double fmtAr = size.width / size.height; + double fmtAr = static_cast(size.width) / size.height; /* Score the dimensions for closeness. */ score = scoreFormat(req.width, size.width);