From patchwork Mon Mar 13 07:28:13 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tomi Valkeinen X-Patchwork-Id: 18382 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 9DBB5BE080 for ; Mon, 13 Mar 2023 07:28:31 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id D32B762709; Mon, 13 Mar 2023 08:28:30 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1678692510; bh=aX7LJO80R3TYxYm9J7A2g+bXXr7TNP1FpDI9yH4BQhk=; h=To:Date:Subject:List-Id:List-Unsubscribe:List-Archive:List-Post: List-Help:List-Subscribe:From:Reply-To:From; b=B2Gk5Ty2amqQK/+s74EstNgrV+j/mTDsBRhec0EjqYjmayWfCO3IFviKzcSprQ+z0 bWSSbpQbIUTMB+xmwnMaDZsC70TWHcGwsNi+wrGZpEDcaMfSIhGa0CNJIxCUS2QW0R RKtH1Prc8lD99vQyOFKCtgUefpG5TkB5HOzAma0fH38iwOvZnF0XGxwRQoq1Bp6UDO T4lw5OKyRIRJSiwMD51aywGy1lOhzDlW5A7VWjhp8TC6wp9gSGY4NfHYLPbzlrKgw4 TyQt4YRM6IMwkXJCixo7ldA7yS78OsL75uTgYHNiFXaILNRXB+4kB9/XBsxzbQpUWq GCP6YqOj1KWJQ== Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id E9DE461ED5 for ; Mon, 13 Mar 2023 08:28:29 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="sGz7Kmh0"; dkim-atps=neutral Received: from desky.lan (91-154-32-225.elisa-laajakaista.fi [91.154.32.225]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 57411563; Mon, 13 Mar 2023 08:28:29 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1678692509; bh=aX7LJO80R3TYxYm9J7A2g+bXXr7TNP1FpDI9yH4BQhk=; h=From:To:Cc:Subject:Date:From; b=sGz7Kmh0IMUEQSzkYS+u5Upc5cMxKCbARqmiyVS1CJ59OHuEVsTlqBm1xiGk7ITnM 4nmAP1lkcfHQGE/5avJFj7Pm8CTWYUariiqGSxyyyu4c/MEjgoND5YrGCJXY7c7TPh RyVrnUqxa/Ed+nteI8Tc8870x2EbrIMKlgalUkXo= To: libcamera-devel@lists.libcamera.org Date: Mon, 13 Mar 2023 09:28:13 +0200 Message-Id: <20230313072813.23430-1-tomi.valkeinen@ideasonboard.com> X-Mailer: git-send-email 2.34.1 MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH] py: cam: Fix demosaic overflow issue X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Tomi Valkeinen via libcamera-devel From: Tomi Valkeinen Reply-To: Tomi Valkeinen Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" The demosaic code first expands the buffer datatype to uint16, and then shifts the data left so that the 8, 10 and 12 bitspp formats all become 16 bitspp. It then, eventually, uses np.einsum to calculate averages, but this averaging sums multiple uint16 values together, and stores them in uint16 storage. As in the first step we shifted the values left, possibly getting values close to the maximum of uint16 range, we, of course, overflow when summing them together. This leads to rather bad looking images. Fix this by dropping the original shift. It serves no purpose, and is probably a remnant of some early testing code. This way the largest numbers we are summing together are 12 bit values, and as we use a 3x3 window from which we fetch values, for a single rgb plane, the max number of 12 bit values is 5 (for green). Sum of 5 12 bit values is well below the 16 bit maximum. Signed-off-by: Tomi Valkeinen Reviewed-by: Laurent Pinchart Reviewed-by: Kieran Bingham --- src/py/cam/helpers.py | 6 ++---- 1 file changed, 2 insertions(+), 4 deletions(-) diff --git a/src/py/cam/helpers.py b/src/py/cam/helpers.py index 6b32a134..2d906667 100644 --- a/src/py/cam/helpers.py +++ b/src/py/cam/helpers.py @@ -117,14 +117,12 @@ def to_rgb(fmt, size, data): bayer_pattern = fmt[1:5] bitspp = int(fmt[5:]) - # \todo shifting leaves the lowest bits 0 if bitspp == 8: data = data.reshape((h, w)) - data = data.astype(np.uint16) << 8 + data = data.astype(np.uint16) elif bitspp in [10, 12]: data = data.view(np.uint16) data = data.reshape((h, w)) - data = data << (16 - bitspp) else: raise Exception('Bad bitspp:' + str(bitspp)) @@ -145,7 +143,7 @@ def to_rgb(fmt, size, data): b0 = (idx % 2, idx // 2) rgb = demosaic(data, r0, g0, g1, b0) - rgb = (rgb >> 8).astype(np.uint8) + rgb = (rgb >> (bitspp - 8)).astype(np.uint8) else: rgb = None