[{"id":23878,"web_url":"https://patchwork.libcamera.org/comment/23878/","msgid":"","date":"2022-07-14T15:27:50","subject":"Re: [libcamera-devel] [PATCH v2 2/2] utils: raspberrypi: ctt:\n\tdng_load_image: Work with DNG files from Picamera2","submitter":{"id":2,"url":"https://patchwork.libcamera.org/api/people/2/","name":"Laurent Pinchart","email":"laurent.pinchart@ideasonboard.com"},"content":"Hi William,\n\nThank you for the patch.\n\nOn Mon, Jul 11, 2022 at 11:26:28AM +0100, william.vinnicombe--- via libcamera-devel wrote:\n> From: William Vinnicombe \n> \n> The exif tags are different between raw files from libcamera-apps and\n> from Picamera2, causing issues loading data.\n> \n> Add code to identify which tags are being used, and then load the\n> metadata from the correct tags.\n\nI haven't received a reply to\nhttps://github.com/schoolpost/PiDNG/issues/65 yet so I'm still puzzled\nby why PiDNG went this way, but that's not a reason to delay this patch.\nI'd like to record more information in the commit message to explain the\nissue though. Here's a proposal:\n\nThe DNG specification is based on the TIFF file format and recommends\nstoring the raw image data in a SubIFD and the Exif tags in an Exif IFD.\nOther options are allowed, even if not recommended, such as storing both\nthe raw image data and the Exif data in IFD0, as done by the TIFF/EP\nspecification.\n\nlibcamera-apps use pyexiv2 to produce DNG files, following the DNG\nrecommendation, while applications based on picamera2 use PiDNG, which\nadopts the TIFF/EP structure. Why it does so is not currently clear (see\nhttps://github.com/schoolpost/PiDNG/issues/65 for discussions on this\ntopic), but as files based on the DNG and TIFF/EP variants exist in the\nwild, both need to be supported by ctt.\n\n> Signed-off-by: William Vinnicombe \n> Reviewed-by: David Plowman \n> Reviewed-by: Naushir Patuck \n> ---\n> utils/raspberrypi/ctt/ctt_image_load.py | 26 ++++++++++++++++++-------\n> 1 file changed, 19 insertions(+), 7 deletions(-)\n> \n> diff --git a/utils/raspberrypi/ctt/ctt_image_load.py b/utils/raspberrypi/ctt/ctt_image_load.py\n> index 934db123..29c17581 100644\n> --- a/utils/raspberrypi/ctt/ctt_image_load.py\n> +++ b/utils/raspberrypi/ctt/ctt_image_load.py\n> @@ -301,17 +301,29 @@ def dng_load_image(Cam, im_str):\n> metadata.read()\n> \n> Img.ver = 100 # random value\n> - Img.w = metadata['Exif.SubImage1.ImageWidth'].value\n> + \"\"\"\n> + libcamera-apps create a separate Exif.Subimage1 for the picture\n> + picamera2 stores everything under Exif.Image\n> + this code detects which one is being used, and therefore extracts the correct values\n\nAnd here, to match the reworded commit message,\n\nThe DNG and TIFF/EP specifications use different IFDs to store the raw\nimage data and the Exif tags. DNG stores them in a SubIFD and in an Exif\nIFD respectively (named \"SubImage1\" and \"Photo\" by pyexiv2), while\nTIFF/EP stores them both in IFD0 (name \"Image\"). Both are used in \"DNG\"\nfiles, with libcamera-apps following the DNG recommendation and\napplications based on picamera2 following TIFF/EP.\n\n> + \"\"\"\n> + try:\n> + Img.w = metadata['Exif.SubImage1.ImageWidth'].value\n> + subimage = \"SubImage1\"\n> + photo = \"Photo\"\n> + except KeyError:\n> + Img.w = metadata['Exif.Image.ImageWidth'].value\n> + subimage = \"Image\"\n> + photo = \"Image\"\n> Img.pad = 0\n> - Img.h = metadata['Exif.SubImage1.ImageLength'].value\n> - white = metadata['Exif.SubImage1.WhiteLevel'].value\n> + Img.h = metadata[f'Exif.{subimage}.ImageLength'].value\n> + white = metadata[f'Exif.{subimage}.WhiteLevel'].value\n> Img.sigbits = int(white).bit_length()\n> Img.fmt = (Img.sigbits - 4) // 2\n> - Img.exposure = int(metadata['Exif.Photo.ExposureTime'].value*1000000)\n> - Img.againQ8 = metadata['Exif.Photo.ISOSpeedRatings'].value*256/100\n> + Img.exposure = int(metadata[f'Exif.{photo}.ExposureTime'].value*1000000)\n> + Img.againQ8 = metadata[f'Exif.{photo}.ISOSpeedRatings'].value*256/100\n> Img.againQ8_norm = Img.againQ8 / 256\n> Img.camName = metadata['Exif.Image.Model'].value\n> - Img.blacklevel = int(metadata['Exif.SubImage1.BlackLevel'].value[0])\n> + Img.blacklevel = int(metadata[f'Exif.{subimage}.BlackLevel'].value[0])\n> Img.blacklevel_16 = Img.blacklevel << (16 - Img.sigbits)\n> bayer_case = {\n> '0 1 1 2': (0, (0, 1, 2, 3)),\n> @@ -319,7 +331,7 @@ def dng_load_image(Cam, im_str):\n> '2 1 1 0': (2, (3, 2, 1, 0)),\n> '1 0 2 1': (3, (1, 0, 3, 2))\n> }\n> - cfa_pattern = metadata['Exif.SubImage1.CFAPattern'].value\n> + cfa_pattern = metadata[f'Exif.{subimage}.CFAPattern'].value\n> Img.pattern = bayer_case[cfa_pattern][0]\n> Img.order = bayer_case[cfa_pattern][1]\n>","headers":{"Return-Path":"","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id 3FBA2BD1F1\n\tfor ;\n\tThu, 14 Jul 2022 15:28:24 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id A642F63312;\n\tThu, 14 Jul 2022 17:28:23 +0200 (CEST)","from perceval.ideasonboard.com (perceval.ideasonboard.com\n\t[IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id F3B0D6330D\n\tfor ;\n\tThu, 14 Jul 2022 17:28:21 +0200 (CEST)","from pendragon.ideasonboard.com (62-78-145-57.bb.dnainternet.fi\n\t[62.78.145.57])\n\tby perceval.ideasonboard.com (Postfix) with ESMTPSA id 70C34383;\n\tThu, 14 Jul 2022 17:28:21 +0200 (CEST)"],"DKIM-Signature":["v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org;\n\ts=mail; t=1657812503;\n\tbh=kUhbfX+GK4WSON7PML5X1bTswyiHmUtEHliOmWz2L/k=;\n\th=Date:To:References:In-Reply-To:Subject:List-Id:List-Unsubscribe:\n\tList-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To:Cc:\n\tFrom;\n\tb=gF/BVg8W2tKeNLesW03+0k9U725z3DF0KLfWmdN5VNTIYstSq3Y9W1j9YCJ+kUGf9\n\t953oHTlPDNZGY+OLPanftZIET9oDaWcgz69zyraaQpkuZhmaZwxhHIJF0wpXnS8Mvg\n\ttGf3/pF9U0L9V8t/RXCBKdED3svULf+/zeUlzRqwC7nICY1OcEIiwYvuQK7H2nietl\n\t7GLLfAspGvSC0dWDHkriBG5Oz5w/4uxAmVx+kPZG1Htl6my39BREGWmLI1gay++gpT\n\tlP+n7SacYPNLppLIFk8H8kd81/jk/DgSmmwp1LlzKERGTrYPKezJZ6lq6aIcfJWyZw\n\t1rYoo4giVcv1Q==","v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com;\n\ts=mail; t=1657812501;\n\tbh=kUhbfX+GK4WSON7PML5X1bTswyiHmUtEHliOmWz2L/k=;\n\th=Date:From:To:Cc:Subject:References:In-Reply-To:From;\n\tb=YZMgl+GE7uJWP3Lay3U6dkkKqp+GnrRJVVkli1q4B/3mwCp9FG8zhtGdc2ixA19wh\n\tgmQXNenkBepODZmeHjQ2z1BDb1at0lLnVb+TJvXe3wQYy/DiNUQfrlQbc3G1u0GmHJ\n\tGVFAeskwan+XCnIhs8Rp2TEGchBr6PJUG+wIlFa4="],"Authentication-Results":"lancelot.ideasonboard.com; dkim=pass (1024-bit key; \n\tunprotected) header.d=ideasonboard.com\n\theader.i=@ideasonboard.com\n\theader.b=\"YZMgl+GE\"; dkim-atps=neutral","Date":"Thu, 14 Jul 2022 18:27:50 +0300","To":"william.vinnicombe@raspberrypi.com","Message-ID":"","References":"<20220711102628.14269-1-william.vinnicombe@raspberrypi.com>\n\t<20220711102628.14269-3-william.vinnicombe@raspberrypi.com>","MIME-Version":"1.0","Content-Type":"text/plain; charset=utf-8","Content-Disposition":"inline","In-Reply-To":"<20220711102628.14269-3-william.vinnicombe@raspberrypi.com>","Subject":"Re: [libcamera-devel] [PATCH v2 2/2] utils: raspberrypi: ctt:\n\tdng_load_image: Work with DNG files from Picamera2","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","From":"Laurent Pinchart via libcamera-devel\n\t","Reply-To":"Laurent Pinchart ","Cc":"libcamera-devel@lists.libcamera.org","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" "}}]