[libcamera-devel,3/3] utils: raspberrypi: ctt: dng_load_image: Work with DNG files from Picamera2
diff mbox series

Message ID 20220706101836.20153-4-david.plowman@raspberrypi.com
State Superseded
Headers show
Series
  • Raspberry Pi tuning tool improvements
Related show

Commit Message

David Plowman July 6, 2022, 10:18 a.m. UTC 
From: William Vinnicombe <william.vinnicombe@raspberrypi.com>

The exif tags are different between raw files from libcamera-apps and
from Picamera2, causing issues loading data.

Add code to identify which tags are being used, and then load the
metadata from the correct tags.

Signed-off-by: William Vinnicombe <william.vinnicombe@raspberrypi.com>
---
 utils/raspberrypi/ctt/ctt_image_load.py | 26 ++++++++++++++++++-------
 1 file changed, 19 insertions(+), 7 deletions(-)

Comments

David Plowman July 6, 2022, 11:53 a.m. UTC | #1 
Hi William

Thanks very much for fixing this!

On Wed, 6 Jul 2022 at 11:18, David Plowman
<david.plowman@raspberrypi.com> wrote:
>
> From: William Vinnicombe <william.vinnicombe@raspberrypi.com>
>
> The exif tags are different between raw files from libcamera-apps and
> from Picamera2, causing issues loading data.
>
> Add code to identify which tags are being used, and then load the
> metadata from the correct tags.
>
> Signed-off-by: William Vinnicombe <william.vinnicombe@raspberrypi.com>

Reviewed-by: David Plowman <david.plowman@raspberrypi.com>

Thanks!
David

> ---
>  utils/raspberrypi/ctt/ctt_image_load.py | 26 ++++++++++++++++++-------
>  1 file changed, 19 insertions(+), 7 deletions(-)
>
> diff --git a/utils/raspberrypi/ctt/ctt_image_load.py b/utils/raspberrypi/ctt/ctt_image_load.py
> index 934db123..29c17581 100644
> --- a/utils/raspberrypi/ctt/ctt_image_load.py
> +++ b/utils/raspberrypi/ctt/ctt_image_load.py
> @@ -301,17 +301,29 @@ def dng_load_image(Cam, im_str):
>          metadata.read()
>
>          Img.ver = 100  # random value
> -        Img.w = metadata['Exif.SubImage1.ImageWidth'].value
> +        """
> +        libcamera-apps create a separate Exif.Subimage1 for the picture
> +        picamera2 stores everything under Exif.Image
> +        this code detects which one is being used, and therefore extracts the correct values
> +        """
> +        try:
> +            Img.w = metadata['Exif.SubImage1.ImageWidth'].value
> +            subimage = "SubImage1"
> +            photo = "Photo"
> +        except KeyError:
> +            Img.w = metadata['Exif.Image.ImageWidth'].value
> +            subimage = "Image"
> +            photo = "Image"
>          Img.pad = 0
> -        Img.h = metadata['Exif.SubImage1.ImageLength'].value
> -        white = metadata['Exif.SubImage1.WhiteLevel'].value
> +        Img.h = metadata[f'Exif.{subimage}.ImageLength'].value
> +        white = metadata[f'Exif.{subimage}.WhiteLevel'].value
>          Img.sigbits = int(white).bit_length()
>          Img.fmt = (Img.sigbits - 4) // 2
> -        Img.exposure = int(metadata['Exif.Photo.ExposureTime'].value*1000000)
> -        Img.againQ8 = metadata['Exif.Photo.ISOSpeedRatings'].value*256/100
> +        Img.exposure = int(metadata[f'Exif.{photo}.ExposureTime'].value*1000000)
> +        Img.againQ8 = metadata[f'Exif.{photo}.ISOSpeedRatings'].value*256/100
>          Img.againQ8_norm = Img.againQ8 / 256
>          Img.camName = metadata['Exif.Image.Model'].value
> -        Img.blacklevel = int(metadata['Exif.SubImage1.BlackLevel'].value[0])
> +        Img.blacklevel = int(metadata[f'Exif.{subimage}.BlackLevel'].value[0])
>          Img.blacklevel_16 = Img.blacklevel << (16 - Img.sigbits)
>          bayer_case = {
>              '0 1 1 2': (0, (0, 1, 2, 3)),
> @@ -319,7 +331,7 @@ def dng_load_image(Cam, im_str):
>              '2 1 1 0': (2, (3, 2, 1, 0)),
>              '1 0 2 1': (3, (1, 0, 3, 2))
>          }
> -        cfa_pattern = metadata['Exif.SubImage1.CFAPattern'].value
> +        cfa_pattern = metadata[f'Exif.{subimage}.CFAPattern'].value
>          Img.pattern = bayer_case[cfa_pattern][0]
>          Img.order = bayer_case[cfa_pattern][1]
>
> --
> 2.30.2
>
Laurent Pinchart July 6, 2022, 6:31 p.m. UTC | #2 
Hi David and William,

Thank you for the patch.

On Wed, Jul 06, 2022 at 11:18:36AM +0100, David Plowman via libcamera-devel wrote:
> From: William Vinnicombe <william.vinnicombe@raspberrypi.com>
> 
> The exif tags are different between raw files from libcamera-apps and
> from Picamera2, causing issues loading data.
> 
> Add code to identify which tags are being used, and then load the
> metadata from the correct tags.
> 
> Signed-off-by: William Vinnicombe <william.vinnicombe@raspberrypi.com>
> ---
>  utils/raspberrypi/ctt/ctt_image_load.py | 26 ++++++++++++++++++-------
>  1 file changed, 19 insertions(+), 7 deletions(-)
> 
> diff --git a/utils/raspberrypi/ctt/ctt_image_load.py b/utils/raspberrypi/ctt/ctt_image_load.py
> index 934db123..29c17581 100644
> --- a/utils/raspberrypi/ctt/ctt_image_load.py
> +++ b/utils/raspberrypi/ctt/ctt_image_load.py
> @@ -301,17 +301,29 @@ def dng_load_image(Cam, im_str):
>          metadata.read()
>  
>          Img.ver = 100  # random value
> -        Img.w = metadata['Exif.SubImage1.ImageWidth'].value
> +        """
> +        libcamera-apps create a separate Exif.Subimage1 for the picture
> +        picamera2 stores everything under Exif.Image

Is this valid according to the DNG specification ? Or is it that
picamera2 produces TIFF/EP files instead of DNG ? Is there a reason not
to use DNG in all cases ?

> +        this code detects which one is being used, and therefore extracts the correct values
> +        """
> +        try:
> +            Img.w = metadata['Exif.SubImage1.ImageWidth'].value
> +            subimage = "SubImage1"
> +            photo = "Photo"
> +        except KeyError:
> +            Img.w = metadata['Exif.Image.ImageWidth'].value
> +            subimage = "Image"
> +            photo = "Image"
>          Img.pad = 0
> -        Img.h = metadata['Exif.SubImage1.ImageLength'].value
> -        white = metadata['Exif.SubImage1.WhiteLevel'].value
> +        Img.h = metadata[f'Exif.{subimage}.ImageLength'].value
> +        white = metadata[f'Exif.{subimage}.WhiteLevel'].value
>          Img.sigbits = int(white).bit_length()
>          Img.fmt = (Img.sigbits - 4) // 2
> -        Img.exposure = int(metadata['Exif.Photo.ExposureTime'].value*1000000)
> -        Img.againQ8 = metadata['Exif.Photo.ISOSpeedRatings'].value*256/100
> +        Img.exposure = int(metadata[f'Exif.{photo}.ExposureTime'].value*1000000)
> +        Img.againQ8 = metadata[f'Exif.{photo}.ISOSpeedRatings'].value*256/100
>          Img.againQ8_norm = Img.againQ8 / 256
>          Img.camName = metadata['Exif.Image.Model'].value
> -        Img.blacklevel = int(metadata['Exif.SubImage1.BlackLevel'].value[0])
> +        Img.blacklevel = int(metadata[f'Exif.{subimage}.BlackLevel'].value[0])
>          Img.blacklevel_16 = Img.blacklevel << (16 - Img.sigbits)
>          bayer_case = {
>              '0 1 1 2': (0, (0, 1, 2, 3)),
> @@ -319,7 +331,7 @@ def dng_load_image(Cam, im_str):
>              '2 1 1 0': (2, (3, 2, 1, 0)),
>              '1 0 2 1': (3, (1, 0, 3, 2))
>          }
> -        cfa_pattern = metadata['Exif.SubImage1.CFAPattern'].value
> +        cfa_pattern = metadata[f'Exif.{subimage}.CFAPattern'].value
>          Img.pattern = bayer_case[cfa_pattern][0]
>          Img.order = bayer_case[cfa_pattern][1]
>
David Plowman July 7, 2022, 7:35 a.m. UTC | #3 
Hi Laurent

Thanks for the review!

On Wed, 6 Jul 2022 at 19:32, Laurent Pinchart
<laurent.pinchart@ideasonboard.com> wrote:
>
> Hi David and William,
>
> Thank you for the patch.
>
> On Wed, Jul 06, 2022 at 11:18:36AM +0100, David Plowman via libcamera-devel wrote:
> > From: William Vinnicombe <william.vinnicombe@raspberrypi.com>
> >
> > The exif tags are different between raw files from libcamera-apps and
> > from Picamera2, causing issues loading data.
> >
> > Add code to identify which tags are being used, and then load the
> > metadata from the correct tags.
> >
> > Signed-off-by: William Vinnicombe <william.vinnicombe@raspberrypi.com>
> > ---
> >  utils/raspberrypi/ctt/ctt_image_load.py | 26 ++++++++++++++++++-------
> >  1 file changed, 19 insertions(+), 7 deletions(-)
> >
> > diff --git a/utils/raspberrypi/ctt/ctt_image_load.py b/utils/raspberrypi/ctt/ctt_image_load.py
> > index 934db123..29c17581 100644
> > --- a/utils/raspberrypi/ctt/ctt_image_load.py
> > +++ b/utils/raspberrypi/ctt/ctt_image_load.py
> > @@ -301,17 +301,29 @@ def dng_load_image(Cam, im_str):
> >          metadata.read()
> >
> >          Img.ver = 100  # random value
> > -        Img.w = metadata['Exif.SubImage1.ImageWidth'].value
> > +        """
> > +        libcamera-apps create a separate Exif.Subimage1 for the picture
> > +        picamera2 stores everything under Exif.Image
>
> Is this valid according to the DNG specification ? Or is it that
> picamera2 produces TIFF/EP files instead of DNG ? Is there a reason not
> to use DNG in all cases ?

Actually DNG and EXIF are all really flavours of TIFF. I would say
that Picamera2, which uses the PiDNG library, does it properly. In C++
we use libtiff which, as far as I can tell, doesn't really support
"modern" flavours of DNG-style TIFF. For example, you can't include
exposure times in the main image which PiDNG can.

But they are all DNGs, and DarkTable, RawTherapee and dcraw support
all of them. Just there are some annoying implementation details...

David

>
> > +        this code detects which one is being used, and therefore extracts the correct values
> > +        """
> > +        try:
> > +            Img.w = metadata['Exif.SubImage1.ImageWidth'].value
> > +            subimage = "SubImage1"
> > +            photo = "Photo"
> > +        except KeyError:
> > +            Img.w = metadata['Exif.Image.ImageWidth'].value
> > +            subimage = "Image"
> > +            photo = "Image"
> >          Img.pad = 0
> > -        Img.h = metadata['Exif.SubImage1.ImageLength'].value
> > -        white = metadata['Exif.SubImage1.WhiteLevel'].value
> > +        Img.h = metadata[f'Exif.{subimage}.ImageLength'].value
> > +        white = metadata[f'Exif.{subimage}.WhiteLevel'].value
> >          Img.sigbits = int(white).bit_length()
> >          Img.fmt = (Img.sigbits - 4) // 2
> > -        Img.exposure = int(metadata['Exif.Photo.ExposureTime'].value*1000000)
> > -        Img.againQ8 = metadata['Exif.Photo.ISOSpeedRatings'].value*256/100
> > +        Img.exposure = int(metadata[f'Exif.{photo}.ExposureTime'].value*1000000)
> > +        Img.againQ8 = metadata[f'Exif.{photo}.ISOSpeedRatings'].value*256/100
> >          Img.againQ8_norm = Img.againQ8 / 256
> >          Img.camName = metadata['Exif.Image.Model'].value
> > -        Img.blacklevel = int(metadata['Exif.SubImage1.BlackLevel'].value[0])
> > +        Img.blacklevel = int(metadata[f'Exif.{subimage}.BlackLevel'].value[0])
> >          Img.blacklevel_16 = Img.blacklevel << (16 - Img.sigbits)
> >          bayer_case = {
> >              '0 1 1 2': (0, (0, 1, 2, 3)),
> > @@ -319,7 +331,7 @@ def dng_load_image(Cam, im_str):
> >              '2 1 1 0': (2, (3, 2, 1, 0)),
> >              '1 0 2 1': (3, (1, 0, 3, 2))
> >          }
> > -        cfa_pattern = metadata['Exif.SubImage1.CFAPattern'].value
> > +        cfa_pattern = metadata[f'Exif.{subimage}.CFAPattern'].value
> >          Img.pattern = bayer_case[cfa_pattern][0]
> >          Img.order = bayer_case[cfa_pattern][1]
> >
>
> --
> Regards,
>
> Laurent Pinchart
Laurent Pinchart July 7, 2022, 7:57 a.m. UTC | #4 
Hi David,

On Thu, Jul 07, 2022 at 08:35:39AM +0100, David Plowman wrote:
> On Wed, 6 Jul 2022 at 19:32, Laurent Pinchart wrote:
> > On Wed, Jul 06, 2022 at 11:18:36AM +0100, David Plowman via libcamera-devel wrote:
> > > From: William Vinnicombe <william.vinnicombe@raspberrypi.com>
> > >
> > > The exif tags are different between raw files from libcamera-apps and
> > > from Picamera2, causing issues loading data.
> > >
> > > Add code to identify which tags are being used, and then load the
> > > metadata from the correct tags.
> > >
> > > Signed-off-by: William Vinnicombe <william.vinnicombe@raspberrypi.com>
> > > ---
> > >  utils/raspberrypi/ctt/ctt_image_load.py | 26 ++++++++++++++++++-------
> > >  1 file changed, 19 insertions(+), 7 deletions(-)
> > >
> > > diff --git a/utils/raspberrypi/ctt/ctt_image_load.py b/utils/raspberrypi/ctt/ctt_image_load.py
> > > index 934db123..29c17581 100644
> > > --- a/utils/raspberrypi/ctt/ctt_image_load.py
> > > +++ b/utils/raspberrypi/ctt/ctt_image_load.py
> > > @@ -301,17 +301,29 @@ def dng_load_image(Cam, im_str):
> > >          metadata.read()
> > >
> > >          Img.ver = 100  # random value
> > > -        Img.w = metadata['Exif.SubImage1.ImageWidth'].value
> > > +        """
> > > +        libcamera-apps create a separate Exif.Subimage1 for the picture
> > > +        picamera2 stores everything under Exif.Image
> >
> > Is this valid according to the DNG specification ? Or is it that
> > picamera2 produces TIFF/EP files instead of DNG ? Is there a reason not
> > to use DNG in all cases ?
> 
> Actually DNG and EXIF are all really flavours of TIFF. I would say

Note that I mentioned TIFF/EP, which is also a different flavour of
TIFF. Those formats are an utter mess :-)

> that Picamera2, which uses the PiDNG library, does it properly. In C++
> we use libtiff which, as far as I can tell, doesn't really support
> "modern" flavours of DNG-style TIFF. For example, you can't include
> exposure times in the main image which PiDNG can.

Where is the "modern" flavour of "DNG-style TIFF" documented ? I'm
looking at the DNG v1.4.0.0. specification, which states, in the subIFDs
trees section,

    DNG recommends the use of SubIFD trees, as described in the TIFF-EP
    specification. SubIFD chains are not supported.

    The highest-resolution and quality IFD should use NewSubFileType
    equal to 0. Reduced resolution (or quality) thumbnails or previews,
    if any, should use NewSubFileType equal to 1 (for a primary preview)
    or 10001.H (for an alternate preview).

    DNG recommends, but does not require, that the first IFD contain a
    low-resolution thumbnail, as described in the TIFF-EP specification.

and in the metadata section,

    Additional metadata may be embedded in DNG in the following ways:

    • Using TIFF-EP or EXIF metadata tags
    • Using the IPTC metadata tag (33723)
    • Using the XMP metadata tag (700)

    Note that TIFF-EP and EXIF use nearly the same metadata tag set, but
    TIFF-EP stores the tags in IFD 0, while EXIF store the tags in a
    separate IFD. Either location is allowed by DNG, but the EXIF
    location is preferred.

While the DNG specification seems to allow storing the EXIF tags in
IFD0, as well as storing the main image there as well, that isn't
recommended. I'm thus curious to know what you mean by "modern
flavours".

> But they are all DNGs, and DarkTable, RawTherapee and dcraw support
> all of them. Just there are some annoying implementation details...
> 
> > > +        this code detects which one is being used, and therefore extracts the correct values
> > > +        """
> > > +        try:
> > > +            Img.w = metadata['Exif.SubImage1.ImageWidth'].value
> > > +            subimage = "SubImage1"
> > > +            photo = "Photo"
> > > +        except KeyError:
> > > +            Img.w = metadata['Exif.Image.ImageWidth'].value
> > > +            subimage = "Image"
> > > +            photo = "Image"
> > >          Img.pad = 0
> > > -        Img.h = metadata['Exif.SubImage1.ImageLength'].value
> > > -        white = metadata['Exif.SubImage1.WhiteLevel'].value
> > > +        Img.h = metadata[f'Exif.{subimage}.ImageLength'].value
> > > +        white = metadata[f'Exif.{subimage}.WhiteLevel'].value
> > >          Img.sigbits = int(white).bit_length()
> > >          Img.fmt = (Img.sigbits - 4) // 2
> > > -        Img.exposure = int(metadata['Exif.Photo.ExposureTime'].value*1000000)
> > > -        Img.againQ8 = metadata['Exif.Photo.ISOSpeedRatings'].value*256/100
> > > +        Img.exposure = int(metadata[f'Exif.{photo}.ExposureTime'].value*1000000)
> > > +        Img.againQ8 = metadata[f'Exif.{photo}.ISOSpeedRatings'].value*256/100
> > >          Img.againQ8_norm = Img.againQ8 / 256
> > >          Img.camName = metadata['Exif.Image.Model'].value
> > > -        Img.blacklevel = int(metadata['Exif.SubImage1.BlackLevel'].value[0])
> > > +        Img.blacklevel = int(metadata[f'Exif.{subimage}.BlackLevel'].value[0])
> > >          Img.blacklevel_16 = Img.blacklevel << (16 - Img.sigbits)
> > >          bayer_case = {
> > >              '0 1 1 2': (0, (0, 1, 2, 3)),
> > > @@ -319,7 +331,7 @@ def dng_load_image(Cam, im_str):
> > >              '2 1 1 0': (2, (3, 2, 1, 0)),
> > >              '1 0 2 1': (3, (1, 0, 3, 2))
> > >          }
> > > -        cfa_pattern = metadata['Exif.SubImage1.CFAPattern'].value
> > > +        cfa_pattern = metadata[f'Exif.{subimage}.CFAPattern'].value
> > >          Img.pattern = bayer_case[cfa_pattern][0]
> > >          Img.order = bayer_case[cfa_pattern][1]
> > >
Naushir Patuck July 11, 2022, 9:38 a.m. UTC | #5 
Hi William,

Thank you for your work.

On Wed, 6 Jul 2022 at 11:18, David Plowman via libcamera-devel <
libcamera-devel@lists.libcamera.org> wrote:

> From: William Vinnicombe <william.vinnicombe@raspberrypi.com>
>
> The exif tags are different between raw files from libcamera-apps and
> from Picamera2, causing issues loading data.
>
> Add code to identify which tags are being used, and then load the
> metadata from the correct tags.
>
> Signed-off-by: William Vinnicombe <william.vinnicombe@raspberrypi.com>
>

Reviewed-by: Naushir Patuck <naush@raspberrypi.com>



> ---
>  utils/raspberrypi/ctt/ctt_image_load.py | 26 ++++++++++++++++++-------
>  1 file changed, 19 insertions(+), 7 deletions(-)
>
> diff --git a/utils/raspberrypi/ctt/ctt_image_load.py
> b/utils/raspberrypi/ctt/ctt_image_load.py
> index 934db123..29c17581 100644
> --- a/utils/raspberrypi/ctt/ctt_image_load.py
> +++ b/utils/raspberrypi/ctt/ctt_image_load.py
> @@ -301,17 +301,29 @@ def dng_load_image(Cam, im_str):
>          metadata.read()
>
>          Img.ver = 100  # random value
> -        Img.w = metadata['Exif.SubImage1.ImageWidth'].value
> +        """
> +        libcamera-apps create a separate Exif.Subimage1 for the picture
> +        picamera2 stores everything under Exif.Image
> +        this code detects which one is being used, and therefore extracts
> the correct values
> +        """
> +        try:
> +            Img.w = metadata['Exif.SubImage1.ImageWidth'].value
> +            subimage = "SubImage1"
> +            photo = "Photo"
> +        except KeyError:
> +            Img.w = metadata['Exif.Image.ImageWidth'].value
> +            subimage = "Image"
> +            photo = "Image"
>          Img.pad = 0
> -        Img.h = metadata['Exif.SubImage1.ImageLength'].value
> -        white = metadata['Exif.SubImage1.WhiteLevel'].value
> +        Img.h = metadata[f'Exif.{subimage}.ImageLength'].value
> +        white = metadata[f'Exif.{subimage}.WhiteLevel'].value
>          Img.sigbits = int(white).bit_length()
>          Img.fmt = (Img.sigbits - 4) // 2
> -        Img.exposure =
> int(metadata['Exif.Photo.ExposureTime'].value*1000000)
> -        Img.againQ8 = metadata['Exif.Photo.ISOSpeedRatings'].value*256/100
> +        Img.exposure =
> int(metadata[f'Exif.{photo}.ExposureTime'].value*1000000)
> +        Img.againQ8 =
> metadata[f'Exif.{photo}.ISOSpeedRatings'].value*256/100
>          Img.againQ8_norm = Img.againQ8 / 256
>          Img.camName = metadata['Exif.Image.Model'].value
> -        Img.blacklevel =
> int(metadata['Exif.SubImage1.BlackLevel'].value[0])
> +        Img.blacklevel =
> int(metadata[f'Exif.{subimage}.BlackLevel'].value[0])
>          Img.blacklevel_16 = Img.blacklevel << (16 - Img.sigbits)
>          bayer_case = {
>              '0 1 1 2': (0, (0, 1, 2, 3)),
> @@ -319,7 +331,7 @@ def dng_load_image(Cam, im_str):
>              '2 1 1 0': (2, (3, 2, 1, 0)),
>              '1 0 2 1': (3, (1, 0, 3, 2))
>          }
> -        cfa_pattern = metadata['Exif.SubImage1.CFAPattern'].value
> +        cfa_pattern = metadata[f'Exif.{subimage}.CFAPattern'].value
>          Img.pattern = bayer_case[cfa_pattern][0]
>          Img.order = bayer_case[cfa_pattern][1]
>
> --
> 2.30.2
>
>
Laurent Pinchart July 11, 2022, 9:43 a.m. UTC | #6 
On Thu, Jul 07, 2022 at 10:57:01AM +0300, Laurent Pinchart via libcamera-devel wrote:
> On Thu, Jul 07, 2022 at 08:35:39AM +0100, David Plowman wrote:
> > On Wed, 6 Jul 2022 at 19:32, Laurent Pinchart wrote:
> > > On Wed, Jul 06, 2022 at 11:18:36AM +0100, David Plowman via libcamera-devel wrote:
> > > > From: William Vinnicombe <william.vinnicombe@raspberrypi.com>
> > > >
> > > > The exif tags are different between raw files from libcamera-apps and
> > > > from Picamera2, causing issues loading data.
> > > >
> > > > Add code to identify which tags are being used, and then load the
> > > > metadata from the correct tags.
> > > >
> > > > Signed-off-by: William Vinnicombe <william.vinnicombe@raspberrypi.com>
> > > > ---
> > > >  utils/raspberrypi/ctt/ctt_image_load.py | 26 ++++++++++++++++++-------
> > > >  1 file changed, 19 insertions(+), 7 deletions(-)
> > > >
> > > > diff --git a/utils/raspberrypi/ctt/ctt_image_load.py b/utils/raspberrypi/ctt/ctt_image_load.py
> > > > index 934db123..29c17581 100644
> > > > --- a/utils/raspberrypi/ctt/ctt_image_load.py
> > > > +++ b/utils/raspberrypi/ctt/ctt_image_load.py
> > > > @@ -301,17 +301,29 @@ def dng_load_image(Cam, im_str):
> > > >          metadata.read()
> > > >
> > > >          Img.ver = 100  # random value
> > > > -        Img.w = metadata['Exif.SubImage1.ImageWidth'].value
> > > > +        """
> > > > +        libcamera-apps create a separate Exif.Subimage1 for the picture
> > > > +        picamera2 stores everything under Exif.Image
> > >
> > > Is this valid according to the DNG specification ? Or is it that
> > > picamera2 produces TIFF/EP files instead of DNG ? Is there a reason not
> > > to use DNG in all cases ?
> > 
> > Actually DNG and EXIF are all really flavours of TIFF. I would say
> 
> Note that I mentioned TIFF/EP, which is also a different flavour of
> TIFF. Those formats are an utter mess :-)
> 
> > that Picamera2, which uses the PiDNG library, does it properly. In C++
> > we use libtiff which, as far as I can tell, doesn't really support
> > "modern" flavours of DNG-style TIFF. For example, you can't include
> > exposure times in the main image which PiDNG can.
> 
> Where is the "modern" flavour of "DNG-style TIFF" documented ? I'm
> looking at the DNG v1.4.0.0. specification, which states, in the subIFDs
> trees section,
> 
>     DNG recommends the use of SubIFD trees, as described in the TIFF-EP
>     specification. SubIFD chains are not supported.
> 
>     The highest-resolution and quality IFD should use NewSubFileType
>     equal to 0. Reduced resolution (or quality) thumbnails or previews,
>     if any, should use NewSubFileType equal to 1 (for a primary preview)
>     or 10001.H (for an alternate preview).
> 
>     DNG recommends, but does not require, that the first IFD contain a
>     low-resolution thumbnail, as described in the TIFF-EP specification.
> 
> and in the metadata section,
> 
>     Additional metadata may be embedded in DNG in the following ways:
> 
>     • Using TIFF-EP or EXIF metadata tags
>     • Using the IPTC metadata tag (33723)
>     • Using the XMP metadata tag (700)
> 
>     Note that TIFF-EP and EXIF use nearly the same metadata tag set, but
>     TIFF-EP stores the tags in IFD 0, while EXIF store the tags in a
>     separate IFD. Either location is allowed by DNG, but the EXIF
>     location is preferred.
> 
> While the DNG specification seems to allow storing the EXIF tags in
> IFD0, as well as storing the main image there as well, that isn't
> recommended. I'm thus curious to know what you mean by "modern
> flavours".

Following up on this, I now understand this is meant to support DNG
files created by PiDNG ([1]). I've asked them ([2]) why they use IFD0
instead of following the recommendations of the Adobe DNG specification,
waiting for a reply.

[1] https://github.com/schoolpost/PiDNG
[2] https://github.com/schoolpost/PiDNG/issues/65

> > But they are all DNGs, and DarkTable, RawTherapee and dcraw support
> > all of them. Just there are some annoying implementation details...
> > 
> > > > +        this code detects which one is being used, and therefore extracts the correct values
> > > > +        """
> > > > +        try:
> > > > +            Img.w = metadata['Exif.SubImage1.ImageWidth'].value
> > > > +            subimage = "SubImage1"
> > > > +            photo = "Photo"
> > > > +        except KeyError:
> > > > +            Img.w = metadata['Exif.Image.ImageWidth'].value
> > > > +            subimage = "Image"
> > > > +            photo = "Image"
> > > >          Img.pad = 0
> > > > -        Img.h = metadata['Exif.SubImage1.ImageLength'].value
> > > > -        white = metadata['Exif.SubImage1.WhiteLevel'].value
> > > > +        Img.h = metadata[f'Exif.{subimage}.ImageLength'].value
> > > > +        white = metadata[f'Exif.{subimage}.WhiteLevel'].value
> > > >          Img.sigbits = int(white).bit_length()
> > > >          Img.fmt = (Img.sigbits - 4) // 2
> > > > -        Img.exposure = int(metadata['Exif.Photo.ExposureTime'].value*1000000)
> > > > -        Img.againQ8 = metadata['Exif.Photo.ISOSpeedRatings'].value*256/100
> > > > +        Img.exposure = int(metadata[f'Exif.{photo}.ExposureTime'].value*1000000)
> > > > +        Img.againQ8 = metadata[f'Exif.{photo}.ISOSpeedRatings'].value*256/100
> > > >          Img.againQ8_norm = Img.againQ8 / 256
> > > >          Img.camName = metadata['Exif.Image.Model'].value
> > > > -        Img.blacklevel = int(metadata['Exif.SubImage1.BlackLevel'].value[0])
> > > > +        Img.blacklevel = int(metadata[f'Exif.{subimage}.BlackLevel'].value[0])
> > > >          Img.blacklevel_16 = Img.blacklevel << (16 - Img.sigbits)
> > > >          bayer_case = {
> > > >              '0 1 1 2': (0, (0, 1, 2, 3)),
> > > > @@ -319,7 +331,7 @@ def dng_load_image(Cam, im_str):
> > > >              '2 1 1 0': (2, (3, 2, 1, 0)),
> > > >              '1 0 2 1': (3, (1, 0, 3, 2))
> > > >          }
> > > > -        cfa_pattern = metadata['Exif.SubImage1.CFAPattern'].value
> > > > +        cfa_pattern = metadata[f'Exif.{subimage}.CFAPattern'].value
> > > >          Img.pattern = bayer_case[cfa_pattern][0]
> > > >          Img.order = bayer_case[cfa_pattern][1]
> > > >

Patch
diff mbox series 

diff --git a/utils/raspberrypi/ctt/ctt_image_load.py b/utils/raspberrypi/ctt/ctt_image_load.py
index 934db123..29c17581 100644
--- a/utils/raspberrypi/ctt/ctt_image_load.py
+++ b/utils/raspberrypi/ctt/ctt_image_load.py
@@ -301,17 +301,29 @@  def dng_load_image(Cam, im_str):
         metadata.read()
 
         Img.ver = 100  # random value
-        Img.w = metadata['Exif.SubImage1.ImageWidth'].value
+        """
+        libcamera-apps create a separate Exif.Subimage1 for the picture
+        picamera2 stores everything under Exif.Image
+        this code detects which one is being used, and therefore extracts the correct values
+        """
+        try:
+            Img.w = metadata['Exif.SubImage1.ImageWidth'].value
+            subimage = "SubImage1"
+            photo = "Photo"
+        except KeyError:
+            Img.w = metadata['Exif.Image.ImageWidth'].value
+            subimage = "Image"
+            photo = "Image"
         Img.pad = 0
-        Img.h = metadata['Exif.SubImage1.ImageLength'].value
-        white = metadata['Exif.SubImage1.WhiteLevel'].value
+        Img.h = metadata[f'Exif.{subimage}.ImageLength'].value
+        white = metadata[f'Exif.{subimage}.WhiteLevel'].value
         Img.sigbits = int(white).bit_length()
         Img.fmt = (Img.sigbits - 4) // 2
-        Img.exposure = int(metadata['Exif.Photo.ExposureTime'].value*1000000)
-        Img.againQ8 = metadata['Exif.Photo.ISOSpeedRatings'].value*256/100
+        Img.exposure = int(metadata[f'Exif.{photo}.ExposureTime'].value*1000000)
+        Img.againQ8 = metadata[f'Exif.{photo}.ISOSpeedRatings'].value*256/100
         Img.againQ8_norm = Img.againQ8 / 256
         Img.camName = metadata['Exif.Image.Model'].value
-        Img.blacklevel = int(metadata['Exif.SubImage1.BlackLevel'].value[0])
+        Img.blacklevel = int(metadata[f'Exif.{subimage}.BlackLevel'].value[0])
         Img.blacklevel_16 = Img.blacklevel << (16 - Img.sigbits)
         bayer_case = {
             '0 1 1 2': (0, (0, 1, 2, 3)),
@@ -319,7 +331,7 @@  def dng_load_image(Cam, im_str):
             '2 1 1 0': (2, (3, 2, 1, 0)),
             '1 0 2 1': (3, (1, 0, 3, 2))
         }
-        cfa_pattern = metadata['Exif.SubImage1.CFAPattern'].value
+        cfa_pattern = metadata[f'Exif.{subimage}.CFAPattern'].value
         Img.pattern = bayer_case[cfa_pattern][0]
         Img.order = bayer_case[cfa_pattern][1]