[libcamera-devel,v3,02/22] libcamera: formats: Add fields to info to ease calculating stride

8623 diff mbox series

Packed formats make it difficult to calculate stride as well as
frame size with the fields that PixelFormatInfo currently has.
bitsPerPixel is defined as the average number of bits per pixel, and
only counts effective bits, so it is not useful for calculating
stride and frame size.

To fix this, we introduce a concept of a "pixel group". The size of this
group is defined as the minimum number of pixels (including padding)
necessary in a row when the image has only one column of effective
pixels. The pixel group has one more attribute, that is the "bytes per
group". This determines how many bytes one pixel group consumes. These
are the fields pixelsPerGroup and bytesPerGroup that are defined in this
patch. Defining these two values makes it really simple to calculate
bytes-per-line, as ceil(width / pixelsPerGroup) * bytesPerGroup, where
width is measured in number of pixels. The ceiling accounts for padding.

Clearly, pixelsPerGroup must be constant for all planes in the format.
The bytesPerGroup then, must be a per-plane attribute. There is one more
field, verticalSubSampling, that is per-plane. This is simply a divider,
to divide the number of rows of pixels by the sub-sampling value, to
obtain the number of rows of pixels for the subsampled plane.

For example, for something simple like BGR888, it is self-explanatory:
the pixel group size is 1, and the bytes necessary is 3, and there is
only one plane with no (= 1) vertical subsampling. For YUYV, the
CbCr pair is shared between two pixels, so even if you have only one
pixel, you would still need a padded second Y, therefore the pixel
group size is 2, and bytes necessary is 4 (as opposed to 1 and 2). YUYV
also has no vertical subsampling. NV12 has a pixel group size of 2
pixels, due to the CbCr plane. The bytes per group then, for both
planes, is 2. The first plane has no vertical subsampling, but the
second plane is subsampled by a factor of 2.
The IPU3 formats are also self-explanatory, as they are single-planar,
and have a pixel group size of 25, consuming 32 bytes. Although a
comment in the driver suggests that it should be 50 and 64,
respectively, this is an attribute of the driver, and not the format, so
this shall be set by the ipu3 pipeline handler.

Signed-off-by: Paul Elder <paul.elder@ideasonboard.com>

---
Changes in v3:
- add planes
- redefine the parameters for the formats
  - pixelsPerGroup is for whole format
  - add verticalSubSampling per plane

Changes in v2:
- add documentation for bytesPerGroup pixelsPerGroup
- fix wording in commit message
  - bytes-per-line -> stride
  - buffer size -> frame size
- changed MJPEG todo to allowing pipeline handlers to set parameters of
  format infos
---
 include/libcamera/internal/formats.h |  11 ++-
 src/libcamera/formats.cpp            | 128 ++++++++++++++++++++++++++-
 2 files changed, 137 insertions(+), 2 deletions(-)

Hi Paul,

Thank you for the patch.

On Sat, Jul 04, 2020 at 10:31:20PM +0900, Paul Elder wrote:
> Packed formats make it difficult to calculate stride as well as
> frame size with the fields that PixelFormatInfo currently has.
> bitsPerPixel is defined as the average number of bits per pixel, and
> only counts effective bits, so it is not useful for calculating
> stride and frame size.
> 
> To fix this, we introduce a concept of a "pixel group". The size of this
> group is defined as the minimum number of pixels (including padding)
> necessary in a row when the image has only one column of effective
> pixels. The pixel group has one more attribute, that is the "bytes per
> group". This determines how many bytes one pixel group consumes. These
> are the fields pixelsPerGroup and bytesPerGroup that are defined in this
> patch. Defining these two values makes it really simple to calculate
> bytes-per-line, as ceil(width / pixelsPerGroup) * bytesPerGroup, where
> width is measured in number of pixels. The ceiling accounts for padding.
> 
> Clearly, pixelsPerGroup must be constant for all planes in the format.
> The bytesPerGroup then, must be a per-plane attribute. There is one more
> field, verticalSubSampling, that is per-plane. This is simply a divider,
> to divide the number of rows of pixels by the sub-sampling value, to
> obtain the number of rows of pixels for the subsampled plane.
> 
> For example, for something simple like BGR888, it is self-explanatory:
> the pixel group size is 1, and the bytes necessary is 3, and there is
> only one plane with no (= 1) vertical subsampling. For YUYV, the
> CbCr pair is shared between two pixels, so even if you have only one
> pixel, you would still need a padded second Y, therefore the pixel
> group size is 2, and bytes necessary is 4 (as opposed to 1 and 2). YUYV
> also has no vertical subsampling. NV12 has a pixel group size of 2
> pixels, due to the CbCr plane. The bytes per group then, for both
> planes, is 2. The first plane has no vertical subsampling, but the
> second plane is subsampled by a factor of 2.
> The IPU3 formats are also self-explanatory, as they are single-planar,
> and have a pixel group size of 25, consuming 32 bytes. Although a
> comment in the driver suggests that it should be 50 and 64,
> respectively, this is an attribute of the driver, and not the format, so
> this shall be set by the ipu3 pipeline handler.
> 
> Signed-off-by: Paul Elder <paul.elder@ideasonboard.com>
> 
> ---
> Changes in v3:
> - add planes
> - redefine the parameters for the formats
>   - pixelsPerGroup is for whole format
>   - add verticalSubSampling per plane
> 
> Changes in v2:
> - add documentation for bytesPerGroup pixelsPerGroup
> - fix wording in commit message
>   - bytes-per-line -> stride
>   - buffer size -> frame size
> - changed MJPEG todo to allowing pipeline handlers to set parameters of
>   format infos
> ---
>  include/libcamera/internal/formats.h |  11 ++-
>  src/libcamera/formats.cpp            | 128 ++++++++++++++++++++++++++-
>  2 files changed, 137 insertions(+), 2 deletions(-)
> 
> diff --git a/include/libcamera/internal/formats.h b/include/libcamera/internal/formats.h
> index f59ac8f..ebd256c 100644
> --- a/include/libcamera/internal/formats.h
> +++ b/include/libcamera/internal/formats.h
> @@ -32,6 +32,12 @@ private:
>  	std::map<unsigned int, std::vector<SizeRange>> data_;
>  };
>  
> +struct PixelFormatPlane

Maybe PixelFormatPlaneInfo ?

> +{
> +	unsigned int bytesPerGroup;
> +	unsigned int verticalSubSampling;
> +};
> +
>  class PixelFormatInfo
>  {
>  public:
> @@ -45,13 +51,16 @@ public:
>  
>  	static const PixelFormatInfo &info(const PixelFormat &format);
>  
> -	/* \todo Add support for non-contiguous memory planes */

I think you still need to keep this, as V4L2 uses different 4CCs for
contiguous and non-contiguous formats (yeah, I know, ...) and we don't
support this yet.

>  	const char *name;
>  	PixelFormat format;
>  	V4L2PixelFormat v4l2Format;
>  	unsigned int bitsPerPixel;
>  	enum ColourEncoding colourEncoding;
>  	bool packed;
> +
> +	unsigned int pixelsPerGroup;
> +
> +	std::array<PixelFormatPlane, 3> planes;
>  };
>  
>  } /* namespace libcamera */
> diff --git a/src/libcamera/formats.cpp b/src/libcamera/formats.cpp
> index d3b722c..6bdd28d 100644
> --- a/src/libcamera/formats.cpp
> +++ b/src/libcamera/formats.cpp
> @@ -110,6 +110,22 @@ const std::map<unsigned int, std::vector<SizeRange>> &ImageFormats::data() const
>  	return data_;
>  }
>  
> +/**
> + * \class PixelFormatPlane
> + * \brief Information about a single plane of a pixel format
> + *
> + * \var PixelFormatPlane::bytesPerGroup
> + * \brief The number of bytes that a pixel group consumes
> + *
> + * \sa PixelFormatInfo::pixelsPerGroup
> + *
> + * \var PixelFormatPlane::verticalSubSampling
> + * \brief Vertical subsampling multiplier
> + *
> + * This value is the ratio between the number of rows of pixels in the frame
> + * to the number of rows of pixels in the plane.
> + */
> +
>  /**
>   * \class PixelFormatInfo
>   * \brief Information about pixel formats
> @@ -152,6 +168,26 @@ const std::map<unsigned int, std::vector<SizeRange>> &ImageFormats::data() const
>   * bytes. For instance, 12-bit Bayer data with two pixels stored in three bytes
>   * is packed, while the same data stored with 4 bits of padding in two bytes
>   * per pixel is not packed.
> + *
> + * \var PixelFormatInfo::pixelsPerGroup
> + * \brief The number of pixels in a pixel group
> + *
> + * The minimum number of pixels (including padding) necessary in a row
> + * when the frame has only one column of effective pixels

s/$/./

> + *
> + * A pixel group is defined as the minimum number of pixels (including padding)
> + * necessary in a row when the image has only one column of effective pixels.
> + * pixelsPerGroup refers to this value. bytesPerGroup, then, refers to the

s/bytesPerGroup/PixelFormatPlane::bytesPerGroup/

> + * number of bytes that a pixel group consumes. This definition of a pixel
> + * group allows simple calculation of stride, as
> + * ceil(width / pixelsPerGroup) * bytesPerGroup. These values are determined
> + * only in terms of a row, and include bytes that are used in all planes (for
> + * multiplanar formats).

What do you mean by "include bytes that are used in all planes" ?

> + *
> + * \var PixelFormatInfo::planes
> + * \brief Information about pixels for each plane
> + *
> + * \sa PixelFormatPlane
>   */
>  
>  /**
> @@ -179,6 +215,8 @@ const std::map<PixelFormat, PixelFormatInfo> pixelFormatInfo{
>  		.bitsPerPixel = 24,
>  		.colourEncoding = PixelFormatInfo::ColourEncodingRGB,
>  		.packed = false,
> +		.pixelsPerGroup = 1,
> +		.planes = {{ { 3, 1 }, { 0, 0 }, { 0, 0 } }},
>  	} },
>  	{ formats::RGB888, {
>  		.name = "RGB888",
> @@ -187,6 +225,8 @@ const std::map<PixelFormat, PixelFormatInfo> pixelFormatInfo{
>  		.bitsPerPixel = 24,
>  		.colourEncoding = PixelFormatInfo::ColourEncodingRGB,
>  		.packed = false,
> +		.pixelsPerGroup = 1,
> +		.planes = {{ { 3, 1 }, { 0, 0 }, { 0, 0 } }},
>  	} },
>  	{ formats::ABGR8888, {
>  		.name = "ABGR8888",
> @@ -195,6 +235,8 @@ const std::map<PixelFormat, PixelFormatInfo> pixelFormatInfo{
>  		.bitsPerPixel = 32,
>  		.colourEncoding = PixelFormatInfo::ColourEncodingRGB,
>  		.packed = false,
> +		.pixelsPerGroup = 1,
> +		.planes = {{ { 4, 1 }, { 0, 0 }, { 0, 0 } }},
>  	} },
>  	{ formats::ARGB8888, {
>  		.name = "ARGB8888",
> @@ -203,6 +245,8 @@ const std::map<PixelFormat, PixelFormatInfo> pixelFormatInfo{
>  		.bitsPerPixel = 32,
>  		.colourEncoding = PixelFormatInfo::ColourEncodingRGB,
>  		.packed = false,
> +		.pixelsPerGroup = 1,
> +		.planes = {{ { 4, 1 }, { 0, 0 }, { 0, 0 } }},
>  	} },
>  	{ formats::BGRA8888, {
>  		.name = "BGRA8888",
> @@ -211,6 +255,8 @@ const std::map<PixelFormat, PixelFormatInfo> pixelFormatInfo{
>  		.bitsPerPixel = 32,
>  		.colourEncoding = PixelFormatInfo::ColourEncodingRGB,
>  		.packed = false,
> +		.pixelsPerGroup = 1,
> +		.planes = {{ { 4, 1 }, { 0, 0 }, { 0, 0 } }},
>  	} },
>  	{ formats::RGBA8888, {
>  		.name = "RGBA8888",
> @@ -219,6 +265,8 @@ const std::map<PixelFormat, PixelFormatInfo> pixelFormatInfo{
>  		.bitsPerPixel = 32,
>  		.colourEncoding = PixelFormatInfo::ColourEncodingRGB,
>  		.packed = false,
> +		.pixelsPerGroup = 1,
> +		.planes = {{ { 4, 1 }, { 0, 0 }, { 0, 0 } }},
>  	} },
>  
>  	/* YUV packed formats. */
> @@ -229,6 +277,8 @@ const std::map<PixelFormat, PixelFormatInfo> pixelFormatInfo{
>  		.bitsPerPixel = 16,
>  		.colourEncoding = PixelFormatInfo::ColourEncodingYUV,
>  		.packed = false,
> +		.pixelsPerGroup = 2,
> +		.planes = {{ { 4, 1 }, { 0, 0 }, { 0, 0 } }},
>  	} },
>  	{ formats::YVYU, {
>  		.name = "YVYU",
> @@ -237,6 +287,8 @@ const std::map<PixelFormat, PixelFormatInfo> pixelFormatInfo{
>  		.bitsPerPixel = 16,
>  		.colourEncoding = PixelFormatInfo::ColourEncodingYUV,
>  		.packed = false,
> +		.pixelsPerGroup = 2,
> +		.planes = {{ { 4, 1 }, { 0, 0 }, { 0, 0 } }},
>  	} },
>  	{ formats::UYVY, {
>  		.name = "UYVY",
> @@ -245,6 +297,8 @@ const std::map<PixelFormat, PixelFormatInfo> pixelFormatInfo{
>  		.bitsPerPixel = 16,
>  		.colourEncoding = PixelFormatInfo::ColourEncodingYUV,
>  		.packed = false,
> +		.pixelsPerGroup = 2,
> +		.planes = {{ { 4, 1 }, { 0, 0 }, { 0, 0 } }},
>  	} },
>  	{ formats::VYUY, {
>  		.name = "VYUY",
> @@ -253,6 +307,8 @@ const std::map<PixelFormat, PixelFormatInfo> pixelFormatInfo{
>  		.bitsPerPixel = 16,
>  		.colourEncoding = PixelFormatInfo::ColourEncodingYUV,
>  		.packed = false,
> +		.pixelsPerGroup = 2,
> +		.planes = {{ { 4, 1 }, { 0, 0 }, { 0, 0 } }},
>  	} },
>  
>  	/* YUV planar formats. */
> @@ -263,6 +319,8 @@ const std::map<PixelFormat, PixelFormatInfo> pixelFormatInfo{
>  		.bitsPerPixel = 12,
>  		.colourEncoding = PixelFormatInfo::ColourEncodingYUV,
>  		.packed = false,
> +		.pixelsPerGroup = 2,
> +		.planes = {{ { 2, 1 }, { 2, 2 }, { 0, 0 } }},
>  	} },
>  	{ formats::NV21, {
>  		.name = "NV21",
> @@ -271,6 +329,8 @@ const std::map<PixelFormat, PixelFormatInfo> pixelFormatInfo{
>  		.bitsPerPixel = 12,
>  		.colourEncoding = PixelFormatInfo::ColourEncodingYUV,
>  		.packed = false,
> +		.pixelsPerGroup = 2,
> +		.planes = {{ { 2, 1 }, { 2, 2 }, { 0, 0 } }},
>  	} },
>  	{ formats::NV16, {
>  		.name = "NV16",
> @@ -279,6 +339,8 @@ const std::map<PixelFormat, PixelFormatInfo> pixelFormatInfo{
>  		.bitsPerPixel = 16,
>  		.colourEncoding = PixelFormatInfo::ColourEncodingYUV,
>  		.packed = false,
> +		.pixelsPerGroup = 2,
> +		.planes = {{ { 2, 1 }, { 2, 1 }, { 0, 0 } }},
>  	} },
>  	{ formats::NV61, {
>  		.name = "NV61",
> @@ -287,6 +349,8 @@ const std::map<PixelFormat, PixelFormatInfo> pixelFormatInfo{
>  		.bitsPerPixel = 16,
>  		.colourEncoding = PixelFormatInfo::ColourEncodingYUV,
>  		.packed = false,
> +		.pixelsPerGroup = 2,
> +		.planes = {{ { 2, 1 }, { 2, 1 }, { 0, 0 } }},
>  	} },
>  	{ formats::NV24, {
>  		.name = "NV24",
> @@ -295,6 +359,8 @@ const std::map<PixelFormat, PixelFormatInfo> pixelFormatInfo{
>  		.bitsPerPixel = 24,
>  		.colourEncoding = PixelFormatInfo::ColourEncodingYUV,
>  		.packed = false,
> +		.pixelsPerGroup = 1,
> +		.planes = {{ { 1, 1 }, { 2, 1 }, { 0, 0 } }},
>  	} },
>  	{ formats::NV42, {
>  		.name = "NV42",
> @@ -303,6 +369,8 @@ const std::map<PixelFormat, PixelFormatInfo> pixelFormatInfo{
>  		.bitsPerPixel = 24,
>  		.colourEncoding = PixelFormatInfo::ColourEncodingYUV,
>  		.packed = false,
> +		.pixelsPerGroup = 1,
> +		.planes = {{ { 1, 1 }, { 2, 1 }, { 0, 0 } }},
>  	} },
>  	{ formats::YUV420, {
>  		.name = "YUV420",
> @@ -311,6 +379,8 @@ const std::map<PixelFormat, PixelFormatInfo> pixelFormatInfo{
>  		.bitsPerPixel = 12,
>  		.colourEncoding = PixelFormatInfo::ColourEncodingYUV,
>  		.packed = false,
> +		.pixelsPerGroup = 2,
> +		.planes = {{ { 2, 1 }, { 1, 2 }, { 1, 2 } }},
>  	} },
>  	{ formats::YUV422, {
>  		.name = "YUV422",
> @@ -319,6 +389,8 @@ const std::map<PixelFormat, PixelFormatInfo> pixelFormatInfo{
>  		.bitsPerPixel = 16,
>  		.colourEncoding = PixelFormatInfo::ColourEncodingYUV,
>  		.packed = false,
> +		.pixelsPerGroup = 2,
> +		.planes = {{ { 2, 1 }, { 1, 1 }, { 1, 1 } }},
>  	} },
>  
>  	/* Greyscale formats. */
> @@ -329,6 +401,8 @@ const std::map<PixelFormat, PixelFormatInfo> pixelFormatInfo{
>  		.bitsPerPixel = 8,
>  		.colourEncoding = PixelFormatInfo::ColourEncodingYUV,
>  		.packed = false,
> +		.pixelsPerGroup = 1,
> +		.planes = {{ { 1, 1 }, { 0, 0 }, { 0, 0 } }},
>  	} },
>  
>  	/* Bayer formats. */
> @@ -339,6 +413,8 @@ const std::map<PixelFormat, PixelFormatInfo> pixelFormatInfo{
>  		.bitsPerPixel = 8,
>  		.colourEncoding = PixelFormatInfo::ColourEncodingRAW,
>  		.packed = false,
> +		.pixelsPerGroup = 2,
> +		.planes = {{ { 2, 1 }, { 0, 0 }, { 0, 0 } }},
>  	} },
>  	{ formats::SGBRG8, {
>  		.name = "SGBRG8",
> @@ -347,6 +423,8 @@ const std::map<PixelFormat, PixelFormatInfo> pixelFormatInfo{
>  		.bitsPerPixel = 8,
>  		.colourEncoding = PixelFormatInfo::ColourEncodingRAW,
>  		.packed = false,
> +		.pixelsPerGroup = 2,
> +		.planes = {{ { 2, 1 }, { 0, 0 }, { 0, 0 } }},
>  	} },
>  	{ formats::SGRBG8, {
>  		.name = "SGRBG8",
> @@ -355,6 +433,8 @@ const std::map<PixelFormat, PixelFormatInfo> pixelFormatInfo{
>  		.bitsPerPixel = 8,
>  		.colourEncoding = PixelFormatInfo::ColourEncodingRAW,
>  		.packed = false,
> +		.pixelsPerGroup = 2,
> +		.planes = {{ { 2, 1 }, { 0, 0 }, { 0, 0 } }},
>  	} },
>  	{ formats::SRGGB8, {
>  		.name = "SRGGB8",
> @@ -363,6 +443,8 @@ const std::map<PixelFormat, PixelFormatInfo> pixelFormatInfo{
>  		.bitsPerPixel = 8,
>  		.colourEncoding = PixelFormatInfo::ColourEncodingRAW,
>  		.packed = false,
> +		.pixelsPerGroup = 2,
> +		.planes = {{ { 2, 1 }, { 0, 0 }, { 0, 0 } }},
>  	} },
>  	{ formats::SBGGR10, {
>  		.name = "SBGGR10",
> @@ -371,6 +453,8 @@ const std::map<PixelFormat, PixelFormatInfo> pixelFormatInfo{
>  		.bitsPerPixel = 10,
>  		.colourEncoding = PixelFormatInfo::ColourEncodingRAW,
>  		.packed = false,
> +		.pixelsPerGroup = 2,
> +		.planes = {{ { 4, 1 }, { 0, 0 }, { 0, 0 } }},
>  	} },
>  	{ formats::SGBRG10, {
>  		.name = "SGBRG10",
> @@ -379,6 +463,8 @@ const std::map<PixelFormat, PixelFormatInfo> pixelFormatInfo{
>  		.bitsPerPixel = 10,
>  		.colourEncoding = PixelFormatInfo::ColourEncodingRAW,
>  		.packed = false,
> +		.pixelsPerGroup = 2,
> +		.planes = {{ { 4, 1 }, { 0, 0 }, { 0, 0 } }},
>  	} },
>  	{ formats::SGRBG10, {
>  		.name = "SGRBG10",
> @@ -387,6 +473,8 @@ const std::map<PixelFormat, PixelFormatInfo> pixelFormatInfo{
>  		.bitsPerPixel = 10,
>  		.colourEncoding = PixelFormatInfo::ColourEncodingRAW,
>  		.packed = false,
> +		.pixelsPerGroup = 2,
> +		.planes = {{ { 4, 1 }, { 0, 0 }, { 0, 0 } }},
>  	} },
>  	{ formats::SRGGB10, {
>  		.name = "SRGGB10",
> @@ -395,6 +483,8 @@ const std::map<PixelFormat, PixelFormatInfo> pixelFormatInfo{
>  		.bitsPerPixel = 10,
>  		.colourEncoding = PixelFormatInfo::ColourEncodingRAW,
>  		.packed = false,
> +		.pixelsPerGroup = 2,
> +		.planes = {{ { 4, 1 }, { 0, 0 }, { 0, 0 } }},
>  	} },
>  	{ formats::SBGGR10_CSI2P, {
>  		.name = "SBGGR10_CSI2P",
> @@ -403,6 +493,8 @@ const std::map<PixelFormat, PixelFormatInfo> pixelFormatInfo{
>  		.bitsPerPixel = 10,
>  		.colourEncoding = PixelFormatInfo::ColourEncodingRAW,
>  		.packed = true,
> +		.pixelsPerGroup = 4,
> +		.planes = {{ { 5, 1 }, { 0, 0 }, { 0, 0 } }},
>  	} },
>  	{ formats::SGBRG10_CSI2P, {
>  		.name = "SGBRG10_CSI2P",
> @@ -411,6 +503,8 @@ const std::map<PixelFormat, PixelFormatInfo> pixelFormatInfo{
>  		.bitsPerPixel = 10,
>  		.colourEncoding = PixelFormatInfo::ColourEncodingRAW,
>  		.packed = true,
> +		.pixelsPerGroup = 4,
> +		.planes = {{ { 5, 1 }, { 0, 0 }, { 0, 0 } }},
>  	} },
>  	{ formats::SGRBG10_CSI2P, {
>  		.name = "SGRBG10_CSI2P",
> @@ -419,6 +513,8 @@ const std::map<PixelFormat, PixelFormatInfo> pixelFormatInfo{
>  		.bitsPerPixel = 10,
>  		.colourEncoding = PixelFormatInfo::ColourEncodingRAW,
>  		.packed = true,
> +		.pixelsPerGroup = 4,
> +		.planes = {{ { 5, 1 }, { 0, 0 }, { 0, 0 } }},
>  	} },
>  	{ formats::SRGGB10_CSI2P, {
>  		.name = "SRGGB10_CSI2P",
> @@ -427,6 +523,8 @@ const std::map<PixelFormat, PixelFormatInfo> pixelFormatInfo{
>  		.bitsPerPixel = 10,
>  		.colourEncoding = PixelFormatInfo::ColourEncodingRAW,
>  		.packed = true,
> +		.pixelsPerGroup = 4,
> +		.planes = {{ { 5, 1 }, { 0, 0 }, { 0, 0 } }},
>  	} },
>  	{ formats::SBGGR12, {
>  		.name = "SBGGR12",
> @@ -435,6 +533,8 @@ const std::map<PixelFormat, PixelFormatInfo> pixelFormatInfo{
>  		.bitsPerPixel = 12,
>  		.colourEncoding = PixelFormatInfo::ColourEncodingRAW,
>  		.packed = false,
> +		.pixelsPerGroup = 2,
> +		.planes = {{ { 4, 1 }, { 0, 0 }, { 0, 0 } }},
>  	} },
>  	{ formats::SGBRG12, {
>  		.name = "SGBRG12",
> @@ -443,6 +543,8 @@ const std::map<PixelFormat, PixelFormatInfo> pixelFormatInfo{
>  		.bitsPerPixel = 12,
>  		.colourEncoding = PixelFormatInfo::ColourEncodingRAW,
>  		.packed = false,
> +		.pixelsPerGroup = 2,
> +		.planes = {{ { 4, 1 }, { 0, 0 }, { 0, 0 } }},
>  	} },
>  	{ formats::SGRBG12, {
>  		.name = "SGRBG12",
> @@ -451,6 +553,8 @@ const std::map<PixelFormat, PixelFormatInfo> pixelFormatInfo{
>  		.bitsPerPixel = 12,
>  		.colourEncoding = PixelFormatInfo::ColourEncodingRAW,
>  		.packed = false,
> +		.pixelsPerGroup = 2,
> +		.planes = {{ { 4, 1 }, { 0, 0 }, { 0, 0 } }},
>  	} },
>  	{ formats::SRGGB12, {
>  		.name = "SRGGB12",
> @@ -459,6 +563,8 @@ const std::map<PixelFormat, PixelFormatInfo> pixelFormatInfo{
>  		.bitsPerPixel = 12,
>  		.colourEncoding = PixelFormatInfo::ColourEncodingRAW,
>  		.packed = false,
> +		.pixelsPerGroup = 2,
> +		.planes = {{ { 4, 1 }, { 0, 0 }, { 0, 0 } }},
>  	} },
>  	{ formats::SBGGR12_CSI2P, {
>  		.name = "SBGGR12_CSI2P",
> @@ -467,6 +573,8 @@ const std::map<PixelFormat, PixelFormatInfo> pixelFormatInfo{
>  		.bitsPerPixel = 12,
>  		.colourEncoding = PixelFormatInfo::ColourEncodingRAW,
>  		.packed = true,
> +		.pixelsPerGroup = 2,
> +		.planes = {{ { 3, 1 }, { 0, 0 }, { 0, 0 } }},
>  	} },
>  	{ formats::SGBRG12_CSI2P, {
>  		.name = "SGBRG12_CSI2P",
> @@ -475,6 +583,8 @@ const std::map<PixelFormat, PixelFormatInfo> pixelFormatInfo{
>  		.bitsPerPixel = 12,
>  		.colourEncoding = PixelFormatInfo::ColourEncodingRAW,
>  		.packed = true,
> +		.pixelsPerGroup = 2,
> +		.planes = {{ { 3, 1 }, { 0, 0 }, { 0, 0 } }},
>  	} },
>  	{ formats::SGRBG12_CSI2P, {
>  		.name = "SGRBG12_CSI2P",
> @@ -483,6 +593,8 @@ const std::map<PixelFormat, PixelFormatInfo> pixelFormatInfo{
>  		.bitsPerPixel = 12,
>  		.colourEncoding = PixelFormatInfo::ColourEncodingRAW,
>  		.packed = true,
> +		.pixelsPerGroup = 2,
> +		.planes = {{ { 3, 1 }, { 0, 0 }, { 0, 0 } }},
>  	} },
>  	{ formats::SRGGB12_CSI2P, {
>  		.name = "SRGGB12_CSI2P",
> @@ -491,6 +603,8 @@ const std::map<PixelFormat, PixelFormatInfo> pixelFormatInfo{
>  		.bitsPerPixel = 12,
>  		.colourEncoding = PixelFormatInfo::ColourEncodingRAW,
>  		.packed = true,
> +		.pixelsPerGroup = 2,
> +		.planes = {{ { 3, 1 }, { 0, 0 }, { 0, 0 } }},
>  	} },
>  	{ formats::SBGGR10_IPU3, {
>  		.name = "SBGGR10_IPU3",
> @@ -499,6 +613,9 @@ const std::map<PixelFormat, PixelFormatInfo> pixelFormatInfo{
>  		.bitsPerPixel = 10,
>  		.colourEncoding = PixelFormatInfo::ColourEncodingRAW,
>  		.packed = true,
> +		/* \todo remember to double this in the ipu3 pipeline handler */
> +		.pixelsPerGroup = 25,
> +		.planes = {{ { 32, 1 }, { 0, 0 }, { 0, 0 } }},
>  	} },
>  	{ formats::SGBRG10_IPU3, {
>  		.name = "SGBRG10_IPU3",
> @@ -507,6 +624,8 @@ const std::map<PixelFormat, PixelFormatInfo> pixelFormatInfo{
>  		.bitsPerPixel = 10,
>  		.colourEncoding = PixelFormatInfo::ColourEncodingRAW,
>  		.packed = true,
> +		.pixelsPerGroup = 25,
> +		.planes = {{ { 32, 1 }, { 0, 0 }, { 0, 0 } }},
>  	} },
>  	{ formats::SGRBG10_IPU3, {
>  		.name = "SGRBG10_IPU3",
> @@ -515,6 +634,8 @@ const std::map<PixelFormat, PixelFormatInfo> pixelFormatInfo{
>  		.bitsPerPixel = 10,
>  		.colourEncoding = PixelFormatInfo::ColourEncodingRAW,
>  		.packed = true,
> +		.pixelsPerGroup = 25,
> +		.planes = {{ { 32, 1 }, { 0, 0 }, { 0, 0 } }},
>  	} },
>  	{ formats::SRGGB10_IPU3, {
>  		.name = "SRGGB10_IPU3",
> @@ -523,16 +644,21 @@ const std::map<PixelFormat, PixelFormatInfo> pixelFormatInfo{
>  		.bitsPerPixel = 10,
>  		.colourEncoding = PixelFormatInfo::ColourEncodingRAW,
>  		.packed = true,
> +		.pixelsPerGroup = 25,
> +		.planes = {{ { 32, 1 }, { 0, 0 }, { 0, 0 } }},
>  	} },
>  
>  	/* Compressed formats. */
> +	/* \todo remember to fill this in from the pipeline handler. */

I don't think this comment is needed, the information here won't be
filled by the pipeline handlers.

>  	{ formats::MJPEG, {
>  		.name = "MJPEG",
>  		.format = formats::MJPEG,
>  		.v4l2Format = V4L2PixelFormat(V4L2_PIX_FMT_MJPEG),
> -		.bitsPerPixel = 0,
> +		.bitsPerPixel = 16,

I'd still set this to 0, it's not valid for MJPEG.

Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>

>  		.colourEncoding = PixelFormatInfo::ColourEncodingYUV,
>  		.packed = false,
> +		.pixelsPerGroup = 1,
> +		.planes = {{ { 1, 1 }, { 0, 0 }, { 0, 0 } }},
>  	} },
>  };
>

Hi Paul,

On Sat, Jul 04, 2020 at 10:31:20PM +0900, Paul Elder wrote:
> Packed formats make it difficult to calculate stride as well as
> frame size with the fields that PixelFormatInfo currently has.
> bitsPerPixel is defined as the average number of bits per pixel, and
> only counts effective bits, so it is not useful for calculating
> stride and frame size.
>
> To fix this, we introduce a concept of a "pixel group". The size of this
> group is defined as the minimum number of pixels (including padding)
> necessary in a row when the image has only one column of effective
> pixels. The pixel group has one more attribute, that is the "bytes per
> group". This determines how many bytes one pixel group consumes. These
> are the fields pixelsPerGroup and bytesPerGroup that are defined in this
> patch. Defining these two values makes it really simple to calculate
> bytes-per-line, as ceil(width / pixelsPerGroup) * bytesPerGroup, where
> width is measured in number of pixels. The ceiling accounts for padding.

How does ceiling accounts for padding ? this depends on the format
defined byte alignement. There's no guarantee 'the next biggest
integer' is enough for that, am I wrong ?

>
> Clearly, pixelsPerGroup must be constant for all planes in the format.
> The bytesPerGroup then, must be a per-plane attribute. There is one more
> field, verticalSubSampling, that is per-plane. This is simply a divider,
> to divide the number of rows of pixels by the sub-sampling value, to
> obtain the number of rows of pixels for the subsampled plane.
>
> For example, for something simple like BGR888, it is self-explanatory:
> the pixel group size is 1, and the bytes necessary is 3, and there is
> only one plane with no (= 1) vertical subsampling. For YUYV, the
> CbCr pair is shared between two pixels, so even if you have only one
> pixel, you would still need a padded second Y, therefore the pixel

s/padded second Y/second sample ?

> group size is 2, and bytes necessary is 4 (as opposed to 1 and 2). YUYV

what do you mean with "as opposed to" ?

> also has no vertical subsampling. NV12 has a pixel group size of 2

Assuming I got this right, YUYV has a single plane, hence no
sub-sampling. Is it true that sub-sampling applies only to the
additional planes, and formats with a single plane always have 1 as
sub-sampling value ?

> pixels, due to the CbCr plane. The bytes per group then, for both
> planes, is 2. The first plane has no vertical subsampling, but the
> second plane is subsampled by a factor of 2.
> The IPU3 formats are also self-explanatory, as they are single-planar,
> and have a pixel group size of 25, consuming 32 bytes. Although a

The IPU3 formats pack pixels in 32 bytes as you said, but how does
this play with the definition of

"The size of [a pixel group]  is defined as the minimum number of pixels
(including padding) necessary in a row when the image has only one column
of effective pixels" ?

I'm mostly missing what you mean by "necessary" here.

I understand how tricky it is to find a way to express what you're
here describing, and I don't have much better wording for that.

For other formats is clear: a pixel group is the minium number of
samples that are required to re-construct one pixel. Sticking to your
examples, for RGB888 a pixel group is 1 sample, of 24bits each.
For YUYV you need two samples to reconstruct a pixel, and each sample
is 16 bits in size. For formats like the IPU3 to reconstruct 1 pixel
you actually need a single sample of 30 bits each (those formats are
equivalent to S[GGBR]10 in sample size and sample ordering). The
requirement to pack 25 pixels in 32bytes is the due to how they pack
one 'tile' of pixels in a byte-aligned memory area.

I know 'tile' has a precise meaing for, say, display formats which do
not apply in the context of video formats (for my understanding) but
would this be a better term to use in place of "pixel group" ?
I would leave out any attempt to try to define the number of samples
per tile in terms of re-constructed pixels, as for the IPU3 format,
that does not apply well.

In example:

-------------------------------------------------------------------------------
To fix this, we introduce a concept of a "memory tile". A tile is
composed by a number of samples (which are combined to re-construct
pixels color informations) defined by the image format, and a
tile size in bytes which includes any necessay padding bits to align
the effective color information samples to a boundary defined
by the format.

The number of samples per tile is reported by the samplesPerTile
attribute, while the overall tile size by the bytesPerTile one.

By using those two format-defined values, calculating the number of
bytes per line becomes trival as in:

        bytesPerLine = (width / samplesPerTile) * bytesPerTile

For RAW formats such as RAW888 the number of samples is 3, with a size
of 1 bytes each. Each tile is then 3 bytes in size, and the number of
bytes per line is simply the number of pixels

        bytesPerLine = (width / 3) * 3 = width

In RAW10 formats each tile is composed of 3 samples (one per colour
information) and each sample is composed by 10-bits of effective color
information and 6 padding bits. A tile has a bytes size of 6 and the
line size is calculated as

        bytesPerLine = (width / 3) * 6 = 2width

For YUYV formats, each tile has a size of two samples, with each
sample being 2 bytes in size. The tile size is then 4 bytes and the
line width is calculated as

        bytesPerLine = (width / 2) * 4 = 2width

Other formats, mostly the platform specific ones, define the number of
samples per tile in order to maximize the format space efficiency or
to align samples to boundaries defined by the platform image
processing components.

For example, the IPU3 platform defines custom RAW10 format where 25
samples are packed in a tile of 32 bytes in size, to maximize the
format space efficiency and to align tiles to the size requirements of
its ISP image processing components. The full line width in bytes
can be calculated as well with

        bytesPerLine = width * (32/25)
-------------------------------------------------------------------------------

I'm not sure if this could be a commit message or part of the
documentation. It tries to express the same concepts that you
expressed but removes the pixel re-construction process from the
definition, as the size of a 'group' (or 'tile') does not only depends
on that.


> comment in the driver suggests that it should be 50 and 64,
> respectively, this is an attribute of the driver, and not the format, so
> this shall be set by the ipu3 pipeline handler.
>
> Signed-off-by: Paul Elder <paul.elder@ideasonboard.com>
>
> ---
> Changes in v3:
> - add planes
> - redefine the parameters for the formats
>   - pixelsPerGroup is for whole format
>   - add verticalSubSampling per plane
>
> Changes in v2:
> - add documentation for bytesPerGroup pixelsPerGroup
> - fix wording in commit message
>   - bytes-per-line -> stride
>   - buffer size -> frame size
> - changed MJPEG todo to allowing pipeline handlers to set parameters of
>   format infos
> ---
>  include/libcamera/internal/formats.h |  11 ++-
>  src/libcamera/formats.cpp            | 128 ++++++++++++++++++++++++++-
>  2 files changed, 137 insertions(+), 2 deletions(-)
>
> diff --git a/include/libcamera/internal/formats.h b/include/libcamera/internal/formats.h
> index f59ac8f..ebd256c 100644
> --- a/include/libcamera/internal/formats.h
> +++ b/include/libcamera/internal/formats.h
> @@ -32,6 +32,12 @@ private:
>  	std::map<unsigned int, std::vector<SizeRange>> data_;
>  };
>
> +struct PixelFormatPlane
> +{
> +	unsigned int bytesPerGroup;
> +	unsigned int verticalSubSampling;
> +};
> +
>  class PixelFormatInfo
>  {
>  public:
> @@ -45,13 +51,16 @@ public:
>
>  	static const PixelFormatInfo &info(const PixelFormat &format);
>
> -	/* \todo Add support for non-contiguous memory planes */

I wish we could remove this :)

>  	const char *name;
>  	PixelFormat format;
>  	V4L2PixelFormat v4l2Format;
>  	unsigned int bitsPerPixel;
>  	enum ColourEncoding colourEncoding;
>  	bool packed;
> +
> +	unsigned int pixelsPerGroup;
> +
> +	std::array<PixelFormatPlane, 3> planes;
>  };
>
>  } /* namespace libcamera */
> diff --git a/src/libcamera/formats.cpp b/src/libcamera/formats.cpp
> index d3b722c..6bdd28d 100644
> --- a/src/libcamera/formats.cpp
> +++ b/src/libcamera/formats.cpp
> @@ -110,6 +110,22 @@ const std::map<unsigned int, std::vector<SizeRange>> &ImageFormats::data() const
>  	return data_;
>  }
>
> +/**
> + * \class PixelFormatPlane
> + * \brief Information about a single plane of a pixel format
> + *
> + * \var PixelFormatPlane::bytesPerGroup
> + * \brief The number of bytes that a pixel group consumes
> + *
> + * \sa PixelFormatInfo::pixelsPerGroup
> + *
> + * \var PixelFormatPlane::verticalSubSampling
> + * \brief Vertical subsampling multiplier
> + *
> + * This value is the ratio between the number of rows of pixels in the frame
> + * to the number of rows of pixels in the plane.
> + */
> +
>  /**
>   * \class PixelFormatInfo
>   * \brief Information about pixel formats
> @@ -152,6 +168,26 @@ const std::map<unsigned int, std::vector<SizeRange>> &ImageFormats::data() const
>   * bytes. For instance, 12-bit Bayer data with two pixels stored in three bytes
>   * is packed, while the same data stored with 4 bits of padding in two bytes
>   * per pixel is not packed.
> + *
> + * \var PixelFormatInfo::pixelsPerGroup
> + * \brief The number of pixels in a pixel group
> + *
> + * The minimum number of pixels (including padding) necessary in a row
> + * when the frame has only one column of effective pixels
> + *
> + * A pixel group is defined as the minimum number of pixels (including padding)
> + * necessary in a row when the image has only one column of effective pixels.
> + * pixelsPerGroup refers to this value. bytesPerGroup, then, refers to the
> + * number of bytes that a pixel group consumes. This definition of a pixel
> + * group allows simple calculation of stride, as
> + * ceil(width / pixelsPerGroup) * bytesPerGroup. These values are determined
> + * only in terms of a row, and include bytes that are used in all planes (for
> + * multiplanar formats).
> + *
> + * \var PixelFormatInfo::planes
> + * \brief Information about pixels for each plane
> + *
> + * \sa PixelFormatPlane
>   */
>
>  /**
> @@ -179,6 +215,8 @@ const std::map<PixelFormat, PixelFormatInfo> pixelFormatInfo{
>  		.bitsPerPixel = 24,
>  		.colourEncoding = PixelFormatInfo::ColourEncodingRGB,
>  		.packed = false,
> +		.pixelsPerGroup = 1,
> +		.planes = {{ { 3, 1 }, { 0, 0 }, { 0, 0 } }},
>  	} },
>  	{ formats::RGB888, {
>  		.name = "RGB888",
> @@ -187,6 +225,8 @@ const std::map<PixelFormat, PixelFormatInfo> pixelFormatInfo{
>  		.bitsPerPixel = 24,
>  		.colourEncoding = PixelFormatInfo::ColourEncodingRGB,
>  		.packed = false,
> +		.pixelsPerGroup = 1,
> +		.planes = {{ { 3, 1 }, { 0, 0 }, { 0, 0 } }},
>  	} },
>  	{ formats::ABGR8888, {
>  		.name = "ABGR8888",
> @@ -195,6 +235,8 @@ const std::map<PixelFormat, PixelFormatInfo> pixelFormatInfo{
>  		.bitsPerPixel = 32,
>  		.colourEncoding = PixelFormatInfo::ColourEncodingRGB,
>  		.packed = false,
> +		.pixelsPerGroup = 1,
> +		.planes = {{ { 4, 1 }, { 0, 0 }, { 0, 0 } }},
>  	} },
>  	{ formats::ARGB8888, {
>  		.name = "ARGB8888",
> @@ -203,6 +245,8 @@ const std::map<PixelFormat, PixelFormatInfo> pixelFormatInfo{
>  		.bitsPerPixel = 32,
>  		.colourEncoding = PixelFormatInfo::ColourEncodingRGB,
>  		.packed = false,
> +		.pixelsPerGroup = 1,
> +		.planes = {{ { 4, 1 }, { 0, 0 }, { 0, 0 } }},
>  	} },
>  	{ formats::BGRA8888, {
>  		.name = "BGRA8888",
> @@ -211,6 +255,8 @@ const std::map<PixelFormat, PixelFormatInfo> pixelFormatInfo{
>  		.bitsPerPixel = 32,
>  		.colourEncoding = PixelFormatInfo::ColourEncodingRGB,
>  		.packed = false,
> +		.pixelsPerGroup = 1,
> +		.planes = {{ { 4, 1 }, { 0, 0 }, { 0, 0 } }},
>  	} },
>  	{ formats::RGBA8888, {
>  		.name = "RGBA8888",
> @@ -219,6 +265,8 @@ const std::map<PixelFormat, PixelFormatInfo> pixelFormatInfo{
>  		.bitsPerPixel = 32,
>  		.colourEncoding = PixelFormatInfo::ColourEncodingRGB,
>  		.packed = false,
> +		.pixelsPerGroup = 1,
> +		.planes = {{ { 4, 1 }, { 0, 0 }, { 0, 0 } }},
>  	} },
>
>  	/* YUV packed formats. */
> @@ -229,6 +277,8 @@ const std::map<PixelFormat, PixelFormatInfo> pixelFormatInfo{
>  		.bitsPerPixel = 16,
>  		.colourEncoding = PixelFormatInfo::ColourEncodingYUV,
>  		.packed = false,
> +		.pixelsPerGroup = 2,
> +		.planes = {{ { 4, 1 }, { 0, 0 }, { 0, 0 } }},
>  	} },
>  	{ formats::YVYU, {
>  		.name = "YVYU",
> @@ -237,6 +287,8 @@ const std::map<PixelFormat, PixelFormatInfo> pixelFormatInfo{
>  		.bitsPerPixel = 16,
>  		.colourEncoding = PixelFormatInfo::ColourEncodingYUV,
>  		.packed = false,
> +		.pixelsPerGroup = 2,
> +		.planes = {{ { 4, 1 }, { 0, 0 }, { 0, 0 } }},
>  	} },
>  	{ formats::UYVY, {
>  		.name = "UYVY",
> @@ -245,6 +297,8 @@ const std::map<PixelFormat, PixelFormatInfo> pixelFormatInfo{
>  		.bitsPerPixel = 16,
>  		.colourEncoding = PixelFormatInfo::ColourEncodingYUV,
>  		.packed = false,
> +		.pixelsPerGroup = 2,
> +		.planes = {{ { 4, 1 }, { 0, 0 }, { 0, 0 } }},
>  	} },
>  	{ formats::VYUY, {
>  		.name = "VYUY",
> @@ -253,6 +307,8 @@ const std::map<PixelFormat, PixelFormatInfo> pixelFormatInfo{
>  		.bitsPerPixel = 16,
>  		.colourEncoding = PixelFormatInfo::ColourEncodingYUV,
>  		.packed = false,
> +		.pixelsPerGroup = 2,
> +		.planes = {{ { 4, 1 }, { 0, 0 }, { 0, 0 } }},
>  	} },
>
>  	/* YUV planar formats. */
> @@ -263,6 +319,8 @@ const std::map<PixelFormat, PixelFormatInfo> pixelFormatInfo{
>  		.bitsPerPixel = 12,
>  		.colourEncoding = PixelFormatInfo::ColourEncodingYUV,
>  		.packed = false,
> +		.pixelsPerGroup = 2,
> +		.planes = {{ { 2, 1 }, { 2, 2 }, { 0, 0 } }},
>  	} },
>  	{ formats::NV21, {
>  		.name = "NV21",
> @@ -271,6 +329,8 @@ const std::map<PixelFormat, PixelFormatInfo> pixelFormatInfo{
>  		.bitsPerPixel = 12,
>  		.colourEncoding = PixelFormatInfo::ColourEncodingYUV,
>  		.packed = false,
> +		.pixelsPerGroup = 2,
> +		.planes = {{ { 2, 1 }, { 2, 2 }, { 0, 0 } }},
>  	} },
>  	{ formats::NV16, {
>  		.name = "NV16",
> @@ -279,6 +339,8 @@ const std::map<PixelFormat, PixelFormatInfo> pixelFormatInfo{
>  		.bitsPerPixel = 16,
>  		.colourEncoding = PixelFormatInfo::ColourEncodingYUV,
>  		.packed = false,
> +		.pixelsPerGroup = 2,
> +		.planes = {{ { 2, 1 }, { 2, 1 }, { 0, 0 } }},
>  	} },
>  	{ formats::NV61, {
>  		.name = "NV61",
> @@ -287,6 +349,8 @@ const std::map<PixelFormat, PixelFormatInfo> pixelFormatInfo{
>  		.bitsPerPixel = 16,
>  		.colourEncoding = PixelFormatInfo::ColourEncodingYUV,
>  		.packed = false,
> +		.pixelsPerGroup = 2,
> +		.planes = {{ { 2, 1 }, { 2, 1 }, { 0, 0 } }},
>  	} },
>  	{ formats::NV24, {
>  		.name = "NV24",
> @@ -295,6 +359,8 @@ const std::map<PixelFormat, PixelFormatInfo> pixelFormatInfo{
>  		.bitsPerPixel = 24,
>  		.colourEncoding = PixelFormatInfo::ColourEncodingYUV,
>  		.packed = false,
> +		.pixelsPerGroup = 1,
> +		.planes = {{ { 1, 1 }, { 2, 1 }, { 0, 0 } }},
>  	} },
>  	{ formats::NV42, {
>  		.name = "NV42",
> @@ -303,6 +369,8 @@ const std::map<PixelFormat, PixelFormatInfo> pixelFormatInfo{
>  		.bitsPerPixel = 24,
>  		.colourEncoding = PixelFormatInfo::ColourEncodingYUV,
>  		.packed = false,
> +		.pixelsPerGroup = 1,
> +		.planes = {{ { 1, 1 }, { 2, 1 }, { 0, 0 } }},
>  	} },
>  	{ formats::YUV420, {
>  		.name = "YUV420",
> @@ -311,6 +379,8 @@ const std::map<PixelFormat, PixelFormatInfo> pixelFormatInfo{
>  		.bitsPerPixel = 12,
>  		.colourEncoding = PixelFormatInfo::ColourEncodingYUV,
>  		.packed = false,
> +		.pixelsPerGroup = 2,
> +		.planes = {{ { 2, 1 }, { 1, 2 }, { 1, 2 } }},
>  	} },
>  	{ formats::YUV422, {
>  		.name = "YUV422",
> @@ -319,6 +389,8 @@ const std::map<PixelFormat, PixelFormatInfo> pixelFormatInfo{
>  		.bitsPerPixel = 16,
>  		.colourEncoding = PixelFormatInfo::ColourEncodingYUV,
>  		.packed = false,
> +		.pixelsPerGroup = 2,
> +		.planes = {{ { 2, 1 }, { 1, 1 }, { 1, 1 } }},
>  	} },
>
>  	/* Greyscale formats. */
> @@ -329,6 +401,8 @@ const std::map<PixelFormat, PixelFormatInfo> pixelFormatInfo{
>  		.bitsPerPixel = 8,
>  		.colourEncoding = PixelFormatInfo::ColourEncodingYUV,
>  		.packed = false,
> +		.pixelsPerGroup = 1,
> +		.planes = {{ { 1, 1 }, { 0, 0 }, { 0, 0 } }},
>  	} },
>
>  	/* Bayer formats. */
> @@ -339,6 +413,8 @@ const std::map<PixelFormat, PixelFormatInfo> pixelFormatInfo{
>  		.bitsPerPixel = 8,
>  		.colourEncoding = PixelFormatInfo::ColourEncodingRAW,
>  		.packed = false,
> +		.pixelsPerGroup = 2,
> +		.planes = {{ { 2, 1 }, { 0, 0 }, { 0, 0 } }},
>  	} },
>  	{ formats::SGBRG8, {
>  		.name = "SGBRG8",
> @@ -347,6 +423,8 @@ const std::map<PixelFormat, PixelFormatInfo> pixelFormatInfo{
>  		.bitsPerPixel = 8,
>  		.colourEncoding = PixelFormatInfo::ColourEncodingRAW,
>  		.packed = false,
> +		.pixelsPerGroup = 2,
> +		.planes = {{ { 2, 1 }, { 0, 0 }, { 0, 0 } }},
>  	} },
>  	{ formats::SGRBG8, {
>  		.name = "SGRBG8",
> @@ -355,6 +433,8 @@ const std::map<PixelFormat, PixelFormatInfo> pixelFormatInfo{
>  		.bitsPerPixel = 8,
>  		.colourEncoding = PixelFormatInfo::ColourEncodingRAW,
>  		.packed = false,
> +		.pixelsPerGroup = 2,
> +		.planes = {{ { 2, 1 }, { 0, 0 }, { 0, 0 } }},
>  	} },
>  	{ formats::SRGGB8, {
>  		.name = "SRGGB8",
> @@ -363,6 +443,8 @@ const std::map<PixelFormat, PixelFormatInfo> pixelFormatInfo{
>  		.bitsPerPixel = 8,
>  		.colourEncoding = PixelFormatInfo::ColourEncodingRAW,
>  		.packed = false,
> +		.pixelsPerGroup = 2,
> +		.planes = {{ { 2, 1 }, { 0, 0 }, { 0, 0 } }},
>  	} },
>  	{ formats::SBGGR10, {
>  		.name = "SBGGR10",
> @@ -371,6 +453,8 @@ const std::map<PixelFormat, PixelFormatInfo> pixelFormatInfo{
>  		.bitsPerPixel = 10,
>  		.colourEncoding = PixelFormatInfo::ColourEncodingRAW,
>  		.packed = false,
> +		.pixelsPerGroup = 2,
> +		.planes = {{ { 4, 1 }, { 0, 0 }, { 0, 0 } }},
>  	} },
>  	{ formats::SGBRG10, {
>  		.name = "SGBRG10",
> @@ -379,6 +463,8 @@ const std::map<PixelFormat, PixelFormatInfo> pixelFormatInfo{
>  		.bitsPerPixel = 10,
>  		.colourEncoding = PixelFormatInfo::ColourEncodingRAW,
>  		.packed = false,
> +		.pixelsPerGroup = 2,
> +		.planes = {{ { 4, 1 }, { 0, 0 }, { 0, 0 } }},
>  	} },
>  	{ formats::SGRBG10, {
>  		.name = "SGRBG10",
> @@ -387,6 +473,8 @@ const std::map<PixelFormat, PixelFormatInfo> pixelFormatInfo{
>  		.bitsPerPixel = 10,
>  		.colourEncoding = PixelFormatInfo::ColourEncodingRAW,
>  		.packed = false,
> +		.pixelsPerGroup = 2,
> +		.planes = {{ { 4, 1 }, { 0, 0 }, { 0, 0 } }},
>  	} },
>  	{ formats::SRGGB10, {
>  		.name = "SRGGB10",
> @@ -395,6 +483,8 @@ const std::map<PixelFormat, PixelFormatInfo> pixelFormatInfo{
>  		.bitsPerPixel = 10,
>  		.colourEncoding = PixelFormatInfo::ColourEncodingRAW,
>  		.packed = false,
> +		.pixelsPerGroup = 2,
> +		.planes = {{ { 4, 1 }, { 0, 0 }, { 0, 0 } }},
>  	} },
>  	{ formats::SBGGR10_CSI2P, {
>  		.name = "SBGGR10_CSI2P",
> @@ -403,6 +493,8 @@ const std::map<PixelFormat, PixelFormatInfo> pixelFormatInfo{
>  		.bitsPerPixel = 10,
>  		.colourEncoding = PixelFormatInfo::ColourEncodingRAW,
>  		.packed = true,
> +		.pixelsPerGroup = 4,
> +		.planes = {{ { 5, 1 }, { 0, 0 }, { 0, 0 } }},
>  	} },
>  	{ formats::SGBRG10_CSI2P, {
>  		.name = "SGBRG10_CSI2P",
> @@ -411,6 +503,8 @@ const std::map<PixelFormat, PixelFormatInfo> pixelFormatInfo{
>  		.bitsPerPixel = 10,
>  		.colourEncoding = PixelFormatInfo::ColourEncodingRAW,
>  		.packed = true,
> +		.pixelsPerGroup = 4,
> +		.planes = {{ { 5, 1 }, { 0, 0 }, { 0, 0 } }},
>  	} },
>  	{ formats::SGRBG10_CSI2P, {
>  		.name = "SGRBG10_CSI2P",
> @@ -419,6 +513,8 @@ const std::map<PixelFormat, PixelFormatInfo> pixelFormatInfo{
>  		.bitsPerPixel = 10,
>  		.colourEncoding = PixelFormatInfo::ColourEncodingRAW,
>  		.packed = true,
> +		.pixelsPerGroup = 4,
> +		.planes = {{ { 5, 1 }, { 0, 0 }, { 0, 0 } }},
>  	} },
>  	{ formats::SRGGB10_CSI2P, {
>  		.name = "SRGGB10_CSI2P",
> @@ -427,6 +523,8 @@ const std::map<PixelFormat, PixelFormatInfo> pixelFormatInfo{
>  		.bitsPerPixel = 10,
>  		.colourEncoding = PixelFormatInfo::ColourEncodingRAW,
>  		.packed = true,
> +		.pixelsPerGroup = 4,
> +		.planes = {{ { 5, 1 }, { 0, 0 }, { 0, 0 } }},
>  	} },
>  	{ formats::SBGGR12, {
>  		.name = "SBGGR12",
> @@ -435,6 +533,8 @@ const std::map<PixelFormat, PixelFormatInfo> pixelFormatInfo{
>  		.bitsPerPixel = 12,
>  		.colourEncoding = PixelFormatInfo::ColourEncodingRAW,
>  		.packed = false,
> +		.pixelsPerGroup = 2,
> +		.planes = {{ { 4, 1 }, { 0, 0 }, { 0, 0 } }},
>  	} },
>  	{ formats::SGBRG12, {
>  		.name = "SGBRG12",
> @@ -443,6 +543,8 @@ const std::map<PixelFormat, PixelFormatInfo> pixelFormatInfo{
>  		.bitsPerPixel = 12,
>  		.colourEncoding = PixelFormatInfo::ColourEncodingRAW,
>  		.packed = false,
> +		.pixelsPerGroup = 2,
> +		.planes = {{ { 4, 1 }, { 0, 0 }, { 0, 0 } }},
>  	} },
>  	{ formats::SGRBG12, {
>  		.name = "SGRBG12",
> @@ -451,6 +553,8 @@ const std::map<PixelFormat, PixelFormatInfo> pixelFormatInfo{
>  		.bitsPerPixel = 12,
>  		.colourEncoding = PixelFormatInfo::ColourEncodingRAW,
>  		.packed = false,
> +		.pixelsPerGroup = 2,
> +		.planes = {{ { 4, 1 }, { 0, 0 }, { 0, 0 } }},
>  	} },
>  	{ formats::SRGGB12, {
>  		.name = "SRGGB12",
> @@ -459,6 +563,8 @@ const std::map<PixelFormat, PixelFormatInfo> pixelFormatInfo{
>  		.bitsPerPixel = 12,
>  		.colourEncoding = PixelFormatInfo::ColourEncodingRAW,
>  		.packed = false,
> +		.pixelsPerGroup = 2,
> +		.planes = {{ { 4, 1 }, { 0, 0 }, { 0, 0 } }},
>  	} },
>  	{ formats::SBGGR12_CSI2P, {
>  		.name = "SBGGR12_CSI2P",
> @@ -467,6 +573,8 @@ const std::map<PixelFormat, PixelFormatInfo> pixelFormatInfo{
>  		.bitsPerPixel = 12,
>  		.colourEncoding = PixelFormatInfo::ColourEncodingRAW,
>  		.packed = true,
> +		.pixelsPerGroup = 2,
> +		.planes = {{ { 3, 1 }, { 0, 0 }, { 0, 0 } }},
>  	} },
>  	{ formats::SGBRG12_CSI2P, {
>  		.name = "SGBRG12_CSI2P",
> @@ -475,6 +583,8 @@ const std::map<PixelFormat, PixelFormatInfo> pixelFormatInfo{
>  		.bitsPerPixel = 12,
>  		.colourEncoding = PixelFormatInfo::ColourEncodingRAW,
>  		.packed = true,
> +		.pixelsPerGroup = 2,
> +		.planes = {{ { 3, 1 }, { 0, 0 }, { 0, 0 } }},
>  	} },
>  	{ formats::SGRBG12_CSI2P, {
>  		.name = "SGRBG12_CSI2P",
> @@ -483,6 +593,8 @@ const std::map<PixelFormat, PixelFormatInfo> pixelFormatInfo{
>  		.bitsPerPixel = 12,
>  		.colourEncoding = PixelFormatInfo::ColourEncodingRAW,
>  		.packed = true,
> +		.pixelsPerGroup = 2,
> +		.planes = {{ { 3, 1 }, { 0, 0 }, { 0, 0 } }},
>  	} },
>  	{ formats::SRGGB12_CSI2P, {
>  		.name = "SRGGB12_CSI2P",
> @@ -491,6 +603,8 @@ const std::map<PixelFormat, PixelFormatInfo> pixelFormatInfo{
>  		.bitsPerPixel = 12,
>  		.colourEncoding = PixelFormatInfo::ColourEncodingRAW,
>  		.packed = true,
> +		.pixelsPerGroup = 2,
> +		.planes = {{ { 3, 1 }, { 0, 0 }, { 0, 0 } }},
>  	} },
>  	{ formats::SBGGR10_IPU3, {
>  		.name = "SBGGR10_IPU3",
> @@ -499,6 +613,9 @@ const std::map<PixelFormat, PixelFormatInfo> pixelFormatInfo{
>  		.bitsPerPixel = 10,
>  		.colourEncoding = PixelFormatInfo::ColourEncodingRAW,
>  		.packed = true,
> +		/* \todo remember to double this in the ipu3 pipeline handler */
> +		.pixelsPerGroup = 25,
> +		.planes = {{ { 32, 1 }, { 0, 0 }, { 0, 0 } }},
>  	} },
>  	{ formats::SGBRG10_IPU3, {
>  		.name = "SGBRG10_IPU3",
> @@ -507,6 +624,8 @@ const std::map<PixelFormat, PixelFormatInfo> pixelFormatInfo{
>  		.bitsPerPixel = 10,
>  		.colourEncoding = PixelFormatInfo::ColourEncodingRAW,
>  		.packed = true,
> +		.pixelsPerGroup = 25,
> +		.planes = {{ { 32, 1 }, { 0, 0 }, { 0, 0 } }},
>  	} },
>  	{ formats::SGRBG10_IPU3, {
>  		.name = "SGRBG10_IPU3",
> @@ -515,6 +634,8 @@ const std::map<PixelFormat, PixelFormatInfo> pixelFormatInfo{
>  		.bitsPerPixel = 10,
>  		.colourEncoding = PixelFormatInfo::ColourEncodingRAW,
>  		.packed = true,
> +		.pixelsPerGroup = 25,
> +		.planes = {{ { 32, 1 }, { 0, 0 }, { 0, 0 } }},
>  	} },
>  	{ formats::SRGGB10_IPU3, {
>  		.name = "SRGGB10_IPU3",
> @@ -523,16 +644,21 @@ const std::map<PixelFormat, PixelFormatInfo> pixelFormatInfo{
>  		.bitsPerPixel = 10,
>  		.colourEncoding = PixelFormatInfo::ColourEncodingRAW,
>  		.packed = true,
> +		.pixelsPerGroup = 25,
> +		.planes = {{ { 32, 1 }, { 0, 0 }, { 0, 0 } }},
>  	} },
>
>  	/* Compressed formats. */
> +	/* \todo remember to fill this in from the pipeline handler. */
>  	{ formats::MJPEG, {
>  		.name = "MJPEG",
>  		.format = formats::MJPEG,
>  		.v4l2Format = V4L2PixelFormat(V4L2_PIX_FMT_MJPEG),
> -		.bitsPerPixel = 0,
> +		.bitsPerPixel = 16,
>  		.colourEncoding = PixelFormatInfo::ColourEncodingYUV,
>  		.packed = false,
> +		.pixelsPerGroup = 1,
> +		.planes = {{ { 1, 1 }, { 0, 0 }, { 0, 0 } }},
>  	} },
>  };
>
> --
> 2.27.0
>
> _______________________________________________
> libcamera-devel mailing list
> libcamera-devel@lists.libcamera.org
> https://lists.libcamera.org/listinfo/libcamera-devel

Hi Jacopo,

On Mon, Jul 06, 2020 at 04:13:51PM +0200, Jacopo Mondi wrote:
> On Sat, Jul 04, 2020 at 10:31:20PM +0900, Paul Elder wrote:
> > Packed formats make it difficult to calculate stride as well as
> > frame size with the fields that PixelFormatInfo currently has.
> > bitsPerPixel is defined as the average number of bits per pixel, and
> > only counts effective bits, so it is not useful for calculating
> > stride and frame size.
> >
> > To fix this, we introduce a concept of a "pixel group". The size of this
> > group is defined as the minimum number of pixels (including padding)
> > necessary in a row when the image has only one column of effective
> > pixels. The pixel group has one more attribute, that is the "bytes per
> > group". This determines how many bytes one pixel group consumes. These
> > are the fields pixelsPerGroup and bytesPerGroup that are defined in this
> > patch. Defining these two values makes it really simple to calculate
> > bytes-per-line, as ceil(width / pixelsPerGroup) * bytesPerGroup, where
> > width is measured in number of pixels. The ceiling accounts for padding.
> 
> How does ceiling accounts for padding ? this depends on the format
> defined byte alignement. There's no guarantee 'the next biggest
> integer' is enough for that, am I wrong ?

Yes and no :-) There's an intrinsic alignment that this patch takes into
account. For instance SGRBG10_CSI2P has an intrinsic alignment of 5
bytes, as it stores 4 pixels in 5 bytes. On top of that, there's a
device-specific alignment, usually related to the implementation of the
DMA engine. Some devices won't have additional alignment constraints,
some will. I've pointed this out to Paul before, we agreed that pipeline
handlers need to be ultimately responsible for aligning the stride.
Still, PixelFormatInfo can help, computing the intrinsic alignment, and
letting the pipeline handler apply further constraints. Paul told me he
was considering adding an alignment parameter to the
PixelFormatInfo::stride() function (it should have a default value of
1). In the vast majority of cases this should be enough for pipeline
handlers, and if they need to take more exotic constraints into account,
they can always do so (either on top of the value returned by stride(),
or by computing the stride manually).

Would that make sense for you ?

> > Clearly, pixelsPerGroup must be constant for all planes in the format.
> > The bytesPerGroup then, must be a per-plane attribute. There is one more
> > field, verticalSubSampling, that is per-plane. This is simply a divider,
> > to divide the number of rows of pixels by the sub-sampling value, to
> > obtain the number of rows of pixels for the subsampled plane.
> >
> > For example, for something simple like BGR888, it is self-explanatory:
> > the pixel group size is 1, and the bytes necessary is 3, and there is
> > only one plane with no (= 1) vertical subsampling. For YUYV, the
> > CbCr pair is shared between two pixels, so even if you have only one
> > pixel, you would still need a padded second Y, therefore the pixel
> 
> s/padded second Y/second sample ?
> 
> > group size is 2, and bytes necessary is 4 (as opposed to 1 and 2). YUYV
> 
> what do you mean with "as opposed to" ?
> 
> > also has no vertical subsampling. NV12 has a pixel group size of 2
> 
> Assuming I got this right, YUYV has a single plane, hence no
> sub-sampling. Is it true that sub-sampling applies only to the
> additional planes, and formats with a single plane always have 1 as
> sub-sampling value ?

I think this makes sense. We can of course change this later if there's
a need to, when we discover new formats that can't fit nicely into the
format info scheme, or just if we figure out we need more information
for existing formats than what is being proposed here.

> > pixels, due to the CbCr plane. The bytes per group then, for both
> > planes, is 2. The first plane has no vertical subsampling, but the
> > second plane is subsampled by a factor of 2.
> > The IPU3 formats are also self-explanatory, as they are single-planar,
> > and have a pixel group size of 25, consuming 32 bytes. Although a
> 
> The IPU3 formats pack pixels in 32 bytes as you said, but how does
> this play with the definition of
> 
> "The size of [a pixel group]  is defined as the minimum number of pixels
> (including padding) necessary in a row when the image has only one column
> of effective pixels" ?
> 
> I'm mostly missing what you mean by "necessary" here.

I also had trouble with this. The idea if that if you have an image of
1xn pixels (a single column of effective pixels), the IPU3 packed format
will still write it as 25 pixels in 32 bytes, with the other 24 pixels
having unknown values (maybe they're set to 0x00, 0xff or a random
value, I don't know what the hardware does). The group size is thus 25
pixels and 32 bytes. After understanding this and re-reading the
documentation I found it quite clear, and wasn't sure how to express it
in a better way, but given that both of us failed to get it in the first
place, an additional effort may be required.

> I understand how tricky it is to find a way to express what you're
> here describing, and I don't have much better wording for that.
> 
> For other formats is clear: a pixel group is the minium number of
> samples that are required to re-construct one pixel. Sticking to your
> examples, for RGB888 a pixel group is 1 sample, of 24bits each.
> For YUYV you need two samples to reconstruct a pixel, and each sample
> is 16 bits in size. For formats like the IPU3 to reconstruct 1 pixel
> you actually need a single sample of 30 bits each (those formats are
> equivalent to S[GGBR]10 in sample size and sample ordering). The
> requirement to pack 25 pixels in 32bytes is the due to how they pack
> one 'tile' of pixels in a byte-aligned memory area.
> 
> I know 'tile' has a precise meaing for, say, display formats which do
> not apply in the context of video formats (for my understanding) but
> would this be a better term to use in place of "pixel group" ?

I'd rather avoid the word tile, exactly because of its meaning in
display contexts.

I initially thought we could define the pixel group as the minimum
number of pixels that can be stored in an integer number of bytes, but
that's not true for the IPU3 packed formats, as it stores 4 pixels in 5
bytes but adds an additional pixel stored in 2 bytes after 24 pixels.
Maybe we could somehow define the pixel group based on a minimal number
of pixels stored in an integer number of bytes with a repeating pattern
? A better wording is needed, but repeating pattern / periodicity may be
the key.

> I would leave out any attempt to try to define the number of samples
> per tile in terms of re-constructed pixels, as for the IPU3 format,
> that does not apply well.
> 
> In example:
> 
> -------------------------------------------------------------------------------
> To fix this, we introduce a concept of a "memory tile". A tile is
> composed by a number of samples (which are combined to re-construct
> pixels color informations) defined by the image format, and a
> tile size in bytes which includes any necessay padding bits to align
> the effective color information samples to a boundary defined
> by the format.
> 
> The number of samples per tile is reported by the samplesPerTile
> attribute, while the overall tile size by the bytesPerTile one.
> 
> By using those two format-defined values, calculating the number of
> bytes per line becomes trival as in:
> 
>         bytesPerLine = (width / samplesPerTile) * bytesPerTile
> 
> For RAW formats such as RAW888 the number of samples is 3, with a size
> of 1 bytes each. Each tile is then 3 bytes in size, and the number of
> bytes per line is simply the number of pixels
> 
>         bytesPerLine = (width / 3) * 3 = width
> 
> In RAW10 formats each tile is composed of 3 samples (one per colour
> information) and each sample is composed by 10-bits of effective color
> information and 6 padding bits. A tile has a bytes size of 6 and the
> line size is calculated as
> 
>         bytesPerLine = (width / 3) * 6 = 2width
> 
> For YUYV formats, each tile has a size of two samples, with each
> sample being 2 bytes in size. The tile size is then 4 bytes and the
> line width is calculated as
> 
>         bytesPerLine = (width / 2) * 4 = 2width
> 
> Other formats, mostly the platform specific ones, define the number of
> samples per tile in order to maximize the format space efficiency or
> to align samples to boundaries defined by the platform image
> processing components.
> 
> For example, the IPU3 platform defines custom RAW10 format where 25
> samples are packed in a tile of 32 bytes in size, to maximize the
> format space efficiency and to align tiles to the size requirements of
> its ISP image processing components. The full line width in bytes
> can be calculated as well with
> 
>         bytesPerLine = width * (32/25)
> -------------------------------------------------------------------------------
> 
> I'm not sure if this could be a commit message or part of the
> documentation. It tries to express the same concepts that you
> expressed but removes the pixel re-construction process from the
> definition, as the size of a 'group' (or 'tile') does not only depends
> on that.

A couple of examples in the documentation may be useful (especially to
explain the weird IPU3 constraints, I think that's the only format we
have where the periodicity isn't identical to the smallest number of
pixels stored in an integer number of bytes).

> > comment in the driver suggests that it should be 50 and 64,
> > respectively, this is an attribute of the driver, and not the format, so
> > this shall be set by the ipu3 pipeline handler.
> >
> > Signed-off-by: Paul Elder <paul.elder@ideasonboard.com>
> >
> > ---
> > Changes in v3:
> > - add planes
> > - redefine the parameters for the formats
> >   - pixelsPerGroup is for whole format
> >   - add verticalSubSampling per plane
> >
> > Changes in v2:
> > - add documentation for bytesPerGroup pixelsPerGroup
> > - fix wording in commit message
> >   - bytes-per-line -> stride
> >   - buffer size -> frame size
> > - changed MJPEG todo to allowing pipeline handlers to set parameters of
> >   format infos
> > ---
> >  include/libcamera/internal/formats.h |  11 ++-
> >  src/libcamera/formats.cpp            | 128 ++++++++++++++++++++++++++-
> >  2 files changed, 137 insertions(+), 2 deletions(-)
> >
> > diff --git a/include/libcamera/internal/formats.h b/include/libcamera/internal/formats.h
> > index f59ac8f..ebd256c 100644
> > --- a/include/libcamera/internal/formats.h
> > +++ b/include/libcamera/internal/formats.h
> > @@ -32,6 +32,12 @@ private:
> >  	std::map<unsigned int, std::vector<SizeRange>> data_;
> >  };
> >
> > +struct PixelFormatPlane
> > +{
> > +	unsigned int bytesPerGroup;
> > +	unsigned int verticalSubSampling;
> > +};
> > +
> >  class PixelFormatInfo
> >  {
> >  public:
> > @@ -45,13 +51,16 @@ public:
> >
> >  	static const PixelFormatInfo &info(const PixelFormat &format);
> >
> > -	/* \todo Add support for non-contiguous memory planes */
> 
> I wish we could remove this :)

We will, we will :-)

> >  	const char *name;
> >  	PixelFormat format;
> >  	V4L2PixelFormat v4l2Format;
> >  	unsigned int bitsPerPixel;
> >  	enum ColourEncoding colourEncoding;
> >  	bool packed;
> > +
> > +	unsigned int pixelsPerGroup;
> > +
> > +	std::array<PixelFormatPlane, 3> planes;
> >  };
> >
> >  } /* namespace libcamera */
> > diff --git a/src/libcamera/formats.cpp b/src/libcamera/formats.cpp
> > index d3b722c..6bdd28d 100644
> > --- a/src/libcamera/formats.cpp
> > +++ b/src/libcamera/formats.cpp
> > @@ -110,6 +110,22 @@ const std::map<unsigned int, std::vector<SizeRange>> &ImageFormats::data() const
> >  	return data_;
> >  }
> >
> > +/**
> > + * \class PixelFormatPlane
> > + * \brief Information about a single plane of a pixel format
> > + *
> > + * \var PixelFormatPlane::bytesPerGroup
> > + * \brief The number of bytes that a pixel group consumes
> > + *
> > + * \sa PixelFormatInfo::pixelsPerGroup
> > + *
> > + * \var PixelFormatPlane::verticalSubSampling
> > + * \brief Vertical subsampling multiplier
> > + *
> > + * This value is the ratio between the number of rows of pixels in the frame
> > + * to the number of rows of pixels in the plane.
> > + */
> > +
> >  /**
> >   * \class PixelFormatInfo
> >   * \brief Information about pixel formats
> > @@ -152,6 +168,26 @@ const std::map<unsigned int, std::vector<SizeRange>> &ImageFormats::data() const
> >   * bytes. For instance, 12-bit Bayer data with two pixels stored in three bytes
> >   * is packed, while the same data stored with 4 bits of padding in two bytes
> >   * per pixel is not packed.
> > + *
> > + * \var PixelFormatInfo::pixelsPerGroup
> > + * \brief The number of pixels in a pixel group
> > + *
> > + * The minimum number of pixels (including padding) necessary in a row
> > + * when the frame has only one column of effective pixels
> > + *
> > + * A pixel group is defined as the minimum number of pixels (including padding)
> > + * necessary in a row when the image has only one column of effective pixels.
> > + * pixelsPerGroup refers to this value. bytesPerGroup, then, refers to the
> > + * number of bytes that a pixel group consumes. This definition of a pixel
> > + * group allows simple calculation of stride, as
> > + * ceil(width / pixelsPerGroup) * bytesPerGroup. These values are determined
> > + * only in terms of a row, and include bytes that are used in all planes (for
> > + * multiplanar formats).
> > + *
> > + * \var PixelFormatInfo::planes
> > + * \brief Information about pixels for each plane
> > + *
> > + * \sa PixelFormatPlane
> >   */
> >
> >  /**
> > @@ -179,6 +215,8 @@ const std::map<PixelFormat, PixelFormatInfo> pixelFormatInfo{
> >  		.bitsPerPixel = 24,
> >  		.colourEncoding = PixelFormatInfo::ColourEncodingRGB,
> >  		.packed = false,
> > +		.pixelsPerGroup = 1,
> > +		.planes = {{ { 3, 1 }, { 0, 0 }, { 0, 0 } }},
> >  	} },
> >  	{ formats::RGB888, {
> >  		.name = "RGB888",
> > @@ -187,6 +225,8 @@ const std::map<PixelFormat, PixelFormatInfo> pixelFormatInfo{
> >  		.bitsPerPixel = 24,
> >  		.colourEncoding = PixelFormatInfo::ColourEncodingRGB,
> >  		.packed = false,
> > +		.pixelsPerGroup = 1,
> > +		.planes = {{ { 3, 1 }, { 0, 0 }, { 0, 0 } }},
> >  	} },
> >  	{ formats::ABGR8888, {
> >  		.name = "ABGR8888",
> > @@ -195,6 +235,8 @@ const std::map<PixelFormat, PixelFormatInfo> pixelFormatInfo{
> >  		.bitsPerPixel = 32,
> >  		.colourEncoding = PixelFormatInfo::ColourEncodingRGB,
> >  		.packed = false,
> > +		.pixelsPerGroup = 1,
> > +		.planes = {{ { 4, 1 }, { 0, 0 }, { 0, 0 } }},
> >  	} },
> >  	{ formats::ARGB8888, {
> >  		.name = "ARGB8888",
> > @@ -203,6 +245,8 @@ const std::map<PixelFormat, PixelFormatInfo> pixelFormatInfo{
> >  		.bitsPerPixel = 32,
> >  		.colourEncoding = PixelFormatInfo::ColourEncodingRGB,
> >  		.packed = false,
> > +		.pixelsPerGroup = 1,
> > +		.planes = {{ { 4, 1 }, { 0, 0 }, { 0, 0 } }},
> >  	} },
> >  	{ formats::BGRA8888, {
> >  		.name = "BGRA8888",
> > @@ -211,6 +255,8 @@ const std::map<PixelFormat, PixelFormatInfo> pixelFormatInfo{
> >  		.bitsPerPixel = 32,
> >  		.colourEncoding = PixelFormatInfo::ColourEncodingRGB,
> >  		.packed = false,
> > +		.pixelsPerGroup = 1,
> > +		.planes = {{ { 4, 1 }, { 0, 0 }, { 0, 0 } }},
> >  	} },
> >  	{ formats::RGBA8888, {
> >  		.name = "RGBA8888",
> > @@ -219,6 +265,8 @@ const std::map<PixelFormat, PixelFormatInfo> pixelFormatInfo{
> >  		.bitsPerPixel = 32,
> >  		.colourEncoding = PixelFormatInfo::ColourEncodingRGB,
> >  		.packed = false,
> > +		.pixelsPerGroup = 1,
> > +		.planes = {{ { 4, 1 }, { 0, 0 }, { 0, 0 } }},
> >  	} },
> >
> >  	/* YUV packed formats. */
> > @@ -229,6 +277,8 @@ const std::map<PixelFormat, PixelFormatInfo> pixelFormatInfo{
> >  		.bitsPerPixel = 16,
> >  		.colourEncoding = PixelFormatInfo::ColourEncodingYUV,
> >  		.packed = false,
> > +		.pixelsPerGroup = 2,
> > +		.planes = {{ { 4, 1 }, { 0, 0 }, { 0, 0 } }},
> >  	} },
> >  	{ formats::YVYU, {
> >  		.name = "YVYU",
> > @@ -237,6 +287,8 @@ const std::map<PixelFormat, PixelFormatInfo> pixelFormatInfo{
> >  		.bitsPerPixel = 16,
> >  		.colourEncoding = PixelFormatInfo::ColourEncodingYUV,
> >  		.packed = false,
> > +		.pixelsPerGroup = 2,
> > +		.planes = {{ { 4, 1 }, { 0, 0 }, { 0, 0 } }},
> >  	} },
> >  	{ formats::UYVY, {
> >  		.name = "UYVY",
> > @@ -245,6 +297,8 @@ const std::map<PixelFormat, PixelFormatInfo> pixelFormatInfo{
> >  		.bitsPerPixel = 16,
> >  		.colourEncoding = PixelFormatInfo::ColourEncodingYUV,
> >  		.packed = false,
> > +		.pixelsPerGroup = 2,
> > +		.planes = {{ { 4, 1 }, { 0, 0 }, { 0, 0 } }},
> >  	} },
> >  	{ formats::VYUY, {
> >  		.name = "VYUY",
> > @@ -253,6 +307,8 @@ const std::map<PixelFormat, PixelFormatInfo> pixelFormatInfo{
> >  		.bitsPerPixel = 16,
> >  		.colourEncoding = PixelFormatInfo::ColourEncodingYUV,
> >  		.packed = false,
> > +		.pixelsPerGroup = 2,
> > +		.planes = {{ { 4, 1 }, { 0, 0 }, { 0, 0 } }},
> >  	} },
> >
> >  	/* YUV planar formats. */
> > @@ -263,6 +319,8 @@ const std::map<PixelFormat, PixelFormatInfo> pixelFormatInfo{
> >  		.bitsPerPixel = 12,
> >  		.colourEncoding = PixelFormatInfo::ColourEncodingYUV,
> >  		.packed = false,
> > +		.pixelsPerGroup = 2,
> > +		.planes = {{ { 2, 1 }, { 2, 2 }, { 0, 0 } }},
> >  	} },
> >  	{ formats::NV21, {
> >  		.name = "NV21",
> > @@ -271,6 +329,8 @@ const std::map<PixelFormat, PixelFormatInfo> pixelFormatInfo{
> >  		.bitsPerPixel = 12,
> >  		.colourEncoding = PixelFormatInfo::ColourEncodingYUV,
> >  		.packed = false,
> > +		.pixelsPerGroup = 2,
> > +		.planes = {{ { 2, 1 }, { 2, 2 }, { 0, 0 } }},
> >  	} },
> >  	{ formats::NV16, {
> >  		.name = "NV16",
> > @@ -279,6 +339,8 @@ const std::map<PixelFormat, PixelFormatInfo> pixelFormatInfo{
> >  		.bitsPerPixel = 16,
> >  		.colourEncoding = PixelFormatInfo::ColourEncodingYUV,
> >  		.packed = false,
> > +		.pixelsPerGroup = 2,
> > +		.planes = {{ { 2, 1 }, { 2, 1 }, { 0, 0 } }},
> >  	} },
> >  	{ formats::NV61, {
> >  		.name = "NV61",
> > @@ -287,6 +349,8 @@ const std::map<PixelFormat, PixelFormatInfo> pixelFormatInfo{
> >  		.bitsPerPixel = 16,
> >  		.colourEncoding = PixelFormatInfo::ColourEncodingYUV,
> >  		.packed = false,
> > +		.pixelsPerGroup = 2,
> > +		.planes = {{ { 2, 1 }, { 2, 1 }, { 0, 0 } }},
> >  	} },
> >  	{ formats::NV24, {
> >  		.name = "NV24",
> > @@ -295,6 +359,8 @@ const std::map<PixelFormat, PixelFormatInfo> pixelFormatInfo{
> >  		.bitsPerPixel = 24,
> >  		.colourEncoding = PixelFormatInfo::ColourEncodingYUV,
> >  		.packed = false,
> > +		.pixelsPerGroup = 1,
> > +		.planes = {{ { 1, 1 }, { 2, 1 }, { 0, 0 } }},
> >  	} },
> >  	{ formats::NV42, {
> >  		.name = "NV42",
> > @@ -303,6 +369,8 @@ const std::map<PixelFormat, PixelFormatInfo> pixelFormatInfo{
> >  		.bitsPerPixel = 24,
> >  		.colourEncoding = PixelFormatInfo::ColourEncodingYUV,
> >  		.packed = false,
> > +		.pixelsPerGroup = 1,
> > +		.planes = {{ { 1, 1 }, { 2, 1 }, { 0, 0 } }},
> >  	} },
> >  	{ formats::YUV420, {
> >  		.name = "YUV420",
> > @@ -311,6 +379,8 @@ const std::map<PixelFormat, PixelFormatInfo> pixelFormatInfo{
> >  		.bitsPerPixel = 12,
> >  		.colourEncoding = PixelFormatInfo::ColourEncodingYUV,
> >  		.packed = false,
> > +		.pixelsPerGroup = 2,
> > +		.planes = {{ { 2, 1 }, { 1, 2 }, { 1, 2 } }},
> >  	} },
> >  	{ formats::YUV422, {
> >  		.name = "YUV422",
> > @@ -319,6 +389,8 @@ const std::map<PixelFormat, PixelFormatInfo> pixelFormatInfo{
> >  		.bitsPerPixel = 16,
> >  		.colourEncoding = PixelFormatInfo::ColourEncodingYUV,
> >  		.packed = false,
> > +		.pixelsPerGroup = 2,
> > +		.planes = {{ { 2, 1 }, { 1, 1 }, { 1, 1 } }},
> >  	} },
> >
> >  	/* Greyscale formats. */
> > @@ -329,6 +401,8 @@ const std::map<PixelFormat, PixelFormatInfo> pixelFormatInfo{
> >  		.bitsPerPixel = 8,
> >  		.colourEncoding = PixelFormatInfo::ColourEncodingYUV,
> >  		.packed = false,
> > +		.pixelsPerGroup = 1,
> > +		.planes = {{ { 1, 1 }, { 0, 0 }, { 0, 0 } }},
> >  	} },
> >
> >  	/* Bayer formats. */
> > @@ -339,6 +413,8 @@ const std::map<PixelFormat, PixelFormatInfo> pixelFormatInfo{
> >  		.bitsPerPixel = 8,
> >  		.colourEncoding = PixelFormatInfo::ColourEncodingRAW,
> >  		.packed = false,
> > +		.pixelsPerGroup = 2,
> > +		.planes = {{ { 2, 1 }, { 0, 0 }, { 0, 0 } }},
> >  	} },
> >  	{ formats::SGBRG8, {
> >  		.name = "SGBRG8",
> > @@ -347,6 +423,8 @@ const std::map<PixelFormat, PixelFormatInfo> pixelFormatInfo{
> >  		.bitsPerPixel = 8,
> >  		.colourEncoding = PixelFormatInfo::ColourEncodingRAW,
> >  		.packed = false,
> > +		.pixelsPerGroup = 2,
> > +		.planes = {{ { 2, 1 }, { 0, 0 }, { 0, 0 } }},
> >  	} },
> >  	{ formats::SGRBG8, {
> >  		.name = "SGRBG8",
> > @@ -355,6 +433,8 @@ const std::map<PixelFormat, PixelFormatInfo> pixelFormatInfo{
> >  		.bitsPerPixel = 8,
> >  		.colourEncoding = PixelFormatInfo::ColourEncodingRAW,
> >  		.packed = false,
> > +		.pixelsPerGroup = 2,
> > +		.planes = {{ { 2, 1 }, { 0, 0 }, { 0, 0 } }},
> >  	} },
> >  	{ formats::SRGGB8, {
> >  		.name = "SRGGB8",
> > @@ -363,6 +443,8 @@ const std::map<PixelFormat, PixelFormatInfo> pixelFormatInfo{
> >  		.bitsPerPixel = 8,
> >  		.colourEncoding = PixelFormatInfo::ColourEncodingRAW,
> >  		.packed = false,
> > +		.pixelsPerGroup = 2,
> > +		.planes = {{ { 2, 1 }, { 0, 0 }, { 0, 0 } }},
> >  	} },
> >  	{ formats::SBGGR10, {
> >  		.name = "SBGGR10",
> > @@ -371,6 +453,8 @@ const std::map<PixelFormat, PixelFormatInfo> pixelFormatInfo{
> >  		.bitsPerPixel = 10,
> >  		.colourEncoding = PixelFormatInfo::ColourEncodingRAW,
> >  		.packed = false,
> > +		.pixelsPerGroup = 2,
> > +		.planes = {{ { 4, 1 }, { 0, 0 }, { 0, 0 } }},
> >  	} },
> >  	{ formats::SGBRG10, {
> >  		.name = "SGBRG10",
> > @@ -379,6 +463,8 @@ const std::map<PixelFormat, PixelFormatInfo> pixelFormatInfo{
> >  		.bitsPerPixel = 10,
> >  		.colourEncoding = PixelFormatInfo::ColourEncodingRAW,
> >  		.packed = false,
> > +		.pixelsPerGroup = 2,
> > +		.planes = {{ { 4, 1 }, { 0, 0 }, { 0, 0 } }},
> >  	} },
> >  	{ formats::SGRBG10, {
> >  		.name = "SGRBG10",
> > @@ -387,6 +473,8 @@ const std::map<PixelFormat, PixelFormatInfo> pixelFormatInfo{
> >  		.bitsPerPixel = 10,
> >  		.colourEncoding = PixelFormatInfo::ColourEncodingRAW,
> >  		.packed = false,
> > +		.pixelsPerGroup = 2,
> > +		.planes = {{ { 4, 1 }, { 0, 0 }, { 0, 0 } }},
> >  	} },
> >  	{ formats::SRGGB10, {
> >  		.name = "SRGGB10",
> > @@ -395,6 +483,8 @@ const std::map<PixelFormat, PixelFormatInfo> pixelFormatInfo{
> >  		.bitsPerPixel = 10,
> >  		.colourEncoding = PixelFormatInfo::ColourEncodingRAW,
> >  		.packed = false,
> > +		.pixelsPerGroup = 2,
> > +		.planes = {{ { 4, 1 }, { 0, 0 }, { 0, 0 } }},
> >  	} },
> >  	{ formats::SBGGR10_CSI2P, {
> >  		.name = "SBGGR10_CSI2P",
> > @@ -403,6 +493,8 @@ const std::map<PixelFormat, PixelFormatInfo> pixelFormatInfo{
> >  		.bitsPerPixel = 10,
> >  		.colourEncoding = PixelFormatInfo::ColourEncodingRAW,
> >  		.packed = true,
> > +		.pixelsPerGroup = 4,
> > +		.planes = {{ { 5, 1 }, { 0, 0 }, { 0, 0 } }},
> >  	} },
> >  	{ formats::SGBRG10_CSI2P, {
> >  		.name = "SGBRG10_CSI2P",
> > @@ -411,6 +503,8 @@ const std::map<PixelFormat, PixelFormatInfo> pixelFormatInfo{
> >  		.bitsPerPixel = 10,
> >  		.colourEncoding = PixelFormatInfo::ColourEncodingRAW,
> >  		.packed = true,
> > +		.pixelsPerGroup = 4,
> > +		.planes = {{ { 5, 1 }, { 0, 0 }, { 0, 0 } }},
> >  	} },
> >  	{ formats::SGRBG10_CSI2P, {
> >  		.name = "SGRBG10_CSI2P",
> > @@ -419,6 +513,8 @@ const std::map<PixelFormat, PixelFormatInfo> pixelFormatInfo{
> >  		.bitsPerPixel = 10,
> >  		.colourEncoding = PixelFormatInfo::ColourEncodingRAW,
> >  		.packed = true,
> > +		.pixelsPerGroup = 4,
> > +		.planes = {{ { 5, 1 }, { 0, 0 }, { 0, 0 } }},
> >  	} },
> >  	{ formats::SRGGB10_CSI2P, {
> >  		.name = "SRGGB10_CSI2P",
> > @@ -427,6 +523,8 @@ const std::map<PixelFormat, PixelFormatInfo> pixelFormatInfo{
> >  		.bitsPerPixel = 10,
> >  		.colourEncoding = PixelFormatInfo::ColourEncodingRAW,
> >  		.packed = true,
> > +		.pixelsPerGroup = 4,
> > +		.planes = {{ { 5, 1 }, { 0, 0 }, { 0, 0 } }},
> >  	} },
> >  	{ formats::SBGGR12, {
> >  		.name = "SBGGR12",
> > @@ -435,6 +533,8 @@ const std::map<PixelFormat, PixelFormatInfo> pixelFormatInfo{
> >  		.bitsPerPixel = 12,
> >  		.colourEncoding = PixelFormatInfo::ColourEncodingRAW,
> >  		.packed = false,
> > +		.pixelsPerGroup = 2,
> > +		.planes = {{ { 4, 1 }, { 0, 0 }, { 0, 0 } }},
> >  	} },
> >  	{ formats::SGBRG12, {
> >  		.name = "SGBRG12",
> > @@ -443,6 +543,8 @@ const std::map<PixelFormat, PixelFormatInfo> pixelFormatInfo{
> >  		.bitsPerPixel = 12,
> >  		.colourEncoding = PixelFormatInfo::ColourEncodingRAW,
> >  		.packed = false,
> > +		.pixelsPerGroup = 2,
> > +		.planes = {{ { 4, 1 }, { 0, 0 }, { 0, 0 } }},
> >  	} },
> >  	{ formats::SGRBG12, {
> >  		.name = "SGRBG12",
> > @@ -451,6 +553,8 @@ const std::map<PixelFormat, PixelFormatInfo> pixelFormatInfo{
> >  		.bitsPerPixel = 12,
> >  		.colourEncoding = PixelFormatInfo::ColourEncodingRAW,
> >  		.packed = false,
> > +		.pixelsPerGroup = 2,
> > +		.planes = {{ { 4, 1 }, { 0, 0 }, { 0, 0 } }},
> >  	} },
> >  	{ formats::SRGGB12, {
> >  		.name = "SRGGB12",
> > @@ -459,6 +563,8 @@ const std::map<PixelFormat, PixelFormatInfo> pixelFormatInfo{
> >  		.bitsPerPixel = 12,
> >  		.colourEncoding = PixelFormatInfo::ColourEncodingRAW,
> >  		.packed = false,
> > +		.pixelsPerGroup = 2,
> > +		.planes = {{ { 4, 1 }, { 0, 0 }, { 0, 0 } }},
> >  	} },
> >  	{ formats::SBGGR12_CSI2P, {
> >  		.name = "SBGGR12_CSI2P",
> > @@ -467,6 +573,8 @@ const std::map<PixelFormat, PixelFormatInfo> pixelFormatInfo{
> >  		.bitsPerPixel = 12,
> >  		.colourEncoding = PixelFormatInfo::ColourEncodingRAW,
> >  		.packed = true,
> > +		.pixelsPerGroup = 2,
> > +		.planes = {{ { 3, 1 }, { 0, 0 }, { 0, 0 } }},
> >  	} },
> >  	{ formats::SGBRG12_CSI2P, {
> >  		.name = "SGBRG12_CSI2P",
> > @@ -475,6 +583,8 @@ const std::map<PixelFormat, PixelFormatInfo> pixelFormatInfo{
> >  		.bitsPerPixel = 12,
> >  		.colourEncoding = PixelFormatInfo::ColourEncodingRAW,
> >  		.packed = true,
> > +		.pixelsPerGroup = 2,
> > +		.planes = {{ { 3, 1 }, { 0, 0 }, { 0, 0 } }},
> >  	} },
> >  	{ formats::SGRBG12_CSI2P, {
> >  		.name = "SGRBG12_CSI2P",
> > @@ -483,6 +593,8 @@ const std::map<PixelFormat, PixelFormatInfo> pixelFormatInfo{
> >  		.bitsPerPixel = 12,
> >  		.colourEncoding = PixelFormatInfo::ColourEncodingRAW,
> >  		.packed = true,
> > +		.pixelsPerGroup = 2,
> > +		.planes = {{ { 3, 1 }, { 0, 0 }, { 0, 0 } }},
> >  	} },
> >  	{ formats::SRGGB12_CSI2P, {
> >  		.name = "SRGGB12_CSI2P",
> > @@ -491,6 +603,8 @@ const std::map<PixelFormat, PixelFormatInfo> pixelFormatInfo{
> >  		.bitsPerPixel = 12,
> >  		.colourEncoding = PixelFormatInfo::ColourEncodingRAW,
> >  		.packed = true,
> > +		.pixelsPerGroup = 2,
> > +		.planes = {{ { 3, 1 }, { 0, 0 }, { 0, 0 } }},
> >  	} },
> >  	{ formats::SBGGR10_IPU3, {
> >  		.name = "SBGGR10_IPU3",
> > @@ -499,6 +613,9 @@ const std::map<PixelFormat, PixelFormatInfo> pixelFormatInfo{
> >  		.bitsPerPixel = 10,
> >  		.colourEncoding = PixelFormatInfo::ColourEncodingRAW,
> >  		.packed = true,
> > +		/* \todo remember to double this in the ipu3 pipeline handler */
> > +		.pixelsPerGroup = 25,
> > +		.planes = {{ { 32, 1 }, { 0, 0 }, { 0, 0 } }},
> >  	} },
> >  	{ formats::SGBRG10_IPU3, {
> >  		.name = "SGBRG10_IPU3",
> > @@ -507,6 +624,8 @@ const std::map<PixelFormat, PixelFormatInfo> pixelFormatInfo{
> >  		.bitsPerPixel = 10,
> >  		.colourEncoding = PixelFormatInfo::ColourEncodingRAW,
> >  		.packed = true,
> > +		.pixelsPerGroup = 25,
> > +		.planes = {{ { 32, 1 }, { 0, 0 }, { 0, 0 } }},
> >  	} },
> >  	{ formats::SGRBG10_IPU3, {
> >  		.name = "SGRBG10_IPU3",
> > @@ -515,6 +634,8 @@ const std::map<PixelFormat, PixelFormatInfo> pixelFormatInfo{
> >  		.bitsPerPixel = 10,
> >  		.colourEncoding = PixelFormatInfo::ColourEncodingRAW,
> >  		.packed = true,
> > +		.pixelsPerGroup = 25,
> > +		.planes = {{ { 32, 1 }, { 0, 0 }, { 0, 0 } }},
> >  	} },
> >  	{ formats::SRGGB10_IPU3, {
> >  		.name = "SRGGB10_IPU3",
> > @@ -523,16 +644,21 @@ const std::map<PixelFormat, PixelFormatInfo> pixelFormatInfo{
> >  		.bitsPerPixel = 10,
> >  		.colourEncoding = PixelFormatInfo::ColourEncodingRAW,
> >  		.packed = true,
> > +		.pixelsPerGroup = 25,
> > +		.planes = {{ { 32, 1 }, { 0, 0 }, { 0, 0 } }},
> >  	} },
> >
> >  	/* Compressed formats. */
> > +	/* \todo remember to fill this in from the pipeline handler. */
> >  	{ formats::MJPEG, {
> >  		.name = "MJPEG",
> >  		.format = formats::MJPEG,
> >  		.v4l2Format = V4L2PixelFormat(V4L2_PIX_FMT_MJPEG),
> > -		.bitsPerPixel = 0,
> > +		.bitsPerPixel = 16,
> >  		.colourEncoding = PixelFormatInfo::ColourEncodingYUV,
> >  		.packed = false,
> > +		.pixelsPerGroup = 1,
> > +		.planes = {{ { 1, 1 }, { 0, 0 }, { 0, 0 } }},
> >  	} },
> >  };
> >