[libcamera-devel,v5,02/23] libcamera: formats: Add fields to info to ease calculating stride

8717 diff mbox series

Packed formats make it difficult to calculate stride as well as
frame size with the fields that PixelFormatInfo currently has.
bitsPerPixel is defined as the average number of bits per pixel, and
only counts effective bits, so it is not useful for calculating
stride and frame size.

To fix this, we introduce a concept of a "pixel group". The size of this
group is defined as the minimum number of pixels (including padding)
necessary in a row when the image has only one column of effective
pixels. The pixel group has one more attribute, that is the "bytes per
group". This determines how many bytes one pixel group consumes. These
are the fields pixelsPerGroup and bytesPerGroup that are defined in this
patch. Defining these two values makes it really simple to calculate
bytes-per-line, as ceil(width / pixelsPerGroup) * bytesPerGroup, where
width is measured in number of pixels. The ceiling accounts for padding.

The pixel group has another contraint, which is that the pixel group
(bytesPerGroup and pixelsPerGroup) is the smallest repeatable unit. What
this means is that, for example, in the IPU3 formats, if there is only
one column of effective pixels, it looks like it could be fit in 5 bytes
with 3 padding pixels (for a total of 4 pixels over 5 bytes). However,
this unit is not repeatable, as at the 7th group in the same row, the
pattern is broken. Therefore, the pixel group for IPU3 formats must be
25 pixels over 32 bytes.

Clearly, pixelsPerGroup must be constant for all planes in the format.
The bytesPerGroup then, must be a per-plane attribute. There is one more
field, verticalSubSampling, that is per-plane. This is simply a divider,
to divide the number of rows of pixels by the sub-sampling value, to
obtain the number of rows of pixels for the subsampled plane.

For example, for something simple like BGR888, it is self-explanatory:
the pixel group size is 1, and the bytes necessary is 3, and there is
only one plane with no (= 1) vertical subsampling. For YUYV, the
CbCr pair is shared between two pixels, so even if you have only one
pixel, you would still need a padded second Y, therefore the pixel
group size is 2, and bytes necessary is 4 (as opposed to 1 and 2). YUYV
also has no vertical subsampling. NV12 has a pixel group size of 2
pixels, due to the CbCr plane. The bytes per group then, for both
planes, is 2. The first plane has no vertical subsampling, but the
second plane is subsampled by a factor of 2.
The IPU3 formats are also self-explanatory, as they are single-planar,
and have a pixel group size of 25, consuming 32 bytes. Although a
comment in the driver suggests that it should be 50 and 64,
respectively, this is an attribute of the driver, and not the format, so
this shall be set by the ipu3 pipeline handler.

Signed-off-by: Paul Elder <paul.elder@ideasonboard.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Jacopo Mondi <jacopo@jmondi.org>
Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>

---
Changes in v5:
- add #include <array> to fix cros compile error

Changes in v4:
- expanded documentation and definition of pixel group

Changes in v3:
- add planes
- redefine the parameters for the formats
  - pixelsPerGroup is for whole format
  - add verticalSubSampling per plane

Changes in v2:
- add documentation for bytesPerGroup pixelsPerGroup
- fix wording in commit message
  - bytes-per-line -> stride
  - buffer size -> frame size
- changed MJPEG todo to allowing pipeline handlers to set parameters of
  format infos
---
 include/libcamera/internal/formats.h |  11 ++
 src/libcamera/formats.cpp            | 148 +++++++++++++++++++++++++++
 2 files changed, 159 insertions(+)

Hi Paul,

Thank you for the patch.

On Thu, Jul 09, 2020 at 10:28:14PM +0900, Paul Elder wrote:
> Packed formats make it difficult to calculate stride as well as
> frame size with the fields that PixelFormatInfo currently has.
> bitsPerPixel is defined as the average number of bits per pixel, and
> only counts effective bits, so it is not useful for calculating
> stride and frame size.
> 
> To fix this, we introduce a concept of a "pixel group". The size of this
> group is defined as the minimum number of pixels (including padding)
> necessary in a row when the image has only one column of effective
> pixels. The pixel group has one more attribute, that is the "bytes per
> group". This determines how many bytes one pixel group consumes. These
> are the fields pixelsPerGroup and bytesPerGroup that are defined in this
> patch. Defining these two values makes it really simple to calculate
> bytes-per-line, as ceil(width / pixelsPerGroup) * bytesPerGroup, where
> width is measured in number of pixels. The ceiling accounts for padding.
> 
> The pixel group has another contraint, which is that the pixel group
> (bytesPerGroup and pixelsPerGroup) is the smallest repeatable unit. What
> this means is that, for example, in the IPU3 formats, if there is only
> one column of effective pixels, it looks like it could be fit in 5 bytes
> with 3 padding pixels (for a total of 4 pixels over 5 bytes). However,
> this unit is not repeatable, as at the 7th group in the same row, the
> pattern is broken. Therefore, the pixel group for IPU3 formats must be
> 25 pixels over 32 bytes.
> 
> Clearly, pixelsPerGroup must be constant for all planes in the format.
> The bytesPerGroup then, must be a per-plane attribute. There is one more
> field, verticalSubSampling, that is per-plane. This is simply a divider,
> to divide the number of rows of pixels by the sub-sampling value, to
> obtain the number of rows of pixels for the subsampled plane.
> 
> For example, for something simple like BGR888, it is self-explanatory:
> the pixel group size is 1, and the bytes necessary is 3, and there is
> only one plane with no (= 1) vertical subsampling. For YUYV, the
> CbCr pair is shared between two pixels, so even if you have only one
> pixel, you would still need a padded second Y, therefore the pixel
> group size is 2, and bytes necessary is 4 (as opposed to 1 and 2). YUYV
> also has no vertical subsampling. NV12 has a pixel group size of 2
> pixels, due to the CbCr plane. The bytes per group then, for both
> planes, is 2. The first plane has no vertical subsampling, but the
> second plane is subsampled by a factor of 2.
> The IPU3 formats are also self-explanatory, as they are single-planar,
> and have a pixel group size of 25, consuming 32 bytes. Although a
> comment in the driver suggests that it should be 50 and 64,
> respectively, this is an attribute of the driver, and not the format, so
> this shall be set by the ipu3 pipeline handler.
> 
> Signed-off-by: Paul Elder <paul.elder@ideasonboard.com>
> Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
> Reviewed-by: Jacopo Mondi <jacopo@jmondi.org>
> Reviewed-by: Kieran Bingham <kieran.bingham@ideasonboard.com>
> 
> ---
> Changes in v5:
> - add #include <array> to fix cros compile error
> 
> Changes in v4:
> - expanded documentation and definition of pixel group
> 
> Changes in v3:
> - add planes
> - redefine the parameters for the formats
>   - pixelsPerGroup is for whole format
>   - add verticalSubSampling per plane
> 
> Changes in v2:
> - add documentation for bytesPerGroup pixelsPerGroup
> - fix wording in commit message
>   - bytes-per-line -> stride
>   - buffer size -> frame size
> - changed MJPEG todo to allowing pipeline handlers to set parameters of
>   format infos
> ---
>  include/libcamera/internal/formats.h |  11 ++
>  src/libcamera/formats.cpp            | 148 +++++++++++++++++++++++++++
>  2 files changed, 159 insertions(+)
> 
> diff --git a/include/libcamera/internal/formats.h b/include/libcamera/internal/formats.h
> index f59ac8f..4e90e5a 100644
> --- a/include/libcamera/internal/formats.h
> +++ b/include/libcamera/internal/formats.h
> @@ -8,6 +8,7 @@
>  #ifndef __LIBCAMERA_INTERNAL_FORMATS_H__
>  #define __LIBCAMERA_INTERNAL_FORMATS_H__
>  
> +#include <array>
>  #include <map>
>  #include <vector>
>  
> @@ -32,6 +33,12 @@ private:
>  	std::map<unsigned int, std::vector<SizeRange>> data_;
>  };
>  
> +struct PixelFormatPlaneInfo
> +{
> +	unsigned int bytesPerGroup;
> +	unsigned int verticalSubSampling;
> +};
> +
>  class PixelFormatInfo
>  {
>  public:
> @@ -52,6 +59,10 @@ public:
>  	unsigned int bitsPerPixel;
>  	enum ColourEncoding colourEncoding;
>  	bool packed;
> +
> +	unsigned int pixelsPerGroup;
> +
> +	std::array<PixelFormatPlaneInfo, 3> planes;
>  };
>  
>  } /* namespace libcamera */
> diff --git a/src/libcamera/formats.cpp b/src/libcamera/formats.cpp
> index d3b722c..e6d3d3b 100644
> --- a/src/libcamera/formats.cpp
> +++ b/src/libcamera/formats.cpp
> @@ -110,6 +110,22 @@ const std::map<unsigned int, std::vector<SizeRange>> &ImageFormats::data() const
>  	return data_;
>  }
>  
> +/**
> + * \class PixelFormatPlaneInfo
> + * \brief Information about a single plane of a pixel format
> + *
> + * \var PixelFormatPlaneInfo::bytesPerGroup
> + * \brief The number of bytes that a pixel group consumes
> + *
> + * \sa PixelFormatInfo::pixelsPerGroup
> + *
> + * \var PixelFormatPlaneInfo::verticalSubSampling
> + * \brief Vertical subsampling multiplier
> + *
> + * This value is the ratio between the number of rows of pixels in the frame
> + * to the number of rows of pixels in the plane.
> + */
> +
>  /**
>   * \class PixelFormatInfo
>   * \brief Information about pixel formats
> @@ -152,6 +168,49 @@ const std::map<unsigned int, std::vector<SizeRange>> &ImageFormats::data() const
>   * bytes. For instance, 12-bit Bayer data with two pixels stored in three bytes
>   * is packed, while the same data stored with 4 bits of padding in two bytes
>   * per pixel is not packed.
> + *
> + * \var PixelFormatInfo::pixelsPerGroup
> + * \brief The number of pixels in a pixel group
> + *
> + * The minimum number of pixels (including padding) necessary in a row
> + * when the frame has only one column of effective pixels.
> + *

I think you can drop this sentence, the next sentence is the same.

> + * A pixel group is defined as the minimum number of pixels (including padding)
> + * necessary in a row when the image has only one column of effective pixels.
> + * pixelsPerGroup refers to this value. PixelFormatPlaneInfo::bytesPerGroup,
> + * then, refers to the number of bytes that a pixel group consumes. This
> + * definition of a pixel group allows simple calculation of stride, as
> + * ceil(width / pixelsPerGroup) * bytesPerGroup. These values are determined
> + * only in terms of a row. The ceiling accounts for padding.
> + *
> + * A pixel group has a second constraint, such that the pixel group
> + * (bytesPerGroup and pixelsPerGroup) is the smallest repeatable unit.
> + * What this means is that, for example, in the IPU3 formats, if there is only
> + * one column of effective pixels, it looks like it could be fit in 5 bytes
> + * with 3 padding pixels (for a total of 4 pixels over 5 bytes). However, this
> + * unit is not repeatable, as at the 7th group in the same row, the pattern
> + * is broken. Therefore, the pixel group for IPU3 formats must be 25 pixels
> + * over 32 bytes.
> + *
> + * For example, for something simple like BGR888, it is self-explanatory:
> + * the pixel group size is 1, and the bytes necessary is 3, and there is
> + * only one plane with no (= 1) vertical subsampling. For YUYV, the
> + * CbCr pair is shared between two pixels, so even if you have only one
> + * pixel, you would still need a padded second Y sample, therefore the pixel
> + * group size is 2, and bytes necessary is 4. YUYV also has no vertical
> + * subsampling. NV12 has a pixel group size of 2 pixels, due to the CbCr plane.
> + * The bytes per group then, for both planes, is 2. The first plane has no
> + * vertical subsampling, but the second plane is subsampled by a factor of 2.
> + * The IPU3 formats are also self-explanatory, as they are single-planar,
> + * and have a pixel group size of 25, consuming 32 bytes. Although a
> + * comment in the driver suggests that it should be 50 and 64,
> + * respectively, this is an attribute of the driver, and not the format, so

s/driver/device/ (it's the DMA engine that sets this limitation, not
just the driver). I'd say

 * The IPU3 raw Bayer formats are single-planar, and have a pixel group size of
 * 25, consuming 32 bytes, due to the packing pattern being repeated in memory
 * every 32 bytes. The IPU3 hardware, however, has an additional constraint on
 * the DMA burst size, requiring lines to be multiple of 64 bytes. This isn't an
 * intrinsic property of the formats and is thus not reflected here. It is
 * instead enforced by the corresponding pipeline handler.

> + * this shall be set by the ipu3 pipeline handler.
> + *
> + * \var PixelFormatInfo::planes
> + * \brief Information about pixels for each plane
> + *
> + * \sa PixelFormatPlaneInfo
>   */
>  
>  /**
> @@ -179,6 +238,8 @@ const std::map<PixelFormat, PixelFormatInfo> pixelFormatInfo{
>  		.bitsPerPixel = 24,
>  		.colourEncoding = PixelFormatInfo::ColourEncodingRGB,
>  		.packed = false,
> +		.pixelsPerGroup = 1,
> +		.planes = {{ { 3, 1 }, { 0, 0 }, { 0, 0 } }},
>  	} },
>  	{ formats::RGB888, {
>  		.name = "RGB888",
> @@ -187,6 +248,8 @@ const std::map<PixelFormat, PixelFormatInfo> pixelFormatInfo{
>  		.bitsPerPixel = 24,
>  		.colourEncoding = PixelFormatInfo::ColourEncodingRGB,
>  		.packed = false,
> +		.pixelsPerGroup = 1,
> +		.planes = {{ { 3, 1 }, { 0, 0 }, { 0, 0 } }},
>  	} },
>  	{ formats::ABGR8888, {
>  		.name = "ABGR8888",
> @@ -195,6 +258,8 @@ const std::map<PixelFormat, PixelFormatInfo> pixelFormatInfo{
>  		.bitsPerPixel = 32,
>  		.colourEncoding = PixelFormatInfo::ColourEncodingRGB,
>  		.packed = false,
> +		.pixelsPerGroup = 1,
> +		.planes = {{ { 4, 1 }, { 0, 0 }, { 0, 0 } }},
>  	} },
>  	{ formats::ARGB8888, {
>  		.name = "ARGB8888",
> @@ -203,6 +268,8 @@ const std::map<PixelFormat, PixelFormatInfo> pixelFormatInfo{
>  		.bitsPerPixel = 32,
>  		.colourEncoding = PixelFormatInfo::ColourEncodingRGB,
>  		.packed = false,
> +		.pixelsPerGroup = 1,
> +		.planes = {{ { 4, 1 }, { 0, 0 }, { 0, 0 } }},
>  	} },
>  	{ formats::BGRA8888, {
>  		.name = "BGRA8888",
> @@ -211,6 +278,8 @@ const std::map<PixelFormat, PixelFormatInfo> pixelFormatInfo{
>  		.bitsPerPixel = 32,
>  		.colourEncoding = PixelFormatInfo::ColourEncodingRGB,
>  		.packed = false,
> +		.pixelsPerGroup = 1,
> +		.planes = {{ { 4, 1 }, { 0, 0 }, { 0, 0 } }},
>  	} },
>  	{ formats::RGBA8888, {
>  		.name = "RGBA8888",
> @@ -219,6 +288,8 @@ const std::map<PixelFormat, PixelFormatInfo> pixelFormatInfo{
>  		.bitsPerPixel = 32,
>  		.colourEncoding = PixelFormatInfo::ColourEncodingRGB,
>  		.packed = false,
> +		.pixelsPerGroup = 1,
> +		.planes = {{ { 4, 1 }, { 0, 0 }, { 0, 0 } }},
>  	} },
>  
>  	/* YUV packed formats. */
> @@ -229,6 +300,8 @@ const std::map<PixelFormat, PixelFormatInfo> pixelFormatInfo{
>  		.bitsPerPixel = 16,
>  		.colourEncoding = PixelFormatInfo::ColourEncodingYUV,
>  		.packed = false,
> +		.pixelsPerGroup = 2,
> +		.planes = {{ { 4, 1 }, { 0, 0 }, { 0, 0 } }},
>  	} },
>  	{ formats::YVYU, {
>  		.name = "YVYU",
> @@ -237,6 +310,8 @@ const std::map<PixelFormat, PixelFormatInfo> pixelFormatInfo{
>  		.bitsPerPixel = 16,
>  		.colourEncoding = PixelFormatInfo::ColourEncodingYUV,
>  		.packed = false,
> +		.pixelsPerGroup = 2,
> +		.planes = {{ { 4, 1 }, { 0, 0 }, { 0, 0 } }},
>  	} },
>  	{ formats::UYVY, {
>  		.name = "UYVY",
> @@ -245,6 +320,8 @@ const std::map<PixelFormat, PixelFormatInfo> pixelFormatInfo{
>  		.bitsPerPixel = 16,
>  		.colourEncoding = PixelFormatInfo::ColourEncodingYUV,
>  		.packed = false,
> +		.pixelsPerGroup = 2,
> +		.planes = {{ { 4, 1 }, { 0, 0 }, { 0, 0 } }},
>  	} },
>  	{ formats::VYUY, {
>  		.name = "VYUY",
> @@ -253,6 +330,8 @@ const std::map<PixelFormat, PixelFormatInfo> pixelFormatInfo{
>  		.bitsPerPixel = 16,
>  		.colourEncoding = PixelFormatInfo::ColourEncodingYUV,
>  		.packed = false,
> +		.pixelsPerGroup = 2,
> +		.planes = {{ { 4, 1 }, { 0, 0 }, { 0, 0 } }},
>  	} },
>  
>  	/* YUV planar formats. */
> @@ -263,6 +342,8 @@ const std::map<PixelFormat, PixelFormatInfo> pixelFormatInfo{
>  		.bitsPerPixel = 12,
>  		.colourEncoding = PixelFormatInfo::ColourEncodingYUV,
>  		.packed = false,
> +		.pixelsPerGroup = 2,
> +		.planes = {{ { 2, 1 }, { 2, 2 }, { 0, 0 } }},
>  	} },
>  	{ formats::NV21, {
>  		.name = "NV21",
> @@ -271,6 +352,8 @@ const std::map<PixelFormat, PixelFormatInfo> pixelFormatInfo{
>  		.bitsPerPixel = 12,
>  		.colourEncoding = PixelFormatInfo::ColourEncodingYUV,
>  		.packed = false,
> +		.pixelsPerGroup = 2,
> +		.planes = {{ { 2, 1 }, { 2, 2 }, { 0, 0 } }},
>  	} },
>  	{ formats::NV16, {
>  		.name = "NV16",
> @@ -279,6 +362,8 @@ const std::map<PixelFormat, PixelFormatInfo> pixelFormatInfo{
>  		.bitsPerPixel = 16,
>  		.colourEncoding = PixelFormatInfo::ColourEncodingYUV,
>  		.packed = false,
> +		.pixelsPerGroup = 2,
> +		.planes = {{ { 2, 1 }, { 2, 1 }, { 0, 0 } }},
>  	} },
>  	{ formats::NV61, {
>  		.name = "NV61",
> @@ -287,6 +372,8 @@ const std::map<PixelFormat, PixelFormatInfo> pixelFormatInfo{
>  		.bitsPerPixel = 16,
>  		.colourEncoding = PixelFormatInfo::ColourEncodingYUV,
>  		.packed = false,
> +		.pixelsPerGroup = 2,
> +		.planes = {{ { 2, 1 }, { 2, 1 }, { 0, 0 } }},
>  	} },
>  	{ formats::NV24, {
>  		.name = "NV24",
> @@ -295,6 +382,8 @@ const std::map<PixelFormat, PixelFormatInfo> pixelFormatInfo{
>  		.bitsPerPixel = 24,
>  		.colourEncoding = PixelFormatInfo::ColourEncodingYUV,
>  		.packed = false,
> +		.pixelsPerGroup = 1,
> +		.planes = {{ { 1, 1 }, { 2, 1 }, { 0, 0 } }},
>  	} },
>  	{ formats::NV42, {
>  		.name = "NV42",
> @@ -303,6 +392,8 @@ const std::map<PixelFormat, PixelFormatInfo> pixelFormatInfo{
>  		.bitsPerPixel = 24,
>  		.colourEncoding = PixelFormatInfo::ColourEncodingYUV,
>  		.packed = false,
> +		.pixelsPerGroup = 1,
> +		.planes = {{ { 1, 1 }, { 2, 1 }, { 0, 0 } }},
>  	} },
>  	{ formats::YUV420, {
>  		.name = "YUV420",
> @@ -311,6 +402,8 @@ const std::map<PixelFormat, PixelFormatInfo> pixelFormatInfo{
>  		.bitsPerPixel = 12,
>  		.colourEncoding = PixelFormatInfo::ColourEncodingYUV,
>  		.packed = false,
> +		.pixelsPerGroup = 2,
> +		.planes = {{ { 2, 1 }, { 1, 2 }, { 1, 2 } }},
>  	} },
>  	{ formats::YUV422, {
>  		.name = "YUV422",
> @@ -319,6 +412,8 @@ const std::map<PixelFormat, PixelFormatInfo> pixelFormatInfo{
>  		.bitsPerPixel = 16,
>  		.colourEncoding = PixelFormatInfo::ColourEncodingYUV,
>  		.packed = false,
> +		.pixelsPerGroup = 2,
> +		.planes = {{ { 2, 1 }, { 1, 1 }, { 1, 1 } }},
>  	} },
>  
>  	/* Greyscale formats. */
> @@ -329,6 +424,8 @@ const std::map<PixelFormat, PixelFormatInfo> pixelFormatInfo{
>  		.bitsPerPixel = 8,
>  		.colourEncoding = PixelFormatInfo::ColourEncodingYUV,
>  		.packed = false,
> +		.pixelsPerGroup = 1,
> +		.planes = {{ { 1, 1 }, { 0, 0 }, { 0, 0 } }},
>  	} },
>  
>  	/* Bayer formats. */
> @@ -339,6 +436,8 @@ const std::map<PixelFormat, PixelFormatInfo> pixelFormatInfo{
>  		.bitsPerPixel = 8,
>  		.colourEncoding = PixelFormatInfo::ColourEncodingRAW,
>  		.packed = false,
> +		.pixelsPerGroup = 2,
> +		.planes = {{ { 2, 1 }, { 0, 0 }, { 0, 0 } }},
>  	} },
>  	{ formats::SGBRG8, {
>  		.name = "SGBRG8",
> @@ -347,6 +446,8 @@ const std::map<PixelFormat, PixelFormatInfo> pixelFormatInfo{
>  		.bitsPerPixel = 8,
>  		.colourEncoding = PixelFormatInfo::ColourEncodingRAW,
>  		.packed = false,
> +		.pixelsPerGroup = 2,
> +		.planes = {{ { 2, 1 }, { 0, 0 }, { 0, 0 } }},
>  	} },
>  	{ formats::SGRBG8, {
>  		.name = "SGRBG8",
> @@ -355,6 +456,8 @@ const std::map<PixelFormat, PixelFormatInfo> pixelFormatInfo{
>  		.bitsPerPixel = 8,
>  		.colourEncoding = PixelFormatInfo::ColourEncodingRAW,
>  		.packed = false,
> +		.pixelsPerGroup = 2,
> +		.planes = {{ { 2, 1 }, { 0, 0 }, { 0, 0 } }},
>  	} },
>  	{ formats::SRGGB8, {
>  		.name = "SRGGB8",
> @@ -363,6 +466,8 @@ const std::map<PixelFormat, PixelFormatInfo> pixelFormatInfo{
>  		.bitsPerPixel = 8,
>  		.colourEncoding = PixelFormatInfo::ColourEncodingRAW,
>  		.packed = false,
> +		.pixelsPerGroup = 2,
> +		.planes = {{ { 2, 1 }, { 0, 0 }, { 0, 0 } }},
>  	} },
>  	{ formats::SBGGR10, {
>  		.name = "SBGGR10",
> @@ -371,6 +476,8 @@ const std::map<PixelFormat, PixelFormatInfo> pixelFormatInfo{
>  		.bitsPerPixel = 10,
>  		.colourEncoding = PixelFormatInfo::ColourEncodingRAW,
>  		.packed = false,
> +		.pixelsPerGroup = 2,
> +		.planes = {{ { 4, 1 }, { 0, 0 }, { 0, 0 } }},
>  	} },
>  	{ formats::SGBRG10, {
>  		.name = "SGBRG10",
> @@ -379,6 +486,8 @@ const std::map<PixelFormat, PixelFormatInfo> pixelFormatInfo{
>  		.bitsPerPixel = 10,
>  		.colourEncoding = PixelFormatInfo::ColourEncodingRAW,
>  		.packed = false,
> +		.pixelsPerGroup = 2,
> +		.planes = {{ { 4, 1 }, { 0, 0 }, { 0, 0 } }},
>  	} },
>  	{ formats::SGRBG10, {
>  		.name = "SGRBG10",
> @@ -387,6 +496,8 @@ const std::map<PixelFormat, PixelFormatInfo> pixelFormatInfo{
>  		.bitsPerPixel = 10,
>  		.colourEncoding = PixelFormatInfo::ColourEncodingRAW,
>  		.packed = false,
> +		.pixelsPerGroup = 2,
> +		.planes = {{ { 4, 1 }, { 0, 0 }, { 0, 0 } }},
>  	} },
>  	{ formats::SRGGB10, {
>  		.name = "SRGGB10",
> @@ -395,6 +506,8 @@ const std::map<PixelFormat, PixelFormatInfo> pixelFormatInfo{
>  		.bitsPerPixel = 10,
>  		.colourEncoding = PixelFormatInfo::ColourEncodingRAW,
>  		.packed = false,
> +		.pixelsPerGroup = 2,
> +		.planes = {{ { 4, 1 }, { 0, 0 }, { 0, 0 } }},
>  	} },
>  	{ formats::SBGGR10_CSI2P, {
>  		.name = "SBGGR10_CSI2P",
> @@ -403,6 +516,8 @@ const std::map<PixelFormat, PixelFormatInfo> pixelFormatInfo{
>  		.bitsPerPixel = 10,
>  		.colourEncoding = PixelFormatInfo::ColourEncodingRAW,
>  		.packed = true,
> +		.pixelsPerGroup = 4,
> +		.planes = {{ { 5, 1 }, { 0, 0 }, { 0, 0 } }},
>  	} },
>  	{ formats::SGBRG10_CSI2P, {
>  		.name = "SGBRG10_CSI2P",
> @@ -411,6 +526,8 @@ const std::map<PixelFormat, PixelFormatInfo> pixelFormatInfo{
>  		.bitsPerPixel = 10,
>  		.colourEncoding = PixelFormatInfo::ColourEncodingRAW,
>  		.packed = true,
> +		.pixelsPerGroup = 4,
> +		.planes = {{ { 5, 1 }, { 0, 0 }, { 0, 0 } }},
>  	} },
>  	{ formats::SGRBG10_CSI2P, {
>  		.name = "SGRBG10_CSI2P",
> @@ -419,6 +536,8 @@ const std::map<PixelFormat, PixelFormatInfo> pixelFormatInfo{
>  		.bitsPerPixel = 10,
>  		.colourEncoding = PixelFormatInfo::ColourEncodingRAW,
>  		.packed = true,
> +		.pixelsPerGroup = 4,
> +		.planes = {{ { 5, 1 }, { 0, 0 }, { 0, 0 } }},
>  	} },
>  	{ formats::SRGGB10_CSI2P, {
>  		.name = "SRGGB10_CSI2P",
> @@ -427,6 +546,8 @@ const std::map<PixelFormat, PixelFormatInfo> pixelFormatInfo{
>  		.bitsPerPixel = 10,
>  		.colourEncoding = PixelFormatInfo::ColourEncodingRAW,
>  		.packed = true,
> +		.pixelsPerGroup = 4,
> +		.planes = {{ { 5, 1 }, { 0, 0 }, { 0, 0 } }},
>  	} },
>  	{ formats::SBGGR12, {
>  		.name = "SBGGR12",
> @@ -435,6 +556,8 @@ const std::map<PixelFormat, PixelFormatInfo> pixelFormatInfo{
>  		.bitsPerPixel = 12,
>  		.colourEncoding = PixelFormatInfo::ColourEncodingRAW,
>  		.packed = false,
> +		.pixelsPerGroup = 2,
> +		.planes = {{ { 4, 1 }, { 0, 0 }, { 0, 0 } }},
>  	} },
>  	{ formats::SGBRG12, {
>  		.name = "SGBRG12",
> @@ -443,6 +566,8 @@ const std::map<PixelFormat, PixelFormatInfo> pixelFormatInfo{
>  		.bitsPerPixel = 12,
>  		.colourEncoding = PixelFormatInfo::ColourEncodingRAW,
>  		.packed = false,
> +		.pixelsPerGroup = 2,
> +		.planes = {{ { 4, 1 }, { 0, 0 }, { 0, 0 } }},
>  	} },
>  	{ formats::SGRBG12, {
>  		.name = "SGRBG12",
> @@ -451,6 +576,8 @@ const std::map<PixelFormat, PixelFormatInfo> pixelFormatInfo{
>  		.bitsPerPixel = 12,
>  		.colourEncoding = PixelFormatInfo::ColourEncodingRAW,
>  		.packed = false,
> +		.pixelsPerGroup = 2,
> +		.planes = {{ { 4, 1 }, { 0, 0 }, { 0, 0 } }},
>  	} },
>  	{ formats::SRGGB12, {
>  		.name = "SRGGB12",
> @@ -459,6 +586,8 @@ const std::map<PixelFormat, PixelFormatInfo> pixelFormatInfo{
>  		.bitsPerPixel = 12,
>  		.colourEncoding = PixelFormatInfo::ColourEncodingRAW,
>  		.packed = false,
> +		.pixelsPerGroup = 2,
> +		.planes = {{ { 4, 1 }, { 0, 0 }, { 0, 0 } }},
>  	} },
>  	{ formats::SBGGR12_CSI2P, {
>  		.name = "SBGGR12_CSI2P",
> @@ -467,6 +596,8 @@ const std::map<PixelFormat, PixelFormatInfo> pixelFormatInfo{
>  		.bitsPerPixel = 12,
>  		.colourEncoding = PixelFormatInfo::ColourEncodingRAW,
>  		.packed = true,
> +		.pixelsPerGroup = 2,
> +		.planes = {{ { 3, 1 }, { 0, 0 }, { 0, 0 } }},
>  	} },
>  	{ formats::SGBRG12_CSI2P, {
>  		.name = "SGBRG12_CSI2P",
> @@ -475,6 +606,8 @@ const std::map<PixelFormat, PixelFormatInfo> pixelFormatInfo{
>  		.bitsPerPixel = 12,
>  		.colourEncoding = PixelFormatInfo::ColourEncodingRAW,
>  		.packed = true,
> +		.pixelsPerGroup = 2,
> +		.planes = {{ { 3, 1 }, { 0, 0 }, { 0, 0 } }},
>  	} },
>  	{ formats::SGRBG12_CSI2P, {
>  		.name = "SGRBG12_CSI2P",
> @@ -483,6 +616,8 @@ const std::map<PixelFormat, PixelFormatInfo> pixelFormatInfo{
>  		.bitsPerPixel = 12,
>  		.colourEncoding = PixelFormatInfo::ColourEncodingRAW,
>  		.packed = true,
> +		.pixelsPerGroup = 2,
> +		.planes = {{ { 3, 1 }, { 0, 0 }, { 0, 0 } }},
>  	} },
>  	{ formats::SRGGB12_CSI2P, {
>  		.name = "SRGGB12_CSI2P",
> @@ -491,6 +626,8 @@ const std::map<PixelFormat, PixelFormatInfo> pixelFormatInfo{
>  		.bitsPerPixel = 12,
>  		.colourEncoding = PixelFormatInfo::ColourEncodingRAW,
>  		.packed = true,
> +		.pixelsPerGroup = 2,
> +		.planes = {{ { 3, 1 }, { 0, 0 }, { 0, 0 } }},
>  	} },
>  	{ formats::SBGGR10_IPU3, {
>  		.name = "SBGGR10_IPU3",
> @@ -499,6 +636,9 @@ const std::map<PixelFormat, PixelFormatInfo> pixelFormatInfo{
>  		.bitsPerPixel = 10,
>  		.colourEncoding = PixelFormatInfo::ColourEncodingRAW,
>  		.packed = true,
> +		/* \todo remember to double this in the ipu3 pipeline handler */
> +		.pixelsPerGroup = 25,
> +		.planes = {{ { 32, 1 }, { 0, 0 }, { 0, 0 } }},
>  	} },
>  	{ formats::SGBRG10_IPU3, {
>  		.name = "SGBRG10_IPU3",
> @@ -507,6 +647,8 @@ const std::map<PixelFormat, PixelFormatInfo> pixelFormatInfo{
>  		.bitsPerPixel = 10,
>  		.colourEncoding = PixelFormatInfo::ColourEncodingRAW,
>  		.packed = true,
> +		.pixelsPerGroup = 25,
> +		.planes = {{ { 32, 1 }, { 0, 0 }, { 0, 0 } }},
>  	} },
>  	{ formats::SGRBG10_IPU3, {
>  		.name = "SGRBG10_IPU3",
> @@ -515,6 +657,8 @@ const std::map<PixelFormat, PixelFormatInfo> pixelFormatInfo{
>  		.bitsPerPixel = 10,
>  		.colourEncoding = PixelFormatInfo::ColourEncodingRAW,
>  		.packed = true,
> +		.pixelsPerGroup = 25,
> +		.planes = {{ { 32, 1 }, { 0, 0 }, { 0, 0 } }},
>  	} },
>  	{ formats::SRGGB10_IPU3, {
>  		.name = "SRGGB10_IPU3",
> @@ -523,6 +667,8 @@ const std::map<PixelFormat, PixelFormatInfo> pixelFormatInfo{
>  		.bitsPerPixel = 10,
>  		.colourEncoding = PixelFormatInfo::ColourEncodingRAW,
>  		.packed = true,
> +		.pixelsPerGroup = 25,
> +		.planes = {{ { 32, 1 }, { 0, 0 }, { 0, 0 } }},
>  	} },
>  
>  	/* Compressed formats. */
> @@ -533,6 +679,8 @@ const std::map<PixelFormat, PixelFormatInfo> pixelFormatInfo{
>  		.bitsPerPixel = 0,
>  		.colourEncoding = PixelFormatInfo::ColourEncodingYUV,
>  		.packed = false,
> +		.pixelsPerGroup = 1,
> +		.planes = {{ { 1, 1 }, { 0, 0 }, { 0, 0 } }},
>  	} },
>  };
>