[libcamera-devel,v5] libcamera: properties: Define pixel array properties

3829 diff mbox series

Add definition of pixel array related properties.

Signed-off-by: Jacopo Mondi <jacopo@jmondi.org>
---

Broke out properties definition to fast track them

---
 src/libcamera/property_ids.yaml | 198 ++++++++++++++++++++++++++++++++
 1 file changed, 198 insertions(+)

--
2.26.2

Hi,

On Wed, May 20, 2020 at 03:19:42PM +0200, Jacopo Mondi wrote:
> Add definition of pixel array related properties.
>
> Signed-off-by: Jacopo Mondi <jacopo@jmondi.org>
> ---
>
> Broke out properties definition to fast track them

Cc-ing Sakari on this patch, with a little summary to see if we could
link the here-defined properties to V4L2 selection targets

Sakari, I'm here trying to define libcamera properties to expose the
camera sensor pixel array properties. There's a preceeding discussion
you've been cc'ed by Laurent. The here defined properties are

- PixelArrayFullSize
- PixelArrayEffectiveSize
- PixelArrayActiveAreas

The description of the properties (hopefully) explains what each
property reports, but the basic idea is the following one:

pr_err("%s:%d\n", __func__, __LINE__);

- PixelArrayFullSize
    Includes optically blank pixels, exposed but not valid, and valid
    pixels
- PixelArrayEffectiveSize
    Total number of exposed but (valid and not valid) pixels, includes all the
    possible 'valid' rectangles
- PixelArrayActiveAreas
    The rectangle(s) of exposed and valid pixels, which represents the
    maximum image size for an image resolution, contained in the
    effective pixels rectangles.

Now, assuming we can agree on libcamera side on these properties, I'm
trying to link them to v4l2 selection targets. From the documentation:

V4L2_SEL_TGT_CROP_DEFAULT
Suggested cropping rectangle that covers the “whole picture”. This
includes only active pixels and excludes other non-active pixels such
as black pixels.

V4L2_SEL_TGT_CROP_BOUNDS
Bounds of the crop rectangle. All valid crop rectangles fit inside the
crop bounds rectangle.

V4L2_SEL_TGT_NATIVE_SIZE
The native size of the device, e.g. a sensor’s pixel array. left and
top fields are zero for this target.

I admit _DEFAULT and _BOUNDS kind of confuses me, as they seem to
serve the same purpose :/ Most i2c drivers use _BOUNDS to report the
active pixel array size (what would be PixelArrayActiveAreas for us).
A few (one actually, imx219, because of a patch of mine) reports the
active pixel array size with _DEFAULT.

Assuming imx219 is 'wrong', what should DEFAULT report ?

Assuming NATIVE_SIZE aims to report the full pixel array size as
PixelArrayFullSize does, is V4L2_SEL_TGT_CROP_DEFAULT a good match for
PixelArrayEffectiveSize (which reports exposed but not 'good' pixels?)

Trivia:
given this picture of a sensor pixel array
https://i.imgur.com/iYh4wbm.png

what would you report?

To me this should probably be
NATIVE_SIZE = 3296x2520 = PixelArrayFullSize
CROP_DEFAULT = 3296x2480 = PixelArrayEffectiveSize
CROP_BOUNDS = 3280x2464  = PixelArrayActiveAreas

If that's the case, do you think the V4L2 target description could be
made more specific ? I undestand they do not only apply to image
sensors, and that makes it complicated to go in detail for that use
case only..

Thanks
  j


>
> ---
>  src/libcamera/property_ids.yaml | 198 ++++++++++++++++++++++++++++++++
>  1 file changed, 198 insertions(+)
>
> diff --git a/src/libcamera/property_ids.yaml b/src/libcamera/property_ids.yaml
> index ce627fa042ba..2ae178a73d52 100644
> --- a/src/libcamera/property_ids.yaml
> +++ b/src/libcamera/property_ids.yaml
> @@ -386,4 +386,202 @@ controls:
>                                |                    |
>                                |                    |
>                                +--------------------+
> +
> +  - UnitCellSize:
> +      type: Size
> +      description: |
> +        The pixel unit cell physical size, in micro-meters.
> +
> +        The UnitCellSize properties defines the horizontal and vertical sizes
> +        of a single pixel unit.
> +
> +        The property can be used to calculate the physical size of the sensor's
> +        pixel array.
> +
> +
> +  - PixelArrayFullSize:
> +      type: Size
> +      description: |
> +        The camera sensor full pixel array vertical and horizontal sizes, in
> +        pixels.
> +
> +        The PixelArrayFullSize property defines the size of the full pixel
> +        array matrix, including blank pixels used for calibration, exposed
> +        pixels which are not considered valid for image capture and active
> +        pixels.
> +
> +        The property describes a rectangle whose top-left corner is placed
> +        in position (0, 0) and whose vertical and horizontal sizes are defined
> +        by the Size element transported by the property.
> +
> +        The PixelArrayEffectiveSize and PixelArrayActiveSize described

s/PixelArrayActiveSize/PixelArrayActiveAreas

> +        rectangles are defined in respect to the rectangle described by this
> +        property.
> +
> +        \todo Rename this property to FullSize once we will have property
> +              categories (i.e. Properties::PixelArray::FullSize)
> +
> +  - PixelArrayEffectiveSize:
> +      type: Rectangle
> +      description: |
> +        The PixelArrayEffectiveSize defines the portion of the camera sensor
> +        full pixel array matrix which is exposed to light but not valid for
> +        image acquisition purposes.
> +
> +        The PixelArrayEffectiveSize property defines the size in pixel units,
> +        and the position of effective pixels area rectangle relatively to the
> +        full pixel array matrix described by PixelArrayFullSize.
> +
> +        The Rectangle top-left corner is defined by its horizontal and vertical
> +        distances from pixel (0, 0) of the full pixel matrix array.
> +
> +        Example
> +        Given a full pixel array size rectangle, whose top-left corner is placed
> +        in position (0, 0) and its horizontal and vertical dimensions defined
> +        by PixelArrayFullSize[0] and PixelArrayFullSize[1] respectively
> +
> +
> +                   PixelArrayFullSize[0]
> +                    /-----------------/
> +                      x1          x2
> +            (0,0)-> +-o------------o-+  /
> +                 y1 o +------------+ |  |
> +                    | |////////////| |  |
> +                    | |////////////| |  | PixelArrayFullSize[1]
> +                    | |////////////| |  |
> +                 y2 o +------------+ |  |
> +                    +----------------+  /
> +
> +        The size and position of the effective pixel array rectangle is defined
> +        as follows
> +
> +            PixelArrayEffectiveSize = (x1, y1, x2 - x1 + 1, y2 - y1 + 1)
> +
> +        \todo Rename this property to EffectiveSize once we will have property
> +              categories (i.e. Properties::PixelArray::EffectiveSize)
> +
> +  - PixelArrayActiveAreas:
> +      type: Rectangle
> +      size: [1 x n]
> +      description: |
> +        The PixelArrayActiveAreas property defines the (possibly multiple and
> +        overlapping) portions of the camera sensor effective pixel matrix
> +        which are considered valid for image acquisition purposes.
> +
> +        Each rectangle is defined relatively to the PixelArrayEffectiveSize
> +        rectangle, with its top-left corner defined by its horizontal and
> +        vertical distances from the PixelArrayEffectiveSize rectangle top-left
> +        corner.
> +
> +        This property describes an arbitrary number of overlapping rectangles,
> +        with each rectangle representing the maximum image size that the camera
> +        sensor can produce for a particular aspect ratio.
> +
> +        When multiple rectangles are reported, they shall be ordered from the
> +        tallest to the shortest.
> +
> +        Example 1
> +        A camera sensor which only produces images in the 4:3 image resolution
> +        will report a single PixelArrayActiveAreas rectangle, from which all
> +        other image formats are obtained by either cropping the field-of-view
> +        and/or applying pixel sub-sampling techniques such as pixel skipping or
> +        binning.
> +
> +                  PixelArrayEffectiveSize[2]
> +                    /-----------------/
> +                      x1          x2
> +                    +-o------------o-+  /
> +                 y1 o +------------+ |  |
> +                    | |////////////| |  |
> +                    | |////////////| |  | PixelArrayEffectiveSize[3]
> +                    | |////////////| |  |
> +                 y2 o +------------+ |  |
> +                    +----------------+  /
> +
> +        The property reports a single rectangle
> +
> +                 PixelArrayActiveAreas = (x1, y1, x2 - x1 + 1, y2 - y1 + 1)
> +
> +        Example 2
> +        A camera sensor which can produce images in different native
> +        resolutions will report several overlapping rectangles, one for each
> +        natively supported resolution.
> +
> +                  PixelArrayEffectiveSize[2]
> +                    /-----------------/
> +                     x1  x2    x3  x4
> +                    +o---o------o---o+  /
> +                 y1 o    +------+    |  |
> +                    |    |//////|    |  |
> +                 y2 o+---+------+---+|  |
> +                    ||///|//////|///||  | PixelArrayEffectiveSize[3]
> +                 y3 o+---+------+---+|  |
> +                    |    |//////|    |  |
> +                 y4 o    +------+    |  |
> +                    +----+------+----+  /
> +
> +        The property reports two rectangles
> +
> +                PixelArrayActiveAreas = ((x2, y1, x3 - x2 + 1, y4 - y1 + 1),
> +                                         (x1, y2, x4 - x1 + 1, y3 - y2 + 1))
> +
> +        The first rectangle describes the maximum field-of-view of all image
> +        formats in the 4:3 resolutions, while the second one describes the
> +        maximum field of view for all image formats in the 16:9 resolutions.
> +
> +        \todo Rename this property to ActiveAreas once we will have property
> +              categories (i.e. Properties::PixelArray::ActiveAreas)
> +
> +  - ColorFilterArrangement:
> +      type: int32_t
> +      description: |
> +        The pixel array color filter arrangement.
> +
> +        This property describes the arrangement and readout sequence of the
> +        three RGB color components of the sensor's Color Filter Array (CFA).
> +
> +        Color filters are usually arranged in line-alternating fashion on the
> +        sensor pixel array. For example, one line might be composed of Red-Green
> +        while the successive is composed of Blue-Green color information.
> +
> +        For Bayer filters, the value of this property represents the arrangement
> +        of color filters in the top-left 2x2 pixel square.
> +
> +        For example, for a sensor with the following color filter pattern
> +
> +                      (0, 0)               (max-col)
> +                +---+    +--------------...---+
> +                |B|G|<---|B|G|B|G|B|G|B|...B|G|
> +                |G|R|<---|G|R|G|R|G|R|G|...G|R|
> +                +---+    |B|G|B|G|B|G|B|...B|G|
> +                         ...                  ..
> +                         ...                  ..
> +                         |G|R|G|R|G|R|G|...G|R|
> +                         |B|G|B|G|B|G|B|...B|G|   (max-lines)
> +                         +--------------...---+
> +
> +        The filter arrangement is represented by the BGGR value, which
> +        correspond to the pixel readout sequence in line interleaved mode.
> +
> +      enum:
> +        - name: BayerFilterBGGR
> +          value: 0
> +          description: |
> +            Bayer color filter array arrangement is Blue-Green/Green-Red
> +
> +        - name: BayerFilterGBRG
> +          value: 1
> +          description: |
> +            Bayer color filter array arrangement is Green-Blue/Red-Green
> +
> +        - name: BayerFilterGRBG
> +          value: 2
> +          description: |
> +            Bayer color filter array arrangement is Green-Red/Blue-Green
> +
> +        - name: BayerFilterRGGB
> +          value: 3
> +          description: |
> +            Bayer color filter array arrangement is Red-Green/Green-Blue
> +
>  ...
> --
> 2.26.2
>