[{"id":4882,"web_url":"https://patchwork.libcamera.org/comment/4882/","msgid":"<20200521141513.oadmi4gozneeuppl@uno.localdomain>","date":"2020-05-21T14:15:13","subject":"Re: [libcamera-devel] [PATCH v5] libcamera: properties: Define\n\tpixel array properties","submitter":{"id":3,"url":"https://patchwork.libcamera.org/api/people/3/","name":"Jacopo Mondi","email":"jacopo@jmondi.org"},"content":"Hi,\n\nOn Wed, May 20, 2020 at 03:19:42PM +0200, Jacopo Mondi wrote:\n> Add definition of pixel array related properties.\n>\n> Signed-off-by: Jacopo Mondi \n> ---\n>\n> Broke out properties definition to fast track them\n\nCc-ing Sakari on this patch, with a little summary to see if we could\nlink the here-defined properties to V4L2 selection targets\n\nSakari, I'm here trying to define libcamera properties to expose the\ncamera sensor pixel array properties. There's a preceeding discussion\nyou've been cc'ed by Laurent. The here defined properties are\n\n- PixelArrayFullSize\n- PixelArrayEffectiveSize\n- PixelArrayActiveAreas\n\nThe description of the properties (hopefully) explains what each\nproperty reports, but the basic idea is the following one:\n\npr_err(\"%s:%d\\n\", __func__, __LINE__);\n\n- PixelArrayFullSize\n Includes optically blank pixels, exposed but not valid, and valid\n pixels\n- PixelArrayEffectiveSize\n Total number of exposed but (valid and not valid) pixels, includes all the\n possible 'valid' rectangles\n- PixelArrayActiveAreas\n The rectangle(s) of exposed and valid pixels, which represents the\n maximum image size for an image resolution, contained in the\n effective pixels rectangles.\n\nNow, assuming we can agree on libcamera side on these properties, I'm\ntrying to link them to v4l2 selection targets. From the documentation:\n\nV4L2_SEL_TGT_CROP_DEFAULT\nSuggested cropping rectangle that covers the “whole picture”. This\nincludes only active pixels and excludes other non-active pixels such\nas black pixels.\n\nV4L2_SEL_TGT_CROP_BOUNDS\nBounds of the crop rectangle. All valid crop rectangles fit inside the\ncrop bounds rectangle.\n\nV4L2_SEL_TGT_NATIVE_SIZE\nThe native size of the device, e.g. a sensor’s pixel array. left and\ntop fields are zero for this target.\n\nI admit _DEFAULT and _BOUNDS kind of confuses me, as they seem to\nserve the same purpose :/ Most i2c drivers use _BOUNDS to report the\nactive pixel array size (what would be PixelArrayActiveAreas for us).\nA few (one actually, imx219, because of a patch of mine) reports the\nactive pixel array size with _DEFAULT.\n\nAssuming imx219 is 'wrong', what should DEFAULT report ?\n\nAssuming NATIVE_SIZE aims to report the full pixel array size as\nPixelArrayFullSize does, is V4L2_SEL_TGT_CROP_DEFAULT a good match for\nPixelArrayEffectiveSize (which reports exposed but not 'good' pixels?)\n\nTrivia:\ngiven this picture of a sensor pixel array\nhttps://i.imgur.com/iYh4wbm.png\n\nwhat would you report?\n\nTo me this should probably be\nNATIVE_SIZE = 3296x2520 = PixelArrayFullSize\nCROP_DEFAULT = 3296x2480 = PixelArrayEffectiveSize\nCROP_BOUNDS = 3280x2464 = PixelArrayActiveAreas\n\nIf that's the case, do you think the V4L2 target description could be\nmade more specific ? I undestand they do not only apply to image\nsensors, and that makes it complicated to go in detail for that use\ncase only..\n\nThanks\n j\n\n\n>\n> ---\n> src/libcamera/property_ids.yaml | 198 ++++++++++++++++++++++++++++++++\n> 1 file changed, 198 insertions(+)\n>\n> diff --git a/src/libcamera/property_ids.yaml b/src/libcamera/property_ids.yaml\n> index ce627fa042ba..2ae178a73d52 100644\n> --- a/src/libcamera/property_ids.yaml\n> +++ b/src/libcamera/property_ids.yaml\n> @@ -386,4 +386,202 @@ controls:\n> | |\n> | |\n> +--------------------+\n> +\n> + - UnitCellSize:\n> + type: Size\n> + description: |\n> + The pixel unit cell physical size, in micro-meters.\n> +\n> + The UnitCellSize properties defines the horizontal and vertical sizes\n> + of a single pixel unit.\n> +\n> + The property can be used to calculate the physical size of the sensor's\n> + pixel array.\n> +\n> +\n> + - PixelArrayFullSize:\n> + type: Size\n> + description: |\n> + The camera sensor full pixel array vertical and horizontal sizes, in\n> + pixels.\n> +\n> + The PixelArrayFullSize property defines the size of the full pixel\n> + array matrix, including blank pixels used for calibration, exposed\n> + pixels which are not considered valid for image capture and active\n> + pixels.\n> +\n> + The property describes a rectangle whose top-left corner is placed\n> + in position (0, 0) and whose vertical and horizontal sizes are defined\n> + by the Size element transported by the property.\n> +\n> + The PixelArrayEffectiveSize and PixelArrayActiveSize described\n\ns/PixelArrayActiveSize/PixelArrayActiveAreas\n\n> + rectangles are defined in respect to the rectangle described by this\n> + property.\n> +\n> + \\todo Rename this property to FullSize once we will have property\n> + categories (i.e. Properties::PixelArray::FullSize)\n> +\n> + - PixelArrayEffectiveSize:\n> + type: Rectangle\n> + description: |\n> + The PixelArrayEffectiveSize defines the portion of the camera sensor\n> + full pixel array matrix which is exposed to light but not valid for\n> + image acquisition purposes.\n> +\n> + The PixelArrayEffectiveSize property defines the size in pixel units,\n> + and the position of effective pixels area rectangle relatively to the\n> + full pixel array matrix described by PixelArrayFullSize.\n> +\n> + The Rectangle top-left corner is defined by its horizontal and vertical\n> + distances from pixel (0, 0) of the full pixel matrix array.\n> +\n> + Example\n> + Given a full pixel array size rectangle, whose top-left corner is placed\n> + in position (0, 0) and its horizontal and vertical dimensions defined\n> + by PixelArrayFullSize[0] and PixelArrayFullSize[1] respectively\n> +\n> +\n> + PixelArrayFullSize[0]\n> + /-----------------/\n> + x1 x2\n> + (0,0)-> +-o------------o-+ /\n> + y1 o +------------+ | |\n> + | |////////////| | |\n> + | |////////////| | | PixelArrayFullSize[1]\n> + | |////////////| | |\n> + y2 o +------------+ | |\n> + +----------------+ /\n> +\n> + The size and position of the effective pixel array rectangle is defined\n> + as follows\n> +\n> + PixelArrayEffectiveSize = (x1, y1, x2 - x1 + 1, y2 - y1 + 1)\n> +\n> + \\todo Rename this property to EffectiveSize once we will have property\n> + categories (i.e. Properties::PixelArray::EffectiveSize)\n> +\n> + - PixelArrayActiveAreas:\n> + type: Rectangle\n> + size: [1 x n]\n> + description: |\n> + The PixelArrayActiveAreas property defines the (possibly multiple and\n> + overlapping) portions of the camera sensor effective pixel matrix\n> + which are considered valid for image acquisition purposes.\n> +\n> + Each rectangle is defined relatively to the PixelArrayEffectiveSize\n> + rectangle, with its top-left corner defined by its horizontal and\n> + vertical distances from the PixelArrayEffectiveSize rectangle top-left\n> + corner.\n> +\n> + This property describes an arbitrary number of overlapping rectangles,\n> + with each rectangle representing the maximum image size that the camera\n> + sensor can produce for a particular aspect ratio.\n> +\n> + When multiple rectangles are reported, they shall be ordered from the\n> + tallest to the shortest.\n> +\n> + Example 1\n> + A camera sensor which only produces images in the 4:3 image resolution\n> + will report a single PixelArrayActiveAreas rectangle, from which all\n> + other image formats are obtained by either cropping the field-of-view\n> + and/or applying pixel sub-sampling techniques such as pixel skipping or\n> + binning.\n> +\n> + PixelArrayEffectiveSize[2]\n> + /-----------------/\n> + x1 x2\n> + +-o------------o-+ /\n> + y1 o +------------+ | |\n> + | |////////////| | |\n> + | |////////////| | | PixelArrayEffectiveSize[3]\n> + | |////////////| | |\n> + y2 o +------------+ | |\n> + +----------------+ /\n> +\n> + The property reports a single rectangle\n> +\n> + PixelArrayActiveAreas = (x1, y1, x2 - x1 + 1, y2 - y1 + 1)\n> +\n> + Example 2\n> + A camera sensor which can produce images in different native\n> + resolutions will report several overlapping rectangles, one for each\n> + natively supported resolution.\n> +\n> + PixelArrayEffectiveSize[2]\n> + /-----------------/\n> + x1 x2 x3 x4\n> + +o---o------o---o+ /\n> + y1 o +------+ | |\n> + | |//////| | |\n> + y2 o+---+------+---+| |\n> + ||///|//////|///|| | PixelArrayEffectiveSize[3]\n> + y3 o+---+------+---+| |\n> + | |//////| | |\n> + y4 o +------+ | |\n> + +----+------+----+ /\n> +\n> + The property reports two rectangles\n> +\n> + PixelArrayActiveAreas = ((x2, y1, x3 - x2 + 1, y4 - y1 + 1),\n> + (x1, y2, x4 - x1 + 1, y3 - y2 + 1))\n> +\n> + The first rectangle describes the maximum field-of-view of all image\n> + formats in the 4:3 resolutions, while the second one describes the\n> + maximum field of view for all image formats in the 16:9 resolutions.\n> +\n> + \\todo Rename this property to ActiveAreas once we will have property\n> + categories (i.e. Properties::PixelArray::ActiveAreas)\n> +\n> + - ColorFilterArrangement:\n> + type: int32_t\n> + description: |\n> + The pixel array color filter arrangement.\n> +\n> + This property describes the arrangement and readout sequence of the\n> + three RGB color components of the sensor's Color Filter Array (CFA).\n> +\n> + Color filters are usually arranged in line-alternating fashion on the\n> + sensor pixel array. For example, one line might be composed of Red-Green\n> + while the successive is composed of Blue-Green color information.\n> +\n> + For Bayer filters, the value of this property represents the arrangement\n> + of color filters in the top-left 2x2 pixel square.\n> +\n> + For example, for a sensor with the following color filter pattern\n> +\n> + (0, 0) (max-col)\n> + +---+ +--------------...---+\n> + |B|G|<---|B|G|B|G|B|G|B|...B|G|\n> + |G|R|<---|G|R|G|R|G|R|G|...G|R|\n> + +---+ |B|G|B|G|B|G|B|...B|G|\n> + ... ..\n> + ... ..\n> + |G|R|G|R|G|R|G|...G|R|\n> + |B|G|B|G|B|G|B|...B|G| (max-lines)\n> + +--------------...---+\n> +\n> + The filter arrangement is represented by the BGGR value, which\n> + correspond to the pixel readout sequence in line interleaved mode.\n> +\n> + enum:\n> + - name: BayerFilterBGGR\n> + value: 0\n> + description: |\n> + Bayer color filter array arrangement is Blue-Green/Green-Red\n> +\n> + - name: BayerFilterGBRG\n> + value: 1\n> + description: |\n> + Bayer color filter array arrangement is Green-Blue/Red-Green\n> +\n> + - name: BayerFilterGRBG\n> + value: 2\n> + description: |\n> + Bayer color filter array arrangement is Green-Red/Blue-Green\n> +\n> + - name: BayerFilterRGGB\n> + value: 3\n> + description: |\n> + Bayer color filter array arrangement is Red-Green/Green-Blue\n> +\n> ...\n> --\n> 2.26.2\n>","headers":{"Return-Path":"","Received":["from relay11.mail.gandi.net (relay11.mail.gandi.net\n\t[217.70.178.231])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id DE9A8603D9\n\tfor ;\n\tThu, 21 May 2020 16:11:55 +0200 (CEST)","from uno.localdomain (93-34-118-233.ip49.fastwebnet.it\n\t[93.34.118.233]) (Authenticated sender: jacopo@jmondi.org)\n\tby relay11.mail.gandi.net (Postfix) with ESMTPSA id 91302100004;\n\tThu, 21 May 2020 14:11:54 +0000 (UTC)"],"Date":"Thu, 21 May 2020 16:15:13 +0200","From":"Jacopo Mondi ","To":"libcamera-devel@lists.libcamera.org,\n\tSakari Ailus ,\n\tLaurent Pinchart ","Message-ID":"<20200521141513.oadmi4gozneeuppl@uno.localdomain>","References":"<20200520131942.444499-1-jacopo@jmondi.org>","MIME-Version":"1.0","Content-Type":"text/plain; charset=utf-8","Content-Disposition":"inline","Content-Transfer-Encoding":"8bit","In-Reply-To":"<20200520131942.444499-1-jacopo@jmondi.org>","Subject":"Re: [libcamera-devel] [PATCH v5] libcamera: properties: Define\n\tpixel array properties","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","X-List-Received-Date":"Thu, 21 May 2020 14:11:56 -0000"}}]