From patchwork Wed Aug 5 10:57:18 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jacopo Mondi X-Patchwork-Id: 9217 X-Patchwork-Delegate: jacopo@jmondi.org Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id C79C7BD87A for ; Wed, 5 Aug 2020 10:53:54 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id A22E260580; Wed, 5 Aug 2020 12:53:54 +0200 (CEST) Received: from relay6-d.mail.gandi.net (relay6-d.mail.gandi.net [217.70.183.198]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id A52BA60555 for ; Wed, 5 Aug 2020 12:53:52 +0200 (CEST) X-Originating-IP: 93.34.118.233 Received: from uno.lan (93-34-118-233.ip49.fastwebnet.it [93.34.118.233]) (Authenticated sender: jacopo@jmondi.org) by relay6-d.mail.gandi.net (Postfix) with ESMTPSA id 2273FC000C; Wed, 5 Aug 2020 10:53:50 +0000 (UTC) From: Jacopo Mondi To: Hans Verkuil , Sakari Ailus , Laurent Pinchart Date: Wed, 5 Aug 2020 12:57:18 +0200 Message-Id: <20200805105721.15445-2-jacopo@jmondi.org> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20200805105721.15445-1-jacopo@jmondi.org> References: <20200805105721.15445-1-jacopo@jmondi.org> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH 1/4] media: docs: Describe pixel array properties X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Ricardo Ribalda Delgado , libcamera-devel@lists.libcamera.org, Sowjanya Komatineni , Linux Media Mailing List Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" The V4L2 selection API are also used to access the pixel array properties of an image sensor, such as the size and position of active pixels and the cropped area of the pixel matrix used to produce images. Currently no clear definition of the different areas that compose an image sensor pixel array matrix is provided in the specification, and the actual meaning of each selection target when applied to an image sensor was not provided. Provide in the sub-device documentation the definition of the pixel matrix properties and the selection target associated to each of them. Signed-off-by: Jacopo Mondi --- .../userspace-api/media/v4l/dev-subdev.rst | 81 +++++++++++++++++++ 1 file changed, 81 insertions(+) diff --git a/Documentation/userspace-api/media/v4l/dev-subdev.rst b/Documentation/userspace-api/media/v4l/dev-subdev.rst index 134d2fb909fa4..c47861dff9b9b 100644 --- a/Documentation/userspace-api/media/v4l/dev-subdev.rst +++ b/Documentation/userspace-api/media/v4l/dev-subdev.rst @@ -386,6 +386,87 @@ requests on all selection targets, unless specifically told otherwise. ``V4L2_SEL_FLAG_GE`` and ``V4L2_SEL_FLAG_LE`` flags may be used to round the image size either up or down. :ref:`v4l2-selection-flags` +.. _v4l2-subdev-pixel-array-properties: + +Selection targets for image sensors properties +---------------------------------------------- + +The V4L2 selection API can be used on sub-devices that represent an image +sensor to retrieve the sensor's pixel array matrix properties by using the +:ref:`selection ` ioctls. + +Sub-device drivers for image sensor usually register a single source pad, but in +the case they expose more, the pixel array properties can be accessed from +any of them. + +The ``V4L2_SEL_TGT_NATIVE``, ``V4L2_SEL_TGT_CROP_BOUNDS``, +``V4L2_SEL_TGT_CROP_DEFAULT`` and ``V4L2_TGT_CROP`` targets are used to retrieve +the immutable properties of the several different areas that compose the sensor +pixel array matrix. Each area describes a rectangle of logically adjacent pixel +units. The logical disposition of pixels is defined by the sensor read-out +starting point and direction, and may differ from the physical disposition of +the pixel units in the pixel array matrix. + +Each pixel matrix portion is contained in a larger rectangle, with the most +largest being the one that describes the pixel matrix physical size. This +defines a hierarchical positional system, where each rectangle is defined +relatively to the largest available one among the ones exposed by the +sub-device driver. Each selection target and the associated pixel array portion +it represents are below presented in order from the largest to the smallest one. + +Pixel array physical size +^^^^^^^^^^^^^^^^^^^^^^^^^ + +The image sensor chip is composed by a number of physical pixels, not all of +them readable by the application processor. Invalid or unreadable lines might +not be transmitted on the data bus at all, or in case on CSI-2 capable sensors +they might be tagged with an invalid data type (DT) so that the receiver +automatically discard them. The size of the whole pixel matrix area is +retrieved using the V4L2_SEL_TGT_NATIVE target, which has its top-left corner +defined as position (0, 0). All the other selection targets are defined +relatively to this, larger, rectangle. The rectangle returned by +V4L2_SEL_TGT_NATIVE describes an immutable property of the image sensor, it +does not change at run-time and cannot be modified from userspace. + +Pixel array readable area +^^^^^^^^^^^^^^^^^^^^^^^^^ + +The V4L2_SEL_TGT_CROP_BOUNDS targets returns size and position of the readable +area of the pixel array matrix, including pixels with valid image data and pixel +used for calibration purposes, such as optical black pixels. It is not unlikely +that valid pixels and optical black pixels are surrounded by non-readable rows +and columns of pixels. Those does not concur in the definition of the +V4L2_SEL_TGT_CROP_BOUNDS rectangle. The rectangle returned by +V4L2_SEL_TGT_CROP_BOUNDS describes an immutable property of the image sensor, it +does not change at run-time and cannot be modified from userspace. + +Pixel array active area +^^^^^^^^^^^^^^^^^^^^^^^ + +The portion of the pixel array which contains valid image data is defined as the +active area of the pixel matrix. The active pixel array is is accessed by mean +of the V4L2_SEL_TGT_CROP_DEFAULT target, and is contained in the larger +V4L2_SEL_TGT_CROP_BOUNDS rectangle. It represents the largest possible frame +resolution the sensor can produce and defines the dimension of the full +field-of-view. The rectangle returned by V4L2_SEL_TGT_CROP_BOUNDS describes an +immutable property of the image sensor, it does not change at run-time and +cannot be modified from userspace. + +Analog crop rectangle +^^^^^^^^^^^^^^^^^^^^^ + +The sensor driver might decide, in order to adjust the image resolution to best +match the one requested by applications, to only process a part of the active +pixel array matrix. The selected area is read-out and processed by the image +sensor on-board ISP in order to produce images of the desired size and +resolution while possible maintaing the largest possible field-of-view. The +cropped portion of the pixel array which is used to produce images is returned +by the V4L2_SEL_TGT_CROP target and represent the only information that can +change at runtime as it depends on the currently configured sensor mode and +desired image resolution. If the sub-device driver supports that, userspace +can set the analog crop rectangle to select which portion of the pixel array +to read out. + Types of selection targets -------------------------- From patchwork Wed Aug 5 10:57:19 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jacopo Mondi X-Patchwork-Id: 9218 X-Patchwork-Delegate: jacopo@jmondi.org Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id E31F1BD87A for ; Wed, 5 Aug 2020 10:53:55 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id BF03D60597; Wed, 5 Aug 2020 12:53:55 +0200 (CEST) Received: from relay6-d.mail.gandi.net (relay6-d.mail.gandi.net [217.70.183.198]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 2E17C60580 for ; Wed, 5 Aug 2020 12:53:54 +0200 (CEST) X-Originating-IP: 93.34.118.233 Received: from uno.lan (93-34-118-233.ip49.fastwebnet.it [93.34.118.233]) (Authenticated sender: jacopo@jmondi.org) by relay6-d.mail.gandi.net (Postfix) with ESMTPSA id B35E2C0006; Wed, 5 Aug 2020 10:53:52 +0000 (UTC) From: Jacopo Mondi To: Hans Verkuil , Sakari Ailus , Laurent Pinchart Date: Wed, 5 Aug 2020 12:57:19 +0200 Message-Id: <20200805105721.15445-3-jacopo@jmondi.org> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20200805105721.15445-1-jacopo@jmondi.org> References: <20200805105721.15445-1-jacopo@jmondi.org> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH 2/4] media: docs: Describe targets for sensor properties X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Ricardo Ribalda Delgado , libcamera-devel@lists.libcamera.org, Sowjanya Komatineni , Linux Media Mailing List Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Provide a table to describe how the V4L2 selection targets can be used to access an image sensor pixel array properties. Reference the table in the sub-device documentation. Signed-off-by: Jacopo Mondi --- .../userspace-api/media/v4l/dev-subdev.rst | 4 ++ .../media/v4l/v4l2-selection-targets.rst | 49 +++++++++++++++++++ 2 files changed, 53 insertions(+) diff --git a/Documentation/userspace-api/media/v4l/dev-subdev.rst b/Documentation/userspace-api/media/v4l/dev-subdev.rst index c47861dff9b9b..2f7da3832f458 100644 --- a/Documentation/userspace-api/media/v4l/dev-subdev.rst +++ b/Documentation/userspace-api/media/v4l/dev-subdev.rst @@ -467,6 +467,10 @@ desired image resolution. If the sub-device driver supports that, userspace can set the analog crop rectangle to select which portion of the pixel array to read out. +A description of each of the above mentioned targets when used to access the +image sensor pixel array properties is provided by +:ref:`v4l2-selection-targets-image-sensor-table` + Types of selection targets -------------------------- diff --git a/Documentation/userspace-api/media/v4l/v4l2-selection-targets.rst b/Documentation/userspace-api/media/v4l/v4l2-selection-targets.rst index 69f500093aa2a..632e6448b784e 100644 --- a/Documentation/userspace-api/media/v4l/v4l2-selection-targets.rst +++ b/Documentation/userspace-api/media/v4l/v4l2-selection-targets.rst @@ -76,3 +76,52 @@ of the two interfaces they are used. modified by hardware. - Yes - No + + +.. _v4l2-selection-targets-image-sensor-table: + +******************************************** +Selection Targets For Pixel Array Properties +******************************************** + +The V4L2 selection API can be used to retrieve the size and disposition of the +pixel units that compose and image sensor pixel matrix when applied to a video +sub-device that represents an image sensor. + +A description of the properties associated with each of the sensor pixel array +areas is provided by the :ref:`v4l2-subdev-pixel-array-properties` section. + +.. tabularcolumns:: |p{6.0cm}|p{1.4cm}|p{7.4cm}|p(1.4cm)| + +.. flat-table:: Selection target definitions + :header-rows: 1 + :stub-columns: 0 + + * - Target name + - id + - Definition + - Read/Write + * - ``V4L2_SEL_TGT_CROP`` + - 0x0000 + - The analog crop rectangle. Represents the portion of the active pixel + array which is processed to produce images. + - RW + * - ``V4L2_SEL_TGT_CROP_DEFAULT`` + - 0x0001 + - The active pixel array rectangle. It includes only active pixels and + excludes other ones such as optical black pixels. Its width and height + represent the maximum image resolution an image sensor can produce. + - RO + * - ``V4L2_SEL_TGT_CROP_BOUNDS`` + - 0x0002 + - The readable portion of the physical pixel array matrix. It includes + pixels that contains valid image data and calibration pixels such as the + optical black ones. + - RO + * - ``V4L2_SEL_TGT_NATIVE_SIZE`` + - 0x0003 + - The physical pixel array size, including readable and not readable + pixels. As pixels that cannot be read from application processor are not + relevant for calibration purposes, this rectangle is useful to calculate + the physical properties of the image sensor. + - RO From patchwork Wed Aug 5 10:57:20 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jacopo Mondi X-Patchwork-Id: 9219 X-Patchwork-Delegate: jacopo@jmondi.org Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 0FB31BD87A for ; Wed, 5 Aug 2020 10:53:58 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id E167560564; Wed, 5 Aug 2020 12:53:57 +0200 (CEST) Received: from relay6-d.mail.gandi.net (relay6-d.mail.gandi.net [217.70.183.198]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id B00F76039D for ; Wed, 5 Aug 2020 12:53:55 +0200 (CEST) X-Originating-IP: 93.34.118.233 Received: from uno.lan (93-34-118-233.ip49.fastwebnet.it [93.34.118.233]) (Authenticated sender: jacopo@jmondi.org) by relay6-d.mail.gandi.net (Postfix) with ESMTPSA id 4BC14C000E; Wed, 5 Aug 2020 10:53:54 +0000 (UTC) From: Jacopo Mondi To: Hans Verkuil , Sakari Ailus , Laurent Pinchart Date: Wed, 5 Aug 2020 12:57:20 +0200 Message-Id: <20200805105721.15445-4-jacopo@jmondi.org> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20200805105721.15445-1-jacopo@jmondi.org> References: <20200805105721.15445-1-jacopo@jmondi.org> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH 3/4] media: docs: USe SUBDEV_G_SELECTION for sensor properties X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Ricardo Ribalda Delgado , libcamera-devel@lists.libcamera.org, Sowjanya Komatineni , Linux Media Mailing List Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Add a small note to the VIDIOC_SUBDEV_G_SELECTION IOCTL documentation to report that the API can be used to access an image sensor pixel array properties. Signed-off-by: Jacopo Mondi --- .../userspace-api/media/v4l/vidioc-subdev-g-selection.rst | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/Documentation/userspace-api/media/v4l/vidioc-subdev-g-selection.rst b/Documentation/userspace-api/media/v4l/vidioc-subdev-g-selection.rst index 06c9553ac48f5..05539f5deace2 100644 --- a/Documentation/userspace-api/media/v4l/vidioc-subdev-g-selection.rst +++ b/Documentation/userspace-api/media/v4l/vidioc-subdev-g-selection.rst @@ -46,6 +46,10 @@ The selections are used to configure various image processing functionality performed by the subdevs which affect the image size. This currently includes cropping, scaling and composition. +This API can also be used to retrieve immutable properties of the device +represented by the subdev, such as the pixel matrix properties of an image +sensor. + The selection API replaces :ref:`the old subdev crop API `. All the function of the crop API, and more, are supported by the selections API. From patchwork Wed Aug 5 10:57:21 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jacopo Mondi X-Patchwork-Id: 9220 X-Patchwork-Delegate: jacopo@jmondi.org Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 3769DBD87A for ; Wed, 5 Aug 2020 10:53:59 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 0FBDB6055E; Wed, 5 Aug 2020 12:53:59 +0200 (CEST) Received: from relay6-d.mail.gandi.net (relay6-d.mail.gandi.net [217.70.183.198]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 72D146039D for ; Wed, 5 Aug 2020 12:53:57 +0200 (CEST) X-Originating-IP: 93.34.118.233 Received: from uno.lan (93-34-118-233.ip49.fastwebnet.it [93.34.118.233]) (Authenticated sender: jacopo@jmondi.org) by relay6-d.mail.gandi.net (Postfix) with ESMTPSA id D167EC000F; Wed, 5 Aug 2020 10:53:55 +0000 (UTC) From: Jacopo Mondi To: Hans Verkuil , Sakari Ailus , Laurent Pinchart Date: Wed, 5 Aug 2020 12:57:21 +0200 Message-Id: <20200805105721.15445-5-jacopo@jmondi.org> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20200805105721.15445-1-jacopo@jmondi.org> References: <20200805105721.15445-1-jacopo@jmondi.org> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH 4/4] media: i2c: imx219: Selection compliance fixes X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: libcamera-devel@lists.libcamera.org, Sowjanya Komatineni , Hans Verkuil , Ricardo Ribalda Delgado , Linux Media Mailing List Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" From: Hans Verkuil To comply with the intended usage of the V4L2 selection target when used to retrieve a sensor image properties, adjust the rectangles returned by the imx219 driver. The top/left crop coordinates of the TGT_CROP rectangle were set to (0, 0) instead of (8, 8) which is the offset from the larger physical pixel array rectangle. This was also a mismatch with the default values crop rectangle value, so this is corrected. Found with v4l2-compliance. While at it, add V4L2_SEL_TGT_CROP_BOUNDS support: CROP_DEFAULT and CROP_BOUNDS have the same size as the non-active pixels are not readable using the selection API. Found with v4l2-compliance. Fixes: e6d4ef7d58aa7 ("media: i2c: imx219: Implement get_selection") Signed-off-by: Hans Verkuil [reword commit message, use macros for pixel offsets] Signed-off-by: Jacopo Mondi --- drivers/media/i2c/imx219.c | 17 +++++++++-------- 1 file changed, 9 insertions(+), 8 deletions(-) diff --git a/drivers/media/i2c/imx219.c b/drivers/media/i2c/imx219.c index 60b23d113fc56..ff48a95b448b1 100644 --- a/drivers/media/i2c/imx219.c +++ b/drivers/media/i2c/imx219.c @@ -473,8 +473,8 @@ static const struct imx219_mode supported_modes[] = { .width = 3280, .height = 2464, .crop = { - .left = 0, - .top = 0, + .left = IMX219_PIXEL_ARRAY_LEFT, + .top = IMX219_PIXEL_ARRAY_TOP, .width = 3280, .height = 2464 }, @@ -489,8 +489,8 @@ static const struct imx219_mode supported_modes[] = { .width = 1920, .height = 1080, .crop = { - .left = 680, - .top = 692, + .left = 688, + .top = 700, .width = 1920, .height = 1080 }, @@ -505,8 +505,8 @@ static const struct imx219_mode supported_modes[] = { .width = 1640, .height = 1232, .crop = { - .left = 0, - .top = 0, + .left = IMX219_PIXEL_ARRAY_LEFT, + .top = IMX219_PIXEL_ARRAY_TOP, .width = 3280, .height = 2464 }, @@ -521,8 +521,8 @@ static const struct imx219_mode supported_modes[] = { .width = 640, .height = 480, .crop = { - .left = 1000, - .top = 752, + .left = 1008, + .top = 760, .width = 1280, .height = 960 }, @@ -1045,6 +1045,7 @@ static int imx219_get_selection(struct v4l2_subdev *sd, return 0; case V4L2_SEL_TGT_CROP_DEFAULT: + case V4L2_SEL_TGT_CROP_BOUNDS: sel->r.top = IMX219_PIXEL_ARRAY_TOP; sel->r.left = IMX219_PIXEL_ARRAY_LEFT; sel->r.width = IMX219_PIXEL_ARRAY_WIDTH;