From patchwork Tue Jun 16 12:08:14 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefan Klug X-Patchwork-Id: 26905 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 70204C3261 for ; Tue, 16 Jun 2026 12:08:51 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 29D8B625C9; Tue, 16 Jun 2026 14:08:51 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="B3WmNeTQ"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 632C1625BE for ; Tue, 16 Jun 2026 14:08:48 +0200 (CEST) Received: from ideasonboard.com (unknown [IPv6:2a00:6020:448c:6c00:f9ec:e24d:190b:e780]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id EDEE8448; Tue, 16 Jun 2026 14:08:14 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1781611695; bh=WJFA8Mm7FeFoElhVwCKj/APNP4z5j2CteTh+OkK1j/g=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=B3WmNeTQ5zFzyUAh+hmMSwlnSYpnAQY/LxQS+Gv7nInOqhsy9vAIKBQlgITSWBQUZ jhGbaCRaeSuBR92UumTZSW9npfeVvwSfZg51KwcWZ/+u5EHoYmnQ1nsfv89VxeBFcu OX/ZP8oHp0n+tv7g0ux49j/10RUV9/5wE383vaVE= From: Stefan Klug To: libcamera-devel@lists.libcamera.org Cc: Stefan Klug Subject: [PATCH v1 1/2] libcamera: Clarify meaning of PixelArraySize and other rectangles Date: Tue, 16 Jun 2026 14:08:14 +0200 Message-ID: <20260616120828.375547-2-stefan.klug@ideasonboard.com> X-Mailer: git-send-email 2.53.0 In-Reply-To: <20260616120828.375547-1-stefan.klug@ideasonboard.com> References: <20260616120828.375547-1-stefan.klug@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Current documented behavior for PixelArraySize is to report the readable array size of the sensor data. If the readable array is offset relative to the physical sensor array, all related rectangles (ScalerCrop, PixelArrayActiveAreas) shall be reported relative to PixelArraySize. However no known pipeline and IPA implements it that way. All pipelines/IPAs treat PixelArraySize as the physical sensor size and handle the rectangles relative to that size. Adjust the documentation to fit the actual implementations. Add another property named PixelArrayReadableArea to convey the information that PixelArraySize was intended for. This has the added benefit that all rectangles are relative to the physical sensor array, which is necessary to align tuning data like lens shading correction. Signed-off-by: Stefan Klug --- ToDo: - Implement the expected behavior in CameraSensorRaw --- include/libcamera/ipa/core.mojom | 2 +- src/libcamera/property_ids_core.yaml | 116 ++++++++++-------- src/libcamera/sensor/camera_sensor_legacy.cpp | 26 ++-- src/libcamera/sensor/camera_sensor_raw.cpp | 16 +-- 4 files changed, 89 insertions(+), 71 deletions(-) diff --git a/include/libcamera/ipa/core.mojom b/include/libcamera/ipa/core.mojom index bce797245829..5da780f5d256 100644 --- a/include/libcamera/ipa/core.mojom +++ b/include/libcamera/ipa/core.mojom @@ -154,7 +154,7 @@ module libcamera; /** * \var IPACameraSensorInfo::activeAreaSize - * \brief The size of the pixel array active area of the sensor + * \brief The size of the physical pixel array of the sensor in pixels */ /** diff --git a/src/libcamera/property_ids_core.yaml b/src/libcamera/property_ids_core.yaml index d5b3d309c055..96c85f9233fd 100644 --- a/src/libcamera/property_ids_core.yaml +++ b/src/libcamera/property_ids_core.yaml @@ -427,11 +427,29 @@ controls: - PixelArraySize: type: Size description: | - The camera sensor pixel array readable area vertical and horizontal - sizes, in pixels. + The camera sensor pixel array vertical and horizontal sizes, in pixels. - The PixelArraySize property defines the size in pixel units of the - readable part of full pixel array matrix, including optical black + The PixelArraySize property defines the size in pixel units of the full + pixel array matrix. It is often not be possible to capture the whole + physical pixel array. For the largest readable area \see + PixelArrayReadableArea. + + This defines a rectangle whose top-left corner is placed in position (0, + 0) and whose vertical and horizontal sizes are defined by this property. + All other rectangles that describe portions of the pixel array, such as + the optical black pixels rectangles and active pixel areas, are defined + relatively to this rectangle. + + \todo Rename this property to Size once we will have property + categories (i.e. Properties::PixelArray::Size) + + - PixelArrayReadableArea: + type: Rectangle + description: | + The camera sensor pixel array readable area. + + The PixelArrayReadableArea property defines the rectangle in pixel units + of the readable part of full pixel array matrix, including optical black pixels used for calibration, pixels which are not considered valid for capture and active pixels containing valid image data. @@ -442,23 +460,25 @@ controls: For example, let's consider a pixel array matrix assembled as follows - +--------------------------------------------------+ - |xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx| - |xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx| - |xxDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDxx| - |xxDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDxx| - |xxDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDxx| - |xxDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDxx| - |xxDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDxx| - |xxDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDxx| + PixelArraySize.width + /--------------------------------------------------/ + +--------------------------------------------------+ / + |xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx| | + |xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx| | + |xxDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDxx| | + |xxDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDxx| | + |xxDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDxx| | + |xxDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDxx| | + |xxDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDxx| | + |xxDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDxx| | PixelArraySize.height ... ... ... ... ... ... ... ... ... ... - |xxDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDxx| - |xxDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDxx| - |xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx| - |xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx| - +--------------------------------------------------+ + |xxDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDxx| | + |xxDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDxx| | + |xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx| | + |xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx| | + +--------------------------------------------------+ / starting with two lines of non-readable pixels (x), followed by N lines of readable data (D) surrounded by two columns of non-readable pixels on @@ -467,35 +487,32 @@ controls: sizes of the largest possible buffer of raw data that can be presented to applications. - PixelArraySize.width - /----------------------------------------------/ - +----------------------------------------------+ / - |DDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDD| | - |DDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDD| | - |DDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDD| | - |DDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDD| | - |DDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDD| | - |DDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDD| | PixelArraySize.height - ... ... ... ... ... - ... ... ... ... ... - |DDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDD| | - |DDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDD| | - +----------------------------------------------+ / + PixelArrayReadableArea.width + /--------------------------------------------/ + PixelArrayReadableArea.x + +--o-----------------------------------------------+ + |xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx| + |xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx| + oxxDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDxx| / PixelArrayReadableArea.y + |xxDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDxx| | + |xxDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDxx| | + |xxDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDxx| | + |xxDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDxx| | + |xxDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDxx| | PixelArrayReadableArea.height + ... ... ... ... ... - This defines a rectangle whose top-left corner is placed in position (0, - 0) and whose vertical and horizontal sizes are defined by this property. - All other rectangles that describe portions of the pixel array, such as - the optical black pixels rectangles and active pixel areas, are defined - relatively to this rectangle. + ... ... ... ... ... + |xxDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDxx| | + |xxDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDxx| / + |xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx| + |xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx| + +--------------------------------------------------+ All the coordinates are expressed relative to the default sensor readout direction, without any transformation (such as horizontal and vertical flipping) applied. When mapping them to the raw pixel buffer, applications shall take any configured transformation into account. - \todo Rename this property to Size once we will have property - categories (i.e. Properties::PixelArray::Size) - - PixelArrayOpticalBlackRectangles: type: Rectangle size: [n] @@ -519,7 +536,7 @@ controls: they could be positioned too close to the chip margins or to the optical black shielding placed on top of optical black pixels. - PixelArraySize.width + PixelArrayReadableArea.width /----------------------------------------------/ x1 x2 +--o---------------------------------------o---+ / @@ -532,7 +549,7 @@ controls: |IIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIII| | |IIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIII| | y3 |IIOOPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPOOII| | - |IIOOPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPOOII| | PixelArraySize.height + |IIOOPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPOOII| | PixelArrayReadableArea.height |IIOOPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPOOII| | ... ... ... ... ... ... ... ... ... ... @@ -560,7 +577,7 @@ controls: skips the invalid lines and columns, producing the following data buffer, when captured to memory - PixelArraySize.width + PixelArrayReadableArea.width /----------------------------------------------/ x1 +--------------------------------------------o-+ / @@ -570,7 +587,7 @@ controls: |OOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOO| | y1 oOOPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPOO| | |OOPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPOO| | - |OOPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPOO| | PixelArraySize.height + |OOPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPOO| | PixelArrayReadableArea.height ... ... ... ... ... | ... ... ... ... ... | |OOPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPOO| | @@ -578,14 +595,17 @@ controls: +----------------------------------------------+ / then the invalid lines and columns should not be reported as part of the - PixelArraySize property in first place. + PixelArrayReadableArea property in first place. As the coordinates of + PixelArrayReadableArea are relative to the physical pixel array, care + must be taken to report it in a way that the valid pixel data maps to + the correct physical position. In this case, the position of the black pixel rectangles will be PixelArrayOpticalBlackRectangles = { - { 0, 0, y1 + 1, PixelArraySize[0] }, - { 0, y1, 2, PixelArraySize[1] - y1 + 1 }, - { x1, y1, 2, PixelArraySize[1] - y1 + 1 }, + { 0, 0, PixelArrayReadableArea.width, y1 + 1 }, + { 0, y1, 2, PixelArrayReadableArea.height - y1 + 1 }, + { x1, y1, 2, PixelArrayReadableArea.height - y1 + 1 }, }; \todo Rename this property to Size once we will have property diff --git a/src/libcamera/sensor/camera_sensor_legacy.cpp b/src/libcamera/sensor/camera_sensor_legacy.cpp index 6a683821f219..482e387d3b24 100644 --- a/src/libcamera/sensor/camera_sensor_legacy.cpp +++ b/src/libcamera/sensor/camera_sensor_legacy.cpp @@ -38,6 +38,7 @@ #include "libcamera/internal/media_device.h" #include "libcamera/internal/sysfs.h" #include "libcamera/internal/v4l2_subdevice.h" +#include "linux/v4l2-common.h" namespace libcamera { @@ -131,6 +132,7 @@ private: controls::draft::TestPatternModeEnum testPatternMode_; Size pixelArraySize_; + Rectangle readableArea_; Rectangle activeArea_; const BayerFormat *bayerFormat_; bool supportFlips_; @@ -390,7 +392,7 @@ int CameraSensorLegacy::validateSensorDriver() * test platforms have been updated. */ Rectangle rect; - int ret = subdev_->getSelection(pad_, V4L2_SEL_TGT_CROP_BOUNDS, &rect); + int ret = subdev_->getSelection(pad_, V4L2_SEL_TGT_NATIVE_SIZE, &rect); if (ret) { /* * Default the pixel array size to the largest size supported @@ -407,9 +409,18 @@ int CameraSensorLegacy::validateSensorDriver() pixelArraySize_ = rect.size(); } + ret = subdev_->getSelection(pad_, V4L2_SEL_TGT_CROP_BOUNDS, &readableArea_); + if (ret) { + readableArea_ = Rectangle(pixelArraySize_); + LOG(CameraSensor, Warning) + << "The PixelArrayReadableArea property has been defaulted to " + << readableArea_; + err = -EINVAL; + } + ret = subdev_->getSelection(pad_, V4L2_SEL_TGT_CROP_DEFAULT, &activeArea_); if (ret) { - activeArea_ = Rectangle(pixelArraySize_); + activeArea_ = readableArea_; LOG(CameraSensor, Warning) << "The PixelArrayActiveAreas property has been defaulted to " << activeArea_; @@ -621,6 +632,7 @@ int CameraSensorLegacy::initProperties() } properties_.set(properties::PixelArraySize, pixelArraySize_); + properties_.set(properties::PixelArrayReadableArea, readableArea_); properties_.set(properties::PixelArrayActiveAreas, { activeArea_ }); /* Color filter array pattern, register only for RAW sensors. */ @@ -882,16 +894,6 @@ int CameraSensorLegacy::sensorInfo(IPACameraSensorInfo *info) const << "The analogue crop rectangle has been defaulted to the active area size"; } - /* - * IPACameraSensorInfo::analogCrop::x and IPACameraSensorInfo::analogCrop::y - * are defined relatively to the active pixel area, while V4L2's - * TGT_CROP target is defined in respect to the full pixel array. - * - * Compensate it by subtracting the active area offset. - */ - info->analogCrop.x -= activeArea_.x; - info->analogCrop.y -= activeArea_.y; - /* The bit depth and image size depend on the currently applied format. */ V4L2SubdeviceFormat format{}; ret = subdev_->getFormat(pad_, &format); diff --git a/src/libcamera/sensor/camera_sensor_raw.cpp b/src/libcamera/sensor/camera_sensor_raw.cpp index 10eba0331fe8..410edeb68983 100644 --- a/src/libcamera/sensor/camera_sensor_raw.cpp +++ b/src/libcamera/sensor/camera_sensor_raw.cpp @@ -416,6 +416,12 @@ std::optional CameraSensorRaw::init() auto last = std::unique(sizes_.begin(), sizes_.end()); sizes_.erase(last, sizes_.end()); + /* + * \todo Implement querying the physical sensor size based on pad 1, + * stream 0. See + * https://lore.kernel.org/linux-media/20260409201501.975242-23-sakari.ailus@linux.intel.com/ + */ + /* * 3. Query selection rectangles. Retrieve properties, and verify that * all the expected selection rectangles are supported. @@ -999,16 +1005,6 @@ int CameraSensorRaw::sensorInfo(IPACameraSensorInfo *info) const if (ret) return ret; - /* - * IPACameraSensorInfo::analogCrop::x and IPACameraSensorInfo::analogCrop::y - * are defined relatively to the active pixel area, while V4L2's - * TGT_CROP target is defined in respect to the full pixel array. - * - * Compensate it by subtracting the active area offset. - */ - info->analogCrop.x -= activeArea_.x; - info->analogCrop.y -= activeArea_.y; - /* The bit depth and image size depend on the currently applied format. */ V4L2SubdeviceFormat format{}; ret = subdev_->getFormat(streams_.image.source, &format);