| Message ID | 20260616120828.375547-2-stefan.klug@ideasonboard.com |
|---|---|
| State | New |
| Headers | show |
| Series |
|
| Related | show |
Quoting Stefan Klug (2026-06-16 21:08:14) > Current documented behavior for PixelArraySize is to report the readable > array size of the sensor data. If the readable array is offset relative > to the physical sensor array, all related rectangles (ScalerCrop, > PixelArrayActiveAreas) shall be reported relative to PixelArraySize. > > However no known pipeline and IPA implements it that way. All > pipelines/IPAs treat PixelArraySize as the physical sensor size and > handle the rectangles relative to that size. > > Adjust the documentation to fit the actual implementations. Add another > property named PixelArrayReadableArea to convey the information that > PixelArraySize was intended for. > > This has the added benefit that all rectangles are relative to the > physical sensor array, which is necessary to align tuning data like lens > shading correction. Makes sense to me. > > Signed-off-by: Stefan Klug <stefan.klug@ideasonboard.com> > > --- > > ToDo: > - Implement the expected behavior in CameraSensorRaw > --- > include/libcamera/ipa/core.mojom | 2 +- > src/libcamera/property_ids_core.yaml | 116 ++++++++++-------- > src/libcamera/sensor/camera_sensor_legacy.cpp | 26 ++-- > src/libcamera/sensor/camera_sensor_raw.cpp | 16 +-- > 4 files changed, 89 insertions(+), 71 deletions(-) > > diff --git a/include/libcamera/ipa/core.mojom b/include/libcamera/ipa/core.mojom > index bce797245829..5da780f5d256 100644 > --- a/include/libcamera/ipa/core.mojom > +++ b/include/libcamera/ipa/core.mojom > @@ -154,7 +154,7 @@ module libcamera; > > /** > * \var IPACameraSensorInfo::activeAreaSize > - * \brief The size of the pixel array active area of the sensor > + * \brief The size of the physical pixel array of the sensor in pixels > */ > > /** > diff --git a/src/libcamera/property_ids_core.yaml b/src/libcamera/property_ids_core.yaml > index d5b3d309c055..96c85f9233fd 100644 > --- a/src/libcamera/property_ids_core.yaml > +++ b/src/libcamera/property_ids_core.yaml > @@ -427,11 +427,29 @@ controls: > - PixelArraySize: > type: Size > description: | > - The camera sensor pixel array readable area vertical and horizontal > - sizes, in pixels. > + The camera sensor pixel array vertical and horizontal sizes, in pixels. > > - The PixelArraySize property defines the size in pixel units of the > - readable part of full pixel array matrix, including optical black > + The PixelArraySize property defines the size in pixel units of the full > + pixel array matrix. It is often not be possible to capture the whole s/be /// > + physical pixel array. For the largest readable area \see > + PixelArrayReadableArea. > + > + This defines a rectangle whose top-left corner is placed in position (0, > + 0) and whose vertical and horizontal sizes are defined by this property. > + All other rectangles that describe portions of the pixel array, such as > + the optical black pixels rectangles and active pixel areas, are defined > + relatively to this rectangle. s/relatively/relative/ Looks good to me. Reviewed-by: Paul Elder <paul.elder@ideasonboard.com> > + > + \todo Rename this property to Size once we will have property > + categories (i.e. Properties::PixelArray::Size) > + > + - PixelArrayReadableArea: > + type: Rectangle > + description: | > + The camera sensor pixel array readable area. > + > + The PixelArrayReadableArea property defines the rectangle in pixel units > + of the readable part of full pixel array matrix, including optical black > pixels used for calibration, pixels which are not considered valid for > capture and active pixels containing valid image data. > > @@ -442,23 +460,25 @@ controls: > > For example, let's consider a pixel array matrix assembled as follows > > - +--------------------------------------------------+ > - |xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx| > - |xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx| > - |xxDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDxx| > - |xxDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDxx| > - |xxDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDxx| > - |xxDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDxx| > - |xxDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDxx| > - |xxDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDxx| > + PixelArraySize.width > + /--------------------------------------------------/ > + +--------------------------------------------------+ / > + |xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx| | > + |xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx| | > + |xxDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDxx| | > + |xxDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDxx| | > + |xxDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDxx| | > + |xxDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDxx| | > + |xxDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDxx| | > + |xxDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDxx| | PixelArraySize.height > ... ... ... ... ... > > ... ... ... ... ... > - |xxDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDxx| > - |xxDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDxx| > - |xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx| > - |xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx| > - +--------------------------------------------------+ > + |xxDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDxx| | > + |xxDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDxx| | > + |xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx| | > + |xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx| | > + +--------------------------------------------------+ / > > starting with two lines of non-readable pixels (x), followed by N lines > of readable data (D) surrounded by two columns of non-readable pixels on > @@ -467,35 +487,32 @@ controls: > sizes of the largest possible buffer of raw data that can be presented > to applications. > > - PixelArraySize.width > - /----------------------------------------------/ > - +----------------------------------------------+ / > - |DDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDD| | > - |DDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDD| | > - |DDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDD| | > - |DDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDD| | > - |DDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDD| | > - |DDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDD| | PixelArraySize.height > - ... ... ... ... ... > - ... ... ... ... ... > - |DDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDD| | > - |DDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDD| | > - +----------------------------------------------+ / > + PixelArrayReadableArea.width > + /--------------------------------------------/ > + PixelArrayReadableArea.x > + +--o-----------------------------------------------+ > + |xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx| > + |xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx| > + oxxDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDxx| / PixelArrayReadableArea.y > + |xxDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDxx| | > + |xxDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDxx| | > + |xxDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDxx| | > + |xxDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDxx| | > + |xxDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDxx| | PixelArrayReadableArea.height > + ... ... ... ... ... > > - This defines a rectangle whose top-left corner is placed in position (0, > - 0) and whose vertical and horizontal sizes are defined by this property. > - All other rectangles that describe portions of the pixel array, such as > - the optical black pixels rectangles and active pixel areas, are defined > - relatively to this rectangle. > + ... ... ... ... ... > + |xxDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDxx| | > + |xxDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDxx| / > + |xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx| > + |xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx| > + +--------------------------------------------------+ > > All the coordinates are expressed relative to the default sensor readout > direction, without any transformation (such as horizontal and vertical > flipping) applied. When mapping them to the raw pixel buffer, > applications shall take any configured transformation into account. > > - \todo Rename this property to Size once we will have property > - categories (i.e. Properties::PixelArray::Size) > - > - PixelArrayOpticalBlackRectangles: > type: Rectangle > size: [n] > @@ -519,7 +536,7 @@ controls: > they could be positioned too close to the chip margins or to the optical > black shielding placed on top of optical black pixels. > > - PixelArraySize.width > + PixelArrayReadableArea.width > /----------------------------------------------/ > x1 x2 > +--o---------------------------------------o---+ / > @@ -532,7 +549,7 @@ controls: > |IIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIII| | > |IIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIII| | > y3 |IIOOPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPOOII| | > - |IIOOPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPOOII| | PixelArraySize.height > + |IIOOPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPOOII| | PixelArrayReadableArea.height > |IIOOPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPOOII| | > ... ... ... ... ... > ... ... ... ... ... > @@ -560,7 +577,7 @@ controls: > skips the invalid lines and columns, producing the following data > buffer, when captured to memory > > - PixelArraySize.width > + PixelArrayReadableArea.width > /----------------------------------------------/ > x1 > +--------------------------------------------o-+ / > @@ -570,7 +587,7 @@ controls: > |OOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOO| | > y1 oOOPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPOO| | > |OOPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPOO| | > - |OOPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPOO| | PixelArraySize.height > + |OOPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPOO| | PixelArrayReadableArea.height > ... ... ... ... ... | > ... ... ... ... ... | > |OOPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPOO| | > @@ -578,14 +595,17 @@ controls: > +----------------------------------------------+ / > > then the invalid lines and columns should not be reported as part of the > - PixelArraySize property in first place. > + PixelArrayReadableArea property in first place. As the coordinates of > + PixelArrayReadableArea are relative to the physical pixel array, care > + must be taken to report it in a way that the valid pixel data maps to > + the correct physical position. > > In this case, the position of the black pixel rectangles will be > > PixelArrayOpticalBlackRectangles = { > - { 0, 0, y1 + 1, PixelArraySize[0] }, > - { 0, y1, 2, PixelArraySize[1] - y1 + 1 }, > - { x1, y1, 2, PixelArraySize[1] - y1 + 1 }, > + { 0, 0, PixelArrayReadableArea.width, y1 + 1 }, > + { 0, y1, 2, PixelArrayReadableArea.height - y1 + 1 }, > + { x1, y1, 2, PixelArrayReadableArea.height - y1 + 1 }, > }; > > \todo Rename this property to Size once we will have property > diff --git a/src/libcamera/sensor/camera_sensor_legacy.cpp b/src/libcamera/sensor/camera_sensor_legacy.cpp > index 6a683821f219..482e387d3b24 100644 > --- a/src/libcamera/sensor/camera_sensor_legacy.cpp > +++ b/src/libcamera/sensor/camera_sensor_legacy.cpp > @@ -38,6 +38,7 @@ > #include "libcamera/internal/media_device.h" > #include "libcamera/internal/sysfs.h" > #include "libcamera/internal/v4l2_subdevice.h" > +#include "linux/v4l2-common.h" > > namespace libcamera { > > @@ -131,6 +132,7 @@ private: > controls::draft::TestPatternModeEnum testPatternMode_; > > Size pixelArraySize_; > + Rectangle readableArea_; > Rectangle activeArea_; > const BayerFormat *bayerFormat_; > bool supportFlips_; > @@ -390,7 +392,7 @@ int CameraSensorLegacy::validateSensorDriver() > * test platforms have been updated. > */ > Rectangle rect; > - int ret = subdev_->getSelection(pad_, V4L2_SEL_TGT_CROP_BOUNDS, &rect); > + int ret = subdev_->getSelection(pad_, V4L2_SEL_TGT_NATIVE_SIZE, &rect); > if (ret) { > /* > * Default the pixel array size to the largest size supported > @@ -407,9 +409,18 @@ int CameraSensorLegacy::validateSensorDriver() > pixelArraySize_ = rect.size(); > } > > + ret = subdev_->getSelection(pad_, V4L2_SEL_TGT_CROP_BOUNDS, &readableArea_); > + if (ret) { > + readableArea_ = Rectangle(pixelArraySize_); > + LOG(CameraSensor, Warning) > + << "The PixelArrayReadableArea property has been defaulted to " > + << readableArea_; > + err = -EINVAL; > + } > + > ret = subdev_->getSelection(pad_, V4L2_SEL_TGT_CROP_DEFAULT, &activeArea_); > if (ret) { > - activeArea_ = Rectangle(pixelArraySize_); > + activeArea_ = readableArea_; > LOG(CameraSensor, Warning) > << "The PixelArrayActiveAreas property has been defaulted to " > << activeArea_; > @@ -621,6 +632,7 @@ int CameraSensorLegacy::initProperties() > } > > properties_.set(properties::PixelArraySize, pixelArraySize_); > + properties_.set(properties::PixelArrayReadableArea, readableArea_); > properties_.set(properties::PixelArrayActiveAreas, { activeArea_ }); > > /* Color filter array pattern, register only for RAW sensors. */ > @@ -882,16 +894,6 @@ int CameraSensorLegacy::sensorInfo(IPACameraSensorInfo *info) const > << "The analogue crop rectangle has been defaulted to the active area size"; > } > > - /* > - * IPACameraSensorInfo::analogCrop::x and IPACameraSensorInfo::analogCrop::y > - * are defined relatively to the active pixel area, while V4L2's > - * TGT_CROP target is defined in respect to the full pixel array. > - * > - * Compensate it by subtracting the active area offset. > - */ > - info->analogCrop.x -= activeArea_.x; > - info->analogCrop.y -= activeArea_.y; > - > /* The bit depth and image size depend on the currently applied format. */ > V4L2SubdeviceFormat format{}; > ret = subdev_->getFormat(pad_, &format); > diff --git a/src/libcamera/sensor/camera_sensor_raw.cpp b/src/libcamera/sensor/camera_sensor_raw.cpp > index 10eba0331fe8..410edeb68983 100644 > --- a/src/libcamera/sensor/camera_sensor_raw.cpp > +++ b/src/libcamera/sensor/camera_sensor_raw.cpp > @@ -416,6 +416,12 @@ std::optional<int> CameraSensorRaw::init() > auto last = std::unique(sizes_.begin(), sizes_.end()); > sizes_.erase(last, sizes_.end()); > > + /* > + * \todo Implement querying the physical sensor size based on pad 1, > + * stream 0. See > + * https://lore.kernel.org/linux-media/20260409201501.975242-23-sakari.ailus@linux.intel.com/ > + */ > + > /* > * 3. Query selection rectangles. Retrieve properties, and verify that > * all the expected selection rectangles are supported. > @@ -999,16 +1005,6 @@ int CameraSensorRaw::sensorInfo(IPACameraSensorInfo *info) const > if (ret) > return ret; > > - /* > - * IPACameraSensorInfo::analogCrop::x and IPACameraSensorInfo::analogCrop::y > - * are defined relatively to the active pixel area, while V4L2's > - * TGT_CROP target is defined in respect to the full pixel array. > - * > - * Compensate it by subtracting the active area offset. > - */ > - info->analogCrop.x -= activeArea_.x; > - info->analogCrop.y -= activeArea_.y; > - > /* The bit depth and image size depend on the currently applied format. */ > V4L2SubdeviceFormat format{}; > ret = subdev_->getFormat(streams_.image.source, &format); > -- > 2.53.0 >
Hi Stefan On Tue, Jun 16, 2026 at 02:08:14PM +0200, Stefan Klug wrote: > Current documented behavior for PixelArraySize is to report the readable > array size of the sensor data. If the readable array is offset relative > to the physical sensor array, all related rectangles (ScalerCrop, > PixelArrayActiveAreas) shall be reported relative to PixelArraySize. > > However no known pipeline and IPA implements it that way. All > pipelines/IPAs treat PixelArraySize as the physical sensor size and > handle the rectangles relative to that size. > > Adjust the documentation to fit the actual implementations. Add another > property named PixelArrayReadableArea to convey the information that > PixelArraySize was intended for. > > This has the added benefit that all rectangles are relative to the > physical sensor array, which is necessary to align tuning data like lens > shading correction. > > Signed-off-by: Stefan Klug <stefan.klug@ideasonboard.com> > > --- > > ToDo: > - Implement the expected behavior in CameraSensorRaw > --- > include/libcamera/ipa/core.mojom | 2 +- > src/libcamera/property_ids_core.yaml | 116 ++++++++++-------- > src/libcamera/sensor/camera_sensor_legacy.cpp | 26 ++-- > src/libcamera/sensor/camera_sensor_raw.cpp | 16 +-- > 4 files changed, 89 insertions(+), 71 deletions(-) > > diff --git a/include/libcamera/ipa/core.mojom b/include/libcamera/ipa/core.mojom > index bce797245829..5da780f5d256 100644 > --- a/include/libcamera/ipa/core.mojom > +++ b/include/libcamera/ipa/core.mojom > @@ -154,7 +154,7 @@ module libcamera; > > /** > * \var IPACameraSensorInfo::activeAreaSize > - * \brief The size of the pixel array active area of the sensor > + * \brief The size of the physical pixel array of the sensor in pixels You should change the description in the next patch, as you're not yet modifying CameraSensorLegacy::sensorInfo() > */ > > /** > diff --git a/src/libcamera/property_ids_core.yaml b/src/libcamera/property_ids_core.yaml > index d5b3d309c055..96c85f9233fd 100644 > --- a/src/libcamera/property_ids_core.yaml > +++ b/src/libcamera/property_ids_core.yaml > @@ -427,11 +427,29 @@ controls: > - PixelArraySize: > type: Size > description: | > - The camera sensor pixel array readable area vertical and horizontal > - sizes, in pixels. > + The camera sensor pixel array vertical and horizontal sizes, in pixels. > > - The PixelArraySize property defines the size in pixel units of the > - readable part of full pixel array matrix, including optical black > + The PixelArraySize property defines the size in pixel units of the full > + pixel array matrix. It is often not be possible to capture the whole > + physical pixel array. For the largest readable area \see > + PixelArrayReadableArea. What about just: The PixelArraySize property defines the size in pixel units of the full pixel array matrix including readable and non-readable pixels. > + > + This defines a rectangle whose top-left corner is placed in position (0, s/This/This property/ > + 0) and whose vertical and horizontal sizes are defined by this property. If you want to make it about rectangles: This property defines the horizontal and vertical sizes of a rectangle whose top-left corner is placed in position (0, 0). > + All other rectangles that describe portions of the pixel array, such as > + the optical black pixels rectangles and active pixel areas, are defined Should we name the properties instead of just describing them ? All the other properties that describe portions of the pixel array such as PixelArrayOpticalBlackRectangles, PixelArrayReadableArea and PixelArrayActiveAreas are defined relatively to this rectangle. > + relatively to this rectangle. > + > + \todo Rename this property to Size once we will have property > + categories (i.e. Properties::PixelArray::Size) > + > + - PixelArrayReadableArea: > + type: Rectangle > + description: | > + The camera sensor pixel array readable area. > + > + The PixelArrayReadableArea property defines the rectangle in pixel units > + of the readable part of full pixel array matrix, including optical black > pixels used for calibration, pixels which are not considered valid for > capture and active pixels containing valid image data. I understand that the description of PixelArraySize now says that all other rectangles are defined relatively to it, but should we repeat it or not ? The PixelArrayReadableArea property defines the rectangle in pixel units relatively to PixelArraySize of the readable part of full pixel array matrix, including optical black Same for PixelArrayOpticalBlackRectangles FTR the PixelArrayActiveAreas property already does that. Should we say that PixelArrayOpticalBlackRectangles and PixelArrayActiveAreas are always contained in PixelArrayReadableArea ? > > @@ -442,23 +460,25 @@ controls: > > For example, let's consider a pixel array matrix assembled as follows > > - +--------------------------------------------------+ > - |xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx| > - |xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx| > - |xxDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDxx| > - |xxDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDxx| > - |xxDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDxx| > - |xxDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDxx| > - |xxDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDxx| > - |xxDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDxx| > + PixelArraySize.width > + /--------------------------------------------------/ > + +--------------------------------------------------+ / > + |xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx| | > + |xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx| | > + |xxDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDxx| | > + |xxDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDxx| | > + |xxDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDxx| | > + |xxDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDxx| | > + |xxDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDxx| | > + |xxDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDxx| | PixelArraySize.height > ... ... ... ... ... > > ... ... ... ... ... > - |xxDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDxx| > - |xxDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDxx| > - |xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx| > - |xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx| > - +--------------------------------------------------+ > + |xxDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDxx| | > + |xxDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDxx| | > + |xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx| | > + |xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx| | > + +--------------------------------------------------+ / > > starting with two lines of non-readable pixels (x), followed by N lines > of readable data (D) surrounded by two columns of non-readable pixels on > @@ -467,35 +487,32 @@ controls: > sizes of the largest possible buffer of raw data that can be presented > to applications. > > - PixelArraySize.width > - /----------------------------------------------/ > - +----------------------------------------------+ / > - |DDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDD| | > - |DDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDD| | > - |DDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDD| | > - |DDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDD| | > - |DDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDD| | > - |DDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDD| | PixelArraySize.height > - ... ... ... ... ... > - ... ... ... ... ... > - |DDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDD| | > - |DDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDD| | > - +----------------------------------------------+ / > + PixelArrayReadableArea.width > + /--------------------------------------------/ > + PixelArrayReadableArea.x > + +--o-----------------------------------------------+ > + |xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx| > + |xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx| > + oxxDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDxx| / PixelArrayReadableArea.y > + |xxDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDxx| | > + |xxDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDxx| | > + |xxDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDxx| | > + |xxDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDxx| | > + |xxDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDxx| | PixelArrayReadableArea.height > + ... ... ... ... ... > > - This defines a rectangle whose top-left corner is placed in position (0, > - 0) and whose vertical and horizontal sizes are defined by this property. > - All other rectangles that describe portions of the pixel array, such as > - the optical black pixels rectangles and active pixel areas, are defined > - relatively to this rectangle. > + ... ... ... ... ... > + |xxDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDxx| | > + |xxDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDxx| / > + |xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx| > + |xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx| > + +--------------------------------------------------+ > > All the coordinates are expressed relative to the default sensor readout > direction, without any transformation (such as horizontal and vertical > flipping) applied. When mapping them to the raw pixel buffer, > applications shall take any configured transformation into account. > > - \todo Rename this property to Size once we will have property > - categories (i.e. Properties::PixelArray::Size) > - > - PixelArrayOpticalBlackRectangles: > type: Rectangle > size: [n] > @@ -519,7 +536,7 @@ controls: > they could be positioned too close to the chip margins or to the optical > black shielding placed on top of optical black pixels. > > - PixelArraySize.width > + PixelArrayReadableArea.width > /----------------------------------------------/ > x1 x2 > +--o---------------------------------------o---+ / > @@ -532,7 +549,7 @@ controls: > |IIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIII| | > |IIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIII| | > y3 |IIOOPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPOOII| | > - |IIOOPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPOOII| | PixelArraySize.height > + |IIOOPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPOOII| | PixelArrayReadableArea.height > |IIOOPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPOOII| | > ... ... ... ... ... > ... ... ... ... ... > @@ -560,7 +577,7 @@ controls: > skips the invalid lines and columns, producing the following data > buffer, when captured to memory > > - PixelArraySize.width > + PixelArrayReadableArea.width > /----------------------------------------------/ > x1 > +--------------------------------------------o-+ / > @@ -570,7 +587,7 @@ controls: > |OOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOO| | > y1 oOOPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPOO| | > |OOPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPOO| | > - |OOPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPOO| | PixelArraySize.height > + |OOPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPOO| | PixelArrayReadableArea.height > ... ... ... ... ... | > ... ... ... ... ... | > |OOPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPOO| | > @@ -578,14 +595,17 @@ controls: > +----------------------------------------------+ / > > then the invalid lines and columns should not be reported as part of the > - PixelArraySize property in first place. > + PixelArrayReadableArea property in first place. As the coordinates of > + PixelArrayReadableArea are relative to the physical pixel array, care > + must be taken to report it in a way that the valid pixel data maps to > + the correct physical position. > > In this case, the position of the black pixel rectangles will be > > PixelArrayOpticalBlackRectangles = { > - { 0, 0, y1 + 1, PixelArraySize[0] }, > - { 0, y1, 2, PixelArraySize[1] - y1 + 1 }, > - { x1, y1, 2, PixelArraySize[1] - y1 + 1 }, > + { 0, 0, PixelArrayReadableArea.width, y1 + 1 }, should the vertical height be 'y1 - 1' ? > + { 0, y1, 2, PixelArrayReadableArea.height - y1 + 1 }, > + { x1, y1, 2, PixelArrayReadableArea.height - y1 + 1 }, > }; > > \todo Rename this property to Size once we will have property > diff --git a/src/libcamera/sensor/camera_sensor_legacy.cpp b/src/libcamera/sensor/camera_sensor_legacy.cpp > index 6a683821f219..482e387d3b24 100644 > --- a/src/libcamera/sensor/camera_sensor_legacy.cpp > +++ b/src/libcamera/sensor/camera_sensor_legacy.cpp > @@ -38,6 +38,7 @@ > #include "libcamera/internal/media_device.h" > #include "libcamera/internal/sysfs.h" > #include "libcamera/internal/v4l2_subdevice.h" > +#include "linux/v4l2-common.h" libcamera/internal/v4l2_subdevice.h includes include/linux/v4l2-subdev.h which includes v4l2-common.h > > namespace libcamera { > > @@ -131,6 +132,7 @@ private: > controls::draft::TestPatternModeEnum testPatternMode_; > > Size pixelArraySize_; > + Rectangle readableArea_; > Rectangle activeArea_; > const BayerFormat *bayerFormat_; > bool supportFlips_; > @@ -390,7 +392,7 @@ int CameraSensorLegacy::validateSensorDriver() > * test platforms have been updated. > */ > Rectangle rect; > - int ret = subdev_->getSelection(pad_, V4L2_SEL_TGT_CROP_BOUNDS, &rect); > + int ret = subdev_->getSelection(pad_, V4L2_SEL_TGT_NATIVE_SIZE, &rect); > if (ret) { > /* > * Default the pixel array size to the largest size supported > @@ -407,9 +409,18 @@ int CameraSensorLegacy::validateSensorDriver() > pixelArraySize_ = rect.size(); > } > > + ret = subdev_->getSelection(pad_, V4L2_SEL_TGT_CROP_BOUNDS, &readableArea_); > + if (ret) { > + readableArea_ = Rectangle(pixelArraySize_); > + LOG(CameraSensor, Warning) > + << "The PixelArrayReadableArea property has been defaulted to " > + << readableArea_; > + err = -EINVAL; > + } > + > ret = subdev_->getSelection(pad_, V4L2_SEL_TGT_CROP_DEFAULT, &activeArea_); > if (ret) { > - activeArea_ = Rectangle(pixelArraySize_); > + activeArea_ = readableArea_; > LOG(CameraSensor, Warning) > << "The PixelArrayActiveAreas property has been defaulted to " > << activeArea_; > @@ -621,6 +632,7 @@ int CameraSensorLegacy::initProperties() > } > > properties_.set(properties::PixelArraySize, pixelArraySize_); > + properties_.set(properties::PixelArrayReadableArea, readableArea_); > properties_.set(properties::PixelArrayActiveAreas, { activeArea_ }); > > /* Color filter array pattern, register only for RAW sensors. */ > @@ -882,16 +894,6 @@ int CameraSensorLegacy::sensorInfo(IPACameraSensorInfo *info) const > << "The analogue crop rectangle has been defaulted to the active area size"; > } > > - /* > - * IPACameraSensorInfo::analogCrop::x and IPACameraSensorInfo::analogCrop::y > - * are defined relatively to the active pixel area, while V4L2's > - * TGT_CROP target is defined in respect to the full pixel array. > - * > - * Compensate it by subtracting the active area offset. > - */ > - info->analogCrop.x -= activeArea_.x; > - info->analogCrop.y -= activeArea_.y; > - The documentation of IPACameraSensorInfo::analogCrop still reports: * The analog crop rectangle top-left corner is defined as the displacement * from the top-left corner of the pixel array active area. The rectangle If you want to make it relative to the physical size you should update the documentation. I would do so (and move this hunk) to the next patch ? > /* The bit depth and image size depend on the currently applied format. */ > V4L2SubdeviceFormat format{}; > ret = subdev_->getFormat(pad_, &format); > diff --git a/src/libcamera/sensor/camera_sensor_raw.cpp b/src/libcamera/sensor/camera_sensor_raw.cpp > index 10eba0331fe8..410edeb68983 100644 > --- a/src/libcamera/sensor/camera_sensor_raw.cpp > +++ b/src/libcamera/sensor/camera_sensor_raw.cpp > @@ -416,6 +416,12 @@ std::optional<int> CameraSensorRaw::init() > auto last = std::unique(sizes_.begin(), sizes_.end()); > sizes_.erase(last, sizes_.end()); > > + /* > + * \todo Implement querying the physical sensor size based on pad 1, > + * stream 0. See > + * https://lore.kernel.org/linux-media/20260409201501.975242-23-sakari.ailus@linux.intel.com/ > + */ > + > /* > * 3. Query selection rectangles. Retrieve properties, and verify that > * all the expected selection rectangles are supported. > @@ -999,16 +1005,6 @@ int CameraSensorRaw::sensorInfo(IPACameraSensorInfo *info) const > if (ret) > return ret; > > - /* > - * IPACameraSensorInfo::analogCrop::x and IPACameraSensorInfo::analogCrop::y > - * are defined relatively to the active pixel area, while V4L2's > - * TGT_CROP target is defined in respect to the full pixel array. > - * > - * Compensate it by subtracting the active area offset. > - */ > - info->analogCrop.x -= activeArea_.x; > - info->analogCrop.y -= activeArea_.y; > - Same here, even if this needs to be reworked anyway > /* The bit depth and image size depend on the currently applied format. */ > V4L2SubdeviceFormat format{}; > ret = subdev_->getFormat(streams_.image.source, &format); > -- > 2.53.0 >
diff --git a/include/libcamera/ipa/core.mojom b/include/libcamera/ipa/core.mojom index bce797245829..5da780f5d256 100644 --- a/include/libcamera/ipa/core.mojom +++ b/include/libcamera/ipa/core.mojom @@ -154,7 +154,7 @@ module libcamera; /** * \var IPACameraSensorInfo::activeAreaSize - * \brief The size of the pixel array active area of the sensor + * \brief The size of the physical pixel array of the sensor in pixels */ /** diff --git a/src/libcamera/property_ids_core.yaml b/src/libcamera/property_ids_core.yaml index d5b3d309c055..96c85f9233fd 100644 --- a/src/libcamera/property_ids_core.yaml +++ b/src/libcamera/property_ids_core.yaml @@ -427,11 +427,29 @@ controls: - PixelArraySize: type: Size description: | - The camera sensor pixel array readable area vertical and horizontal - sizes, in pixels. + The camera sensor pixel array vertical and horizontal sizes, in pixels. - The PixelArraySize property defines the size in pixel units of the - readable part of full pixel array matrix, including optical black + The PixelArraySize property defines the size in pixel units of the full + pixel array matrix. It is often not be possible to capture the whole + physical pixel array. For the largest readable area \see + PixelArrayReadableArea. + + This defines a rectangle whose top-left corner is placed in position (0, + 0) and whose vertical and horizontal sizes are defined by this property. + All other rectangles that describe portions of the pixel array, such as + the optical black pixels rectangles and active pixel areas, are defined + relatively to this rectangle. + + \todo Rename this property to Size once we will have property + categories (i.e. Properties::PixelArray::Size) + + - PixelArrayReadableArea: + type: Rectangle + description: | + The camera sensor pixel array readable area. + + The PixelArrayReadableArea property defines the rectangle in pixel units + of the readable part of full pixel array matrix, including optical black pixels used for calibration, pixels which are not considered valid for capture and active pixels containing valid image data. @@ -442,23 +460,25 @@ controls: For example, let's consider a pixel array matrix assembled as follows - +--------------------------------------------------+ - |xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx| - |xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx| - |xxDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDxx| - |xxDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDxx| - |xxDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDxx| - |xxDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDxx| - |xxDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDxx| - |xxDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDxx| + PixelArraySize.width + /--------------------------------------------------/ + +--------------------------------------------------+ / + |xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx| | + |xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx| | + |xxDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDxx| | + |xxDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDxx| | + |xxDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDxx| | + |xxDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDxx| | + |xxDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDxx| | + |xxDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDxx| | PixelArraySize.height ... ... ... ... ... ... ... ... ... ... - |xxDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDxx| - |xxDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDxx| - |xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx| - |xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx| - +--------------------------------------------------+ + |xxDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDxx| | + |xxDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDxx| | + |xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx| | + |xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx| | + +--------------------------------------------------+ / starting with two lines of non-readable pixels (x), followed by N lines of readable data (D) surrounded by two columns of non-readable pixels on @@ -467,35 +487,32 @@ controls: sizes of the largest possible buffer of raw data that can be presented to applications. - PixelArraySize.width - /----------------------------------------------/ - +----------------------------------------------+ / - |DDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDD| | - |DDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDD| | - |DDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDD| | - |DDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDD| | - |DDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDD| | - |DDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDD| | PixelArraySize.height - ... ... ... ... ... - ... ... ... ... ... - |DDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDD| | - |DDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDD| | - +----------------------------------------------+ / + PixelArrayReadableArea.width + /--------------------------------------------/ + PixelArrayReadableArea.x + +--o-----------------------------------------------+ + |xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx| + |xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx| + oxxDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDxx| / PixelArrayReadableArea.y + |xxDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDxx| | + |xxDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDxx| | + |xxDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDxx| | + |xxDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDxx| | + |xxDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDxx| | PixelArrayReadableArea.height + ... ... ... ... ... - This defines a rectangle whose top-left corner is placed in position (0, - 0) and whose vertical and horizontal sizes are defined by this property. - All other rectangles that describe portions of the pixel array, such as - the optical black pixels rectangles and active pixel areas, are defined - relatively to this rectangle. + ... ... ... ... ... + |xxDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDxx| | + |xxDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDxx| / + |xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx| + |xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx| + +--------------------------------------------------+ All the coordinates are expressed relative to the default sensor readout direction, without any transformation (such as horizontal and vertical flipping) applied. When mapping them to the raw pixel buffer, applications shall take any configured transformation into account. - \todo Rename this property to Size once we will have property - categories (i.e. Properties::PixelArray::Size) - - PixelArrayOpticalBlackRectangles: type: Rectangle size: [n] @@ -519,7 +536,7 @@ controls: they could be positioned too close to the chip margins or to the optical black shielding placed on top of optical black pixels. - PixelArraySize.width + PixelArrayReadableArea.width /----------------------------------------------/ x1 x2 +--o---------------------------------------o---+ / @@ -532,7 +549,7 @@ controls: |IIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIII| | |IIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIII| | y3 |IIOOPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPOOII| | - |IIOOPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPOOII| | PixelArraySize.height + |IIOOPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPOOII| | PixelArrayReadableArea.height |IIOOPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPOOII| | ... ... ... ... ... ... ... ... ... ... @@ -560,7 +577,7 @@ controls: skips the invalid lines and columns, producing the following data buffer, when captured to memory - PixelArraySize.width + PixelArrayReadableArea.width /----------------------------------------------/ x1 +--------------------------------------------o-+ / @@ -570,7 +587,7 @@ controls: |OOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOO| | y1 oOOPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPOO| | |OOPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPOO| | - |OOPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPOO| | PixelArraySize.height + |OOPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPOO| | PixelArrayReadableArea.height ... ... ... ... ... | ... ... ... ... ... | |OOPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPPOO| | @@ -578,14 +595,17 @@ controls: +----------------------------------------------+ / then the invalid lines and columns should not be reported as part of the - PixelArraySize property in first place. + PixelArrayReadableArea property in first place. As the coordinates of + PixelArrayReadableArea are relative to the physical pixel array, care + must be taken to report it in a way that the valid pixel data maps to + the correct physical position. In this case, the position of the black pixel rectangles will be PixelArrayOpticalBlackRectangles = { - { 0, 0, y1 + 1, PixelArraySize[0] }, - { 0, y1, 2, PixelArraySize[1] - y1 + 1 }, - { x1, y1, 2, PixelArraySize[1] - y1 + 1 }, + { 0, 0, PixelArrayReadableArea.width, y1 + 1 }, + { 0, y1, 2, PixelArrayReadableArea.height - y1 + 1 }, + { x1, y1, 2, PixelArrayReadableArea.height - y1 + 1 }, }; \todo Rename this property to Size once we will have property diff --git a/src/libcamera/sensor/camera_sensor_legacy.cpp b/src/libcamera/sensor/camera_sensor_legacy.cpp index 6a683821f219..482e387d3b24 100644 --- a/src/libcamera/sensor/camera_sensor_legacy.cpp +++ b/src/libcamera/sensor/camera_sensor_legacy.cpp @@ -38,6 +38,7 @@ #include "libcamera/internal/media_device.h" #include "libcamera/internal/sysfs.h" #include "libcamera/internal/v4l2_subdevice.h" +#include "linux/v4l2-common.h" namespace libcamera { @@ -131,6 +132,7 @@ private: controls::draft::TestPatternModeEnum testPatternMode_; Size pixelArraySize_; + Rectangle readableArea_; Rectangle activeArea_; const BayerFormat *bayerFormat_; bool supportFlips_; @@ -390,7 +392,7 @@ int CameraSensorLegacy::validateSensorDriver() * test platforms have been updated. */ Rectangle rect; - int ret = subdev_->getSelection(pad_, V4L2_SEL_TGT_CROP_BOUNDS, &rect); + int ret = subdev_->getSelection(pad_, V4L2_SEL_TGT_NATIVE_SIZE, &rect); if (ret) { /* * Default the pixel array size to the largest size supported @@ -407,9 +409,18 @@ int CameraSensorLegacy::validateSensorDriver() pixelArraySize_ = rect.size(); } + ret = subdev_->getSelection(pad_, V4L2_SEL_TGT_CROP_BOUNDS, &readableArea_); + if (ret) { + readableArea_ = Rectangle(pixelArraySize_); + LOG(CameraSensor, Warning) + << "The PixelArrayReadableArea property has been defaulted to " + << readableArea_; + err = -EINVAL; + } + ret = subdev_->getSelection(pad_, V4L2_SEL_TGT_CROP_DEFAULT, &activeArea_); if (ret) { - activeArea_ = Rectangle(pixelArraySize_); + activeArea_ = readableArea_; LOG(CameraSensor, Warning) << "The PixelArrayActiveAreas property has been defaulted to " << activeArea_; @@ -621,6 +632,7 @@ int CameraSensorLegacy::initProperties() } properties_.set(properties::PixelArraySize, pixelArraySize_); + properties_.set(properties::PixelArrayReadableArea, readableArea_); properties_.set(properties::PixelArrayActiveAreas, { activeArea_ }); /* Color filter array pattern, register only for RAW sensors. */ @@ -882,16 +894,6 @@ int CameraSensorLegacy::sensorInfo(IPACameraSensorInfo *info) const << "The analogue crop rectangle has been defaulted to the active area size"; } - /* - * IPACameraSensorInfo::analogCrop::x and IPACameraSensorInfo::analogCrop::y - * are defined relatively to the active pixel area, while V4L2's - * TGT_CROP target is defined in respect to the full pixel array. - * - * Compensate it by subtracting the active area offset. - */ - info->analogCrop.x -= activeArea_.x; - info->analogCrop.y -= activeArea_.y; - /* The bit depth and image size depend on the currently applied format. */ V4L2SubdeviceFormat format{}; ret = subdev_->getFormat(pad_, &format); diff --git a/src/libcamera/sensor/camera_sensor_raw.cpp b/src/libcamera/sensor/camera_sensor_raw.cpp index 10eba0331fe8..410edeb68983 100644 --- a/src/libcamera/sensor/camera_sensor_raw.cpp +++ b/src/libcamera/sensor/camera_sensor_raw.cpp @@ -416,6 +416,12 @@ std::optional<int> CameraSensorRaw::init() auto last = std::unique(sizes_.begin(), sizes_.end()); sizes_.erase(last, sizes_.end()); + /* + * \todo Implement querying the physical sensor size based on pad 1, + * stream 0. See + * https://lore.kernel.org/linux-media/20260409201501.975242-23-sakari.ailus@linux.intel.com/ + */ + /* * 3. Query selection rectangles. Retrieve properties, and verify that * all the expected selection rectangles are supported. @@ -999,16 +1005,6 @@ int CameraSensorRaw::sensorInfo(IPACameraSensorInfo *info) const if (ret) return ret; - /* - * IPACameraSensorInfo::analogCrop::x and IPACameraSensorInfo::analogCrop::y - * are defined relatively to the active pixel area, while V4L2's - * TGT_CROP target is defined in respect to the full pixel array. - * - * Compensate it by subtracting the active area offset. - */ - info->analogCrop.x -= activeArea_.x; - info->analogCrop.y -= activeArea_.y; - /* The bit depth and image size depend on the currently applied format. */ V4L2SubdeviceFormat format{}; ret = subdev_->getFormat(streams_.image.source, &format);
Current documented behavior for PixelArraySize is to report the readable array size of the sensor data. If the readable array is offset relative to the physical sensor array, all related rectangles (ScalerCrop, PixelArrayActiveAreas) shall be reported relative to PixelArraySize. However no known pipeline and IPA implements it that way. All pipelines/IPAs treat PixelArraySize as the physical sensor size and handle the rectangles relative to that size. Adjust the documentation to fit the actual implementations. Add another property named PixelArrayReadableArea to convey the information that PixelArraySize was intended for. This has the added benefit that all rectangles are relative to the physical sensor array, which is necessary to align tuning data like lens shading correction. Signed-off-by: Stefan Klug <stefan.klug@ideasonboard.com> --- ToDo: - Implement the expected behavior in CameraSensorRaw --- include/libcamera/ipa/core.mojom | 2 +- src/libcamera/property_ids_core.yaml | 116 ++++++++++-------- src/libcamera/sensor/camera_sensor_legacy.cpp | 26 ++-- src/libcamera/sensor/camera_sensor_raw.cpp | 16 +-- 4 files changed, 89 insertions(+), 71 deletions(-)