Message ID | 20210730082832.152626-3-umang.jain@ideasonboard.com |
---|---|
State | Superseded |
Delegated to: | Umang Jain |
Headers | show |
Series |
|
Related | show |
On Fri, Jul 30, 2021 at 01:58:27PM +0530, Umang Jain wrote: > This prepares a base to introduce custom selection of sensor format > based on platform(soraka or nautilus) constraints. The changes in > selection policy will be introduced in a subsequent patch. > > No functional changes in this patch. > > Signed-off-by: Umang Jain <umang.jain@ideasonboard.com> > --- > src/libcamera/pipeline/ipu3/cio2.cpp | 101 ++++++++++++++++++++++++++- > src/libcamera/pipeline/ipu3/cio2.h | 5 ++ > 2 files changed, 104 insertions(+), 2 deletions(-) > > diff --git a/src/libcamera/pipeline/ipu3/cio2.cpp b/src/libcamera/pipeline/ipu3/cio2.cpp > index 1bcd580e..aef88afd 100644 > --- a/src/libcamera/pipeline/ipu3/cio2.cpp > +++ b/src/libcamera/pipeline/ipu3/cio2.cpp > @@ -7,6 +7,8 @@ > > #include "cio2.h" > > +#include <float.h> > + #include <math.h> #include <limits> > #include <linux/media-bus-format.h> > > #include <libcamera/formats.h> > @@ -120,6 +122,14 @@ int CIO2Device::init(const MediaDevice *media, unsigned int index) > if (ret) > return ret; > > + formats_ = sensor_->formats(); > + if (formats_.empty()) { > + LOG(IPU3, Error) > + << "Sensor " << sensor_->entity()->name() > + << "has empty sensor format list"; > + return -EINVAL; > + } > + in CameraSensor /* Enumerate, sort and cache media bus codes and sizes. */ formats_ = subdev_->formats(pad_); if (formats_.empty()) { LOG(CameraSensor, Error) << "No image format found"; return -EINVAL; } > /* > * Make sure the sensor produces at least one format compatible with > * the CIO2 requirements. > @@ -168,7 +178,7 @@ int CIO2Device::configure(const Size &size, V4L2DeviceFormat *outputFormat) > * the CIO2 output device. > */ > std::vector<unsigned int> mbusCodes = utils::map_keys(mbusCodesToPixelFormat); > - sensorFormat = sensor_->getFormat(mbusCodes, size); > + sensorFormat = getSensorFormat(mbusCodes, size); > ret = sensor_->setFormat(&sensorFormat); > if (ret) > return ret; > @@ -206,7 +216,7 @@ StreamConfiguration CIO2Device::generateConfiguration(Size size) const > > /* Query the sensor static information for closest match. */ > std::vector<unsigned int> mbusCodes = utils::map_keys(mbusCodesToPixelFormat); > - V4L2SubdeviceFormat sensorFormat = sensor_->getFormat(mbusCodes, size); > + V4L2SubdeviceFormat sensorFormat = getSensorFormat(mbusCodes, size); > if (!sensorFormat.mbus_code) { > LOG(IPU3, Error) << "Sensor does not support mbus code"; > return {}; > @@ -219,6 +229,93 @@ StreamConfiguration CIO2Device::generateConfiguration(Size size) const > return cfg; > } > > +/** > + * \brief Retrieve the best sensor format for a desired output > + * \param[in] mbusCodes The list of acceptable media bus codes > + * \param[in] size The desired size > + * > + * Media bus codes are selected from \a mbusCodes, which lists all acceptable > + * codes in decreasing order of preference. Media bus codes supported by the > + * sensor but not listed in \a mbusCodes are ignored. If none of the desired > + * codes is supported, it returns an error. > + * > + * \a size indicates the desired size at the output of the sensor. This method > + * selects the best media bus code and size supported by the sensor according > + * to the following criteria. > + * > + * - The desired \a size shall fit in the sensor output size to avoid the need > + * to up-scale. > + * - The sensor output size shall match the desired aspect ratio to avoid the > + * need to crop the field of view. > + * - The sensor output size shall be as small as possible to lower the required > + * bandwidth. > + * - The desired \a size shall be supported by one of the media bus code listed > + * in \a mbusCodes. > + * > + * When multiple media bus codes can produce the same size, the code at the > + * lowest position in \a mbusCodes is selected. > + * > + * The use of this method is optional, as the above criteria may not match the > + * needs of all pipeline handlers. Pipeline handlers may implement custom > + * sensor format selection when needed. > + * > + * The returned sensor output format is guaranteed to be acceptable by the > + * setFormat() method without any modification. > + * > + * \return The best sensor output format matching the desired media bus codes > + * and size on success, or an empty format otherwise. > + */ > +V4L2SubdeviceFormat CIO2Device::getSensorFormat(const std::vector<unsigned int> &mbusCodes, > + const Size &size) const > +{ > + unsigned int desiredArea = size.width * size.height; > + unsigned int bestArea = UINT_MAX; > + float desiredRatio = static_cast<float>(size.width) / size.height; > + float bestRatio = FLT_MAX; > + const Size *bestSize = nullptr; > + uint32_t bestCode = 0; > + > + for (unsigned int code : mbusCodes) { > + const auto formats = formats_.find(code); > + if (formats == formats_.end()) > + continue; > + > + for (const SizeRange &range : formats->second) { > + const Size &sz = range.max; > + > + if (sz.width < size.width || sz.height < size.height) > + continue; > + > + float ratio = static_cast<float>(sz.width) / sz.height; > + float ratioDiff = fabsf(ratio - desiredRatio); > + unsigned int area = sz.width * sz.height; > + unsigned int areaDiff = area - desiredArea; > + > + if (ratioDiff > bestRatio) > + continue; > + > + if (ratioDiff < bestRatio || areaDiff < bestArea) { > + bestRatio = ratioDiff; > + bestArea = areaDiff; > + bestSize = &sz; > + bestCode = code; > + } > + } > + } > + > + if (!bestSize) { > + LOG(IPU3, Debug) << "No supported format or size found"; > + return {}; > + } > + > + V4L2SubdeviceFormat format{ > + .mbus_code = bestCode, > + .size = *bestSize, > + }; > + > + return format; > +} > + I think a verbatim copy of the function implemented in CameraSensor is a good idea to highlight changes on top that happen in the subsequent patches I'm always a bit hesitant about using 'get' in functions that perform computations, as a getter to me is just a field accessor. But that might just be me. Reviewed-by: Jacopo Mondi <jacopo@jmondi.org> Thanks j > int CIO2Device::exportBuffers(unsigned int count, > std::vector<std::unique_ptr<FrameBuffer>> *buffers) > { > diff --git a/src/libcamera/pipeline/ipu3/cio2.h b/src/libcamera/pipeline/ipu3/cio2.h > index f28e9f1d..f3f12706 100644 > --- a/src/libcamera/pipeline/ipu3/cio2.h > +++ b/src/libcamera/pipeline/ipu3/cio2.h > @@ -45,6 +45,9 @@ public: > int exportBuffers(unsigned int count, > std::vector<std::unique_ptr<FrameBuffer>> *buffers); > > + V4L2SubdeviceFormat getSensorFormat(const std::vector<unsigned int> &mbusCodes, > + const Size &size) const; > + > int start(); > int stop(); > > @@ -67,6 +70,8 @@ private: > std::unique_ptr<V4L2Subdevice> csi2_; > std::unique_ptr<V4L2VideoDevice> output_; > > + V4L2Subdevice::Formats formats_; > + > std::vector<std::unique_ptr<FrameBuffer>> buffers_; > std::queue<FrameBuffer *> availableBuffers_; > }; > -- > 2.31.0 >
diff --git a/src/libcamera/pipeline/ipu3/cio2.cpp b/src/libcamera/pipeline/ipu3/cio2.cpp index 1bcd580e..aef88afd 100644 --- a/src/libcamera/pipeline/ipu3/cio2.cpp +++ b/src/libcamera/pipeline/ipu3/cio2.cpp @@ -7,6 +7,8 @@ #include "cio2.h" +#include <float.h> + #include <linux/media-bus-format.h> #include <libcamera/formats.h> @@ -120,6 +122,14 @@ int CIO2Device::init(const MediaDevice *media, unsigned int index) if (ret) return ret; + formats_ = sensor_->formats(); + if (formats_.empty()) { + LOG(IPU3, Error) + << "Sensor " << sensor_->entity()->name() + << "has empty sensor format list"; + return -EINVAL; + } + /* * Make sure the sensor produces at least one format compatible with * the CIO2 requirements. @@ -168,7 +178,7 @@ int CIO2Device::configure(const Size &size, V4L2DeviceFormat *outputFormat) * the CIO2 output device. */ std::vector<unsigned int> mbusCodes = utils::map_keys(mbusCodesToPixelFormat); - sensorFormat = sensor_->getFormat(mbusCodes, size); + sensorFormat = getSensorFormat(mbusCodes, size); ret = sensor_->setFormat(&sensorFormat); if (ret) return ret; @@ -206,7 +216,7 @@ StreamConfiguration CIO2Device::generateConfiguration(Size size) const /* Query the sensor static information for closest match. */ std::vector<unsigned int> mbusCodes = utils::map_keys(mbusCodesToPixelFormat); - V4L2SubdeviceFormat sensorFormat = sensor_->getFormat(mbusCodes, size); + V4L2SubdeviceFormat sensorFormat = getSensorFormat(mbusCodes, size); if (!sensorFormat.mbus_code) { LOG(IPU3, Error) << "Sensor does not support mbus code"; return {}; @@ -219,6 +229,93 @@ StreamConfiguration CIO2Device::generateConfiguration(Size size) const return cfg; } +/** + * \brief Retrieve the best sensor format for a desired output + * \param[in] mbusCodes The list of acceptable media bus codes + * \param[in] size The desired size + * + * Media bus codes are selected from \a mbusCodes, which lists all acceptable + * codes in decreasing order of preference. Media bus codes supported by the + * sensor but not listed in \a mbusCodes are ignored. If none of the desired + * codes is supported, it returns an error. + * + * \a size indicates the desired size at the output of the sensor. This method + * selects the best media bus code and size supported by the sensor according + * to the following criteria. + * + * - The desired \a size shall fit in the sensor output size to avoid the need + * to up-scale. + * - The sensor output size shall match the desired aspect ratio to avoid the + * need to crop the field of view. + * - The sensor output size shall be as small as possible to lower the required + * bandwidth. + * - The desired \a size shall be supported by one of the media bus code listed + * in \a mbusCodes. + * + * When multiple media bus codes can produce the same size, the code at the + * lowest position in \a mbusCodes is selected. + * + * The use of this method is optional, as the above criteria may not match the + * needs of all pipeline handlers. Pipeline handlers may implement custom + * sensor format selection when needed. + * + * The returned sensor output format is guaranteed to be acceptable by the + * setFormat() method without any modification. + * + * \return The best sensor output format matching the desired media bus codes + * and size on success, or an empty format otherwise. + */ +V4L2SubdeviceFormat CIO2Device::getSensorFormat(const std::vector<unsigned int> &mbusCodes, + const Size &size) const +{ + unsigned int desiredArea = size.width * size.height; + unsigned int bestArea = UINT_MAX; + float desiredRatio = static_cast<float>(size.width) / size.height; + float bestRatio = FLT_MAX; + const Size *bestSize = nullptr; + uint32_t bestCode = 0; + + for (unsigned int code : mbusCodes) { + const auto formats = formats_.find(code); + if (formats == formats_.end()) + continue; + + for (const SizeRange &range : formats->second) { + const Size &sz = range.max; + + if (sz.width < size.width || sz.height < size.height) + continue; + + float ratio = static_cast<float>(sz.width) / sz.height; + float ratioDiff = fabsf(ratio - desiredRatio); + unsigned int area = sz.width * sz.height; + unsigned int areaDiff = area - desiredArea; + + if (ratioDiff > bestRatio) + continue; + + if (ratioDiff < bestRatio || areaDiff < bestArea) { + bestRatio = ratioDiff; + bestArea = areaDiff; + bestSize = &sz; + bestCode = code; + } + } + } + + if (!bestSize) { + LOG(IPU3, Debug) << "No supported format or size found"; + return {}; + } + + V4L2SubdeviceFormat format{ + .mbus_code = bestCode, + .size = *bestSize, + }; + + return format; +} + int CIO2Device::exportBuffers(unsigned int count, std::vector<std::unique_ptr<FrameBuffer>> *buffers) { diff --git a/src/libcamera/pipeline/ipu3/cio2.h b/src/libcamera/pipeline/ipu3/cio2.h index f28e9f1d..f3f12706 100644 --- a/src/libcamera/pipeline/ipu3/cio2.h +++ b/src/libcamera/pipeline/ipu3/cio2.h @@ -45,6 +45,9 @@ public: int exportBuffers(unsigned int count, std::vector<std::unique_ptr<FrameBuffer>> *buffers); + V4L2SubdeviceFormat getSensorFormat(const std::vector<unsigned int> &mbusCodes, + const Size &size) const; + int start(); int stop(); @@ -67,6 +70,8 @@ private: std::unique_ptr<V4L2Subdevice> csi2_; std::unique_ptr<V4L2VideoDevice> output_; + V4L2Subdevice::Formats formats_; + std::vector<std::unique_ptr<FrameBuffer>> buffers_; std::queue<FrameBuffer *> availableBuffers_; };
This prepares a base to introduce custom selection of sensor format based on platform(soraka or nautilus) constraints. The changes in selection policy will be introduced in a subsequent patch. No functional changes in this patch. Signed-off-by: Umang Jain <umang.jain@ideasonboard.com> --- src/libcamera/pipeline/ipu3/cio2.cpp | 101 ++++++++++++++++++++++++++- src/libcamera/pipeline/ipu3/cio2.h | 5 ++ 2 files changed, 104 insertions(+), 2 deletions(-)