Message ID | 20210803133205.6599-3-umang.jain@ideasonboard.com |
---|---|
State | Superseded |
Delegated to: | Umang Jain |
Headers | show |
Series |
|
Related | show |
Hi Umang, On Tue, Aug 03, 2021 at 07:02:03PM +0530, Umang Jain wrote: > This prepares a base to introduce custom selection of sensor format > based on platform(soraka or nautilus) constraints. The changes in > selection policy will be introduced in a subsequent patch. > > The replication of the function is not verbatim and has small changes. I would say that this copied CameraSensor::getFormat() in the PH code to be later changed on top. The "the function" here above is clear to us that know the context. > CameraSensor::getFormats() has accesss to all the supported sensor > formats via a V4L2Subdevice::Formats internally and use it directly. > We do not want to expose V4L2Subdevice::Formats directly to the > pipeline-handler hence, the patch is adapted to use the > CameraSensor::sizes(mbusCode) instead, to get the range of sensor sizes > supported for desired media bus codes. > > No functional changes in this patch. > > Signed-off-by: Umang Jain <umang.jain@ideasonboard.com> > --- > src/libcamera/pipeline/ipu3/cio2.cpp | 91 +++++++++++++++++++++++++++- > src/libcamera/pipeline/ipu3/cio2.h | 3 + > 2 files changed, 92 insertions(+), 2 deletions(-) > > diff --git a/src/libcamera/pipeline/ipu3/cio2.cpp b/src/libcamera/pipeline/ipu3/cio2.cpp > index 6f2ef321..be063613 100644 > --- a/src/libcamera/pipeline/ipu3/cio2.cpp > +++ b/src/libcamera/pipeline/ipu3/cio2.cpp > @@ -7,6 +7,8 @@ > > #include "cio2.h" > > +#include <float.h> > + > #include <linux/media-bus-format.h> > > #include <libcamera/formats.h> > @@ -170,7 +172,7 @@ int CIO2Device::configure(const Size &size, V4L2DeviceFormat *outputFormat) > * the CIO2 output device. > */ > std::vector<unsigned int> mbusCodes = utils::map_keys(mbusCodesToPixelFormat); > - sensorFormat = sensor_->getFormat(mbusCodes, size); > + sensorFormat = getSensorFormat(mbusCodes, size); > ret = sensor_->setFormat(&sensorFormat); > if (ret) > return ret; > @@ -208,7 +210,7 @@ StreamConfiguration CIO2Device::generateConfiguration(Size size) const > > /* Query the sensor static information for closest match. */ > std::vector<unsigned int> mbusCodes = utils::map_keys(mbusCodesToPixelFormat); > - V4L2SubdeviceFormat sensorFormat = sensor_->getFormat(mbusCodes, size); > + V4L2SubdeviceFormat sensorFormat = getSensorFormat(mbusCodes, size); > if (!sensorFormat.mbus_code) { > LOG(IPU3, Error) << "Sensor does not support mbus code"; > return {}; > @@ -221,6 +223,91 @@ StreamConfiguration CIO2Device::generateConfiguration(Size size) const > return cfg; > } > > +/** > + * \brief Retrieve the best sensor format for a desired output > + * \param[in] mbusCodes The list of acceptable media bus codes > + * \param[in] size The desired size > + * > + * Media bus codes are selected from \a mbusCodes, which lists all acceptable > + * codes in decreasing order of preference. Media bus codes supported by the > + * sensor but not listed in \a mbusCodes are ignored. If none of the desired > + * codes is supported, it returns an error. > + * > + * \a size indicates the desired size at the output of the sensor. This method > + * selects the best media bus code and size supported by the sensor according > + * to the following criteria. > + * > + * - The desired \a size shall fit in the sensor output size to avoid the need > + * to up-scale. > + * - The sensor output size shall match the desired aspect ratio to avoid the > + * need to crop the field of view. > + * - The sensor output size shall be as small as possible to lower the required > + * bandwidth. > + * - The desired \a size shall be supported by one of the media bus code listed > + * in \a mbusCodes. > + * > + * When multiple media bus codes can produce the same size, the code at the > + * lowest position in \a mbusCodes is selected. > + * > + * The use of this method is optional, as the above criteria may not match the > + * needs of all pipeline handlers. Pipeline handlers may implement custom > + * sensor format selection when needed. > + * > + * The returned sensor output format is guaranteed to be acceptable by the > + * setFormat() method without any modification. > + * > + * \return The best sensor output format matching the desired media bus codes > + * and size on success, or an empty format otherwise. > + */ > +V4L2SubdeviceFormat CIO2Device::getSensorFormat(const std::vector<unsigned int> &mbusCodes, > + const Size &size) const > +{ > + unsigned int desiredArea = size.width * size.height; > + unsigned int bestArea = UINT_MAX; we should drop the C versions and use std::numeric_limits<> instead Maybe on top ? > + float desiredRatio = static_cast<float>(size.width) / size.height; > + float bestRatio = FLT_MAX; > + Size bestSize; > + uint32_t bestCode = 0; > + > + for (unsigned int code : mbusCodes) { > + const auto sizes = sensor_->sizes(code); > + if (!sizes.size()) > + continue; > + > + for (const Size &sz : sizes) { > + if (sz.width < size.width || sz.height < size.height) > + continue; > + > + float ratio = static_cast<float>(sz.width) / sz.height; > + float ratioDiff = fabsf(ratio - desiredRatio); > + unsigned int area = sz.width * sz.height; > + unsigned int areaDiff = area - desiredArea; > + > + if (ratioDiff > bestRatio) > + continue; > + > + if (ratioDiff < bestRatio || areaDiff < bestArea) { > + bestRatio = ratioDiff; > + bestArea = areaDiff; > + bestSize = sz; > + bestCode = code; > + } > + } > + } > + > + if (bestSize.isNull()) { > + LOG(IPU3, Debug) << "No supported format or size found"; > + return {}; > + } > + > + V4L2SubdeviceFormat format{ > + .mbus_code = bestCode, > + .size = bestSize, > + }; > + > + return format; > +} > + > int CIO2Device::exportBuffers(unsigned int count, > std::vector<std::unique_ptr<FrameBuffer>> *buffers) > { > diff --git a/src/libcamera/pipeline/ipu3/cio2.h b/src/libcamera/pipeline/ipu3/cio2.h > index f28e9f1d..da024d45 100644 > --- a/src/libcamera/pipeline/ipu3/cio2.h > +++ b/src/libcamera/pipeline/ipu3/cio2.h > @@ -45,6 +45,9 @@ public: > int exportBuffers(unsigned int count, > std::vector<std::unique_ptr<FrameBuffer>> *buffers); > > + V4L2SubdeviceFormat getSensorFormat(const std::vector<unsigned int> &mbusCodes, > + const Size &size) const; The patch is good! Reviewed-by: Jacopo Mondi <jacopo@jmondi.org> Thanks j > + > int start(); > int stop(); > > -- > 2.31.0 >
diff --git a/src/libcamera/pipeline/ipu3/cio2.cpp b/src/libcamera/pipeline/ipu3/cio2.cpp index 6f2ef321..be063613 100644 --- a/src/libcamera/pipeline/ipu3/cio2.cpp +++ b/src/libcamera/pipeline/ipu3/cio2.cpp @@ -7,6 +7,8 @@ #include "cio2.h" +#include <float.h> + #include <linux/media-bus-format.h> #include <libcamera/formats.h> @@ -170,7 +172,7 @@ int CIO2Device::configure(const Size &size, V4L2DeviceFormat *outputFormat) * the CIO2 output device. */ std::vector<unsigned int> mbusCodes = utils::map_keys(mbusCodesToPixelFormat); - sensorFormat = sensor_->getFormat(mbusCodes, size); + sensorFormat = getSensorFormat(mbusCodes, size); ret = sensor_->setFormat(&sensorFormat); if (ret) return ret; @@ -208,7 +210,7 @@ StreamConfiguration CIO2Device::generateConfiguration(Size size) const /* Query the sensor static information for closest match. */ std::vector<unsigned int> mbusCodes = utils::map_keys(mbusCodesToPixelFormat); - V4L2SubdeviceFormat sensorFormat = sensor_->getFormat(mbusCodes, size); + V4L2SubdeviceFormat sensorFormat = getSensorFormat(mbusCodes, size); if (!sensorFormat.mbus_code) { LOG(IPU3, Error) << "Sensor does not support mbus code"; return {}; @@ -221,6 +223,91 @@ StreamConfiguration CIO2Device::generateConfiguration(Size size) const return cfg; } +/** + * \brief Retrieve the best sensor format for a desired output + * \param[in] mbusCodes The list of acceptable media bus codes + * \param[in] size The desired size + * + * Media bus codes are selected from \a mbusCodes, which lists all acceptable + * codes in decreasing order of preference. Media bus codes supported by the + * sensor but not listed in \a mbusCodes are ignored. If none of the desired + * codes is supported, it returns an error. + * + * \a size indicates the desired size at the output of the sensor. This method + * selects the best media bus code and size supported by the sensor according + * to the following criteria. + * + * - The desired \a size shall fit in the sensor output size to avoid the need + * to up-scale. + * - The sensor output size shall match the desired aspect ratio to avoid the + * need to crop the field of view. + * - The sensor output size shall be as small as possible to lower the required + * bandwidth. + * - The desired \a size shall be supported by one of the media bus code listed + * in \a mbusCodes. + * + * When multiple media bus codes can produce the same size, the code at the + * lowest position in \a mbusCodes is selected. + * + * The use of this method is optional, as the above criteria may not match the + * needs of all pipeline handlers. Pipeline handlers may implement custom + * sensor format selection when needed. + * + * The returned sensor output format is guaranteed to be acceptable by the + * setFormat() method without any modification. + * + * \return The best sensor output format matching the desired media bus codes + * and size on success, or an empty format otherwise. + */ +V4L2SubdeviceFormat CIO2Device::getSensorFormat(const std::vector<unsigned int> &mbusCodes, + const Size &size) const +{ + unsigned int desiredArea = size.width * size.height; + unsigned int bestArea = UINT_MAX; + float desiredRatio = static_cast<float>(size.width) / size.height; + float bestRatio = FLT_MAX; + Size bestSize; + uint32_t bestCode = 0; + + for (unsigned int code : mbusCodes) { + const auto sizes = sensor_->sizes(code); + if (!sizes.size()) + continue; + + for (const Size &sz : sizes) { + if (sz.width < size.width || sz.height < size.height) + continue; + + float ratio = static_cast<float>(sz.width) / sz.height; + float ratioDiff = fabsf(ratio - desiredRatio); + unsigned int area = sz.width * sz.height; + unsigned int areaDiff = area - desiredArea; + + if (ratioDiff > bestRatio) + continue; + + if (ratioDiff < bestRatio || areaDiff < bestArea) { + bestRatio = ratioDiff; + bestArea = areaDiff; + bestSize = sz; + bestCode = code; + } + } + } + + if (bestSize.isNull()) { + LOG(IPU3, Debug) << "No supported format or size found"; + return {}; + } + + V4L2SubdeviceFormat format{ + .mbus_code = bestCode, + .size = bestSize, + }; + + return format; +} + int CIO2Device::exportBuffers(unsigned int count, std::vector<std::unique_ptr<FrameBuffer>> *buffers) { diff --git a/src/libcamera/pipeline/ipu3/cio2.h b/src/libcamera/pipeline/ipu3/cio2.h index f28e9f1d..da024d45 100644 --- a/src/libcamera/pipeline/ipu3/cio2.h +++ b/src/libcamera/pipeline/ipu3/cio2.h @@ -45,6 +45,9 @@ public: int exportBuffers(unsigned int count, std::vector<std::unique_ptr<FrameBuffer>> *buffers); + V4L2SubdeviceFormat getSensorFormat(const std::vector<unsigned int> &mbusCodes, + const Size &size) const; + int start(); int stop();
This prepares a base to introduce custom selection of sensor format based on platform(soraka or nautilus) constraints. The changes in selection policy will be introduced in a subsequent patch. The replication of the function is not verbatim and has small changes. CameraSensor::getFormats() has accesss to all the supported sensor formats via a V4L2Subdevice::Formats internally and use it directly. We do not want to expose V4L2Subdevice::Formats directly to the pipeline-handler hence, the patch is adapted to use the CameraSensor::sizes(mbusCode) instead, to get the range of sensor sizes supported for desired media bus codes. No functional changes in this patch. Signed-off-by: Umang Jain <umang.jain@ideasonboard.com> --- src/libcamera/pipeline/ipu3/cio2.cpp | 91 +++++++++++++++++++++++++++- src/libcamera/pipeline/ipu3/cio2.h | 3 + 2 files changed, 92 insertions(+), 2 deletions(-)