[libcamera-devel,v2,14/17] android: capabilties: Fix ANDROID_CONTROL_AE_AVAILABLE_TARGET_FPS_RANGES

13745 diff mbox series

As reported by the CONTROL_AE_AVAILABLE_TARGET_FPS_RANGES documentation
in the Android developer reference:

"For devices advertising any color filter arrangement other than NIR, or
devices not advertising color filter arrangement, this list will always
include (min, max) and (max, max) where min <= 15 and max = the maximum
output frame rate of the maximum YUV_420_888 output size."

Collect the higher FPS of the larger YUV stream and use it with the
minimum FPS rate the camera can produce to populate the
ANDROID_CONTROL_AE_AVAILABLE_TARGET_FPS_RANGES static metadata.

Signed-off-by: Jacopo Mondi <jacopo@jmondi.org>
Reviewed-by: Paul Elder <paul.elder@ideasonboard.com>
---
 src/android/camera_capabilities.cpp | 75 +++++++++++------------------
 1 file changed, 28 insertions(+), 47 deletions(-)

Hi Jacopo,

Thank you for the patch.

On Tue, Sep 07, 2021 at 09:41:04PM +0200, Jacopo Mondi wrote:
> As reported by the CONTROL_AE_AVAILABLE_TARGET_FPS_RANGES documentation
> in the Android developer reference:
> 
> "For devices advertising any color filter arrangement other than NIR, or
> devices not advertising color filter arrangement, this list will always
> include (min, max) and (max, max) where min <= 15 and max = the maximum
> output frame rate of the maximum YUV_420_888 output size."
> 
> Collect the higher FPS of the larger YUV stream and use it with the
> minimum FPS rate the camera can produce to populate the
> ANDROID_CONTROL_AE_AVAILABLE_TARGET_FPS_RANGES static metadata.
> 
> Signed-off-by: Jacopo Mondi <jacopo@jmondi.org>
> Reviewed-by: Paul Elder <paul.elder@ideasonboard.com>
> ---
>  src/android/camera_capabilities.cpp | 75 +++++++++++------------------
>  1 file changed, 28 insertions(+), 47 deletions(-)
> 
> diff --git a/src/android/camera_capabilities.cpp b/src/android/camera_capabilities.cpp
> index 194ede866806..deaec3282fd7 100644
> --- a/src/android/camera_capabilities.cpp
> +++ b/src/android/camera_capabilities.cpp
> @@ -866,53 +866,6 @@ int CameraCapabilities::initializeStaticMetadata()
>  	staticMetadata_->addEntry(ANDROID_CONTROL_AE_AVAILABLE_MODES,
>  				  aeAvailableModes);
>  
> -	/* Initialize the AE frame duration limits. */
> -	const auto frameDurationsInfo = controlsInfo.find(&controls::FrameDurationLimits);
> -	int64_t minFrameDurationNsec = frameDurationsInfo->second.min().get<int64_t>() * 1000;
> -	int64_t maxFrameDurationNsec = frameDurationsInfo->second.max().get<int64_t>() * 1000;
> -
> -	/*
> -	 * Adjust the minimum frame duration to comply with Android
> -	 * requirements. The camera service mandates all preview/record
> -	 * streams to have a minimum frame duration < 33,366 milliseconds
> -	 * (see MAX_PREVIEW_RECORD_DURATION_NS in the camera service
> -	 * implementation).
> -	 *
> -	 * If we're close enough (+ 500 useconds) to that value, round
> -	 * the minimum frame duration of the camera to an accepted
> -	 * value.
> -	 */
> -	static constexpr int64_t MAX_PREVIEW_RECORD_DURATION_NS = 1e9 / 29.97;
> -	if (minFrameDurationNsec > MAX_PREVIEW_RECORD_DURATION_NS &&
> -	    minFrameDurationNsec < MAX_PREVIEW_RECORD_DURATION_NS + 500000)
> -		minFrameDurationNsec = MAX_PREVIEW_RECORD_DURATION_NS - 1000;

I really like how we finally drop this.

Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>

> -
> -	/*
> -	 * The AE routine frame rate limits are computed using the frame
> -	 * duration limits, as libcamera clips the AE routine to the
> -	 * frame durations.
> -	 */
> -	int32_t maxFps = std::round(1e9 / minFrameDurationNsec);
> -	int32_t minFps = std::round(1e9 / maxFrameDurationNsec);
> -	minFps = std::max(1, minFps);
> -
> -	/*
> -	 * Force rounding errors so that we have the proper frame
> -	 * durations for when we reuse these variables later
> -	 */
> -	minFrameDurationNsec = 1e9 / maxFps;
> -	maxFrameDurationNsec = 1e9 / minFps;
> -
> -	/*
> -	 * Register to the camera service {min, max} and {max, max}
> -	 * intervals as requested by the metadata documentation.
> -	 */
> -	int32_t availableAeFpsTarget[] = {
> -		minFps, maxFps, maxFps, maxFps
> -	};
> -	staticMetadata_->addEntry(ANDROID_CONTROL_AE_AVAILABLE_TARGET_FPS_RANGES,
> -				  availableAeFpsTarget);
> -
>  	std::vector<int32_t> aeCompensationRange = {
>  		0, 0,
>  	};
> @@ -1258,8 +1211,12 @@ int CameraCapabilities::initializeStaticMetadata()
>  
>  	std::vector<uint32_t> availableStreamConfigurations;
>  	std::vector<int64_t> minFrameDurations;
> +	int maxYUVFps = 0;
> +	Size maxYUVSize;
> +
>  	availableStreamConfigurations.reserve(streamConfigurations_.size() * 4);
>  	minFrameDurations.reserve(streamConfigurations_.size() * 4);
> +
>  	for (const auto &entry : streamConfigurations_) {
>  		/*
>  		 * Filter out YUV streams not capable of running at 30 FPS.
> @@ -1276,6 +1233,16 @@ int CameraCapabilities::initializeStaticMetadata()
>  		if (entry.androidFormat != HAL_PIXEL_FORMAT_BLOB && fps < 30)
>  			continue;
>  
> +		/*
> +		 * Collect the FPS of the maximum YUV output size to populate
> +		 * AE_AVAILABLE_TARGET_FPS_RANGE
> +		 */
> +		if (entry.androidFormat == HAL_PIXEL_FORMAT_YCbCr_420_888 &&
> +		    entry.resolution > maxYUVSize) {
> +			maxYUVSize = entry.resolution;
> +			maxYUVFps = fps;
> +		}
> +
>  		/* Stream configuration map. */
>  		availableStreamConfigurations.push_back(entry.androidFormat);
>  		availableStreamConfigurations.push_back(entry.resolution.width);
> @@ -1303,6 +1270,20 @@ int CameraCapabilities::initializeStaticMetadata()
>  	staticMetadata_->addEntry(ANDROID_SCALER_AVAILABLE_MIN_FRAME_DURATIONS,
>  				  minFrameDurations);
>  
> +	/*
> +	 * Register to the camera service {min, max} and {max, max} with
> +	 * 'max' being the larger YUV stream maximum frame rate and 'min' being
> +	 * the globally minimum frame rate rounded to the next largest integer
> +	 * as the camera service expects the camera maximum frame duration to be
> +	 * smaller than 10^9 / minFps.
> +	 */
> +	int32_t minFps = std::ceil(1e9 / maxFrameDuration_);
> +	int32_t availableAeFpsTarget[] = {
> +		minFps, maxYUVFps, maxYUVFps, maxYUVFps,
> +	};
> +	staticMetadata_->addEntry(ANDROID_CONTROL_AE_AVAILABLE_TARGET_FPS_RANGES,
> +				  availableAeFpsTarget);
> +
>  	std::vector<int64_t> availableStallDurations;
>  	for (const auto &entry : streamConfigurations_) {
>  		if (entry.androidFormat != HAL_PIXEL_FORMAT_BLOB)