From patchwork Fri Jul 30 08:28:28 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Umang Jain X-Patchwork-Id: 13157 X-Patchwork-Delegate: umang.jain@ideasonboard.com Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id AF0B7C322E for ; Fri, 30 Jul 2021 08:28:55 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 61966687C6; Fri, 30 Jul 2021 10:28:55 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="jd9qs1QS"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 8BB74687BD for ; Fri, 30 Jul 2021 10:28:53 +0200 (CEST) Received: from perceval.ideasonboard.com (unknown [103.251.226.16]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 79C5789B; Fri, 30 Jul 2021 10:28:52 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1627633733; bh=6Pk6VqmTnnDdgsN5+EP2HhmtIsGtTJY1s05p3DgZQAY=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=jd9qs1QS5RTry7xMEN+yZOZBNKCIb7CN7rgtoIVeZTO0te1+d3bWXod39fmHjPJ36 9q3Pb8jzr+NrQc7KcYeXuIobiORVIn/IdLaVyGhEuDpyU4yON2kcD1+7vpJo6irRIX cXvDBHJkj9SMi43ygu/F2yiL9kzB9q3mUmo2PYAc= From: Umang Jain To: libcamera-devel@lists.libcamera.org Date: Fri, 30 Jul 2021 13:58:28 +0530 Message-Id: <20210730082832.152626-4-umang.jain@ideasonboard.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210730082832.152626-1-umang.jain@ideasonboard.com> References: <20210730082832.152626-1-umang.jain@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [RFC PATCH 3/3] ipu3: cio2: Customize sensor format selection logic X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Adapt the sensor format selection logic such that it address platform constraints on Soraka and Nautilus. This is best-effort basis hence, new platforms can bring new constraints in future and we will need to adapt accordingly. Currently we prioritise sensor format resolutions which closest to the FoV of desired output. However, Intel seems to prioritize the selection based on the closest FoV with respect to sensor's maximum resolution. For e.g. for a desired output of 1080p, Soraka will select 2112x1568 since it's a better FoV match to sensor's maximum resolution (4224x3136). It will not match the provided resolution of 2112x1188, even if that matches exactly to the desired ratio of 1080p. On the other hand, on Nautilus, currently the sensor's maximum resolution(4208x3118) is the only selection for any 16:9 desired formats, whether it is 640x360 or 1080p or 3840x2160. That means the sensor will run at full bandwidth even for ridiculously smaller resolutions so, we probably don't want that either. This patch how the sensor format resolution is selected: - It prioritizes FoV with respect to sensor's maximum resolution size - It shall never return sensor's maximum resolution provided if there are other lower resolutions available. /* DNI: Preliminary testing of getSensorFormat(): * cam -c1 -swidth=640,height=360,role=raw * cam -c1 -swidth=1280,height=720,role=raw * cam -c1 -swidth=1920,height=1080,role=raw * cam -c1 -swidth=3840,height=2160,role=raw */ Signed-off-by: Umang Jain --- Output of preliminary testing: ($) cam -c1 -swidth=640,height=360,role=raw INFO IPU3 cio2.cpp:326 Desired Size: 640x360, Found SGRBG10_IPU3 with Size: 2104x1560 ($) cam -c1 -swidth=1280,height=720,role=raw INFO IPU3 cio2.cpp:326 Desired Size: 1280x720, Found SGRBG10_IPU3 with Size: 2104x1560 ($) cam -c1 -swidth=1920,height=1080,role=raw INFO IPU3 cio2.cpp:326 Desired Size: 1920x1080, Found SGRBG10_IPU3 with Size: 2104x1560 ($) cam -c1 -swidth=3840,height=2160,role=raw INFO IPU3 cio2.cpp:326 Desired Size: 3840x2160, Found SGRBG10_IPU3 with Size: 4208x3118 --- src/libcamera/pipeline/ipu3/cio2.cpp | 62 +++++++++++++++++----------- 1 file changed, 38 insertions(+), 24 deletions(-) diff --git a/src/libcamera/pipeline/ipu3/cio2.cpp b/src/libcamera/pipeline/ipu3/cio2.cpp index aef88afd..4fd57ef7 100644 --- a/src/libcamera/pipeline/ipu3/cio2.cpp +++ b/src/libcamera/pipeline/ipu3/cio2.cpp @@ -245,20 +245,16 @@ StreamConfiguration CIO2Device::generateConfiguration(Size size) const * * - The desired \a size shall fit in the sensor output size to avoid the need * to up-scale. - * - The sensor output size shall match the desired aspect ratio to avoid the - * need to crop the field of view. - * - The sensor output size shall be as small as possible to lower the required - * bandwidth. + * - The sensor output size will be set to maximum resolution only when there + * are no other available lower sensor resolutions from any of \a mbusCodes. + * - The sensor output size shall have the closest FoV with respect + * to the sensor's maximum resolution. * - The desired \a size shall be supported by one of the media bus code listed * in \a mbusCodes. * * When multiple media bus codes can produce the same size, the code at the * lowest position in \a mbusCodes is selected. * - * The use of this method is optional, as the above criteria may not match the - * needs of all pipeline handlers. Pipeline handlers may implement custom - * sensor format selection when needed. - * * The returned sensor output format is guaranteed to be acceptable by the * setFormat() method without any modification. * @@ -268,14 +264,15 @@ StreamConfiguration CIO2Device::generateConfiguration(Size size) const V4L2SubdeviceFormat CIO2Device::getSensorFormat(const std::vector &mbusCodes, const Size &size) const { - unsigned int desiredArea = size.width * size.height; - unsigned int bestArea = UINT_MAX; - float desiredRatio = static_cast(size.width) / size.height; - float bestRatio = FLT_MAX; - const Size *bestSize = nullptr; + std::map selectedSizes; + Size bestSize; + float maxResRatio = FLT_MAX; + float bestRatioDiff = -1.0; uint32_t bestCode = 0; for (unsigned int code : mbusCodes) { + Size maxSensorResolution; + const auto formats = formats_.find(code); if (formats == formats_.end()) continue; @@ -286,33 +283,50 @@ V4L2SubdeviceFormat CIO2Device::getSensorFormat(const std::vector if (sz.width < size.width || sz.height < size.height) continue; + if (sz > maxSensorResolution) + maxSensorResolution = sz; + float ratio = static_cast(sz.width) / sz.height; - float ratioDiff = fabsf(ratio - desiredRatio); - unsigned int area = sz.width * sz.height; - unsigned int areaDiff = area - desiredArea; + selectedSizes.emplace(sz, ratio); + } - if (ratioDiff > bestRatio) - continue; + if (!selectedSizes.empty()) { + maxResRatio = selectedSizes[maxSensorResolution]; - if (ratioDiff < bestRatio || areaDiff < bestArea) { - bestRatio = ratioDiff; - bestArea = areaDiff; - bestSize = &sz; + selectedSizes.erase(maxSensorResolution); + if (selectedSizes.empty()) { bestCode = code; + bestSize = maxSensorResolution; + } else { /* Find the best FoV to sensor's max resolution */ + for (auto iter : selectedSizes) { + float ratioDiff = fabsf(maxResRatio - iter.second); + + if (bestRatioDiff < 0 || (ratioDiff < bestRatioDiff)) { + bestRatioDiff = ratioDiff; + bestCode = code; + bestSize = iter.first; + } + } } } + selectedSizes.clear(); } - if (!bestSize) { + if (bestSize.isNull()) { LOG(IPU3, Debug) << "No supported format or size found"; return {}; } V4L2SubdeviceFormat format{ .mbus_code = bestCode, - .size = *bestSize, + .size = bestSize, }; + LOG(IPU3, Info) + << "Desired Size: " << size.toString() + << ", Found " << mbusCodesToPixelFormat.at(format.mbus_code).toString() + << " with Size: " << format.size.toString(); + return format; }