[{"id":19145,"web_url":"https://patchwork.libcamera.org/comment/19145/","msgid":"<8b451b0c-c43a-f46c-c9cf-05673e2d3710@ideasonboard.com>","date":"2021-08-27T12:41:43","subject":"Re: [libcamera-devel] [PATCH v2 2/4] ipu3: cio2: Replicate\n\tCameraSensor::getFormats() to a member function","submitter":{"id":4,"url":"https://patchwork.libcamera.org/api/people/4/","name":"Kieran Bingham","email":"kieran.bingham@ideasonboard.com"},"content":"On 10/08/2021 08:58, Umang Jain wrote:\n> This prepares a base to introduce custom selection of sensor format\n> based on platform(soraka or nautilus) constraints. The changes in\n\nas nouns/names, you can capitalize Soraka and Nautilus.\n\n\n> selection policy will be introduced in a subsequent patch.\n> \n> The function is copied from CameraSensor::getFormat() in the IPU3\n> pipeline handler code to be later changed on top.\n> \n> The copy is not verbatim and has a minor change as follows:\n> CameraSensor::getFormats() has access to a V4L2Subdevice::Formats\n> interally and use it directly to iterate over supported camera sensor\n\ns/interally/internally/\n\n\n> frame sizes. The copy is adapted to use the CameraSensor::sizes(mbusCode)\n> instead, to enumerate over the supported frame sizes as per\n> mbusCodesToPixelFormat map.\n> \n> No functional changes in this patch.\n> \n> Signed-off-by: Umang Jain \n> Tested-by: Umang Jain \n> Reviewed-by: Jacopo Mondi \n> Tested-by: Jacopo Mondi \n> ---\n> src/libcamera/pipeline/ipu3/cio2.cpp | 92 +++++++++++++++++++++++++++-\n> src/libcamera/pipeline/ipu3/cio2.h | 3 +\n> 2 files changed, 93 insertions(+), 2 deletions(-)\n> \n> diff --git a/src/libcamera/pipeline/ipu3/cio2.cpp b/src/libcamera/pipeline/ipu3/cio2.cpp\n> index 8a40e955..88dd364b 100644\n> --- a/src/libcamera/pipeline/ipu3/cio2.cpp\n> +++ b/src/libcamera/pipeline/ipu3/cio2.cpp\n> @@ -7,6 +7,9 @@\n> \n> #include \"cio2.h\"\n> \n> +#include \n> +#include \n> +\n> #include \n> \n> #include \n> @@ -178,7 +181,7 @@ int CIO2Device::configure(const Size &size, V4L2DeviceFormat *outputFormat)\n> \t * the CIO2 output device.\n> \t */\n> \tstd::vector mbusCodes = utils::map_keys(mbusCodesToPixelFormat);\n> -\tsensorFormat = sensor_->getFormat(mbusCodes, size);\n> +\tsensorFormat = getSensorFormat(mbusCodes, size);\n> \tret = sensor_->setFormat(&sensorFormat);\n> \tif (ret)\n> \t\treturn ret;\n> @@ -216,7 +219,7 @@ StreamConfiguration CIO2Device::generateConfiguration(Size size) const\n> \n> \t/* Query the sensor static information for closest match. */\n> \tstd::vector mbusCodes = utils::map_keys(mbusCodesToPixelFormat);\n> -\tV4L2SubdeviceFormat sensorFormat = sensor_->getFormat(mbusCodes, size);\n> +\tV4L2SubdeviceFormat sensorFormat = getSensorFormat(mbusCodes, size);\n> \tif (!sensorFormat.mbus_code) {\n> \t\tLOG(IPU3, Error) << \"Sensor does not support mbus code\";\n> \t\treturn {};\n> @@ -229,6 +232,91 @@ StreamConfiguration CIO2Device::generateConfiguration(Size size) const\n> \treturn cfg;\n> }\n> \n> +/**\n> + * \\brief Retrieve the best sensor format for a desired output\n> + * \\param[in] mbusCodes The list of acceptable media bus codes\n> + * \\param[in] size The desired size\n> + *\n> + * Media bus codes are selected from \\a mbusCodes, which lists all acceptable\n> + * codes in decreasing order of preference. Media bus codes supported by the\n> + * sensor but not listed in \\a mbusCodes are ignored. If none of the desired\n> + * codes is supported, it returns an error.\n> + *\n> + * \\a size indicates the desired size at the output of the sensor. This method\n> + * selects the best media bus code and size supported by the sensor according\n> + * to the following criteria.\n> + *\n> + * - The desired \\a size shall fit in the sensor output size to avoid the need\n> + * to up-scale.\n> + * - The sensor output size shall match the desired aspect ratio to avoid the\n> + * need to crop the field of view.\n> + * - The sensor output size shall be as small as possible to lower the required\n> + * bandwidth.\n> + * - The desired \\a size shall be supported by one of the media bus code listed\n> + * in \\a mbusCodes.\n> + *\n> + * When multiple media bus codes can produce the same size, the code at the\n> + * lowest position in \\a mbusCodes is selected.\n> + *\n> + * The use of this method is optional, as the above criteria may not match the\n> + * needs of all pipeline handlers. Pipeline handlers may implement custom\n> + * sensor format selection when needed.\n\nAs part of copying in the function, you are doing what this statement\ndescribes. So I think you can remove that paragraph from \"The use of\nthis method...\"\n\nReviewed-by: Kieran Bingham \n\n\n> + *\n> + * The returned sensor output format is guaranteed to be acceptable by the\n> + * setFormat() method without any modification.\n> + *\n> + * \\return The best sensor output format matching the desired media bus codes\n> + * and size on success, or an empty format otherwise.\n> + */\n> +V4L2SubdeviceFormat CIO2Device::getSensorFormat(const std::vector &mbusCodes,\n> +\t\t\t\t\t\tconst Size &size) const\n> +{\n> +\tunsigned int desiredArea = size.width * size.height;\n> +\tunsigned int bestArea = std::numeric_limits::max();\n> +\tfloat desiredRatio = static_cast(size.width) / size.height;\n> +\tfloat bestRatio = std::numeric_limits::max();\n> +\tSize bestSize;\n> +\tuint32_t bestCode = 0;\n> +\n> +\tfor (unsigned int code : mbusCodes) {\n> +\t\tconst auto sizes = sensor_->sizes(code);\n> +\t\tif (!sizes.size())\n> +\t\t\tcontinue;\n> +\n> +\t\tfor (const Size &sz : sizes) {\n> +\t\t\tif (sz.width < size.width || sz.height < size.height)\n> +\t\t\t\tcontinue;\n> +\n> +\t\t\tfloat ratio = static_cast(sz.width) / sz.height;\n> +\t\t\tfloat ratioDiff = fabsf(ratio - desiredRatio);\n> +\t\t\tunsigned int area = sz.width * sz.height;\n> +\t\t\tunsigned int areaDiff = area - desiredArea;\n> +\n> +\t\t\tif (ratioDiff > bestRatio)\n> +\t\t\t\tcontinue;\n> +\n> +\t\t\tif (ratioDiff < bestRatio || areaDiff < bestArea) {\n> +\t\t\t\tbestRatio = ratioDiff;\n> +\t\t\t\tbestArea = areaDiff;\n> +\t\t\t\tbestSize = sz;\n> +\t\t\t\tbestCode = code;\n> +\t\t\t}\n> +\t\t}\n> +\t}\n> +\n> +\tif (bestSize.isNull()) {\n> +\t\tLOG(IPU3, Debug) << \"No supported format or size found\";\n> +\t\treturn {};\n> +\t}\n> +\n> +\tV4L2SubdeviceFormat format{\n> +\t\t.mbus_code = bestCode,\n> +\t\t.size = bestSize,\n> +\t};\n> +\n> +\treturn format;\n> +}\n> +\n> int CIO2Device::exportBuffers(unsigned int count,\n> \t\t\t std::vector> *buffers)\n> {\n> diff --git a/src/libcamera/pipeline/ipu3/cio2.h b/src/libcamera/pipeline/ipu3/cio2.h\n> index 24272dc5..5fffe921 100644\n> --- a/src/libcamera/pipeline/ipu3/cio2.h\n> +++ b/src/libcamera/pipeline/ipu3/cio2.h\n> @@ -45,6 +45,9 @@ public:\n> \tint exportBuffers(unsigned int count,\n> \t\t\t std::vector> *buffers);\n> \n> +\tV4L2SubdeviceFormat getSensorFormat(const std::vector &mbusCodes,\n> +\t\t\t\t\t const Size &size) const;\n> +\n> \tint start();\n> \tint stop();\n> \n>","headers":{"Return-Path":"","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id 49517BD87C\n\tfor ;\n\tFri, 27 Aug 2021 12:41:48 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id B01886891F;\n\tFri, 27 Aug 2021 14:41:47 +0200 (CEST)","from perceval.ideasonboard.com (perceval.ideasonboard.com\n\t[213.167.242.64])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id 66D4E60256\n\tfor ;\n\tFri, 27 Aug 2021 14:41:46 +0200 (CEST)","from [192.168.0.20]\n\t(cpc89244-aztw30-2-0-cust3082.18-1.cable.virginm.net [86.31.172.11])\n\tby perceval.ideasonboard.com (Postfix) with ESMTPSA id EA8F75A1;\n\tFri, 27 Aug 2021 14:41:45 +0200 (CEST)"],"Authentication-Results":"lancelot.ideasonboard.com; dkim=pass (1024-bit key;\n\tunprotected) header.d=ideasonboard.com header.i=@ideasonboard.com\n\theader.b=\"aBT2zeoI\"; dkim-atps=neutral","DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com;\n\ts=mail; t=1630068106;\n\tbh=68aVQATAsVOUuRYbxGWELHrZxkvXCGhA7Wgq52OuFsw=;\n\th=To:References:From:Subject:Date:In-Reply-To:From;\n\tb=aBT2zeoIbG1Y9jJclKLiXvweGCAPAaFE03uUJFZq5RhHNk12qyerytKQF+c6SrbhS\n\tiHWJ7/Q7sBlgsmR0ADoTsRLg0wKwkbmfYJz+OYQdxtpvrhZPehpcfZXGHIq8tDuqW+\n\tMhcopwHaILlgluCMFUFTYpgcqIr0AY4FpJ7fxKm4=","To":"Umang Jain ,\n\tlibcamera-devel@lists.libcamera.org","References":"<20210810075854.86191-1-umang.jain@ideasonboard.com>\n\t<20210810075854.86191-3-umang.jain@ideasonboard.com>","From":"Kieran Bingham ","Message-ID":"<8b451b0c-c43a-f46c-c9cf-05673e2d3710@ideasonboard.com>","Date":"Fri, 27 Aug 2021 13:41:43 +0100","User-Agent":"Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101\n\tThunderbird/78.11.0","MIME-Version":"1.0","In-Reply-To":"<20210810075854.86191-3-umang.jain@ideasonboard.com>","Content-Type":"text/plain; charset=utf-8","Content-Language":"en-GB","Content-Transfer-Encoding":"8bit","Subject":"Re: [libcamera-devel] [PATCH v2 2/4] ipu3: cio2: Replicate\n\tCameraSensor::getFormats() to a member function","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" "}}]