[{"id":14243,"web_url":"https://patchwork.libcamera.org/comment/14243/","msgid":"<19547176-38c5-201a-549a-6be68d25b3c8@collabora.com>","date":"2020-12-14T12:04:50","subject":"Re: [libcamera-devel] [PATCH v3]\n\tpipeline-rkisp1-Fix-sensor-ISP-format-mismatch","submitter":{"id":20,"url":"https://patchwork.libcamera.org/api/people/20/","name":"Helen Koike","email":"helen.koike@collabora.com"},"content":"Hi Sebastian,\n\nOn 12/13/20 10:42 AM, Sebastian Fricke wrote:\n> This patch fixes a mismatch of image formats during the pipeline\n> creation of the RkISP1. The mismatch happens because the current code\n> does not check if the configured format exceeds the maximum viable\n> resolution of the ISP.\n> \n> Make sure to use a sensor format resolution that is smaller or equal to\n> the maximum allowed resolution for the RkISP1. The maximum resolution\n> is defined within the `rkisp1-common.h` file as:\n> define RKISP1_ISP_MAX_WIDTH        4032\n> define RKISP1_ISP_MAX_HEIGHT        3024\n> \n> Enumerate the frame-sizes of the ISP entity and compare the maximum with\n> the configured resolution.\n> \n> This means that some camera sensors can never operate with their maximum\n> resolution, for example on my OV13850 camera sensor, there are two\n> possible resolutions: 4224x3136 & 2112x1568, the first of those two will\n> never be picked as it surpasses the maximum of the ISP.\n> \n> Signed-off-by: Sebastian Fricke <sebastian.fricke.linux@gmail.com>\n> ---\n> \n> Attention: This patch depends on the following patch for the Linux\n> Kernel media subsystem.\n> https://patchwork.kernel.org/project/linux-media/patch/20201212185306.19135-1-sebastian.fricke.linux@gmail.com/\n> \n> v1: https://lists.libcamera.org/pipermail/libcamera-devel/2020-November/015113.html\n> v2: https://lists.libcamera.org/pipermail/libcamera-devel/2020-November/015136.html\n> \n> Changelog:\n> \n> Changes since v2:\n> \n> * Replace the act of attempting to set the ISP format before negotiating\n>   the actual format for both the ISP input pad and the sensor in order\n>   to get the maximum frame size. With a logic that involves enumerating\n>   the maximum size directly from the subdevice and using that size for\n>   the negotiation process.\n> * Improve the log messages\n> \n> Changes since v1:\n> \n> * Change snake_case variable names to camelCase\n> * Use the request comment style\n> * Correct the scope of the newly implemented variables\n> * Correct the subject of the debug log for the ISP format configuration\n> * Update the comment above the ISP format configuration\n> * Check if the original format is not equal to the configured ISP format\n>   instead of checking if it is greater, this denies a false positive\n>   where the height exceeds the maximum while the width is smaller.\n>   If the configured format does not exceed the maximum resolution of the\n>   ISP, it will stay untouched so the inequality always means that we\n>   have to reconfigure the format.\n> * Adjust the comparison of the ISP format size with the available sensor\n>   formats, to detect a false-positive were the width is smaller while\n>   the height exceeds the maximum\n> * Use the standard function `max`\n> \n> -----\n> \n> The following tests were all able to create a working camera pipeline:\n> 1. Without stream configuration\n> 2. With a normal stream configuration\n> 3. With a stream configuration that exceeds the maximum of the ISP\n> 4. With a very small resolution stream configuration\n> 5. With a configuration that is closer to the upper than to the lower\n> resolution\n> \n> -----\n> \n> 1. LIBCAMERA_LOG_LEVELS=0 cam -c 1 --capture=3\n> \n> [7:56:14.564557994] [9200] DEBUG Camera camera.cpp:831 streams configuration: (0) 1920x1920-NV12\n> [7:56:14.565294449] [9200]  INFO Camera camera.cpp:890 configuring streams: (0) 1920x1920-NV12\n> ...\n> [7:56:14.567584898] [9201]  INFO RkISP1 rkisp1.cpp:689 Sensor format 4224x3136 is not supported by the ISP (maximum: 4032x3024), trying to re-configure to a smaller sensor format\n> [7:56:14.567738605] [9201] DEBUG RkISP1 rkisp1.cpp:713 Sensor configured with 2112x1568-SBGGR10_1X10\n> [7:56:14.567941896] [9201] DEBUG RkISP1 rkisp1.cpp:724 ISP input pad configured with 2112x1568-SBGGR10_1X10 crop (0x0)/2112x1568\n> [7:56:14.568196812] [9201] DEBUG RkISP1 rkisp1.cpp:730 Configuring ISP output pad with 2112x1568-YUYV8_2X8 crop (0x0)/2112x1568\n> [7:56:14.568360144] [9201] DEBUG RkISP1 rkisp1.cpp:742 ISP output pad configured with 2112x1568-YUYV8_2X8 crop (0x0)/2112x1568\n> [7:56:14.568529602] [9201] DEBUG RkISP1 rkisp1_path.cpp:119 Configured main resizer input pad with 2112x1568-YUYV8_2X8 crop (0x0)/2112x1568\n> [7:56:14.568647435] [9201] DEBUG RkISP1 rkisp1_path.cpp:125 Configuring main resizer output pad with 1920x1920-YUYV8_2X8\n> [7:56:14.568762351] [9201] DEBUG RkISP1 rkisp1_path.cpp:143 Configured main resizer output pad with 1920x1920-YUYV8_1_5X8\n> \n> 2. LIBCAMERA_LOG_LEVELS=0 cam -c 1 --capture=3 -s width=900,height=600,pixelformat=NV12,role=video\n> \n> [7:57:58.012056371] [9204] DEBUG Camera camera.cpp:831 streams configuration: (0) 1920x1920-NV12\n> [7:57:58.012524202] [9204]  INFO Camera camera.cpp:890 configuring streams: (0) 900x600-NV12\n> ...\n> [7:57:58.014565860] [9205] DEBUG RkISP1 rkisp1.cpp:713 Sensor configured with 2112x1568-SBGGR10_1X10\n> [7:57:58.014776442] [9205] DEBUG RkISP1 rkisp1.cpp:724 ISP input pad configured with 2112x1568-SBGGR10_1X10 crop (0x0)/2112x1568\n> [7:57:58.014913816] [9205] DEBUG RkISP1 rkisp1.cpp:730 Configuring ISP output pad with 2112x1568-YUYV8_2X8 crop (0x0)/2112x1568\n> [7:57:58.015064607] [9205] DEBUG RkISP1 rkisp1.cpp:742 ISP output pad configured with 2112x1568-YUYV8_2X8 crop (0x0)/2112x1568\n> [7:57:58.015230273] [9205] DEBUG RkISP1 rkisp1_path.cpp:119 Configured main resizer input pad with 2112x1568-YUYV8_2X8 crop (0x0)/2112x1568\n> [7:57:58.015348689] [9205] DEBUG RkISP1 rkisp1_path.cpp:125 Configuring main resizer output pad with 900x600-YUYV8_2X8\n> [7:57:58.015463314] [9205] DEBUG RkISP1 rkisp1_path.cpp:143 Configured main resizer output pad with 900x600-YUYV8_1_5X8\n> \n> 3. LIBCAMERA_LOG_LEVELS=0 cam -c 1 --capture=3 -s width=4500,height=3500,pixelformat=NV12,role=video\n> \n> [7:59:52.764968202] [9210] DEBUG Camera camera.cpp:831 streams configuration: (0) 1920x1920-NV12\n> [7:59:52.765415033] [9210] DEBUG RkISP1 rkisp1_path.cpp:94 Adjusting format from 4500x3500-NV12 to 4416x3312-NV12\n> [7:59:52.765671115] [9210] DEBUG RkISP1 rkisp1_path.cpp:94 Adjusting format from 4500x3500-NV12 to 1920x1920-NV12\n> [7:59:52.765905613] [9210] DEBUG RkISP1 rkisp1_path.cpp:94 Adjusting format from 4500x3500-NV12 to 4416x3312-NV12\n> [7:59:52.766147987] [9210] DEBUG CameraSensor camera_sensor.cpp:436 'ov13850 1-0010': No supported format or size found\n> Camera configuration adjusted\n> [7:59:52.766558943] [9210] DEBUG CameraSensor camera_sensor.cpp:436 'ov13850 1-0010': No supported format or size found\n> [7:59:52.766702151] [9210]  INFO Camera camera.cpp:890 configuring streams: (0) 4416x3312-NV12\n> ...\n> [7:59:52.768968097] [9211]  INFO RkISP1 rkisp1.cpp:689 Sensor format 4224x3136 is not supported by the ISP (maximum: 4032x3024), trying to re-configure to a smaller sensor format\n> [7:59:52.769120347] [9211] DEBUG RkISP1 rkisp1.cpp:713 Sensor configured with 2112x1568-SBGGR10_1X10\n> [7:59:52.769320721] [9211] DEBUG RkISP1 rkisp1.cpp:724 ISP input pad configured with 2112x1568-SBGGR10_1X10 crop (0x0)/2112x1568\n> [7:59:52.769455762] [9211] DEBUG RkISP1 rkisp1.cpp:730 Configuring ISP output pad with 2112x1568-YUYV8_2X8 crop (0x0)/2112x1568\n> [7:59:52.769606552] [9211] DEBUG RkISP1 rkisp1.cpp:742 ISP output pad configured with 2112x1568-YUYV8_2X8 crop (0x0)/2112x1568\n> [7:59:52.769771927] [9211] DEBUG RkISP1 rkisp1_path.cpp:119 Configured main resizer input pad with 2112x1568-YUYV8_2X8 crop (0x0)/2112x1568\n> [7:59:52.769888301] [9211] DEBUG RkISP1 rkisp1_path.cpp:125 Configuring main resizer output pad with 4416x3312-YUYV8_2X8\n> [7:59:52.770003217] [9211] DEBUG RkISP1 rkisp1_path.cpp:143 Configured main resizer output pad with 4416x3312-YUYV8_1_5X8\n> \n> 4. LIBCAMERA_LOG_LEVELS=0 cam -c 1 --capture=3 -s width=40,height=30,pixelformat=NV12,role=video\n> \n> [8:03:00.496627233] [9218] DEBUG Camera camera.cpp:831 streams configuration: (0) 1920x1920-NV12\n> [8:03:00.497419103] [9218]  INFO Camera camera.cpp:890 configuring streams: (0) 40x30-NV12\n> ...\n> [8:03:00.499715966] [9219] DEBUG RkISP1 rkisp1.cpp:713 Sensor configured with 2112x1568-SBGGR10_1X10\n> [8:03:00.499924215] [9219] DEBUG RkISP1 rkisp1.cpp:724 ISP input pad configured with 2112x1568-SBGGR10_1X10 crop (0x0)/2112x1568\n> [8:03:00.500168047] [9219] DEBUG RkISP1 rkisp1.cpp:730 Configuring ISP output pad with 2112x1568-YUYV8_2X8 crop (0x0)/2112x1568\n> [8:03:00.500336630] [9219] DEBUG RkISP1 rkisp1.cpp:742 ISP output pad configured with 2112x1568-YUYV8_2X8 crop (0x0)/2112x1568\n> [8:03:00.500513087] [9219] DEBUG RkISP1 rkisp1_path.cpp:119 Configured main resizer input pad with 2112x1568-YUYV8_2X8 crop (0x0)/2112x1568\n> [8:03:00.500632962] [9219] DEBUG RkISP1 rkisp1_path.cpp:125 Configuring main resizer output pad with 40x30-YUYV8_2X8\n> [8:03:00.500745836] [9219] DEBUG RkISP1 rkisp1_path.cpp:143 Configured main resizer output pad with 40x30-YUYV8_1_5X8\n> \n> 5. LIBCAMERA_LOG_LEVELS=0 cam -c 1 --capture=3 -s width=3450,height=2456,pixelformat=NV12,role=video\n> \n> [8:01:23.689086810] [9215] DEBUG Camera camera.cpp:831 streams configuration: (0) 1920x1920-NV12\n> [8:01:23.689893848] [9215]  INFO Camera camera.cpp:890 configuring streams: (0) 3450x2456-NV12\n> ...\n> [8:01:23.692257211] [9216]  INFO RkISP1 rkisp1.cpp:689 Sensor format 4224x3136 is not supported by the ISP (maximum: 4032x3024), trying to re-configure to a smaller sensor format\n> [8:01:23.692415876] [9216] DEBUG RkISP1 rkisp1.cpp:713 Sensor configured with 2112x1568-SBGGR10_1X10\n> [8:01:23.692615084] [9216] DEBUG RkISP1 rkisp1.cpp:724 ISP input pad configured with 2112x1568-SBGGR10_1X10 crop (0x0)/2112x1568\n> [8:01:23.692753041] [9216] DEBUG RkISP1 rkisp1.cpp:730 Configuring ISP output pad with 2112x1568-YUYV8_2X8 crop (0x0)/2112x1568\n> [8:01:23.692901499] [9216] DEBUG RkISP1 rkisp1.cpp:742 ISP output pad configured with 2112x1568-YUYV8_2X8 crop (0x0)/2112x1568\n> [8:01:23.693066581] [9216] DEBUG RkISP1 rkisp1_path.cpp:119 Configured main resizer input pad with 2112x1568-YUYV8_2X8 crop (0x0)/2112x1568\n> [8:01:23.693183831] [9216] DEBUG RkISP1 rkisp1_path.cpp:125 Configuring main resizer output pad with 3450x2456-YUYV8_2X8\n> [8:01:23.693297289] [9216] DEBUG RkISP1 rkisp1_path.cpp:143 Configured main resizer output pad with 3450x2456-YUYV8_1_5X8\n> \n> ---\n>  src/libcamera/pipeline/rkisp1/rkisp1.cpp | 34 +++++++++++++++++++++++-\n>  1 file changed, 33 insertions(+), 1 deletion(-)\n> \n> diff --git a/src/libcamera/pipeline/rkisp1/rkisp1.cpp b/src/libcamera/pipeline/rkisp1/rkisp1.cpp\n> index 4d98c902..d6fc624c 100644\n> --- a/src/libcamera/pipeline/rkisp1/rkisp1.cpp\n> +++ b/src/libcamera/pipeline/rkisp1/rkisp1.cpp\n> @@ -667,12 +667,44 @@ int PipelineHandlerRkISP1::configure(Camera *camera, CameraConfiguration *c)\n>  \tif (ret)\n>  \t\treturn ret;\n>  \n> +\tV4L2Subdevice::Formats ispFormats = isp_->formats(0);\n> +\tif (ispFormats.empty())\n> +\t\treturn -1;\n\nReturning -1 doesn't seem appropriated. Please check how other functions return errors.\nI would guess -EINVAL.\n\n> +\n> +\t/*\n> +\t * The maximum resolution is identical for all media bus codes on\n> +\t * the RkISP1 isp entity. Therefore take the first available resolution.\n> +\t */\n> +\tSize ispMaximum = ispFormats.begin()->second[0].max;\n> +\n>  \t/*\n>  \t * Configure the format on the sensor output and propagate it through\n>  \t * the pipeline.\n>  \t */\n>  \tV4L2SubdeviceFormat format = config->sensorFormat();\n> -\tLOG(RkISP1, Debug) << \"Configuring sensor with \" << format.toString();\n> +\n> +\tif (ispMaximum.width < format.size.width ||\n> +\t    ispMaximum.height < format.size.height) {\n> +\t\tSize maxSize;\n> +\t\tLOG(RkISP1, Info) << \"Sensor format \" << format.size.toString()\n> +\t\t\t\t  << \" is not supported by the ISP (maximum: \"\n> +\t\t\t\t  << ispMaximum.toString() << \"), trying to \"\n> +\t\t\t\t  << \"re-configure to a smaller sensor format\";\n> +\n> +\t\tfor (const Size &size : sensor->sizes()) {\n> +\t\t\tif (size.width > ispMaximum.width ||\n> +\t\t\t    size.height > ispMaximum.height)\n> +\t\t\t\tcontinue;\n> +\t\t\tmaxSize = std::max(maxSize, size);\n> +\t\t}\n> +\t\tif (maxSize == Size(0, 0)) {\n> +\t\t\tLOG(RkISP1, Error) << \"No available sensor resolution\"\n> +\t\t\t\t\t      \"that is smaller or equal to \"\n> +\t\t\t\t\t   << format.toString();\n> +\t\t\treturn -1;\n\nReturning -1 doesn't seem appropriated. Please check how other functions return errors.\nI would guess -EINVAL.\n\nWith this change:\n\nReviewed-by: Helen Koike <helen.koike@collabora.com>\n\nThanks\nHelen\n\n> +\t\t}\n> +\t\tformat.size = maxSize;\n> +\t}\n>  \n>  \tret = sensor->setFormat(&format);\n>  \tif (ret < 0)\n>","headers":{"Return-Path":"<libcamera-devel-bounces@lists.libcamera.org>","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id 7C4C1BD80A\n\tfor <parsemail@patchwork.libcamera.org>;\n\tMon, 14 Dec 2020 12:04:57 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id 117A46158C;\n\tMon, 14 Dec 2020 13:04:57 +0100 (CET)","from bhuna.collabora.co.uk (bhuna.collabora.co.uk\n\t[IPv6:2a00:1098:0:82:1000:25:2eeb:e3e3])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id 3333260321\n\tfor <libcamera-devel@lists.libcamera.org>;\n\tMon, 14 Dec 2020 13:04:56 +0100 (CET)","from [127.0.0.1] (localhost [127.0.0.1])\n\t(Authenticated sender: koike) with ESMTPSA id 930C81F44C82"],"To":"Sebastian Fricke <sebastian.fricke.linux@gmail.com>,\n\tlibcamera-devel@lists.libcamera.org","References":"<20201213134251.7574-1-sebastian.fricke.linux@gmail.com>","From":"Helen Koike <helen.koike@collabora.com>","Message-ID":"<19547176-38c5-201a-549a-6be68d25b3c8@collabora.com>","Date":"Mon, 14 Dec 2020 09:04:50 -0300","User-Agent":"Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101\n\tThunderbird/78.4.0","MIME-Version":"1.0","In-Reply-To":"<20201213134251.7574-1-sebastian.fricke.linux@gmail.com>","Content-Language":"en-US","Subject":"Re: [libcamera-devel] [PATCH v3]\n\tpipeline-rkisp1-Fix-sensor-ISP-format-mismatch","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"<libcamera-devel.lists.libcamera.org>","List-Unsubscribe":"<https://lists.libcamera.org/options/libcamera-devel>,\n\t<mailto:libcamera-devel-request@lists.libcamera.org?subject=unsubscribe>","List-Archive":"<https://lists.libcamera.org/pipermail/libcamera-devel/>","List-Post":"<mailto:libcamera-devel@lists.libcamera.org>","List-Help":"<mailto:libcamera-devel-request@lists.libcamera.org?subject=help>","List-Subscribe":"<https://lists.libcamera.org/listinfo/libcamera-devel>,\n\t<mailto:libcamera-devel-request@lists.libcamera.org?subject=subscribe>","Content-Type":"text/plain; charset=\"utf-8\"","Content-Transfer-Encoding":"base64","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" <libcamera-devel-bounces@lists.libcamera.org>"}},{"id":14246,"web_url":"https://patchwork.libcamera.org/comment/14246/","msgid":"<05aee61e-cf29-a397-ef6e-b53123264c53@collabora.com>","date":"2020-12-14T15:57:26","subject":"Re: [libcamera-devel] [PATCH v3]\n\tpipeline-rkisp1-Fix-sensor-ISP-format-mismatch","submitter":{"id":46,"url":"https://patchwork.libcamera.org/api/people/46/","name":"Dafna Hirschfeld","email":"dafna.hirschfeld@collabora.com"},"content":"Am 13.12.20 um 14:42 schrieb Sebastian Fricke:\n> This patch fixes a mismatch of image formats during the pipeline\n> creation of the RkISP1. The mismatch happens because the current code\n> does not check if the configured format exceeds the maximum viable\n> resolution of the ISP.\n> \n> Make sure to use a sensor format resolution that is smaller or equal to\n> the maximum allowed resolution for the RkISP1. The maximum resolution\n> is defined within the `rkisp1-common.h` file as:\n> define RKISP1_ISP_MAX_WIDTH        4032\n> define RKISP1_ISP_MAX_HEIGHT        3024\n> \n> Enumerate the frame-sizes of the ISP entity and compare the maximum with\n> the configured resolution.\n> \n> This means that some camera sensors can never operate with their maximum\n> resolution, for example on my OV13850 camera sensor, there are two\n> possible resolutions: 4224x3136 & 2112x1568, the first of those two will\n> never be picked as it surpasses the maximum of the ISP.\n> \n> Signed-off-by: Sebastian Fricke <sebastian.fricke.linux@gmail.com>\n> ---\n> \n> Attention: This patch depends on the following patch for the Linux\n> Kernel media subsystem.\n> https://patchwork.kernel.org/project/linux-media/patch/20201212185306.19135-1-sebastian.fricke.linux@gmail.com/\n\nYou can add a patch that uses the new VIDIOC_SUBDEV_ENUM_FRAME_SIZE  in the end of the 'RkISP1CameraConfiguration::validate()'\nfunction instead of that hard-coded list:\n\n  sensorFormat_ = sensor->getFormat({ MEDIA_BUS_FMT_SBGGR12_1X12,\n                                                 MEDIA_BUS_FMT_SGBRG12_1X12,\n                                                 MEDIA_BUS_FMT_SGRBG12_1X12,\n                                                 MEDIA_BUS_FMT_SRGGB12_1X12,\n                                                 MEDIA_BUS_FMT_SBGGR10_1X10,\n                                                 MEDIA_BUS_FMT_SGBRG10_1X10,\n                                                MEDIA_BUS_FMT_SGRBG10_1X10,\n                                                MEDIA_BUS_FMT_SRGGB10_1X10,\n                                                MEDIA_BUS_FMT_SBGGR8_1X8,\n                                                MEDIA_BUS_FMT_SGBRG8_1X8,\n                                                MEDIA_BUS_FMT_SGRBG8_1X8,\n                                                MEDIA_BUS_FMT_SRGGB8_1X8 },\n                                              maxSize);\n\n> \n> v1: https://lists.libcamera.org/pipermail/libcamera-devel/2020-November/015113.html\n> v2: https://lists.libcamera.org/pipermail/libcamera-devel/2020-November/015136.html\n> \n> Changelog:\n> \n> Changes since v2:\n> \n> * Replace the act of attempting to set the ISP format before negotiating\n>    the actual format for both the ISP input pad and the sensor in order\n>    to get the maximum frame size. With a logic that involves enumerating\n>    the maximum size directly from the subdevice and using that size for\n>    the negotiation process.\n> * Improve the log messages\n> \n> Changes since v1:\n> \n> * Change snake_case variable names to camelCase\n> * Use the request comment style\n> * Correct the scope of the newly implemented variables\n> * Correct the subject of the debug log for the ISP format configuration\n> * Update the comment above the ISP format configuration\n> * Check if the original format is not equal to the configured ISP format\n>    instead of checking if it is greater, this denies a false positive\n>    where the height exceeds the maximum while the width is smaller.\n>    If the configured format does not exceed the maximum resolution of the\n>    ISP, it will stay untouched so the inequality always means that we\n>    have to reconfigure the format.\n> * Adjust the comparison of the ISP format size with the available sensor\n>    formats, to detect a false-positive were the width is smaller while\n>    the height exceeds the maximum\n> * Use the standard function `max`\n> \n> -----\n> \n> The following tests were all able to create a working camera pipeline:\n> 1. Without stream configuration\n> 2. With a normal stream configuration\n> 3. With a stream configuration that exceeds the maximum of the ISP\n> 4. With a very small resolution stream configuration\n> 5. With a configuration that is closer to the upper than to the lower\n> resolution\n> \n> -----\n> \n> 1. LIBCAMERA_LOG_LEVELS=0 cam -c 1 --capture=3\n> \n> [7:56:14.564557994] [9200] DEBUG Camera camera.cpp:831 streams configuration: (0) 1920x1920-NV12\n> [7:56:14.565294449] [9200]  INFO Camera camera.cpp:890 configuring streams: (0) 1920x1920-NV12\n> ...\n> [7:56:14.567584898] [9201]  INFO RkISP1 rkisp1.cpp:689 Sensor format 4224x3136 is not supported by the ISP (maximum: 4032x3024), trying to re-configure to a smaller sensor format\n> [7:56:14.567738605] [9201] DEBUG RkISP1 rkisp1.cpp:713 Sensor configured with 2112x1568-SBGGR10_1X10\n> [7:56:14.567941896] [9201] DEBUG RkISP1 rkisp1.cpp:724 ISP input pad configured with 2112x1568-SBGGR10_1X10 crop (0x0)/2112x1568\n> [7:56:14.568196812] [9201] DEBUG RkISP1 rkisp1.cpp:730 Configuring ISP output pad with 2112x1568-YUYV8_2X8 crop (0x0)/2112x1568\n> [7:56:14.568360144] [9201] DEBUG RkISP1 rkisp1.cpp:742 ISP output pad configured with 2112x1568-YUYV8_2X8 crop (0x0)/2112x1568\n> [7:56:14.568529602] [9201] DEBUG RkISP1 rkisp1_path.cpp:119 Configured main resizer input pad with 2112x1568-YUYV8_2X8 crop (0x0)/2112x1568\n> [7:56:14.568647435] [9201] DEBUG RkISP1 rkisp1_path.cpp:125 Configuring main resizer output pad with 1920x1920-YUYV8_2X8\n> [7:56:14.568762351] [9201] DEBUG RkISP1 rkisp1_path.cpp:143 Configured main resizer output pad with 1920x1920-YUYV8_1_5X8\n> \n> 2. LIBCAMERA_LOG_LEVELS=0 cam -c 1 --capture=3 -s width=900,height=600,pixelformat=NV12,role=video\n> \n> [7:57:58.012056371] [9204] DEBUG Camera camera.cpp:831 streams configuration: (0) 1920x1920-NV12\n> [7:57:58.012524202] [9204]  INFO Camera camera.cpp:890 configuring streams: (0) 900x600-NV12\n> ...\n> [7:57:58.014565860] [9205] DEBUG RkISP1 rkisp1.cpp:713 Sensor configured with 2112x1568-SBGGR10_1X10\n> [7:57:58.014776442] [9205] DEBUG RkISP1 rkisp1.cpp:724 ISP input pad configured with 2112x1568-SBGGR10_1X10 crop (0x0)/2112x1568\n> [7:57:58.014913816] [9205] DEBUG RkISP1 rkisp1.cpp:730 Configuring ISP output pad with 2112x1568-YUYV8_2X8 crop (0x0)/2112x1568\n> [7:57:58.015064607] [9205] DEBUG RkISP1 rkisp1.cpp:742 ISP output pad configured with 2112x1568-YUYV8_2X8 crop (0x0)/2112x1568\n> [7:57:58.015230273] [9205] DEBUG RkISP1 rkisp1_path.cpp:119 Configured main resizer input pad with 2112x1568-YUYV8_2X8 crop (0x0)/2112x1568\n> [7:57:58.015348689] [9205] DEBUG RkISP1 rkisp1_path.cpp:125 Configuring main resizer output pad with 900x600-YUYV8_2X8\n> [7:57:58.015463314] [9205] DEBUG RkISP1 rkisp1_path.cpp:143 Configured main resizer output pad with 900x600-YUYV8_1_5X8\n> \n> 3. LIBCAMERA_LOG_LEVELS=0 cam -c 1 --capture=3 -s width=4500,height=3500,pixelformat=NV12,role=video\n> \n> [7:59:52.764968202] [9210] DEBUG Camera camera.cpp:831 streams configuration: (0) 1920x1920-NV12\n> [7:59:52.765415033] [9210] DEBUG RkISP1 rkisp1_path.cpp:94 Adjusting format from 4500x3500-NV12 to 4416x3312-NV12\n> [7:59:52.765671115] [9210] DEBUG RkISP1 rkisp1_path.cpp:94 Adjusting format from 4500x3500-NV12 to 1920x1920-NV12\n> [7:59:52.765905613] [9210] DEBUG RkISP1 rkisp1_path.cpp:94 Adjusting format from 4500x3500-NV12 to 4416x3312-NV12\n> [7:59:52.766147987] [9210] DEBUG CameraSensor camera_sensor.cpp:436 'ov13850 1-0010': No supported format or size found\n\nNot sure, I think that instead of adjusting the sizes to 2112x1568 in \"configure\", they should be adjusted in the 'validate'.\nSo we avoid reaching this case of \"No supported format or size found\".\n\n\n> Camera configuration adjusted\n> [7:59:52.766558943] [9210] DEBUG CameraSensor camera_sensor.cpp:436 'ov13850 1-0010': No supported format or size found\n> [7:59:52.766702151] [9210]  INFO Camera camera.cpp:890 configuring streams: (0) 4416x3312-NV12\n> ...\n> [7:59:52.768968097] [9211]  INFO RkISP1 rkisp1.cpp:689 Sensor format 4224x3136 is not supported by the ISP (maximum: 4032x3024), trying to re-configure to a smaller sensor format\n> [7:59:52.769120347] [9211] DEBUG RkISP1 rkisp1.cpp:713 Sensor configured with 2112x1568-SBGGR10_1X10\n> [7:59:52.769320721] [9211] DEBUG RkISP1 rkisp1.cpp:724 ISP input pad configured with 2112x1568-SBGGR10_1X10 crop (0x0)/2112x1568\n> [7:59:52.769455762] [9211] DEBUG RkISP1 rkisp1.cpp:730 Configuring ISP output pad with 2112x1568-YUYV8_2X8 crop (0x0)/2112x1568\n> [7:59:52.769606552] [9211] DEBUG RkISP1 rkisp1.cpp:742 ISP output pad configured with 2112x1568-YUYV8_2X8 crop (0x0)/2112x1568\n> [7:59:52.769771927] [9211] DEBUG RkISP1 rkisp1_path.cpp:119 Configured main resizer input pad with 2112x1568-YUYV8_2X8 crop (0x0)/2112x1568\n> [7:59:52.769888301] [9211] DEBUG RkISP1 rkisp1_path.cpp:125 Configuring main resizer output pad with 4416x3312-YUYV8_2X8\n> [7:59:52.770003217] [9211] DEBUG RkISP1 rkisp1_path.cpp:143 Configured main resizer output pad with 4416x3312-YUYV8_1_5X8\n> \n> 4. LIBCAMERA_LOG_LEVELS=0 cam -c 1 --capture=3 -s width=40,height=30,pixelformat=NV12,role=video\n> \n> [8:03:00.496627233] [9218] DEBUG Camera camera.cpp:831 streams configuration: (0) 1920x1920-NV12\n> [8:03:00.497419103] [9218]  INFO Camera camera.cpp:890 configuring streams: (0) 40x30-NV12\n> ...\n> [8:03:00.499715966] [9219] DEBUG RkISP1 rkisp1.cpp:713 Sensor configured with 2112x1568-SBGGR10_1X10\n> [8:03:00.499924215] [9219] DEBUG RkISP1 rkisp1.cpp:724 ISP input pad configured with 2112x1568-SBGGR10_1X10 crop (0x0)/2112x1568\n> [8:03:00.500168047] [9219] DEBUG RkISP1 rkisp1.cpp:730 Configuring ISP output pad with 2112x1568-YUYV8_2X8 crop (0x0)/2112x1568\n> [8:03:00.500336630] [9219] DEBUG RkISP1 rkisp1.cpp:742 ISP output pad configured with 2112x1568-YUYV8_2X8 crop (0x0)/2112x1568\n> [8:03:00.500513087] [9219] DEBUG RkISP1 rkisp1_path.cpp:119 Configured main resizer input pad with 2112x1568-YUYV8_2X8 crop (0x0)/2112x1568\n> [8:03:00.500632962] [9219] DEBUG RkISP1 rkisp1_path.cpp:125 Configuring main resizer output pad with 40x30-YUYV8_2X8\n> [8:03:00.500745836] [9219] DEBUG RkISP1 rkisp1_path.cpp:143 Configured main resizer output pad with 40x30-YUYV8_1_5X8\n> \n> 5. LIBCAMERA_LOG_LEVELS=0 cam -c 1 --capture=3 -s width=3450,height=2456,pixelformat=NV12,role=video\n> \n> [8:01:23.689086810] [9215] DEBUG Camera camera.cpp:831 streams configuration: (0) 1920x1920-NV12\n> [8:01:23.689893848] [9215]  INFO Camera camera.cpp:890 configuring streams: (0) 3450x2456-NV12\n> ...\n> [8:01:23.692257211] [9216]  INFO RkISP1 rkisp1.cpp:689 Sensor format 4224x3136 is not supported by the ISP (maximum: 4032x3024), trying to re-configure to a smaller sensor format\n> [8:01:23.692415876] [9216] DEBUG RkISP1 rkisp1.cpp:713 Sensor configured with 2112x1568-SBGGR10_1X10\n> [8:01:23.692615084] [9216] DEBUG RkISP1 rkisp1.cpp:724 ISP input pad configured with 2112x1568-SBGGR10_1X10 crop (0x0)/2112x1568\n> [8:01:23.692753041] [9216] DEBUG RkISP1 rkisp1.cpp:730 Configuring ISP output pad with 2112x1568-YUYV8_2X8 crop (0x0)/2112x1568\n> [8:01:23.692901499] [9216] DEBUG RkISP1 rkisp1.cpp:742 ISP output pad configured with 2112x1568-YUYV8_2X8 crop (0x0)/2112x1568\n> [8:01:23.693066581] [9216] DEBUG RkISP1 rkisp1_path.cpp:119 Configured main resizer input pad with 2112x1568-YUYV8_2X8 crop (0x0)/2112x1568\n> [8:01:23.693183831] [9216] DEBUG RkISP1 rkisp1_path.cpp:125 Configuring main resizer output pad with 3450x2456-YUYV8_2X8\n> [8:01:23.693297289] [9216] DEBUG RkISP1 rkisp1_path.cpp:143 Configured main resizer output pad with 3450x2456-YUYV8_1_5X8\n> \n> ---\n>   src/libcamera/pipeline/rkisp1/rkisp1.cpp | 34 +++++++++++++++++++++++-\n>   1 file changed, 33 insertions(+), 1 deletion(-)\n> \n> diff --git a/src/libcamera/pipeline/rkisp1/rkisp1.cpp b/src/libcamera/pipeline/rkisp1/rkisp1.cpp\n> index 4d98c902..d6fc624c 100644\n> --- a/src/libcamera/pipeline/rkisp1/rkisp1.cpp\n> +++ b/src/libcamera/pipeline/rkisp1/rkisp1.cpp\n> @@ -667,12 +667,44 @@ int PipelineHandlerRkISP1::configure(Camera *camera, CameraConfiguration *c)\n>   \tif (ret)\n>   \t\treturn ret;\n>   \n> +\tV4L2Subdevice::Formats ispFormats = isp_->formats(0);\n> +\tif (ispFormats.empty())\n> +\t\treturn -1;\n> +\n> +\t/*\n> +\t * The maximum resolution is identical for all media bus codes on\n> +\t * the RkISP1 isp entity. Therefore take the first available resolution.\n> +\t */\n> +\tSize ispMaximum = ispFormats.begin()->second[0].max;\n> +\n>   \t/*\n>   \t * Configure the format on the sensor output and propagate it through\n>   \t * the pipeline.\n>   \t */\n>   \tV4L2SubdeviceFormat format = config->sensorFormat();\n> -\tLOG(RkISP1, Debug) << \"Configuring sensor with \" << format.toString();\n> +\n> +\tif (ispMaximum.width < format.size.width ||\n> +\t    ispMaximum.height < format.size.height) {\n> +\t\tSize maxSize;\n> +\t\tLOG(RkISP1, Info) << \"Sensor format \" << format.size.toString()\n> +\t\t\t\t  << \" is not supported by the ISP (maximum: \"\n> +\t\t\t\t  << ispMaximum.toString() << \"), trying to \"\n> +\t\t\t\t  << \"re-configure to a smaller sensor format\";\n> +\n> +\t\tfor (const Size &size : sensor->sizes()) {\n> +\t\t\tif (size.width > ispMaximum.width ||\n> +\t\t\t    size.height > ispMaximum.height)\n> +\t\t\t\tcontinue;\n> +\t\t\tmaxSize = std::max(maxSize, size);\n\nNote that is it not enough to make sure that the sizes match but also that the\nmbus codes match.\n\nThanks,\nDafna\n\n> +\t\t}\n> +\t\tif (maxSize == Size(0, 0)) {\n> +\t\t\tLOG(RkISP1, Error) << \"No available sensor resolution\"\n> +\t\t\t\t\t      \"that is smaller or equal to \"\n> +\t\t\t\t\t   << format.toString();\n> +\t\t\treturn -1;\n> +\t\t}\n> +\t\tformat.size = maxSize;\n> +\t}\n>   \n>   \tret = sensor->setFormat(&format);\n>   \tif (ret < 0)\n>","headers":{"Return-Path":"<libcamera-devel-bounces@lists.libcamera.org>","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id B6931C0F1A\n\tfor <parsemail@patchwork.libcamera.org>;\n\tMon, 14 Dec 2020 15:57:31 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id 52A0F6158F;\n\tMon, 14 Dec 2020 16:57:31 +0100 (CET)","from bhuna.collabora.co.uk (bhuna.collabora.co.uk [46.235.227.227])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id 7CC0B60321\n\tfor <libcamera-devel@lists.libcamera.org>;\n\tMon, 14 Dec 2020 16:57:30 +0100 (CET)","from [127.0.0.1] (localhost [127.0.0.1])\n\t(Authenticated sender: dafna) with ESMTPSA id 0D2B11F44B43"],"To":"Sebastian Fricke <sebastian.fricke.linux@gmail.com>,\n\tlibcamera-devel@lists.libcamera.org","References":"<20201213134251.7574-1-sebastian.fricke.linux@gmail.com>","From":"Dafna Hirschfeld <dafna.hirschfeld@collabora.com>","Message-ID":"<05aee61e-cf29-a397-ef6e-b53123264c53@collabora.com>","Date":"Mon, 14 Dec 2020 16:57:26 +0100","User-Agent":"Mozilla/5.0 (X11; Linux x86_64; rv:68.0) Gecko/20100101\n\tThunderbird/68.10.0","MIME-Version":"1.0","In-Reply-To":"<20201213134251.7574-1-sebastian.fricke.linux@gmail.com>","Content-Language":"en-US","Subject":"Re: [libcamera-devel] [PATCH v3]\n\tpipeline-rkisp1-Fix-sensor-ISP-format-mismatch","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"<libcamera-devel.lists.libcamera.org>","List-Unsubscribe":"<https://lists.libcamera.org/options/libcamera-devel>,\n\t<mailto:libcamera-devel-request@lists.libcamera.org?subject=unsubscribe>","List-Archive":"<https://lists.libcamera.org/pipermail/libcamera-devel/>","List-Post":"<mailto:libcamera-devel@lists.libcamera.org>","List-Help":"<mailto:libcamera-devel-request@lists.libcamera.org?subject=help>","List-Subscribe":"<https://lists.libcamera.org/listinfo/libcamera-devel>,\n\t<mailto:libcamera-devel-request@lists.libcamera.org?subject=subscribe>","Content-Transfer-Encoding":"base64","Content-Type":"text/plain; charset=\"utf-8\"; Format=\"flowed\"","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" <libcamera-devel-bounces@lists.libcamera.org>"}}]