From patchwork Thu Nov 19 13:13:30 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sebastian Fricke X-Patchwork-Id: 10450 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 3E771BE176 for ; Thu, 19 Nov 2020 13:13:37 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id B6CFD615A8; Thu, 19 Nov 2020 14:13:36 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.b="XvBvGAqY"; dkim-atps=neutral Received: from mail-ed1-x536.google.com (mail-ed1-x536.google.com [IPv6:2a00:1450:4864:20::536]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id CD222615A2 for ; Thu, 19 Nov 2020 14:13:34 +0100 (CET) Received: by mail-ed1-x536.google.com with SMTP id v22so5754692edt.9 for ; Thu, 19 Nov 2020 05:13:34 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:mime-version :content-transfer-encoding; bh=5q5ZE80Ex4cGaMzfDQ7cgmoJDwR3tivQc9lEFndLJ6Q=; b=XvBvGAqYrnwicQ0oVX6+b3WAGMXsstDy/hlsMzh83xF+GqNerE8FVsR1aIT/6qeSt8 39C2UC8P1U/uSdfQhhdID5ay6SIJomlzjCgf8ZkrcbR+fv2cdk9TCrJpoQzsfBOrmedq pUxIZczqwESMkSJrkMwQALKp8pW2xwZP/75tP6VTM4kNLKG7TwE4ac5SGy+KrcVk8wrN uJlm980U6imyHCACNsLIejUOBLbUjnZBrGEXignDSH+nhryPr0Egf5WOIq4v7BD9J3yv ZpNrVc8nCJZ2HJTsRSXEKpCH4cXxJkznTcXFoh2DFTh/xAuglKeeOqP5Oz2Ptt8AyyZx hPVQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:mime-version :content-transfer-encoding; bh=5q5ZE80Ex4cGaMzfDQ7cgmoJDwR3tivQc9lEFndLJ6Q=; b=YWSEVAki8+xYIVaqSUmkoZyTcDtESRwYxovG0oV85uGDBSuVXiAVuE5EjPJe4upwmm jQO5txydM71dHOvvobzcb6hvaosQL+XyFRtT2LDXAXQA4TT8UuaFuNZj1QDO9WN3lPXs psDV72K+x0Tj8Mf42579ooaAIwOpJdQkA5p39yX3d2xlpC2T/zORW6UzG73ksZ1ysLdV sPdx4fqhIH0NAqMVamgU3NHJZBErEAmpHsr24FKwmc1s40RZeUTZTycQ4/S0JUTh4jRB bT8d0lmkgQVo/D6Ryg6ZBGs/hNZMdlb1xvo35HrMAxlDos3/rgrr0I7tt44GxKPfJfg/ dJ6A== X-Gm-Message-State: AOAM533X9/W9di+eBsBr3v8lCRq2J6LvQFnCV8QS8J02tljZP2kNDl/A CDyhk4SVSEMB/FqZNo8XvrgINDtzVtmRwQ== X-Google-Smtp-Source: ABdhPJwQdljq93trghRmxWJ05ZIISHkv5kJxLrs4MbRPihxefw2b5GBhf4k8dc4Z+rDpZOpyIgJ6yw== X-Received: by 2002:a50:ed96:: with SMTP id h22mr31043186edr.336.1605791613925; Thu, 19 Nov 2020 05:13:33 -0800 (PST) Received: from basti.fritz.box (p200300d1ff000d00631290ba0e9702d8.dip0.t-ipconnect.de. [2003:d1:ff00:d00:6312:90ba:e97:2d8]) by smtp.gmail.com with ESMTPSA id d19sm15056135eds.31.2020.11.19.05.13.32 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 19 Nov 2020 05:13:33 -0800 (PST) From: Sebastian Fricke To: libcamera-devel@lists.libcamera.org Date: Thu, 19 Nov 2020 14:13:30 +0100 Message-Id: <20201119131331.30847-1-sebastian.fricke.linux@gmail.com> X-Mailer: git-send-email 2.20.1 MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH 0/1] pipeline-rkisp1-Fix-sensor-ISP-format-mismatch X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" This patch fixes a mismatch of image formats during the pipeline creation of the RkISP1. The mismatch happens because the current code does not check if the configured format exceeded the maximum viable resolution of the ISP. This patch fixes the issue by first setting the ISP, checking if the given resolution was taken and searching for another viable format that is lower or equal to the ISP maximum. Here are a couple of test runs: 1. Without stream configuration 2. With a normal stream configuration 3. With a stream configuration that exceeds the maximum of the ISP 4. With a very small resolution stream configuration 5. With a configuration that is closer to the upper than to the lower resolution ---- 1. LIBCAMERA_LOG_LEVELS=0 cam -c 1 --capture=3 [8:05:49.668381732] [9973] DEBUG Camera camera.cpp:771 streams configuration: (0) 1920x1920-NV12 [8:05:49.669108269] [9973] INFO Camera camera.cpp:830 configuring streams: (0) 1920x1920-NV12 ... [8:05:49.670965005] [9974] DEBUG RkISP1 rkisp1.cpp:687 Configuring sensor with 4224x3136-SBGGR10_1X10 [8:05:49.671126004] [9974] DEBUG RkISP1 rkisp1.cpp:694 ISP configured with 4032x3024-SBGGR10_1X10 [8:05:49.671233045] [9974] INFO RkISP1 rkisp1.cpp:697 Configured resolution is greater than the maximum resolution for the ISP, trying to re-configure to a smaller valid sensor format [8:05:49.671378294] [9974] DEBUG RkISP1 rkisp1.cpp:716 ISP re-configured with 2112x1568-SBGGR10_1X10 [8:05:49.671510710] [9974] DEBUG RkISP1 rkisp1.cpp:724 Sensor configured with 2112x1568-SBGGR10_1X10 [8:05:49.671676667] [9974] DEBUG RkISP1 rkisp1.cpp:731 ISP input pad configured with 2112x1568-SBGGR10_1X10 [8:05:49.671787791] [9974] DEBUG RkISP1 rkisp1.cpp:735 Configuring ISP output pad with 2112x1568-YUYV8_2X8 [8:05:49.671894540] [9974] DEBUG RkISP1 rkisp1.cpp:741 ISP output pad configured with 800x600-YUYV8_2X8 [8:05:49.672107747] [9974] DEBUG RkISP1 rkisp1_path.cpp:120 Configured main resizer input pad with 800x600-YUYV8_2X8 [8:05:49.672226163] [9974] DEBUG RkISP1 rkisp1_path.cpp:126 Configuring main resizer output pad with 1920x1920-YUYV8_2X8 [8:05:49.672341662] [9974] DEBUG RkISP1 rkisp1_path.cpp:144 Configured main resizer output pad with 1920x1920-YUYV8_1_5X8 2. LIBCAMERA_LOG_LEVELS=0 cam -c 1 --capture=3 -s width=900,height=600,pixelformat=NV12,role=video [8:07:05.725799205] [9977] DEBUG Camera camera.cpp:771 streams configuration: (0) 1920x1920-NV12 [8:07:05.726242535] [9977] INFO Camera camera.cpp:830 configuring streams: (0) 900x600-NV12 ... [8:07:05.727718025] [9978] DEBUG RkISP1 rkisp1.cpp:687 Configuring sensor with 2112x1568-SBGGR10_1X10 [8:07:05.727873482] [9978] DEBUG RkISP1 rkisp1.cpp:694 ISP configured with 2112x1568-SBGGR10_1X10 [8:07:05.728056356] [9978] DEBUG RkISP1 rkisp1.cpp:724 Sensor configured with 2112x1568-SBGGR10_1X10 [8:07:05.728233104] [9978] DEBUG RkISP1 rkisp1.cpp:731 ISP input pad configured with 2112x1568-SBGGR10_1X10 [8:07:05.728340437] [9978] DEBUG RkISP1 rkisp1.cpp:735 Configuring ISP output pad with 2112x1568-YUYV8_2X8 [8:07:05.728448353] [9978] DEBUG RkISP1 rkisp1.cpp:741 ISP output pad configured with 800x600-YUYV8_2X8 [8:07:05.728567352] [9978] DEBUG RkISP1 rkisp1_path.cpp:120 Configured main resizer input pad with 800x600-YUYV8_2X8 [8:07:05.728673809] [9978] DEBUG RkISP1 rkisp1_path.cpp:126 Configuring main resizer output pad with 900x600-YUYV8_2X8 [8:07:05.728785517] [9978] DEBUG RkISP1 rkisp1_path.cpp:144 Configured main resizer output pad with 900x600-YUYV8_1_5X8 3. LIBCAMERA_LOG_LEVELS=0 cam -c 1 --capture=3 -s width=4500,height=3500,pixelformat=NV12,role=video [8:08:02.372820979] [9981] DEBUG Camera camera.cpp:771 streams configuration: (0) 1920x1920-NV12 [8:08:02.373280350] [9981] DEBUG RkISP1 rkisp1_path.cpp:100 Adjusting format from 4500x3500-NV12 to 4416x3312-NV12 [8:08:02.373534098] [9981] DEBUG RkISP1 rkisp1_path.cpp:100 Adjusting format from 4500x3500-NV12 to 1920x1920-NV12 [8:08:02.373767722] [9981] DEBUG RkISP1 rkisp1_path.cpp:100 Adjusting format from 4500x3500-NV12 to 4416x3312-NV12 [8:08:02.374010095] [9981] DEBUG CameraSensor camera_sensor.cpp:408 'ov13850 1-0010': No supported format or size found Camera configuration adjusted [8:08:02.374449634] [9981] DEBUG CameraSensor camera_sensor.cpp:408 'ov13850 1-0010': No supported format or size found [8:08:02.374591091] [9981] INFO Camera camera.cpp:830 configuring streams: (0) 4416x3312-NV12 ... [8:08:02.376301704] [9982] DEBUG RkISP1 rkisp1.cpp:687 Configuring sensor with 4224x3136-0x0000 [8:08:02.376469994] [9982] DEBUG RkISP1 rkisp1.cpp:694 ISP configured with 4032x3024-SRGGB10_1X10 [8:08:02.376576160] [9982] INFO RkISP1 rkisp1.cpp:697 Configured resolution is greater than the maximum resolution for the ISP, trying to re-configure to a smaller valid sensor format [8:08:02.376710909] [9982] DEBUG RkISP1 rkisp1.cpp:716 ISP re-configured with 2112x1568-SBGGR10_1X10 [8:08:02.376839241] [9982] DEBUG RkISP1 rkisp1.cpp:724 Sensor configured with 2112x1568-SBGGR10_1X10 [8:08:02.377001115] [9982] DEBUG RkISP1 rkisp1.cpp:731 ISP input pad configured with 2112x1568-SBGGR10_1X10 [8:08:02.377104364] [9982] DEBUG RkISP1 rkisp1.cpp:735 Configuring ISP output pad with 2112x1568-YUYV8_2X8 [8:08:02.377208780] [9982] DEBUG RkISP1 rkisp1.cpp:741 ISP output pad configured with 800x600-YUYV8_2X8 [8:08:02.377324863] [9982] DEBUG RkISP1 rkisp1_path.cpp:120 Configured main resizer input pad with 800x600-YUYV8_2X8 [8:08:02.377425779] [9982] DEBUG RkISP1 rkisp1_path.cpp:126 Configuring main resizer output pad with 4416x3312-YUYV8_2X8 [8:08:02.377533695] [9982] DEBUG RkISP1 rkisp1_path.cpp:144 Configured main resizer output pad with 4416x3312-YUYV8_1_5X8 4. LIBCAMERA_LOG_LEVELS=0 cam -c 1 --capture=3 -s width=40,height=30,pixelformat=NV12,role=video [8:10:41.140561164] [9989] DEBUG Camera camera.cpp:771 streams configuration: (0) 1920x1920-NV12 [8:10:41.141348367] [9989] INFO Camera camera.cpp:830 configuring streams: (0) 40x30-NV12 ... [8:10:41.143016688] [9990] DEBUG RkISP1 rkisp1.cpp:687 Configuring sensor with 2112x1568-SBGGR10_1X10 [8:10:41.143161062] [9990] DEBUG RkISP1 rkisp1.cpp:694 ISP configured with 2112x1568-SBGGR10_1X10 [8:10:41.143283561] [9990] DEBUG RkISP1 rkisp1.cpp:724 Sensor configured with 2112x1568-SBGGR10_1X10 [8:10:41.143442810] [9990] DEBUG RkISP1 rkisp1.cpp:731 ISP input pad configured with 2112x1568-SBGGR10_1X10 [8:10:41.143544893] [9990] DEBUG RkISP1 rkisp1.cpp:735 Configuring ISP output pad with 2112x1568-YUYV8_2X8 [8:10:41.143644934] [9990] DEBUG RkISP1 rkisp1.cpp:741 ISP output pad configured with 800x600-YUYV8_2X8 [8:10:41.143756058] [9990] DEBUG RkISP1 rkisp1_path.cpp:120 Configured main resizer input pad with 800x600-YUYV8_2X8 [8:10:41.143854640] [9990] DEBUG RkISP1 rkisp1_path.cpp:126 Configuring main resizer output pad with 40x30-YUYV8_2X8 [8:10:41.143960223] [9990] DEBUG RkISP1 rkisp1_path.cpp:144 Configured main resizer output pad with 40x30-YUYV8_1_5X8 5. LIBCAMERA_LOG_LEVELS=0 cam -c 1 --capture=3 -s width=3450,height=2456,pixelformat=NV12,role=video [8:11:40.321895811] [9992] DEBUG Camera camera.cpp:771 streams configuration: (0) 1920x1920-NV12 [8:11:40.322697014] [9992] INFO Camera camera.cpp:830 configuring streams: (0) 3450x2456-NV12 ... [8:11:40.324431543] [9993] DEBUG RkISP1 rkisp1.cpp:687 Configuring sensor with 4224x3136-SBGGR10_1X10 [8:11:40.324579125] [9993] DEBUG RkISP1 rkisp1.cpp:694 ISP configured with 4032x3024-SBGGR10_1X10 [8:11:40.324680041] [9993] INFO RkISP1 rkisp1.cpp:697 Configured resolution is greater than the maximum resolution for the ISP, trying to re-configure to a smaller valid sensor format [8:11:40.324815082] [9993] DEBUG RkISP1 rkisp1.cpp:716 ISP re-configured with 2112x1568-SBGGR10_1X10 [8:11:40.324946039] [9993] DEBUG RkISP1 rkisp1.cpp:724 Sensor configured with 2112x1568-SBGGR10_1X10 [8:11:40.325111121] [9993] DEBUG RkISP1 rkisp1.cpp:731 ISP input pad configured with 2112x1568-SBGGR10_1X10 [8:11:40.325216121] [9993] DEBUG RkISP1 rkisp1.cpp:735 Configuring ISP output pad with 2112x1568-YUYV8_2X8 [8:11:40.325323453] [9993] DEBUG RkISP1 rkisp1.cpp:741 ISP output pad configured with 800x600-YUYV8_2X8 [8:11:40.325441286] [9993] DEBUG RkISP1 rkisp1_path.cpp:120 Configured main resizer input pad with 800x600-YUYV8_2X8 [8:11:40.325547451] [9993] DEBUG RkISP1 rkisp1_path.cpp:126 Configuring main resizer output pad with 3450x2456-YUYV8_2X8 [8:11:40.325660909] [9993] DEBUG RkISP1 rkisp1_path.cpp:144 Configured main resizer output pad with 3450x2456-YUYV8_1_5X8 ---- Sebastian Fricke (1): pipeline: rkisp1: Fix sensor-ISP format mismatch src/libcamera/pipeline/rkisp1/rkisp1.cpp | 36 ++++++++++++++++++++++-- 1 file changed, 33 insertions(+), 3 deletions(-)