[{"id":15626,"web_url":"https://patchwork.libcamera.org/comment/15626/","msgid":"<3346fa50-6ffd-7fef-fed5-c5f5743a4701@ideasonboard.com>","date":"2021-03-12T14:44:52","subject":"Re: [libcamera-devel] [PATCH v3 0/2] lc-compliance: Add a libcamera\n\tcompliance tool","submitter":{"id":4,"url":"https://patchwork.libcamera.org/api/people/4/","name":"Kieran Bingham","email":"kieran.bingham@ideasonboard.com"},"content":"Hi Niklas,\n\nOn 10/03/2021 15:44, Niklas Söderlund wrote:\n> Hello,\n> \n> This series adds a compliance tool to libcamera. It was developed out of\n> necessity while extending and debugging the IPU3 pipeline hander. Each\n> test in the tool so far have at one point triggered fatal issues in one\n> pipeline or another. All known issues are however either fixed or have\n> patches on ML to address them.\n\nThere's more (or at least now) but this tool has helped in flushing out\na few.\n\nThe raspberry pi pipeline handler seems to fail a few of these tests.\nBut all the more reason to integrate this.\n\nI can't say I've done a detailed review but I can say this:\n\n\nFor both:\n\nAcked-by: Kieran Bingham \nTested-by: Kieran Bingham \n\n\nOn IPU3 (with my stability branch):\n120 tests executed, 96 tests passed, 24 tests skipped and 0 tests failed\n\nOn vivid:\n120 tests executed, 96 tests passed, 24 tests skipped and 0 tests failed\n\nOn UVC:\n120 tests executed, 96 tests passed, 24 tests skipped and 0 tests failed\n\n\nOn RPi:\n120 tests executed, 78 tests passed, 12 tests skipped and 30 tests failed\n\nNaush - I had to do a quick fudge fix in the configure path to get this\nto run at all - so it might be interesting for you to try running this.\n\n\n> The tool is simple at the moment and only tests a single stream. As our\n> use-cases grow more complex I hope to find time to extend the tool to\n> cover more areas.\n\nFuture feature request\n --fail-fast\n\nSo if it fails, it halts? (To help see what to fix)\n\n> \n> Niklas Söderlund (2):\n> lc-compliance: Add a libcamera compliance tool\n> lc-compliance: Add test stopping single stream with requests queued\n> \n> src/lc-compliance/main.cpp | 139 ++++++++++++++++++\n> src/lc-compliance/meson.build | 25 ++++\n> src/lc-compliance/results.cpp | 75 ++++++++++\n> src/lc-compliance/results.h | 45 ++++++\n> src/lc-compliance/simple_capture.cpp | 212 +++++++++++++++++++++++++++\n> src/lc-compliance/simple_capture.h | 68 +++++++++\n> src/lc-compliance/single_stream.cpp | 98 +++++++++++++\n> src/lc-compliance/tests.h | 16 ++\n> src/meson.build | 2 +\n> 9 files changed, 680 insertions(+)\n> create mode 100644 src/lc-compliance/main.cpp\n> create mode 100644 src/lc-compliance/meson.build\n> create mode 100644 src/lc-compliance/results.cpp\n> create mode 100644 src/lc-compliance/results.h\n> create mode 100644 src/lc-compliance/simple_capture.cpp\n> create mode 100644 src/lc-compliance/simple_capture.h\n> create mode 100644 src/lc-compliance/single_stream.cpp\n> create mode 100644 src/lc-compliance/tests.h\n>","headers":{"Return-Path":"","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id BA514BD80C\n\tfor ;\n\tFri, 12 Mar 2021 14:44:57 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id 29D1D68C6B;\n\tFri, 12 Mar 2021 15:44:57 +0100 (CET)","from perceval.ideasonboard.com (perceval.ideasonboard.com\n\t[IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id 0DF3668C68\n\tfor ;\n\tFri, 12 Mar 2021 15:44:56 +0100 (CET)","from [192.168.0.20]\n\t(cpc89244-aztw30-2-0-cust3082.18-1.cable.virginm.net [86.31.172.11])\n\tby perceval.ideasonboard.com (Postfix) with ESMTPSA id 788FDA2A;\n\tFri, 12 Mar 2021 15:44:55 +0100 (CET)"],"Authentication-Results":"lancelot.ideasonboard.com;\n\tdkim=fail reason=\"signature verification failed\" (1024-bit key;\n\tunprotected) header.d=ideasonboard.com header.i=@ideasonboard.com\n\theader.b=\"BIzERJnP\"; dkim-atps=neutral","DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com;\n\ts=mail; t=1615560295;\n\tbh=dpmhA6FMDcZOxQaMyVKjF1T5M56q6+K7OFEfdnVLIVM=;\n\th=Reply-To:Subject:To:References:From:Date:In-Reply-To:From;\n\tb=BIzERJnP/0xgF10KilF3xg8ijczW10vi1kYz4B0p58Vr3GDptIX94+bIseKnTd6/D\n\tiLbRUYsdpj6r6NsFROiW3unV7WtbnOwqrLH4Crk5kqM3WlohUVN0uz+0zRtiec1Wch\n\tcMfDYVdFh8CRE7xL81Tc+rdXGoiCfpiHUC337Hfw=","To":"=?utf-8?q?Niklas_S=C3=B6derlund?= ,\n\tlibcamera-devel@lists.libcamera.org,\n\tNaushir Patuck ","References":"<20210310154414.3560115-1-niklas.soderlund@ragnatech.se>","From":"Kieran Bingham ","Autocrypt":"addr=kieran.bingham@ideasonboard.com; keydata=\n\tmQINBFYE/WYBEACs1PwjMD9rgCu1hlIiUA1AXR4rv2v+BCLUq//vrX5S5bjzxKAryRf0uHat\n\tV/zwz6hiDrZuHUACDB7X8OaQcwhLaVlq6byfoBr25+hbZG7G3+5EUl9cQ7dQEdvNj6V6y/SC\n\trRanWfelwQThCHckbobWiQJfK9n7rYNcPMq9B8e9F020LFH7Kj6YmO95ewJGgLm+idg1Kb3C\n\tpotzWkXc1xmPzcQ1fvQMOfMwdS+4SNw4rY9f07Xb2K99rjMwZVDgESKIzhsDB5GY465sCsiQ\n\tcSAZRxqE49RTBq2+EQsbrQpIc8XiffAB8qexh5/QPzCmR4kJgCGeHIXBtgRj+nIkCJPZvZtf\n\tKr2EAbc6tgg6DkAEHJb+1okosV09+0+TXywYvtEop/WUOWQ+zo+Y/OBd+8Ptgt1pDRyOBzL8\n\tRXa8ZqRf0Mwg75D+dKntZeJHzPRJyrlfQokngAAs4PaFt6UfS+ypMAF37T6CeDArQC41V3ko\n\tlPn1yMsVD0p+6i3DPvA/GPIksDC4owjnzVX9kM8Zc5Cx+XoAN0w5Eqo4t6qEVbuettxx55gq\n\t8K8FieAjgjMSxngo/HST8TpFeqI5nVeq0/lqtBRQKumuIqDg+Bkr4L1V/PSB6XgQcOdhtd36\n\tOe9X9dXB8YSNt7VjOcO7BTmFn/Z8r92mSAfHXpb07YJWJosQOQARAQABtDBLaWVyYW4gQmlu\n\tZ2hhbSA8a2llcmFuLmJpbmdoYW1AaWRlYXNvbmJvYXJkLmNvbT6JAlcEEwEKAEECGwMFCwkI\n\tBwIGFQgJCgsCBBYCAwECHgECF4ACGQEWIQSQLdeYP70o/eNy1HqhHkZyEKRh/QUCXWTtygUJ\n\tCyJXZAAKCRChHkZyEKRh/f8dEACTDsbLN2nioNZMwyLuQRUAFcXNolDX48xcUXsWS2QjxaPm\n\tVsJx8Uy8aYkS85mdPBh0C83OovQR/OVbr8AxhGvYqBs3nQvbWuTl/+4od7DfK2VZOoKBAu5S\n\tQK2FYuUcikDqYcFWJ8DQnubxfE8dvzojHEkXw0sA4igINHDDFX3HJGZtLio+WpEFQtCbfTAG\n\tYZslasz1YZRbwEdSsmO3/kqy5eMnczlm8a21A3fKUo3g8oAZEFM+f4DUNzqIltg31OAB/kZS\n\tenKZQ/SWC8PmLg/ZXBrReYakxXtkP6w3FwMlzOlhGxqhIRNiAJfXJBaRhuUWzPOpEDE9q5YJ\n\tBmqQL2WJm1VSNNVxbXJHpaWMH1sA2R00vmvRrPXGwyIO0IPYeUYQa3gsy6k+En/aMQJd27dp\n\taScf9am9PFICPY5T4ppneeJLif2lyLojo0mcHOV+uyrds9XkLpp14GfTkeKPdPMrLLTsHRfH\n\tfA4I4OBpRrEPiGIZB/0im98MkGY/Mu6qxeZmYLCcgD6qz4idOvfgVOrNh+aA8HzIVR+RMW8H\n\tQGBN9f0E3kfwxuhl3omo6V7lDw8XOdmuWZNC9zPq1UfryVHANYbLGz9KJ4Aw6M+OgBC2JpkD\n\thXMdHUkC+d20dwXrwHTlrJi1YNp6rBc+xald3wsUPOZ5z8moTHUX/uPA/qhGsbkCDQRWBP1m\n\tARAAzijkb+Sau4hAncr1JjOY+KyFEdUNxRy+hqTJdJfaYihxyaj0Ee0P0zEi35CbE6lgU0Uz\n\ttih9fiUbSV3wfsWqg1Ut3/5rTKu7kLFp15kF7eqvV4uezXRD3Qu4yjv/rMmEJbbD4cTvGCYI\n\td6MDC417f7vK3hCbCVIZSp3GXxyC1LU+UQr3fFcOyCwmP9vDUR9JV0BSqHHxRDdpUXE26Dk6\n\tmhf0V1YkspE5St814ETXpEus2urZE5yJIUROlWPIL+hm3NEWfAP06vsQUyLvr/GtbOT79vXl\n\tEn1aulcYyu20dRRxhkQ6iILaURcxIAVJJKPi8dsoMnS8pB0QW12AHWuirPF0g6DiuUfPmrA5\n\tPKe56IGlpkjc8cO51lIxHkWTpCMWigRdPDexKX+Sb+W9QWK/0JjIc4t3KBaiG8O4yRX8ml2R\n\t+rxfAVKM6V769P/hWoRGdgUMgYHFpHGSgEt80OKK5HeUPy2cngDUXzwrqiM5Sz6Od0qw5pCk\n\tNlXqI0W/who0iSVM+8+RmyY0OEkxEcci7rRLsGnM15B5PjLJjh1f2ULYkv8s4SnDwMZ/kE04\n\t/UqCMK/KnX8pwXEMCjz0h6qWNpGwJ0/tYIgQJZh6bqkvBrDogAvuhf60Sogw+mH8b+PBlx1L\n\toeTK396wc+4c3BfiC6pNtUS5GpsPMMjYMk7kVvEAEQEAAYkCPAQYAQoAJgIbDBYhBJAt15g/\n\tvSj943LUeqEeRnIQpGH9BQJdizzIBQkLSKZiAAoJEKEeRnIQpGH9eYgQAJpjaWNgqNOnMTmD\n\tMJggbwjIotypzIXfhHNCeTkG7+qCDlSaBPclcPGYrTwCt0YWPU2TgGgJrVhYT20ierN8LUvj\n\t6qOPTd+Uk7NFzL65qkh80ZKNBFddx1AabQpSVQKbdcLb8OFs85kuSvFdgqZwgxA1vl4TFhNz\n\tPZ79NAmXLackAx3sOVFhk4WQaKRshCB7cSl+RIng5S/ThOBlwNlcKG7j7W2MC06BlTbdEkUp\n\tECzuuRBv8wX4OQl+hbWbB/VKIx5HKlLu1eypen/5lNVzSqMMIYkkZcjV2SWQyUGxSwq0O/sx\n\tS0A8/atCHUXOboUsn54qdxrVDaK+6jIAuo8JiRWctP16KjzUM7MO0/+4zllM8EY57rXrj48j\n\tsbEYX0YQnzaj+jO6kJtoZsIaYR7rMMq9aUAjyiaEZpmP1qF/2sYenDx0Fg2BSlLvLvXM0vU8\n\tpQk3kgDu7kb/7PRYrZvBsr21EIQoIjXbZxDz/o7z95frkP71EaICttZ6k9q5oxxA5WC6sTXc\n\tMW8zs8avFNuA9VpXt0YupJd2ijtZy2mpZNG02fFVXhIn4G807G7+9mhuC4XG5rKlBBUXTvPU\n\tAfYnB4JBDLmLzBFavQfvonSfbitgXwCG3vS+9HEwAjU30Bar1PEOmIbiAoMzuKeRm2LVpmq4\n\tWZw01QYHU/GUV/zHJSFk","Organization":"Ideas on Board","Message-ID":"<3346fa50-6ffd-7fef-fed5-c5f5743a4701@ideasonboard.com>","Date":"Fri, 12 Mar 2021 14:44:52 +0000","User-Agent":"Mozilla/5.0 (X11; Linux x86_64; rv:68.0) Gecko/20100101\n\tThunderbird/68.10.0","MIME-Version":"1.0","In-Reply-To":"<20210310154414.3560115-1-niklas.soderlund@ragnatech.se>","Content-Language":"en-GB","Subject":"Re: [libcamera-devel] [PATCH v3 0/2] lc-compliance: Add a libcamera\n\tcompliance tool","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Reply-To":"kieran.bingham@ideasonboard.com","Content-Type":"text/plain; charset=\"utf-8\"","Content-Transfer-Encoding":"base64","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" "}},{"id":15629,"web_url":"https://patchwork.libcamera.org/comment/15629/","msgid":"","date":"2021-03-12T17:02:48","subject":"Re: [libcamera-devel] [PATCH v3 0/2] lc-compliance: Add a libcamera\n\tcompliance tool","submitter":{"id":34,"url":"https://patchwork.libcamera.org/api/people/34/","name":"Naushir Patuck","email":"naush@raspberrypi.com"},"content":"Hi Kieran and Niklas,\n\nOn Fri, 12 Mar 2021 at 14:44, Kieran Bingham <\nkieran.bingham@ideasonboard.com> wrote:\n\n> Hi Niklas,\n>\n> On 10/03/2021 15:44, Niklas Söderlund wrote:\n> > Hello,\n> >\n> > This series adds a compliance tool to libcamera. It was developed out of\n> > necessity while extending and debugging the IPU3 pipeline hander. Each\n> > test in the tool so far have at one point triggered fatal issues in one\n> > pipeline or another. All known issues are however either fixed or have\n> > patches on ML to address them.\n>\n> There's more (or at least now) but this tool has helped in flushing out\n> a few.\n>\n> The raspberry pi pipeline handler seems to fail a few of these tests.\n> But all the more reason to integrate this.\n>\n> I can't say I've done a detailed review but I can say this:\n>\n>\n> For both:\n>\n> Acked-by: Kieran Bingham \n> Tested-by: Kieran Bingham \n>\n>\n> On IPU3 (with my stability branch):\n> 120 tests executed, 96 tests passed, 24 tests skipped and 0 tests failed\n>\n> On vivid:\n> 120 tests executed, 96 tests passed, 24 tests skipped and 0 tests failed\n>\n> On UVC:\n> 120 tests executed, 96 tests passed, 24 tests skipped and 0 tests failed\n>\n>\n> On RPi:\n> 120 tests executed, 78 tests passed, 12 tests skipped and 30 tests failed\n>\n> Naush - I had to do a quick fudge fix in the configure path to get this\n> to run at all - so it might be interesting for you to try running this.\n>\n\nThanks for the heads-up. This is indeed going to be a very handy tool for\neveryone.\n\nRegarding the results, I think I see the cause of the failures. It is due\nto\nthe RPi pipeline handler not correctly handling only a RAW stream (i.e. no\nISP output). Our entire flow depends on it, including running the IPA for\nmetering stats. The solution here would be to allow our ISP to run so that\nit only generates stats output and no image output. However, this is not\na trivial change, and since we have never run this way before, my\nconfidence levels are not all that high.\n\nA simpler fix would be to get the ISP to generate a small (say 320x240)\noutput image for internal use only so the pipeline behaves correctly.\nHaving\na quick go at this change seems to fix all failures.\n\nFor the skipped tests, I think we did have a brief discussion with Niklas\nsome\ntime back when the tool was first introduced. We advertise a particular\nbuffer count for each SteramRole in generateConfiguration(), and I think the\ntool skips the tests with number of Requests lower than that count.\nHowever,\nthis number is merely a guide and our pipeline handler ought to work\ncorrectly with\nonly a single Request queued. We did not reach a conclusion on what was\nthe right\nway to approach this back then, but I think we said that I should not set\nour buffer\ncounts down to one. Niklas should we consider another number besides buffer\ncount\nreturned out of generateConfiguration()?\n\nThanks,\nNaush\n\n\n>\n> > The tool is simple at the moment and only tests a single stream. As our\n> > use-cases grow more complex I hope to find time to extend the tool to\n> > cover more areas.\n>\n> Future feature request\n> --fail-fast\n>\n> So if it fails, it halts? (To help see what to fix)\n>\n> >\n> > Niklas Söderlund (2):\n> > lc-compliance: Add a libcamera compliance tool\n> > lc-compliance: Add test stopping single stream with requests queued\n> >\n> > src/lc-compliance/main.cpp | 139 ++++++++++++++++++\n> > src/lc-compliance/meson.build | 25 ++++\n> > src/lc-compliance/results.cpp | 75 ++++++++++\n> > src/lc-compliance/results.h | 45 ++++++\n> > src/lc-compliance/simple_capture.cpp | 212 +++++++++++++++++++++++++++\n> > src/lc-compliance/simple_capture.h | 68 +++++++++\n> > src/lc-compliance/single_stream.cpp | 98 +++++++++++++\n> > src/lc-compliance/tests.h | 16 ++\n> > src/meson.build | 2 +\n> > 9 files changed, 680 insertions(+)\n> > create mode 100644 src/lc-compliance/main.cpp\n> > create mode 100644 src/lc-compliance/meson.build\n> > create mode 100644 src/lc-compliance/results.cpp\n> > create mode 100644 src/lc-compliance/results.h\n> > create mode 100644 src/lc-compliance/simple_capture.cpp\n> > create mode 100644 src/lc-compliance/simple_capture.h\n> > create mode 100644 src/lc-compliance/single_stream.cpp\n> > create mode 100644 src/lc-compliance/tests.h\n> >\n>\n> --\n> Regards\n> --\n> Kieran\n>","headers":{"Return-Path":"","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id 89703BD1F1\n\tfor ;\n\tFri, 12 Mar 2021 17:03:07 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id D585768C69;\n\tFri, 12 Mar 2021 18:03:06 +0100 (CET)","from mail-lf1-x130.google.com (mail-lf1-x130.google.com\n\t[IPv6:2a00:1450:4864:20::130])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id 6113768C66\n\tfor ;\n\tFri, 12 Mar 2021 18:03:05 +0100 (CET)","by mail-lf1-x130.google.com with SMTP id u4so46892923lfs.0\n\tfor ;\n\tFri, 12 Mar 2021 09:03:05 -0800 (PST)"],"Authentication-Results":"lancelot.ideasonboard.com;\n\tdkim=fail reason=\"signature verification failed\" (2048-bit key;\n\tunprotected) header.d=raspberrypi.com header.i=@raspberrypi.com\n\theader.b=\"lbpOOcCD\"; dkim-atps=neutral","DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed;\n\td=raspberrypi.com; s=google;\n\th=mime-version:references:in-reply-to:from:date:message-id:subject:to\n\t:cc; bh=9Hg8UppcK9lNTDU+t7HkSVlrx4Afbp+b3/N/sMOlzY8=;\n\tb=lbpOOcCDlLwKeBgjDT7p/hvaTMY5FWU0pEVxZsCVEsoGaDD2bD4i9tok8vJ97ALFjF\n\tbkzZGUqWgcnz3W4cBoXeiY+4Y+PqExRbyS2S+C8wQCsUngRbUk4Wf92glibIAbC/Suue\n\ttma4/mLGlA2yWxqhgzCHAEHv2TN6GEpaWyOglJt2RH2lCK4CAa4GJWRkJY6F2o97zV0r\n\tsfrYYA8EVXDwNsydPREpL7Zr71LXPoWI6HH1OVlWb7xJm8d0Ek7Qcq83HEHNjniHDy4B\n\tdE6/JqELn2aM3qNFq5G/mF7O2LcTktehJSTKuTP2WASnxUj1K0tyiTGznfawoUPnY8P5\n\tva3g==","X-Google-DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/relaxed;\n\td=1e100.net; s=20161025;\n\th=x-gm-message-state:mime-version:references:in-reply-to:from:date\n\t:message-id:subject:to:cc;\n\tbh=9Hg8UppcK9lNTDU+t7HkSVlrx4Afbp+b3/N/sMOlzY8=;\n\tb=SCPbh1L2nVoC6H+QnOlw1xh9pCmLa9kwSOchW6gy80FS0NGLNHC2dL9XJv6RwQPxsU\n\t4iDUVrIBy9sqqQK0eCIAMGj4WNrPFKEvAuiSDCBpgsJAuoBaQ5Jq8cs1KXqvLmb3J6sF\n\tnzu0CQ33YmGMOp3ieF/g7pnMo5318QmPlpCbXCaP16gpNSeYrfUENRD4oZ9wL8gORLv8\n\t/s3Yc2C7ZiqxXFKFFuUJKgD9wEwhHFYiePa12wnhG5dA62cFCpInQl+WM5tXKmJ9NmXR\n\tQPs6bjYVoDyw6qYr7rnXlLfMaqGzUF8CMNi5o/TT/4I4g27LlMpTeVRn0T8uT1kmUL7o\n\tqrSg==","X-Gm-Message-State":"AOAM531Nlg0aOLxZ3prSBNHylJVNl8IVUE1mBPo4eFwyYDyrUp2QIn3/\n\tPFZRTFV2BlXIJbD0ZY1IRZY6B3jEpZLDLVZdGXE4C8isKyjsng==","X-Google-Smtp-Source":"ABdhPJz6I/BSfKYIHfk55RNzQ18i0/MsCVmyh50NkhPj+E7a7PdINd1pU3OjQY7oj4wdu2rv11BrNk8YNf9hWzeFhXQ=","X-Received":"by 2002:a05:6512:210b:: with SMTP id\n\tq11mr107339lfr.133.1615568584718; \n\tFri, 12 Mar 2021 09:03:04 -0800 (PST)","MIME-Version":"1.0","References":"<20210310154414.3560115-1-niklas.soderlund@ragnatech.se>\n\t<3346fa50-6ffd-7fef-fed5-c5f5743a4701@ideasonboard.com>","In-Reply-To":"<3346fa50-6ffd-7fef-fed5-c5f5743a4701@ideasonboard.com>","From":"Naushir Patuck ","Date":"Fri, 12 Mar 2021 17:02:48 +0000","Message-ID":"","To":"Kieran Bingham ","Subject":"Re: [libcamera-devel] [PATCH v3 0/2] lc-compliance: Add a libcamera\n\tcompliance tool","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Cc":"libcamera devel ","Content-Type":"multipart/mixed;\n\tboundary=\"===============3125639844951855138==\"","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" "}},{"id":15680,"web_url":"https://patchwork.libcamera.org/comment/15680/","msgid":"","date":"2021-03-14T20:59:04","subject":"Re: [libcamera-devel] [PATCH v3 0/2] lc-compliance: Add a libcamera\n\tcompliance tool","submitter":{"id":2,"url":"https://patchwork.libcamera.org/api/people/2/","name":"Laurent Pinchart","email":"laurent.pinchart@ideasonboard.com"},"content":"Hi Niklas,\n\nOn Wed, Mar 10, 2021 at 04:44:12PM +0100, Niklas Söderlund wrote:\n> Hello,\n> \n> This series adds a compliance tool to libcamera. It was developed out of\n> necessity while extending and debugging the IPU3 pipeline hander. Each\n> test in the tool so far have at one point triggered fatal issues in one\n> pipeline or another. All known issues are however either fixed or have\n> patches on ML to address them.\n> \n> The tool is simple at the moment and only tests a single stream. As our\n> use-cases grow more complex I hope to find time to extend the tool to\n> cover more areas.\n\nLots of work remaining on the test front :-) This is a good starting\npoint. I've reviewed both patches, with lots of comments. Please address\nthe ones you think should be addressed immediately, and record the other\nones in \\todo comments. I see no blocking issue, so I expect to ack v4.\n\n> Niklas Söderlund (2):\n> lc-compliance: Add a libcamera compliance tool\n> lc-compliance: Add test stopping single stream with requests queued\n> \n> src/lc-compliance/main.cpp | 139 ++++++++++++++++++\n> src/lc-compliance/meson.build | 25 ++++\n> src/lc-compliance/results.cpp | 75 ++++++++++\n> src/lc-compliance/results.h | 45 ++++++\n> src/lc-compliance/simple_capture.cpp | 212 +++++++++++++++++++++++++++\n> src/lc-compliance/simple_capture.h | 68 +++++++++\n> src/lc-compliance/single_stream.cpp | 98 +++++++++++++\n> src/lc-compliance/tests.h | 16 ++\n> src/meson.build | 2 +\n> 9 files changed, 680 insertions(+)\n> create mode 100644 src/lc-compliance/main.cpp\n> create mode 100644 src/lc-compliance/meson.build\n> create mode 100644 src/lc-compliance/results.cpp\n> create mode 100644 src/lc-compliance/results.h\n> create mode 100644 src/lc-compliance/simple_capture.cpp\n> create mode 100644 src/lc-compliance/simple_capture.h\n> create mode 100644 src/lc-compliance/single_stream.cpp\n> create mode 100644 src/lc-compliance/tests.h","headers":{"Return-Path":"","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id 851D6BD80C\n\tfor ;\n\tSun, 14 Mar 2021 20:59:41 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id 405F360850;\n\tSun, 14 Mar 2021 21:59:41 +0100 (CET)","from perceval.ideasonboard.com (perceval.ideasonboard.com\n\t[213.167.242.64])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id 80BD0602E3\n\tfor ;\n\tSun, 14 Mar 2021 21:59:40 +0100 (CET)","from pendragon.ideasonboard.com (62-78-145-57.bb.dnainternet.fi\n\t[62.78.145.57])\n\tby perceval.ideasonboard.com (Postfix) with ESMTPSA id 127733E6;\n\tSun, 14 Mar 2021 21:59:40 +0100 (CET)"],"Authentication-Results":"lancelot.ideasonboard.com;\n\tdkim=fail reason=\"signature verification failed\" (1024-bit key;\n\tunprotected) header.d=ideasonboard.com header.i=@ideasonboard.com\n\theader.b=\"PiMqVmK4\"; dkim-atps=neutral","DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com;\n\ts=mail; t=1615755580;\n\tbh=txbhb5mP1CGzzIl7YweGj7Z1P8Di9JFNfpH9Frw+eBg=;\n\th=Date:From:To:Cc:Subject:References:In-Reply-To:From;\n\tb=PiMqVmK4NnT2/+uBgE2+m87UzNIsAF/x6nCeYPllN4V5ZQ4PFGRMcSW9OlkYv7UYI\n\tknrk6HIFlFkDpnVcVE11eJc3rQkAPDRnjAOSHrauV5bd+AJY5QKLT2l79q7VHwmmTy\n\tjFDGCfHZ4QEQwxk34ischDy8PstzcDjZlKP1+drM=","Date":"Sun, 14 Mar 2021 22:59:04 +0200","From":"Laurent Pinchart ","To":"Niklas =?utf-8?q?S=C3=B6derlund?= ","Message-ID":"","References":"<20210310154414.3560115-1-niklas.soderlund@ragnatech.se>","MIME-Version":"1.0","Content-Disposition":"inline","In-Reply-To":"<20210310154414.3560115-1-niklas.soderlund@ragnatech.se>","Subject":"Re: [libcamera-devel] [PATCH v3 0/2] lc-compliance: Add a libcamera\n\tcompliance tool","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Cc":"libcamera-devel@lists.libcamera.org","Content-Type":"text/plain; charset=\"utf-8\"","Content-Transfer-Encoding":"base64","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" "}}]