From patchwork Thu Apr 22 21:06:07 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?b?TsOtY29sYXMgRi4gUi4gQS4gUHJhZG8=?= X-Patchwork-Id: 12082 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id F379ABDB1A for ; Thu, 22 Apr 2021 21:06:51 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 324436885C; Thu, 22 Apr 2021 23:06:51 +0200 (CEST) Received: from bhuna.collabora.co.uk (bhuna.collabora.co.uk [IPv6:2a00:1098:0:82:1000:25:2eeb:e3e3]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id B8EDB60516 for ; Thu, 22 Apr 2021 23:06:49 +0200 (CEST) Received: from [127.0.0.1] (localhost [127.0.0.1]) (Authenticated sender: nfraprado) with ESMTPSA id B8B7D1F43738 From: =?utf-8?b?TsOtY29sYXMgRi4gUi4gQS4gUHJhZG8=?= To: libcamera-devel@lists.libcamera.org Date: Thu, 22 Apr 2021 18:06:07 -0300 Message-Id: <20210422210609.425987-1-nfraprado@collabora.com> X-Mailer: git-send-email 2.31.1 MIME-Version: 1.0 Subject: [libcamera-devel] [RFC PATCH 0/2] lc-compliance: Refactor using Googletest X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kernel@collabora.com, =?utf-8?q?Andr=C3=A9_Almeida?= Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" This is another prototype of the refactoring of lc-compliance, this time using Googletest as the framework. Patch 1 is just to simplify cleanup and is the same patch used in the previous refactoring series using Catch2 [1]. Patch 2 does the actual refactor. The biggest advantage this has over the previous one using Catch2, is that Googletest has a GTEST_SKIP() macro which skips the current test during runtime, which is essential for lc-compliance. One drawback of Gtest is the need to use different assertion macros for each type of comparison, like ASSERT_LE(a, b), instead of a single one used in Catch2, ASSERT(a <= b), which makes the code a little less clear, but is not a show stopper. The parameters for Gtest have yet to be wired up, but can currently be passed after double dashes, like lc-compliance -c '' -- --gtest_list_tests to list all current tests defined. The parameter --gtest_filter can be used to filter the individual tests or test suites to run. It uses string matching and globing for the test names so it's easy to run a whole test suite MyTestSuite with --gtest_list_tests='*MyTestSuite*'. Four INSTANTIATE_TEST_SUITE_P(), one for each stream role, were used in the code to make the StreamRole part of the test name string, making it possible to filter only tests of a single StreamRole to run. In order to not have to define each new parametrized test suite four times, a macro could be used for the substitution. This is a snippet of a successful run: [==========] Running 120 tests from 4 test suites. [----------] Global test environment set-up. [----------] 30 tests from Raw/FixedRequestsTest [ RUN ] Raw/FixedRequestsTest.BalancedSingleCycle/0 [79:31:45.624016307] [421717] INFO Camera camera.cpp:905 configuring streams: (0) 1280x720-MJPEG Camera needs 4 requests, can't test only 1 [ SKIPPED ] Raw/FixedRequestsTest.BalancedSingleCycle/0 (64 ms) [...] [ RUN ] Viewfinder/FixedRequestsTest.Unbalanced/9 [79:34:38.653213305] [421717] INFO Camera camera.cpp:905 configuring streams: (0) 1280x720-MJPEG [ OK ] Viewfinder/FixedRequestsTest.Unbalanced/9 (3146 ms) [----------] 30 tests from Viewfinder/FixedRequestsTest (43964 ms total) [----------] Global test environment tear-down [==========] 120 tests from 4 test suites ran. (176177 ms total) [ PASSED ] 96 tests. [ SKIPPED ] 24 tests, listed below: [ SKIPPED ] Raw/FixedRequestsTest.BalancedSingleCycle/0 [ SKIPPED ] Raw/FixedRequestsTest.BalancedSingleCycle/1 [...] And how a failure looks like: [ RUN ] Raw/FixedRequestsTest.BalancedSingleCycle/3 [79:46:37.536423187] [424054] INFO Camera camera.cpp:905 configuring streams: (0) 1280x720-MJPEG ../src/lc-compliance/simple_capture.cpp:110: Failure Expected equality of these values: captureCount_ Which is: 5 captureLimit_ Which is: 4 [ FAILED ] Raw/FixedRequestsTest.BalancedSingleCycle/3, where GetParam() = (0, 5) (318 ms) The current complete list of tests is shown below: Raw/FixedRequestsTest. BalancedSingleCycle/0 # GetParam() = (0, 1) BalancedSingleCycle/1 # GetParam() = (0, 2) BalancedSingleCycle/2 # GetParam() = (0, 3) BalancedSingleCycle/3 # GetParam() = (0, 5) BalancedSingleCycle/4 # GetParam() = (0, 8) BalancedSingleCycle/5 # GetParam() = (0, 13) BalancedSingleCycle/6 # GetParam() = (0, 21) BalancedSingleCycle/7 # GetParam() = (0, 34) BalancedSingleCycle/8 # GetParam() = (0, 55) BalancedSingleCycle/9 # GetParam() = (0, 89) BalancedMultiCycle/0 # GetParam() = (0, 1) BalancedMultiCycle/1 # GetParam() = (0, 2) BalancedMultiCycle/2 # GetParam() = (0, 3) BalancedMultiCycle/3 # GetParam() = (0, 5) BalancedMultiCycle/4 # GetParam() = (0, 8) BalancedMultiCycle/5 # GetParam() = (0, 13) BalancedMultiCycle/6 # GetParam() = (0, 21) BalancedMultiCycle/7 # GetParam() = (0, 34) BalancedMultiCycle/8 # GetParam() = (0, 55) BalancedMultiCycle/9 # GetParam() = (0, 89) Unbalanced/0 # GetParam() = (0, 1) Unbalanced/1 # GetParam() = (0, 2) Unbalanced/2 # GetParam() = (0, 3) Unbalanced/3 # GetParam() = (0, 5) Unbalanced/4 # GetParam() = (0, 8) Unbalanced/5 # GetParam() = (0, 13) Unbalanced/6 # GetParam() = (0, 21) Unbalanced/7 # GetParam() = (0, 34) Unbalanced/8 # GetParam() = (0, 55) Unbalanced/9 # GetParam() = (0, 89) StillCapture/FixedRequestsTest. BalancedSingleCycle/0 # GetParam() = (1, 1) BalancedSingleCycle/1 # GetParam() = (1, 2) BalancedSingleCycle/2 # GetParam() = (1, 3) BalancedSingleCycle/3 # GetParam() = (1, 5) BalancedSingleCycle/4 # GetParam() = (1, 8) BalancedSingleCycle/5 # GetParam() = (1, 13) BalancedSingleCycle/6 # GetParam() = (1, 21) BalancedSingleCycle/7 # GetParam() = (1, 34) BalancedSingleCycle/8 # GetParam() = (1, 55) BalancedSingleCycle/9 # GetParam() = (1, 89) BalancedMultiCycle/0 # GetParam() = (1, 1) BalancedMultiCycle/1 # GetParam() = (1, 2) BalancedMultiCycle/2 # GetParam() = (1, 3) BalancedMultiCycle/3 # GetParam() = (1, 5) BalancedMultiCycle/4 # GetParam() = (1, 8) BalancedMultiCycle/5 # GetParam() = (1, 13) BalancedMultiCycle/6 # GetParam() = (1, 21) BalancedMultiCycle/7 # GetParam() = (1, 34) BalancedMultiCycle/8 # GetParam() = (1, 55) BalancedMultiCycle/9 # GetParam() = (1, 89) Unbalanced/0 # GetParam() = (1, 1) Unbalanced/1 # GetParam() = (1, 2) Unbalanced/2 # GetParam() = (1, 3) Unbalanced/3 # GetParam() = (1, 5) Unbalanced/4 # GetParam() = (1, 8) Unbalanced/5 # GetParam() = (1, 13) Unbalanced/6 # GetParam() = (1, 21) Unbalanced/7 # GetParam() = (1, 34) Unbalanced/8 # GetParam() = (1, 55) Unbalanced/9 # GetParam() = (1, 89) VideoRecording/FixedRequestsTest. BalancedSingleCycle/0 # GetParam() = (2, 1) BalancedSingleCycle/1 # GetParam() = (2, 2) BalancedSingleCycle/2 # GetParam() = (2, 3) BalancedSingleCycle/3 # GetParam() = (2, 5) BalancedSingleCycle/4 # GetParam() = (2, 8) BalancedSingleCycle/5 # GetParam() = (2, 13) BalancedSingleCycle/6 # GetParam() = (2, 21) BalancedSingleCycle/7 # GetParam() = (2, 34) BalancedSingleCycle/8 # GetParam() = (2, 55) BalancedSingleCycle/9 # GetParam() = (2, 89) BalancedMultiCycle/0 # GetParam() = (2, 1) BalancedMultiCycle/1 # GetParam() = (2, 2) BalancedMultiCycle/2 # GetParam() = (2, 3) BalancedMultiCycle/3 # GetParam() = (2, 5) BalancedMultiCycle/4 # GetParam() = (2, 8) BalancedMultiCycle/5 # GetParam() = (2, 13) BalancedMultiCycle/6 # GetParam() = (2, 21) BalancedMultiCycle/7 # GetParam() = (2, 34) BalancedMultiCycle/8 # GetParam() = (2, 55) BalancedMultiCycle/9 # GetParam() = (2, 89) Unbalanced/0 # GetParam() = (2, 1) Unbalanced/1 # GetParam() = (2, 2) Unbalanced/2 # GetParam() = (2, 3) Unbalanced/3 # GetParam() = (2, 5) Unbalanced/4 # GetParam() = (2, 8) Unbalanced/5 # GetParam() = (2, 13) Unbalanced/6 # GetParam() = (2, 21) Unbalanced/7 # GetParam() = (2, 34) Unbalanced/8 # GetParam() = (2, 55) Unbalanced/9 # GetParam() = (2, 89) Viewfinder/FixedRequestsTest. BalancedSingleCycle/0 # GetParam() = (3, 1) BalancedSingleCycle/1 # GetParam() = (3, 2) BalancedSingleCycle/2 # GetParam() = (3, 3) BalancedSingleCycle/3 # GetParam() = (3, 5) BalancedSingleCycle/4 # GetParam() = (3, 8) BalancedSingleCycle/5 # GetParam() = (3, 13) BalancedSingleCycle/6 # GetParam() = (3, 21) BalancedSingleCycle/7 # GetParam() = (3, 34) BalancedSingleCycle/8 # GetParam() = (3, 55) BalancedSingleCycle/9 # GetParam() = (3, 89) BalancedMultiCycle/0 # GetParam() = (3, 1) BalancedMultiCycle/1 # GetParam() = (3, 2) BalancedMultiCycle/2 # GetParam() = (3, 3) BalancedMultiCycle/3 # GetParam() = (3, 5) BalancedMultiCycle/4 # GetParam() = (3, 8) BalancedMultiCycle/5 # GetParam() = (3, 13) BalancedMultiCycle/6 # GetParam() = (3, 21) BalancedMultiCycle/7 # GetParam() = (3, 34) BalancedMultiCycle/8 # GetParam() = (3, 55) BalancedMultiCycle/9 # GetParam() = (3, 89) Unbalanced/0 # GetParam() = (3, 1) Unbalanced/1 # GetParam() = (3, 2) Unbalanced/2 # GetParam() = (3, 3) Unbalanced/3 # GetParam() = (3, 5) Unbalanced/4 # GetParam() = (3, 8) Unbalanced/5 # GetParam() = (3, 13) Unbalanced/6 # GetParam() = (3, 21) Unbalanced/7 # GetParam() = (3, 34) Unbalanced/8 # GetParam() = (3, 55) Unbalanced/9 # GetParam() = (3, 89) [1] https://lists.libcamera.org/pipermail/libcamera-devel/2021-April/019358.html Nícolas F. R. A. Prado (2): lc-compliance: Make SimpleCapture::stop() idempotent lc-compliance: Refactor using Googletest src/lc-compliance/main.cpp | 60 +++++------ src/lc-compliance/meson.build | 3 + src/lc-compliance/simple_capture.cpp | 85 ++++++---------- src/lc-compliance/simple_capture.h | 8 +- src/lc-compliance/single_stream.cpp | 142 ++++++++++++++------------- 5 files changed, 137 insertions(+), 161 deletions(-)