From patchwork Fri May 14 13:16:48 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?b?TsOtY29sYXMgRi4gUi4gQS4gUHJhZG8=?= X-Patchwork-Id: 12293 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 170CAC31F8 for ; Fri, 14 May 2021 13:18:07 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 60D7D6891C; Fri, 14 May 2021 15:18:06 +0200 (CEST) Received: from bhuna.collabora.co.uk (bhuna.collabora.co.uk [IPv6:2a00:1098:0:82:1000:25:2eeb:e3e3]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 67E0F602B5 for ; Fri, 14 May 2021 15:18:05 +0200 (CEST) Received: from localhost.localdomain (unknown [IPv6:2804:14c:1a9:2978:985c:7892:ebcf:7a90]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) (Authenticated sender: nfraprado) by bhuna.collabora.co.uk (Postfix) with ESMTPSA id D4E3B1F432B8; Fri, 14 May 2021 14:18:02 +0100 (BST) From: =?utf-8?b?TsOtY29sYXMgRi4gUi4gQS4gUHJhZG8=?= To: libcamera-devel@lists.libcamera.org Date: Fri, 14 May 2021 10:16:48 -0300 Message-Id: <20210514131652.345486-1-nfraprado@collabora.com> X-Mailer: git-send-email 2.31.1 MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v3 0/4] lc-compliance: Refactor using Googletest X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: kernel@collabora.com, =?utf-8?q?Andr=C3=A9_Almeida?= Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" This is a prototype of the refactoring of lc-compliance using Googletest as the framework. Patches 1 and 2 are just to simplify the cleanup path from the tests. Patch 3 does the actual refactor. Patch 4 adds two new parameters, --list and --filter. lc-compliance is built around testing a single camera (and possibly optional secondary cameras in the future), which is passed through the -c flag. The list of tests can be queried with: lc-compliance --list By default all tests are run, but it's possible to select which tests to run by using: lc-compliance -c '' --filter '' where the test_filter is in the same format as Googletest's --gtest_filter. Quoting its help: --gtest_filter=POSTIVE_PATTERNS[-NEGATIVE_PATTERNS] Run only the tests whose name matches one of the positive patterns but none of the negative patterns. '?' matches any single character; '*' matches any substring; ':' separates two patterns. This is a snippet of a successful run: [==========] Running 120 tests from 1 test suite. [----------] Global test environment set-up. [----------] 120 tests from RolesAndRequests/FixedRequestsTest [ RUN ] RolesAndRequests/FixedRequestsTest.BalancedSingleCycle/raw1 [51:19:15.995527857] [339440] INFO Camera camera.cpp:915 configuring streams: (0) 1280x720-MJPEG Camera needs 4 requests, can't test only 1 [ SKIPPED ] RolesAndRequests/FixedRequestsTest.BalancedSingleCycle/raw1 (63 ms) [...] [ RUN ] RolesAndRequests/FixedRequestsTest.Unbalanced/viewfinder89 [51:22:08.969389923] [339440] INFO Camera camera.cpp:915 configuring streams: (0) 1280x720-MJPEG [ OK ] RolesAndRequests/FixedRequestsTest.Unbalanced/viewfinder89 (3144 ms) [----------] 120 tests from RolesAndRequests/FixedRequestsTest (176118 ms total) [----------] Global test environment tear-down [==========] 120 tests from 1 test suite ran. (176118 ms total) [ PASSED ] 96 tests. [ SKIPPED ] 24 tests, listed below: [ SKIPPED ] RolesAndRequests/FixedRequestsTest.BalancedSingleCycle/raw1 [ SKIPPED ] RolesAndRequests/FixedRequestsTest.BalancedSingleCycle/raw2 [ SKIPPED ] RolesAndRequests/FixedRequestsTest.BalancedSingleCycle/raw3 [ SKIPPED ] RolesAndRequests/FixedRequestsTest.BalancedSingleCycle/still1 [...] And how a failure looks like: [ RUN ] RolesAndRequests/FixedRequestsTest.BalancedSingleCycle/raw5 [51:30:10.270474586] [340392] INFO Camera camera.cpp:915 configuring streams: (0) 1280x720-MJPEG ../src/lc-compliance/simple_capture.cpp:117: Failure Expected equality of these values: captureCount_ Which is: 5 captureLimit_ Which is: 6 [ FAILED ] RolesAndRequests/FixedRequestsTest.BalancedSingleCycle/raw5, where GetParam() = (0, 5) (317 ms) The current complete list of tests is shown below: RolesAndRequests/FixedRequestsTest. BalancedSingleCycle/raw1 # GetParam() = (0, 1) BalancedSingleCycle/raw2 # GetParam() = (0, 2) BalancedSingleCycle/raw3 # GetParam() = (0, 3) BalancedSingleCycle/raw5 # GetParam() = (0, 5) BalancedSingleCycle/raw8 # GetParam() = (0, 8) BalancedSingleCycle/raw13 # GetParam() = (0, 13) BalancedSingleCycle/raw21 # GetParam() = (0, 21) BalancedSingleCycle/raw34 # GetParam() = (0, 34) BalancedSingleCycle/raw55 # GetParam() = (0, 55) BalancedSingleCycle/raw89 # GetParam() = (0, 89) BalancedSingleCycle/still1 # GetParam() = (1, 1) BalancedSingleCycle/still2 # GetParam() = (1, 2) BalancedSingleCycle/still3 # GetParam() = (1, 3) BalancedSingleCycle/still5 # GetParam() = (1, 5) BalancedSingleCycle/still8 # GetParam() = (1, 8) BalancedSingleCycle/still13 # GetParam() = (1, 13) BalancedSingleCycle/still21 # GetParam() = (1, 21) BalancedSingleCycle/still34 # GetParam() = (1, 34) BalancedSingleCycle/still55 # GetParam() = (1, 55) BalancedSingleCycle/still89 # GetParam() = (1, 89) BalancedSingleCycle/video1 # GetParam() = (2, 1) BalancedSingleCycle/video2 # GetParam() = (2, 2) BalancedSingleCycle/video3 # GetParam() = (2, 3) BalancedSingleCycle/video5 # GetParam() = (2, 5) BalancedSingleCycle/video8 # GetParam() = (2, 8) BalancedSingleCycle/video13 # GetParam() = (2, 13) BalancedSingleCycle/video21 # GetParam() = (2, 21) BalancedSingleCycle/video34 # GetParam() = (2, 34) BalancedSingleCycle/video55 # GetParam() = (2, 55) BalancedSingleCycle/video89 # GetParam() = (2, 89) BalancedSingleCycle/viewfinder1 # GetParam() = (3, 1) BalancedSingleCycle/viewfinder2 # GetParam() = (3, 2) BalancedSingleCycle/viewfinder3 # GetParam() = (3, 3) BalancedSingleCycle/viewfinder5 # GetParam() = (3, 5) BalancedSingleCycle/viewfinder8 # GetParam() = (3, 8) BalancedSingleCycle/viewfinder13 # GetParam() = (3, 13) BalancedSingleCycle/viewfinder21 # GetParam() = (3, 21) BalancedSingleCycle/viewfinder34 # GetParam() = (3, 34) BalancedSingleCycle/viewfinder55 # GetParam() = (3, 55) BalancedSingleCycle/viewfinder89 # GetParam() = (3, 89) BalancedMultiCycle/raw1 # GetParam() = (0, 1) BalancedMultiCycle/raw2 # GetParam() = (0, 2) BalancedMultiCycle/raw3 # GetParam() = (0, 3) BalancedMultiCycle/raw5 # GetParam() = (0, 5) BalancedMultiCycle/raw8 # GetParam() = (0, 8) BalancedMultiCycle/raw13 # GetParam() = (0, 13) BalancedMultiCycle/raw21 # GetParam() = (0, 21) BalancedMultiCycle/raw34 # GetParam() = (0, 34) BalancedMultiCycle/raw55 # GetParam() = (0, 55) BalancedMultiCycle/raw89 # GetParam() = (0, 89) BalancedMultiCycle/still1 # GetParam() = (1, 1) BalancedMultiCycle/still2 # GetParam() = (1, 2) BalancedMultiCycle/still3 # GetParam() = (1, 3) BalancedMultiCycle/still5 # GetParam() = (1, 5) BalancedMultiCycle/still8 # GetParam() = (1, 8) BalancedMultiCycle/still13 # GetParam() = (1, 13) BalancedMultiCycle/still21 # GetParam() = (1, 21) BalancedMultiCycle/still34 # GetParam() = (1, 34) BalancedMultiCycle/still55 # GetParam() = (1, 55) BalancedMultiCycle/still89 # GetParam() = (1, 89) BalancedMultiCycle/video1 # GetParam() = (2, 1) BalancedMultiCycle/video2 # GetParam() = (2, 2) BalancedMultiCycle/video3 # GetParam() = (2, 3) BalancedMultiCycle/video5 # GetParam() = (2, 5) BalancedMultiCycle/video8 # GetParam() = (2, 8) BalancedMultiCycle/video13 # GetParam() = (2, 13) BalancedMultiCycle/video21 # GetParam() = (2, 21) BalancedMultiCycle/video34 # GetParam() = (2, 34) BalancedMultiCycle/video55 # GetParam() = (2, 55) BalancedMultiCycle/video89 # GetParam() = (2, 89) BalancedMultiCycle/viewfinder1 # GetParam() = (3, 1) BalancedMultiCycle/viewfinder2 # GetParam() = (3, 2) BalancedMultiCycle/viewfinder3 # GetParam() = (3, 3) BalancedMultiCycle/viewfinder5 # GetParam() = (3, 5) BalancedMultiCycle/viewfinder8 # GetParam() = (3, 8) BalancedMultiCycle/viewfinder13 # GetParam() = (3, 13) BalancedMultiCycle/viewfinder21 # GetParam() = (3, 21) BalancedMultiCycle/viewfinder34 # GetParam() = (3, 34) BalancedMultiCycle/viewfinder55 # GetParam() = (3, 55) BalancedMultiCycle/viewfinder89 # GetParam() = (3, 89) Unbalanced/raw1 # GetParam() = (0, 1) Unbalanced/raw2 # GetParam() = (0, 2) Unbalanced/raw3 # GetParam() = (0, 3) Unbalanced/raw5 # GetParam() = (0, 5) Unbalanced/raw8 # GetParam() = (0, 8) Unbalanced/raw13 # GetParam() = (0, 13) Unbalanced/raw21 # GetParam() = (0, 21) Unbalanced/raw34 # GetParam() = (0, 34) Unbalanced/raw55 # GetParam() = (0, 55) Unbalanced/raw89 # GetParam() = (0, 89) Unbalanced/still1 # GetParam() = (1, 1) Unbalanced/still2 # GetParam() = (1, 2) Unbalanced/still3 # GetParam() = (1, 3) Unbalanced/still5 # GetParam() = (1, 5) Unbalanced/still8 # GetParam() = (1, 8) Unbalanced/still13 # GetParam() = (1, 13) Unbalanced/still21 # GetParam() = (1, 21) Unbalanced/still34 # GetParam() = (1, 34) Unbalanced/still55 # GetParam() = (1, 55) Unbalanced/still89 # GetParam() = (1, 89) Unbalanced/video1 # GetParam() = (2, 1) Unbalanced/video2 # GetParam() = (2, 2) Unbalanced/video3 # GetParam() = (2, 3) Unbalanced/video5 # GetParam() = (2, 5) Unbalanced/video8 # GetParam() = (2, 8) Unbalanced/video13 # GetParam() = (2, 13) Unbalanced/video21 # GetParam() = (2, 21) Unbalanced/video34 # GetParam() = (2, 34) Unbalanced/video55 # GetParam() = (2, 55) Unbalanced/video89 # GetParam() = (2, 89) Unbalanced/viewfinder1 # GetParam() = (3, 1) Unbalanced/viewfinder2 # GetParam() = (3, 2) Unbalanced/viewfinder3 # GetParam() = (3, 3) Unbalanced/viewfinder5 # GetParam() = (3, 5) Unbalanced/viewfinder8 # GetParam() = (3, 8) Unbalanced/viewfinder13 # GetParam() = (3, 13) Unbalanced/viewfinder21 # GetParam() = (3, 21) Unbalanced/viewfinder34 # GetParam() = (3, 34) Unbalanced/viewfinder55 # GetParam() = (3, 55) Unbalanced/viewfinder89 # GetParam() = (3, 89) Changes in v3: - Thanks to Niklas: - Went back to static test registration, and created a Singleton Environment class to store the camera global variable - Added patch 4 implementing list and filter parameters - Added a nameParameters() function to give meaningful names to the test parameters, removing the need to register each test suite for every role v2: https://lists.libcamera.org/pipermail/libcamera-devel/2021-May/020056.html Changes in v2: - Added patch 1 to make Camera::stop() idempotent as well from [1]. - Made test registration dynamic in patch 3 - Some other minor fixes [1] https://lists.libcamera.org/pipermail/libcamera-devel/2021-April/019735.html v1: https://lists.libcamera.org/pipermail/libcamera-devel/2021-April/019702.html Nícolas F. R. A. Prado (4): libcamera: camera: Make stop() idempotent lc-compliance: Make SimpleCapture::stop() idempotent lc-compliance: Refactor using Googletest lc-compliance: Add list and filter parameters src/lc-compliance/main.cpp | 152 +++++++++++++++++++++------ src/lc-compliance/meson.build | 3 + src/lc-compliance/simple_capture.cpp | 95 ++++++----------- src/lc-compliance/simple_capture.h | 8 +- src/lc-compliance/single_stream.cpp | 148 +++++++++++++------------- src/lc-compliance/tests.h | 19 +++- src/libcamera/camera.cpp | 20 +++- 7 files changed, 275 insertions(+), 170 deletions(-)