{"id":12293,"url":"https://patchwork.libcamera.org/api/covers/12293/?format=json","web_url":"https://patchwork.libcamera.org/cover/12293/","project":{"id":1,"url":"https://patchwork.libcamera.org/api/projects/1/?format=json","name":"libcamera","link_name":"libcamera","list_id":"libcamera_core","list_email":"libcamera-devel@lists.libcamera.org","web_url":"","scm_url":"","webscm_url":""},"msgid":"<20210514131652.345486-1-nfraprado@collabora.com>","date":"2021-05-14T13:16:48","name":"[libcamera-devel,v3,0/4] lc-compliance: Refactor using Googletest","submitter":{"id":84,"url":"https://patchwork.libcamera.org/api/people/84/?format=json","name":"Nícolas F. R. A. Prado","email":"nfraprado@collabora.com"},"mbox":"https://patchwork.libcamera.org/cover/12293/mbox/","series":[{"id":2031,"url":"https://patchwork.libcamera.org/api/series/2031/?format=json","web_url":"https://patchwork.libcamera.org/project/libcamera/list/?series=2031","date":"2021-05-14T13:16:48","name":"lc-compliance: Refactor using Googletest","version":3,"mbox":"https://patchwork.libcamera.org/series/2031/mbox/"}],"comments":"https://patchwork.libcamera.org/api/covers/12293/comments/","headers":{"Return-Path":"","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id 170CAC31F8\n\tfor ;\n\tFri, 14 May 2021 13:18:07 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id 60D7D6891C;\n\tFri, 14 May 2021 15:18:06 +0200 (CEST)","from bhuna.collabora.co.uk (bhuna.collabora.co.uk\n\t[IPv6:2a00:1098:0:82:1000:25:2eeb:e3e3])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id 67E0F602B5\n\tfor ;\n\tFri, 14 May 2021 15:18:05 +0200 (CEST)","from localhost.localdomain (unknown\n\t[IPv6:2804:14c:1a9:2978:985c:7892:ebcf:7a90])\n\t(using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128\n\tbits))\n\t(No client certificate requested) (Authenticated sender: nfraprado)\n\tby bhuna.collabora.co.uk (Postfix) with ESMTPSA id D4E3B1F432B8;\n\tFri, 14 May 2021 14:18:02 +0100 (BST)"],"From":"=?utf-8?b?TsOtY29sYXMgRi4gUi4gQS4gUHJhZG8=?= ","To":"libcamera-devel@lists.libcamera.org","Date":"Fri, 14 May 2021 10:16:48 -0300","Message-Id":"<20210514131652.345486-1-nfraprado@collabora.com>","X-Mailer":"git-send-email 2.31.1","MIME-Version":"1.0","Subject":"[libcamera-devel] [PATCH v3 0/4] lc-compliance: Refactor using\n\tGoogletest","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Cc":"kernel@collabora.com, =?utf-8?q?Andr=C3=A9_Almeida?=\n\t","Content-Type":"text/plain; charset=\"utf-8\"","Content-Transfer-Encoding":"base64","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" "},"content":"This is a prototype of the refactoring of lc-compliance using Googletest as the\nframework.\n\nPatches 1 and 2 are just to simplify the cleanup path from the tests. Patch 3\ndoes the actual refactor. Patch 4 adds two new parameters, --list and --filter.\n\nlc-compliance is built around testing a single camera (and possibly optional\nsecondary cameras in the future), which is passed through the -c flag. The list\nof tests can be queried with:\n\n\tlc-compliance --list\n\nBy default all tests are run, but it's possible to select which tests to run by\nusing:\n\n\tlc-compliance -c '' --filter ''\n\nwhere the test_filter is in the same format as Googletest's --gtest_filter.\nQuoting its help:\n\n --gtest_filter=POSTIVE_PATTERNS[-NEGATIVE_PATTERNS]\n Run only the tests whose name matches one of the positive patterns but\n none of the negative patterns. '?' matches any single character; '*'\n matches any substring; ':' separates two patterns.\n\nThis is a snippet of a successful run:\n\n\t[==========] Running 120 tests from 1 test suite.\n\t[----------] Global test environment set-up.\n\t[----------] 120 tests from RolesAndRequests/FixedRequestsTest\n\t[ RUN ] RolesAndRequests/FixedRequestsTest.BalancedSingleCycle/raw1\n\t[51:19:15.995527857] [339440] INFO Camera camera.cpp:915 configuring streams: (0) 1280x720-MJPEG\n\tCamera needs 4 requests, can't test only 1\n\t[ SKIPPED ] RolesAndRequests/FixedRequestsTest.BalancedSingleCycle/raw1 (63 ms)\n\n\t[...]\n\n\t[ RUN ] RolesAndRequests/FixedRequestsTest.Unbalanced/viewfinder89\n\t[51:22:08.969389923] [339440] INFO Camera camera.cpp:915 configuring streams: (0) 1280x720-MJPEG\n\t[ OK ] RolesAndRequests/FixedRequestsTest.Unbalanced/viewfinder89 (3144 ms)\n\t[----------] 120 tests from RolesAndRequests/FixedRequestsTest (176118 ms total)\n\n\t[----------] Global test environment tear-down\n\t[==========] 120 tests from 1 test suite ran. (176118 ms total)\n\t[ PASSED ] 96 tests.\n\t[ SKIPPED ] 24 tests, listed below:\n\t[ SKIPPED ] RolesAndRequests/FixedRequestsTest.BalancedSingleCycle/raw1\n\t[ SKIPPED ] RolesAndRequests/FixedRequestsTest.BalancedSingleCycle/raw2\n\t[ SKIPPED ] RolesAndRequests/FixedRequestsTest.BalancedSingleCycle/raw3\n\t[ SKIPPED ] RolesAndRequests/FixedRequestsTest.BalancedSingleCycle/still1\n\n\t[...]\n\nAnd how a failure looks like:\n\n\t[ RUN ] RolesAndRequests/FixedRequestsTest.BalancedSingleCycle/raw5\n\t[51:30:10.270474586] [340392] INFO Camera camera.cpp:915 configuring streams: (0) 1280x720-MJPEG\n\t../src/lc-compliance/simple_capture.cpp:117: Failure\n\tExpected equality of these values:\n\t captureCount_\n\t Which is: 5\n\t captureLimit_\n\t Which is: 6\n\t[ FAILED ] RolesAndRequests/FixedRequestsTest.BalancedSingleCycle/raw5, where GetParam() = (0, 5) (317 ms)\n\nThe current complete list of tests is shown below:\n\n\tRolesAndRequests/FixedRequestsTest.\n\t BalancedSingleCycle/raw1 # GetParam() = (0, 1)\n\t BalancedSingleCycle/raw2 # GetParam() = (0, 2)\n\t BalancedSingleCycle/raw3 # GetParam() = (0, 3)\n\t BalancedSingleCycle/raw5 # GetParam() = (0, 5)\n\t BalancedSingleCycle/raw8 # GetParam() = (0, 8)\n\t BalancedSingleCycle/raw13 # GetParam() = (0, 13)\n\t BalancedSingleCycle/raw21 # GetParam() = (0, 21)\n\t BalancedSingleCycle/raw34 # GetParam() = (0, 34)\n\t BalancedSingleCycle/raw55 # GetParam() = (0, 55)\n\t BalancedSingleCycle/raw89 # GetParam() = (0, 89)\n\t BalancedSingleCycle/still1 # GetParam() = (1, 1)\n\t BalancedSingleCycle/still2 # GetParam() = (1, 2)\n\t BalancedSingleCycle/still3 # GetParam() = (1, 3)\n\t BalancedSingleCycle/still5 # GetParam() = (1, 5)\n\t BalancedSingleCycle/still8 # GetParam() = (1, 8)\n\t BalancedSingleCycle/still13 # GetParam() = (1, 13)\n\t BalancedSingleCycle/still21 # GetParam() = (1, 21)\n\t BalancedSingleCycle/still34 # GetParam() = (1, 34)\n\t BalancedSingleCycle/still55 # GetParam() = (1, 55)\n\t BalancedSingleCycle/still89 # GetParam() = (1, 89)\n\t BalancedSingleCycle/video1 # GetParam() = (2, 1)\n\t BalancedSingleCycle/video2 # GetParam() = (2, 2)\n\t BalancedSingleCycle/video3 # GetParam() = (2, 3)\n\t BalancedSingleCycle/video5 # GetParam() = (2, 5)\n\t BalancedSingleCycle/video8 # GetParam() = (2, 8)\n\t BalancedSingleCycle/video13 # GetParam() = (2, 13)\n\t BalancedSingleCycle/video21 # GetParam() = (2, 21)\n\t BalancedSingleCycle/video34 # GetParam() = (2, 34)\n\t BalancedSingleCycle/video55 # GetParam() = (2, 55)\n\t BalancedSingleCycle/video89 # GetParam() = (2, 89)\n\t BalancedSingleCycle/viewfinder1 # GetParam() = (3, 1)\n\t BalancedSingleCycle/viewfinder2 # GetParam() = (3, 2)\n\t BalancedSingleCycle/viewfinder3 # GetParam() = (3, 3)\n\t BalancedSingleCycle/viewfinder5 # GetParam() = (3, 5)\n\t BalancedSingleCycle/viewfinder8 # GetParam() = (3, 8)\n\t BalancedSingleCycle/viewfinder13 # GetParam() = (3, 13)\n\t BalancedSingleCycle/viewfinder21 # GetParam() = (3, 21)\n\t BalancedSingleCycle/viewfinder34 # GetParam() = (3, 34)\n\t BalancedSingleCycle/viewfinder55 # GetParam() = (3, 55)\n\t BalancedSingleCycle/viewfinder89 # GetParam() = (3, 89)\n\t BalancedMultiCycle/raw1 # GetParam() = (0, 1)\n\t BalancedMultiCycle/raw2 # GetParam() = (0, 2)\n\t BalancedMultiCycle/raw3 # GetParam() = (0, 3)\n\t BalancedMultiCycle/raw5 # GetParam() = (0, 5)\n\t BalancedMultiCycle/raw8 # GetParam() = (0, 8)\n\t BalancedMultiCycle/raw13 # GetParam() = (0, 13)\n\t BalancedMultiCycle/raw21 # GetParam() = (0, 21)\n\t BalancedMultiCycle/raw34 # GetParam() = (0, 34)\n\t BalancedMultiCycle/raw55 # GetParam() = (0, 55)\n\t BalancedMultiCycle/raw89 # GetParam() = (0, 89)\n\t BalancedMultiCycle/still1 # GetParam() = (1, 1)\n\t BalancedMultiCycle/still2 # GetParam() = (1, 2)\n\t BalancedMultiCycle/still3 # GetParam() = (1, 3)\n\t BalancedMultiCycle/still5 # GetParam() = (1, 5)\n\t BalancedMultiCycle/still8 # GetParam() = (1, 8)\n\t BalancedMultiCycle/still13 # GetParam() = (1, 13)\n\t BalancedMultiCycle/still21 # GetParam() = (1, 21)\n\t BalancedMultiCycle/still34 # GetParam() = (1, 34)\n\t BalancedMultiCycle/still55 # GetParam() = (1, 55)\n\t BalancedMultiCycle/still89 # GetParam() = (1, 89)\n\t BalancedMultiCycle/video1 # GetParam() = (2, 1)\n\t BalancedMultiCycle/video2 # GetParam() = (2, 2)\n\t BalancedMultiCycle/video3 # GetParam() = (2, 3)\n\t BalancedMultiCycle/video5 # GetParam() = (2, 5)\n\t BalancedMultiCycle/video8 # GetParam() = (2, 8)\n\t BalancedMultiCycle/video13 # GetParam() = (2, 13)\n\t BalancedMultiCycle/video21 # GetParam() = (2, 21)\n\t BalancedMultiCycle/video34 # GetParam() = (2, 34)\n\t BalancedMultiCycle/video55 # GetParam() = (2, 55)\n\t BalancedMultiCycle/video89 # GetParam() = (2, 89)\n\t BalancedMultiCycle/viewfinder1 # GetParam() = (3, 1)\n\t BalancedMultiCycle/viewfinder2 # GetParam() = (3, 2)\n\t BalancedMultiCycle/viewfinder3 # GetParam() = (3, 3)\n\t BalancedMultiCycle/viewfinder5 # GetParam() = (3, 5)\n\t BalancedMultiCycle/viewfinder8 # GetParam() = (3, 8)\n\t BalancedMultiCycle/viewfinder13 # GetParam() = (3, 13)\n\t BalancedMultiCycle/viewfinder21 # GetParam() = (3, 21)\n\t BalancedMultiCycle/viewfinder34 # GetParam() = (3, 34)\n\t BalancedMultiCycle/viewfinder55 # GetParam() = (3, 55)\n\t BalancedMultiCycle/viewfinder89 # GetParam() = (3, 89)\n\t Unbalanced/raw1 # GetParam() = (0, 1)\n\t Unbalanced/raw2 # GetParam() = (0, 2)\n\t Unbalanced/raw3 # GetParam() = (0, 3)\n\t Unbalanced/raw5 # GetParam() = (0, 5)\n\t Unbalanced/raw8 # GetParam() = (0, 8)\n\t Unbalanced/raw13 # GetParam() = (0, 13)\n\t Unbalanced/raw21 # GetParam() = (0, 21)\n\t Unbalanced/raw34 # GetParam() = (0, 34)\n\t Unbalanced/raw55 # GetParam() = (0, 55)\n\t Unbalanced/raw89 # GetParam() = (0, 89)\n\t Unbalanced/still1 # GetParam() = (1, 1)\n\t Unbalanced/still2 # GetParam() = (1, 2)\n\t Unbalanced/still3 # GetParam() = (1, 3)\n\t Unbalanced/still5 # GetParam() = (1, 5)\n\t Unbalanced/still8 # GetParam() = (1, 8)\n\t Unbalanced/still13 # GetParam() = (1, 13)\n\t Unbalanced/still21 # GetParam() = (1, 21)\n\t Unbalanced/still34 # GetParam() = (1, 34)\n\t Unbalanced/still55 # GetParam() = (1, 55)\n\t Unbalanced/still89 # GetParam() = (1, 89)\n\t Unbalanced/video1 # GetParam() = (2, 1)\n\t Unbalanced/video2 # GetParam() = (2, 2)\n\t Unbalanced/video3 # GetParam() = (2, 3)\n\t Unbalanced/video5 # GetParam() = (2, 5)\n\t Unbalanced/video8 # GetParam() = (2, 8)\n\t Unbalanced/video13 # GetParam() = (2, 13)\n\t Unbalanced/video21 # GetParam() = (2, 21)\n\t Unbalanced/video34 # GetParam() = (2, 34)\n\t Unbalanced/video55 # GetParam() = (2, 55)\n\t Unbalanced/video89 # GetParam() = (2, 89)\n\t Unbalanced/viewfinder1 # GetParam() = (3, 1)\n\t Unbalanced/viewfinder2 # GetParam() = (3, 2)\n\t Unbalanced/viewfinder3 # GetParam() = (3, 3)\n\t Unbalanced/viewfinder5 # GetParam() = (3, 5)\n\t Unbalanced/viewfinder8 # GetParam() = (3, 8)\n\t Unbalanced/viewfinder13 # GetParam() = (3, 13)\n\t Unbalanced/viewfinder21 # GetParam() = (3, 21)\n\t Unbalanced/viewfinder34 # GetParam() = (3, 34)\n\t Unbalanced/viewfinder55 # GetParam() = (3, 55)\n\t Unbalanced/viewfinder89 # GetParam() = (3, 89)\n\nChanges in v3:\n- Thanks to Niklas:\n - Went back to static test registration, and created a Singleton Environment\n class to store the camera global variable\n- Added patch 4 implementing list and filter parameters\n- Added a nameParameters() function to give meaningful names to the test\n parameters, removing the need to register each test suite for every role\n\nv2: https://lists.libcamera.org/pipermail/libcamera-devel/2021-May/020056.html\n\nChanges in v2:\n- Added patch 1 to make Camera::stop() idempotent as well from [1].\n- Made test registration dynamic in patch 3\n- Some other minor fixes\n\n[1] https://lists.libcamera.org/pipermail/libcamera-devel/2021-April/019735.html\n\nv1: https://lists.libcamera.org/pipermail/libcamera-devel/2021-April/019702.html\n\nNícolas F. R. A. Prado (4):\n libcamera: camera: Make stop() idempotent\n lc-compliance: Make SimpleCapture::stop() idempotent\n lc-compliance: Refactor using Googletest\n lc-compliance: Add list and filter parameters\n\n src/lc-compliance/main.cpp | 152 +++++++++++++++++++++------\n src/lc-compliance/meson.build | 3 +\n src/lc-compliance/simple_capture.cpp | 95 ++++++-----------\n src/lc-compliance/simple_capture.h | 8 +-\n src/lc-compliance/single_stream.cpp | 148 +++++++++++++-------------\n src/lc-compliance/tests.h | 19 +++-\n src/libcamera/camera.cpp | 20 +++-\n 7 files changed, 275 insertions(+), 170 deletions(-)"}