From patchwork Sat Jan 18 03:54:43 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nicolas Dufresne X-Patchwork-Id: 2668 Return-Path: Received: from bhuna.collabora.co.uk (bhuna.collabora.co.uk [IPv6:2a00:1098:0:82:1000:25:2eeb:e3e3]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id E0CC560452 for ; Sat, 18 Jan 2020 04:59:48 +0100 (CET) Received: from nicolas-tpx395.localdomain (unknown [IPv6:2002:c0de:c115:0:66fc:8b:2a38:8313]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) (Authenticated sender: nicolas) by bhuna.collabora.co.uk (Postfix) with ESMTPSA id AD7562913C4; Sat, 18 Jan 2020 03:59:47 +0000 (GMT) From: Nicolas Dufresne To: libcamera-devel@lists.libcamera.org Cc: Nicolas Dufresne , Kieran Bingham Date: Fri, 17 Jan 2020 22:54:43 -0500 Message-Id: <20200118035448.230530-2-nicolas@ndufresne.ca> X-Mailer: git-send-email 2.24.1 In-Reply-To: <20200118035448.230530-1-nicolas@ndufresne.ca> References: <20200118035448.230530-1-nicolas@ndufresne.ca> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v3 1/6] checkstyle: Move from pep8 to pycodestyle X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 18 Jan 2020 03:59:49 -0000 From: Nicolas Dufresne The tool has been renamed in 2016 to make it more obvious what it is doing. There is no other changes needed on our side. See https://github.com/PyCQA/pycodestyle/issues/466 Signed-off-by: Nicolas Dufresne Reviewed-by: Kieran Bingham Reviewed-by: Laurent Pinchart --- utils/checkstyle.py | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/utils/checkstyle.py b/utils/checkstyle.py index 93f0291..7edea25 100755 --- a/utils/checkstyle.py +++ b/utils/checkstyle.py @@ -321,10 +321,10 @@ class Pep8Checker(StyleChecker): data = ''.join(self.__content).encode('utf-8') try: - ret = subprocess.run(['pep8', '--ignore=E501', '-'], + ret = subprocess.run(['pycodestyle', '--ignore=E501', '-'], input=data, stdout=subprocess.PIPE) except FileNotFoundError: - issues.append(StyleIssue(0, None, "Please install pep8 to validate python additions")) + issues.append(StyleIssue(0, None, "Please install pycodestyle to validate python additions")) return issues results = ret.stdout.decode('utf-8').splitlines() From patchwork Sat Jan 18 03:54:44 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Nicolas Dufresne X-Patchwork-Id: 2669 Return-Path: Received: from bhuna.collabora.co.uk (bhuna.collabora.co.uk [IPv6:2a00:1098:0:82:1000:25:2eeb:e3e3]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 9B3E860452 for ; Sat, 18 Jan 2020 04:59:56 +0100 (CET) Received: from nicolas-tpx395.localdomain (unknown [IPv6:2002:c0de:c115:0:66fc:8b:2a38:8313]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) (Authenticated sender: nicolas) by bhuna.collabora.co.uk (Postfix) with ESMTPSA id B41812913C4; Sat, 18 Jan 2020 03:59:51 +0000 (GMT) From: Nicolas Dufresne To: libcamera-devel@lists.libcamera.org Cc: Nicolas Dufresne , =?utf-8?q?Niklas_S?= =?utf-8?b?w7ZkZXJsdW5k?= , Laurent Pinchart Date: Fri, 17 Jan 2020 22:54:44 -0500 Message-Id: <20200118035448.230530-3-nicolas@ndufresne.ca> X-Mailer: git-send-email 2.24.1 In-Reply-To: <20200118035448.230530-1-nicolas@ndufresne.ca> References: <20200118035448.230530-1-nicolas@ndufresne.ca> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v3 2/6] checkstyle: Exit with 1 status if issues are found X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 18 Jan 2020 03:59:56 -0000 From: Nicolas Dufresne Makes the tool return 1 if there is any potential issues. This is needed when using this tool for pre-commit hook in order to abort the commit process. Signed-off-by: Nicolas Dufresne Reviewed-by: Niklas Söderlund Reviewed-by: Laurent Pinchart --- utils/checkstyle.py | 12 +++++++++--- 1 file changed, 9 insertions(+), 3 deletions(-) diff --git a/utils/checkstyle.py b/utils/checkstyle.py index 7edea25..4a14309 100755 --- a/utils/checkstyle.py +++ b/utils/checkstyle.py @@ -541,7 +541,7 @@ def check_style(top_level, commit): files = [f for f in files if len([p for p in patterns if fnmatch.fnmatch(os.path.basename(f), p)])] if len(files) == 0: print("Commit doesn't touch source files, skipping") - return + return 0 issues = 0 for f in files: @@ -554,6 +554,8 @@ def check_style(top_level, commit): print("%u potential style %s detected, please review" % \ (issues, 'issue' if issues == 1 else 'issues')) + return issues + def extract_revlist(revs): """Extract a list of commits on which to operate from a revision or revision @@ -632,11 +634,15 @@ def main(argv): revlist = extract_revlist(args.revision_range) + issues = 0 for commit in revlist: - check_style(top_level, commit) + issues += check_style(top_level, commit) print('') - return 0 + if issues: + return 1 + else: + return 0 if __name__ == '__main__': From patchwork Sat Jan 18 03:54:45 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nicolas Dufresne X-Patchwork-Id: 2670 Return-Path: Received: from bhuna.collabora.co.uk (bhuna.collabora.co.uk [IPv6:2a00:1098:0:82:1000:25:2eeb:e3e3]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 760D760452 for ; Sat, 18 Jan 2020 04:59:58 +0100 (CET) Received: from nicolas-tpx395.localdomain (unknown [IPv6:2002:c0de:c115:0:66fc:8b:2a38:8313]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) (Authenticated sender: nicolas) by bhuna.collabora.co.uk (Postfix) with ESMTPSA id AA51C2932F6; Sat, 18 Jan 2020 03:59:56 +0000 (GMT) From: Nicolas Dufresne To: libcamera-devel@lists.libcamera.org Cc: Nicolas Dufresne Date: Fri, 17 Jan 2020 22:54:45 -0500 Message-Id: <20200118035448.230530-4-nicolas@ndufresne.ca> X-Mailer: git-send-email 2.24.1 In-Reply-To: <20200118035448.230530-1-nicolas@ndufresne.ca> References: <20200118035448.230530-1-nicolas@ndufresne.ca> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v3 3/6] checkstyle: Introduce a Commit class X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 18 Jan 2020 03:59:58 -0000 From: Nicolas Dufresne This introduce a Commit class used in the final revlist list. All the git command are moved into that class. This class will be used to introduce new type of commit (StagedChanges and Amendment) needed to implement pre-commit hook support. Signed-off-by: Nicolas Dufresne Reviewed-by: Laurent Pinchart --- utils/checkstyle.py | 46 +++++++++++++++++++++++++++++---------------- 1 file changed, 30 insertions(+), 16 deletions(-) diff --git a/utils/checkstyle.py b/utils/checkstyle.py index 4a14309..828605a 100755 --- a/utils/checkstyle.py +++ b/utils/checkstyle.py @@ -458,12 +458,33 @@ class StripTrailingSpaceFormatter(Formatter): # Style checking # +class Commit: + def __init__(self, commit): + self.commit = commit + + def get_info(self, top_level): + # Get the commit title and list of files. + ret = subprocess.run(['git', 'show', '--pretty=oneline', '--name-only', + self.commit], + stdout=subprocess.PIPE).stdout.decode('utf-8') + files = ret.splitlines() + # Returning title and files list as a tuple + return files[0], files[1:] + + def get_diff(self, top_level, filename): + return subprocess.run(['git', 'diff', '%s~..%s' % (self.commit, self.commit), + '--', '%s/%s' % (top_level, filename)], + stdout=subprocess.PIPE).stdout.decode('utf-8') + + def get_file(self, filename): + return subprocess.run(['git', 'show', '%s:%s' % (self.commit, filename)], + stdout=subprocess.PIPE).stdout.decode('utf-8') + + def check_file(top_level, commit, filename): # Extract the line numbers touched by the commit. - diff = subprocess.run(['git', 'diff', '%s~..%s' % (commit, commit), '--', - '%s/%s' % (top_level, filename)], - stdout=subprocess.PIPE).stdout - diff = diff.decode('utf-8').splitlines(True) + diff = commit.get_diff(top_level, filename) + diff = diff.splitlines(True) commit_diff = parse_diff(diff) lines = [] @@ -476,9 +497,7 @@ def check_file(top_level, commit, filename): # Format the file after the commit with all formatters and compute the diff # between the unformatted and formatted contents. - after = subprocess.run(['git', 'show', '%s:%s' % (commit, filename)], - stdout=subprocess.PIPE).stdout - after = after.decode('utf-8') + after = commit.get_file(filename) formatted = after for formatter in Formatter.formatters(filename): @@ -522,12 +541,7 @@ def check_file(top_level, commit, filename): def check_style(top_level, commit): - # Get the commit title and list of files. - ret = subprocess.run(['git', 'show', '--pretty=oneline','--name-only', commit], - stdout=subprocess.PIPE) - files = ret.stdout.decode('utf-8').splitlines() - title = files[0] - files = files[1:] + title, files = commit.get_info(top_level) separator = '-' * len(title) print(separator) @@ -557,7 +571,7 @@ def check_style(top_level, commit): return issues -def extract_revlist(revs): +def extract_commits(revs): """Extract a list of commits on which to operate from a revision or revision range. """ @@ -576,7 +590,7 @@ def extract_revlist(revs): revlist = ret.stdout.decode('utf-8').splitlines() revlist.reverse() - return revlist + return [Commit(x) for x in revlist] def git_top_level(): @@ -632,7 +646,7 @@ def main(argv): if top_level is None: return 1 - revlist = extract_revlist(args.revision_range) + revlist = extract_commits(args.revision_range) issues = 0 for commit in revlist: From patchwork Sat Jan 18 03:54:46 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nicolas Dufresne X-Patchwork-Id: 2671 Return-Path: Received: from bhuna.collabora.co.uk (bhuna.collabora.co.uk [IPv6:2a00:1098:0:82:1000:25:2eeb:e3e3]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id AEC5260452 for ; Sat, 18 Jan 2020 05:00:02 +0100 (CET) Received: from nicolas-tpx395.localdomain (unknown [IPv6:2002:c0de:c115:0:66fc:8b:2a38:8313]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) (Authenticated sender: nicolas) by bhuna.collabora.co.uk (Postfix) with ESMTPSA id 538DA2913C4; Sat, 18 Jan 2020 03:59:58 +0000 (GMT) From: Nicolas Dufresne To: libcamera-devel@lists.libcamera.org Cc: Nicolas Dufresne Date: Fri, 17 Jan 2020 22:54:46 -0500 Message-Id: <20200118035448.230530-5-nicolas@ndufresne.ca> X-Mailer: git-send-email 2.24.1 In-Reply-To: <20200118035448.230530-1-nicolas@ndufresne.ca> References: <20200118035448.230530-1-nicolas@ndufresne.ca> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v3 4/6] checkstyle: Add support for checking style on staged changes X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 18 Jan 2020 04:00:02 -0000 From: Nicolas Dufresne This introduce a new command line "--staged" and a new special type of commit "StagedChanges". It will check the style of changes that are in the index, so the changes that would be committed by "git commit". "--staged" was chosen to match with "git diff --staged" command line. Other valid name could have been "--index" or "--cached". This was my personal preference, alias can be added later. Note that we must not confuse this with working tree changes, as these changes are not picked by "git commit". This feature is needed to implement pre-commit hook. Signed-off-by: Nicolas Dufresne Reviewed-by: Laurent Pinchart --- utils/checkstyle.py | 36 ++++++++++++++++++++++++++++++++++-- 1 file changed, 34 insertions(+), 2 deletions(-) diff --git a/utils/checkstyle.py b/utils/checkstyle.py index 828605a..1cd5476 100755 --- a/utils/checkstyle.py +++ b/utils/checkstyle.py @@ -481,6 +481,25 @@ class Commit: stdout=subprocess.PIPE).stdout.decode('utf-8') +class StagedChanges(Commit): + def __init__(self): + Commit.__init__(self, None) + + def get_info(self, top_level): + ret = subprocess.run(['git', 'diff', '--staged', '--name-only'], + stdout=subprocess.PIPE).stdout.decode('utf-8') + return "Staged changes", ret.splitlines() + + def get_diff(self, top_level, filename): + return subprocess.run(['git', 'diff', '--staged', '--', + '%s/%s' % (top_level, filename)], + stdout=subprocess.PIPE).stdout.decode('utf-8') + + def get_file(self, filename): + return subprocess.run(['git', 'show', ':%s' % (filename)], + stdout=subprocess.PIPE).stdout.decode('utf-8') + + def check_file(top_level, commit, filename): # Extract the line numbers touched by the commit. diff = commit.get_diff(top_level, filename) @@ -611,7 +630,9 @@ def main(argv): parser = argparse.ArgumentParser() parser.add_argument('--formatter', '-f', type=str, choices=['astyle', 'clang-format'], help='Code formatter. Default to clang-format if not specified.') - parser.add_argument('revision_range', type=str, default='HEAD', nargs='?', + parser.add_argument('--staged', '-s', action='store_true', + help='Include the changes in the index. Defaults to False') + parser.add_argument('revision_range', type=str, default=None, nargs='?', help='Revision range (as defined by git rev-parse). Defaults to HEAD if not specified.') args = parser.parse_args(argv[1:]) @@ -646,7 +667,18 @@ def main(argv): if top_level is None: return 1 - revlist = extract_commits(args.revision_range) + revlist = [] + if args.staged: + revlist.append(StagedChanges()) + + # If not --staged + if len(revlist) == 0: + # And no revisions was passed, then default to HEAD + if not args.revision_range: + args.revision_range = 'HEAD' + + if args.revision_range: + revlist += extract_commits(args.revision_range) issues = 0 for commit in revlist: From patchwork Sat Jan 18 03:54:47 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Nicolas Dufresne X-Patchwork-Id: 2672 Return-Path: Received: from bhuna.collabora.co.uk (bhuna.collabora.co.uk [46.235.227.227]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 4ACC2607A2 for ; Sat, 18 Jan 2020 05:00:04 +0100 (CET) Received: from nicolas-tpx395.localdomain (unknown [IPv6:2002:c0de:c115:0:66fc:8b:2a38:8313]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) (Authenticated sender: nicolas) by bhuna.collabora.co.uk (Postfix) with ESMTPSA id B113A2932F6; Sat, 18 Jan 2020 04:00:02 +0000 (GMT) From: Nicolas Dufresne To: libcamera-devel@lists.libcamera.org Cc: Nicolas Dufresne , =?utf-8?q?Niklas_S?= =?utf-8?b?w7ZkZXJsdW5k?= Date: Fri, 17 Jan 2020 22:54:47 -0500 Message-Id: <20200118035448.230530-6-nicolas@ndufresne.ca> X-Mailer: git-send-email 2.24.1 In-Reply-To: <20200118035448.230530-1-nicolas@ndufresne.ca> References: <20200118035448.230530-1-nicolas@ndufresne.ca> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v3 5/6] checkstyle: Add support for checking style on amendments X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 18 Jan 2020 04:00:04 -0000 From: Nicolas Dufresne This introduce a new argument "--amend" and a new special type of commit "Amendment". It will check the style of changes that are in the index combined with the changes of the last commit. So this is the changes that would be applied by "git commit --amend" hence the name of the argument. This is needed to implement pre-commit hook. Signed-off-by: Nicolas Dufresne Reviewed-by: Niklas Söderlund Reviewed-by: Laurent Pinchart --- utils/checkstyle.py | 26 +++++++++++++++++++++++++- 1 file changed, 25 insertions(+), 1 deletion(-) diff --git a/utils/checkstyle.py b/utils/checkstyle.py index 1cd5476..8591e4e 100755 --- a/utils/checkstyle.py +++ b/utils/checkstyle.py @@ -500,6 +500,26 @@ class StagedChanges(Commit): stdout=subprocess.PIPE).stdout.decode('utf-8') +class Amendment(StagedChanges): + def __init__(self): + Commit.__init__(self, None) + + def get_info(self, top_level): + # Create a title using HEAD commit + ret = subprocess.run(['git', 'show', '--pretty=oneline', '--name-only', 'HEAD'], + stdout=subprocess.PIPE).stdout.decode('utf-8') + title = 'Amendment of: ' + ret.splitlines()[0] + # Extract the list of modifier files + ret = subprocess.run(['git', 'diff', '--staged', '--name-only', 'HEAD~'], + stdout=subprocess.PIPE).stdout.decode('utf-8') + return title, ret.splitlines() + + def get_diff(self, top_level, filename): + return subprocess.run(['git', 'diff', '--staged', 'HEAD~', '--', + '%s/%s' % (top_level, filename)], + stdout=subprocess.PIPE).stdout.decode('utf-8') + + def check_file(top_level, commit, filename): # Extract the line numbers touched by the commit. diff = commit.get_diff(top_level, filename) @@ -632,6 +652,8 @@ def main(argv): help='Code formatter. Default to clang-format if not specified.') parser.add_argument('--staged', '-s', action='store_true', help='Include the changes in the index. Defaults to False') + parser.add_argument('--amend', '-a', action='store_true', + help='Include changes in the index and the previous patch combined. Defaults to False') parser.add_argument('revision_range', type=str, default=None, nargs='?', help='Revision range (as defined by git rev-parse). Defaults to HEAD if not specified.') args = parser.parse_args(argv[1:]) @@ -670,8 +692,10 @@ def main(argv): revlist = [] if args.staged: revlist.append(StagedChanges()) + if args.amend: + revlist.append(Amendment()) - # If not --staged + # If none of --staged or --amend was passed if len(revlist) == 0: # And no revisions was passed, then default to HEAD if not args.revision_range: From patchwork Sat Jan 18 03:54:48 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nicolas Dufresne X-Patchwork-Id: 2673 Return-Path: Received: from bhuna.collabora.co.uk (bhuna.collabora.co.uk [IPv6:2a00:1098:0:82:1000:25:2eeb:e3e3]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 87434607AD for ; Sat, 18 Jan 2020 05:00:05 +0100 (CET) Received: from nicolas-tpx395.localdomain (unknown [IPv6:2002:c0de:c115:0:66fc:8b:2a38:8313]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) (Authenticated sender: nicolas) by bhuna.collabora.co.uk (Postfix) with ESMTPSA id 66F50293B1A; Sat, 18 Jan 2020 04:00:04 +0000 (GMT) From: Nicolas Dufresne To: libcamera-devel@lists.libcamera.org Cc: Nicolas Dufresne Date: Fri, 17 Jan 2020 22:54:48 -0500 Message-Id: <20200118035448.230530-7-nicolas@ndufresne.ca> X-Mailer: git-send-email 2.24.1 In-Reply-To: <20200118035448.230530-1-nicolas@ndufresne.ca> References: <20200118035448.230530-1-nicolas@ndufresne.ca> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v3 6/6] checkstyle: Add a pre-commit hook script X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 18 Jan 2020 04:00:05 -0000 From: Nicolas Dufresne This adds support for pre-commit hook workflow. In pre-commit hook we check the style on the changes currently staged or the combination of the index and the last commit if "git commit --amend" is being used. Signed-off-by: Nicolas Dufresne Reviewed-by: Laurent Pinchart --- utils/hooks/pre-commit | 17 +++++++++++++++++ 1 file changed, 17 insertions(+) create mode 100755 utils/hooks/pre-commit diff --git a/utils/hooks/pre-commit b/utils/hooks/pre-commit new file mode 100755 index 0000000..7a5554d --- /dev/null +++ b/utils/hooks/pre-commit @@ -0,0 +1,17 @@ +#!/bin/sh + +# Execute the checkstyle script after committing any code. This allows the +# commit to succeed, but ensures that the developer is aware of any potential +# issues immediately, and can resolve them and fix rapidly with: +# git commit --amend +# +# To utilise this hook, install this file with: +# cp utils/hooks/post-commit .git/hooks/post-commit + +args="--staged" +if ps -ocommand= -p $PPID | grep -- "--amend" +then + args="--amend" +fi + +./utils/checkstyle.py $args