From patchwork Sat Jan 18 20:00:10 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nicolas Dufresne X-Patchwork-Id: 2675 Return-Path: Received: from bhuna.collabora.co.uk (bhuna.collabora.co.uk [46.235.227.227]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id EBBF2607B0 for ; Sat, 18 Jan 2020 21:02:43 +0100 (CET) Received: from nicolas-tpx395.localdomain (unknown [IPv6:2002:c0de:c115:0:66fc:8b:2a38:8313]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) (Authenticated sender: nicolas) by bhuna.collabora.co.uk (Postfix) with ESMTPSA id 96C902911C1; Sat, 18 Jan 2020 20:02:42 +0000 (GMT) From: Nicolas Dufresne To: libcamera-devel@lists.libcamera.org Cc: Nicolas Dufresne , Kieran Bingham , Laurent Pinchart Date: Sat, 18 Jan 2020 15:00:10 -0500 Message-Id: <20200118200015.16531-2-nicolas@ndufresne.ca> X-Mailer: git-send-email 2.24.1 In-Reply-To: <20200118200015.16531-1-nicolas@ndufresne.ca> References: <20200118200015.16531-1-nicolas@ndufresne.ca> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v4 1/6] checkstyle: Move from pep8 to pycodestyle X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 18 Jan 2020 20:02:44 -0000 From: Nicolas Dufresne The tool has been renamed in 2016 to make it more obvious what it is doing. There is no other changes needed on our side. See https://github.com/PyCQA/pycodestyle/issues/466 Signed-off-by: Nicolas Dufresne Reviewed-by: Kieran Bingham Reviewed-by: Laurent Pinchart --- utils/checkstyle.py | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/utils/checkstyle.py b/utils/checkstyle.py index 93f0291..7edea25 100755 --- a/utils/checkstyle.py +++ b/utils/checkstyle.py @@ -321,10 +321,10 @@ class Pep8Checker(StyleChecker): data = ''.join(self.__content).encode('utf-8') try: - ret = subprocess.run(['pep8', '--ignore=E501', '-'], + ret = subprocess.run(['pycodestyle', '--ignore=E501', '-'], input=data, stdout=subprocess.PIPE) except FileNotFoundError: - issues.append(StyleIssue(0, None, "Please install pep8 to validate python additions")) + issues.append(StyleIssue(0, None, "Please install pycodestyle to validate python additions")) return issues results = ret.stdout.decode('utf-8').splitlines() From patchwork Sat Jan 18 20:00:11 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Nicolas Dufresne X-Patchwork-Id: 2676 Return-Path: Received: from bhuna.collabora.co.uk (bhuna.collabora.co.uk [IPv6:2a00:1098:0:82:1000:25:2eeb:e3e3]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 3A56A607B1 for ; Sat, 18 Jan 2020 21:02:45 +0100 (CET) Received: from nicolas-tpx395.localdomain (unknown [IPv6:2002:c0de:c115:0:66fc:8b:2a38:8313]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) (Authenticated sender: nicolas) by bhuna.collabora.co.uk (Postfix) with ESMTPSA id D185A2925A2; Sat, 18 Jan 2020 20:02:43 +0000 (GMT) From: Nicolas Dufresne To: libcamera-devel@lists.libcamera.org Cc: Nicolas Dufresne , =?utf-8?q?Niklas_S?= =?utf-8?b?w7ZkZXJsdW5k?= , Laurent Pinchart Date: Sat, 18 Jan 2020 15:00:11 -0500 Message-Id: <20200118200015.16531-3-nicolas@ndufresne.ca> X-Mailer: git-send-email 2.24.1 In-Reply-To: <20200118200015.16531-1-nicolas@ndufresne.ca> References: <20200118200015.16531-1-nicolas@ndufresne.ca> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v4 2/6] checkstyle: Exit with 1 status if issues are found X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 18 Jan 2020 20:02:45 -0000 From: Nicolas Dufresne Makes the tool return 1 if there is any potential issues. This is needed when using this tool for pre-commit hook in order to abort the commit process. Signed-off-by: Nicolas Dufresne Reviewed-by: Niklas Söderlund Reviewed-by: Laurent Pinchart --- utils/checkstyle.py | 12 +++++++++--- 1 file changed, 9 insertions(+), 3 deletions(-) diff --git a/utils/checkstyle.py b/utils/checkstyle.py index 7edea25..4a14309 100755 --- a/utils/checkstyle.py +++ b/utils/checkstyle.py @@ -541,7 +541,7 @@ def check_style(top_level, commit): files = [f for f in files if len([p for p in patterns if fnmatch.fnmatch(os.path.basename(f), p)])] if len(files) == 0: print("Commit doesn't touch source files, skipping") - return + return 0 issues = 0 for f in files: @@ -554,6 +554,8 @@ def check_style(top_level, commit): print("%u potential style %s detected, please review" % \ (issues, 'issue' if issues == 1 else 'issues')) + return issues + def extract_revlist(revs): """Extract a list of commits on which to operate from a revision or revision @@ -632,11 +634,15 @@ def main(argv): revlist = extract_revlist(args.revision_range) + issues = 0 for commit in revlist: - check_style(top_level, commit) + issues += check_style(top_level, commit) print('') - return 0 + if issues: + return 1 + else: + return 0 if __name__ == '__main__': From patchwork Sat Jan 18 20:00:12 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nicolas Dufresne X-Patchwork-Id: 2677 Return-Path: Received: from bhuna.collabora.co.uk (bhuna.collabora.co.uk [IPv6:2a00:1098:0:82:1000:25:2eeb:e3e3]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id E653B60455 for ; Sat, 18 Jan 2020 21:02:49 +0100 (CET) Received: from nicolas-tpx395.localdomain (unknown [IPv6:2002:c0de:c115:0:66fc:8b:2a38:8313]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) (Authenticated sender: nicolas) by bhuna.collabora.co.uk (Postfix) with ESMTPSA id 422C72911C1; Sat, 18 Jan 2020 20:02:45 +0000 (GMT) From: Nicolas Dufresne To: libcamera-devel@lists.libcamera.org Cc: Nicolas Dufresne , Laurent Pinchart Date: Sat, 18 Jan 2020 15:00:12 -0500 Message-Id: <20200118200015.16531-4-nicolas@ndufresne.ca> X-Mailer: git-send-email 2.24.1 In-Reply-To: <20200118200015.16531-1-nicolas@ndufresne.ca> References: <20200118200015.16531-1-nicolas@ndufresne.ca> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v4 3/6] checkstyle: Introduce a Commit class X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 18 Jan 2020 20:02:50 -0000 From: Nicolas Dufresne This introduces a Commit class used in the final revlist list. All the git commands are moved into that class. This class will be used to introduce new types of commit (StagedChanges and Amendment) needed to implement pre-commit hook support. Signed-off-by: Nicolas Dufresne Reviewed-by: Laurent Pinchart --- utils/checkstyle.py | 48 +++++++++++++++++++++++++++++---------------- 1 file changed, 31 insertions(+), 17 deletions(-) diff --git a/utils/checkstyle.py b/utils/checkstyle.py index 4a14309..d928a5f 100755 --- a/utils/checkstyle.py +++ b/utils/checkstyle.py @@ -458,12 +458,33 @@ class StripTrailingSpaceFormatter(Formatter): # Style checking # +class Commit: + def __init__(self, commit): + self.commit = commit + + def get_info(self): + # Get the commit title and list of files. + ret = subprocess.run(['git', 'show', '--pretty=oneline', '--name-only', + self.commit], + stdout=subprocess.PIPE).stdout.decode('utf-8') + files = ret.splitlines() + # Returning title and files list as a tuple + return files[0], files[1:] + + def get_diff(self, top_level, filename): + return subprocess.run(['git', 'diff', '%s~..%s' % (self.commit, self.commit), + '--', '%s/%s' % (top_level, filename)], + stdout=subprocess.PIPE).stdout.decode('utf-8') + + def get_file(self, filename): + return subprocess.run(['git', 'show', '%s:%s' % (self.commit, filename)], + stdout=subprocess.PIPE).stdout.decode('utf-8') + + def check_file(top_level, commit, filename): # Extract the line numbers touched by the commit. - diff = subprocess.run(['git', 'diff', '%s~..%s' % (commit, commit), '--', - '%s/%s' % (top_level, filename)], - stdout=subprocess.PIPE).stdout - diff = diff.decode('utf-8').splitlines(True) + diff = commit.get_diff(top_level, filename) + diff = diff.splitlines(True) commit_diff = parse_diff(diff) lines = [] @@ -476,9 +497,7 @@ def check_file(top_level, commit, filename): # Format the file after the commit with all formatters and compute the diff # between the unformatted and formatted contents. - after = subprocess.run(['git', 'show', '%s:%s' % (commit, filename)], - stdout=subprocess.PIPE).stdout - after = after.decode('utf-8') + after = commit.get_file(filename) formatted = after for formatter in Formatter.formatters(filename): @@ -522,12 +541,7 @@ def check_file(top_level, commit, filename): def check_style(top_level, commit): - # Get the commit title and list of files. - ret = subprocess.run(['git', 'show', '--pretty=oneline','--name-only', commit], - stdout=subprocess.PIPE) - files = ret.stdout.decode('utf-8').splitlines() - title = files[0] - files = files[1:] + title, files = commit.get_info() separator = '-' * len(title) print(separator) @@ -557,7 +571,7 @@ def check_style(top_level, commit): return issues -def extract_revlist(revs): +def extract_commits(revs): """Extract a list of commits on which to operate from a revision or revision range. """ @@ -576,7 +590,7 @@ def extract_revlist(revs): revlist = ret.stdout.decode('utf-8').splitlines() revlist.reverse() - return revlist + return [Commit(x) for x in revlist] def git_top_level(): @@ -632,10 +646,10 @@ def main(argv): if top_level is None: return 1 - revlist = extract_revlist(args.revision_range) + commits = extract_commits(args.revision_range) issues = 0 - for commit in revlist: + for commit in commits: issues += check_style(top_level, commit) print('') From patchwork Sat Jan 18 20:00:13 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nicolas Dufresne X-Patchwork-Id: 2678 Return-Path: Received: from bhuna.collabora.co.uk (bhuna.collabora.co.uk [46.235.227.227]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id BA0D160455 for ; Sat, 18 Jan 2020 21:02:52 +0100 (CET) Received: from nicolas-tpx395.localdomain (unknown [IPv6:2002:c0de:c115:0:66fc:8b:2a38:8313]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) (Authenticated sender: nicolas) by bhuna.collabora.co.uk (Postfix) with ESMTPSA id 6BEBE2911C1; Sat, 18 Jan 2020 20:02:50 +0000 (GMT) From: Nicolas Dufresne To: libcamera-devel@lists.libcamera.org Cc: Nicolas Dufresne , Laurent Pinchart Date: Sat, 18 Jan 2020 15:00:13 -0500 Message-Id: <20200118200015.16531-5-nicolas@ndufresne.ca> X-Mailer: git-send-email 2.24.1 In-Reply-To: <20200118200015.16531-1-nicolas@ndufresne.ca> References: <20200118200015.16531-1-nicolas@ndufresne.ca> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v4 4/6] checkstyle: Add support for checking style on staged changes X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 18 Jan 2020 20:02:52 -0000 From: Nicolas Dufresne This introduces a new command line "--staged" and a new special type of commit "StagedChanges". It will check the style of changes that are in the index, so the changes that would be committed by "git commit". "--staged" was chosen to match with "git diff --staged" command line. Other valid name could have been "--index" or "--cached". This was my personal preference, aliases can be added later. Note that we must not confuse this with working tree changes, as these changes are not picked by "git commit". This feature is needed to implement pre-commit hook. Signed-off-by: Nicolas Dufresne Reviewed-by: Laurent Pinchart --- utils/checkstyle.py | 32 ++++++++++++++++++++++++++++++-- 1 file changed, 30 insertions(+), 2 deletions(-) diff --git a/utils/checkstyle.py b/utils/checkstyle.py index d928a5f..e7c3ae2 100755 --- a/utils/checkstyle.py +++ b/utils/checkstyle.py @@ -481,6 +481,21 @@ class Commit: stdout=subprocess.PIPE).stdout.decode('utf-8') +class StagedChanges(Commit): + def __init__(self): + Commit.__init__(self, '') + + def get_info(self): + ret = subprocess.run(['git', 'diff', '--staged', '--name-only'], + stdout=subprocess.PIPE).stdout.decode('utf-8') + return "Staged changes", ret.splitlines() + + def get_diff(self, top_level, filename): + return subprocess.run(['git', 'diff', '--staged', '--', + '%s/%s' % (top_level, filename)], + stdout=subprocess.PIPE).stdout.decode('utf-8') + + def check_file(top_level, commit, filename): # Extract the line numbers touched by the commit. diff = commit.get_diff(top_level, filename) @@ -611,7 +626,9 @@ def main(argv): parser = argparse.ArgumentParser() parser.add_argument('--formatter', '-f', type=str, choices=['astyle', 'clang-format'], help='Code formatter. Default to clang-format if not specified.') - parser.add_argument('revision_range', type=str, default='HEAD', nargs='?', + parser.add_argument('--staged', '-s', action='store_true', + help='Include the changes in the index. Defaults to False') + parser.add_argument('revision_range', type=str, default=None, nargs='?', help='Revision range (as defined by git rev-parse). Defaults to HEAD if not specified.') args = parser.parse_args(argv[1:]) @@ -646,7 +663,18 @@ def main(argv): if top_level is None: return 1 - commits = extract_commits(args.revision_range) + commits = [] + if args.staged: + commits.append(StagedChanges()) + + # If not --staged + if len(commits) == 0: + # And no revisions were passed, then default to HEAD + if not args.revision_range: + args.revision_range = 'HEAD' + + if args.revision_range: + commits += extract_commits(args.revision_range) issues = 0 for commit in commits: From patchwork Sat Jan 18 20:00:14 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Nicolas Dufresne X-Patchwork-Id: 2679 Return-Path: Received: from bhuna.collabora.co.uk (bhuna.collabora.co.uk [IPv6:2a00:1098:0:82:1000:25:2eeb:e3e3]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 8B704607B2 for ; Sat, 18 Jan 2020 21:02:54 +0100 (CET) Received: from nicolas-tpx395.localdomain (unknown [IPv6:2002:c0de:c115:0:66fc:8b:2a38:8313]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) (Authenticated sender: nicolas) by bhuna.collabora.co.uk (Postfix) with ESMTPSA id D1B5E2925A2; Sat, 18 Jan 2020 20:02:52 +0000 (GMT) From: Nicolas Dufresne To: libcamera-devel@lists.libcamera.org Cc: Nicolas Dufresne , =?utf-8?q?Niklas_S?= =?utf-8?b?w7ZkZXJsdW5k?= , Laurent Pinchart Date: Sat, 18 Jan 2020 15:00:14 -0500 Message-Id: <20200118200015.16531-6-nicolas@ndufresne.ca> X-Mailer: git-send-email 2.24.1 In-Reply-To: <20200118200015.16531-1-nicolas@ndufresne.ca> References: <20200118200015.16531-1-nicolas@ndufresne.ca> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v4 5/6] checkstyle: Add support for checking style on amendments X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 18 Jan 2020 20:02:54 -0000 From: Nicolas Dufresne This introduces a new argument "--amend" and a new special type of commit "Amendment". It will check the style of changes that are in the index combined with the changes of the last commit. So this is the changes that would be applied by "git commit --amend" hence the name of the argument. This is needed to implement pre-commit hook. Signed-off-by: Nicolas Dufresne Reviewed-by: Niklas Söderlund Reviewed-by: Laurent Pinchart --- utils/checkstyle.py | 24 ++++++++++++++++++++++++ 1 file changed, 24 insertions(+) diff --git a/utils/checkstyle.py b/utils/checkstyle.py index e7c3ae2..37f74c8 100755 --- a/utils/checkstyle.py +++ b/utils/checkstyle.py @@ -496,6 +496,26 @@ class StagedChanges(Commit): stdout=subprocess.PIPE).stdout.decode('utf-8') +class Amendment(StagedChanges): + def __init__(self): + StagedChanges.__init__(self) + + def get_info(self): + # Create a title using HEAD commit + ret = subprocess.run(['git', 'show', '--pretty=oneline', '--no-patch', 'HEAD'], + stdout=subprocess.PIPE).stdout.decode('utf-8') + title = 'Amendment of: ' + ret.splitlines()[0] + # Extract the list of modified files + ret = subprocess.run(['git', 'diff', '--staged', '--name-only', 'HEAD~'], + stdout=subprocess.PIPE).stdout.decode('utf-8') + return title, ret.splitlines() + + def get_diff(self, top_level, filename): + return subprocess.run(['git', 'diff', '--staged', 'HEAD~', '--', + '%s/%s' % (top_level, filename)], + stdout=subprocess.PIPE).stdout.decode('utf-8') + + def check_file(top_level, commit, filename): # Extract the line numbers touched by the commit. diff = commit.get_diff(top_level, filename) @@ -628,6 +648,8 @@ def main(argv): help='Code formatter. Default to clang-format if not specified.') parser.add_argument('--staged', '-s', action='store_true', help='Include the changes in the index. Defaults to False') + parser.add_argument('--amend', '-a', action='store_true', + help='Include changes in the index and the previous patch combined. Defaults to False') parser.add_argument('revision_range', type=str, default=None, nargs='?', help='Revision range (as defined by git rev-parse). Defaults to HEAD if not specified.') args = parser.parse_args(argv[1:]) @@ -666,6 +688,8 @@ def main(argv): commits = [] if args.staged: commits.append(StagedChanges()) + if args.amend: + commits.append(Amendment()) # If not --staged if len(commits) == 0: From patchwork Sat Jan 18 20:00:15 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nicolas Dufresne X-Patchwork-Id: 2680 Return-Path: Received: from bhuna.collabora.co.uk (bhuna.collabora.co.uk [IPv6:2a00:1098:0:82:1000:25:2eeb:e3e3]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id E752460455 for ; Sat, 18 Jan 2020 21:02:56 +0100 (CET) Received: from nicolas-tpx395.localdomain (unknown [IPv6:2002:c0de:c115:0:66fc:8b:2a38:8313]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) (Authenticated sender: nicolas) by bhuna.collabora.co.uk (Postfix) with ESMTPSA id 9B0052911C1; Sat, 18 Jan 2020 20:02:54 +0000 (GMT) From: Nicolas Dufresne To: libcamera-devel@lists.libcamera.org Cc: Nicolas Dufresne , Laurent Pinchart Date: Sat, 18 Jan 2020 15:00:15 -0500 Message-Id: <20200118200015.16531-7-nicolas@ndufresne.ca> X-Mailer: git-send-email 2.24.1 In-Reply-To: <20200118200015.16531-1-nicolas@ndufresne.ca> References: <20200118200015.16531-1-nicolas@ndufresne.ca> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v4 6/6] checkstyle: Add a pre-commit hook script X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 18 Jan 2020 20:02:57 -0000 From: Nicolas Dufresne This adds support for pre-commit hook workflow. In pre-commit hook we check the style on the changes currently staged or the combination of the index and the last commit if "git commit --amend" is being used. Signed-off-by: Nicolas Dufresne Reviewed-by: Laurent Pinchart --- utils/hooks/pre-commit | 18 ++++++++++++++++++ 1 file changed, 18 insertions(+) create mode 100755 utils/hooks/pre-commit diff --git a/utils/hooks/pre-commit b/utils/hooks/pre-commit new file mode 100755 index 0000000..47fcbc8 --- /dev/null +++ b/utils/hooks/pre-commit @@ -0,0 +1,18 @@ +#!/bin/sh + +# Execute the checkstyle script before committing any code. This will fail the +# commit in case of style issues, ensuring that the developer will notice them. +# The pre-commit hook can be bypassed with git commit -n to ignore selective +# changes. +# +# To utilise this hook, install this file with: +# cp utils/hooks/pre-commit .git/hooks/pre-commit + +if ps -ocommand= -p $PPID | grep -- "--amend" +then + args="--amend" +else + args="--staged" +fi + +./utils/checkstyle.py $args