From patchwork Fri Jan 17 19:17:28 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Nicolas Dufresne X-Patchwork-Id: 2661 Return-Path: Received: from bhuna.collabora.co.uk (bhuna.collabora.co.uk [IPv6:2a00:1098:0:82:1000:25:2eeb:e3e3]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 2D42660787 for ; Fri, 17 Jan 2020 20:17:53 +0100 (CET) Received: from nicolas-tpx395.localdomain (unknown [IPv6:2610:98:8005::127]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) (Authenticated sender: nicolas) by bhuna.collabora.co.uk (Postfix) with ESMTPSA id 7E4782949D4; Fri, 17 Jan 2020 19:17:52 +0000 (GMT) From: Nicolas Dufresne To: libcamera-devel@lists.libcamera.org Cc: Nicolas Dufresne , Kieran Bingham Date: Fri, 17 Jan 2020 14:17:28 -0500 Message-Id: <20200117191733.198897-2-nicolas@ndufresne.ca> X-Mailer: git-send-email 2.24.1 In-Reply-To: <20200117191733.198897-1-nicolas@ndufresne.ca> References: <20200117191733.198897-1-nicolas@ndufresne.ca> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v2 1/6] checkstyle: Move from pep8 to pycodestyle X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 17 Jan 2020 19:17:53 -0000 From: Nicolas Dufresne The tool has been renamed in 2016 to make it more obvious what it is doing. There is no other changes needed on our side. See https://github.com/PyCQA/pycodestyle/issues/466 Signed-off-by: Nicolas Dufresne Reviewed-by: Kieran Bingham Reviewed-by: Niklas Söderlund --- utils/checkstyle.py | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/utils/checkstyle.py b/utils/checkstyle.py index 93f0291..7edea25 100755 --- a/utils/checkstyle.py +++ b/utils/checkstyle.py @@ -321,10 +321,10 @@ class Pep8Checker(StyleChecker): data = ''.join(self.__content).encode('utf-8') try: - ret = subprocess.run(['pep8', '--ignore=E501', '-'], + ret = subprocess.run(['pycodestyle', '--ignore=E501', '-'], input=data, stdout=subprocess.PIPE) except FileNotFoundError: - issues.append(StyleIssue(0, None, "Please install pep8 to validate python additions")) + issues.append(StyleIssue(0, None, "Please install pycodestyle to validate python additions")) return issues results = ret.stdout.decode('utf-8').splitlines() From patchwork Fri Jan 17 19:17:29 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Nicolas Dufresne X-Patchwork-Id: 2662 Return-Path: Received: from bhuna.collabora.co.uk (bhuna.collabora.co.uk [IPv6:2a00:1098:0:82:1000:25:2eeb:e3e3]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 760EB6077B for ; Fri, 17 Jan 2020 20:17:57 +0100 (CET) Received: from nicolas-tpx395.localdomain (unknown [IPv6:2610:98:8005::127]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) (Authenticated sender: nicolas) by bhuna.collabora.co.uk (Postfix) with ESMTPSA id E20162949D4; Fri, 17 Jan 2020 19:17:56 +0000 (GMT) From: Nicolas Dufresne To: libcamera-devel@lists.libcamera.org Cc: Nicolas Dufresne Date: Fri, 17 Jan 2020 14:17:29 -0500 Message-Id: <20200117191733.198897-3-nicolas@ndufresne.ca> X-Mailer: git-send-email 2.24.1 In-Reply-To: <20200117191733.198897-1-nicolas@ndufresne.ca> References: <20200117191733.198897-1-nicolas@ndufresne.ca> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v2 2/6] checkstyle: Exit with 1 status if issues are found X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 17 Jan 2020 19:17:57 -0000 From: Nicolas Dufresne Makes the tool return 1 if there is any potential issues. This is needed when using this tool for pre-commit hook in order to abort the commit process. Signed-off-by: Nicolas Dufresne Reviewed-by: Niklas Söderlund Reviewed-by: Laurent Pinchart --- utils/checkstyle.py | 9 ++++++--- 1 file changed, 6 insertions(+), 3 deletions(-) mode change 100755 => 100644 utils/checkstyle.py diff --git a/utils/checkstyle.py b/utils/checkstyle.py old mode 100755 new mode 100644 index 7edea25..e7375b3 --- a/utils/checkstyle.py +++ b/utils/checkstyle.py @@ -541,7 +541,7 @@ def check_style(top_level, commit): files = [f for f in files if len([p for p in patterns if fnmatch.fnmatch(os.path.basename(f), p)])] if len(files) == 0: print("Commit doesn't touch source files, skipping") - return + return 0 issues = 0 for f in files: @@ -554,6 +554,8 @@ def check_style(top_level, commit): print("%u potential style %s detected, please review" % \ (issues, 'issue' if issues == 1 else 'issues')) + return issues + def extract_revlist(revs): """Extract a list of commits on which to operate from a revision or revision @@ -632,11 +634,12 @@ def main(argv): revlist = extract_revlist(args.revision_range) + issues = 0 for commit in revlist: - check_style(top_level, commit) + issues += check_style(top_level, commit) print('') - return 0 + return min(issues, 1) if __name__ == '__main__': From patchwork Fri Jan 17 19:17:30 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Nicolas Dufresne X-Patchwork-Id: 2663 Return-Path: Received: from bhuna.collabora.co.uk (bhuna.collabora.co.uk [46.235.227.227]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 8B9FA60787 for ; Fri, 17 Jan 2020 20:17:58 +0100 (CET) Received: from nicolas-tpx395.localdomain (unknown [IPv6:2610:98:8005::127]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) (Authenticated sender: nicolas) by bhuna.collabora.co.uk (Postfix) with ESMTPSA id 15DC52949D4; Fri, 17 Jan 2020 19:17:58 +0000 (GMT) From: Nicolas Dufresne To: libcamera-devel@lists.libcamera.org Cc: Nicolas Dufresne Date: Fri, 17 Jan 2020 14:17:30 -0500 Message-Id: <20200117191733.198897-4-nicolas@ndufresne.ca> X-Mailer: git-send-email 2.24.1 In-Reply-To: <20200117191733.198897-1-nicolas@ndufresne.ca> References: <20200117191733.198897-1-nicolas@ndufresne.ca> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v2 3/6] checkstyle: Introduce a Commit class X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 17 Jan 2020 19:17:58 -0000 From: Nicolas Dufresne This introduce a Commit class used in the final revlist list. All the git command are moved into that class. This class will be used to introduce new type of commit (index and amendment) needed to implement pre-commit hook support. Signed-off-by: Nicolas Dufresne Reviewed-by: Niklas Söderlund Reviewed-by: Laurent Pinchart --- utils/checkstyle.py | 43 +++++++++++++++++++++++++++++-------------- 1 file changed, 29 insertions(+), 14 deletions(-) diff --git a/utils/checkstyle.py b/utils/checkstyle.py index e7375b3..fb865c8 100644 --- a/utils/checkstyle.py +++ b/utils/checkstyle.py @@ -458,12 +458,34 @@ class StripTrailingSpaceFormatter(Formatter): # Style checking # +class Commit: + commit = None + + def __init__(self, commit): + self.commit = commit + + def get_info(self, top_level): + # Get the commit title and list of files. + ret = subprocess.run(['git', 'show', '--pretty=oneline', '--name-only', + self.commit], + stdout=subprocess.PIPE).stdout.decode('utf-8') + files = ret.splitlines() + return files[0], files[1:] + + def get_diff(self, top_level, filename): + return subprocess.run(['git', 'diff', '%s~..%s' % (self.commit, self.commit), + '--', '%s/%s' % (top_level, filename)], + stdout=subprocess.PIPE).stdout.decode('utf-8') + + def get_file(self, filename): + return subprocess.run(['git', 'show', '%s:%s' % (self.commit, filename)], + stdout=subprocess.PIPE).stdout.decode('utf-8') + + def check_file(top_level, commit, filename): # Extract the line numbers touched by the commit. - diff = subprocess.run(['git', 'diff', '%s~..%s' % (commit, commit), '--', - '%s/%s' % (top_level, filename)], - stdout=subprocess.PIPE).stdout - diff = diff.decode('utf-8').splitlines(True) + diff = commit.get_diff(top_level, filename) + diff = diff.splitlines(True) commit_diff = parse_diff(diff) lines = [] @@ -476,9 +498,7 @@ def check_file(top_level, commit, filename): # Format the file after the commit with all formatters and compute the diff # between the unformatted and formatted contents. - after = subprocess.run(['git', 'show', '%s:%s' % (commit, filename)], - stdout=subprocess.PIPE).stdout - after = after.decode('utf-8') + after = commit.get_file(filename) formatted = after for formatter in Formatter.formatters(filename): @@ -522,12 +542,7 @@ def check_file(top_level, commit, filename): def check_style(top_level, commit): - # Get the commit title and list of files. - ret = subprocess.run(['git', 'show', '--pretty=oneline','--name-only', commit], - stdout=subprocess.PIPE) - files = ret.stdout.decode('utf-8').splitlines() - title = files[0] - files = files[1:] + title, files = commit.get_info(top_level) separator = '-' * len(title) print(separator) @@ -576,7 +591,7 @@ def extract_revlist(revs): revlist = ret.stdout.decode('utf-8').splitlines() revlist.reverse() - return revlist + return [Commit(x) for x in revlist] def git_top_level(): From patchwork Fri Jan 17 19:17:31 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nicolas Dufresne X-Patchwork-Id: 2664 Return-Path: Received: from bhuna.collabora.co.uk (bhuna.collabora.co.uk [46.235.227.227]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id A10A260783 for ; Fri, 17 Jan 2020 20:17:59 +0100 (CET) Received: from nicolas-tpx395.localdomain (unknown [IPv6:2610:98:8005::127]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) (Authenticated sender: nicolas) by bhuna.collabora.co.uk (Postfix) with ESMTPSA id 2CBEA2949D5; Fri, 17 Jan 2020 19:17:59 +0000 (GMT) From: Nicolas Dufresne To: libcamera-devel@lists.libcamera.org Cc: Nicolas Dufresne Date: Fri, 17 Jan 2020 14:17:31 -0500 Message-Id: <20200117191733.198897-5-nicolas@ndufresne.ca> X-Mailer: git-send-email 2.24.1 In-Reply-To: <20200117191733.198897-1-nicolas@ndufresne.ca> References: <20200117191733.198897-1-nicolas@ndufresne.ca> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v2 4/6] checkstyle: Add support for checking style on indexed changes X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 17 Jan 2020 19:18:00 -0000 From: Nicolas Dufresne This introduce a new command line "--staged" and a new special type of commit "Index". It will check the style of changes that are in the index, so the changes that would be committed by "git commit". "--staged" was chosen to match with "git diff --staged" command line. Other valid name could have been "--index" or "--cached". This was my personal preference, alias can be added later. Note that we must not confuse this with working tree changes, as these changes are not picked by "git commit". This feature is needed to implement pre-commit hook. Signed-off-by: Nicolas Dufresne --- utils/checkstyle.py | 34 ++++++++++++++++++++++++++++++++-- 1 file changed, 32 insertions(+), 2 deletions(-) diff --git a/utils/checkstyle.py b/utils/checkstyle.py index fb865c8..8e456cd 100644 --- a/utils/checkstyle.py +++ b/utils/checkstyle.py @@ -482,6 +482,25 @@ class Commit: stdout=subprocess.PIPE).stdout.decode('utf-8') +class Index(Commit): + def __init__(self): + Commit.__init__(self, None) + + def get_info(self, top_level): + ret = subprocess.run(['git', 'diff', '--staged', '--name-only'], + stdout=subprocess.PIPE).stdout.decode('utf-8') + return "Staged changes", ret.splitlines() + + def get_diff(self, top_level, filename): + return subprocess.run(['git', 'diff', '--staged', '--', + '%s/%s' % (top_level, filename)], + stdout=subprocess.PIPE).stdout.decode('utf-8') + + def get_file(self, filename): + return subprocess.run(['git', 'show', ':%s' % (filename)], + stdout=subprocess.PIPE).stdout.decode('utf-8') + + def check_file(top_level, commit, filename): # Extract the line numbers touched by the commit. diff = commit.get_diff(top_level, filename) @@ -612,7 +631,9 @@ def main(argv): parser = argparse.ArgumentParser() parser.add_argument('--formatter', '-f', type=str, choices=['astyle', 'clang-format'], help='Code formatter. Default to clang-format if not specified.') - parser.add_argument('revision_range', type=str, default='HEAD', nargs='?', + parser.add_argument('--staged', '-s', action='store_true', + help='Include the changes in the index. Defaults to False') + parser.add_argument('revision_range', type=str, default=None, nargs='?', help='Revision range (as defined by git rev-parse). Defaults to HEAD if not specified.') args = parser.parse_args(argv[1:]) @@ -647,7 +668,16 @@ def main(argv): if top_level is None: return 1 - revlist = extract_revlist(args.revision_range) + revlist = [] + if args.staged: + revlist.append(Index()) + + # If nothing of --staged or --amend was passed, defaults to HEAD + if len(revlist) == 0 and not args.revision_range: + args.revision_range = 'HEAD' + + if args.revision_range: + revlist += extract_revlist(args.revision_range) issues = 0 for commit in revlist: From patchwork Fri Jan 17 19:17:32 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Nicolas Dufresne X-Patchwork-Id: 2665 Return-Path: Received: from bhuna.collabora.co.uk (bhuna.collabora.co.uk [IPv6:2a00:1098:0:82:1000:25:2eeb:e3e3]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id C510F60791 for ; Fri, 17 Jan 2020 20:18:00 +0100 (CET) Received: from nicolas-tpx395.localdomain (unknown [IPv6:2610:98:8005::127]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) (Authenticated sender: nicolas) by bhuna.collabora.co.uk (Postfix) with ESMTPSA id 3BE5E2949D4; Fri, 17 Jan 2020 19:18:00 +0000 (GMT) From: Nicolas Dufresne To: libcamera-devel@lists.libcamera.org Cc: Nicolas Dufresne Date: Fri, 17 Jan 2020 14:17:32 -0500 Message-Id: <20200117191733.198897-6-nicolas@ndufresne.ca> X-Mailer: git-send-email 2.24.1 In-Reply-To: <20200117191733.198897-1-nicolas@ndufresne.ca> References: <20200117191733.198897-1-nicolas@ndufresne.ca> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v2 5/6] checkstyle: Add support for checking style on amendments X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 17 Jan 2020 19:18:01 -0000 From: Nicolas Dufresne This introduce a new argument "--amend" and a new special type of commit "Amendment". It will check the style of changes that are in the index combined with the changes of the last commit. So this is the changes that would be applied by "git commit --amend" hence the name of the argument. This is needed to implement pre-commit hook. Signed-off-by: Nicolas Dufresne Reviewed-by: Niklas Söderlund --- utils/checkstyle.py | 24 ++++++++++++++++++++++++ 1 file changed, 24 insertions(+) mode change 100644 => 100755 utils/checkstyle.py diff --git a/utils/checkstyle.py b/utils/checkstyle.py old mode 100644 new mode 100755 index 8e456cd..7c2ce00 --- a/utils/checkstyle.py +++ b/utils/checkstyle.py @@ -501,6 +501,26 @@ class Index(Commit): stdout=subprocess.PIPE).stdout.decode('utf-8') +class Amendment(Index): + def __init__(self): + Commit.__init__(self, None) + + def det_info(self, top_level): + # Create a title using HEAD commit + ret = subprocess.run(['git', 'show', '--pretty=oneline', '--name-only', 'HEAD'], + stdout=subprocess.PIPE).stdout.decode('utf-8') + title = 'Amendment of: ' + ret.splitlines()[0] + # Extract the list of modifier files + ret = subprocess.run(['git', 'diff', '--staged', '--name-only', 'HEAD~'], + stdout=subprocess.PIPE).stdout.decode('utf-8') + return title, ret.splitlines() + + def get_diff(self, top_level, filename): + return subprocess.run(['git', 'diff', '--staged', 'HEAD~', '--', + '%s/%s' % (top_level, filename)], + stdout=subprocess.PIPE).stdout.decode('utf-8') + + def check_file(top_level, commit, filename): # Extract the line numbers touched by the commit. diff = commit.get_diff(top_level, filename) @@ -633,6 +653,8 @@ def main(argv): help='Code formatter. Default to clang-format if not specified.') parser.add_argument('--staged', '-s', action='store_true', help='Include the changes in the index. Defaults to False') + parser.add_argument('--amend', '-a', action='store_true', + help='Includes changes in the index and the previous patch combined. Defaults to False') parser.add_argument('revision_range', type=str, default=None, nargs='?', help='Revision range (as defined by git rev-parse). Defaults to HEAD if not specified.') args = parser.parse_args(argv[1:]) @@ -671,6 +693,8 @@ def main(argv): revlist = [] if args.staged: revlist.append(Index()) + if args.amend: + revlist.append(Amendment()) # If nothing of --staged or --amend was passed, defaults to HEAD if len(revlist) == 0 and not args.revision_range: From patchwork Fri Jan 17 19:17:33 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nicolas Dufresne X-Patchwork-Id: 2666 Return-Path: Received: from bhuna.collabora.co.uk (bhuna.collabora.co.uk [IPv6:2a00:1098:0:82:1000:25:2eeb:e3e3]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id C2EAB6078C for ; Fri, 17 Jan 2020 20:18:01 +0100 (CET) Received: from nicolas-tpx395.localdomain (unknown [IPv6:2610:98:8005::127]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) (Authenticated sender: nicolas) by bhuna.collabora.co.uk (Postfix) with ESMTPSA id 39D812949D4; Fri, 17 Jan 2020 19:18:01 +0000 (GMT) From: Nicolas Dufresne To: libcamera-devel@lists.libcamera.org Cc: Nicolas Dufresne Date: Fri, 17 Jan 2020 14:17:33 -0500 Message-Id: <20200117191733.198897-7-nicolas@ndufresne.ca> X-Mailer: git-send-email 2.24.1 In-Reply-To: <20200117191733.198897-1-nicolas@ndufresne.ca> References: <20200117191733.198897-1-nicolas@ndufresne.ca> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v2 6/6] checkstyle: Add a pre-commit hook script X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 17 Jan 2020 19:18:02 -0000 From: Nicolas Dufresne This adds support for pre-commit hook workflow. In pre-commit hook we check the style on the changes currently staged or the combination of the index and the last commit if "git commit --amend" is being used. Signed-off-by: Nicolas Dufresne --- utils/hooks/pre-commit | 17 +++++++++++++++++ 1 file changed, 17 insertions(+) create mode 100755 utils/hooks/pre-commit diff --git a/utils/hooks/pre-commit b/utils/hooks/pre-commit new file mode 100755 index 0000000..be2f8b3 --- /dev/null +++ b/utils/hooks/pre-commit @@ -0,0 +1,17 @@ +#!/bin/sh + +# Execute the checkstyle script after committing any code. This allows the +# commit to succeed, but ensures that the developer is aware of any potential +# issues immediately, and can resolve them and fix rapidly with: +# git commit --amend +# +# To utilise this hook, install this file with: +# cp utils/hooks/post-commit .git/hooks/post-commit + +args="--staged" +if ps -ocommand= -p $PPID | grep -- "--amend" +then + commit="--amend" +fi + +./utils/checkstyle.py $args