From patchwork Thu Dec 24 12:28:48 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Laurent Pinchart X-Patchwork-Id: 10737 X-Patchwork-Delegate: laurent.pinchart@ideasonboard.com Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 59ABFC0F1A for ; Thu, 24 Dec 2020 12:29:11 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 6DA63619AF; Thu, 24 Dec 2020 13:29:09 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="wiG++nfE"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 6B23460528 for ; Thu, 24 Dec 2020 13:29:08 +0100 (CET) Received: from pendragon.lan (62-78-145-57.bb.dnainternet.fi [62.78.145.57]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id F15CFDFC for ; Thu, 24 Dec 2020 13:29:07 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1608812948; bh=zHoIUWSLaD0q02RhJWnqD6Uw8yZ235rdSFwoTc1c/9c=; h=From:To:Subject:Date:In-Reply-To:References:From; b=wiG++nfEEoEshQNaWjjS6SJGstKvvWLPdmyTKDfamWPLlTr4kq+JJk/K3mwpvzAp0 61sBdSC7fvCyL3ItHdyYqf9XOlEE//DEfXQOvSn7P+FV9LCmfx6ZBaMI3VY7H0deFi Tgau4rLTUviy89VksLkdhFuXGL6K38RHVeR3vjyc= From: Laurent Pinchart To: libcamera-devel@lists.libcamera.org Date: Thu, 24 Dec 2020 14:28:48 +0200 Message-Id: <20201224122855.22200-2-laurent.pinchart@ideasonboard.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20201224122855.22200-1-laurent.pinchart@ideasonboard.com> References: <20201224122855.22200-1-laurent.pinchart@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH 1/8] utils: checkstyle.py: Drop arguments to super() when possible X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" The super() call is a shortcut syntax for super(__class__, ). Drom the arguments when they match the default. Signed-off-by: Laurent Pinchart Reviewed-by: Niklas Söderlund Reviewed-by: Paul Elder --- utils/checkstyle.py | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/utils/checkstyle.py b/utils/checkstyle.py index 7225cac47a4e..c04bf3850dcd 100755 --- a/utils/checkstyle.py +++ b/utils/checkstyle.py @@ -198,7 +198,7 @@ _style_checkers = [] class StyleCheckerRegistry(type): def __new__(cls, clsname, bases, attrs): - newclass = super(StyleCheckerRegistry, cls).__new__(cls, clsname, bases, attrs) + newclass = super().__new__(cls, clsname, bases, attrs) if clsname != 'StyleChecker': _style_checkers.append(newclass) return newclass @@ -387,7 +387,7 @@ _formatters = [] class FormatterRegistry(type): def __new__(cls, clsname, bases, attrs): - newclass = super(FormatterRegistry, cls).__new__(cls, clsname, bases, attrs) + newclass = super().__new__(cls, clsname, bases, attrs) if clsname != 'Formatter': _formatters.append(newclass) return newclass From patchwork Thu Dec 24 12:28:49 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Laurent Pinchart X-Patchwork-Id: 10738 X-Patchwork-Delegate: laurent.pinchart@ideasonboard.com Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 08C5CC0F1A for ; Thu, 24 Dec 2020 12:29:13 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id CCD78615B0; Thu, 24 Dec 2020 13:29:12 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="fVnESeqV"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id AC52160528 for ; Thu, 24 Dec 2020 13:29:08 +0100 (CET) Received: from pendragon.lan (62-78-145-57.bb.dnainternet.fi [62.78.145.57]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 4CC2FA1D for ; Thu, 24 Dec 2020 13:29:08 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1608812948; bh=9n2XxMWv5HmxGxi11lL7HODHwJPLMDLZV43LTTuKVIM=; h=From:To:Subject:Date:In-Reply-To:References:From; b=fVnESeqVkqv5Ffmx7TYDJ/Ys8uW1AQaMz6K9dmUNE8FuIjPYgt4AVtBJsWrUbzxeR FWQGfddwHOwAiBIgghhhdWAa/psdw/XhoC0OODOvflyv7L8OLeI9OAKFUuQ6ZneTBD m0MOakVxlkGc5SlQ7di/ZR2vbXzrMqwYnjffwPDo= From: Laurent Pinchart To: libcamera-devel@lists.libcamera.org Date: Thu, 24 Dec 2020 14:28:49 +0200 Message-Id: <20201224122855.22200-3-laurent.pinchart@ideasonboard.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20201224122855.22200-1-laurent.pinchart@ideasonboard.com> References: <20201224122855.22200-1-laurent.pinchart@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH 2/8] utils: checkstyle.py: Factor out automatic class registry X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" The style checkers and formatters duplicate automatic class registry code. Factor it out to a common ClassRegistry helper class. The list of subclasses is moved to a class member variable of the auto-registered base class type. Signed-off-by: Laurent Pinchart Reviewed-by: Niklas Söderlund --- utils/checkstyle.py | 45 ++++++++++++++++++++------------------------- 1 file changed, 20 insertions(+), 25 deletions(-) diff --git a/utils/checkstyle.py b/utils/checkstyle.py index c04bf3850dcd..76267d5ea764 100755 --- a/utils/checkstyle.py +++ b/utils/checkstyle.py @@ -190,21 +190,25 @@ def parse_diff(diff): return hunks +# ------------------------------------------------------------------------------ +# Helpers +# + +class ClassRegistry(type): + def __new__(cls, clsname, bases, attrs): + newclass = super().__new__(cls, clsname, bases, attrs) + if bases: + bases[0].subclasses.append(newclass) + return newclass + + # ------------------------------------------------------------------------------ # Style Checkers # -_style_checkers = [] +class StyleChecker(metaclass=ClassRegistry): + subclasses = [] -class StyleCheckerRegistry(type): - def __new__(cls, clsname, bases, attrs): - newclass = super().__new__(cls, clsname, bases, attrs) - if clsname != 'StyleChecker': - _style_checkers.append(newclass) - return newclass - - -class StyleChecker(metaclass=StyleCheckerRegistry): def __init__(self): pass @@ -213,7 +217,7 @@ class StyleChecker(metaclass=StyleCheckerRegistry): # @classmethod def checkers(cls, filename): - for checker in _style_checkers: + for checker in cls.subclasses: if checker.supports(filename): yield checker @@ -227,7 +231,7 @@ class StyleChecker(metaclass=StyleCheckerRegistry): @classmethod def all_patterns(cls): patterns = set() - for checker in _style_checkers: + for checker in cls.subclasses: patterns.update(checker.patterns) return patterns @@ -383,18 +387,9 @@ class ShellChecker(StyleChecker): # Formatters # -_formatters = [] - -class FormatterRegistry(type): - def __new__(cls, clsname, bases, attrs): - newclass = super().__new__(cls, clsname, bases, attrs) - if clsname != 'Formatter': - _formatters.append(newclass) - return newclass - - -class Formatter(metaclass=FormatterRegistry): +class Formatter(metaclass=ClassRegistry): enabled = True + subclasses = [] def __init__(self): pass @@ -404,7 +399,7 @@ class Formatter(metaclass=FormatterRegistry): # @classmethod def formatters(cls, filename): - for formatter in _formatters: + for formatter in cls.subclasses: if not cls.enabled: continue if formatter.supports(filename): @@ -422,7 +417,7 @@ class Formatter(metaclass=FormatterRegistry): @classmethod def all_patterns(cls): patterns = set() - for formatter in _formatters: + for formatter in cls.subclasses: if not cls.enabled: continue patterns.update(formatter.patterns) From patchwork Thu Dec 24 12:28:50 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Laurent Pinchart X-Patchwork-Id: 10739 X-Patchwork-Delegate: laurent.pinchart@ideasonboard.com Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 9D090C0F1A for ; Thu, 24 Dec 2020 12:29:13 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 4C6F8615AF; Thu, 24 Dec 2020 13:29:13 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="I4h+YFIf"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 0978F60528 for ; Thu, 24 Dec 2020 13:29:09 +0100 (CET) Received: from pendragon.lan (62-78-145-57.bb.dnainternet.fi [62.78.145.57]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 9ED28DFC for ; Thu, 24 Dec 2020 13:29:08 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1608812948; bh=6yCYwRzf5QvfKinGmTRNPvWOY+FQEs89TQ6h/scAPW8=; h=From:To:Subject:Date:In-Reply-To:References:From; b=I4h+YFIfEuBm4HCfMUVTqy6Qhlbe3L0juQEZ01HhKtaWBx71FrVokPToMtNQGh44D +eBHQZjtCOkFfoCjDE16+EZs+O4TMk7BXscTAzMWZd67TR2JSoW/6cW6lHdypM/DQe 3hd56yZHOSZSDlCwqT32F+vtVKMHyyAnMmTWF8zw= From: Laurent Pinchart To: libcamera-devel@lists.libcamera.org Date: Thu, 24 Dec 2020 14:28:50 +0200 Message-Id: <20201224122855.22200-4-laurent.pinchart@ideasonboard.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20201224122855.22200-1-laurent.pinchart@ideasonboard.com> References: <20201224122855.22200-1-laurent.pinchart@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH 3/8] utils: checkstyle.py: Move commit handling to a separate section X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" To prepare for checkers that operate directly on commits, move the related classes to a separate section. No functional change is included. Signed-off-by: Laurent Pinchart Reviewed-by: Niklas Söderlund Reviewed-by: Paul Elder --- utils/checkstyle.py | 120 +++++++++++++++++++++++--------------------- 1 file changed, 62 insertions(+), 58 deletions(-) diff --git a/utils/checkstyle.py b/utils/checkstyle.py index 76267d5ea764..4ba65e5a4ff1 100755 --- a/utils/checkstyle.py +++ b/utils/checkstyle.py @@ -190,6 +190,68 @@ def parse_diff(diff): return hunks +# ------------------------------------------------------------------------------ +# Commit, Staged Changes & Amendments +# + +class Commit: + def __init__(self, commit): + self.commit = commit + + def get_info(self): + # Get the commit title and list of files. + ret = subprocess.run(['git', 'show', '--pretty=oneline', '--name-only', + self.commit], + stdout=subprocess.PIPE).stdout.decode('utf-8') + files = ret.splitlines() + # Returning title and files list as a tuple + return files[0], files[1:] + + def get_diff(self, top_level, filename): + return subprocess.run(['git', 'diff', '%s~..%s' % (self.commit, self.commit), + '--', '%s/%s' % (top_level, filename)], + stdout=subprocess.PIPE).stdout.decode('utf-8') + + def get_file(self, filename): + return subprocess.run(['git', 'show', '%s:%s' % (self.commit, filename)], + stdout=subprocess.PIPE).stdout.decode('utf-8') + + +class StagedChanges(Commit): + def __init__(self): + Commit.__init__(self, '') + + def get_info(self): + ret = subprocess.run(['git', 'diff', '--staged', '--name-only'], + stdout=subprocess.PIPE).stdout.decode('utf-8') + return "Staged changes", ret.splitlines() + + def get_diff(self, top_level, filename): + return subprocess.run(['git', 'diff', '--staged', '--', + '%s/%s' % (top_level, filename)], + stdout=subprocess.PIPE).stdout.decode('utf-8') + + +class Amendment(StagedChanges): + def __init__(self): + StagedChanges.__init__(self) + + def get_info(self): + # Create a title using HEAD commit + ret = subprocess.run(['git', 'show', '--pretty=oneline', '--no-patch'], + stdout=subprocess.PIPE).stdout.decode('utf-8') + title = 'Amendment of ' + ret.strip() + # Extract the list of modified files + ret = subprocess.run(['git', 'diff', '--staged', '--name-only', 'HEAD~'], + stdout=subprocess.PIPE).stdout.decode('utf-8') + return title, ret.splitlines() + + def get_diff(self, top_level, filename): + return subprocess.run(['git', 'diff', '--staged', 'HEAD~', '--', + '%s/%s' % (top_level, filename)], + stdout=subprocess.PIPE).stdout.decode('utf-8') + + # ------------------------------------------------------------------------------ # Helpers # @@ -564,64 +626,6 @@ class StripTrailingSpaceFormatter(Formatter): # Style checking # -class Commit: - def __init__(self, commit): - self.commit = commit - - def get_info(self): - # Get the commit title and list of files. - ret = subprocess.run(['git', 'show', '--pretty=oneline', '--name-only', - self.commit], - stdout=subprocess.PIPE).stdout.decode('utf-8') - files = ret.splitlines() - # Returning title and files list as a tuple - return files[0], files[1:] - - def get_diff(self, top_level, filename): - return subprocess.run(['git', 'diff', '%s~..%s' % (self.commit, self.commit), - '--', '%s/%s' % (top_level, filename)], - stdout=subprocess.PIPE).stdout.decode('utf-8') - - def get_file(self, filename): - return subprocess.run(['git', 'show', '%s:%s' % (self.commit, filename)], - stdout=subprocess.PIPE).stdout.decode('utf-8') - - -class StagedChanges(Commit): - def __init__(self): - Commit.__init__(self, '') - - def get_info(self): - ret = subprocess.run(['git', 'diff', '--staged', '--name-only'], - stdout=subprocess.PIPE).stdout.decode('utf-8') - return "Staged changes", ret.splitlines() - - def get_diff(self, top_level, filename): - return subprocess.run(['git', 'diff', '--staged', '--', - '%s/%s' % (top_level, filename)], - stdout=subprocess.PIPE).stdout.decode('utf-8') - - -class Amendment(StagedChanges): - def __init__(self): - StagedChanges.__init__(self) - - def get_info(self): - # Create a title using HEAD commit - ret = subprocess.run(['git', 'show', '--pretty=oneline', '--no-patch'], - stdout=subprocess.PIPE).stdout.decode('utf-8') - title = 'Amendment of ' + ret.strip() - # Extract the list of modified files - ret = subprocess.run(['git', 'diff', '--staged', '--name-only', 'HEAD~'], - stdout=subprocess.PIPE).stdout.decode('utf-8') - return title, ret.splitlines() - - def get_diff(self, top_level, filename): - return subprocess.run(['git', 'diff', '--staged', 'HEAD~', '--', - '%s/%s' % (top_level, filename)], - stdout=subprocess.PIPE).stdout.decode('utf-8') - - def check_file(top_level, commit, filename): # Extract the line numbers touched by the commit. diff = commit.get_diff(top_level, filename) From patchwork Thu Dec 24 12:28:51 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Laurent Pinchart X-Patchwork-Id: 10740 X-Patchwork-Delegate: laurent.pinchart@ideasonboard.com Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 3FFBEC0F1A for ; Thu, 24 Dec 2020 12:29:14 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id E7C9060526; Thu, 24 Dec 2020 13:29:13 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="dyDxFH9S"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 58478615D5 for ; Thu, 24 Dec 2020 13:29:09 +0100 (CET) Received: from pendragon.lan (62-78-145-57.bb.dnainternet.fi [62.78.145.57]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id ED680A1D for ; Thu, 24 Dec 2020 13:29:08 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1608812949; bh=z4u4x2raluKiHJaKWTFtx/4Qil1hDuWMhc/94oYSOn0=; h=From:To:Subject:Date:In-Reply-To:References:From; b=dyDxFH9Sn/WkhlM2NEnKSA0W95pbQyOqhS+fUp2bk0KwrPq7XL4XsLQxhujbPaWkE 23oHvQd6ltcN9Jv1wtnLqg9nnsJevP66g5EeyeTUd1PzYpPdW2b+cnSCcur68155R6 EpMlv3e5DFFaY9lP1BkfOiGsfv+pKG7tNQUsrkok= From: Laurent Pinchart To: libcamera-devel@lists.libcamera.org Date: Thu, 24 Dec 2020 14:28:51 +0200 Message-Id: <20201224122855.22200-5-laurent.pinchart@ideasonboard.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20201224122855.22200-1-laurent.pinchart@ideasonboard.com> References: <20201224122855.22200-1-laurent.pinchart@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH 4/8] utils: checkstyle.py: Make title and files properties of commit class X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Make the API of the Commit class more explicit by exposing the title and files as properties instead of through a get_info() method. Signed-off-by: Laurent Pinchart Reviewed-by: Niklas Söderlund Reviewed-by: Paul Elder --- utils/checkstyle.py | 34 +++++++++++++++++++++------------- 1 file changed, 21 insertions(+), 13 deletions(-) diff --git a/utils/checkstyle.py b/utils/checkstyle.py index 4ba65e5a4ff1..77e635dc5154 100755 --- a/utils/checkstyle.py +++ b/utils/checkstyle.py @@ -197,15 +197,24 @@ def parse_diff(diff): class Commit: def __init__(self, commit): self.commit = commit + self.__parse() - def get_info(self): + def __parse(self): # Get the commit title and list of files. ret = subprocess.run(['git', 'show', '--pretty=oneline', '--name-only', self.commit], stdout=subprocess.PIPE).stdout.decode('utf-8') files = ret.splitlines() - # Returning title and files list as a tuple - return files[0], files[1:] + self.__files = files[1:] + self.__title = files[0] + + @property + def files(self): + return self.__files + + @property + def title(self): + return self.__title def get_diff(self, top_level, filename): return subprocess.run(['git', 'diff', '%s~..%s' % (self.commit, self.commit), @@ -221,10 +230,11 @@ class StagedChanges(Commit): def __init__(self): Commit.__init__(self, '') - def get_info(self): + def __parse(self): ret = subprocess.run(['git', 'diff', '--staged', '--name-only'], stdout=subprocess.PIPE).stdout.decode('utf-8') - return "Staged changes", ret.splitlines() + self.__title = "Staged changes" + self.__files = ret.splitlines() def get_diff(self, top_level, filename): return subprocess.run(['git', 'diff', '--staged', '--', @@ -236,15 +246,15 @@ class Amendment(StagedChanges): def __init__(self): StagedChanges.__init__(self) - def get_info(self): + def __parse(self): # Create a title using HEAD commit ret = subprocess.run(['git', 'show', '--pretty=oneline', '--no-patch'], stdout=subprocess.PIPE).stdout.decode('utf-8') - title = 'Amendment of ' + ret.strip() + self.__title = 'Amendment of ' + ret.strip() # Extract the list of modified files ret = subprocess.run(['git', 'diff', '--staged', '--name-only', 'HEAD~'], stdout=subprocess.PIPE).stdout.decode('utf-8') - return title, ret.splitlines() + self.__files = ret.splitlines() def get_diff(self, top_level, filename): return subprocess.run(['git', 'diff', '--staged', 'HEAD~', '--', @@ -686,18 +696,16 @@ def check_file(top_level, commit, filename): def check_style(top_level, commit): - title, files = commit.get_info() - - separator = '-' * len(title) + separator = '-' * len(commit.title) print(separator) - print(title) + print(commit.title) print(separator) # Filter out files we have no checker for. patterns = set() patterns.update(StyleChecker.all_patterns()) patterns.update(Formatter.all_patterns()) - files = [f for f in files if len([p for p in patterns if fnmatch.fnmatch(os.path.basename(f), p)])] + files = [f for f in commit.files if len([p for p in patterns if fnmatch.fnmatch(os.path.basename(f), p)])] if len(files) == 0: print("Commit doesn't touch source files, skipping") return 0 From patchwork Thu Dec 24 12:28:52 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Laurent Pinchart X-Patchwork-Id: 10741 X-Patchwork-Delegate: laurent.pinchart@ideasonboard.com Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id B6FC9C0F1A for ; Thu, 24 Dec 2020 12:29:14 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 740E361FF6; Thu, 24 Dec 2020 13:29:14 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="V3SzuRIV"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 9E56F61FF3 for ; Thu, 24 Dec 2020 13:29:09 +0100 (CET) Received: from pendragon.lan (62-78-145-57.bb.dnainternet.fi [62.78.145.57]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 485B1DFC for ; Thu, 24 Dec 2020 13:29:09 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1608812949; bh=uKANu3/kKLScJOKlHXIhE61UJF0R/3MmN5xzJ/AGvNE=; h=From:To:Subject:Date:In-Reply-To:References:From; b=V3SzuRIVHIgOZbx6iHyyYoJ/q0Ps8/1QSCmF3Zvk+juNKk7/QQRASYnSlaC2SvC5S F6G/omlRrCDTbd6EptmGfHcVa3XXOvgPlEHiOoba1hVe2MHMDnHXot0vkKVFL88Y2M AVtBjEXq+cPAyDngXciOC21sJHuuREJch9Vo2Wn4= From: Laurent Pinchart To: libcamera-devel@lists.libcamera.org Date: Thu, 24 Dec 2020 14:28:52 +0200 Message-Id: <20201224122855.22200-6-laurent.pinchart@ideasonboard.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20201224122855.22200-1-laurent.pinchart@ideasonboard.com> References: <20201224122855.22200-1-laurent.pinchart@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH 5/8] utils: checkstyle.py: Add ability to filter files by status in a commit X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" A commit can perform different operations on a file. Record the file status (added, modified, renamed, deleted, ...) and add the ability to filter files by status when listing the files touched by a commit. Signed-off-by: Laurent Pinchart Reviewed-by: Niklas Söderlund --- utils/checkstyle.py | 39 +++++++++++++++++++++++++++++---------- 1 file changed, 29 insertions(+), 10 deletions(-) diff --git a/utils/checkstyle.py b/utils/checkstyle.py index 77e635dc5154..07e896d9819f 100755 --- a/utils/checkstyle.py +++ b/utils/checkstyle.py @@ -194,6 +194,26 @@ def parse_diff(diff): # Commit, Staged Changes & Amendments # +class CommitFile: + def __init__(self, name): + info = name.split() + self.__status = info[0][0] + + # For renamed files, store the new name + if self.__status == 'R': + self.__filename = info[2] + else: + self.__filename = info[1] + + @property + def filename(self): + return self.__filename + + @property + def status(self): + return self.__status + + class Commit: def __init__(self, commit): self.commit = commit @@ -201,16 +221,15 @@ class Commit: def __parse(self): # Get the commit title and list of files. - ret = subprocess.run(['git', 'show', '--pretty=oneline', '--name-only', + ret = subprocess.run(['git', 'show', '--pretty=oneline', '--name-status', self.commit], stdout=subprocess.PIPE).stdout.decode('utf-8') files = ret.splitlines() - self.__files = files[1:] + self.__files = [CommitFile(f) for f in files[1:]] self.__title = files[0] - @property - def files(self): - return self.__files + def files(self, filter='AM'): + return [f.filename for f in self.__files if f.status in filter] @property def title(self): @@ -231,10 +250,10 @@ class StagedChanges(Commit): Commit.__init__(self, '') def __parse(self): - ret = subprocess.run(['git', 'diff', '--staged', '--name-only'], + ret = subprocess.run(['git', 'diff', '--staged', '--name-status'], stdout=subprocess.PIPE).stdout.decode('utf-8') self.__title = "Staged changes" - self.__files = ret.splitlines() + self.__files = [CommitFile(f) for f in ret.splitlines()] def get_diff(self, top_level, filename): return subprocess.run(['git', 'diff', '--staged', '--', @@ -252,9 +271,9 @@ class Amendment(StagedChanges): stdout=subprocess.PIPE).stdout.decode('utf-8') self.__title = 'Amendment of ' + ret.strip() # Extract the list of modified files - ret = subprocess.run(['git', 'diff', '--staged', '--name-only', 'HEAD~'], + ret = subprocess.run(['git', 'diff', '--staged', '--name-status', 'HEAD~'], stdout=subprocess.PIPE).stdout.decode('utf-8') - self.__files = ret.splitlines() + self.__files = [CommitFile(f) for f in ret.splitlines()] def get_diff(self, top_level, filename): return subprocess.run(['git', 'diff', '--staged', 'HEAD~', '--', @@ -705,7 +724,7 @@ def check_style(top_level, commit): patterns = set() patterns.update(StyleChecker.all_patterns()) patterns.update(Formatter.all_patterns()) - files = [f for f in commit.files if len([p for p in patterns if fnmatch.fnmatch(os.path.basename(f), p)])] + files = [f for f in commit.files() if len([p for p in patterns if fnmatch.fnmatch(os.path.basename(f), p)])] if len(files) == 0: print("Commit doesn't touch source files, skipping") return 0 From patchwork Thu Dec 24 12:28:53 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Laurent Pinchart X-Patchwork-Id: 10742 X-Patchwork-Delegate: laurent.pinchart@ideasonboard.com Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 3857AC0F1A for ; Thu, 24 Dec 2020 12:29:15 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id E594561FF3; Thu, 24 Dec 2020 13:29:14 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="wK0U1oyW"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 0A5EA615B0 for ; Thu, 24 Dec 2020 13:29:10 +0100 (CET) Received: from pendragon.lan (62-78-145-57.bb.dnainternet.fi [62.78.145.57]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 978B1A1D for ; Thu, 24 Dec 2020 13:29:09 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1608812949; bh=842OaJz8aXP5QHYae35b2PfuGuUyo05f13sH0nuiqjI=; h=From:To:Subject:Date:In-Reply-To:References:From; b=wK0U1oyW8/AUsY2492T8+HP7mu3hzY2iSxu89UqfzazHvjgaPY2FiNYuMa6HllTB1 EK1XoHP2nYRbt2m9K5BCLadtKomw6gK4sCBiTGD2jXLxjxBinc0lhsnkCakuZZ9kuy 4rLOfvKkLckPbJX8s+viNXuYJVIaxGy0IeTr2OEA= From: Laurent Pinchart To: libcamera-devel@lists.libcamera.org Date: Thu, 24 Dec 2020 14:28:53 +0200 Message-Id: <20201224122855.22200-7-laurent.pinchart@ideasonboard.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20201224122855.22200-1-laurent.pinchart@ideasonboard.com> References: <20201224122855.22200-1-laurent.pinchart@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH 6/8] utils: checkstyle.py: Move diff parsing to Commit class X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" To avoid duplicating diff parsing in commit checkers, move it to the Commit class. Signed-off-by: Laurent Pinchart Reviewed-by: Niklas Söderlund --- utils/checkstyle.py | 13 +++++++------ 1 file changed, 7 insertions(+), 6 deletions(-) diff --git a/utils/checkstyle.py b/utils/checkstyle.py index 07e896d9819f..9c2a1837aa11 100755 --- a/utils/checkstyle.py +++ b/utils/checkstyle.py @@ -236,9 +236,10 @@ class Commit: return self.__title def get_diff(self, top_level, filename): - return subprocess.run(['git', 'diff', '%s~..%s' % (self.commit, self.commit), + diff = subprocess.run(['git', 'diff', '%s~..%s' % (self.commit, self.commit), '--', '%s/%s' % (top_level, filename)], stdout=subprocess.PIPE).stdout.decode('utf-8') + return parse_diff(diff.splitlines(True)) def get_file(self, filename): return subprocess.run(['git', 'show', '%s:%s' % (self.commit, filename)], @@ -256,9 +257,10 @@ class StagedChanges(Commit): self.__files = [CommitFile(f) for f in ret.splitlines()] def get_diff(self, top_level, filename): - return subprocess.run(['git', 'diff', '--staged', '--', + diff = subprocess.run(['git', 'diff', '--staged', '--', '%s/%s' % (top_level, filename)], stdout=subprocess.PIPE).stdout.decode('utf-8') + return parse_diff(diff.splitlines(True)) class Amendment(StagedChanges): @@ -276,9 +278,10 @@ class Amendment(StagedChanges): self.__files = [CommitFile(f) for f in ret.splitlines()] def get_diff(self, top_level, filename): - return subprocess.run(['git', 'diff', '--staged', 'HEAD~', '--', + diff = subprocess.run(['git', 'diff', '--staged', 'HEAD~', '--', '%s/%s' % (top_level, filename)], stdout=subprocess.PIPE).stdout.decode('utf-8') + return parse_diff(diff.splitlines(True)) # ------------------------------------------------------------------------------ @@ -657,9 +660,7 @@ class StripTrailingSpaceFormatter(Formatter): def check_file(top_level, commit, filename): # Extract the line numbers touched by the commit. - diff = commit.get_diff(top_level, filename) - diff = diff.splitlines(True) - commit_diff = parse_diff(diff) + commit_diff = commit.get_diff(top_level, filename) lines = [] for hunk in commit_diff: From patchwork Thu Dec 24 12:28:54 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Laurent Pinchart X-Patchwork-Id: 10743 X-Patchwork-Delegate: laurent.pinchart@ideasonboard.com Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id CB076C0F1A for ; Thu, 24 Dec 2020 12:29:16 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 9B22C62005; Thu, 24 Dec 2020 13:29:16 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="TB85nlZb"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 50A82615AC for ; Thu, 24 Dec 2020 13:29:10 +0100 (CET) Received: from pendragon.lan (62-78-145-57.bb.dnainternet.fi [62.78.145.57]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id EB866DFC for ; Thu, 24 Dec 2020 13:29:09 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1608812950; bh=LXq3q8vXJNF3MJqbcdQQ/ujxaZIqwAZYEuyeR+1e0AA=; h=From:To:Subject:Date:In-Reply-To:References:From; b=TB85nlZbVt1ga9hir0emhP3hFKax/2CQ4WecXJQ4vd9HK7ZFCyuc1vNn0YnbCXqi+ VgESeXzWFTWCq/+wN2AaPQRoY9FwS7yMMXF1GWBueHVnAfSmfawz/LbEixJXabQH55 GzftOKtzmFrqyMzlllkDhkB7HHlxLnU/ieJLiPg0= From: Laurent Pinchart To: libcamera-devel@lists.libcamera.org Date: Thu, 24 Dec 2020 14:28:54 +0200 Message-Id: <20201224122855.22200-8-laurent.pinchart@ideasonboard.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20201224122855.22200-1-laurent.pinchart@ideasonboard.com> References: <20201224122855.22200-1-laurent.pinchart@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH 7/8] utils: checkstyle.py: Add commit checkers X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Add a new category of checkers that operate on a whole commit. Signed-off-by: Laurent Pinchart Reviewed-by: Niklas Söderlund --- utils/checkstyle.py | 42 +++++++++++++++++++++++++++++++++++------- 1 file changed, 35 insertions(+), 7 deletions(-) diff --git a/utils/checkstyle.py b/utils/checkstyle.py index 9c2a1837aa11..c0a6b7ab06cd 100755 --- a/utils/checkstyle.py +++ b/utils/checkstyle.py @@ -296,6 +296,30 @@ class ClassRegistry(type): return newclass +# ------------------------------------------------------------------------------ +# Commit Checkers +# + +class CommitChecker(metaclass=ClassRegistry): + subclasses = [] + + def __init__(self): + pass + + # + # Class methods + # + @classmethod + def checkers(cls): + for checker in cls.subclasses: + yield checker + + +class CommitIssue(object): + def __init__(self, msg): + self.msg = msg + + # ------------------------------------------------------------------------------ # Style Checkers # @@ -721,25 +745,29 @@ def check_style(top_level, commit): print(commit.title) print(separator) + issues = 0 + + # Apply the commit checkers first. + for checker in CommitChecker.checkers(): + for issue in checker.check(commit, top_level): + print('%s%s%s' % (Colours.fg(Colours.Yellow), issue.msg, Colours.reset())) + issues += 1 + # Filter out files we have no checker for. patterns = set() patterns.update(StyleChecker.all_patterns()) patterns.update(Formatter.all_patterns()) files = [f for f in commit.files() if len([p for p in patterns if fnmatch.fnmatch(os.path.basename(f), p)])] - if len(files) == 0: - print("Commit doesn't touch source files, skipping") - return 0 - issues = 0 for f in files: issues += check_file(top_level, commit, f) if issues == 0: - print("No style issue detected") + print("No issue detected") else: print('---') - print("%u potential style %s detected, please review" % \ - (issues, 'issue' if issues == 1 else 'issues')) + print("%u potential %s detected, please review" % + (issues, 'issue' if issues == 1 else 'issues')) return issues From patchwork Thu Dec 24 12:28:55 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Laurent Pinchart X-Patchwork-Id: 10744 X-Patchwork-Delegate: laurent.pinchart@ideasonboard.com Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 540C3C0F1A for ; Thu, 24 Dec 2020 12:29:17 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 0C220615AF; Thu, 24 Dec 2020 13:29:17 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="YqrcMpaE"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id A86C7615AF for ; Thu, 24 Dec 2020 13:29:10 +0100 (CET) Received: from pendragon.lan (62-78-145-57.bb.dnainternet.fi [62.78.145.57]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 47E92FE0 for ; Thu, 24 Dec 2020 13:29:10 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1608812950; bh=yD71QmB73qmX72gSMLoN0LKZWfl2nUJ+FCw9LcubTDw=; h=From:To:Subject:Date:In-Reply-To:References:From; b=YqrcMpaEHiBabiUDKcqM2kPMM+NlCqVTDNDGOv6RjvsAEyYHPb7KX3KmTB97oA4Fe CCoBpgD9q1/RWeHingsFsimTNhkp6ECCk5m+MHZutf6mB/70YhNdWq9adOMIQqT+Hq ho89Uqmb0tdvjVR1NYtpjYFlPjCcK/y2E/0Yiiqo= From: Laurent Pinchart To: libcamera-devel@lists.libcamera.org Date: Thu, 24 Dec 2020 14:28:55 +0200 Message-Id: <20201224122855.22200-9-laurent.pinchart@ideasonboard.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20201224122855.22200-1-laurent.pinchart@ideasonboard.com> References: <20201224122855.22200-1-laurent.pinchart@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH 8/8] utils: checkstyle.py: Add header add checker X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Add a commit checker that ensures that all header files added to the libcamera includes (public or internal) are accompanied by a corresponding update of the meson.build file in the same directory. Here's the output of the new checker when run against a commit that forgot to update meson.build. $ ./utils/checkstyle.py b3383da79f1d --------------------------------------------------------------------------------- b3383da79f1d513b0d76db220a7104e1c1035e30 libcamera: buffer: Create a MappedBuffer --------------------------------------------------------------------------------- Header include/libcamera/internal/buffer.h added without corresponding update to include/libcamera/internal/meson.build --- 1 potential issue detected, please review In theory we could extend the checker to cover .cpp files too, but the issue will be quite noticeable as meson won't build the file if meson.build isn't updated. Header files are more tricky as problems would only occur at when installing the headers (for public headers), or would result in race conditions in the build. Both of those issues are harder to catch. Signed-off-by: Laurent Pinchart Reviewed-by: Niklas Söderlund --- utils/checkstyle.py | 44 ++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 44 insertions(+) diff --git a/utils/checkstyle.py b/utils/checkstyle.py index c0a6b7ab06cd..e618db937c2b 100755 --- a/utils/checkstyle.py +++ b/utils/checkstyle.py @@ -320,6 +320,50 @@ class CommitIssue(object): self.msg = msg +class HeaderAddChecker(CommitChecker): + @classmethod + def check(cls, commit, top_level): + issues = [] + + meson_files = [f for f in commit.files('M') + if os.path.basename(f) == 'meson.build'] + + for filename in commit.files('A'): + if not filename.startswith('include/libcamera/') or \ + not filename.endswith('.h'): + continue + + meson = os.path.dirname(filename) + '/meson.build' + header = os.path.basename(filename) + + issue = CommitIssue('Header %s added without corresponding update to %s' % + (filename, meson)) + + if meson not in meson_files: + issues.append(issue) + continue + + diff = commit.get_diff(top_level, meson) + found = False + + for hunk in diff: + for line in hunk.lines: + if line[0] != '+': + continue + + if line.find("'%s'" % header) != -1: + found = True + break + + if found: + break + + if not found: + issues.append(issue) + + return issues + + # ------------------------------------------------------------------------------ # Style Checkers #