From patchwork Mon Jun 12 22:47:48 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Laurent Pinchart X-Patchwork-Id: 18727 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 38FFDC3290 for ; Mon, 12 Jun 2023 22:47:55 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id D2B7561E5F; Tue, 13 Jun 2023 00:47:54 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1686610074; bh=pFIsOSDRcc8pIfEa2RsLe82QaIv0gE1BAISsqJvUaLk=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To: From; b=YwiJZIk6BF4PMadquYWpCMmEBhAOCvZP52CQGiFb68YmlOjekBwjrVlWwhQ2nFHDR 3IhAe35hPNtIDnd2Y+0mP33lByEqeQG8MgrLGxiQy2J2qBzJK7vpFaLDvRGhcnAZ4H Y0Ta+H7vwOnkO2mkb6aTkRwikZlneBf/vRJJ0T87ej/7SUPsBkOXgM7Jf9VdYculbW FjauSqd8xbNg7Y5/HnQ+q2capNjjJJ0VQPdAv3keoG6FCCAVbgdiZipPdGMiDd5Vnw 5PD5fpCmIIhM1Bv+oXLzYn0HIco4qDlxLk5tkmhY8ZmByxgPBGvYCqnswhlX6Ktjxb 2gD4QNy5ZQarA== Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 02E8361E50 for ; Tue, 13 Jun 2023 00:47:53 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="i6Whi86E"; dkim-atps=neutral Received: from pendragon.ideasonboard.com (213-243-189-158.bb.dnainternet.fi [213.243.189.158]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 3822AC7F for ; Tue, 13 Jun 2023 00:47:23 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1686610043; bh=pFIsOSDRcc8pIfEa2RsLe82QaIv0gE1BAISsqJvUaLk=; h=From:To:Subject:Date:In-Reply-To:References:From; b=i6Whi86EiWzMN9w3S/eq7eetGERX5B9vOmOgb/B4Dr7Iqp5ALy6zKqEBVfj8WArpE noljrgkxN+W+kMM9HCxDgVu7PMJEWd9Zc7uUbllUqNvgjdyaPqJPZjvWvHAU2+bqyq 6zwMPW1Ev55avoOHAqh9aQ2tb8oFFnwxKU9Cc9ko= To: libcamera-devel@lists.libcamera.org Date: Tue, 13 Jun 2023 01:47:48 +0300 Message-Id: <20230612224751.4437-2-laurent.pinchart@ideasonboard.com> X-Mailer: git-send-email 2.39.3 In-Reply-To: <20230612224751.4437-1-laurent.pinchart@ideasonboard.com> References: <20230612224751.4437-1-laurent.pinchart@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v1 1/4] utils: checkstyle: Add __repr__ method to CommitFile class X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Laurent Pinchart via libcamera-devel From: Laurent Pinchart Reply-To: Laurent Pinchart Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Add a custom representation to the CommitFile class in order to facilitate debugging. Signed-off-by: Laurent Pinchart Reviewed-by: Kieran Bingham --- utils/checkstyle.py | 3 +++ 1 file changed, 3 insertions(+) diff --git a/utils/checkstyle.py b/utils/checkstyle.py index 5cc523b6fba3..7da888d8c365 100755 --- a/utils/checkstyle.py +++ b/utils/checkstyle.py @@ -191,6 +191,9 @@ class CommitFile: else: self.__filename = info[1] + def __repr__(self): + return f'{self.__status} {self.__filename}' + @property def filename(self): return self.__filename From patchwork Mon Jun 12 22:47:49 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Laurent Pinchart X-Patchwork-Id: 18728 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 784FAC31E9 for ; Mon, 12 Jun 2023 22:47:56 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 4C9716289F; Tue, 13 Jun 2023 00:47:56 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1686610076; bh=pFFY84X4S/spKigWytzPQVbkgxCjjLhWG9Rj+tMYnds=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To: From; b=ick64gzLzY4S35vLNuOcxy6YnGu6AbS5cmt3rUcmvHcunw1u+AY1zclfc1M6mJ+Kx azw9xeVNe14B2PQzpaMtuQze4vdA9otFR8sUkLCH8F2KHGjXmcj4CGWkgDOqc1T8dU 5mfwEfveHwUx6HCqM0CqbDJli9v5SJpNyBvP3uJdW/TMr/0pcctsVUwzm57utFPZ5g LE6bsnz6ffBD0wnp5H59EmRQnLKhNKIHB/sV7kreW33cfsS+FowRDVqk+SsCJ4TU8n FItGbiBJzpH+VekpiVDO8e/7GbIxKbi6+7U25ZIbgfr59GsUH2AKUuz1hSjjPbHE3F TDh6ysZfK7jsg== Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 9D51F6020C for ; Tue, 13 Jun 2023 00:47:54 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="mSJjXB70"; dkim-atps=neutral Received: from pendragon.ideasonboard.com (213-243-189-158.bb.dnainternet.fi [213.243.189.158]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id CF9B4D80 for ; Tue, 13 Jun 2023 00:47:24 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1686610045; bh=pFFY84X4S/spKigWytzPQVbkgxCjjLhWG9Rj+tMYnds=; h=From:To:Subject:Date:In-Reply-To:References:From; b=mSJjXB70M5gWssCOmrC7fcJQVDnY0QtgqiMJDHKcYoRsC10vLZru1q08qFvbRXCJr FckUDDbxG1LFc9ex/LtiILTsKj1kixM48OwG/397+1jFkntbPEL58oGnemjMW/oJBb c/HRfhPhkvw7A51sUp7f2BqMb6eBYSn7L8SbzJog= To: libcamera-devel@lists.libcamera.org Date: Tue, 13 Jun 2023 01:47:49 +0300 Message-Id: <20230612224751.4437-3-laurent.pinchart@ideasonboard.com> X-Mailer: git-send-email 2.39.3 In-Reply-To: <20230612224751.4437-1-laurent.pinchart@ideasonboard.com> References: <20230612224751.4437-1-laurent.pinchart@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v1 2/4] utils: checkstyle: Support running checkers selectively X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Laurent Pinchart via libcamera-devel From: Laurent Pinchart Reply-To: Laurent Pinchart Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" During development of the checkstyle.py script, it can be useful to run only a subset of the checker. Add the ability to do so with a '--checkers' command line argument. Signed-off-by: Laurent Pinchart Reviewed-by: Mattijs Korpershoek Reviewed-by: Kieran Bingham --- utils/checkstyle.py | 31 +++++++++++++++++++++---------- 1 file changed, 21 insertions(+), 10 deletions(-) diff --git a/utils/checkstyle.py b/utils/checkstyle.py index 7da888d8c365..3104acfa2065 100755 --- a/utils/checkstyle.py +++ b/utils/checkstyle.py @@ -301,8 +301,10 @@ class CommitChecker(metaclass=ClassRegistry): # Class methods # @classmethod - def checkers(cls): + def checkers(cls, names): for checker in cls.subclasses: + if names and checker.__name__ not in names: + continue yield checker @@ -436,8 +438,10 @@ class StyleChecker(metaclass=ClassRegistry): # Class methods # @classmethod - def checkers(cls, filename): + def checkers(cls, filename, names): for checker in cls.subclasses: + if names and checker.__name__ not in names: + continue if checker.supports(filename): yield checker @@ -617,8 +621,10 @@ class Formatter(metaclass=ClassRegistry): # Class methods # @classmethod - def formatters(cls, filename): + def formatters(cls, filename, names): for formatter in cls.subclasses: + if names and formatter.__name__ not in names: + continue if formatter.supports(filename): yield formatter @@ -780,7 +786,7 @@ class StripTrailingSpaceFormatter(Formatter): # Style checking # -def check_file(top_level, commit, filename): +def check_file(top_level, commit, filename, checkers): # Extract the line numbers touched by the commit. commit_diff = commit.get_diff(top_level, filename) @@ -797,7 +803,7 @@ def check_file(top_level, commit, filename): after = commit.get_file(filename) formatted = after - for formatter in Formatter.formatters(filename): + for formatter in Formatter.formatters(filename, checkers): formatted = formatter.format(filename, formatted) after = after.splitlines(True) @@ -811,7 +817,7 @@ def check_file(top_level, commit, filename): # Check for code issues not related to formatting. issues = [] - for checker in StyleChecker.checkers(filename): + for checker in StyleChecker.checkers(filename, checkers): checker = checker(after) for hunk in commit_diff: issues += checker.check(hunk.side('to').touched) @@ -839,7 +845,7 @@ def check_file(top_level, commit, filename): return len(formatted_diff) + len(issues) -def check_style(top_level, commit): +def check_style(top_level, commit, checkers): separator = '-' * len(commit.title) print(separator) print(commit.title) @@ -848,7 +854,7 @@ def check_style(top_level, commit): issues = 0 # Apply the commit checkers first. - for checker in CommitChecker.checkers(): + for checker in CommitChecker.checkers(checkers): for issue in checker.check(commit, top_level): print('%s%s%s' % (Colours.fg(Colours.Yellow), issue.msg, Colours.reset())) issues += 1 @@ -860,7 +866,7 @@ def check_style(top_level, commit): files = [f for f in commit.files() if len([p for p in patterns if fnmatch.fnmatch(os.path.basename(f), p)])] for f in files: - issues += check_file(top_level, commit, f) + issues += check_file(top_level, commit, f, checkers) if issues == 0: print('No issue detected') @@ -910,6 +916,8 @@ def main(argv): # Parse command line arguments parser = argparse.ArgumentParser() + parser.add_argument('--checkers', '-c', type=str, + help='Specify which checkers to run as a comma-separated list. Defaults to all checkers') parser.add_argument('--staged', '-s', action='store_true', help='Include the changes in the index. Defaults to False') parser.add_argument('--amend', '-a', action='store_true', @@ -918,6 +926,9 @@ def main(argv): help='Revision range (as defined by git rev-parse). Defaults to HEAD if not specified.') args = parser.parse_args(argv[1:]) + if args.checkers: + args.checkers = args.checkers.split(',') + # Check for required dependencies. for command, mandatory in dependencies.items(): found = shutil.which(command) @@ -951,7 +962,7 @@ def main(argv): issues = 0 for commit in commits: - issues += check_style(top_level, commit) + issues += check_style(top_level, commit, args.checkers) print('') if issues: From patchwork Mon Jun 12 22:47:50 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Laurent Pinchart X-Patchwork-Id: 18729 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 0BF94C3290 for ; Mon, 12 Jun 2023 22:47:58 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id BF4CC628A2; Tue, 13 Jun 2023 00:47:57 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1686610077; bh=He3f3szaxuhPEncGvV+Xel2qkY3nXK9yIB9c71GbCmE=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To: From; b=3mprioAKHLJeqzOytGepc34f/H1Npk602sHsfm3Me8byfz3fo1nMSwvKFr46iXNQz GkbBvewOnVWP2rIGCOMLrukp07bMvQ0pMEy9mgb02iqOt1R/lezIQYP4NEZqVWI2/4 YRCk5euwVVud6uMuvJH7Se93jlkI1kCAk/T5PaXI1Rh9KUYts5+j6dv6101NOne1Te l7q8HLQknajOQUbd+erv/+yZz7cR8Chqr/Az++3zUBLG4GZnnGs5u78Vpy7pYYcAoZ C2+Vlwl76HHJadjnetrognuYj2VHa/aNcXrXhXLgXRVgWIQi4cRSEf6QTyDZ7S2x36 ccKBnlpwi5u3g== Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 09EBE61E59 for ; Tue, 13 Jun 2023 00:47:56 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="cE6DN1RS"; dkim-atps=neutral Received: from pendragon.ideasonboard.com (213-243-189-158.bb.dnainternet.fi [213.243.189.158]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 3FA34D80 for ; Tue, 13 Jun 2023 00:47:26 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1686610046; bh=He3f3szaxuhPEncGvV+Xel2qkY3nXK9yIB9c71GbCmE=; h=From:To:Subject:Date:In-Reply-To:References:From; b=cE6DN1RS3n4OquHuT5TXPWcFoiuTyjNdLDRd7o7aQwZ35x3QGfHceu6esB8j/l2xn zR6EJn+lycjKNANDKQYTPtdj+Gt5VhOp4d5sDhAj44UGo/1ajlrUIrF8Jn5xpDIYYN FtX373w/ezw8G4NtomzEwl8G7K5jDxUCaRrvYu6c= To: libcamera-devel@lists.libcamera.org Date: Tue, 13 Jun 2023 01:47:50 +0300 Message-Id: <20230612224751.4437-4-laurent.pinchart@ideasonboard.com> X-Mailer: git-send-email 2.39.3 In-Reply-To: <20230612224751.4437-1-laurent.pinchart@ideasonboard.com> References: <20230612224751.4437-1-laurent.pinchart@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v1 3/4] utils: checkstyle: Don't include commit ID in commit title X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Laurent Pinchart via libcamera-devel From: Laurent Pinchart Reply-To: Laurent Pinchart Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" The commit title and commit ID are two different pieces of information. Don't include the latter in the former, to simplify code that only needs the commit title. Constructing a string from the ID and title is easier than splitting the combined string back into its elements. Signed-off-by: Laurent Pinchart Reviewed-by: Kieran Bingham --- utils/checkstyle.py | 17 +++++++++-------- 1 file changed, 9 insertions(+), 8 deletions(-) diff --git a/utils/checkstyle.py b/utils/checkstyle.py index 3104acfa2065..e68c874609bc 100755 --- a/utils/checkstyle.py +++ b/utils/checkstyle.py @@ -210,12 +210,12 @@ class Commit: def _parse(self): # Get the commit title and list of files. - ret = subprocess.run(['git', 'show', '--pretty=oneline', '--name-status', + ret = subprocess.run(['git', 'show', '--format=%s', '--name-status', self.commit], stdout=subprocess.PIPE).stdout.decode('utf-8') - files = ret.splitlines() - self._files = [CommitFile(f) for f in files[1:]] - self._title = files[0] + lines = ret.splitlines() + self._files = [CommitFile(f) for f in lines[1:] if f] + self._title = lines[0] def files(self, filter='AMR'): return [f.filename for f in self._files if f.status in filter] @@ -358,7 +358,7 @@ class HeaderAddChecker(CommitChecker): class TitleChecker(CommitChecker): - prefix_regex = re.compile(r'[0-9a-f]+ (([a-zA-Z0-9_.-]+: )+)') + prefix_regex = re.compile(r'^([a-zA-Z0-9_.-]+: )+') release_regex = re.compile(r'libcamera v[0-9]+\.[0-9]+\.[0-9]+') @classmethod @@ -388,7 +388,7 @@ class TitleChecker(CommitChecker): if not prefix: continue - prefix = prefix.group(1) + prefix = prefix.group(0) if prefix in prefixes: prefixes[prefix] += 1 else: @@ -846,9 +846,10 @@ def check_file(top_level, commit, filename, checkers): def check_style(top_level, commit, checkers): - separator = '-' * len(commit.title) + title = commit.commit + ' ' + commit.title + separator = '-' * len(title) print(separator) - print(commit.title) + print(title) print(separator) issues = 0 From patchwork Mon Jun 12 22:47:51 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Laurent Pinchart X-Patchwork-Id: 18730 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id A91F1C31E9 for ; Mon, 12 Jun 2023 22:47:59 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 6950C61E50; Tue, 13 Jun 2023 00:47:59 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1686610079; bh=ndBRtEmIBZGjH91lpJCqt/EeeCr4N7TvaJHIhQx5eL4=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To: From; b=BCwDPhz7pTkFVaue5iHz9w7pfe8WPmmOL54f3A/M6eD7UsZuDGBYCFI08KrCpAVeS sA7WKhsALeym8nOGrFGpO3Uvda4DpULbZ/yQ+rvadHwS/hViFGQmq4mbPODIwN2TGx /faoSP38IanX9NJTZvv3EJ17jd2hk/anRx72DSFN9nZZMkEHqjpoJyIb7Cu9IR0K0V kc56WiAtjjwzfODOORDjjyhnkp6yjmVviyHZVk8hXWEV6Vf2i5Y4p8A1HGX2Beo+W0 Jdw4m2NKOP5XV2v6CpX2OjO2V4O4Ye60HCHLNdkCyJ9LpQPEMywqTGCzlqDKHiAl83 idZv2wL4urKoA== Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 9019461E51 for ; Tue, 13 Jun 2023 00:47:57 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="exv113ct"; dkim-atps=neutral Received: from pendragon.ideasonboard.com (213-243-189-158.bb.dnainternet.fi [213.243.189.158]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id D5915D80 for ; Tue, 13 Jun 2023 00:47:27 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1686610048; bh=ndBRtEmIBZGjH91lpJCqt/EeeCr4N7TvaJHIhQx5eL4=; h=From:To:Subject:Date:In-Reply-To:References:From; b=exv113ctSynJl/h/mqr08CCJSyPp029GwwlekcjdAQc/voDo8KvUxcTqX1iBmg/ay MahXo2h11UeJg0SkZMvt0/fxMAKkKJ7x0OyExZzrMqHhPDRZWgw359NYWp7PmMau9h 4yQh27Cs7Jid6UBiPgnuq7dXbBSEX8WY00S8VmvI= To: libcamera-devel@lists.libcamera.org Date: Tue, 13 Jun 2023 01:47:51 +0300 Message-Id: <20230612224751.4437-5-laurent.pinchart@ideasonboard.com> X-Mailer: git-send-email 2.39.3 In-Reply-To: <20230612224751.4437-1-laurent.pinchart@ideasonboard.com> References: <20230612224751.4437-1-laurent.pinchart@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v1 4/4] utils: checkstyle: Add trailers checker X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Laurent Pinchart via libcamera-devel From: Laurent Pinchart Reply-To: Laurent Pinchart Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" The libcamera git history contains numerous examples of incorrect commit message trailers due to invalid trailer types (e.g. Change-Id), typos and other small issues. Those went unnoticed through reviews, which shows that an automated checker is required. Add a trailers checker to checkstyle.py to catch invalid or malformed trailers, with a set of supported trailers that match libcamera's commit message practices. New trailer keys can easily be added later as new needs arise. Signed-off-by: Laurent Pinchart Reviewed-by: Kieran Bingham --- This reports a total of 42 issues through the project's history. 37 of them should not be controversial: - Trailer keys not valid for libcamera ('Cc', 'Change-Id', 'Inspired-by' and 'Reported-on') - Typos in trailer keys ('Fixed' instead of 'Fixes', 'Reviewed' instead of 'Reviewed-by' and 'Signed-off-By' instead of 'Signed-off-by') - Typos in e-mail address (missing display name, missing space before '<' and missing trailing '>') - Link in 'Fixes' trailer (should be a commit) - Too short commit ID in 'Fixes' trailer - Typos in 'Fixes' trailer (extra 'commit' before commit ID, missing space or extra ':' after commit ID, missing '"' around commit subject) The five remaining issues may benefit from discussions: - Invalid trailer key 'Co-developed-by' (one instance). This is a trailer key commonly used in the kernel, but git..b both recommend Co-authored-by. I'm sure which option would be best, so I haven't included either for now. - Typo in 'Reported-by' trailer for issues reported by Coverity (one instance). 'Reported-by' usually has an e-mail address value, but we have commonly used 'Coverity CID=' for issues reported by Coverity. I've tentatively added support for this (feedback is welcome), and one commit still got flagged as its 'Reported-by' trailer has a space instead of an '=' after 'CID'. - Usage of a github user URL in 'Reported-by' (one instance). Our policy is to be able to identify users by name and e-mail address for 'Signed-off-by' trailers, and I would prefer covering 'Reported-by' trailers too. If someone *really* doesn't want their name included in the git log when reporting an issue, we can simply omit the 'Reported-by' trailer. - Usage of URLs in 'Reported-by' to point to buildbot.libcamera.org (two instances). This should use a 'Link' trailer instead. --- utils/checkstyle.py | 80 +++++++++++++++++++++++++++++++++++++++++++-- 1 file changed, 78 insertions(+), 2 deletions(-) diff --git a/utils/checkstyle.py b/utils/checkstyle.py index e68c874609bc..3558740d389d 100755 --- a/utils/checkstyle.py +++ b/utils/checkstyle.py @@ -210,13 +210,23 @@ class Commit: def _parse(self): # Get the commit title and list of files. - ret = subprocess.run(['git', 'show', '--format=%s', '--name-status', + ret = subprocess.run(['git', 'show', '--format=%s%n%(trailers:only,unfold)', '--name-status', self.commit], stdout=subprocess.PIPE).stdout.decode('utf-8') lines = ret.splitlines() - self._files = [CommitFile(f) for f in lines[1:] if f] + self._title = lines[0] + self._trailers = [] + for index in range(1, len(lines)): + line = lines[index] + if not line: + break + + self._trailers.append(line) + + self._files = [CommitFile(f) for f in lines[index:] if f] + def files(self, filter='AMR'): return [f.filename for f in self._files if f.status in filter] @@ -224,6 +234,10 @@ class Commit: def title(self): return self._title + @property + def trailers(self): + return self._trailers + def get_diff(self, top_level, filename): diff = subprocess.run(['git', 'diff', '%s~..%s' % (self.commit, self.commit), '--', '%s/%s' % (top_level, filename)], @@ -424,6 +438,68 @@ class TitleChecker(CommitChecker): 'possible candidates are ' + candidates)] +class TrailersChecker(CommitChecker): + commit_regex = re.compile(r'[0-9a-f]{12}[0-9a-f]* \(".*"\)') + + coverity_regex = re.compile(r'Coverity CID=.*') + + # Simple e-mail address validator regex, with an additional trailing + # comment. The complexity of a full RFC6531 validator isn't worth the + # additional invalid addresses it would reject. + email_regex = re.compile(r'[^<]+ <[^@>]+@[^>]+>( # .*)?') + + link_regex = re.compile(r'https?://.*') + + @staticmethod + def validate_reported_by(value): + if TrailersChecker.email_regex.fullmatch(value): + return True + if TrailersChecker.coverity_regex.fullmatch(value): + return True + return False + + known_trailers = { + 'Acked-by': email_regex, + 'Bug': link_regex, + 'Fixes': commit_regex, + 'Link': link_regex, + 'Reported-by': validate_reported_by, + 'Reviewed-by': email_regex, + 'Signed-off-by': email_regex, + 'Suggested-by': email_regex, + 'Tested-by': email_regex, + } + + trailer_regex = re.compile(r'([A-Z][a-zA-Z-]*)\s*:\s*(.*)') + + @classmethod + def check(cls, commit, top_level): + issues = [] + + for trailer in commit.trailers: + match = TrailersChecker.trailer_regex.fullmatch(trailer) + if not match: + raise RuntimeError(f"Malformed commit trailer '{trailer}'") + + key, value = match.groups() + + validator = TrailersChecker.known_trailers.get(key) + if not validator: + issues.append(CommitIssue(f"Invalid commit trailer key '{key}'")) + continue + + if isinstance(validator, re.Pattern): + valid = bool(validator.fullmatch(value)) + else: + valid = validator(value) + + if not valid: + issues.append(CommitIssue(f"Malformed value '{value}' for commit trailer '{key}'")) + continue + + return issues + + # ------------------------------------------------------------------------------ # Style Checkers #