From patchwork Mon Jun 12 22:47:51 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Laurent Pinchart X-Patchwork-Id: 18730 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id A91F1C31E9 for ; Mon, 12 Jun 2023 22:47:59 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 6950C61E50; Tue, 13 Jun 2023 00:47:59 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1686610079; bh=ndBRtEmIBZGjH91lpJCqt/EeeCr4N7TvaJHIhQx5eL4=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To: From; b=BCwDPhz7pTkFVaue5iHz9w7pfe8WPmmOL54f3A/M6eD7UsZuDGBYCFI08KrCpAVeS sA7WKhsALeym8nOGrFGpO3Uvda4DpULbZ/yQ+rvadHwS/hViFGQmq4mbPODIwN2TGx /faoSP38IanX9NJTZvv3EJ17jd2hk/anRx72DSFN9nZZMkEHqjpoJyIb7Cu9IR0K0V kc56WiAtjjwzfODOORDjjyhnkp6yjmVviyHZVk8hXWEV6Vf2i5Y4p8A1HGX2Beo+W0 Jdw4m2NKOP5XV2v6CpX2OjO2V4O4Ye60HCHLNdkCyJ9LpQPEMywqTGCzlqDKHiAl83 idZv2wL4urKoA== Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 9019461E51 for ; Tue, 13 Jun 2023 00:47:57 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="exv113ct"; dkim-atps=neutral Received: from pendragon.ideasonboard.com (213-243-189-158.bb.dnainternet.fi [213.243.189.158]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id D5915D80 for ; Tue, 13 Jun 2023 00:47:27 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1686610048; bh=ndBRtEmIBZGjH91lpJCqt/EeeCr4N7TvaJHIhQx5eL4=; h=From:To:Subject:Date:In-Reply-To:References:From; b=exv113ctSynJl/h/mqr08CCJSyPp029GwwlekcjdAQc/voDo8KvUxcTqX1iBmg/ay MahXo2h11UeJg0SkZMvt0/fxMAKkKJ7x0OyExZzrMqHhPDRZWgw359NYWp7PmMau9h 4yQh27Cs7Jid6UBiPgnuq7dXbBSEX8WY00S8VmvI= To: libcamera-devel@lists.libcamera.org Date: Tue, 13 Jun 2023 01:47:51 +0300 Message-Id: <20230612224751.4437-5-laurent.pinchart@ideasonboard.com> X-Mailer: git-send-email 2.39.3 In-Reply-To: <20230612224751.4437-1-laurent.pinchart@ideasonboard.com> References: <20230612224751.4437-1-laurent.pinchart@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v1 4/4] utils: checkstyle: Add trailers checker X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Laurent Pinchart via libcamera-devel From: Laurent Pinchart Reply-To: Laurent Pinchart Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" The libcamera git history contains numerous examples of incorrect commit message trailers due to invalid trailer types (e.g. Change-Id), typos and other small issues. Those went unnoticed through reviews, which shows that an automated checker is required. Add a trailers checker to checkstyle.py to catch invalid or malformed trailers, with a set of supported trailers that match libcamera's commit message practices. New trailer keys can easily be added later as new needs arise. Signed-off-by: Laurent Pinchart Reviewed-by: Kieran Bingham --- This reports a total of 42 issues through the project's history. 37 of them should not be controversial: - Trailer keys not valid for libcamera ('Cc', 'Change-Id', 'Inspired-by' and 'Reported-on') - Typos in trailer keys ('Fixed' instead of 'Fixes', 'Reviewed' instead of 'Reviewed-by' and 'Signed-off-By' instead of 'Signed-off-by') - Typos in e-mail address (missing display name, missing space before '<' and missing trailing '>') - Link in 'Fixes' trailer (should be a commit) - Too short commit ID in 'Fixes' trailer - Typos in 'Fixes' trailer (extra 'commit' before commit ID, missing space or extra ':' after commit ID, missing '"' around commit subject) The five remaining issues may benefit from discussions: - Invalid trailer key 'Co-developed-by' (one instance). This is a trailer key commonly used in the kernel, but git..b both recommend Co-authored-by. I'm sure which option would be best, so I haven't included either for now. - Typo in 'Reported-by' trailer for issues reported by Coverity (one instance). 'Reported-by' usually has an e-mail address value, but we have commonly used 'Coverity CID=' for issues reported by Coverity. I've tentatively added support for this (feedback is welcome), and one commit still got flagged as its 'Reported-by' trailer has a space instead of an '=' after 'CID'. - Usage of a github user URL in 'Reported-by' (one instance). Our policy is to be able to identify users by name and e-mail address for 'Signed-off-by' trailers, and I would prefer covering 'Reported-by' trailers too. If someone *really* doesn't want their name included in the git log when reporting an issue, we can simply omit the 'Reported-by' trailer. - Usage of URLs in 'Reported-by' to point to buildbot.libcamera.org (two instances). This should use a 'Link' trailer instead. --- utils/checkstyle.py | 80 +++++++++++++++++++++++++++++++++++++++++++-- 1 file changed, 78 insertions(+), 2 deletions(-) diff --git a/utils/checkstyle.py b/utils/checkstyle.py index e68c874609bc..3558740d389d 100755 --- a/utils/checkstyle.py +++ b/utils/checkstyle.py @@ -210,13 +210,23 @@ class Commit: def _parse(self): # Get the commit title and list of files. - ret = subprocess.run(['git', 'show', '--format=%s', '--name-status', + ret = subprocess.run(['git', 'show', '--format=%s%n%(trailers:only,unfold)', '--name-status', self.commit], stdout=subprocess.PIPE).stdout.decode('utf-8') lines = ret.splitlines() - self._files = [CommitFile(f) for f in lines[1:] if f] + self._title = lines[0] + self._trailers = [] + for index in range(1, len(lines)): + line = lines[index] + if not line: + break + + self._trailers.append(line) + + self._files = [CommitFile(f) for f in lines[index:] if f] + def files(self, filter='AMR'): return [f.filename for f in self._files if f.status in filter] @@ -224,6 +234,10 @@ class Commit: def title(self): return self._title + @property + def trailers(self): + return self._trailers + def get_diff(self, top_level, filename): diff = subprocess.run(['git', 'diff', '%s~..%s' % (self.commit, self.commit), '--', '%s/%s' % (top_level, filename)], @@ -424,6 +438,68 @@ class TitleChecker(CommitChecker): 'possible candidates are ' + candidates)] +class TrailersChecker(CommitChecker): + commit_regex = re.compile(r'[0-9a-f]{12}[0-9a-f]* \(".*"\)') + + coverity_regex = re.compile(r'Coverity CID=.*') + + # Simple e-mail address validator regex, with an additional trailing + # comment. The complexity of a full RFC6531 validator isn't worth the + # additional invalid addresses it would reject. + email_regex = re.compile(r'[^<]+ <[^@>]+@[^>]+>( # .*)?') + + link_regex = re.compile(r'https?://.*') + + @staticmethod + def validate_reported_by(value): + if TrailersChecker.email_regex.fullmatch(value): + return True + if TrailersChecker.coverity_regex.fullmatch(value): + return True + return False + + known_trailers = { + 'Acked-by': email_regex, + 'Bug': link_regex, + 'Fixes': commit_regex, + 'Link': link_regex, + 'Reported-by': validate_reported_by, + 'Reviewed-by': email_regex, + 'Signed-off-by': email_regex, + 'Suggested-by': email_regex, + 'Tested-by': email_regex, + } + + trailer_regex = re.compile(r'([A-Z][a-zA-Z-]*)\s*:\s*(.*)') + + @classmethod + def check(cls, commit, top_level): + issues = [] + + for trailer in commit.trailers: + match = TrailersChecker.trailer_regex.fullmatch(trailer) + if not match: + raise RuntimeError(f"Malformed commit trailer '{trailer}'") + + key, value = match.groups() + + validator = TrailersChecker.known_trailers.get(key) + if not validator: + issues.append(CommitIssue(f"Invalid commit trailer key '{key}'")) + continue + + if isinstance(validator, re.Pattern): + valid = bool(validator.fullmatch(value)) + else: + valid = validator(value) + + if not valid: + issues.append(CommitIssue(f"Malformed value '{value}' for commit trailer '{key}'")) + continue + + return issues + + # ------------------------------------------------------------------------------ # Style Checkers #