[{"id":35670,"web_url":"https://patchwork.libcamera.org/comment/35670/","msgid":"<175673311470.560048.10560463581093758798@ping.linuxembedded.co.uk>","date":"2025-09-01T13:25:14","subject":"Re: [PATCH v1 1/2] utils: codegen: gen-formats.py: Use jinja","submitter":{"id":4,"url":"https://patchwork.libcamera.org/api/people/4/","name":"Kieran Bingham","email":"kieran.bingham@ideasonboard.com"},"content":"Quoting Barnabás Pőcze (2025-09-01 10:23:19)\n> Currently the gen-formats.py script can only be used to generate C++\n> code because it hard-codes part of the template. Use jinja to fully\n> remove any such dependency.\n> \n\n\nIs there a plan to generate format information elsewhere than the C++\nheader after this ?\n\n\n\n> Signed-off-by: Barnabás Pőcze \n> ---\n> include/libcamera/formats.h.in | 4 +++-\n> utils/codegen/gen-formats.py | 34 ++++++++++++----------------------\n> 2 files changed, 15 insertions(+), 23 deletions(-)\n> \n> diff --git a/include/libcamera/formats.h.in b/include/libcamera/formats.h.in\n> index 6ae7634fe..5ff9c3bf4 100644\n> --- a/include/libcamera/formats.h.in\n> +++ b/include/libcamera/formats.h.in\n> @@ -35,7 +35,9 @@ constexpr uint64_t __mod(unsigned int vendor, unsigned int mod)\n> \n> } /* namespace */\n> \n> -${formats}\n> +{% for f in formats %}\n> +constexpr PixelFormat {{f.name}}(__fourcc({{f.fourcc}}), __mod({{f.mod}}));\n> +{%- endfor %}\n> \n> } /* namespace formats */\n> \n> diff --git a/utils/codegen/gen-formats.py b/utils/codegen/gen-formats.py\n> index 0c0932a5b..872f3fe34 100755\n> --- a/utils/codegen/gen-formats.py\n> +++ b/utils/codegen/gen-formats.py\n> @@ -7,6 +7,7 @@\n> # Generate formats definitions from YAML\n> \n> import argparse\n> +import jinja2\n> import re\n> import string\n> import sys\n> @@ -52,9 +53,7 @@ class DRMFourCC(object):\n> return self.vendors[vendor], value\n> \n> \n> -def generate_h(formats, drm_fourcc):\n> - template = string.Template('constexpr PixelFormat ${name}{ __fourcc(${fourcc}), __mod(${mod}) };')\n> -\n> +def generate_formats(formats, drm_fourcc):\n> fmts = []\n> \n> for format in formats:\n> @@ -73,24 +72,17 @@ def generate_h(formats, drm_fourcc):\n> if mod:\n> data['mod'] = '%u, %u' % drm_fourcc.mod(mod)\n> \n> - fmts.append(template.substitute(data))\n> -\n> - return {'formats': '\\n'.join(fmts)}\n> -\n> -\n> -def fill_template(template, data):\n> + fmts.append(data)\n> \n> - template = open(template, 'rb').read()\n> - template = template.decode('utf-8')\n> - template = string.Template(template)\n> - return template.substitute(data)\n> + return fmts\n> \n> \n> def main(argv):\n> \n> # Parse command line arguments\n> parser = argparse.ArgumentParser()\n> - parser.add_argument('-o', dest='output', metavar='file', type=str,\n> + parser.add_argument('-o', dest='output', metavar='file',\n> + type=argparse.FileType('w', encoding='utf-8'), default=sys.stdout,\n> help='Output file name. Defaults to standard output if not specified.')\n> parser.add_argument('input', type=str,\n> help='Input file name.')\n> @@ -104,15 +96,13 @@ def main(argv):\n> formats = yaml.safe_load(data)['formats']\n> drm_fourcc = DRMFourCC(args.drm_fourcc)\n> \n> - data = generate_h(formats, drm_fourcc)\n> - data = fill_template(args.template, data)\n> + env = jinja2.Environment()\n> + template = env.from_string(open(args.template, 'r', encoding='utf-8').read())\n> + string = template.render({\n> + 'formats': generate_formats(formats, drm_fourcc),\n> + })\n> \n> - if args.output:\n> - output = open(args.output, 'wb')\n> - output.write(data.encode('utf-8'))\n> - output.close()\n> - else:\n> - sys.stdout.write(data)\n> + args.output.write(string)\n\nThis looks reasonable to me.\n\nAll looks ok to me and we already use jinja so :\n\nReviewed-by: Kieran Bingham \n\n> \n> return 0\n> \n> -- \n> 2.51.0\n>","headers":{"Return-Path":"","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id A7C53BDB13\n\tfor ;\n\tMon, 1 Sep 2025 13:25:20 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id 9C82669324;\n\tMon, 1 Sep 2025 15:25:19 +0200 (CEST)","from perceval.ideasonboard.com (perceval.ideasonboard.com\n\t[IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id 6085E69323\n\tfor ;\n\tMon, 1 Sep 2025 15:25:17 +0200 (CEST)","from pendragon.ideasonboard.com\n\t(cpc89244-aztw30-2-0-cust6594.18-1.cable.virginm.net [86.31.185.195])\n\tby perceval.ideasonboard.com (Postfix) with ESMTPSA id 0231B50A;\n\tMon, 1 Sep 2025 15:24:09 +0200 (CEST)"],"Authentication-Results":"lancelot.ideasonboard.com; dkim=pass (1024-bit key;\n\tunprotected) header.d=ideasonboard.com header.i=@ideasonboard.com\n\theader.b=\"cYWa8Sfg\"; dkim-atps=neutral","DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com;\n\ts=mail; t=1756733050;\n\tbh=UCWa+ygFEwhAPbNYGQKKlZcAPp3lhJx46VaMVkwn4TA=;\n\th=In-Reply-To:References:Subject:From:To:Date:From;\n\tb=cYWa8Sfg2SJWrz/m141Y//OFd2LNBRXoBtekY0DNPn1mWj6PrxxR03yBJd5Xz6hqn\n\tTDxoPHCCUgtnfCRZL4c2mxEMJVSpB8V8BEocqLltZwqSXAIowKubzZdCqfOMYVbNzr\n\txFEfhg+d47bBhmiuKjFBDcHsTFomYzAeXkVhlco4=","Content-Type":"text/plain; charset=\"utf-8\"","MIME-Version":"1.0","Content-Transfer-Encoding":"quoted-printable","In-Reply-To":"<20250901092320.44615-1-barnabas.pocze@ideasonboard.com>","References":"<20250901092320.44615-1-barnabas.pocze@ideasonboard.com>","Subject":"Re: [PATCH v1 1/2] utils: codegen: gen-formats.py: Use jinja","From":"Kieran Bingham ","To":"=?utf-8?q?Barnab=C3=A1s_P=C5=91cze?= ,\n\tlibcamera-devel@lists.libcamera.org","Date":"Mon, 01 Sep 2025 14:25:14 +0100","Message-ID":"<175673311470.560048.10560463581093758798@ping.linuxembedded.co.uk>","User-Agent":"alot/0.9.1","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" "}},{"id":35672,"web_url":"https://patchwork.libcamera.org/comment/35672/","msgid":"","date":"2025-09-01T13:34:08","subject":"Re: [PATCH v1 1/2] utils: codegen: gen-formats.py: Use jinja","submitter":{"id":216,"url":"https://patchwork.libcamera.org/api/people/216/","name":"Barnabás Pőcze","email":"barnabas.pocze@ideasonboard.com"},"content":"Hi\n\n2025. 09. 01. 15:25 keltezéssel, Kieran Bingham írta:\n> Quoting Barnabás Pőcze (2025-09-01 10:23:19)\n>> Currently the gen-formats.py script can only be used to generate C++\n>> code because it hard-codes part of the template. Use jinja to fully\n>> remove any such dependency.\n>>\n> \n> \n> Is there a plan to generate format information elsewhere than the C++\n> header after this ?\n\nPossibly, I'm not entirely sure yet. I needed it for some experiments.\nAnd the next change also depends on it. And many things already use jinja.\n\n\nRegards,\nBarnabás Pőcze\n\n\n> \n> \n> \n>> Signed-off-by: Barnabás Pőcze \n>> ---\n>> include/libcamera/formats.h.in | 4 +++-\n>> utils/codegen/gen-formats.py | 34 ++++++++++++----------------------\n>> 2 files changed, 15 insertions(+), 23 deletions(-)\n>>\n>> diff --git a/include/libcamera/formats.h.in b/include/libcamera/formats.h.in\n>> index 6ae7634fe..5ff9c3bf4 100644\n>> --- a/include/libcamera/formats.h.in\n>> +++ b/include/libcamera/formats.h.in\n>> @@ -35,7 +35,9 @@ constexpr uint64_t __mod(unsigned int vendor, unsigned int mod)\n>> \n>> } /* namespace */\n>> \n>> -${formats}\n>> +{% for f in formats %}\n>> +constexpr PixelFormat {{f.name}}(__fourcc({{f.fourcc}}), __mod({{f.mod}}));\n>> +{%- endfor %}\n>> \n>> } /* namespace formats */\n>> \n>> diff --git a/utils/codegen/gen-formats.py b/utils/codegen/gen-formats.py\n>> index 0c0932a5b..872f3fe34 100755\n>> --- a/utils/codegen/gen-formats.py\n>> +++ b/utils/codegen/gen-formats.py\n>> @@ -7,6 +7,7 @@\n>> # Generate formats definitions from YAML\n>> \n>> import argparse\n>> +import jinja2\n>> import re\n>> import string\n>> import sys\n>> @@ -52,9 +53,7 @@ class DRMFourCC(object):\n>> return self.vendors[vendor], value\n>> \n>> \n>> -def generate_h(formats, drm_fourcc):\n>> - template = string.Template('constexpr PixelFormat ${name}{ __fourcc(${fourcc}), __mod(${mod}) };')\n>> -\n>> +def generate_formats(formats, drm_fourcc):\n>> fmts = []\n>> \n>> for format in formats:\n>> @@ -73,24 +72,17 @@ def generate_h(formats, drm_fourcc):\n>> if mod:\n>> data['mod'] = '%u, %u' % drm_fourcc.mod(mod)\n>> \n>> - fmts.append(template.substitute(data))\n>> -\n>> - return {'formats': '\\n'.join(fmts)}\n>> -\n>> -\n>> -def fill_template(template, data):\n>> + fmts.append(data)\n>> \n>> - template = open(template, 'rb').read()\n>> - template = template.decode('utf-8')\n>> - template = string.Template(template)\n>> - return template.substitute(data)\n>> + return fmts\n>> \n>> \n>> def main(argv):\n>> \n>> # Parse command line arguments\n>> parser = argparse.ArgumentParser()\n>> - parser.add_argument('-o', dest='output', metavar='file', type=str,\n>> + parser.add_argument('-o', dest='output', metavar='file',\n>> + type=argparse.FileType('w', encoding='utf-8'), default=sys.stdout,\n>> help='Output file name. Defaults to standard output if not specified.')\n>> parser.add_argument('input', type=str,\n>> help='Input file name.')\n>> @@ -104,15 +96,13 @@ def main(argv):\n>> formats = yaml.safe_load(data)['formats']\n>> drm_fourcc = DRMFourCC(args.drm_fourcc)\n>> \n>> - data = generate_h(formats, drm_fourcc)\n>> - data = fill_template(args.template, data)\n>> + env = jinja2.Environment()\n>> + template = env.from_string(open(args.template, 'r', encoding='utf-8').read())\n>> + string = template.render({\n>> + 'formats': generate_formats(formats, drm_fourcc),\n>> + })\n>> \n>> - if args.output:\n>> - output = open(args.output, 'wb')\n>> - output.write(data.encode('utf-8'))\n>> - output.close()\n>> - else:\n>> - sys.stdout.write(data)\n>> + args.output.write(string)\n> \n> This looks reasonable to me.\n> \n> All looks ok to me and we already use jinja so :\n> \n> Reviewed-by: Kieran Bingham \n> \n>> \n>> return 0\n>> \n>> -- \n>> 2.51.0\n>>","headers":{"Return-Path":"","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id 5E82FBDB13\n\tfor ;\n\tMon, 1 Sep 2025 13:34:15 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id CBD1C69338;\n\tMon, 1 Sep 2025 15:34:13 +0200 (CEST)","from perceval.ideasonboard.com (perceval.ideasonboard.com\n\t[IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id 2CAA269323\n\tfor ;\n\tMon, 1 Sep 2025 15:34:12 +0200 (CEST)","from [192.168.33.23] (185.221.143.232.nat.pool.zt.hu\n\t[185.221.143.232])\n\tby perceval.ideasonboard.com (Postfix) with ESMTPSA id AD533EAE;\n\tMon, 1 Sep 2025 15:33:04 +0200 (CEST)"],"Authentication-Results":"lancelot.ideasonboard.com; dkim=pass (1024-bit key;\n\tunprotected) header.d=ideasonboard.com header.i=@ideasonboard.com\n\theader.b=\"cljl7ecg\"; dkim-atps=neutral","DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com;\n\ts=mail; t=1756733584;\n\tbh=wlFso3pPxgHWX5lnija1RixaFbm8oy806kv0EzTzmuY=;\n\th=Date:Subject:To:References:From:In-Reply-To:From;\n\tb=cljl7ecgBil0zyrDU/D6ZgVpfsbCQDVSOeZVgdFpZ56kwPxktIeKHWYFL0goE/Ax9\n\trIt3EWDCzhrxXuCvGjG5bnkv3WzRbNgdHOu/XPlwrinmLSFCWwui7otpvYWkPZiDoi\n\tZZ3LqzrFS+8vNrJvPgx3LwWx713m51bQkUEvLoks=","Message-ID":"","Date":"Mon, 1 Sep 2025 15:34:08 +0200","MIME-Version":"1.0","User-Agent":"Mozilla Thunderbird","Subject":"Re: [PATCH v1 1/2] utils: codegen: gen-formats.py: Use jinja","To":"Kieran Bingham ,\n\tlibcamera-devel@lists.libcamera.org","References":"<20250901092320.44615-1-barnabas.pocze@ideasonboard.com>\n\t<175673311470.560048.10560463581093758798@ping.linuxembedded.co.uk>","From":"=?utf-8?q?Barnab=C3=A1s_P=C5=91cze?= ","Content-Language":"en-US, hu-HU","In-Reply-To":"<175673311470.560048.10560463581093758798@ping.linuxembedded.co.uk>","Content-Type":"text/plain; charset=UTF-8; format=flowed","Content-Transfer-Encoding":"8bit","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" "}},{"id":35676,"web_url":"https://patchwork.libcamera.org/comment/35676/","msgid":"<20250901143428.GC1705@pendragon.ideasonboard.com>","date":"2025-09-01T14:34:28","subject":"Re: [PATCH v1 1/2] utils: codegen: gen-formats.py: Use jinja","submitter":{"id":2,"url":"https://patchwork.libcamera.org/api/people/2/","name":"Laurent Pinchart","email":"laurent.pinchart@ideasonboard.com"},"content":"Hi Barnabás,\n\nThank you for the patch.\n\nOn Mon, Sep 01, 2025 at 11:23:19AM +0200, Barnabás Pőcze wrote:\n> Currently the gen-formats.py script can only be used to generate C++\n> code because it hard-codes part of the template. Use jinja to fully\n> remove any such dependency.\n> \n> Signed-off-by: Barnabás Pőcze \n> ---\n> include/libcamera/formats.h.in | 4 +++-\n> utils/codegen/gen-formats.py | 34 ++++++++++++----------------------\n> 2 files changed, 15 insertions(+), 23 deletions(-)\n> \n> diff --git a/include/libcamera/formats.h.in b/include/libcamera/formats.h.in\n> index 6ae7634fe..5ff9c3bf4 100644\n> --- a/include/libcamera/formats.h.in\n> +++ b/include/libcamera/formats.h.in\n> @@ -35,7 +35,9 @@ constexpr uint64_t __mod(unsigned int vendor, unsigned int mod)\n> \n> } /* namespace */\n> \n> -${formats}\n> +{% for f in formats %}\n> +constexpr PixelFormat {{f.name}}(__fourcc({{f.fourcc}}), __mod({{f.mod}}));\n> +{%- endfor %}\n> \n> } /* namespace formats */\n> \n> diff --git a/utils/codegen/gen-formats.py b/utils/codegen/gen-formats.py\n> index 0c0932a5b..872f3fe34 100755\n> --- a/utils/codegen/gen-formats.py\n> +++ b/utils/codegen/gen-formats.py\n> @@ -7,6 +7,7 @@\n> # Generate formats definitions from YAML\n> \n> import argparse\n> +import jinja2\n> import re\n> import string\n> import sys\n> @@ -52,9 +53,7 @@ class DRMFourCC(object):\n> return self.vendors[vendor], value\n> \n> \n> -def generate_h(formats, drm_fourcc):\n> - template = string.Template('constexpr PixelFormat ${name}{ __fourcc(${fourcc}), __mod(${mod}) };')\n> -\n> +def generate_formats(formats, drm_fourcc):\n> fmts = []\n> \n> for format in formats:\n> @@ -73,24 +72,17 @@ def generate_h(formats, drm_fourcc):\n> if mod:\n> data['mod'] = '%u, %u' % drm_fourcc.mod(mod)\n> \n> - fmts.append(template.substitute(data))\n> -\n> - return {'formats': '\\n'.join(fmts)}\n> -\n> -\n> -def fill_template(template, data):\n> + fmts.append(data)\n> \n> - template = open(template, 'rb').read()\n> - template = template.decode('utf-8')\n> - template = string.Template(template)\n> - return template.substitute(data)\n> + return fmts\n> \n> \n> def main(argv):\n> \n> # Parse command line arguments\n> parser = argparse.ArgumentParser()\n> - parser.add_argument('-o', dest='output', metavar='file', type=str,\n> + parser.add_argument('-o', dest='output', metavar='file',\n> + type=argparse.FileType('w', encoding='utf-8'), default=sys.stdout,\n\nI like this. Could you send a patch to do the same in the other code\ngeneration scripts ?\n\nReviewed-by: Laurent Pinchart \n\n> help='Output file name. Defaults to standard output if not specified.')\n> parser.add_argument('input', type=str,\n> help='Input file name.')\n> @@ -104,15 +96,13 @@ def main(argv):\n> formats = yaml.safe_load(data)['formats']\n> drm_fourcc = DRMFourCC(args.drm_fourcc)\n> \n> - data = generate_h(formats, drm_fourcc)\n> - data = fill_template(args.template, data)\n> + env = jinja2.Environment()\n> + template = env.from_string(open(args.template, 'r', encoding='utf-8').read())\n> + string = template.render({\n> + 'formats': generate_formats(formats, drm_fourcc),\n> + })\n> \n> - if args.output:\n> - output = open(args.output, 'wb')\n> - output.write(data.encode('utf-8'))\n> - output.close()\n> - else:\n> - sys.stdout.write(data)\n> + args.output.write(string)\n> \n> return 0\n>","headers":{"Return-Path":"","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id 602F2BDB13\n\tfor ;\n\tMon, 1 Sep 2025 14:34:52 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id 8169169332;\n\tMon, 1 Sep 2025 16:34:51 +0200 (CEST)","from perceval.ideasonboard.com (perceval.ideasonboard.com\n\t[IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id 4A30069323\n\tfor ;\n\tMon, 1 Sep 2025 16:34:49 +0200 (CEST)","from pendragon.ideasonboard.com\n\t(230.215-178-91.adsl-dyn.isp.belgacom.be [91.178.215.230])\n\tby perceval.ideasonboard.com (Postfix) with UTF8SMTPSA id E582C226;\n\tMon, 1 Sep 2025 16:33:41 +0200 (CEST)"],"Authentication-Results":"lancelot.ideasonboard.com; dkim=pass (1024-bit key;\n\tunprotected) header.d=ideasonboard.com header.i=@ideasonboard.com\n\theader.b=\"sSHkMeIR\"; dkim-atps=neutral","DKIM-Signature":"v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com;\n\ts=mail; t=1756737222;\n\tbh=kHoFTKLcIW1LHCbOjQKBKIG4nenflnkV4TWim4vYfos=;\n\th=Date:From:To:Cc:Subject:References:In-Reply-To:From;\n\tb=sSHkMeIRoIJk2WAPvBGEXDe8IkgKwK1dhWdkZnynz6bNzm0IC9DJGAK5MX8yseYK9\n\tsXBlhI1uA7RQpTaFNtrmqhRGI6EOqunoh/nNE+v2FUqZYUGUrk2cJm7EM4cmkah5ct\n\tGpYAnekpeoeA3COdHMcjATWUf+aegVKg4QOp/gFY=","Date":"Mon, 1 Sep 2025 16:34:28 +0200","From":"Laurent Pinchart ","To":"=?utf-8?q?Barnab=C3=A1s_P=C5=91cze?= ","Cc":"libcamera-devel@lists.libcamera.org","Subject":"Re: [PATCH v1 1/2] utils: codegen: gen-formats.py: Use jinja","Message-ID":"<20250901143428.GC1705@pendragon.ideasonboard.com>","References":"<20250901092320.44615-1-barnabas.pocze@ideasonboard.com>","MIME-Version":"1.0","Content-Type":"text/plain; charset=utf-8","Content-Disposition":"inline","Content-Transfer-Encoding":"8bit","In-Reply-To":"<20250901092320.44615-1-barnabas.pocze@ideasonboard.com>","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" "}}]