From patchwork Mon Jul 18 08:16:00 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Naushir Patuck X-Patchwork-Id: 16673 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 93D15BD1F1 for ; Mon, 18 Jul 2022 08:16:15 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 255856332A; Mon, 18 Jul 2022 10:16:15 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1658132175; bh=y9QEoioXKnqCMjHNtfAWEm/WKvhWfBTuVbqwgrko3aI=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To: From; b=bZSenOfYQDwyXtahjiGiRsK+t6qWHpRnu7/eR3yCbN8QTE/r0pDFy1dbX32c+Z1sR 9OesYRq5XcWdJ1Jl3GIvtBtD5elFW6fb8ZCkJ1iQIffqeAkc0UDg/Jl1vZzBJ7rPTP oAkUgRt++Sd5dkt3sh1sFr5Vkp6VRjsB8PocdhfTPk/LlV+/82smoUiUK/EcqczW5H r36HEndG2sSFSuEBcyJCFwFYqHJ+M4YGPxxZknHASm3ZenxeB3v2wZu2fNbXlOyWkW TLMq2/vUmIANx5vVUScN9d9ar3BqJImAN5UBDSJaEpYt7cPtRne6JZL9iUqq5H4zOo VN+FM3VHv91Rw== Received: from mail-wr1-x436.google.com (mail-wr1-x436.google.com [IPv6:2a00:1450:4864:20::436]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 4019B6048A for ; Mon, 18 Jul 2022 10:16:11 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="NQVpsMWd"; dkim-atps=neutral Received: by mail-wr1-x436.google.com with SMTP id bu1so15871437wrb.9 for ; Mon, 18 Jul 2022 01:16:11 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=lKai8tSSfjJFtWqCINHlH3zNlWNwg7ZRxTrsnBkmxuI=; b=NQVpsMWdMm0q/8QCP5E/llxsi5r2Ei37UFt+u8UuQzOorc0sMmNpQ1rK8Z69A8+D1w DVSc4lkIQW86aTa45gWlgaflHFWFqwMN0oXW0OuWtYF/PAo9eoQR6qr3WOqFDMVeORGO Hx0svj/BMp6lKe9hCpZrDT0I2zEgyxUuTtyXu1k5v5uD7cP71TM5iIcq/fRnaar1IaM5 d0P0SHuQajphocfo0dFF2vTg5lP+AoHRo6cozC2pbivi8DFyvb2ti08fIMPk+Ebh44y7 lSwODYtigfZoFUgQLy5HO8P1X+BItELP5ELvUUMUMofZoVriYtOiwu9rp+aTFmmSrmLi fLrw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=lKai8tSSfjJFtWqCINHlH3zNlWNwg7ZRxTrsnBkmxuI=; b=BaZ8YIVHBNZgRqhJ9DRtJWt5nQ98e/W/ntf4j+yvgcIhC/bP4x2bEWqcdg6KaH/ksD +ek0G00Iw0nkBe28Gb0b92LKoxKXcLG48BaoQjP1bLhIruN0a6PhxrlJDIPDVfeN6Mhz LBFrofuLHAuM3560a50AeHLbR1eqK27bFLGSCXz86EPbWANYV1GQ3peBbNjplzqlW0tP JajUkAzNvBTC3grx1yUiqVUGDesZ5DB5jm4u4lwpb8gWHxZkyZTPdWW3N0CfHE6nY6Y2 /s3Cyyn3jLy4/qvckTu3C5WRU0Iag2fVaJlk6FJhJqxvUl7L57ToebTz72cJ6L4r231f +ULg== X-Gm-Message-State: AJIora+u/pv6OlQh1H8mObCkADOY3VLfQDJoFkBgsQPsstuSdqyXcRhJ V9J39P6Uuh5w43d8wxf58ETgfCFTcrgU5Q== X-Google-Smtp-Source: AGRyM1v4IGsjPC2MBEBjj7NcGmiUWD6Rr7+4yn0WSw+yZ9966MUDxibRAFUKlRMRlaZuJTJtHbAcxQ== X-Received: by 2002:a05:6000:250:b0:21d:b3b5:3438 with SMTP id m16-20020a056000025000b0021db3b53438mr20242055wrz.203.1658132170604; Mon, 18 Jul 2022 01:16:10 -0700 (PDT) Received: from naush-laptop.localdomain ([93.93.133.154]) by smtp.gmail.com with ESMTPSA id a15-20020adffb8f000000b0021dbac444a7sm10206795wrr.59.2022.07.18.01.16.09 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 18 Jul 2022 01:16:09 -0700 (PDT) To: libcamera-devel@lists.libcamera.org Date: Mon, 18 Jul 2022 09:16:00 +0100 Message-Id: <20220718081602.32535-7-naush@raspberrypi.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220718081602.32535-1-naush@raspberrypi.com> References: <20220718081602.32535-1-naush@raspberrypi.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v6 6/8] utils: raspberrypi: ctt: Output version 2.0 format tuning files X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Naushir Patuck via libcamera-devel From: Naushir Patuck Reply-To: Naushir Patuck Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Update the ctt_pretty_print_json.py script to generate the new version 2.0 format camera tuning file. This script can be called through the command line to prettify an existing JSON file, or programatically by the CTT to format a new JSON config dictionary. Update the CTT to produce a version 2.0 format json structure and use ctt_pretty_print_json.pretty_print to prettify the output. Signed-off-by: Naushir Patuck Reviewed-by: Laurent Pinchart --- utils/raspberrypi/ctt/ctt.py | 18 +- .../raspberrypi/ctt/ctt_pretty_print_json.py | 194 +++++++++--------- 2 files changed, 113 insertions(+), 99 deletions(-) mode change 100644 => 100755 utils/raspberrypi/ctt/ctt_pretty_print_json.py diff --git a/utils/raspberrypi/ctt/ctt.py b/utils/raspberrypi/ctt/ctt.py index 15064634c67f..0ec5b49016df 100755 --- a/utils/raspberrypi/ctt/ctt.py +++ b/utils/raspberrypi/ctt/ctt.py @@ -15,7 +15,7 @@ from ctt_alsc import * from ctt_lux import * from ctt_noise import * from ctt_geq import * -from ctt_pretty_print_json import * +from ctt_pretty_print_json import pretty_print import random import json import re @@ -511,13 +511,17 @@ class Camera: """ def write_json(self): """ - Write json dictionary to file + Write json dictionary to file using our version 2 format """ - jstring = json.dumps(self.json, sort_keys=False) - """ - make it pretty :) - """ - pretty_print_json(jstring, self.jf) + + out_json = { + "version": 2.0, + 'target': 'bcm2835', + "algorithms": [{name: data} for name, data in self.json.items()], + } + + with open(self.jf, 'w') as f: + f.write(pretty_print(out_json)) """ add a new section to the log file diff --git a/utils/raspberrypi/ctt/ctt_pretty_print_json.py b/utils/raspberrypi/ctt/ctt_pretty_print_json.py old mode 100644 new mode 100755 index d38ae6178524..ee5e8e10e437 --- a/utils/raspberrypi/ctt/ctt_pretty_print_json.py +++ b/utils/raspberrypi/ctt/ctt_pretty_print_json.py @@ -1,106 +1,116 @@ +#!/usr/bin/env python3 +# # SPDX-License-Identifier: BSD-2-Clause # -# Copyright (C) 2019, Raspberry Pi (Trading) Limited +# Script to pretty print a Raspberry Pi tuning config JSON structure in +# version 2.0 and later formats. # -# ctt_pretty_print_json.py - camera tuning tool JSON formatter - -import sys - - -class JSONPrettyPrinter(object): - """ - Take a collapsed JSON file and make it more readable - """ - def __init__(self, fout): - self.state = { - "indent": 0, - "inarray": [False], - "arraycount": [], - "skipnewline": True, - "need_indent": False, - "need_space": False, +# Copyright 2022 Raspberry Pi Ltd. + +import argparse +import json +import textwrap + + +class Encoder(json.JSONEncoder): + + def __init__(self, *args, **kwargs): + super().__init__(*args, **kwargs) + self.indentation_level = 0 + self.hard_break = 120 + self.custom_elems = { + 'table': 16, + 'luminance_lut': 16, + 'ct_curve': 3, + 'ccm': 3, + 'gamma_curve': 2, + 'y_target': 2, + 'prior': 2 } - self.fout = fout - - def newline(self): - if not self.state["skipnewline"]: - self.fout.write('\n') - self.state["need_indent"] = True - self.state["need_space"] = False - self.state["skipnewline"] = True - - def write(self, c): - if self.state["need_indent"]: - self.fout.write(' ' * self.state["indent"] * 4) - self.state["need_indent"] = False - if self.state["need_space"]: - self.fout.write(' ') - self.state["need_space"] = False - self.fout.write(c) - self.state["skipnewline"] = False - - def process_char(self, c): - if c == '{': - self.newline() - self.write(c) - self.state["indent"] += 1 - self.newline() - elif c == '}': - self.state["indent"] -= 1 - self.newline() - self.write(c) - elif c == '[': - self.newline() - self.write(c) - self.state["indent"] += 1 - self.newline() - self.state["inarray"] = [True] + self.state["inarray"] - self.state["arraycount"] = [0] + self.state["arraycount"] - elif c == ']': - self.state["indent"] -= 1 - self.newline() - self.state["inarray"].pop(0) - self.state["arraycount"].pop(0) - self.write(c) - elif c == ':': - self.write(c) - self.state["need_space"] = True - elif c == ',': - if not self.state["inarray"][0]: - self.write(c) - self.newline() + def encode(self, o, node_key=None): + if isinstance(o, (list, tuple)): + # Check if we are a flat list of numbers. + if not any(isinstance(el, (list, tuple, dict)) for el in o): + s = ', '.join(json.dumps(el) for el in o) + if node_key in self.custom_elems.keys(): + # Special case handling to specify number of elements in a row for tables, ccm, etc. + self.indentation_level += 1 + sl = s.split(', ') + num = self.custom_elems[node_key] + chunk = [self.indent_str + ', '.join(sl[x:x + num]) for x in range(0, len(sl), num)] + t = ',\n'.join(chunk) + self.indentation_level -= 1 + output = f'\n{self.indent_str}[\n{t}\n{self.indent_str}]' + elif len(s) > self.hard_break - len(self.indent_str): + # Break a long list with wraps. + self.indentation_level += 1 + t = textwrap.fill(s, self.hard_break, break_long_words=False, + initial_indent=self.indent_str, subsequent_indent=self.indent_str) + self.indentation_level -= 1 + output = f'\n{self.indent_str}[\n{t}\n{self.indent_str}]' + else: + # Smaller lists can remain on a single line. + output = f' [ {s} ]' + return output else: - self.write(c) - self.state["arraycount"][0] += 1 - if self.state["arraycount"][0] == 16: - self.state["arraycount"][0] = 0 - self.newline() + # Sub-structures in the list case. + self.indentation_level += 1 + output = [self.indent_str + self.encode(el) for el in o] + self.indentation_level -= 1 + output = ',\n'.join(output) + return f' [\n{output}\n{self.indent_str}]' + + elif isinstance(o, dict): + self.indentation_level += 1 + output = [] + for k, v in o.items(): + if isinstance(v, dict) and len(v) == 0: + # Empty config block special case. + output.append(self.indent_str + f'{json.dumps(k)}: {{ }}') else: - self.state["need_space"] = True - elif c.isspace(): - pass + # Only linebreak if the next node is a config block. + sep = f'\n{self.indent_str}' if isinstance(v, dict) else '' + output.append(self.indent_str + f'{json.dumps(k)}:{sep}{self.encode(v, k)}') + output = ',\n'.join(output) + self.indentation_level -= 1 + return f'{{\n{output}\n{self.indent_str}}}' + else: - self.write(c) + return ' ' + json.dumps(o) + + @property + def indent_str(self) -> str: + return ' ' * self.indentation_level * self.indent + + def iterencode(self, o, **kwargs): + return self.encode(o) + + +def pretty_print(in_json: dict) -> str: + + if 'version' not in in_json or \ + 'target' not in in_json or \ + 'algorithms' not in in_json or \ + in_json['version'] < 2.0: + raise RuntimeError('Incompatible JSON dictionary has been provided') - def print(self, string): - for c in string: - self.process_char(c) - self.newline() + return json.dumps(in_json, cls=Encoder, indent=4, sort_keys=False) -def pretty_print_json(str_in, output_filename): - with open(output_filename, "w") as fout: - printer = JSONPrettyPrinter(fout) - printer.print(str_in) +if __name__ == "__main__": + parser = argparse.ArgumentParser(formatter_class=argparse.RawTextHelpFormatter, description= + 'Prettify a version 2.0 camera tuning config JSON file.') + parser.add_argument('input', type=str, help='Input tuning file.') + parser.add_argument('output', type=str, nargs='?', + help='Output converted tuning file. If not provided, the input file will be updated in-place.', + default=None) + args = parser.parse_args() + with open(args.input, 'r') as f: + in_json = json.load(f) -if __name__ == '__main__': - if len(sys.argv) != 2: - print("Usage: %s filename" % sys.argv[0]) - sys.exit(1) + out_json = pretty_print(in_json) - input_filename = sys.argv[1] - with open(input_filename, "r") as fin: - printer = JSONPrettyPrinter(sys.stdout) - printer.print(fin.read()) + with open(args.output if args.output is not None else args.input, 'w') as f: + f.write(out_json)