From patchwork Thu Nov 24 11:35:38 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Paul Elder X-Patchwork-Id: 17867 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 2FD4ABDE6B for ; Thu, 24 Nov 2022 11:36:04 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 97F216331A; Thu, 24 Nov 2022 12:36:03 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1669289763; bh=fHurnCMbzcStEKGykH5upEThLGj24CLlPryxaPDKOYI=; h=To:Date:Subject:List-Id:List-Unsubscribe:List-Archive:List-Post: List-Help:List-Subscribe:From:Reply-To:From; b=EtNEXFYkLNzcdUpq4EHGG65rYyaSW/gm/358ZlmYCweXz/SCkKBNCSY8PJcWcrnTF pHxg5GDKu8MTM3ViVseet4j3+76X6eTvzMPVqOli175oU4B+7IqsF8aGYCuzSd3Jgx +moeB/W41QDyiugAuM5GDNIh1hpZOKWuASjGBf5Bne8h+O9/Zfy/jaCRCrSeRP4fro 1YsfGw/I+Lq3kBBlDUliq4sbEjCfQwIou7oI2NbxTBCIuRxKkcHUaoC6JoWeR61ems +MkJiLOWO8tXOATXgSnVze6qH595I3+YbmgidetijJ2L1bH3bZ1gF/47P3Z0BffyqQ BVEORhxnanq6w== Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id F0EF6632EA for ; Thu, 24 Nov 2022 12:36:01 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="OFo9UPYf"; dkim-atps=neutral Received: from pyrite.tail37cf.ts.net (h175-177-042-159.catv02.itscom.jp [175.177.42.159]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 7B630496; Thu, 24 Nov 2022 12:36:00 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1669289761; bh=fHurnCMbzcStEKGykH5upEThLGj24CLlPryxaPDKOYI=; h=From:To:Cc:Subject:Date:From; b=OFo9UPYfzaZmHIl6Mf2Txvgi/HhHWwcO1haDz8/vS5GWFM4RXQR4JZcad94Oov8zV smr1aFKFksuQurU/v1JwVheJ2WjG43L0QDZgpohnjwNxk7fJw34DQx4jtVKpRWAsiZ oChP2MEBg95v3OudlqdHD7ZyankuRWtoX92bCk0s= To: libcamera-devel@lists.libcamera.org Date: Thu, 24 Nov 2022 20:35:38 +0900 Message-Id: <20221124113550.2182342-1-paul.elder@ideasonboard.com> X-Mailer: git-send-email 2.35.1 MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v4 00/12] utils: tuning: Add a new tuning infrastructure X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Paul Elder via libcamera-devel From: Paul Elder Reply-To: Paul Elder Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" This patch series adds a new tuning infrastructure called libtuning, inspired by ctt. The new changes in v4 are fairly minor. One notable change is that the less-accurate rounding copied from ctt was replaced with simple rounding, so the output no longer matches *exactly* what ctt outputs. Main non-cosmetic changes in v3: - remove full raspberry pi tuning script - split core libtuning implementation into three patches The design modularizes common components of tuning tools such that new tuning scripts for new platforms can be created easily, as show in patches 11/12 to 12/12 (v3: the full Raspberry Pi tuning script was removed because it is DNI and clearly incomplete). The common "core" components include file management, argument parsing, image loading and validation, and macbeth chart detection, as well as miscellaneous (but tedious) math utilities. It connects everything together such that a platform's tuning script can very easily customize tuning modules in a variety of ways, even including the format of the input configuration file and the output tuning file. These are all implemented in patches 1/12 to 3/12 (split from one to three patches as of v3). The input configuration file and output tuning file could have different formats as well, hence why these have their own classes. As of v1, only the raspberrypi's formats were implemented, in patches 7/12 and 8/12 respectively. As of v2, output for yaml was added in patch 10/12. A skeleton for yaml input is added as well in patch 9/12, though it is not implemented yet as there is no specification for the yaml input configuration format, and the only existing user of it doesn't actually need a configuration file. As of v2, it became apparent that it was infeasible to create an ALSC module (that's what we're starting with) that could completely support different platforms with configuration options alone. So, the ALSC module was split into a base one (patch 4/12), with variations for raspberrypi and rkisp1 implemented on top of it in patches 5/12 and 6/12 respectively. I think they came out quite nice, and they are still more manageable than an entirely new tuning module per platform. v2: I have also since managed to get test images and so this entire thing runs! Even though the rkisp1 tuning script (patch 12/12) says "WIP", it does run and outputs a valid tuning file. I haven't tried using the tuning file with libcamera though, as it is missng the /other/ algorithm tuning results. The output from the libtuning-based alsc-only raspberrypi tuning script (patch 11/12) has been confirmed to be character-for-character exactly the same as the output from ctt's alsc-only tuning script. Paul Elder (12): utils: tuning: libtuning: Implement the core of libtuning utils: tuning: libtuning: Implement math helpers utils: tuning: libtuning: Implement extensible components of libtuning utils: libtuning: modules: Add base LSC module utils: libtuning: modules: alsc: Add raspberrypi ALSC module utils: libtuning: modules: alsc: Add rkisp1 LSC module utils: libtuning: parsers: Add raspberrypi parser utils: libtuning: generators: Add raspberrypi output utils: libtuning: parsers: Add yaml parser utils: libtuning: generators: Add yaml output utils: tuning: Add alsc-only libtuning raspberrypi tuning script utils: tuning: Add tuning script for rkisp1 utils/tuning/README.rst | 11 + utils/tuning/libtuning/__init__.py | 13 + utils/tuning/libtuning/average.py | 21 + utils/tuning/libtuning/generators/__init__.py | 6 + .../tuning/libtuning/generators/generator.py | 15 + .../generators/raspberrypi_output.py | 114 ++++ .../libtuning/generators/yaml_output.py | 123 +++++ utils/tuning/libtuning/gradient.py | 75 +++ utils/tuning/libtuning/image.py | 136 +++++ utils/tuning/libtuning/libtuning.py | 208 +++++++ utils/tuning/libtuning/macbeth.py | 516 ++++++++++++++++++ utils/tuning/libtuning/macbeth_ref.pgm | 6 + utils/tuning/libtuning/modules/__init__.py | 0 .../tuning/libtuning/modules/lsc/__init__.py | 7 + utils/tuning/libtuning/modules/lsc/lsc.py | 72 +++ .../libtuning/modules/lsc/raspberrypi.py | 246 +++++++++ utils/tuning/libtuning/modules/lsc/rkisp1.py | 112 ++++ utils/tuning/libtuning/modules/module.py | 32 ++ utils/tuning/libtuning/parsers/__init__.py | 6 + utils/tuning/libtuning/parsers/parser.py | 21 + .../libtuning/parsers/raspberrypi_parser.py | 93 ++++ utils/tuning/libtuning/parsers/yaml_parser.py | 17 + utils/tuning/libtuning/smoothing.py | 24 + utils/tuning/libtuning/utils.py | 125 +++++ utils/tuning/raspberrypi/__init__.py | 0 utils/tuning/raspberrypi/alsc.py | 19 + utils/tuning/raspberrypi_alsc_only.py | 23 + utils/tuning/rkisp1.py | 40 ++ 28 files changed, 2081 insertions(+) create mode 100644 utils/tuning/README.rst create mode 100644 utils/tuning/libtuning/__init__.py create mode 100644 utils/tuning/libtuning/average.py create mode 100644 utils/tuning/libtuning/generators/__init__.py create mode 100644 utils/tuning/libtuning/generators/generator.py create mode 100644 utils/tuning/libtuning/generators/raspberrypi_output.py create mode 100644 utils/tuning/libtuning/generators/yaml_output.py create mode 100644 utils/tuning/libtuning/gradient.py create mode 100644 utils/tuning/libtuning/image.py create mode 100644 utils/tuning/libtuning/libtuning.py create mode 100644 utils/tuning/libtuning/macbeth.py create mode 100644 utils/tuning/libtuning/macbeth_ref.pgm create mode 100644 utils/tuning/libtuning/modules/__init__.py create mode 100644 utils/tuning/libtuning/modules/lsc/__init__.py create mode 100644 utils/tuning/libtuning/modules/lsc/lsc.py create mode 100644 utils/tuning/libtuning/modules/lsc/raspberrypi.py create mode 100644 utils/tuning/libtuning/modules/lsc/rkisp1.py create mode 100644 utils/tuning/libtuning/modules/module.py create mode 100644 utils/tuning/libtuning/parsers/__init__.py create mode 100644 utils/tuning/libtuning/parsers/parser.py create mode 100644 utils/tuning/libtuning/parsers/raspberrypi_parser.py create mode 100644 utils/tuning/libtuning/parsers/yaml_parser.py create mode 100644 utils/tuning/libtuning/smoothing.py create mode 100644 utils/tuning/libtuning/utils.py create mode 100644 utils/tuning/raspberrypi/__init__.py create mode 100644 utils/tuning/raspberrypi/alsc.py create mode 100755 utils/tuning/raspberrypi_alsc_only.py create mode 100755 utils/tuning/rkisp1.py Reviewed-by: Laurent Pinchart