From patchwork Sat Oct 22 06:22:59 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Paul Elder X-Patchwork-Id: 17652 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id DF313C0DA4 for ; Sat, 22 Oct 2022 06:23:23 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 18D8262EC6; Sat, 22 Oct 2022 08:23:23 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1666419803; bh=FhneLda9TqzA9v/1BCMsyNYoXUd+/PSlMi4oLZdjCNs=; h=To:Date:Subject:List-Id:List-Unsubscribe:List-Archive:List-Post: List-Help:List-Subscribe:From:Reply-To:From; b=BWQFKeNkepS9PWhSx4j5xAXm17dRnva7ZIPkhIQjlqAie3otFaXEiSQzwM9aSLEl/ dDfAlBHrxwHC3rZUm95pZkW8r4cgQr8pFpS2NBjcRFsTAcOJnucenrNAa3lCPyKbqa hJlWYBdv/ij7HbSmCj35qwV5+lMq86kv89V8iNzrxpBDg8rhCqRM/Nivc6Dh1ZiLKq KW0J6Fd0KhtdqFnk3M5Tgf2shf5myEql2kocL8uZ+4OnxgDEaZ1LLGcOQHbu/eSasK kl1G/GksALcIkSLipMWBA027fittyxZv6Kjm5fDUViVy/t6fhjpSbXYgoawSUMY9aI 8yLEGC3+6Cuiw== Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id DF91F61F51 for ; Sat, 22 Oct 2022 08:23:20 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="fIZI97AU"; dkim-atps=neutral Received: from pyrite.rasen.tech (h175-177-042-159.catv02.itscom.jp [175.177.42.159]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 523DE106; Sat, 22 Oct 2022 08:23:19 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1666419800; bh=FhneLda9TqzA9v/1BCMsyNYoXUd+/PSlMi4oLZdjCNs=; h=From:To:Cc:Subject:Date:From; b=fIZI97AUUasMggyRAov9JUL0ckNpdTIWeB/mutgLq8IGcuACN2AfnBl82Lc17j9tm TXl7QHO2aj7oq3yccbT/oEwW+ET+4rV9dJ2zzymY+h30SEO9g1RjNRmFHmIZw2Eo49 BxI5XCsHVZlwbuvIPbBQJYHrkndAUi1bDkO4+N9w= To: libcamera-devel@lists.libcamera.org Date: Sat, 22 Oct 2022 15:22:59 +0900 Message-Id: <20221022062310.2545463-1-paul.elder@ideasonboard.com> X-Mailer: git-send-email 2.30.2 MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v2 00/11] utils: tuning: Add a new tuning infrastructure X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Paul Elder via libcamera-devel From: Paul Elder Reply-To: Paul Elder Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" This patch series adds a new tuning infrastructure called libtuning, inspired by ctt. The design modularizes common components of tuning tools such that new tuning scripts for new platforms can be created easily, as show in patches 9/11 to 11/11. The common "core" components include file management, argument parsing, image loading and validation, and macbeth chart detection, as well as miscellaneous (but tedious) math utilities. It connects everything together such that a platform's tuning script can very easily customize tuning modules in a variety of ways, even including the format of the input configuration file and the output tuning file. These are all implemented in patch 1/7, as they are all interconnected. The input configuration file and output tuning file could have different formats as well, hence why these have their own classes. As of v1, only the raspberrypi's formats were implemented, in patches 5/11 and 6/11 respectively. As of v2, output for yaml is added in patch 8/11. A skeleton for yaml input is added as well in patch 7/11, though it is not implemented yet as there is no specification for the yaml input configuration format, and the only existing user of it doesn't actually need a configuration file. As of v2, it became apparent that it was infeasible to create an ALSC module (that's what we're starting with) that could completely support different platforms with configuration options alone. So, the ALSC module was split into a base one (patch 2/11), with variations for raspberrypi and rkisp1 implemented on top of it in patches 3/11 and 4/11 respectively. I think they came out quite nice, and they are still more manageable than an entirely new tuning module per platform. I have also since managed to get test images and so this entire thing runs! Even though the rkisp1 tuning script (patch 11/11) says "WIP", it does run and outputs a valid tuning file. I haven't tried using the tuning file with libcamera though, as it is missng the /other/ algorithm tuning results. The output from the libtuning-based alsc-only raspberrypi tuning script (patch 9/11) has been confirmed to be character-for-character exactly the same as the output from ctt's alsc-only tuning script. Paul Elder (11): utils: tuning: libtuning: Implement the core of libtuning utils: libtuning: modules: Add ALSC module utils: libtuning: modules: alsc: Add raspberrypi ALSC module utils: libtuning: modules: alsc: Add rkisp1 ALSC module utils: libtuning: parsers: Add raspberrypi parser utils: libtuning: generators: Add raspberrypi output [WIP] utils: libtuning: parsers: Add yaml parser utils: libtuning: generators: Add yaml output utils: tuning: Add alsc-only libtuning raspberrypi tuning script [DNI] utils: tuning: Add full libtuning raspberrypi tuning script [WIP] utils: tuning: Add tuning script for rkisp1 utils/tuning/README.md | 7 + utils/tuning/libtuning/__init__.py | 13 + utils/tuning/libtuning/average.py | 22 + utils/tuning/libtuning/generators/__init__.py | 6 + .../tuning/libtuning/generators/generator.py | 16 + .../generators/raspberrypi_output.py | 115 ++++ .../libtuning/generators/yaml_output.py | 121 +++++ utils/tuning/libtuning/gradient.py | 109 ++++ utils/tuning/libtuning/image.py | 137 +++++ utils/tuning/libtuning/libtuning.py | 205 +++++++ utils/tuning/libtuning/macbeth.py | 505 ++++++++++++++++++ utils/tuning/libtuning/macbeth_ref.pgm | 6 + utils/tuning/libtuning/modules/__init__.py | 0 .../tuning/libtuning/modules/alsc/__init__.py | 7 + utils/tuning/libtuning/modules/alsc/alsc.py | 78 +++ .../libtuning/modules/alsc/raspberrypi.py | 246 +++++++++ utils/tuning/libtuning/modules/alsc/rkisp1.py | 112 ++++ utils/tuning/libtuning/modules/module.py | 48 ++ utils/tuning/libtuning/parsers/__init__.py | 6 + utils/tuning/libtuning/parsers/parser.py | 22 + .../libtuning/parsers/raspberrypi_parser.py | 91 ++++ utils/tuning/libtuning/parsers/yaml_parser.py | 15 + utils/tuning/libtuning/smoothing.py | 25 + utils/tuning/libtuning/utils.py | 151 ++++++ utils/tuning/raspberrypi.py | 44 ++ utils/tuning/raspberrypi/__init__.py | 0 utils/tuning/raspberrypi/alsc.py | 17 + utils/tuning/raspberrypi_alsc_only.py | 22 + utils/tuning/rkisp1.py | 58 ++ 29 files changed, 2204 insertions(+) create mode 100644 utils/tuning/README.md create mode 100644 utils/tuning/libtuning/__init__.py create mode 100644 utils/tuning/libtuning/average.py create mode 100644 utils/tuning/libtuning/generators/__init__.py create mode 100644 utils/tuning/libtuning/generators/generator.py create mode 100644 utils/tuning/libtuning/generators/raspberrypi_output.py create mode 100644 utils/tuning/libtuning/generators/yaml_output.py create mode 100644 utils/tuning/libtuning/gradient.py create mode 100644 utils/tuning/libtuning/image.py create mode 100644 utils/tuning/libtuning/libtuning.py create mode 100644 utils/tuning/libtuning/macbeth.py create mode 100644 utils/tuning/libtuning/macbeth_ref.pgm create mode 100644 utils/tuning/libtuning/modules/__init__.py create mode 100644 utils/tuning/libtuning/modules/alsc/__init__.py create mode 100644 utils/tuning/libtuning/modules/alsc/alsc.py create mode 100644 utils/tuning/libtuning/modules/alsc/raspberrypi.py create mode 100644 utils/tuning/libtuning/modules/alsc/rkisp1.py create mode 100644 utils/tuning/libtuning/modules/module.py create mode 100644 utils/tuning/libtuning/parsers/__init__.py create mode 100644 utils/tuning/libtuning/parsers/parser.py create mode 100644 utils/tuning/libtuning/parsers/raspberrypi_parser.py create mode 100644 utils/tuning/libtuning/parsers/yaml_parser.py create mode 100644 utils/tuning/libtuning/smoothing.py create mode 100644 utils/tuning/libtuning/utils.py create mode 100644 utils/tuning/raspberrypi.py create mode 100644 utils/tuning/raspberrypi/__init__.py create mode 100644 utils/tuning/raspberrypi/alsc.py create mode 100755 utils/tuning/raspberrypi_alsc_only.py create mode 100755 utils/tuning/rkisp1.py