From patchwork Thu Nov 10 17:31:42 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Paul Elder X-Patchwork-Id: 17772 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id D38CABD16B for ; Thu, 10 Nov 2022 17:32:18 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 27EB96307B; Thu, 10 Nov 2022 18:32:18 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1668101538; bh=lKhbnIFB+V6BCKgPOWGcXBL46cg5uMwafLR/iFaKj8k=; h=To:Date:Subject:List-Id:List-Unsubscribe:List-Archive:List-Post: List-Help:List-Subscribe:From:Reply-To:From; b=rEJhp6uSZ5SSFSQK2qbKdX/yAwJ++K5psG4VywgOYKh2B+kB0OVIVz5/0cf9pbUmz B0J6wyH4XClAwUsAHmRvCYduv1vlOgHda3NvzdeW/JKPyWTcqAIIgbX6hQFD89kzfP MKFomx0OjWsT2c/xhEk/jHt0I8zBAQ6C2lXyKeRVgN0WprfqHiLvcTYMOKiuPB3ba2 BnjNeM9FJdBW6MCm8/wCIyQr0boS/IaFu3/t04aytjOSgLpWynS4QhY8OdZLZl1Xm2 /T1uQtLgJppsrJXtavoyCkDeaZP7WTzoChF+xnlwTTyFs/CyiA7GYbyXlSpozSF39I KpJXgcfTt5nGw== Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 32ED263075 for ; Thu, 10 Nov 2022 18:32:16 +0100 (CET) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="alLuzwz1"; dkim-atps=neutral Received: from pyrite.tail37cf.ts.net (h175-177-042-159.catv02.itscom.jp [175.177.42.159]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 72BBF32A; Thu, 10 Nov 2022 18:32:14 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1668101535; bh=lKhbnIFB+V6BCKgPOWGcXBL46cg5uMwafLR/iFaKj8k=; h=From:To:Cc:Subject:Date:From; b=alLuzwz1thzUXSGLQgY2b4hKkijG1Mm41xA9wsvWpQG7lCOpWaXMfFTgYaFUJ0K+a NrNPSUeGI1EaKNiSeFiw76tWetbqnq1AKD9G6l3n0Hnu0LmcQU/oNDXBGIJLdCksIJ NBsaIw2/5U0OrqUV5zuoPYn8CLKYSJ/LmXKZj4JI= To: libcamera-devel@lists.libcamera.org Date: Fri, 11 Nov 2022 02:31:42 +0900 Message-Id: <20221110173154.488445-1-paul.elder@ideasonboard.com> X-Mailer: git-send-email 2.35.1 MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v3 00/12] utils: tuning: Add a new tuning infrastructure X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Paul Elder via libcamera-devel From: Paul Elder Reply-To: Paul Elder Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" This patch series adds a new tuning infrastructure called libtuning, inspired by ctt. Main non-cosmetic changes in v3: - remove full raspberry pi tuning script - split core libtuning implementation into three patches The design modularizes common components of tuning tools such that new tuning scripts for new platforms can be created easily, as show in patches 11/12 to 12/12 (v3: the full Raspberry Pi tuning script was removed because it is DNI and clearly incomplete). The common "core" components include file management, argument parsing, image loading and validation, and macbeth chart detection, as well as miscellaneous (but tedious) math utilities. It connects everything together such that a platform's tuning script can very easily customize tuning modules in a variety of ways, even including the format of the input configuration file and the output tuning file. These are all implemented in patches 1/12 to 3/12 (split from one to three patches as of v3). The input configuration file and output tuning file could have different formats as well, hence why these have their own classes. As of v1, only the raspberrypi's formats were implemented, in patches 7/12 and 8/12 respectively. As of v2, output for yaml was added in patch 10/12. A skeleton for yaml input is added as well in patch 9/12, though it is not implemented yet as there is no specification for the yaml input configuration format, and the only existing user of it doesn't actually need a configuration file. As of v2, it became apparent that it was infeasible to create an ALSC module (that's what we're starting with) that could completely support different platforms with configuration options alone. So, the ALSC module was split into a base one (patch 4/12), with variations for raspberrypi and rkisp1 implemented on top of it in patches 5/12 and 6/12 respectively. I think they came out quite nice, and they are still more manageable than an entirely new tuning module per platform. v2: I have also since managed to get test images and so this entire thing runs! Even though the rkisp1 tuning script (patch 12/12) says "WIP", it does run and outputs a valid tuning file. I haven't tried using the tuning file with libcamera though, as it is missng the /other/ algorithm tuning results. The output from the libtuning-based alsc-only raspberrypi tuning script (patch 11/12) has been confirmed to be character-for-character exactly the same as the output from ctt's alsc-only tuning script. (P.S. I'll be uncontactable by IRC for a while because of DNS.) Paul Elder (12): utils: tuning: libtuning: Implement the core of libtuning utils: tuning: libtuning: Implement math helpers utils: tuning: libtuning: Implement extensible components of libtuning utils: libtuning: modules: Add base LSC module utils: libtuning: modules: alsc: Add raspberrypi ALSC module utils: libtuning: modules: alsc: Add rkisp1 LSC module utils: libtuning: parsers: Add raspberrypi parser utils: libtuning: generators: Add raspberrypi output utils: libtuning: parsers: Add yaml parser utils: libtuning: generators: Add yaml output utils: tuning: Add alsc-only libtuning raspberrypi tuning script utils: tuning: Add tuning script for rkisp1 utils/tuning/README.rst | 11 + utils/tuning/libtuning/__init__.py | 13 + utils/tuning/libtuning/average.py | 21 + utils/tuning/libtuning/generators/__init__.py | 6 + .../tuning/libtuning/generators/generator.py | 15 + .../generators/raspberrypi_output.py | 114 ++++ .../libtuning/generators/yaml_output.py | 123 +++++ utils/tuning/libtuning/gradient.py | 75 +++ utils/tuning/libtuning/image.py | 133 +++++ utils/tuning/libtuning/libtuning.py | 203 +++++++ utils/tuning/libtuning/macbeth.py | 516 ++++++++++++++++++ utils/tuning/libtuning/macbeth_ref.pgm | 6 + utils/tuning/libtuning/modules/__init__.py | 0 .../tuning/libtuning/modules/lsc/__init__.py | 7 + utils/tuning/libtuning/modules/lsc/lsc.py | 72 +++ .../libtuning/modules/lsc/raspberrypi.py | 250 +++++++++ utils/tuning/libtuning/modules/lsc/rkisp1.py | 114 ++++ utils/tuning/libtuning/modules/module.py | 33 ++ utils/tuning/libtuning/parsers/__init__.py | 6 + utils/tuning/libtuning/parsers/parser.py | 21 + .../libtuning/parsers/raspberrypi_parser.py | 93 ++++ utils/tuning/libtuning/parsers/yaml_parser.py | 17 + utils/tuning/libtuning/smoothing.py | 24 + utils/tuning/libtuning/utils.py | 152 ++++++ utils/tuning/raspberrypi/__init__.py | 0 utils/tuning/raspberrypi/alsc.py | 19 + utils/tuning/raspberrypi_alsc_only.py | 23 + utils/tuning/rkisp1.py | 43 ++ 28 files changed, 2110 insertions(+) create mode 100644 utils/tuning/README.rst create mode 100644 utils/tuning/libtuning/__init__.py create mode 100644 utils/tuning/libtuning/average.py create mode 100644 utils/tuning/libtuning/generators/__init__.py create mode 100644 utils/tuning/libtuning/generators/generator.py create mode 100644 utils/tuning/libtuning/generators/raspberrypi_output.py create mode 100644 utils/tuning/libtuning/generators/yaml_output.py create mode 100644 utils/tuning/libtuning/gradient.py create mode 100644 utils/tuning/libtuning/image.py create mode 100644 utils/tuning/libtuning/libtuning.py create mode 100644 utils/tuning/libtuning/macbeth.py create mode 100644 utils/tuning/libtuning/macbeth_ref.pgm create mode 100644 utils/tuning/libtuning/modules/__init__.py create mode 100644 utils/tuning/libtuning/modules/lsc/__init__.py create mode 100644 utils/tuning/libtuning/modules/lsc/lsc.py create mode 100644 utils/tuning/libtuning/modules/lsc/raspberrypi.py create mode 100644 utils/tuning/libtuning/modules/lsc/rkisp1.py create mode 100644 utils/tuning/libtuning/modules/module.py create mode 100644 utils/tuning/libtuning/parsers/__init__.py create mode 100644 utils/tuning/libtuning/parsers/parser.py create mode 100644 utils/tuning/libtuning/parsers/raspberrypi_parser.py create mode 100644 utils/tuning/libtuning/parsers/yaml_parser.py create mode 100644 utils/tuning/libtuning/smoothing.py create mode 100644 utils/tuning/libtuning/utils.py create mode 100644 utils/tuning/raspberrypi/__init__.py create mode 100644 utils/tuning/raspberrypi/alsc.py create mode 100755 utils/tuning/raspberrypi_alsc_only.py create mode 100755 utils/tuning/rkisp1.py