From patchwork Mon Sep 26 05:09:31 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Paul Elder X-Patchwork-Id: 17394 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 325BFC0DA4 for ; Mon, 26 Sep 2022 05:09:51 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 7F8A262247; Mon, 26 Sep 2022 07:09:50 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1664168990; bh=z5wPVcp4Id5mk2x5bY5AU8zC3pjYCB0eR0EjwEEbse8=; h=To:Date:Subject:List-Id:List-Unsubscribe:List-Archive:List-Post: List-Help:List-Subscribe:From:Reply-To:From; b=iEEYrw8ISVr3VGkzJEin4UMk0iyR099xbofYzXNilEsjQx9aF2uH4cKiD/ERduZU/ ed5Puvh191NzppJqGbjZhJSyQl3FOWw6cMiLL///suhM+MfbkZv2jHK+gmwoRkze4X Do1+U1Soi2zxTwOWMVAYuTa7bHRJLd0JFiMpPIkuV16jJIgaw1HPJR8euYxJWcO4+6 LmaaVwveomQQ1B6IDjREee88eOGCZs+W9pppCwKDxf1f84MJpvESoGeen4333klm6E YrndGJdzFHxX4ilp9qUNj1GJD4+HSYjEY4aIfRQxfMIhmVSRuDGIRHOFru04eS1Mlo WX9PjiMNf+OHw== Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id A027A603DC for ; Mon, 26 Sep 2022 07:09:49 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="BlEHGO1g"; dkim-atps=neutral Received: from pyrite.rasen.tech (h175-177-042-159.catv02.itscom.jp [175.177.42.159]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 068344F7; Mon, 26 Sep 2022 07:09:46 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1664168989; bh=z5wPVcp4Id5mk2x5bY5AU8zC3pjYCB0eR0EjwEEbse8=; h=From:To:Cc:Subject:Date:From; b=BlEHGO1guQcLY3cPyKvmkL1fmXDxzcw1Fw4i3Fcxj/pQvtUQduhWflU7fIqKji+3p G9lsv2rjPOw9CDa2sj966mX78gMGtuLjEMD+AsHFPEcX62ArAfGNIkIFqY5To3ZRHi ASUpMizy9e/MgSBAeuXDeylMtVw4ds36RLLNhrPs= To: libcamera-devel@lists.libcamera.org Date: Mon, 26 Sep 2022 14:09:31 +0900 Message-Id: <20220926050931.686428-1-paul.elder@ideasonboard.com> X-Mailer: git-send-email 2.30.2 MIME-Version: 1.0 Subject: [libcamera-devel] [RFC PATCH] [RFC] [DNI] utils: tuning: Draft a new tuning infrastructure X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Paul Elder via libcamera-devel From: Paul Elder Reply-To: Paul Elder Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" This patch introduces the interface of a new tuning infrastructure, with samples of how I imagine different platforms would use it (aptly named "libtuning" for now) to create their own tuning scripts. The samples include (in reverse order of how they appear in the patch, but in semantic order of how I want to show them): - rkisp1, which for now only has ALSC, and comes with most of the descriptions in the form of comments - raspberrypi's alsc_only, which is similar to rkisp1's, to show how different platforms could implement similar-but-slightly-different tuning scripts without much duplication - raspberrypi's main one, which only has ALSC and a dummy AWB one for now The two raspberrypi tuning scripts have a shared ALSC component, which is in a separate file to show where platforms could place custom components if libtuning doesn't support exactly what the platform's tuning script wants. Obviously none of this runs (hence the DNI) and none of it is implemented yet, but I wanted to show how it would be used first and to gather comments on it. There are some questions nested in the descriptions comments as well. Signed-off-by: Paul Elder --- utils/tuning/libtuning/__init__.py | 0 utils/tuning/libtuning/modules/__init__.py | 0 utils/tuning/raspberrypi.py | 37 ++++++++++++ utils/tuning/raspberrypi/__init__.py | 0 utils/tuning/raspberrypi/alsc.py | 16 ++++++ utils/tuning/raspberrypi_alsc_only.py | 16 ++++++ utils/tuning/rkisp1.py | 67 ++++++++++++++++++++++ 7 files changed, 136 insertions(+) create mode 100644 utils/tuning/libtuning/__init__.py create mode 100644 utils/tuning/libtuning/modules/__init__.py create mode 100644 utils/tuning/raspberrypi.py create mode 100644 utils/tuning/raspberrypi/__init__.py create mode 100644 utils/tuning/raspberrypi/alsc.py create mode 100644 utils/tuning/raspberrypi_alsc_only.py create mode 100644 utils/tuning/rkisp1.py diff --git a/utils/tuning/libtuning/__init__.py b/utils/tuning/libtuning/__init__.py new file mode 100644 index 00000000..e69de29b diff --git a/utils/tuning/libtuning/modules/__init__.py b/utils/tuning/libtuning/modules/__init__.py new file mode 100644 index 00000000..e69de29b diff --git a/utils/tuning/raspberrypi.py b/utils/tuning/raspberrypi.py new file mode 100644 index 00000000..322b9c36 --- /dev/null +++ b/utils/tuning/raspberrypi.py @@ -0,0 +1,37 @@ +import sys + +import libtuning as lt +from libtuning.modules import AWB +from libtuning.parsers import RaspberryPiParser +from libtuning.generators import RaspberryPiOutput + +from raspberrypi.alsc import ALSC as RaspberryPiALSC + +tuner = lt.Camera() + +# These modules can also be custom modules. libtuning will come with utilities +# for handling stuff like images, so there shouldn't be too much boilerplate +# involved in creating custom modules, though I don't yet have a concrete +# vision on how custom implementations of modules would look. +tuner.add(RaspberryPiALSC) + +# Other tuning modules can be added like so. +# The order that the tuning modules will be executed is determined by the order +# that they're added. +# This is kind of an implementation detail, but the "context" is saved +# internally in lt.Camera, so modules that are added (and therefore executed) +# later can use the output of the previous modules. I'm thinking that a module +# that depends on values from another module has two modes of execution, for +# when those values are available and another for when they're not. Not quite +# sure concretely how to handle this yet. +tuner.add(AWB( # module parameters + )) + +tuner.setInputType(RaspberryPiParser) +tuner.setOutputType(RaspberryPiOutput) + +# The order of the output doesn't necessarily have to be the same as the order +# of input, which is specified by the order of adding the modules above. +tuner.setOutputOrder(AWB, ALSC) + +tuner.run(sys.argv) diff --git a/utils/tuning/raspberrypi/__init__.py b/utils/tuning/raspberrypi/__init__.py new file mode 100644 index 00000000..e69de29b diff --git a/utils/tuning/raspberrypi/alsc.py b/utils/tuning/raspberrypi/alsc.py new file mode 100644 index 00000000..ff8a02fd --- /dev/null +++ b/utils/tuning/raspberrypi/alsc.py @@ -0,0 +1,16 @@ +ALSC(do_color = lt.paramsIfUnset(True), \ + debug = [lt.debug.Plot], \ + luminance_strength = lt.params.IfUnSet(0.5) \ + sector_shape = (16, 12), \ + sector_x_gradient = lt.gradients.Linear, \ + sector_y_gradient = lt.gradients.Linear, \ + sector_x_remainder = lt.remainder.Append, \ + sector_y_remainder = lt.remainder.Append, \ + sector_average_function = lt.average_functions.Mean, \ + smoothing_function = lt.smoothing.MedianBlur, \ + smoothing_parameters = [3], \ + output_type = lt.type.Float, \ + output_color_channels = [lt.color.R, lt.color.G, lt.color.B], \ + output_luminance_channels = [lt.color.G] \ + output_range = [0, 3.999] \ + ) diff --git a/utils/tuning/raspberrypi_alsc_only.py b/utils/tuning/raspberrypi_alsc_only.py new file mode 100644 index 00000000..11a1fc23 --- /dev/null +++ b/utils/tuning/raspberrypi_alsc_only.py @@ -0,0 +1,16 @@ +import sys + +import libtuning as lt +from libtuning.modules import ALSC +from libtuning.parsers import RaspberryPiParser +from libtuning.generators import RaspberryPiOutput + +from raspberrypi.alsc import ALSC as RaspberryPiALSC + +tuner = lt.Camera() +tuner.add(RaspberryPiALSC) +tuner.setInputType(RaspberryPiParser) +tuner.setOutputType(RaspberryPiOutput) +tuner.setOutputOrder(ALSC) + +tuner.run(sys.argv) diff --git a/utils/tuning/rkisp1.py b/utils/tuning/rkisp1.py new file mode 100644 index 00000000..8e26beba --- /dev/null +++ b/utils/tuning/rkisp1.py @@ -0,0 +1,67 @@ +import sys + +import libtuning as lt +from libtuning.modules import ALSC +from libtuning.parsers import YamlParser +from libtuning.generators import YamlOutput + +tuner = lt.Camera() +tuner.add(ALSC(do_color = lt.paramsIfUnset(True), \ + + # This can support other debug options (I can't think of any rn + # but for future-proofing) + debug = [lt.debug.Plot], \ + + # The name of IfUnSet can obviously be changed, but really I + # just want something that says "it must be specified in the + # configuration, and get the value from there" or "if the value + # is not in the configuration, use this" + luminance_strength = lt.params.IfUnSet(0.5) \ + + sector_shape = (16, 16), \ + + # Other functions might include Circular, Hyperbolic, Gaussian, + # Linear, Exponential, Logarithmic, etc + # Of course, both need not be the same function + # Some functions would need a sector_x_parameter (eg. sigma for Gaussian) + # Alternatively: sector_x_sizes = [] ? I don't think it would work tho + sector_x_gradient = lt.gradients.Parabolic, \ + sector_y_gradient = lt.gradients.Parabolic, \ + + # This is the function that will be used to average the pixels in each sector + # This can also be a custom function. + sector_average_function = lt.average_functions.Mean, \ + + # This is the function that will be used to smooth the color ratio values + # This can also be a custom function. + smoothing_function = lt.smoothing.MedianBlur, \ + smoothing_parameters = [3], \ + + # Are there any platforms that use integer values for their lsc table? + output_type = lt.type.Float, \ + + # Required if and only if do_color can be or is True + output_color_channels = [lt.color.R, lt.color.GR, lt.color.GB, lt.color.B], \ + + # Required if and only if do_color can be or is False + output_luminance_channels = [lt.color.GR, lt.color.GB], \ + + # Automatically get the precision from this + output_range = [0, 3.999] \ + + # Do we need a flag to enable/disable calculating sigmas? afaik + # the rkisp1 IPA doesn't use it? But we could output it anyway + # and algorithms can decide whether or not if they want to use + # it. Oh well, no flag for now, we can always add it later if + # there's a big demand, plus it's only two to three values and + # not an entire table. + )) +tuner.setInputType(YamlParser) +tuner.setOutputType(YamlOutput) +tuner.setOutputOrder(ALSC) + +# Maybe this should be wrapped in an if __main__ = '__main__' ? That way the +# developer can control which tuner they want to be executed based on another +# layer of arguments? But I was thinking that this would handle *all* arguments +# based on the modules' and the input/output configurations. +tuner.run(sys.argv)