From patchwork Tue Aug 13 08:44:18 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefan Klug X-Patchwork-Id: 20894 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 37AFFC32A9 for ; Tue, 13 Aug 2024 08:45:08 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id CA016633B5; Tue, 13 Aug 2024 10:45:07 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="EIhAtKy0"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id A798B6337E for ; Tue, 13 Aug 2024 10:45:06 +0200 (CEST) Received: from ideasonboard.com (unknown [IPv6:2a00:6020:448c:6c00:7025:8d00:1ffd:751a]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id C1EF8827; Tue, 13 Aug 2024 10:44:09 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1723538649; bh=g0EGizF8N6P27EL622ay/ZuWAfTMJgo8pFvqFqDWh+s=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=EIhAtKy0McuoXUjTtSIgRi8Vqmm/Ykl+ta3y39NpLrJTAWamtQ36jw6R3xS0bxV7j RNL4QSG7O9mdMBNqItf/4AUeTK1Sd2bv2kTKv2bRagrLK95hVCXm5UihUlGs1u6XCT rkGA5+bqpnFg640HoGL4GMmihYgfme5/cLUTlOAU= From: Stefan Klug To: libcamera-devel@lists.libcamera.org Cc: Stefan Klug Subject: [PATCH v4 1/6] libcamera: controls: Update the ColourTemperature control to be writable Date: Tue, 13 Aug 2024 10:44:18 +0200 Message-ID: <20240813084451.44099-2-stefan.klug@ideasonboard.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240813084451.44099-1-stefan.klug@ideasonboard.com> References: <20240813084451.44099-1-stefan.klug@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" For manual control it is helpful to be able to specify a fixed colour temperature. It also provides an easy way to apply the temperature specific CCMs and colour gains that are contained in the tuning files. Document this and update the control dependencies. Signed-off-by: Stefan Klug Reviewed-by: Paul Elder --- src/libcamera/control_ids_core.yaml | 24 ++++++++++++++++++++---- 1 file changed, 20 insertions(+), 4 deletions(-) diff --git a/src/libcamera/control_ids_core.yaml b/src/libcamera/control_ids_core.yaml index cf40771d3896..04dcc4af67fc 100644 --- a/src/libcamera/control_ids_core.yaml +++ b/src/libcamera/control_ids_core.yaml @@ -252,9 +252,12 @@ controls: - AwbEnable: type: bool description: | - Enable or disable the AWB. + Enable or disable the AWB. Disabling AWB stops updates to the + ColourGains and to the ColourCorrectionMatrix. + \sa ColourCorrectionMatrix \sa ColourGains + \sa ColourTemperature # AwbMode needs further attention: # - Auto-generate max enum value. @@ -309,13 +312,24 @@ controls: disabled. \sa AwbEnable + \sa ColourTemperature size: [2] - ColourTemperature: type: int32_t - description: Report the current estimate of the colour temperature, in - kelvin, for this frame. The ColourTemperature control can only be - returned in metadata. + description: | + Report the current estimate of the colour temperature, in kelvin, for + this frame. An implementation may also allow this control to be set when + AWB is disabled. In that case ColourGains and the ColourCorrectionMatrix + get set accordingly. If either ColourGains or ColourCorrectionMatrix are + specified at the same time, they take precedence. + + The metadata will only report measured colour temperature values when + available, even if set manually. + + \sa AwbEnable + \sa ColourCorrectionMatrix + \sa ColourGains - Saturation: type: float @@ -365,6 +379,8 @@ controls: transformation. The 3x3 matrix is stored in conventional reading order in an array of 9 floating point values. + \sa AwbEnable + \sa ColourTemperature size: [3,3] - ScalerCrop: From patchwork Tue Aug 13 08:44:19 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefan Klug X-Patchwork-Id: 20895 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id CA842C323E for ; Tue, 13 Aug 2024 08:45:11 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id E7A13633BF; Tue, 13 Aug 2024 10:45:10 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="nBBjVHk8"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 6705E6337E for ; Tue, 13 Aug 2024 10:45:09 +0200 (CEST) Received: from ideasonboard.com (unknown [IPv6:2a00:6020:448c:6c00:7025:8d00:1ffd:751a]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 7C6D2A1A; Tue, 13 Aug 2024 10:44:12 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1723538652; bh=nezsLkdaQM9qFBjjmieiWNaHqQnHtd/J8wCcftKrbIo=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=nBBjVHk8M3n05fTeXNoO9pP/bhtSgKXy1dzD29H7eoQyTxpzDrhEdBjn2f7MpAftd 3cQSAxD3U7hPJ5iCzAAunwEdtz3U3K6tHY80MEzghJrEnMk3H5F7Oid5Wv1DQKZNCd MaZmkX8FDzCZLvKrtyfKiEM4f7hvu6FCsLJPhec4= From: Stefan Klug To: libcamera-devel@lists.libcamera.org Cc: Stefan Klug , Kieran Bingham Subject: [PATCH v4 2/6] ipa: rkisp1: awb: Load white balance gains from tuning file Date: Tue, 13 Aug 2024 10:44:19 +0200 Message-ID: <20240813084451.44099-3-stefan.klug@ideasonboard.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240813084451.44099-1-stefan.klug@ideasonboard.com> References: <20240813084451.44099-1-stefan.klug@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" For the implementation of a manual colour temperature setting, it is necessary to read predefined colour gains per colour temperature from the tuning file. Implement this in a backwards compatible way. If no gains are contained in the tuning file, loading just continues as before. Signed-off-by: Stefan Klug Reviewed-by: Kieran Bingham Reviewed-by: Paul Elder Reviewed-by: Laurent Pinchart --- src/ipa/rkisp1/algorithms/awb.cpp | 18 ++++++++++++++++++ src/ipa/rkisp1/algorithms/awb.h | 6 ++++++ 2 files changed, 24 insertions(+) diff --git a/src/ipa/rkisp1/algorithms/awb.cpp b/src/ipa/rkisp1/algorithms/awb.cpp index 4ccafd48dedd..c23f749c192b 100644 --- a/src/ipa/rkisp1/algorithms/awb.cpp +++ b/src/ipa/rkisp1/algorithms/awb.cpp @@ -39,6 +39,24 @@ Awb::Awb() { } +/** + * \copydoc libcamera::ipa::Algorithm::init + */ +int Awb::init([[maybe_unused]] IPAContext &context, const YamlObject &tuningData) +{ + MatrixInterpolator gains; + int ret = gains.readYaml(tuningData["gains"], "ct", "gains"); + if (ret < 0) + LOG(RkISP1Awb, Warning) + << "Failed to parse 'gains' " + << "parameter from tuning file; " + << "rgb gains will not be set based on colour temperature"; + else + gains_ = gains; + + return 0; +} + /** * \copydoc libcamera::ipa::Algorithm::configure */ diff --git a/src/ipa/rkisp1/algorithms/awb.h b/src/ipa/rkisp1/algorithms/awb.h index 06c92896e2dc..a010e6d1cb3c 100644 --- a/src/ipa/rkisp1/algorithms/awb.h +++ b/src/ipa/rkisp1/algorithms/awb.h @@ -7,6 +7,10 @@ #pragma once +#include + +#include "libipa/matrix_interpolator.h" + #include "algorithm.h" namespace libcamera { @@ -19,6 +23,7 @@ public: Awb(); ~Awb() = default; + int init(IPAContext &context, const YamlObject &tuningData) override; int configure(IPAContext &context, const IPACameraSensorInfo &configInfo) override; void queueRequest(IPAContext &context, const uint32_t frame, IPAFrameContext &frameContext, @@ -34,6 +39,7 @@ public: private: uint32_t estimateCCT(double red, double green, double blue); + std::optional> gains_; bool rgbMode_; }; From patchwork Tue Aug 13 08:44:20 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefan Klug X-Patchwork-Id: 20896 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 3F06FC32A9 for ; Tue, 13 Aug 2024 08:45:14 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id E4921633C0; Tue, 13 Aug 2024 10:45:13 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="WHSO9KC9"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id AFA57633B5 for ; Tue, 13 Aug 2024 10:45:11 +0200 (CEST) Received: from ideasonboard.com (unknown [IPv6:2a00:6020:448c:6c00:7025:8d00:1ffd:751a]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 4C8A2827; Tue, 13 Aug 2024 10:44:14 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1723538654; bh=/KOBVKAao58HbFs5enY7FFTibZIebIgtD3qfN8Dq52s=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=WHSO9KC9z7Th48Z/Of+bL3/AbCk4kVIDyT8mOZhfPNQjaGeUxf1RksHwgQwv/BIDx hwpfhtImCDCOyR+M/3TfpUsCzyDxU8E5TViGIalYlQSOVfTsdDkAAljLUD7fUqJV71 1PbCE+PETr9sQ7LtwbE47p+cRpn7ktwk/WTCFeBk= From: Stefan Klug To: libcamera-devel@lists.libcamera.org Cc: Stefan Klug Subject: [PATCH v4 3/6] ipa: rkisp1: awb: Implement ColourTemperature control Date: Tue, 13 Aug 2024 10:44:20 +0200 Message-ID: <20240813084451.44099-4-stefan.klug@ideasonboard.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240813084451.44099-1-stefan.klug@ideasonboard.com> References: <20240813084451.44099-1-stefan.klug@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" There are many use-cases (tuning-validation, working in static environments) where a manual ColourTemperature control is helpful. Implement that by interpolating and applying the white balance gains from the tuning file according to the requested colour temperature. If colour gains are provided on the same request, they take precedence. As the colour temperature reported in the metadata is always based on the measurements, we don't have to touch that. Note that in the automatic case, the colour gains are still based on the gray world model and the ones from the tuning file get ignored. Signed-off-by: Stefan Klug Reviewed-by: Paul Elder --- src/ipa/rkisp1/algorithms/awb.cpp | 22 +++++++++++++++++++++- 1 file changed, 21 insertions(+), 1 deletion(-) diff --git a/src/ipa/rkisp1/algorithms/awb.cpp b/src/ipa/rkisp1/algorithms/awb.cpp index c23f749c192b..d482eda5b541 100644 --- a/src/ipa/rkisp1/algorithms/awb.cpp +++ b/src/ipa/rkisp1/algorithms/awb.cpp @@ -31,6 +31,10 @@ namespace ipa::rkisp1::algorithms { LOG_DEFINE_CATEGORY(RkISP1Awb) +constexpr int32_t kMinColourTemperature = 2500; +constexpr int32_t kMaxColourTemperature = 10000; +constexpr int32_t kDefaultColourTemperature = 6500; + /* Minimum mean value below which AWB can't operate. */ constexpr double kMeanMinThreshold = 2.0; @@ -42,8 +46,13 @@ Awb::Awb() /** * \copydoc libcamera::ipa::Algorithm::init */ -int Awb::init([[maybe_unused]] IPAContext &context, const YamlObject &tuningData) +int Awb::init(IPAContext &context, const YamlObject &tuningData) { + auto &cmap = context.ctrlMap; + cmap[&controls::ColourTemperature] = ControlInfo(kMinColourTemperature, + kMaxColourTemperature, + kDefaultColourTemperature); + MatrixInterpolator gains; int ret = gains.readYaml(tuningData["gains"], "ct", "gains"); if (ret < 0) @@ -113,6 +122,17 @@ void Awb::queueRequest(IPAContext &context, << ", blue: " << awb.gains.manual.blue; } + const auto &colourTemperature = controls.get(controls::ColourTemperature); + if (colourTemperature && !awb.autoEnabled && gains_ && !colourGains) { + Matrix gains = gains_->get(*colourTemperature); + awb.gains.manual.red = gains[0][0]; + awb.gains.manual.blue = gains[1][0]; + + LOG(RkISP1Awb, Debug) + << "Set colour gains to red: " << awb.gains.manual.red + << ", blue: " << awb.gains.manual.blue; + } + frameContext.awb.autoEnabled = awb.autoEnabled; if (!awb.autoEnabled) { From patchwork Tue Aug 13 08:44:21 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefan Klug X-Patchwork-Id: 20897 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id ED938C32D5 for ; Tue, 13 Aug 2024 08:45:16 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 7A2BD633BE; Tue, 13 Aug 2024 10:45:16 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="l8rbyQuC"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id C9FC7633BE for ; Tue, 13 Aug 2024 10:45:13 +0200 (CEST) Received: from ideasonboard.com (unknown [IPv6:2a00:6020:448c:6c00:7025:8d00:1ffd:751a]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id E02B5827; Tue, 13 Aug 2024 10:44:16 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1723538657; bh=o4QRQPyBZWqPobyF3eAI8iX37EScWXUuSO7hIcCVgU0=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=l8rbyQuCUv1PhmX90AgU2wKqQIz5wRufAAnoG+VxMo82K6ygifzWJs6C06fu/zZDp Spd8MGL6ukaWYGjdxw8GCbcuiohX26FRKyKjVJLVhLfh9pAL+Dh+FQtHRL+4op3Y5n BaOecbzqr7xMN4LBmqHMc2mz1E55raY6uBNKUaB4= From: Stefan Klug To: libcamera-devel@lists.libcamera.org Cc: Stefan Klug , Kieran Bingham Subject: [PATCH v4 4/6] libtuning: Modify ctt_awb.awb() so that it can run in our context Date: Tue, 13 Aug 2024 10:44:21 +0200 Message-ID: <20240813084451.44099-5-stefan.klug@ideasonboard.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240813084451.44099-1-stefan.klug@ideasonboard.com> References: <20240813084451.44099-1-stefan.klug@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Modify the awb function that was copied from the Raspberry Pi tuning scripts in a way that it can easily be called from the libtuning code. In essence the logging was replaced by calls to a python logger and the need for the Cam object was removed by providing a list of images to the function. Signed-off-by: Stefan Klug Reviewed-by: Kieran Bingham Reviewed-by: Paul Elder --- utils/tuning/libtuning/ctt_awb.py | 55 ++++++++++++++++--------------- 1 file changed, 28 insertions(+), 27 deletions(-) diff --git a/utils/tuning/libtuning/ctt_awb.py b/utils/tuning/libtuning/ctt_awb.py index abf22321a0ea..73a1bc1a840c 100644 --- a/utils/tuning/libtuning/ctt_awb.py +++ b/utils/tuning/libtuning/ctt_awb.py @@ -4,6 +4,8 @@ # # camera tuning tool for AWB +import logging + import matplotlib.pyplot as plt from bisect import bisect_left from scipy.optimize import fmin @@ -11,12 +13,12 @@ import numpy as np from .image import Image +logger = logging.getLogger(__name__) """ obtain piecewise linear approximation for colour curve """ -def awb(Cam, cal_cr_list, cal_cb_list, plot): - imgs = Cam.imgs +def awb(imgs, cal_cr_list, cal_cb_list, plot): """ condense alsc calibration tables into one dictionary """ @@ -39,7 +41,7 @@ def awb(Cam, cal_cr_list, cal_cb_list, plot): rb_raw = [] rbs_hat = [] for Img in imgs: - Cam.log += '\nProcessing '+Img.name + logger.info(f'Processing {Img.name}') """ get greyscale patches with alsc applied if alsc enabled. Note: if alsc is disabled then colour_cals will be set to None and the @@ -51,7 +53,7 @@ def awb(Cam, cal_cr_list, cal_cb_list, plot): """ r_g = np.mean(r_patchs/g_patchs) b_g = np.mean(b_patchs/g_patchs) - Cam.log += '\n r : {:.4f} b : {:.4f}'.format(r_g, b_g) + logger.info(' r : {:.4f} b : {:.4f}'.format(r_g, b_g)) """ The curve tends to be better behaved in so-called hatspace. R, B, G represent the individual channels. The colour curve is plotted in @@ -74,12 +76,11 @@ def awb(Cam, cal_cr_list, cal_cb_list, plot): """ r_g_hat = r_g/(1+r_g+b_g) b_g_hat = b_g/(1+r_g+b_g) - Cam.log += '\n r_hat : {:.4f} b_hat : {:.4f}'.format(r_g_hat, b_g_hat) - rbs_hat.append((r_g_hat, b_g_hat, Img.col)) + logger.info('\n r_hat : {:.4f} b_hat : {:.4f}'.format(r_g_hat, b_g_hat)) + rbs_hat.append((r_g_hat, b_g_hat, Img.color)) rb_raw.append((r_g, b_g)) - Cam.log += '\n' - Cam.log += '\nFinished processing images' + logger.info('Finished processing images') """ sort all lits simultaneously by r_hat """ @@ -95,7 +96,7 @@ def awb(Cam, cal_cr_list, cal_cb_list, plot): fit quadratic fit to r_g hat and b_g_hat """ a, b, c = np.polyfit(rbs_hat[0], rbs_hat[1], 2) - Cam.log += '\nFit quadratic curve in hatspace' + logger.info('Fit quadratic curve in hatspace') """ the algorithm now approximates the shortest distance from each point to the curve in dehatspace. Since the fit is done in hatspace, it is easier to @@ -151,14 +152,14 @@ def awb(Cam, cal_cr_list, cal_cb_list, plot): if (x+y) > (rr+bb): dist *= -1 dists.append(dist) - Cam.log += '\nFound closest point on fit line to each point in dehatspace' + logger.info('Found closest point on fit line to each point in dehatspace') """ calculate wiggle factors in awb. 10% added since this is an upper bound """ transverse_neg = - np.min(dists) * 1.1 transverse_pos = np.max(dists) * 1.1 - Cam.log += '\nTransverse pos : {:.5f}'.format(transverse_pos) - Cam.log += '\nTransverse neg : {:.5f}'.format(transverse_neg) + logger.info('Transverse pos : {:.5f}'.format(transverse_pos)) + logger.info('Transverse neg : {:.5f}'.format(transverse_neg)) """ set minimum transverse wiggles to 0.1 . Wiggle factors dictate how far off of the curve the algorithm searches. 0.1 @@ -167,10 +168,10 @@ def awb(Cam, cal_cr_list, cal_cb_list, plot): """ if transverse_pos < 0.01: transverse_pos = 0.01 - Cam.log += '\nForced transverse pos to 0.01' + logger.info('Forced transverse pos to 0.01') if transverse_neg < 0.01: transverse_neg = 0.01 - Cam.log += '\nForced transverse neg to 0.01' + logger.info('Forced transverse neg to 0.01') """ generate new b_hat values at each r_hat according to fit @@ -202,25 +203,25 @@ def awb(Cam, cal_cr_list, cal_cb_list, plot): i = len(c_fit) - 1 while i > 0: if c_fit[i] > c_fit[i-1]: - Cam.log += '\nColour temperature increase found\n' - Cam.log += '{} K at r = {} to '.format(c_fit[i-1], r_fit[i-1]) - Cam.log += '{} K at r = {}'.format(c_fit[i], r_fit[i]) + logger.info('Colour temperature increase found') + logger.info('{} K at r = {} to '.format(c_fit[i-1], r_fit[i-1])) + logger.info('{} K at r = {}'.format(c_fit[i], r_fit[i])) """ if colour temperature increases then discard point furthest from the transformed fit (dehatspace) """ error_1 = abs(dists[i-1]) error_2 = abs(dists[i]) - Cam.log += '\nDistances from fit:\n' - Cam.log += '{} K : {:.5f} , '.format(c_fit[i], error_1) - Cam.log += '{} K : {:.5f}'.format(c_fit[i-1], error_2) + logger.info('Distances from fit:') + logger.info('{} K : {:.5f} , '.format(c_fit[i], error_1)) + logger.info('{} K : {:.5f}'.format(c_fit[i-1], error_2)) """ find bad index note that in python false = 0 and true = 1 """ bad = i - (error_1 < error_2) - Cam.log += '\nPoint at {} K deleted as '.format(c_fit[bad]) - Cam.log += 'it is furthest from fit' + logger.info('Point at {} K deleted as '.format(c_fit[bad])) + logger.info('it is furthest from fit') """ delete bad point """ @@ -239,12 +240,12 @@ def awb(Cam, cal_cr_list, cal_cb_list, plot): return formatted ct curve, ordered by increasing colour temperature """ ct_curve = list(np.array(list(zip(b_fit, r_fit, c_fit))).flatten())[::-1] - Cam.log += '\nFinal CT curve:' + logger.info('Final CT curve:') for i in range(len(ct_curve)//3): j = 3*i - Cam.log += '\n ct: {} '.format(ct_curve[j]) - Cam.log += ' r: {} '.format(ct_curve[j+1]) - Cam.log += ' b: {} '.format(ct_curve[j+2]) + logger.info(' ct: {} '.format(ct_curve[j])) + logger.info(' r: {} '.format(ct_curve[j+1])) + logger.info(' b: {} '.format(ct_curve[j+2])) """ plotting code for debug @@ -303,7 +304,7 @@ def get_alsc_patches(Img, colour_cals, grey=True): """ if grey: cen_coords = Img.cen_coords[3::4] - col = Img.col + col = Img.color patches = [np.array(Img.patches[i]) for i in Img.order] r_patchs = patches[0][3::4] - Img.blacklevel_16 b_patchs = patches[3][3::4] - Img.blacklevel_16 From patchwork Tue Aug 13 08:44:22 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefan Klug X-Patchwork-Id: 20898 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 7DE8EC323E for ; Tue, 13 Aug 2024 08:45:19 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 2B068633C5; Tue, 13 Aug 2024 10:45:19 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="gA9IvLHC"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id BFBFC633C3 for ; Tue, 13 Aug 2024 10:45:16 +0200 (CEST) Received: from ideasonboard.com (unknown [IPv6:2a00:6020:448c:6c00:7025:8d00:1ffd:751a]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id DF9FD827; Tue, 13 Aug 2024 10:44:19 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1723538660; bh=V9Oha5C3YDbyGAPUEltpK+alEolxP0AseaHmNlRfq6s=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=gA9IvLHCfHctNFtjawOTMalMKIUh/3nH1NbC6Bah8+MBRzg1v6o4raw5A54X+wYZK l/nc6g3gfOH3+oCWzyZEzf7BMYEnJFtm/V33Y6DsbH6Y0Soa1C05qx1LsfI/az27Pr JJdoqwBRNcGC4LcaxwVeLVhsdXtRTfwsrPQ/l9B8= From: Stefan Klug To: libcamera-devel@lists.libcamera.org Cc: Stefan Klug , Kieran Bingham Subject: [PATCH v4 5/6] libtuning: Add initial AWB module Date: Tue, 13 Aug 2024 10:44:22 +0200 Message-ID: <20240813084451.44099-6-stefan.klug@ideasonboard.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240813084451.44099-1-stefan.klug@ideasonboard.com> References: <20240813084451.44099-1-stefan.klug@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" This AWB modules uses the awb function from Raspberry Pi to calculate the needed white balance gains per colour temperature. It stores these gains in the tuning file. Currently they are only used for the manual colour temperature control. It is likely that they will be used with more complex awb algorithms. Signed-off-by: Stefan Klug Reviewed-by: Kieran Bingham Reviewed-by: Paul Elder Reviewed-by: Laurent Pinchart --- .../tuning/libtuning/modules/awb/__init__.py | 6 +++ utils/tuning/libtuning/modules/awb/awb.py | 42 +++++++++++++++++++ utils/tuning/libtuning/modules/awb/rkisp1.py | 27 ++++++++++++ 3 files changed, 75 insertions(+) create mode 100644 utils/tuning/libtuning/modules/awb/__init__.py create mode 100644 utils/tuning/libtuning/modules/awb/awb.py create mode 100644 utils/tuning/libtuning/modules/awb/rkisp1.py diff --git a/utils/tuning/libtuning/modules/awb/__init__.py b/utils/tuning/libtuning/modules/awb/__init__.py new file mode 100644 index 000000000000..2d67f10cfc4f --- /dev/null +++ b/utils/tuning/libtuning/modules/awb/__init__.py @@ -0,0 +1,6 @@ +# SPDX-License-Identifier: GPL-2.0-or-later +# +# Copyright (C) 2024, Ideas On Board + +from libtuning.modules.awb.awb import AWB +from libtuning.modules.awb.rkisp1 import AWBRkISP1 diff --git a/utils/tuning/libtuning/modules/awb/awb.py b/utils/tuning/libtuning/modules/awb/awb.py new file mode 100644 index 000000000000..5680a44fd3e3 --- /dev/null +++ b/utils/tuning/libtuning/modules/awb/awb.py @@ -0,0 +1,42 @@ +# SPDX-License-Identifier: GPL-2.0-or-later +# +# Copyright (C) 2024, Ideas On Board + +import logging + +from ..module import Module + +from libtuning.ctt_awb import awb +import numpy as np + +logger = logging.getLogger(__name__) + + +class AWB(Module): + type = 'awb' + hr_name = 'AWB (Base)' + out_name = 'GenericAWB' + + def __init__(self, *, + debug: list): + super().__init__() + + self.debug = debug + + def do_calculation(self, images): + logger.info('Starting AWB calculation') + + imgs = [img for img in images if img.macbeth is not None] + + gains, _, _ = awb(imgs, None, None, False) + gains = np.array(gains) + gains = gains.reshape(-1, 3) + + res = [] + for v in gains: + res.append({ + 'ct': int(v[0]), + 'gains': [float(1.0 / v[1]), float(1.0 / v[2])] + }) + + return res diff --git a/utils/tuning/libtuning/modules/awb/rkisp1.py b/utils/tuning/libtuning/modules/awb/rkisp1.py new file mode 100644 index 000000000000..e3ddeb50c9a1 --- /dev/null +++ b/utils/tuning/libtuning/modules/awb/rkisp1.py @@ -0,0 +1,27 @@ +# SPDX-License-Identifier: GPL-2.0-or-later +# +# Copyright (C) 2024, Ideas On Board +# +# AWB module for tuning rkisp1 + +from .awb import AWB + +import libtuning as lt + + +class AWBRkISP1(AWB): + hr_name = 'AWB (RkISP1)' + out_name = 'Awb' + + def __init__(self, **kwargs): + super().__init__(**kwargs) + + def validate_config(self, config: dict) -> bool: + return True + + def process(self, config: dict, images: list, outputs: dict) -> dict: + output = {} + + output['gains'] = self.do_calculation(images) + + return output From patchwork Tue Aug 13 08:44:23 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefan Klug X-Patchwork-Id: 20899 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id B497DC32A9 for ; Tue, 13 Aug 2024 08:45:21 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 47D29633C5; Tue, 13 Aug 2024 10:45:21 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="WdRj+tlR"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id D501B633BD for ; Tue, 13 Aug 2024 10:45:18 +0200 (CEST) Received: from ideasonboard.com (unknown [IPv6:2a00:6020:448c:6c00:7025:8d00:1ffd:751a]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id F3B59827; Tue, 13 Aug 2024 10:44:21 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1723538662; bh=9z8+IGgL9sP+6ZAt7OvDwSu986dzi/ux1kdCVuAYVWo=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=WdRj+tlRX8/Oj0bxdLnKUygGnviAugCFfNCF8LDeTfwvVK9w6riuqQn2HWPyl6zry 6OhjnyBpzGZCGqXWnK8WdPUTv4DjxwlfSCi5tnYKYa+lnDEJOa9fqgYtkf1ixJvUEz 4w4jIkuk5yb3ghISpTQ6LUVRWrD8PsqM4QgA4ptY= From: Stefan Klug To: libcamera-devel@lists.libcamera.org Cc: Stefan Klug , Kieran Bingham Subject: [PATCH v4 6/6] utils: tuning: rkisp1: Replace static AWB with new AWB module Date: Tue, 13 Aug 2024 10:44:23 +0200 Message-ID: <20240813084451.44099-7-stefan.klug@ideasonboard.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240813084451.44099-1-stefan.klug@ideasonboard.com> References: <20240813084451.44099-1-stefan.klug@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" The rkisp1 tuner used a static module to insert the AWB algorithm into the tuning file. Replace that with the new AWB module. Signed-off-by: Stefan Klug Reviewed-by: Kieran Bingham Reviewed-by: Paul Elder Reviewed-by: Laurent Pinchart --- utils/tuning/rkisp1.py | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/utils/tuning/rkisp1.py b/utils/tuning/rkisp1.py index f5c42a61d15e..9f40fd8bd63b 100755 --- a/utils/tuning/rkisp1.py +++ b/utils/tuning/rkisp1.py @@ -2,6 +2,7 @@ # SPDX-License-Identifier: GPL-2.0-or-later # # Copyright (C) 2022, Paul Elder +# Copyright (C) 2024, Ideas On Board # # Tuning script for rkisp1 @@ -14,13 +15,14 @@ from libtuning.parsers import YamlParser from libtuning.generators import YamlOutput from libtuning.modules.lsc import LSCRkISP1 from libtuning.modules.agc import AGCRkISP1 +from libtuning.modules.awb import AWBRkISP1 from libtuning.modules.ccm import CCMRkISP1 from libtuning.modules.static import StaticModule coloredlogs.install(level=logging.INFO, fmt='%(name)s %(levelname)s %(message)s') agc = AGCRkISP1(debug=[lt.Debug.Plot]) -awb = StaticModule('Awb') +awb = AWBRkISP1(debug=[lt.Debug.Plot]) blc = StaticModule('BlackLevelCorrection') ccm = CCMRkISP1(debug=[lt.Debug.Plot]) color_processing = StaticModule('ColorProcessing')