From patchwork Wed Jul 3 14:16:50 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefan Klug X-Patchwork-Id: 20515 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 853FBBEFBE for ; Wed, 3 Jul 2024 14:17:38 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 404DD62E27; Wed, 3 Jul 2024 16:17:38 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="S23eorpz"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 48F9B62C96 for ; Wed, 3 Jul 2024 16:17:34 +0200 (CEST) Received: from ideasonboard.com (unknown [IPv6:2a00:6020:448c:6c00:9263:c199:9587:576]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 249208A9; Wed, 3 Jul 2024 16:17:06 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1720016226; bh=i9AN0dt4XQULDpcikbR2FIlp+kwamVULwLdL7P7zjPM=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=S23eorpzOQocHMI69GXT6360/FioRrQSf0InwboiwvpXoYkCG2W+JJqgnFARqonRj Yv3rOjDJVMu621WPzSpNanH/1xQYTbMvZSJ3IBJ1TspvjzE1niAF0aH+Rvye2A16oE WFhiFMhBoylgwg1EBDP5DK8moEhtAj7fLgs/lOGs= From: Stefan Klug To: libcamera-devel@lists.libcamera.org Cc: Stefan Klug , Paul Elder , Kieran Bingham Subject: [PATCH v3 01/23] libtuning: Backport improvements in MacBeth search reliability Date: Wed, 3 Jul 2024 16:16:50 +0200 Message-ID: <20240703141726.252368-2-stefan.klug@ideasonboard.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240703141726.252368-1-stefan.klug@ideasonboard.com> References: <20240703141726.252368-1-stefan.klug@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Port 66479605baca4a22e2b7a17c2a8cf9f9be9a7724 into libtuning. Original message: Improve the Macbeth Chart search reliability Previously the code would brighten up images in case the Macbeth Chart is slightly dark, and also zoom in on sections of it to look for charts occupying less of the field of view. But it would not do both together. This change makes the search for smaller charts also repeat that search for the brightened up images that it made earlier, thereby increasing the chances of success for non-optimal tuning images. Signed-off-by: Stefan Klug Reviewed-by: Paul Elder Reviewed-by: Kieran Bingham Reviewed-by: Laurent Pinchart --- utils/tuning/libtuning/macbeth.py | 38 ++++++++++++++++++------------- 1 file changed, 22 insertions(+), 16 deletions(-) diff --git a/utils/tuning/libtuning/macbeth.py b/utils/tuning/libtuning/macbeth.py index e11824646a4f..81f3e87c9088 100644 --- a/utils/tuning/libtuning/macbeth.py +++ b/utils/tuning/libtuning/macbeth.py @@ -403,10 +403,15 @@ def find_macbeth(img, mac_config): # nothing more is tried as this is a high enough confidence to ensure # reliable macbeth square centre placement. + # Keep a list that will include this and any brightened up versions of + # the image for reuse. + all_images = [img] + for brightness in [2, 4]: if cor >= 0.75: break img_br = cv2.convertScaleAbs(img, alpha=brightness, beta=0) + all_images.append(img_br) cor_b, mac_b, coords_b, ret_b = get_macbeth_chart(img_br, ref_data) if cor_b > cor: cor, mac, coords, ret = cor_b, mac_b, coords_b, ret_b @@ -456,23 +461,24 @@ def find_macbeth(img, mac_config): w_inc = int(w * pair['inc']) h_inc = int(h * pair['inc']) - loop = ((1 - pair['sel']) / pair['inc']) + 1 + loop = int(((1 - pair['sel']) / pair['inc']) + 1) # For each subselection, look for a macbeth chart - for i in range(loop): - for j in range(loop): - w_s, h_s = i * w_inc, j * h_inc - img_sel = img[w_s:w_s + w_sel, h_s:h_s + h_sel] - cor_ij, mac_ij, coords_ij, ret_ij = get_macbeth_chart(img_sel, ref_data) - - # If the correlation is better than the best then record the - # scale and current subselection at which macbeth chart was - # found. Also record the coordinates, macbeth chart and message. - if cor_ij > cor: - cor = cor_ij - mac, coords, ret = mac_ij, coords_ij, ret_ij - ii, jj = i, j - w_best, h_best = w_inc, h_inc - d_best = index + 1 + for img_br in all_images: + for i in range(loop): + for j in range(loop): + w_s, h_s = i * w_inc, j * h_inc + img_sel = img_br[w_s:w_s + w_sel, h_s:h_s + h_sel] + cor_ij, mac_ij, coords_ij, ret_ij = get_macbeth_chart(img_sel, ref_data) + + # If the correlation is better than the best then record the + # scale and current subselection at which macbeth chart was + # found. Also record the coordinates, macbeth chart and message. + if cor_ij > cor: + cor = cor_ij + mac, coords, ret = mac_ij, coords_ij, ret_ij + ii, jj = i, j + w_best, h_best = w_inc, h_inc + d_best = index + 1 # Transform coordinates from subselection to original image if ii != -1: From patchwork Wed Jul 3 14:16:51 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefan Klug X-Patchwork-Id: 20516 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 874C2BEFBE for ; Wed, 3 Jul 2024 14:17:40 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 3C86F62E22; Wed, 3 Jul 2024 16:17:39 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="rq3DLFxR"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 3CC5B62C96 for ; Wed, 3 Jul 2024 16:17:37 +0200 (CEST) Received: from ideasonboard.com (unknown [IPv6:2a00:6020:448c:6c00:9263:c199:9587:576]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 164814CA; Wed, 3 Jul 2024 16:17:09 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1720016229; bh=ag55GuB867JmULS+QEKPGbsOqxOjnA1YznN9wONUmVk=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=rq3DLFxRBcRImbohgGAQgthbxdcyyj1LGLIwHcuak7FLj0h+XE0AU2RNdyKKwtdRc bymwTsuwEN5NqLQ8r1bxcHUnkzq/KE3Lcks7Sv8jTH+MIm3TlzqVrdg82/TYjNV3Li ZNr36p0+C+0iLjYY5mS4UJFZJHAMJ3LodRRc0Ttg= From: Stefan Klug To: libcamera-devel@lists.libcamera.org Cc: Stefan Klug , Paul ELder , Kieran Bingham Subject: [PATCH v3 02/23] libtuning: Fix reference image Date: Wed, 3 Jul 2024 16:16:51 +0200 Message-ID: <20240703141726.252368-3-stefan.klug@ideasonboard.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240703141726.252368-1-stefan.klug@ideasonboard.com> References: <20240703141726.252368-1-stefan.klug@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Opencv fails to load the image. The added license destroys the magic number. Fix, by moving the licence below the magic number. Signed-off-by: Stefan Klug Reviewed-by: Paul ELder Reviewed-by: Kieran Bingham Reviewed-by: Laurent Pinchart --- utils/tuning/libtuning/macbeth_ref.pgm | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/utils/tuning/libtuning/macbeth_ref.pgm b/utils/tuning/libtuning/macbeth_ref.pgm index 37897140ce84..089ea91f7f12 100644 --- a/utils/tuning/libtuning/macbeth_ref.pgm +++ b/utils/tuning/libtuning/macbeth_ref.pgm @@ -1,5 +1,5 @@ -# SPDX-License-Identifier: BSD-2-Clause P5 +# SPDX-License-Identifier: BSD-2-Clause # Reference macbeth chart 120 80 255 From patchwork Wed Jul 3 14:16:52 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Stefan Klug X-Patchwork-Id: 20517 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id ECCD4BEFBE for ; Wed, 3 Jul 2024 14:17:43 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 6F30263339; Wed, 3 Jul 2024 16:17:43 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="PfCMLF6z"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id A9BFE62E26 for ; Wed, 3 Jul 2024 16:17:40 +0200 (CEST) Received: from ideasonboard.com (unknown [IPv6:2a00:6020:448c:6c00:9263:c199:9587:576]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 760F04CA; Wed, 3 Jul 2024 16:17:12 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1720016232; bh=mwxRs/47IiVj7qsxbGZe700L2GMD/pAI93M/ZSxRqUg=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=PfCMLF6zhfgMzbSHbIW30VvLLsa+VwHHIpIq6i6vPiUdlzJKWBVzAJM+gnPoPTs9V uu7L5Tk3MUeF1ZUYK6YVrmIaF6QDNu69P9eIotlniMTWCNGhJ6V57o1Uih6sjbbQwr hZXC0PE3yA1uRiqrbNG9MriCY9z8254OWzFO8zjc= From: Stefan Klug To: libcamera-devel@lists.libcamera.org Cc: Stefan Klug , Kieran Bingham Subject: [PATCH v3 03/23] libtuning: Copy files from raspberrypi Date: Wed, 3 Jul 2024 16:16:52 +0200 Message-ID: <20240703141726.252368-4-stefan.klug@ideasonboard.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240703141726.252368-1-stefan.klug@ideasonboard.com> References: <20240703141726.252368-1-stefan.klug@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Copy ctt_{awb,ccm,colors,ransac} from the raspberrypi tuning scripts as basis for the libcamera implementation. color.py was renamed to ctt_colors.py to better express the origin. The files were taken from commit 66479605baca4a22e2b Signed-off-by: Stefan Klug Acked-by: Kieran Bingham Acked-by: Paul Elder Acked-by: Laurent Pinchart --- utils/tuning/libtuning/ctt_awb.py | 376 +++++++++++++++++++++++++ utils/tuning/libtuning/ctt_ccm.py | 406 +++++++++++++++++++++++++++ utils/tuning/libtuning/ctt_colors.py | 30 ++ utils/tuning/libtuning/ctt_ransac.py | 71 +++++ 4 files changed, 883 insertions(+) create mode 100644 utils/tuning/libtuning/ctt_awb.py create mode 100644 utils/tuning/libtuning/ctt_ccm.py create mode 100644 utils/tuning/libtuning/ctt_colors.py create mode 100644 utils/tuning/libtuning/ctt_ransac.py diff --git a/utils/tuning/libtuning/ctt_awb.py b/utils/tuning/libtuning/ctt_awb.py new file mode 100644 index 000000000000..5ba6f978a228 --- /dev/null +++ b/utils/tuning/libtuning/ctt_awb.py @@ -0,0 +1,376 @@ +# SPDX-License-Identifier: BSD-2-Clause +# +# Copyright (C) 2019, Raspberry Pi Ltd +# +# camera tuning tool for AWB + +from ctt_image_load import * +import matplotlib.pyplot as plt +from bisect import bisect_left +from scipy.optimize import fmin + + +""" +obtain piecewise linear approximation for colour curve +""" +def awb(Cam, cal_cr_list, cal_cb_list, plot): + imgs = Cam.imgs + """ + condense alsc calibration tables into one dictionary + """ + if cal_cr_list is None: + colour_cals = None + else: + colour_cals = {} + for cr, cb in zip(cal_cr_list, cal_cb_list): + cr_tab = cr['table'] + cb_tab = cb['table'] + """ + normalise tables so min value is 1 + """ + cr_tab = cr_tab/np.min(cr_tab) + cb_tab = cb_tab/np.min(cb_tab) + colour_cals[cr['ct']] = [cr_tab, cb_tab] + """ + obtain data from greyscale macbeth patches + """ + rb_raw = [] + rbs_hat = [] + for Img in imgs: + Cam.log += '\nProcessing '+Img.name + """ + get greyscale patches with alsc applied if alsc enabled. + Note: if alsc is disabled then colour_cals will be set to None and the + function will just return the greyscale patches + """ + r_patchs, b_patchs, g_patchs = get_alsc_patches(Img, colour_cals) + """ + calculate ratio of r, b to g + """ + r_g = np.mean(r_patchs/g_patchs) + b_g = np.mean(b_patchs/g_patchs) + Cam.log += '\n r : {:.4f} b : {:.4f}'.format(r_g, b_g) + """ + The curve tends to be better behaved in so-called hatspace. + R, B, G represent the individual channels. The colour curve is plotted in + r, b space, where: + r = R/G + b = B/G + This will be referred to as dehatspace... (sorry) + Hatspace is defined as: + r_hat = R/(R+B+G) + b_hat = B/(R+B+G) + To convert from dehatspace to hastpace (hat operation): + r_hat = r/(1+r+b) + b_hat = b/(1+r+b) + To convert from hatspace to dehatspace (dehat operation): + r = r_hat/(1-r_hat-b_hat) + b = b_hat/(1-r_hat-b_hat) + Proof is left as an excercise to the reader... + Throughout the code, r and b are sometimes referred to as r_g and b_g + as a reminder that they are ratios + """ + r_g_hat = r_g/(1+r_g+b_g) + b_g_hat = b_g/(1+r_g+b_g) + Cam.log += '\n r_hat : {:.4f} b_hat : {:.4f}'.format(r_g_hat, b_g_hat) + rbs_hat.append((r_g_hat, b_g_hat, Img.col)) + rb_raw.append((r_g, b_g)) + Cam.log += '\n' + + Cam.log += '\nFinished processing images' + """ + sort all lits simultaneously by r_hat + """ + rbs_zip = list(zip(rbs_hat, rb_raw)) + rbs_zip.sort(key=lambda x: x[0][0]) + rbs_hat, rb_raw = list(zip(*rbs_zip)) + """ + unzip tuples ready for processing + """ + rbs_hat = list(zip(*rbs_hat)) + rb_raw = list(zip(*rb_raw)) + """ + fit quadratic fit to r_g hat and b_g_hat + """ + a, b, c = np.polyfit(rbs_hat[0], rbs_hat[1], 2) + Cam.log += '\nFit quadratic curve in hatspace' + """ + the algorithm now approximates the shortest distance from each point to the + curve in dehatspace. Since the fit is done in hatspace, it is easier to + find the actual shortest distance in hatspace and use the projection back + into dehatspace as an overestimate. + The distance will be used for two things: + 1) In the case that colour temperature does not strictly decrease with + increasing r/g, the closest point to the line will be chosen out of an + increasing pair of colours. + + 2) To calculate transverse negative an dpositive, the maximum positive + and negative distance from the line are chosen. This benefits from the + overestimate as the transverse pos/neg are upper bound values. + """ + """ + define fit function + """ + def f(x): + return a*x**2 + b*x + c + """ + iterate over points (R, B are x and y coordinates of points) and calculate + distance to line in dehatspace + """ + dists = [] + for i, (R, B) in enumerate(zip(rbs_hat[0], rbs_hat[1])): + """ + define function to minimise as square distance between datapoint and + point on curve. Squaring is monotonic so minimising radius squared is + equivalent to minimising radius + """ + def f_min(x): + y = f(x) + return((x-R)**2+(y-B)**2) + """ + perform optimisation with scipy.optmisie.fmin + """ + x_hat = fmin(f_min, R, disp=0)[0] + y_hat = f(x_hat) + """ + dehat + """ + x = x_hat/(1-x_hat-y_hat) + y = y_hat/(1-x_hat-y_hat) + rr = R/(1-R-B) + bb = B/(1-R-B) + """ + calculate euclidean distance in dehatspace + """ + dist = ((x-rr)**2+(y-bb)**2)**0.5 + """ + return negative if point is below the fit curve + """ + if (x+y) > (rr+bb): + dist *= -1 + dists.append(dist) + Cam.log += '\nFound closest point on fit line to each point in dehatspace' + """ + calculate wiggle factors in awb. 10% added since this is an upper bound + """ + transverse_neg = - np.min(dists) * 1.1 + transverse_pos = np.max(dists) * 1.1 + Cam.log += '\nTransverse pos : {:.5f}'.format(transverse_pos) + Cam.log += '\nTransverse neg : {:.5f}'.format(transverse_neg) + """ + set minimum transverse wiggles to 0.1 . + Wiggle factors dictate how far off of the curve the algorithm searches. 0.1 + is a suitable minimum that gives better results for lighting conditions not + within calibration dataset. Anything less will generalise poorly. + """ + if transverse_pos < 0.01: + transverse_pos = 0.01 + Cam.log += '\nForced transverse pos to 0.01' + if transverse_neg < 0.01: + transverse_neg = 0.01 + Cam.log += '\nForced transverse neg to 0.01' + + """ + generate new b_hat values at each r_hat according to fit + """ + r_hat_fit = np.array(rbs_hat[0]) + b_hat_fit = a*r_hat_fit**2 + b*r_hat_fit + c + """ + transform from hatspace to dehatspace + """ + r_fit = r_hat_fit/(1-r_hat_fit-b_hat_fit) + b_fit = b_hat_fit/(1-r_hat_fit-b_hat_fit) + c_fit = np.round(rbs_hat[2], 0) + """ + round to 4dp + """ + r_fit = np.where((1000*r_fit) % 1 <= 0.05, r_fit+0.0001, r_fit) + r_fit = np.where((1000*r_fit) % 1 >= 0.95, r_fit-0.0001, r_fit) + b_fit = np.where((1000*b_fit) % 1 <= 0.05, b_fit+0.0001, b_fit) + b_fit = np.where((1000*b_fit) % 1 >= 0.95, b_fit-0.0001, b_fit) + r_fit = np.round(r_fit, 4) + b_fit = np.round(b_fit, 4) + """ + The following code ensures that colour temperature decreases with + increasing r/g + """ + """ + iterate backwards over list for easier indexing + """ + i = len(c_fit) - 1 + while i > 0: + if c_fit[i] > c_fit[i-1]: + Cam.log += '\nColour temperature increase found\n' + Cam.log += '{} K at r = {} to '.format(c_fit[i-1], r_fit[i-1]) + Cam.log += '{} K at r = {}'.format(c_fit[i], r_fit[i]) + """ + if colour temperature increases then discard point furthest from + the transformed fit (dehatspace) + """ + error_1 = abs(dists[i-1]) + error_2 = abs(dists[i]) + Cam.log += '\nDistances from fit:\n' + Cam.log += '{} K : {:.5f} , '.format(c_fit[i], error_1) + Cam.log += '{} K : {:.5f}'.format(c_fit[i-1], error_2) + """ + find bad index + note that in python false = 0 and true = 1 + """ + bad = i - (error_1 < error_2) + Cam.log += '\nPoint at {} K deleted as '.format(c_fit[bad]) + Cam.log += 'it is furthest from fit' + """ + delete bad point + """ + r_fit = np.delete(r_fit, bad) + b_fit = np.delete(b_fit, bad) + c_fit = np.delete(c_fit, bad).astype(np.uint16) + """ + note that if a point has been discarded then the length has decreased + by one, meaning that decreasing the index by one will reassess the kept + point against the next point. It is therefore possible, in theory, for + two adjacent points to be discarded, although probably rare + """ + i -= 1 + + """ + return formatted ct curve, ordered by increasing colour temperature + """ + ct_curve = list(np.array(list(zip(b_fit, r_fit, c_fit))).flatten())[::-1] + Cam.log += '\nFinal CT curve:' + for i in range(len(ct_curve)//3): + j = 3*i + Cam.log += '\n ct: {} '.format(ct_curve[j]) + Cam.log += ' r: {} '.format(ct_curve[j+1]) + Cam.log += ' b: {} '.format(ct_curve[j+2]) + + """ + plotting code for debug + """ + if plot: + x = np.linspace(np.min(rbs_hat[0]), np.max(rbs_hat[0]), 100) + y = a*x**2 + b*x + c + plt.subplot(2, 1, 1) + plt.title('hatspace') + plt.plot(rbs_hat[0], rbs_hat[1], ls='--', color='blue') + plt.plot(x, y, color='green', ls='-') + plt.scatter(rbs_hat[0], rbs_hat[1], color='red') + for i, ct in enumerate(rbs_hat[2]): + plt.annotate(str(ct), (rbs_hat[0][i], rbs_hat[1][i])) + plt.xlabel('$\\hat{r}$') + plt.ylabel('$\\hat{b}$') + """ + optional set axes equal to shortest distance so line really does + looks perpendicular and everybody is happy + """ + # ax = plt.gca() + # ax.set_aspect('equal') + plt.grid() + plt.subplot(2, 1, 2) + plt.title('dehatspace - indoors?') + plt.plot(r_fit, b_fit, color='blue') + plt.scatter(rb_raw[0], rb_raw[1], color='green') + plt.scatter(r_fit, b_fit, color='red') + for i, ct in enumerate(c_fit): + plt.annotate(str(ct), (r_fit[i], b_fit[i])) + plt.xlabel('$r$') + plt.ylabel('$b$') + """ + optional set axes equal to shortest distance so line really does + looks perpendicular and everybody is happy + """ + # ax = plt.gca() + # ax.set_aspect('equal') + plt.subplots_adjust(hspace=0.5) + plt.grid() + plt.show() + """ + end of plotting code + """ + return(ct_curve, np.round(transverse_pos, 5), np.round(transverse_neg, 5)) + + +""" +obtain greyscale patches and perform alsc colour correction +""" +def get_alsc_patches(Img, colour_cals, grey=True): + """ + get patch centre coordinates, image colour and the actual + patches for each channel, remembering to subtract blacklevel + If grey then only greyscale patches considered + """ + if grey: + cen_coords = Img.cen_coords[3::4] + col = Img.col + patches = [np.array(Img.patches[i]) for i in Img.order] + r_patchs = patches[0][3::4] - Img.blacklevel_16 + b_patchs = patches[3][3::4] - Img.blacklevel_16 + """ + note two green channels are averages + """ + g_patchs = (patches[1][3::4]+patches[2][3::4])/2 - Img.blacklevel_16 + else: + cen_coords = Img.cen_coords + col = Img.col + patches = [np.array(Img.patches[i]) for i in Img.order] + r_patchs = patches[0] - Img.blacklevel_16 + b_patchs = patches[3] - Img.blacklevel_16 + g_patchs = (patches[1]+patches[2])/2 - Img.blacklevel_16 + + if colour_cals is None: + return r_patchs, b_patchs, g_patchs + """ + find where image colour fits in alsc colour calibration tables + """ + cts = list(colour_cals.keys()) + pos = bisect_left(cts, col) + """ + if img colour is below minimum or above maximum alsc calibration colour, simply + pick extreme closest to img colour + """ + if pos % len(cts) == 0: + """ + this works because -0 = 0 = first and -1 = last index + """ + col_tabs = np.array(colour_cals[cts[-pos//len(cts)]]) + """ + else, perform linear interpolation between existing alsc colour + calibration tables + """ + else: + bef = cts[pos-1] + aft = cts[pos] + da = col-bef + db = aft-col + bef_tabs = np.array(colour_cals[bef]) + aft_tabs = np.array(colour_cals[aft]) + col_tabs = (bef_tabs*db + aft_tabs*da)/(da+db) + col_tabs = np.reshape(col_tabs, (2, 12, 16)) + """ + calculate dx, dy used to calculate alsc table + """ + w, h = Img.w/2, Img.h/2 + dx, dy = int(-(-(w-1)//16)), int(-(-(h-1)//12)) + """ + make list of pairs of gains for each patch by selecting the correct value + in alsc colour calibration table + """ + patch_gains = [] + for cen in cen_coords: + x, y = cen[0]//dx, cen[1]//dy + # We could probably do with some better spatial interpolation here? + col_gains = (col_tabs[0][y][x], col_tabs[1][y][x]) + patch_gains.append(col_gains) + + """ + multiply the r and b channels in each patch by the respective gain, finally + performing the alsc colour correction + """ + for i, gains in enumerate(patch_gains): + r_patchs[i] = r_patchs[i] * gains[0] + b_patchs[i] = b_patchs[i] * gains[1] + + """ + return greyscale patches, g channel and correct r, b channels + """ + return r_patchs, b_patchs, g_patchs diff --git a/utils/tuning/libtuning/ctt_ccm.py b/utils/tuning/libtuning/ctt_ccm.py new file mode 100644 index 000000000000..59753e332ee9 --- /dev/null +++ b/utils/tuning/libtuning/ctt_ccm.py @@ -0,0 +1,406 @@ +# SPDX-License-Identifier: BSD-2-Clause +# +# Copyright (C) 2019, Raspberry Pi Ltd +# +# camera tuning tool for CCM (colour correction matrix) + +from ctt_image_load import * +from ctt_awb import get_alsc_patches +import colors +from scipy.optimize import minimize +from ctt_visualise import visualise_macbeth_chart +import numpy as np +""" +takes 8-bit macbeth chart values, degammas and returns 16 bit +""" + +''' +This program has many options from which to derive the color matrix from. +The first is average. This minimises the average delta E across all patches of +the macbeth chart. Testing across all cameras yeilded this as the most color +accurate and vivid. Other options are avalible however. +Maximum minimises the maximum Delta E of the patches. It iterates through till +a minimum maximum is found (so that there is +not one patch that deviates wildly.) +This yields generally good results but overall the colors are less accurate +Have a fiddle with maximum and see what you think. +The final option allows you to select the patches for which to average across. +This means that you can bias certain patches, for instance if you want the +reds to be more accurate. +''' + +matrix_selection_types = ["average", "maximum", "patches"] +typenum = 0 # select from array above, 0 = average, 1 = maximum, 2 = patches +test_patches = [1, 2, 5, 8, 9, 12, 14] + +''' +Enter patches to test for. Can also be entered twice if you +would like twice as much bias on one patch. +''' + + +def degamma(x): + x = x / ((2 ** 8) - 1) # takes 255 and scales it down to one + x = np.where(x < 0.04045, x / 12.92, ((x + 0.055) / 1.055) ** 2.4) + x = x * ((2 ** 16) - 1) # takes one and scales up to 65535, 16 bit color + return x + + +def gamma(x): + # Take 3 long array of color values and gamma them + return [((colour / 255) ** (1 / 2.4) * 1.055 - 0.055) * 255 for colour in x] + + +""" +FInds colour correction matrices for list of images +""" + + +def ccm(Cam, cal_cr_list, cal_cb_list): + global matrix_selection_types, typenum + imgs = Cam.imgs + """ + standard macbeth chart colour values + """ + m_rgb = np.array([ # these are in RGB + [116, 81, 67], # dark skin + [199, 147, 129], # light skin + [91, 122, 156], # blue sky + [90, 108, 64], # foliage + [130, 128, 176], # blue flower + [92, 190, 172], # bluish green + [224, 124, 47], # orange + [68, 91, 170], # purplish blue + [198, 82, 97], # moderate red + [94, 58, 106], # purple + [159, 189, 63], # yellow green + [230, 162, 39], # orange yellow + [35, 63, 147], # blue + [67, 149, 74], # green + [180, 49, 57], # red + [238, 198, 20], # yellow + [193, 84, 151], # magenta + [0, 136, 170], # cyan (goes out of gamut) + [245, 245, 243], # white 9.5 + [200, 202, 202], # neutral 8 + [161, 163, 163], # neutral 6.5 + [121, 121, 122], # neutral 5 + [82, 84, 86], # neutral 3.5 + [49, 49, 51] # black 2 + ]) + """ + convert reference colours from srgb to rgb + """ + m_srgb = degamma(m_rgb) # now in 16 bit color. + + # Produce array of LAB values for ideal color chart + m_lab = [colors.RGB_to_LAB(color / 256) for color in m_srgb] + + """ + reorder reference values to match how patches are ordered + """ + m_srgb = np.array([m_srgb[i::6] for i in range(6)]).reshape((24, 3)) + m_lab = np.array([m_lab[i::6] for i in range(6)]).reshape((24, 3)) + m_rgb = np.array([m_rgb[i::6] for i in range(6)]).reshape((24, 3)) + """ + reformat alsc correction tables or set colour_cals to None if alsc is + deactivated + """ + if cal_cr_list is None: + colour_cals = None + else: + colour_cals = {} + for cr, cb in zip(cal_cr_list, cal_cb_list): + cr_tab = cr['table'] + cb_tab = cb['table'] + """ + normalise tables so min value is 1 + """ + cr_tab = cr_tab / np.min(cr_tab) + cb_tab = cb_tab / np.min(cb_tab) + colour_cals[cr['ct']] = [cr_tab, cb_tab] + + """ + for each image, perform awb and alsc corrections. + Then calculate the colour correction matrix for that image, recording the + ccm and the colour tempertaure. + """ + ccm_tab = {} + for Img in imgs: + Cam.log += '\nProcessing image: ' + Img.name + """ + get macbeth patches with alsc applied if alsc enabled. + Note: if alsc is disabled then colour_cals will be set to None and no + the function will simply return the macbeth patches + """ + r, b, g = get_alsc_patches(Img, colour_cals, grey=False) + # 256 values for each patch of sRGB values + + """ + do awb + Note: awb is done by measuring the macbeth chart in the image, rather + than from the awb calibration. This is done so the awb will be perfect + and the ccm matrices will be more accurate. + """ + r_greys, b_greys, g_greys = r[3::4], b[3::4], g[3::4] + r_g = np.mean(r_greys / g_greys) + b_g = np.mean(b_greys / g_greys) + r = r / r_g + b = b / b_g + """ + normalise brightness wrt reference macbeth colours and then average + each channel for each patch + """ + gain = np.mean(m_srgb) / np.mean((r, g, b)) + Cam.log += '\nGain with respect to standard colours: {:.3f}'.format(gain) + r = np.mean(gain * r, axis=1) + b = np.mean(gain * b, axis=1) + g = np.mean(gain * g, axis=1) + """ + calculate ccm matrix + """ + # ==== All of below should in sRGB ===## + sumde = 0 + ccm = do_ccm(r, g, b, m_srgb) + # This is the initial guess that our optimisation code works with. + original_ccm = ccm + r1 = ccm[0] + r2 = ccm[1] + g1 = ccm[3] + g2 = ccm[4] + b1 = ccm[6] + b2 = ccm[7] + ''' + COLOR MATRIX LOOKS AS BELOW + R1 R2 R3 Rval Outr + G1 G2 G3 * Gval = G + B1 B2 B3 Bval B + Will be optimising 6 elements and working out the third element using 1-r1-r2 = r3 + ''' + + x0 = [r1, r2, g1, g2, b1, b2] + ''' + We use our old CCM as the initial guess for the program to find the + optimised matrix + ''' + result = minimize(guess, x0, args=(r, g, b, m_lab), tol=0.01) + ''' + This produces a color matrix which has the lowest delta E possible, + based off the input data. Note it is impossible for this to reach + zero since the input data is imperfect + ''' + + Cam.log += ("\n \n Optimised Matrix Below: \n \n") + [r1, r2, g1, g2, b1, b2] = result.x + # The new, optimised color correction matrix values + optimised_ccm = [r1, r2, (1 - r1 - r2), g1, g2, (1 - g1 - g2), b1, b2, (1 - b1 - b2)] + + # This is the optimised Color Matrix (preserving greys by summing rows up to 1) + Cam.log += str(optimised_ccm) + Cam.log += "\n Old Color Correction Matrix Below \n" + Cam.log += str(ccm) + + formatted_ccm = np.array(original_ccm).reshape((3, 3)) + + ''' + below is a whole load of code that then applies the latest color + matrix, and returns LAB values for color. This can then be used + to calculate the final delta E + ''' + optimised_ccm_rgb = [] # Original Color Corrected Matrix RGB / LAB + optimised_ccm_lab = [] + + formatted_optimised_ccm = np.array(optimised_ccm).reshape((3, 3)) + after_gamma_rgb = [] + after_gamma_lab = [] + + for RGB in zip(r, g, b): + ccm_applied_rgb = np.dot(formatted_ccm, (np.array(RGB) / 256)) + optimised_ccm_rgb.append(gamma(ccm_applied_rgb)) + optimised_ccm_lab.append(colors.RGB_to_LAB(ccm_applied_rgb)) + + optimised_ccm_applied_rgb = np.dot(formatted_optimised_ccm, np.array(RGB) / 256) + after_gamma_rgb.append(gamma(optimised_ccm_applied_rgb)) + after_gamma_lab.append(colors.RGB_to_LAB(optimised_ccm_applied_rgb)) + ''' + Gamma After RGB / LAB - not used in calculations, only used for visualisation + We now want to spit out some data that shows + how the optimisation has improved the color matrices + ''' + Cam.log += "Here are the Improvements" + + # CALCULATE WORST CASE delta e + old_worst_delta_e = 0 + before_average = transform_and_evaluate(formatted_ccm, r, g, b, m_lab) + new_worst_delta_e = 0 + after_average = transform_and_evaluate(formatted_optimised_ccm, r, g, b, m_lab) + for i in range(24): + old_delta_e = deltae(optimised_ccm_lab[i], m_lab[i]) # Current Old Delta E + new_delta_e = deltae(after_gamma_lab[i], m_lab[i]) # Current New Delta E + if old_delta_e > old_worst_delta_e: + old_worst_delta_e = old_delta_e + if new_delta_e > new_worst_delta_e: + new_worst_delta_e = new_delta_e + + Cam.log += "Before color correction matrix was optimised, we got an average delta E of " + str(before_average) + " and a maximum delta E of " + str(old_worst_delta_e) + Cam.log += "After color correction matrix was optimised, we got an average delta E of " + str(after_average) + " and a maximum delta E of " + str(new_worst_delta_e) + + visualise_macbeth_chart(m_rgb, optimised_ccm_rgb, after_gamma_rgb, str(Img.col) + str(matrix_selection_types[typenum])) + ''' + The program will also save some visualisations of improvements. + Very pretty to look at. Top rectangle is ideal, Left square is + before optimisation, right square is after. + ''' + + """ + if a ccm has already been calculated for that temperature then don't + overwrite but save both. They will then be averaged later on + """ # Now going to use optimised color matrix, optimised_ccm + if Img.col in ccm_tab.keys(): + ccm_tab[Img.col].append(optimised_ccm) + else: + ccm_tab[Img.col] = [optimised_ccm] + Cam.log += '\n' + + Cam.log += '\nFinished processing images' + """ + average any ccms that share a colour temperature + """ + for k, v in ccm_tab.items(): + tab = np.mean(v, axis=0) + tab = np.where((10000 * tab) % 1 <= 0.05, tab + 0.00001, tab) + tab = np.where((10000 * tab) % 1 >= 0.95, tab - 0.00001, tab) + ccm_tab[k] = list(np.round(tab, 5)) + Cam.log += '\nMatrix calculated for colour temperature of {} K'.format(k) + + """ + return all ccms with respective colour temperature in the correct format, + sorted by their colour temperature + """ + sorted_ccms = sorted(ccm_tab.items(), key=lambda kv: kv[0]) + ccms = [] + for i in sorted_ccms: + ccms.append({ + 'ct': i[0], + 'ccm': i[1] + }) + return ccms + + +def guess(x0, r, g, b, m_lab): # provides a method of numerical feedback for the optimisation code + [r1, r2, g1, g2, b1, b2] = x0 + ccm = np.array([r1, r2, (1 - r1 - r2), + g1, g2, (1 - g1 - g2), + b1, b2, (1 - b1 - b2)]).reshape((3, 3)) # format the matrix correctly + return transform_and_evaluate(ccm, r, g, b, m_lab) + + +def transform_and_evaluate(ccm, r, g, b, m_lab): # Transforms colors to LAB and applies the correction matrix + # create list of matrix changed colors + realrgb = [] + for RGB in zip(r, g, b): + rgb_post_ccm = np.dot(ccm, np.array(RGB) / 256) # This is RGB values after the color correction matrix has been applied + realrgb.append(colors.RGB_to_LAB(rgb_post_ccm)) + # now compare that with m_lab and return numeric result, averaged for each patch + return (sumde(realrgb, m_lab) / 24) # returns an average result of delta E + + +def sumde(listA, listB): + global typenum, test_patches + sumde = 0 + maxde = 0 + patchde = [] # Create array of the delta E values for each patch. useful for optimisation of certain patches + for listA_item, listB_item in zip(listA, listB): + if maxde < (deltae(listA_item, listB_item)): + maxde = deltae(listA_item, listB_item) + patchde.append(deltae(listA_item, listB_item)) + sumde += deltae(listA_item, listB_item) + ''' + The different options specified at the start allow for + the maximum to be returned, average or specific patches + ''' + if typenum == 0: + return sumde + if typenum == 1: + return maxde + if typenum == 2: + output = sum([patchde[test_patch] for test_patch in test_patches]) + # Selects only certain patches and returns the output for them + return output + + +""" +calculates the ccm for an individual image. +ccms are calculated in rgb space, and are fit by hand. Although it is a 3x3 +matrix, each row must add up to 1 in order to conserve greyness, simplifying +calculation. +The initial CCM is calculated in RGB, and then optimised in LAB color space +This simplifies the initial calculation but then gets us the accuracy of +using LAB color space. +""" + + +def do_ccm(r, g, b, m_srgb): + rb = r-b + gb = g-b + rb_2s = (rb * rb) + rb_gbs = (rb * gb) + gb_2s = (gb * gb) + + r_rbs = rb * (m_srgb[..., 0] - b) + r_gbs = gb * (m_srgb[..., 0] - b) + g_rbs = rb * (m_srgb[..., 1] - b) + g_gbs = gb * (m_srgb[..., 1] - b) + b_rbs = rb * (m_srgb[..., 2] - b) + b_gbs = gb * (m_srgb[..., 2] - b) + + """ + Obtain least squares fit + """ + rb_2 = np.sum(rb_2s) + gb_2 = np.sum(gb_2s) + rb_gb = np.sum(rb_gbs) + r_rb = np.sum(r_rbs) + r_gb = np.sum(r_gbs) + g_rb = np.sum(g_rbs) + g_gb = np.sum(g_gbs) + b_rb = np.sum(b_rbs) + b_gb = np.sum(b_gbs) + + det = rb_2 * gb_2 - rb_gb * rb_gb + + """ + Raise error if matrix is singular... + This shouldn't really happen with real data but if it does just take new + pictures and try again, not much else to be done unfortunately... + """ + if det < 0.001: + raise ArithmeticError + + r_a = (gb_2 * r_rb - rb_gb * r_gb) / det + r_b = (rb_2 * r_gb - rb_gb * r_rb) / det + """ + Last row can be calculated by knowing the sum must be 1 + """ + r_c = 1 - r_a - r_b + + g_a = (gb_2 * g_rb - rb_gb * g_gb) / det + g_b = (rb_2 * g_gb - rb_gb * g_rb) / det + g_c = 1 - g_a - g_b + + b_a = (gb_2 * b_rb - rb_gb * b_gb) / det + b_b = (rb_2 * b_gb - rb_gb * b_rb) / det + b_c = 1 - b_a - b_b + + """ + format ccm + """ + ccm = [r_a, r_b, r_c, g_a, g_b, g_c, b_a, b_b, b_c] + + return ccm + + +def deltae(colorA, colorB): + return ((colorA[0] - colorB[0]) ** 2 + (colorA[1] - colorB[1]) ** 2 + (colorA[2] - colorB[2]) ** 2) ** 0.5 + # return ((colorA[1]-colorB[1]) * * 2 + (colorA[2]-colorB[2]) * * 2) * * 0.5 + # UNCOMMENT IF YOU WANT TO NEGLECT LUMINANCE FROM CALCULATION OF DELTA E diff --git a/utils/tuning/libtuning/ctt_colors.py b/utils/tuning/libtuning/ctt_colors.py new file mode 100644 index 000000000000..cb4d236b04d7 --- /dev/null +++ b/utils/tuning/libtuning/ctt_colors.py @@ -0,0 +1,30 @@ +# Program to convert from RGB to LAB color space +def RGB_to_LAB(RGB): # where RGB is a 1x3 array. e.g RGB = [100, 255, 230] + num = 0 + XYZ = [0, 0, 0] + # converted all the three R, G, B to X, Y, Z + X = RGB[0] * 0.4124 + RGB[1] * 0.3576 + RGB[2] * 0.1805 + Y = RGB[0] * 0.2126 + RGB[1] * 0.7152 + RGB[2] * 0.0722 + Z = RGB[0] * 0.0193 + RGB[1] * 0.1192 + RGB[2] * 0.9505 + + XYZ[0] = X / 255 * 100 + XYZ[1] = Y / 255 * 100 # XYZ Must be in range 0 -> 100, so scale down from 255 + XYZ[2] = Z / 255 * 100 + XYZ[0] = XYZ[0] / 95.047 # ref_X = 95.047 Observer= 2°, Illuminant= D65 + XYZ[1] = XYZ[1] / 100.0 # ref_Y = 100.000 + XYZ[2] = XYZ[2] / 108.883 # ref_Z = 108.883 + num = 0 + for value in XYZ: + if value > 0.008856: + value = value ** (0.3333333333333333) + else: + value = (7.787 * value) + (16 / 116) + XYZ[num] = value + num = num + 1 + + # L, A, B, values calculated below + L = (116 * XYZ[1]) - 16 + a = 500 * (XYZ[0] - XYZ[1]) + b = 200 * (XYZ[1] - XYZ[2]) + + return [L, a, b] diff --git a/utils/tuning/libtuning/ctt_ransac.py b/utils/tuning/libtuning/ctt_ransac.py new file mode 100644 index 000000000000..01bba3022ef0 --- /dev/null +++ b/utils/tuning/libtuning/ctt_ransac.py @@ -0,0 +1,71 @@ +# SPDX-License-Identifier: BSD-2-Clause +# +# Copyright (C) 2019, Raspberry Pi Ltd +# +# camera tuning tool RANSAC selector for Macbeth chart locator + +import numpy as np + +scale = 2 + + +""" +constructs normalised macbeth chart corners for ransac algorithm +""" +def get_square_verts(c_err=0.05, scale=scale): + """ + define macbeth chart corners + """ + b_bord_x, b_bord_y = scale*8.5, scale*13 + s_bord = 6*scale + side = 41*scale + x_max = side*6 + 5*s_bord + 2*b_bord_x + y_max = side*4 + 3*s_bord + 2*b_bord_y + c1 = (0, 0) + c2 = (0, y_max) + c3 = (x_max, y_max) + c4 = (x_max, 0) + mac_norm = np.array((c1, c2, c3, c4), np.float32) + mac_norm = np.array([mac_norm]) + + square_verts = [] + square_0 = np.array(((0, 0), (0, side), + (side, side), (side, 0)), np.float32) + offset_0 = np.array((b_bord_x, b_bord_y), np.float32) + c_off = side * c_err + offset_cont = np.array(((c_off, c_off), (c_off, -c_off), + (-c_off, -c_off), (-c_off, c_off)), np.float32) + square_0 += offset_0 + square_0 += offset_cont + """ + define macbeth square corners + """ + for i in range(6): + shift_i = np.array(((i*side, 0), (i*side, 0), + (i*side, 0), (i*side, 0)), np.float32) + shift_bord = np.array(((i*s_bord, 0), (i*s_bord, 0), + (i*s_bord, 0), (i*s_bord, 0)), np.float32) + square_i = square_0 + shift_i + shift_bord + for j in range(4): + shift_j = np.array(((0, j*side), (0, j*side), + (0, j*side), (0, j*side)), np.float32) + shift_bord = np.array(((0, j*s_bord), + (0, j*s_bord), (0, j*s_bord), + (0, j*s_bord)), np.float32) + square_j = square_i + shift_j + shift_bord + square_verts.append(square_j) + # print('square_verts') + # print(square_verts) + return np.array(square_verts, np.float32), mac_norm + + +def get_square_centres(c_err=0.05, scale=scale): + """ + define macbeth square centres + """ + verts, mac_norm = get_square_verts(c_err, scale=scale) + + centres = np.mean(verts, axis=1) + # print('centres') + # print(centres) + return np.array(centres, np.float32) From patchwork Wed Jul 3 14:16:53 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefan Klug X-Patchwork-Id: 20518 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id C0048BEFBE for ; Wed, 3 Jul 2024 14:17:46 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 7DB5E63339; Wed, 3 Jul 2024 16:17:46 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="kxTlf01I"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 9BF826333B for ; Wed, 3 Jul 2024 16:17:43 +0200 (CEST) Received: from ideasonboard.com (unknown [IPv6:2a00:6020:448c:6c00:9263:c199:9587:576]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 5F50B4CA; Wed, 3 Jul 2024 16:17:15 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1720016235; bh=5fXRCD14xWuqf3/uVumQEH9XBl0Wg7gzKSUIW7HYpiA=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=kxTlf01Ix3NMMaJWLEykeq9YptLoxyUodUy35pXGZjka41CppFZobv3dpx6OUrUGY FhJmYJrTyrWBbT80RAIj5CYDlip2I6AfyS9SkAI0zuduMoJSdae5cZDT+Lsyn3MJWe vy3hnKC32VIsUkIuEEW7x8pplmU/RntBJkl2xf24= From: Stefan Klug To: libcamera-devel@lists.libcamera.org Cc: Stefan Klug , Laurent Pinchart Subject: [PATCH v3 04/23] libtuning: Copy visualize_macbeth_chart from raspberry pi Date: Wed, 3 Jul 2024 16:16:53 +0200 Message-ID: <20240703141726.252368-5-stefan.klug@ideasonboard.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240703141726.252368-1-stefan.klug@ideasonboard.com> References: <20240703141726.252368-1-stefan.klug@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Copy visualize_macbeth_chart from raspberry pi. It is copied verbatim and does not work in this state. Signed-off-by: Stefan Klug Reviewed-by: Laurent Pinchart Acked-by: Paul Elder --- utils/tuning/libtuning/utils.py | 43 +++++++++++++++++++++++++++++++++ 1 file changed, 43 insertions(+) diff --git a/utils/tuning/libtuning/utils.py b/utils/tuning/libtuning/utils.py index 1e8128ea0571..f099c0ed685c 100644 --- a/utils/tuning/libtuning/utils.py +++ b/utils/tuning/libtuning/utils.py @@ -123,3 +123,46 @@ def load_images(input_dir: str, config: dict, load_nonlsc: bool, load_lsc: bool) return None return images + + + +""" +Some code that will save virtual macbeth charts that show the difference between optimised matrices and non optimised matrices + +The function creates an image that is 1550 by 1050 pixels wide, and fills it with patches which are 200x200 pixels in size +Each patch contains the ideal color, the color from the original matrix, and the color from the final matrix +_________________ +| | +| Ideal Color | +|_______________| +| Old | new | +| Color | Color | +|_______|_______| + +Nice way of showing how the optimisation helps change the colors and the color matricies +""" +def visualise_macbeth_chart(macbeth_rgb, original_rgb, new_rgb, output_filename): + image = np.zeros((1050, 1550, 3), dtype=np.uint8) + colorindex = -1 + for y in range(6): + for x in range(4): # Creates 6 x 4 grid of macbeth chart + colorindex += 1 + xlocation = 50 + 250 * x # Means there is 50px of black gap between each square, more like the real macbeth chart. + ylocation = 50 + 250 * y + for g in range(200): + for i in range(100): + image[xlocation + i, ylocation + g] = macbeth_rgb[colorindex] + xlocation = 150 + 250 * x + ylocation = 50 + 250 * y + for i in range(100): + for g in range(100): + image[xlocation + i, ylocation + g] = original_rgb[colorindex] # Smaller squares below to compare the old colors with the new ones + xlocation = 150 + 250 * x + ylocation = 150 + 250 * y + for i in range(100): + for g in range(100): + image[xlocation + i, ylocation + g] = new_rgb[colorindex] + + img = Image.fromarray(image, 'RGB') + img.save(str(output_filename) + 'Generated Macbeth Chart.png') + From patchwork Wed Jul 3 14:16:54 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefan Klug X-Patchwork-Id: 20519 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 9B67FBEFBE for ; Wed, 3 Jul 2024 14:17:49 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 47ACF63352; Wed, 3 Jul 2024 16:17:49 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="YCdh+L56"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 2468463340 for ; Wed, 3 Jul 2024 16:17:47 +0200 (CEST) Received: from ideasonboard.com (unknown [IPv6:2a00:6020:448c:6c00:9263:c199:9587:576]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 077F14CA; Wed, 3 Jul 2024 16:17:19 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1720016239; bh=mQmlD9VVezWjUEQ6QLbsJLSU9RqmB8vdyw97qO9yw38=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=YCdh+L56MEMdEwUeCPkL0klxs0nUd+LqFhNFT7AL5KEudusg3d5jw/7DYa3Z/A9xS MLHsqS15IciO0FaepGdYJ9l4guXJehc3X7hzWnntNkrgUpxh+4DseFarsHGyk/t968 pK4MspvwABQoZ1rm7Z4t8urRvEOSTAElv60JvJsI= From: Stefan Klug To: libcamera-devel@lists.libcamera.org Cc: Stefan Klug , Laurent Pinchart Subject: [PATCH v3 05/23] utils: tuning: Add requirements file and update readme Date: Wed, 3 Jul 2024 16:16:54 +0200 Message-ID: <20240703141726.252368-6-stefan.klug@ideasonboard.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240703141726.252368-1-stefan.klug@ideasonboard.com> References: <20240703141726.252368-1-stefan.klug@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Add a requirements file to ease the installation and use of the tuning scripts. Document that in the readme. No debian packages are provided as rawpy is not packaged as deb. So pip has to be used anyways. Also add pyyaml which was missing in the dependencies. Signed-off-by: Stefan Klug Reviewed-by: Laurent Pinchart Reviewed-by: Paul Elder --- utils/tuning/README.rst | 23 ++++++++++++++++------- utils/tuning/requirements.txt | 5 +++++ 2 files changed, 21 insertions(+), 7 deletions(-) create mode 100644 utils/tuning/requirements.txt diff --git a/utils/tuning/README.rst b/utils/tuning/README.rst index ef3e6ad76f1b..ec05c116c536 100644 --- a/utils/tuning/README.rst +++ b/utils/tuning/README.rst @@ -1,11 +1,20 @@ .. SPDX-License-Identifier: CC-BY-SA-4.0 -.. TODO: Write an overview of libtuning +libcamera tuning tools +====================== -Dependencies ------------- +.. Note:: The tuning tool are still very much work in progress. If in doubt, + please ask on the mailing list. + +.. todo:: + Write documentation + +Installation of dependencies +---------------------------- + +:: + # Using a venv + python3 -m venv venv + . ./venv/bin/activate + pip3 install -r requirements.txt -- numpy -- opencv-python -- py3exiv2 -- rawpy diff --git a/utils/tuning/requirements.txt b/utils/tuning/requirements.txt new file mode 100644 index 000000000000..d1dc589d0329 --- /dev/null +++ b/utils/tuning/requirements.txt @@ -0,0 +1,5 @@ +numpy +opencv-python +py3exiv2 +pyyaml +rawpy From patchwork Wed Jul 3 14:16:55 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefan Klug X-Patchwork-Id: 20520 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 6E2FABEFBE for ; Wed, 3 Jul 2024 14:17:53 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 146766333B; Wed, 3 Jul 2024 16:17:53 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="BacAVCJp"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id C6AE66333B for ; Wed, 3 Jul 2024 16:17:49 +0200 (CEST) Received: from ideasonboard.com (unknown [IPv6:2a00:6020:448c:6c00:9263:c199:9587:576]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id A88094CA; Wed, 3 Jul 2024 16:17:21 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1720016241; bh=7aE52oCUh9ZNIxxT8s5bnyRaWjcCtyo80eVfD4vW8jY=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=BacAVCJptjzS1PwzFf/bNMTJBlOp2h9JRjEz2ZWWyJcjwAbbW12/FTCF6hikm/dU3 gRP4S/KDlllQbkDxHkX2jzpiNcGsxJtVEGFFymh66D+C/iI5JNtVu1wxr3+c/cQsuo Ml/nrdYBIjiaTDXovNBhuMwrbH9Y/sA+DCTaZDjo= From: Stefan Klug To: libcamera-devel@lists.libcamera.org Cc: Stefan Klug , Paul Elder , Laurent Pinchart Subject: [PATCH v3 06/23] libtuning: Fix imports Date: Wed, 3 Jul 2024 16:16:55 +0200 Message-ID: <20240703141726.252368-7-stefan.klug@ideasonboard.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240703141726.252368-1-stefan.klug@ideasonboard.com> References: <20240703141726.252368-1-stefan.klug@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Fix imports to match new structure in the files copied from Raspberry Pi. Add missing imports in macbeth.py. Signed-off-by: Stefan Klug Reviewed-by: Paul Elder Reviewed-by: Laurent Pinchart --- utils/tuning/libtuning/ctt_awb.py | 4 +++- utils/tuning/libtuning/ctt_ccm.py | 12 +++++++----- utils/tuning/libtuning/macbeth.py | 5 +++++ 3 files changed, 15 insertions(+), 6 deletions(-) diff --git a/utils/tuning/libtuning/ctt_awb.py b/utils/tuning/libtuning/ctt_awb.py index 5ba6f978a228..f3a1ce779e21 100644 --- a/utils/tuning/libtuning/ctt_awb.py +++ b/utils/tuning/libtuning/ctt_awb.py @@ -4,10 +4,12 @@ # # camera tuning tool for AWB -from ctt_image_load import * import matplotlib.pyplot as plt from bisect import bisect_left from scipy.optimize import fmin +import numpy as np + +from .image import Image """ diff --git a/utils/tuning/libtuning/ctt_ccm.py b/utils/tuning/libtuning/ctt_ccm.py index 59753e332ee9..f37adaf45538 100644 --- a/utils/tuning/libtuning/ctt_ccm.py +++ b/utils/tuning/libtuning/ctt_ccm.py @@ -4,12 +4,14 @@ # # camera tuning tool for CCM (colour correction matrix) -from ctt_image_load import * -from ctt_awb import get_alsc_patches -import colors -from scipy.optimize import minimize -from ctt_visualise import visualise_macbeth_chart import numpy as np +from scipy.optimize import minimize + +from . import ctt_colors as colors +from .image import Image +from .ctt_awb import get_alsc_patches +from .utils import visualise_macbeth_chart + """ takes 8-bit macbeth chart values, degammas and returns 16 bit """ diff --git a/utils/tuning/libtuning/macbeth.py b/utils/tuning/libtuning/macbeth.py index 81f3e87c9088..265a33d68378 100644 --- a/utils/tuning/libtuning/macbeth.py +++ b/utils/tuning/libtuning/macbeth.py @@ -1,6 +1,7 @@ # SPDX-License-Identifier: BSD-2-Clause # # Copyright (C) 2019, Raspberry Pi Ltd +# Copyright (C) 2024, Ideas on Board Oy # # Locate and extract Macbeth charts from images # (Copied from: ctt_macbeth_locator.py) @@ -11,6 +12,10 @@ import cv2 import os from pathlib import Path import numpy as np +import warnings +from sklearn import cluster as cluster + +from .ctt_ransac import get_square_verts, get_square_centres from libtuning.image import Image From patchwork Wed Jul 3 14:16:56 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefan Klug X-Patchwork-Id: 20521 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 2D8EEBEFBE for ; Wed, 3 Jul 2024 14:17:56 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id D25976333B; Wed, 3 Jul 2024 16:17:55 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="ij8pm13K"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id E41D66333E for ; Wed, 3 Jul 2024 16:17:52 +0200 (CEST) Received: from ideasonboard.com (unknown [IPv6:2a00:6020:448c:6c00:9263:c199:9587:576]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id C11814CA; Wed, 3 Jul 2024 16:17:24 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1720016244; bh=2/0kLR7tryEdS0J0pDqxoUdQUJxadgI3BbWkuYkyK9Y=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=ij8pm13KIiUFozhcWW3rGC6k2ky3Xn5m+PYAGdpIsC/h8QKTPgT4IwsTTaQVoz3Go XELiyXiVAIUqvB2Klb8L4kentsgdcRUo9IGSebfz3sBvZTau20YLxc/oOKsTODgQn3 li9AdFFpPVO8YQnAeIb7CI7SSpBdrNNGIzZNCPno= From: Stefan Klug To: libcamera-devel@lists.libcamera.org Cc: Stefan Klug Subject: [PATCH v3 07/23] libtuning: Migrate prints to python logging framework Date: Wed, 3 Jul 2024 16:16:56 +0200 Message-ID: <20240703141726.252368-8-stefan.klug@ideasonboard.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240703141726.252368-1-stefan.klug@ideasonboard.com> References: <20240703141726.252368-1-stefan.klug@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" In ctt_ccm.py the logging functionality of the Cam object was used. As we don't want to port over that class, it needs to be replaced anyways. While at it, also replace the eprint function as it doesn't add any value over the logging framework and misses the ability for easy log formatting. For nice output formatting add the coloredlogs library. Signed-off-by: Stefan Klug Reviewed-by: Paul Elder --- utils/tuning/libtuning/ctt_ccm.py | 27 ++++++++++--------- .../libtuning/generators/yaml_output.py | 5 ++-- utils/tuning/libtuning/image.py | 7 +++-- utils/tuning/libtuning/libtuning.py | 21 ++++++++------- utils/tuning/libtuning/macbeth.py | 13 +++++---- .../libtuning/modules/lsc/raspberrypi.py | 12 +++++---- utils/tuning/libtuning/utils.py | 17 ++++++------ utils/tuning/requirements.txt | 1 + utils/tuning/rkisp1.py | 5 ++++ 9 files changed, 62 insertions(+), 46 deletions(-) diff --git a/utils/tuning/libtuning/ctt_ccm.py b/utils/tuning/libtuning/ctt_ccm.py index f37adaf45538..c4362756c3c0 100644 --- a/utils/tuning/libtuning/ctt_ccm.py +++ b/utils/tuning/libtuning/ctt_ccm.py @@ -4,6 +4,8 @@ # # camera tuning tool for CCM (colour correction matrix) +import logging + import numpy as np from scipy.optimize import minimize @@ -12,6 +14,8 @@ from .image import Image from .ctt_awb import get_alsc_patches from .utils import visualise_macbeth_chart +logger = logging.getLogger(__name__) + """ takes 8-bit macbeth chart values, degammas and returns 16 bit """ @@ -129,7 +133,7 @@ def ccm(Cam, cal_cr_list, cal_cb_list): """ ccm_tab = {} for Img in imgs: - Cam.log += '\nProcessing image: ' + Img.name + logger.info('Processing image: ' + Img.name) """ get macbeth patches with alsc applied if alsc enabled. Note: if alsc is disabled then colour_cals will be set to None and no @@ -154,7 +158,7 @@ def ccm(Cam, cal_cr_list, cal_cb_list): each channel for each patch """ gain = np.mean(m_srgb) / np.mean((r, g, b)) - Cam.log += '\nGain with respect to standard colours: {:.3f}'.format(gain) + logger.info(f'Gain with respect to standard colours: {gain:.3f}') r = np.mean(gain * r, axis=1) b = np.mean(gain * b, axis=1) g = np.mean(gain * g, axis=1) @@ -192,15 +196,13 @@ def ccm(Cam, cal_cr_list, cal_cb_list): zero since the input data is imperfect ''' - Cam.log += ("\n \n Optimised Matrix Below: \n \n") [r1, r2, g1, g2, b1, b2] = result.x # The new, optimised color correction matrix values + # This is the optimised Color Matrix (preserving greys by summing rows up to 1) optimised_ccm = [r1, r2, (1 - r1 - r2), g1, g2, (1 - g1 - g2), b1, b2, (1 - b1 - b2)] - # This is the optimised Color Matrix (preserving greys by summing rows up to 1) - Cam.log += str(optimised_ccm) - Cam.log += "\n Old Color Correction Matrix Below \n" - Cam.log += str(ccm) + logger.info(f'Optimized Matrix: {np.round(optimised_ccm, 4)}') + logger.info(f'Old Matrix: {np.round(ccm, 4)}') formatted_ccm = np.array(original_ccm).reshape((3, 3)) @@ -229,7 +231,7 @@ def ccm(Cam, cal_cr_list, cal_cb_list): We now want to spit out some data that shows how the optimisation has improved the color matrices ''' - Cam.log += "Here are the Improvements" + logger.info("Here are the Improvements") # CALCULATE WORST CASE delta e old_worst_delta_e = 0 @@ -244,8 +246,8 @@ def ccm(Cam, cal_cr_list, cal_cb_list): if new_delta_e > new_worst_delta_e: new_worst_delta_e = new_delta_e - Cam.log += "Before color correction matrix was optimised, we got an average delta E of " + str(before_average) + " and a maximum delta E of " + str(old_worst_delta_e) - Cam.log += "After color correction matrix was optimised, we got an average delta E of " + str(after_average) + " and a maximum delta E of " + str(new_worst_delta_e) + logger.info(f'delta E optimized: average: {after_average:.2f} max:{new_worst_delta_e:.2f}') + logger.info(f'delta E old: average: {before_average:.2f} max:{old_worst_delta_e:.2f}') visualise_macbeth_chart(m_rgb, optimised_ccm_rgb, after_gamma_rgb, str(Img.col) + str(matrix_selection_types[typenum])) ''' @@ -262,9 +264,8 @@ def ccm(Cam, cal_cr_list, cal_cb_list): ccm_tab[Img.col].append(optimised_ccm) else: ccm_tab[Img.col] = [optimised_ccm] - Cam.log += '\n' - Cam.log += '\nFinished processing images' + logger.info('Finished processing images') """ average any ccms that share a colour temperature """ @@ -273,7 +274,7 @@ def ccm(Cam, cal_cr_list, cal_cb_list): tab = np.where((10000 * tab) % 1 <= 0.05, tab + 0.00001, tab) tab = np.where((10000 * tab) % 1 >= 0.95, tab - 0.00001, tab) ccm_tab[k] = list(np.round(tab, 5)) - Cam.log += '\nMatrix calculated for colour temperature of {} K'.format(k) + logger.info(f'Matrix calculated for colour temperature of {k} K') """ return all ccms with respective colour temperature in the correct format, diff --git a/utils/tuning/libtuning/generators/yaml_output.py b/utils/tuning/libtuning/generators/yaml_output.py index 8f22d386f1b3..31e265df4ea7 100644 --- a/utils/tuning/libtuning/generators/yaml_output.py +++ b/utils/tuning/libtuning/generators/yaml_output.py @@ -9,8 +9,9 @@ from .generator import Generator from numbers import Number from pathlib import Path -import libtuning.utils as utils +import logging +logger = logging.getLogger(__name__) class YamlOutput(Generator): def __init__(self): @@ -112,7 +113,7 @@ class YamlOutput(Generator): continue if not isinstance(output_dict[module], dict): - utils.eprint(f'Error: Output of {module.type} is not a dictionary') + logger.error(f'Error: Output of {module.type} is not a dictionary') continue lines = self._stringify_dict(output_dict[module]) diff --git a/utils/tuning/libtuning/image.py b/utils/tuning/libtuning/image.py index 6ff60ec17ec4..2c4d774f11e2 100644 --- a/utils/tuning/libtuning/image.py +++ b/utils/tuning/libtuning/image.py @@ -13,6 +13,9 @@ import re import libtuning as lt import libtuning.utils as utils +import logging + +logger = logging.getLogger(__name__) class Image: @@ -25,13 +28,13 @@ class Image: try: self._load_metadata_exif() except Exception as e: - utils.eprint(f'Failed to load metadata from {self.path}: {e}') + logger.error(f'Failed to load metadata from {self.path}: {e}') raise e try: self._read_image_dng() except Exception as e: - utils.eprint(f'Failed to load image data from {self.path}: {e}') + logger.error(f'Failed to load image data from {self.path}: {e}') raise e @property diff --git a/utils/tuning/libtuning/libtuning.py b/utils/tuning/libtuning/libtuning.py index 5e22288df49b..5342e5d6daaa 100644 --- a/utils/tuning/libtuning/libtuning.py +++ b/utils/tuning/libtuning/libtuning.py @@ -5,13 +5,14 @@ # An infrastructure for camera tuning tools import argparse +import logging import libtuning as lt import libtuning.utils as utils -from libtuning.utils import eprint from enum import Enum, IntEnum +logger = logging.getLogger(__name__) class Color(IntEnum): R = 0 @@ -112,10 +113,10 @@ class Tuner(object): for module_type in output_order: modules = [module for module in self.modules if module.type == module_type.type] if len(modules) > 1: - eprint(f'Multiple modules found for module type "{module_type.type}"') + logger.error(f'Multiple modules found for module type "{module_type.type}"') return False if len(modules) < 1: - eprint(f'No module found for module type "{module_type.type}"') + logger.error(f'No module found for module type "{module_type.type}"') return False self.output_order.append(modules[0]) @@ -124,19 +125,19 @@ class Tuner(object): # \todo Validate parser and generator at Tuner construction time? def _validate_settings(self): if self.parser is None: - eprint('Missing parser') + logger.error('Missing parser') return False if self.generator is None: - eprint('Missing generator') + logger.error('Missing generator') return False if len(self.modules) == 0: - eprint('No modules added') + logger.error('No modules added') return False if len(self.output_order) != len(self.modules): - eprint('Number of outputs does not match number of modules') + logger.error('Number of outputs does not match number of modules') return False return True @@ -183,7 +184,7 @@ class Tuner(object): for module in self.modules: if not module.validate_config(self.config): - eprint(f'Config is invalid for module {module.type}') + logger.error(f'Config is invalid for module {module.type}') return -1 has_lsc = any(isinstance(m, lt.modules.lsc.LSC) for m in self.modules) @@ -192,14 +193,14 @@ class Tuner(object): images = utils.load_images(args.input, self.config, not has_only_lsc, has_lsc) if images is None or len(images) == 0: - eprint(f'No images were found, or able to load') + logger.error(f'No images were found, or able to load') return -1 # Do the tuning for module in self.modules: out = module.process(self.config, images, self.output) if out is None: - eprint(f'Module {module.name} failed to process, aborting') + logger.error(f'Module {module.hr_name} failed to process...') break self.output[module] = out diff --git a/utils/tuning/libtuning/macbeth.py b/utils/tuning/libtuning/macbeth.py index 265a33d68378..28051de8155c 100644 --- a/utils/tuning/libtuning/macbeth.py +++ b/utils/tuning/libtuning/macbeth.py @@ -13,12 +13,15 @@ import os from pathlib import Path import numpy as np import warnings +import logging from sklearn import cluster as cluster from .ctt_ransac import get_square_verts, get_square_centres from libtuning.image import Image +logger = logging.getLogger(__name__) + # Reshape image to fixed width without distorting returns image and scale # factor @@ -374,7 +377,7 @@ def get_macbeth_chart(img, ref_data): # Catch macbeth errors and continue with code except MacbethError as error: - eprint(error) + logger.warning(error) return (0, None, None, False) @@ -497,7 +500,7 @@ def find_macbeth(img, mac_config): coords_fit = coords if cor < 0.75: - eprint(f'Warning: Low confidence {cor:.3f} for macbeth chart in {img.path.name}') + logger.warning(f'Low confidence {cor:.3f} for macbeth chart') if show: draw_macbeth_results(img, coords_fit) @@ -510,18 +513,18 @@ def locate_macbeth(image: Image, config: dict): av_chan = (np.mean(np.array(image.channels), axis=0) / (2**16)) av_val = np.mean(av_chan) if av_val < image.blacklevel_16 / (2**16) + 1 / 64: - eprint(f'Image {image.path.name} too dark') + logger.warning(f'Image {image.path.name} too dark') return None macbeth = find_macbeth(av_chan, config['general']['macbeth']) if macbeth is None: - eprint(f'No macbeth chart found in {image.path.name}') + logger.warning(f'No macbeth chart found in {image.path.name}') return None mac_cen_coords = macbeth[1] if not image.get_patches(mac_cen_coords): - eprint(f'Macbeth patches have saturated in {image.path.name}') + logger.warning(f'Macbeth patches have saturated in {image.path.name}') return None return macbeth diff --git a/utils/tuning/libtuning/modules/lsc/raspberrypi.py b/utils/tuning/libtuning/modules/lsc/raspberrypi.py index f19c71637b89..99bc4fe6e07f 100644 --- a/utils/tuning/libtuning/modules/lsc/raspberrypi.py +++ b/utils/tuning/libtuning/modules/lsc/raspberrypi.py @@ -12,7 +12,9 @@ import libtuning.utils as utils from numbers import Number import numpy as np +import logging +logger = logging.getLogger(__name__) class ALSCRaspberryPi(LSC): # Override the type name so that the parser can match the entry in the @@ -35,7 +37,7 @@ class ALSCRaspberryPi(LSC): def validate_config(self, config: dict) -> bool: if self not in config: - utils.eprint(f'{self.type} not in config') + logger.error(f'{self.type} not in config') return False valid = True @@ -46,14 +48,14 @@ class ALSCRaspberryPi(LSC): color_key = self.do_color.name if lum_key not in conf and self.luminance_strength.required: - utils.eprint(f'{lum_key} is not in config') + logger.error(f'{lum_key} is not in config') valid = False if lum_key in conf and (conf[lum_key] < 0 or conf[lum_key] > 1): - utils.eprint(f'Warning: {lum_key} is not in range [0, 1]; defaulting to 0.5') + logger.warning(f'{lum_key} is not in range [0, 1]; defaulting to 0.5') if color_key not in conf and self.do_color.required: - utils.eprint(f'{color_key} is not in config') + logger.error(f'{color_key} is not in config') valid = False return valid @@ -235,7 +237,7 @@ class ALSCRaspberryPi(LSC): if count == 1: output['sigma'] = 0.005 output['sigma_Cb'] = 0.005 - utils.eprint('Warning: Only one alsc calibration found; standard sigmas used for adaptive algorithm.') + logger.warning('Only one alsc calibration found; standard sigmas used for adaptive algorithm.') return output # Obtain worst-case scenario residual sigmas diff --git a/utils/tuning/libtuning/utils.py b/utils/tuning/libtuning/utils.py index f099c0ed685c..872341407b7b 100644 --- a/utils/tuning/libtuning/utils.py +++ b/utils/tuning/libtuning/utils.py @@ -12,16 +12,15 @@ import os from pathlib import Path import re import sys +import logging import libtuning as lt from libtuning.image import Image from libtuning.macbeth import locate_macbeth -# Utility functions - +logger = logging.getLogger(__name__) -def eprint(*args, **kwargs): - print(*args, file=sys.stderr, **kwargs) +# Utility functions def get_module_by_type_name(modules, name): @@ -45,7 +44,7 @@ def _list_image_files(directory): def _parse_image_filename(fn: Path): result = re.search(r'^(alsc_)?(\d+)[kK]_(\d+)?[lLuU]?.\w{3,4}$', fn.name) if result is None: - eprint(f'The file name of {fn.name} is incorrectly formatted') + logger.error(f'The file name of {fn.name} is incorrectly formatted') return None, None, None color = int(result.group(2)) @@ -72,7 +71,7 @@ def _validate_images(images): def load_images(input_dir: str, config: dict, load_nonlsc: bool, load_lsc: bool) -> list: files = _list_image_files(input_dir) if len(files) == 0: - eprint(f'No images found in {input_dir}') + logger.error(f'No images found in {input_dir}') return None images = [] @@ -83,19 +82,19 @@ def load_images(input_dir: str, config: dict, load_nonlsc: bool, load_lsc: bool) # Skip lsc image if we don't need it if lsc_only and not load_lsc: - eprint(f'Skipping {f.name} as this tuner has no LSC module') + logger.warning(f'Skipping {f.name} as this tuner has no LSC module') continue # Skip non-lsc image if we don't need it if not lsc_only and not load_nonlsc: - eprint(f'Skipping {f.name} as this tuner only has an LSC module') + logger.warning(f'Skipping {f.name} as this tuner only has an LSC module') continue # Load image try: image = Image(f) except Exception as e: - eprint(f'Failed to load image {f.name}: {e}') + logger.error(f'Failed to load image {f.name}: {e}') continue # Populate simple fields diff --git a/utils/tuning/requirements.txt b/utils/tuning/requirements.txt index d1dc589d0329..c3c20cee1263 100644 --- a/utils/tuning/requirements.txt +++ b/utils/tuning/requirements.txt @@ -1,3 +1,4 @@ +coloredlogs numpy opencv-python py3exiv2 diff --git a/utils/tuning/rkisp1.py b/utils/tuning/rkisp1.py index d0ce15d5ed7a..2606e07a93f7 100755 --- a/utils/tuning/rkisp1.py +++ b/utils/tuning/rkisp1.py @@ -5,6 +5,8 @@ # # Tuning script for rkisp1 +import coloredlogs +import logging import sys import libtuning as lt @@ -13,6 +15,9 @@ from libtuning.generators import YamlOutput from libtuning.modules.lsc import LSCRkISP1 from libtuning.modules.agc import AGCRkISP1 + +coloredlogs.install(level=logging.INFO, fmt='%(name)s %(levelname)s %(message)s') + tuner = lt.Tuner('RkISP1') tuner.add(LSCRkISP1( debug=[lt.Debug.Plot], From patchwork Wed Jul 3 14:16:57 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefan Klug X-Patchwork-Id: 20522 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 40CD2BEFBE for ; Wed, 3 Jul 2024 14:17:59 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id E47CF63339; Wed, 3 Jul 2024 16:17:58 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="PghGND7c"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id B1D2063339 for ; Wed, 3 Jul 2024 16:17:55 +0200 (CEST) Received: from ideasonboard.com (unknown [IPv6:2a00:6020:448c:6c00:9263:c199:9587:576]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 961C44CA; Wed, 3 Jul 2024 16:17:27 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1720016247; bh=dTqRfHvl3XMGum+nbZJgo4z99tFr+h4U19miy1r5r2o=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=PghGND7c5ggRl/1MRwQPng6uWxG+Wqm+QljKqIjVL2j/FNZ5XmL2thSKG7ZJH6eUV Dd8Q3T0qagrGiZyjnZZakqBtKOAWJL4MuuuUq6uQFlztTJTClkKEY/QDd0bMjjwdqC n0iVt/HURjidzg2/kYeVdjkUdyjrkJNsITjM408Q= From: Stefan Klug To: libcamera-devel@lists.libcamera.org Cc: Stefan Klug , Paul Elder , Laurent Pinchart Subject: [PATCH v3 08/23] libtuning: Fix visualize_macbeth_chart() Date: Wed, 3 Jul 2024 16:16:57 +0200 Message-ID: <20240703141726.252368-9-stefan.klug@ideasonboard.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240703141726.252368-1-stefan.klug@ideasonboard.com> References: <20240703141726.252368-1-stefan.klug@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" The old function uses PIL to save the image, which is not in the requirements file. As we are already requiring opencv, use that to save images instead of an additional dependency Signed-off-by: Stefan Klug Reviewed-by: Paul Elder Reviewed-by: Laurent Pinchart --- utils/tuning/libtuning/utils.py | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/utils/tuning/libtuning/utils.py b/utils/tuning/libtuning/utils.py index 872341407b7b..90fd7072a0fd 100644 --- a/utils/tuning/libtuning/utils.py +++ b/utils/tuning/libtuning/utils.py @@ -5,6 +5,7 @@ # # Utilities for libtuning +import cv2 import decimal import math import numpy as np @@ -162,6 +163,6 @@ def visualise_macbeth_chart(macbeth_rgb, original_rgb, new_rgb, output_filename) for g in range(100): image[xlocation + i, ylocation + g] = new_rgb[colorindex] - img = Image.fromarray(image, 'RGB') - img.save(str(output_filename) + 'Generated Macbeth Chart.png') + im_bgr = cv2.cvtColor(image, cv2.COLOR_RGB2BGR) + cv2.imwrite(f'{output_filename} Generated Macbeth Chart.png', im_bgr) From patchwork Wed Jul 3 14:16:58 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefan Klug X-Patchwork-Id: 20523 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 60D2CBEFBE for ; Wed, 3 Jul 2024 14:18:02 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id EF38763352; Wed, 3 Jul 2024 16:18:01 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="H47AgwEv"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 59DCE63362 for ; Wed, 3 Jul 2024 16:17:59 +0200 (CEST) Received: from ideasonboard.com (unknown [IPv6:2a00:6020:448c:6c00:9263:c199:9587:576]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 33A994CA; Wed, 3 Jul 2024 16:17:31 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1720016251; bh=YivJPVzYaTF92WKi8YwCCPOvj6rqc8S0NZ+QdhsD7J8=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=H47AgwEv412/6fXxcUzwNFWL1lzw4dh5mfdaHQFsHRK+8zSp8I7Q1ESy99oPQEElI GPzbW9h8F0FhLeDL3ZYPjSb5pC6Nq8qm2/PlPHvpLYKGobtNzSYwtrj2WDIV9pwru4 kEZSZt/X0nj1chH2JjGekBeXiRjwN0Eo/sSL9aeI= From: Stefan Klug To: libcamera-devel@lists.libcamera.org Cc: Stefan Klug , Paul Elder Subject: [PATCH v3 09/23] libtuning: Improve filename parsing Date: Wed, 3 Jul 2024 16:16:58 +0200 Message-ID: <20240703141726.252368-10-stefan.klug@ideasonboard.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240703141726.252368-1-stefan.klug@ideasonboard.com> References: <20240703141726.252368-1-stefan.klug@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" In the tuning datasets, the files had names like 'imx335_1600l_3000k_1.dng'. That failed on the old filename parsing function. As there is no need to dictate the order of the tags, split the big regex into chunks and parse them one by one. This also makes the code easier to digest. Signed-off-by: Stefan Klug Reviewed-by: Paul Elder Reviewed-by: Laurent Pinchart --- utils/tuning/libtuning/utils.py | 30 ++++++++++++++++++++++-------- 1 file changed, 22 insertions(+), 8 deletions(-) diff --git a/utils/tuning/libtuning/utils.py b/utils/tuning/libtuning/utils.py index 90fd7072a0fd..c70dfae049bc 100644 --- a/utils/tuning/libtuning/utils.py +++ b/utils/tuning/libtuning/utils.py @@ -43,16 +43,30 @@ def _list_image_files(directory): def _parse_image_filename(fn: Path): - result = re.search(r'^(alsc_)?(\d+)[kK]_(\d+)?[lLuU]?.\w{3,4}$', fn.name) - if result is None: - logger.error(f'The file name of {fn.name} is incorrectly formatted') - return None, None, None + lsc_only = False + color_temperature = None + lux = None + + parts = fn.stem.split('_') + for part in parts: + if part == 'alsc': + lsc_only = True + continue + r = re.match(r'(\d+)[kK]', part) + if r: + color_temperature = int(r.group(1)) + continue + r = re.match(r'(\d+)[lLuU]', part) + if r: + lux = int(r.group(1)) + + if color_temperature is None: + logger.error(f'The file name of "{fn.name}" does not contain a color temperature') - color = int(result.group(2)) - lsc_only = result.group(1) is not None - lux = None if lsc_only else int(result.group(3)) + if lux is None and lsc_only is False: + logger.error(f'The file name of "{fn.name}" must either contain alsc or a lux level') - return color, lux, lsc_only + return color_temperature, lux, lsc_only # \todo Implement this from check_imgs() in ctt.py From patchwork Wed Jul 3 14:16:59 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefan Klug X-Patchwork-Id: 20524 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 50B6FBEFBE for ; Wed, 3 Jul 2024 14:18:05 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 0D42D63364; Wed, 3 Jul 2024 16:18:05 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="wmWT4yRC"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 4934163363 for ; Wed, 3 Jul 2024 16:18:02 +0200 (CEST) Received: from ideasonboard.com (unknown [IPv6:2a00:6020:448c:6c00:9263:c199:9587:576]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id E1EF24CA; Wed, 3 Jul 2024 16:17:33 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1720016254; bh=QLzmsrZJD4p2yyDrdaopL/53SEkuwl5v9CLDaSjQzWY=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=wmWT4yRCEvZGHjLwNM12fJO1Nxr+xg2K3mnANU/u4YUexTx/Lb9YntxIqx96rSZDW x6BF2g736Plx3nRRbAUUL01GcIZOhFxi1dQ6XW1Dkp/v5IReHIK9IuGSDjjRZUiBnL LhzJ7cKaOGdjerkzjhusjx332tPSIt6K84a+jRu0= From: Stefan Klug To: libcamera-devel@lists.libcamera.org Cc: Stefan Klug , Paul Elder , Laurent Pinchart Subject: [PATCH v3 10/23] libtuning: Implement a minimal yaml parser Date: Wed, 3 Jul 2024 16:16:59 +0200 Message-ID: <20240703141726.252368-11-stefan.klug@ideasonboard.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240703141726.252368-1-stefan.klug@ideasonboard.com> References: <20240703141726.252368-1-stefan.klug@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" At the moment this just reads the yaml file and returns it verbatim. This needs to evolve further in the near future. Signed-off-by: Stefan Klug Reviewed-by: Paul Elder Reviewed-by: Laurent Pinchart --- utils/tuning/config-example.yaml | 12 ++++++++++++ utils/tuning/libtuning/parsers/yaml_parser.py | 9 ++++++--- 2 files changed, 18 insertions(+), 3 deletions(-) create mode 100644 utils/tuning/config-example.yaml diff --git a/utils/tuning/config-example.yaml b/utils/tuning/config-example.yaml new file mode 100644 index 000000000000..1b7f52cd2fff --- /dev/null +++ b/utils/tuning/config-example.yaml @@ -0,0 +1,12 @@ +general: + disable: [] + plot: [] + alsc: + do_alsc_colour: 1 + luminance_strength: 0.5 + awb: + greyworld: 0 + macbeth: + small: 1 + show: 0 +# blacklevel: 32 \ No newline at end of file diff --git a/utils/tuning/libtuning/parsers/yaml_parser.py b/utils/tuning/libtuning/parsers/yaml_parser.py index 244db24daeb1..1fa6b7a86ddd 100644 --- a/utils/tuning/libtuning/parsers/yaml_parser.py +++ b/utils/tuning/libtuning/parsers/yaml_parser.py @@ -5,13 +5,16 @@ # Parser for YAML format config file from .parser import Parser +import yaml class YamlParser(Parser): def __init__(self): super().__init__() - # \todo Implement this (it's fine for now as we don't need a config for - # rkisp1 LSC, which is the only user of this so far) def parse(self, config_file: str, modules: list) -> (dict, list): - return {}, [] + # Dummy implementation that just reads the file + with open(config_file, 'r') as f: + config = yaml.safe_load(f) + + return config, [] From patchwork Wed Jul 3 14:17:00 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefan Klug X-Patchwork-Id: 20525 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id A1709BEFBE for ; Wed, 3 Jul 2024 14:18:08 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 5901763363; Wed, 3 Jul 2024 16:18:08 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="i2W1EZPQ"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 2445F63365 for ; Wed, 3 Jul 2024 16:18:05 +0200 (CEST) Received: from ideasonboard.com (unknown [IPv6:2a00:6020:448c:6c00:9263:c199:9587:576]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 0532F4CA; Wed, 3 Jul 2024 16:17:36 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1720016257; bh=WDvuEgRZJSn8ft56e1J+vD0dxPn56D/Aap9b1kKGZWM=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=i2W1EZPQfW8OFdxnwJvlHebWeQ7Gno/i8CX1HYnUTGTgj3qcE8J2LlWVKmt1tkP1b fDACb0Ycq4aB6PI+2s00gDBBzebJy1w3ObUSNq9lUe51xijmHMO34ZOEED/lX0L58B JDlAxiQQao0cXk7JdcdLif24VSGQfoWWDQ7frDis= From: Stefan Klug To: libcamera-devel@lists.libcamera.org Cc: Stefan Klug Subject: [PATCH v3 11/23] libtuning: Reactivate macbeth locator Date: Wed, 3 Jul 2024 16:17:00 +0200 Message-ID: <20240703141726.252368-12-stefan.klug@ideasonboard.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240703141726.252368-1-stefan.klug@ideasonboard.com> References: <20240703141726.252368-1-stefan.klug@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Add the missing pieces and store the result inside the image object. Signed-off-by: Stefan Klug Reviewed-by: Paul Elder --- utils/tuning/libtuning/image.py | 1 + utils/tuning/libtuning/macbeth.py | 13 ++++++++++--- utils/tuning/libtuning/utils.py | 4 ++-- 3 files changed, 13 insertions(+), 5 deletions(-) diff --git a/utils/tuning/libtuning/image.py b/utils/tuning/libtuning/image.py index 2c4d774f11e2..c8911a0ff125 100644 --- a/utils/tuning/libtuning/image.py +++ b/utils/tuning/libtuning/image.py @@ -24,6 +24,7 @@ class Image: self.lsc_only = False self.color = -1 self.lux = -1 + self.macbeth = None try: self._load_metadata_exif() diff --git a/utils/tuning/libtuning/macbeth.py b/utils/tuning/libtuning/macbeth.py index 28051de8155c..4a2006b013dc 100644 --- a/utils/tuning/libtuning/macbeth.py +++ b/utils/tuning/libtuning/macbeth.py @@ -17,12 +17,15 @@ import logging from sklearn import cluster as cluster from .ctt_ransac import get_square_verts, get_square_centres - -from libtuning.image import Image +from .image import Image logger = logging.getLogger(__name__) +class MacbethError(Exception): + pass + + # Reshape image to fixed width without distorting returns image and scale # factor def reshape(img, width): @@ -377,7 +380,9 @@ def get_macbeth_chart(img, ref_data): # Catch macbeth errors and continue with code except MacbethError as error: - logger.warning(error) + # \todo: This happens so many times in a normal run, that it shadows + # all the relevant output + # logger.warning(error) return (0, None, None, False) @@ -527,4 +532,6 @@ def locate_macbeth(image: Image, config: dict): logger.warning(f'Macbeth patches have saturated in {image.path.name}') return None + image.macbeth = macbeth + return macbeth diff --git a/utils/tuning/libtuning/utils.py b/utils/tuning/libtuning/utils.py index c70dfae049bc..93c6c94ea2b0 100644 --- a/utils/tuning/libtuning/utils.py +++ b/utils/tuning/libtuning/utils.py @@ -17,7 +17,7 @@ import logging import libtuning as lt from libtuning.image import Image -from libtuning.macbeth import locate_macbeth +from .macbeth import locate_macbeth logger = logging.getLogger(__name__) @@ -127,7 +127,7 @@ def load_images(input_dir: str, config: dict, load_nonlsc: bool, load_lsc: bool) continue # Handle macbeth - macbeth = locate_macbeth(config) + macbeth = locate_macbeth(image, config) if macbeth is None: continue From patchwork Wed Jul 3 14:17:01 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefan Klug X-Patchwork-Id: 20526 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 636DFBEFBE for ; Wed, 3 Jul 2024 14:18:11 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 142FA63363; Wed, 3 Jul 2024 16:18:11 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="JZuWiqea"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 041A062C95 for ; Wed, 3 Jul 2024 16:18:08 +0200 (CEST) Received: from ideasonboard.com (unknown [IPv6:2a00:6020:448c:6c00:9263:c199:9587:576]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id D9FBF4CA; Wed, 3 Jul 2024 16:17:39 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1720016260; bh=qT3ViRvNBToMyvzzVKMUNpQKz9K4yfyXl3UI08KAXIo=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=JZuWiqeaXWzSSyZIhixOd3GsszP7O7B/N/jRuJGrWvSiR9PIYJkPsmoJzVxPEfw0u G40pgHKxAjYUd+UwWld2FLFgW96sJDwQrCDKy98EX/ioDOri4orYQEtQE1VlXSvkho wYWdTp7bjKJJPQ6bvkPp4VJJfdr4iHLhUtFOyuGo= From: Stefan Klug To: libcamera-devel@lists.libcamera.org Cc: Stefan Klug Subject: [PATCH v3 12/23] libtuning: Be a bit more verbose Date: Wed, 3 Jul 2024 16:17:01 +0200 Message-ID: <20240703141726.252368-13-stefan.klug@ideasonboard.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240703141726.252368-1-stefan.klug@ideasonboard.com> References: <20240703141726.252368-1-stefan.klug@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Print a info on every image that gets processed and a warning on every image that gets ignored. Signed-off-by: Stefan Klug Reviewed-by: Paul Elder --- utils/tuning/libtuning/utils.py | 2 ++ 1 file changed, 2 insertions(+) diff --git a/utils/tuning/libtuning/utils.py b/utils/tuning/libtuning/utils.py index 93c6c94ea2b0..dacda11312a2 100644 --- a/utils/tuning/libtuning/utils.py +++ b/utils/tuning/libtuning/utils.py @@ -92,7 +92,9 @@ def load_images(input_dir: str, config: dict, load_nonlsc: bool, load_lsc: bool) images = [] for f in files: color, lux, lsc_only = _parse_image_filename(f) + logger.info(f'Process image "{f.name}" (color={color}, lux={lux}, lsc_only={lsc_only})') if color is None: + logger.warning(f'Ignoring "{f.name}" as it has no associated color temperature') continue # Skip lsc image if we don't need it From patchwork Wed Jul 3 14:17:02 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefan Klug X-Patchwork-Id: 20527 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 8AE3CBEFBE for ; Wed, 3 Jul 2024 14:18:15 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 2CCA96336B; Wed, 3 Jul 2024 16:18:15 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="F9EA/xom"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id A75B863352 for ; Wed, 3 Jul 2024 16:18:11 +0200 (CEST) Received: from ideasonboard.com (unknown [IPv6:2a00:6020:448c:6c00:9263:c199:9587:576]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 7B6E74CA; Wed, 3 Jul 2024 16:17:43 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1720016263; bh=cJM1KWSOc4I49q8BvtWNe5IEdJqGhlI/wUssO/a32m0=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=F9EA/xomand5OjRf7Q4ppWQJaaSIvUE4+BfTUnvSLuPbeSBpqYVeg74ktDaK05AwK Q8hLAF1WvvqzdUXu68SdwRYGj02vv0hs81RK4UzoDip2WZiphk1PPUeBTHdythJRrO BLKjMDitYSuKsKzjEx+opM11P2IDQ2ES/X/aqyac= From: Stefan Klug To: libcamera-devel@lists.libcamera.org Cc: Stefan Klug , Laurent Pinchart Subject: [PATCH v3 13/23] libtuning: lsc: rkisp1: Clip lsc values to valid range Date: Wed, 3 Jul 2024 16:17:02 +0200 Message-ID: <20240703141726.252368-14-stefan.klug@ideasonboard.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240703141726.252368-1-stefan.klug@ideasonboard.com> References: <20240703141726.252368-1-stefan.klug@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Based on the input images, the lsc values could exceed the range allowed by the rkisp1. As we are now clipping the values, we can simplify the value mapping. Signed-off-by: Stefan Klug Reviewed-by: Laurent Pinchart Reviewed-by: Paul Elder --- utils/tuning/libtuning/modules/lsc/rkisp1.py | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/utils/tuning/libtuning/modules/lsc/rkisp1.py b/utils/tuning/libtuning/modules/lsc/rkisp1.py index 20406e436a6a..57004104fc7a 100644 --- a/utils/tuning/libtuning/modules/lsc/rkisp1.py +++ b/utils/tuning/libtuning/modules/lsc/rkisp1.py @@ -80,7 +80,8 @@ class LSCRkISP1(LSC): tables = [] for lis in [list_cr, list_cgr, list_cgb, list_cb]: table = np.mean(lis[indices], axis=0) - table = output_map_func((1, 3.999), (1024, 4095), table) + table = output_map_func((1, 4), (1024, 4096), table) + table = np.clip(table, 1024, 4095) table = np.round(table).astype('int32').tolist() tables.append(table) From patchwork Wed Jul 3 14:17:03 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefan Klug X-Patchwork-Id: 20528 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id DAD9BBEFBE for ; Wed, 3 Jul 2024 14:18:19 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 3EF0263368; Wed, 3 Jul 2024 16:18:19 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="P5U/Hlb2"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id C2FF363363 for ; Wed, 3 Jul 2024 16:18:14 +0200 (CEST) Received: from ideasonboard.com (unknown [IPv6:2a00:6020:448c:6c00:9263:c199:9587:576]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 968334CA; Wed, 3 Jul 2024 16:17:46 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1720016266; bh=5MpIYAHJLL4DqpfdN0/TluQv04AANiSdXaGOkaHuJ4I=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=P5U/Hlb2RHlchdtw7N0/txyyOTF+MrIIF+K/dw7rBgu5b3l9q/At2VN1gmOzry3Ej efHf0cpZwR2lXfstem3twNKbhx5o6SoMGOxe0Gj5gjU/3DQZ007thbMrJBHFHberlQ zZrLfxfoHNhudeQB2PRW7scDvGrCXTxNv6LnUvLI= From: Stefan Klug To: libcamera-devel@lists.libcamera.org Cc: Stefan Klug , Laurent Pinchart Subject: [PATCH v3 14/23] libtuning: Use the color member of the Image class Date: Wed, 3 Jul 2024 16:17:03 +0200 Message-ID: <20240703141726.252368-15-stefan.klug@ideasonboard.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240703141726.252368-1-stefan.klug@ideasonboard.com> References: <20240703141726.252368-1-stefan.klug@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" In the Image class the variable holding the color temperature is named color instead of col which was used by the raspberry pi scripts. Rename accordingly. Signed-off-by: Stefan Klug Reviewed-by: Laurent Pinchart Reviewed-by: Paul Elder --- utils/tuning/libtuning/ctt_awb.py | 2 +- utils/tuning/libtuning/ctt_ccm.py | 8 ++++---- 2 files changed, 5 insertions(+), 5 deletions(-) diff --git a/utils/tuning/libtuning/ctt_awb.py b/utils/tuning/libtuning/ctt_awb.py index f3a1ce779e21..abf22321a0ea 100644 --- a/utils/tuning/libtuning/ctt_awb.py +++ b/utils/tuning/libtuning/ctt_awb.py @@ -313,7 +313,7 @@ def get_alsc_patches(Img, colour_cals, grey=True): g_patchs = (patches[1][3::4]+patches[2][3::4])/2 - Img.blacklevel_16 else: cen_coords = Img.cen_coords - col = Img.col + col = Img.color patches = [np.array(Img.patches[i]) for i in Img.order] r_patchs = patches[0] - Img.blacklevel_16 b_patchs = patches[3] - Img.blacklevel_16 diff --git a/utils/tuning/libtuning/ctt_ccm.py b/utils/tuning/libtuning/ctt_ccm.py index c4362756c3c0..086ef8a2580b 100644 --- a/utils/tuning/libtuning/ctt_ccm.py +++ b/utils/tuning/libtuning/ctt_ccm.py @@ -249,7 +249,7 @@ def ccm(Cam, cal_cr_list, cal_cb_list): logger.info(f'delta E optimized: average: {after_average:.2f} max:{new_worst_delta_e:.2f}') logger.info(f'delta E old: average: {before_average:.2f} max:{old_worst_delta_e:.2f}') - visualise_macbeth_chart(m_rgb, optimised_ccm_rgb, after_gamma_rgb, str(Img.col) + str(matrix_selection_types[typenum])) + visualise_macbeth_chart(m_rgb, optimised_ccm_rgb, after_gamma_rgb, str(Img.color) + str(matrix_selection_types[typenum])) ''' The program will also save some visualisations of improvements. Very pretty to look at. Top rectangle is ideal, Left square is @@ -260,10 +260,10 @@ def ccm(Cam, cal_cr_list, cal_cb_list): if a ccm has already been calculated for that temperature then don't overwrite but save both. They will then be averaged later on """ # Now going to use optimised color matrix, optimised_ccm - if Img.col in ccm_tab.keys(): - ccm_tab[Img.col].append(optimised_ccm) + if Img.color in ccm_tab.keys(): + ccm_tab[Img.color].append(optimised_ccm) else: - ccm_tab[Img.col] = [optimised_ccm] + ccm_tab[Img.color] = [optimised_ccm] logger.info('Finished processing images') """ From patchwork Wed Jul 3 14:17:04 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefan Klug X-Patchwork-Id: 20529 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id EAFA0BEFBE for ; Wed, 3 Jul 2024 14:18:21 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 96D6A6336F; Wed, 3 Jul 2024 16:18:21 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="HpKweF+n"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id A13F063352 for ; Wed, 3 Jul 2024 16:18:17 +0200 (CEST) Received: from ideasonboard.com (unknown [IPv6:2a00:6020:448c:6c00:9263:c199:9587:576]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 7ABC66D6; Wed, 3 Jul 2024 16:17:49 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1720016269; bh=azdBtdduYJjovqhTZPgLSeoMrz8G7YyWb+h7GkdtzVw=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=HpKweF+njqSYaVszR4wnQPiUD2q5dIU/SHgKVe2htbgTF/rbY6HCgiEEr15uCJrhT hwQHFbFYkIOZTr81Cv7JBfzT+RZ3jsE0witUxaUXxsh0Hn6rMUjuk30Uh2Fh6PH4Z1 DrKGO3PBKgw3f6yCJYpm1MZh4BPdJFG8woj9fBqg= From: Stefan Klug To: libcamera-devel@lists.libcamera.org Cc: Stefan Klug , Laurent Pinchart Subject: [PATCH v3 15/23] libtuning: Remove need for Cam object from ccm Date: Wed, 3 Jul 2024 16:17:04 +0200 Message-ID: <20240703141726.252368-16-stefan.klug@ideasonboard.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240703141726.252368-1-stefan.klug@ideasonboard.com> References: <20240703141726.252368-1-stefan.klug@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Remove the need for the Cam object, as we don't want to port it from Raspberry Pi. Signed-off-by: Stefan Klug Reviewed-by: Laurent Pinchart Reviewed-by: Paul Elder --- utils/tuning/libtuning/ctt_ccm.py | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/utils/tuning/libtuning/ctt_ccm.py b/utils/tuning/libtuning/ctt_ccm.py index 086ef8a2580b..2e87a6679f20 100644 --- a/utils/tuning/libtuning/ctt_ccm.py +++ b/utils/tuning/libtuning/ctt_ccm.py @@ -62,9 +62,8 @@ FInds colour correction matrices for list of images """ -def ccm(Cam, cal_cr_list, cal_cb_list): +def ccm(imgs, cal_cr_list, cal_cb_list): global matrix_selection_types, typenum - imgs = Cam.imgs """ standard macbeth chart colour values """ From patchwork Wed Jul 3 14:17:05 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefan Klug X-Patchwork-Id: 20530 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id AEC99BEFBE for ; Wed, 3 Jul 2024 14:18:24 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 57B4C6336F; Wed, 3 Jul 2024 16:18:24 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="CUxq2DE2"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id E05B263363 for ; Wed, 3 Jul 2024 16:18:19 +0200 (CEST) Received: from ideasonboard.com (unknown [IPv6:2a00:6020:448c:6c00:9263:c199:9587:576]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id BF2D74CA; Wed, 3 Jul 2024 16:17:51 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1720016271; bh=R3xauIACzw6NgLS3ELw+s6R+znhK+BDcDxiDTY+v3Ao=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=CUxq2DE2Vonip+otwci0GqlBsZMpWpb6klGxwjMph6sGi4iOk4h90CfJl82A81eF5 754r9gOlziHJpJvlPKE80qu0R6ElJ8xIlQr/4BpH4O4iAKSIEa6HEHeKDjBsXQq/rD pjDTGGp4P6Edll1M/aRee6h2C1rfgQX20+VJs2Rc= From: Stefan Klug To: libcamera-devel@lists.libcamera.org Cc: Paul Elder , Stefan Klug , Laurent Pinchart Subject: [PATCH v3 16/23] libtuning: modules: Add initial CCM module Date: Wed, 3 Jul 2024 16:17:05 +0200 Message-ID: <20240703141726.252368-17-stefan.klug@ideasonboard.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240703141726.252368-1-stefan.klug@ideasonboard.com> References: <20240703141726.252368-1-stefan.klug@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" From: Paul Elder Implement a minimal ccm calibration module. For now it doesn't take the results from lsc into account and supports rkisp1 only. Signed-off-by: Paul Elder Signed-off-by: Stefan Klug Reviewed-by: Laurent Pinchart --- .../tuning/libtuning/modules/ccm/__init__.py | 6 +++ utils/tuning/libtuning/modules/ccm/ccm.py | 41 +++++++++++++++++++ utils/tuning/libtuning/modules/ccm/rkisp1.py | 27 ++++++++++++ utils/tuning/rkisp1.py | 4 +- 4 files changed, 77 insertions(+), 1 deletion(-) create mode 100644 utils/tuning/libtuning/modules/ccm/__init__.py create mode 100644 utils/tuning/libtuning/modules/ccm/ccm.py create mode 100644 utils/tuning/libtuning/modules/ccm/rkisp1.py diff --git a/utils/tuning/libtuning/modules/ccm/__init__.py b/utils/tuning/libtuning/modules/ccm/__init__.py new file mode 100644 index 000000000000..322602afe63b --- /dev/null +++ b/utils/tuning/libtuning/modules/ccm/__init__.py @@ -0,0 +1,6 @@ +# SPDX-License-Identifier: GPL-2.0-or-later +# +# Copyright (C) 2024, Paul Elder + +from libtuning.modules.ccm.ccm import CCM +from libtuning.modules.ccm.rkisp1 import CCMRkISP1 diff --git a/utils/tuning/libtuning/modules/ccm/ccm.py b/utils/tuning/libtuning/modules/ccm/ccm.py new file mode 100644 index 000000000000..ba433404f069 --- /dev/null +++ b/utils/tuning/libtuning/modules/ccm/ccm.py @@ -0,0 +1,41 @@ +# SPDX-License-Identifier: GPL-2.0-or-later +# +# Copyright (C) 2024, Paul Elder +# Copyright (C) 2024, Ideas on Board +# +# Base Ccm tuning module + +from ..module import Module + +from libtuning.ctt_ccm import ccm +import logging + +logger = logging.getLogger(__name__) + + +class CCM(Module): + type = 'ccm' + hr_name = 'CCM (Base)' + out_name = 'GenericCCM' + + def __init__(self, debug: list): + super().__init__() + + self.debug = debug + + def do_calibration(self, images): + logger.info('Starting CCM calibration') + + imgs = [img for img in images if img.macbeth is not None] + + # todo: Take LSC calibration results into account. + cal_cr_list = None + cal_cb_list = None + + try: + ccms = ccm(imgs, cal_cr_list, cal_cb_list) + except ArithmeticError: + logger.error('CCM calibration failed') + return 1 + + return ccms diff --git a/utils/tuning/libtuning/modules/ccm/rkisp1.py b/utils/tuning/libtuning/modules/ccm/rkisp1.py new file mode 100644 index 000000000000..00381abe2893 --- /dev/null +++ b/utils/tuning/libtuning/modules/ccm/rkisp1.py @@ -0,0 +1,27 @@ +# SPDX-License-Identifier: GPL-2.0-or-later +# +# Copyright (C) 2024, Paul Elder +# Copyright (C) 2024, Ideas on Board +# +# Ccm module for tuning rkisp1 + +from .ccm import CCM + +class CCMRkISP1(CCM): + hr_name = 'Crosstalk Correction (RkISP1)' + out_name = 'Ccm' + + def __init__(self, **kwargs): + super().__init__(**kwargs) + + # We don't need anything from the config file. + def validate_config(self, config: dict) -> bool: + return True + + def process(self, config: dict, images: list, outputs: dict) -> dict: + output = {} + + ccms = self.do_calibration(images) + output['ccms'] = ccms + + return output diff --git a/utils/tuning/rkisp1.py b/utils/tuning/rkisp1.py index 2606e07a93f7..fae35783c656 100755 --- a/utils/tuning/rkisp1.py +++ b/utils/tuning/rkisp1.py @@ -14,6 +14,7 @@ from libtuning.parsers import YamlParser from libtuning.generators import YamlOutput from libtuning.modules.lsc import LSCRkISP1 from libtuning.modules.agc import AGCRkISP1 +from libtuning.modules.ccm import CCMRkISP1 coloredlogs.install(level=logging.INFO, fmt='%(name)s %(levelname)s %(message)s') @@ -39,9 +40,10 @@ tuner.add(LSCRkISP1( smoothing_function=lt.smoothing.MedianBlur(3), )) tuner.add(AGCRkISP1(debug=[lt.Debug.Plot])) +tuner.add(CCMRkISP1(debug=[lt.Debug.Plot])) tuner.set_input_parser(YamlParser()) tuner.set_output_formatter(YamlOutput()) -tuner.set_output_order([AGCRkISP1, LSCRkISP1]) +tuner.set_output_order([AGCRkISP1, CCMRkISP1, LSCRkISP1]) if __name__ == '__main__': sys.exit(tuner.run(sys.argv)) From patchwork Wed Jul 3 14:17:06 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefan Klug X-Patchwork-Id: 20531 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 8152DBEFBE for ; Wed, 3 Jul 2024 14:18:26 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 2CF8063370; Wed, 3 Jul 2024 16:18:26 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="pJrx6Res"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 849E463363 for ; Wed, 3 Jul 2024 16:18:23 +0200 (CEST) Received: from ideasonboard.com (unknown [IPv6:2a00:6020:448c:6c00:9263:c199:9587:576]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 5A6B06D6; Wed, 3 Jul 2024 16:17:55 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1720016275; bh=f/QNn7n07AHjXdqNKS3gVvVHJg0cEDp4fJUPiszmD48=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=pJrx6ResC68H1VJTst9UXlER1zqVUfQaxg8iFetUSWIm6ppdk05C1aRfdu4Mn/oph Ntr1QwX7gmL0ZLdU3CmmLAU15o6Gswqu3Wbv0rZ1pntzbYUrpd9aLBTTUMEP+PwZ+O rHsq+KSAG9CySvE0Xack/iiLq+IZiWDQUXIowUxo= From: Stefan Klug To: libcamera-devel@lists.libcamera.org Cc: Stefan Klug , Laurent Pinchart Subject: [PATCH v3 17/23] libtuning: Handle cases, where no lsc tuning images are present Date: Wed, 3 Jul 2024 16:17:06 +0200 Message-ID: <20240703141726.252368-18-stefan.klug@ideasonboard.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240703141726.252368-1-stefan.klug@ideasonboard.com> References: <20240703141726.252368-1-stefan.klug@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Make it clear that no lsc calibration was done by returning None instead of a incomplete configuration. Signed-off-by: Stefan Klug Reviewed-by: Laurent Pinchart Reviewed-by: Paul Elder --- utils/tuning/libtuning/modules/lsc/rkisp1.py | 3 +++ 1 file changed, 3 insertions(+) diff --git a/utils/tuning/libtuning/modules/lsc/rkisp1.py b/utils/tuning/libtuning/modules/lsc/rkisp1.py index 57004104fc7a..512233aeae9d 100644 --- a/utils/tuning/libtuning/modules/lsc/rkisp1.py +++ b/utils/tuning/libtuning/modules/lsc/rkisp1.py @@ -107,6 +107,9 @@ class LSCRkISP1(LSC): output['sets'] = self._do_all_lsc(images) + if len(output['sets']) == 0: + return None + # \todo Validate images from greyscale camera and force grescale mode # \todo Debug functionality From patchwork Wed Jul 3 14:17:07 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefan Klug X-Patchwork-Id: 20532 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 52862BEFBE for ; Wed, 3 Jul 2024 14:18:29 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 01D1563372; Wed, 3 Jul 2024 16:18:29 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="EM/gHc98"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 9EE986336E for ; Wed, 3 Jul 2024 16:18:26 +0200 (CEST) Received: from ideasonboard.com (unknown [IPv6:2a00:6020:448c:6c00:9263:c199:9587:576]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 7F4684CA; Wed, 3 Jul 2024 16:17:58 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1720016278; bh=K33EG7elE+JgYUBh7qjIArnf9ym+nG5nGPpBVymhUbA=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=EM/gHc98/Zucg3wPhr09xjc5yUKKZ4mgxb2Tku1MGKMBG5ZQ7m2TcnJYCIpZ0Lxww e38huvzq+CbW0UG21DzuRllUzxrt2zgatoD1mtM8uomW45sA9anjIV8hL6Mq6AwdIr mW1DeWwOZOw3g8vTp8SCscBZeOwi8mvi6fRmFVLI= From: Stefan Klug To: libcamera-devel@lists.libcamera.org Cc: Stefan Klug , Laurent Pinchart Subject: [PATCH v3 18/23] libtuning: Only warn if processing returns None Date: Wed, 3 Jul 2024 16:17:07 +0200 Message-ID: <20240703141726.252368-19-stefan.klug@ideasonboard.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240703141726.252368-1-stefan.klug@ideasonboard.com> References: <20240703141726.252368-1-stefan.klug@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" There are valid cases where a module returns None. E.g no images were provided for lsc calibration. We should however define proper semantics there. Continue with a warning for now. Signed-off-by: Stefan Klug Reviewed-by: Laurent Pinchart Reviewed-by: Paul Elder --- utils/tuning/libtuning/generators/yaml_output.py | 3 +++ utils/tuning/libtuning/libtuning.py | 4 ++-- 2 files changed, 5 insertions(+), 2 deletions(-) diff --git a/utils/tuning/libtuning/generators/yaml_output.py b/utils/tuning/libtuning/generators/yaml_output.py index 31e265df4ea7..c490081d7de7 100644 --- a/utils/tuning/libtuning/generators/yaml_output.py +++ b/utils/tuning/libtuning/generators/yaml_output.py @@ -107,6 +107,9 @@ class YamlOutput(Generator): ] for module in output_order: + if module not in output_dict: + continue + out_lines.append(f' - {module.out_name}:') if len(output_dict[module]) == 0: diff --git a/utils/tuning/libtuning/libtuning.py b/utils/tuning/libtuning/libtuning.py index 5342e5d6daaa..e7c63535fefd 100644 --- a/utils/tuning/libtuning/libtuning.py +++ b/utils/tuning/libtuning/libtuning.py @@ -200,8 +200,8 @@ class Tuner(object): for module in self.modules: out = module.process(self.config, images, self.output) if out is None: - logger.error(f'Module {module.hr_name} failed to process...') - break + logger.warning(f'Module {module.hr_name} failed to process...') + continue self.output[module] = out self.generator.write(args.output, self.output, self.output_order) From patchwork Wed Jul 3 14:17:08 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefan Klug X-Patchwork-Id: 20533 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 1EA7ABEFBE for ; Wed, 3 Jul 2024 14:18:32 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id CD1D363376; Wed, 3 Jul 2024 16:18:31 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="JavZuqVd"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 88E006336E for ; Wed, 3 Jul 2024 16:18:29 +0200 (CEST) Received: from ideasonboard.com (unknown [IPv6:2a00:6020:448c:6c00:9263:c199:9587:576]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 5EAB04CA; Wed, 3 Jul 2024 16:18:01 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1720016281; bh=UrljjAhio6rU7Y4wHigmMab7p8CuYd+D8fkroZn46ok=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=JavZuqVdeJHNb2Wn6XbI57YwxvVI6EIRWGLDLtKn1xmcYbKCbeJ7b6GZv8qGufwQh g5sV4D1YpYaar/+p7qFhqSvQTbZxl+MFdb0t6R5pRg/k6tjt4JlDf/LotW+mWqYqR0 QgZJrFhykB2yGqJSYaC0zJ8frovs7tpH7W2ePMkg= From: Stefan Klug To: libcamera-devel@lists.libcamera.org Cc: Stefan Klug , Laurent Pinchart Subject: [PATCH v3 19/23] libtuning: Add static module Date: Wed, 3 Jul 2024 16:17:08 +0200 Message-ID: <20240703141726.252368-20-stefan.klug@ideasonboard.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240703141726.252368-1-stefan.klug@ideasonboard.com> References: <20240703141726.252368-1-stefan.klug@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Add a static module class, that can be used to add static data to the tuning file. This is propably not the best solution, but allows us to progress without writing lots of dummy classes for static cases. Signed-off-by: Stefan Klug Reviewed-by: Laurent Pinchart Reviewed-by: Paul Elder --- utils/tuning/libtuning/modules/static.py | 24 ++++++++++++++++++++++++ 1 file changed, 24 insertions(+) create mode 100644 utils/tuning/libtuning/modules/static.py diff --git a/utils/tuning/libtuning/modules/static.py b/utils/tuning/libtuning/modules/static.py new file mode 100644 index 000000000000..4d0f7e18c24e --- /dev/null +++ b/utils/tuning/libtuning/modules/static.py @@ -0,0 +1,24 @@ +# SPDX-License-Identifier: GPL-2.0-or-later +# +# Copyright (C) 2024, Ideas on Board +# +# Module implementation for static data + +from .module import Module + + +# This module can be used in cases where the tuning file should contain +# static data. +class StaticModule(Module): + def __init__(self, out_name: str, output: dict = {}): + super().__init__() + self.out_name = out_name + self.hr_name = f'Static {out_name}' + self.type = f'static_{out_name}' + self.output = output + + def validate_config(self, config: dict) -> bool: + return True + + def process(self, config: dict, images: list, outputs: dict) -> dict: + return self.output From patchwork Wed Jul 3 14:17:09 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefan Klug X-Patchwork-Id: 20534 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 1CC18BEFBE for ; Wed, 3 Jul 2024 14:18:36 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id C184463370; Wed, 3 Jul 2024 16:18:35 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="rXzZTssX"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id EB18163377 for ; Wed, 3 Jul 2024 16:18:31 +0200 (CEST) Received: from ideasonboard.com (unknown [IPv6:2a00:6020:448c:6c00:9263:c199:9587:576]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id C9A5F4CA; Wed, 3 Jul 2024 16:18:03 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1720016283; bh=99m8MFKipaRARjY+1DjYk51S4MxdGkm5FF0JBXdMeqQ=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=rXzZTssXZazNrgzaAaZifGp0z57m8Zu+cE4B8DWF7LNE3rdhZpwZ8JEGe+yUE+09Y cA8LziOHuKQ41qvAC+1QRinHABl2LsZImcTA80ES7nF6Qvmjwp7IDIY1Vsc2Ohqz/2 +0d/u0J5Hb1isjyNE8Z0CbfdO8i+hqk9lphAsPTA= From: Stefan Klug To: libcamera-devel@lists.libcamera.org Cc: Stefan Klug Subject: [PATCH v3 20/23] tuning: rkisp1: Add some static modules Date: Wed, 3 Jul 2024 16:17:09 +0200 Message-ID: <20240703141726.252368-21-stefan.klug@ideasonboard.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240703141726.252368-1-stefan.klug@ideasonboard.com> References: <20240703141726.252368-1-stefan.klug@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Add awb, blc, cproc, filter, and gamma by default to the tuning file. These don't need any configuration. While at it, sort the modules alphabetically. Signed-off-by: Stefan Klug Reviewed-by: Paul Elder --- utils/tuning/rkisp1.py | 21 +++++++++++++++++---- 1 file changed, 17 insertions(+), 4 deletions(-) diff --git a/utils/tuning/rkisp1.py b/utils/tuning/rkisp1.py index fae35783c656..0d279a39ab1b 100755 --- a/utils/tuning/rkisp1.py +++ b/utils/tuning/rkisp1.py @@ -15,11 +15,24 @@ from libtuning.generators import YamlOutput from libtuning.modules.lsc import LSCRkISP1 from libtuning.modules.agc import AGCRkISP1 from libtuning.modules.ccm import CCMRkISP1 - +from libtuning.modules.static import StaticModule coloredlogs.install(level=logging.INFO, fmt='%(name)s %(levelname)s %(message)s') +awb = StaticModule('Awb') +blc = StaticModule('BlackLevelCorrection') +color_processing = StaticModule('ColorProcessing') +filter = StaticModule('Filter') +gamma_out = StaticModule('GammaOutCorrection', {'gamma': 2.2}) + tuner = lt.Tuner('RkISP1') +tuner.add(AGCRkISP1(debug=[lt.Debug.Plot])) +tuner.add(awb) +tuner.add(blc) +tuner.add(CCMRkISP1(debug=[lt.Debug.Plot])) +tuner.add(color_processing) +tuner.add(filter) +tuner.add(gamma_out) tuner.add(LSCRkISP1( debug=[lt.Debug.Plot], # This is for the actual LSC tuning, and is part of the base LSC @@ -39,11 +52,11 @@ tuner.add(LSCRkISP1( # values. This can also be a custom function. smoothing_function=lt.smoothing.MedianBlur(3), )) -tuner.add(AGCRkISP1(debug=[lt.Debug.Plot])) -tuner.add(CCMRkISP1(debug=[lt.Debug.Plot])) + tuner.set_input_parser(YamlParser()) tuner.set_output_formatter(YamlOutput()) -tuner.set_output_order([AGCRkISP1, CCMRkISP1, LSCRkISP1]) +tuner.set_output_order([AGCRkISP1, awb, blc, CCMRkISP1, color_processing, + filter, gamma_out, LSCRkISP1]) if __name__ == '__main__': sys.exit(tuner.run(sys.argv)) From patchwork Wed Jul 3 14:17:10 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefan Klug X-Patchwork-Id: 20535 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id B68DFBEFBE for ; Wed, 3 Jul 2024 14:18:37 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 7495B6336F; Wed, 3 Jul 2024 16:18:37 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="ki3bNeyy"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 1D88B63370 for ; Wed, 3 Jul 2024 16:18:35 +0200 (CEST) Received: from ideasonboard.com (unknown [IPv6:2a00:6020:448c:6c00:9263:c199:9587:576]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id F39C64CA; Wed, 3 Jul 2024 16:18:06 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1720016287; bh=/gVqIs0u6onaUPch87D8xNTLx3TCQPtgdC96/KTuCqc=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=ki3bNeyyh+1/KKbFnbYQry2+7puEHlYdj6bIhU3mvcLlIsvIFbVeA+0Iz62h5ISMI ssk7Cgyj9UZiJdiZ7eRcxeZrKveDjY5xe9p3+7mxV5gz5fiUOBcZ/NkhNjIzYiQUKV a3cr6XWcLw2NiWkjHtXKoDXTZ6d0pF81poAunR+I= From: Stefan Klug To: libcamera-devel@lists.libcamera.org Cc: Stefan Klug Subject: [PATCH v3 21/23] libtuning: lsc: rkisp1: Do not calculate ratios to green Date: Wed, 3 Jul 2024 16:17:10 +0200 Message-ID: <20240703141726.252368-22-stefan.klug@ideasonboard.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240703141726.252368-1-stefan.klug@ideasonboard.com> References: <20240703141726.252368-1-stefan.klug@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" The current LSC algorithm for the rkisp1 just forwards the LSC tables to the hardware, so absolute factors are needed and not ratios compared to green. Therefore every channel needs to be calculated independently. Signed-off-by: Stefan Klug Reviewed-by: Laurent Pinchart Reviewed-by: Paul Elder --- utils/tuning/libtuning/modules/lsc/rkisp1.py | 8 ++++++-- 1 file changed, 6 insertions(+), 2 deletions(-) diff --git a/utils/tuning/libtuning/modules/lsc/rkisp1.py b/utils/tuning/libtuning/modules/lsc/rkisp1.py index 512233aeae9d..56df23ec7b42 100644 --- a/utils/tuning/libtuning/modules/lsc/rkisp1.py +++ b/utils/tuning/libtuning/modules/lsc/rkisp1.py @@ -38,8 +38,12 @@ class LSCRkISP1(LSC): # \todo Should these ratio against the average of both greens or just # each green like we've done here? - cr, _ = self._lsc_single_channel(image.channels[lt.Color.R], image, gr) - cb, _ = self._lsc_single_channel(image.channels[lt.Color.B], image, gb) + + # The LSC tables in the our rkisp1 algorithm represent gains on the + # corresponding channel, not ratios with respect to green, so calculate + # them independently + cr, _ = self._lsc_single_channel(image.channels[lt.Color.R], image, None) + cb, _ = self._lsc_single_channel(image.channels[lt.Color.B], image, None) return image.color, cr.flatten(), cb.flatten(), cgr.flatten(), cgb.flatten() From patchwork Wed Jul 3 14:17:11 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefan Klug X-Patchwork-Id: 20536 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 58E14BEFBE for ; Wed, 3 Jul 2024 14:18:41 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 1766363372; Wed, 3 Jul 2024 16:18:41 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="Wi/tT4V/"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 1534762E22 for ; Wed, 3 Jul 2024 16:18:38 +0200 (CEST) Received: from ideasonboard.com (unknown [IPv6:2a00:6020:448c:6c00:9263:c199:9587:576]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id E39984CA; Wed, 3 Jul 2024 16:18:09 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1720016290; bh=faXPQZCwK6fG+KAnkat5u5LDzhhbxj3syAQB9obo2P8=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=Wi/tT4V/G9YbZ9e8PE0p+efUaTPBftMH1PRfL53h9Jrmbewyf60F3PHQmMPM0eO5G wSBsw/oe2ZYzRTChrFAITXlB+JGYhnmCPFl4q8Xw2kV5m0QSrfI7Dr7Pva/vgPn/oM b/A008VKf9iAu4d8cukD6CqEN0nujtFz2LAst/O0= From: Stefan Klug To: libcamera-devel@lists.libcamera.org Cc: Stefan Klug , Laurent Pinchart Subject: [PATCH v3 22/23] libtuning: lsc: Prevent negative values Date: Wed, 3 Jul 2024 16:17:11 +0200 Message-ID: <20240703141726.252368-23-stefan.klug@ideasonboard.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240703141726.252368-1-stefan.klug@ideasonboard.com> References: <20240703141726.252368-1-stefan.klug@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" In cases where the calibration image contains super dark areas, or when an invalid blacklevel was supplied, the grid might get close to zero or negative. This would have bad effects on the 1/grid later. So clamp the values to a small positive number. Signed-off-by: Stefan Klug Reviewed-by: Laurent Pinchart Reviewed-by: Paul Elder --- utils/tuning/libtuning/modules/lsc/lsc.py | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/utils/tuning/libtuning/modules/lsc/lsc.py b/utils/tuning/libtuning/modules/lsc/lsc.py index 344a07a3d443..e0ca22ebe9bd 100644 --- a/utils/tuning/libtuning/modules/lsc/lsc.py +++ b/utils/tuning/libtuning/modules/lsc/lsc.py @@ -59,7 +59,10 @@ class LSC(Module): def _lsc_single_channel(self, channel: np.array, image: lt.Image, green_grid: np.array = None): grid = self._get_grid(channel, image.w, image.h) - grid -= image.blacklevel_16 + # Clamp the values to a small positive, so that the following 1/grid + # doesn't produce negative results. + grid = np.maximum(grid - image.blacklevel_16, 0.1) + if green_grid is None: table = np.reshape(1 / grid, self.sector_shape[::-1]) else: From patchwork Wed Jul 3 14:17:12 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stefan Klug X-Patchwork-Id: 20537 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 6E107BEFBE for ; Wed, 3 Jul 2024 14:18:44 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 212B863372; Wed, 3 Jul 2024 16:18:44 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="rVboq7/6"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 636A663370 for ; Wed, 3 Jul 2024 16:18:41 +0200 (CEST) Received: from ideasonboard.com (unknown [IPv6:2a00:6020:448c:6c00:9263:c199:9587:576]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 435994CA; Wed, 3 Jul 2024 16:18:13 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1720016293; bh=8REfSy+49hMeQ0J7u/T/NBeOTW1n8QoejuZ9uBZaysY=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=rVboq7/6nCPXjD3+cmfk208XDyuA21Shx9VRYAu8yn2u4YBVMM8joEGOvClMgBKcN 2ERb15LQx1nlaS1cJ1k2Eh29dzdfc2B6bIojARM+wcazJk4BSoqNr6pmRHX3Ba9DJA GBnwnsq3rct4sZg31I+Ug/vZPk/gcUl1OZVLdAjA= From: Stefan Klug To: libcamera-devel@lists.libcamera.org Cc: Stefan Klug , Laurent Pinchart Subject: [PATCH v3 23/23] libtuning: agc: rkisp1: Increase y-target Date: Wed, 3 Jul 2024 16:17:12 +0200 Message-ID: <20240703141726.252368-24-stefan.klug@ideasonboard.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240703141726.252368-1-stefan.klug@ideasonboard.com> References: <20240703141726.252368-1-stefan.klug@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" With the addition of gamma out correction the relative luminance target was set too low. As brightness is a bit subjective it is difficult to come up with the "correct" value. With 0.5 the patch 22 on the mac beth chart (neutral grey, 18% reflectance) ended up a bit below 50% grey, which seems reasonable. Signed-off-by: Stefan Klug Reviewed-by: Laurent Pinchart Reviewed-by: Paul Elder --- utils/tuning/libtuning/modules/agc/rkisp1.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/utils/tuning/libtuning/modules/agc/rkisp1.py b/utils/tuning/libtuning/modules/agc/rkisp1.py index 19a5555b6111..7147028a774a 100644 --- a/utils/tuning/libtuning/modules/agc/rkisp1.py +++ b/utils/tuning/libtuning/modules/agc/rkisp1.py @@ -64,7 +64,7 @@ class AGCRkISP1(AGC): return {'ConstraintNormal': normal, 'ConstraintHighlight': highlight} def _generate_y_target(self) -> list: - return 0.16 + return 0.5 def process(self, config: dict, images: list, outputs: dict) -> dict: output = {}