From patchwork Fri Jul 7 03:17:00 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Ben Benson X-Patchwork-Id: 18816 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id BEE3FC3243 for ; Thu, 13 Jul 2023 10:35:22 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id AC1BD628C7; Thu, 13 Jul 2023 12:35:20 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1689244520; bh=o+EiVNZ6qzxOgWZd8C4VLbV5zeBWUy3cyNN09w8W7qY=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To:Cc: From; b=wrxf/2mEHg5S6lbBImAodUVqwxFEnsP6qoc8b2BPlk+wsaofvYqFNLpmic0Jjha+F ZdIGk2E15pyBgUH64PEMgqFaf4ZLFJyRsSriqajyFlyUWWI11/YtrRqyBAmcx7cXQj g7uF6A2QrOaoBryxAHk70rF08jdngGssiAa10RVu0piPNciTNGtt1nmLFd04UVvFRj wh8uobSzg03o738AgEXEKzdvcfSCbOPJZiBiHrg6HoxW3xic+ipHZfYhGcUPLAhm4a ymuDGnFsVHdjRhg3JvPB7FVmOdhHVFCcKVvSrQ87IildNQwa7odrlO9+orS5E1cmzL JelUUXFwE1D0Q== Received: from mail-wr1-x42c.google.com (mail-wr1-x42c.google.com [IPv6:2a00:1450:4864:20::42c]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 194B261E37 for ; Thu, 13 Jul 2023 12:35:19 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="dSSb9qLi"; dkim-atps=neutral Received: by mail-wr1-x42c.google.com with SMTP id ffacd0b85a97d-3142a9ff6d8so615435f8f.3 for ; Thu, 13 Jul 2023 03:35:19 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; t=1689244518; x=1691836518; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=w0Nt3OgYoUoR8NrIlF4fNRYpoIzlhcYW+sZSO2rIvAQ=; b=dSSb9qLiqeJALdzgFdppmm2Il1iP8jxB49CoxYMmRvnaIaozXMHBnxIkmYE6VjFaI3 EU+5mSpgUsgtsKdK48G0K9CvAy2X2LM9M9SYt5z7JeVAjqD5a9HbROdjdpHF8vJphCXu Rfm4LhOMxlm/gN4DjDBnmzJVPoay/aUR3ItfUn6+7YoShkUiZg7FxYScVIh8sCI913h5 OQ1oVPTsrgKqpi0hSv/uoUPKZhuZjdHW/nZYYWGBwp+nCpJxOYyTWGDm2Iwir50zYrdG JWvup1jZIKQvehQlxJoLEoFdvHbCj4ayMClEQrwyoshatVdzUMJEP6VYe0/PpgrfHVgP lRNQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1689244518; x=1691836518; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=w0Nt3OgYoUoR8NrIlF4fNRYpoIzlhcYW+sZSO2rIvAQ=; b=dPQf+laEoIatzbjGOgTUkSVlOwN/vJZJMhJcy8z7puGYQihIw1lq3V5wLru6AGDKYq hRv7U/eXr24kBEiM33bTrjQQjiz4KYg8Kn8da0XaoWm6Q2Ugktn5ztDUCXSjKOat2QrB QQ1As7gdlqP18pfe1PF5Y0h0gwDD95T9AQ8OWPYyfDg/Lbeptr59f5C5qOGPHXtxBxQb pZ+AnHBCR2wzRUSx03tKXI4ae2Nyl1hWQ5mJKs75Ji/qydiZ2UmUi4n1nNr16gvLMp7T JXDC/Vgdqva6+QnsKsIxtpdI4ImWRL/vBmF9BC5zh31GXw8+/5xLIayxBHVxauEEBlpu ApUw== X-Gm-Message-State: ABy/qLb/TpPlkd1K9v7lz2g/aHF/CEZwZuXpU3ZKOviwj/9aw4JLZbrO exfxOJbRc9sj1YEv+oNVnTlCJBVUOua/t/8h7a7dxyKH X-Google-Smtp-Source: APBJJlF4jGwYuJztwHb8B/t/Jf7FVwcYzBPKt2sczNMEky5Fw0HR7OqIz2y359cYyAcoVGB0to2Jyw== X-Received: by 2002:a5d:6850:0:b0:314:53a4:2d3f with SMTP id o16-20020a5d6850000000b0031453a42d3fmr1149577wrw.63.1689244518179; Thu, 13 Jul 2023 03:35:18 -0700 (PDT) Received: from localhost.localdomain ([93.93.133.154]) by smtp.gmail.com with ESMTPSA id r8-20020a056000014800b0030fa3567541sm7551143wrx.48.2023.07.13.03.35.17 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 13 Jul 2023 03:35:17 -0700 (PDT) To: libcamera-devel@lists.libcamera.org Date: Fri, 7 Jul 2023 04:17:00 +0100 Message-Id: <20230707031702.603261-2-ben.benson@raspberrypi.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230707031702.603261-1-ben.benson@raspberrypi.com> References: <20230707031702.603261-1-ben.benson@raspberrypi.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v2 1/3] utils: raspberrypi: ctt: Improved color matrix fitting X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Ben Benson via libcamera-devel From: Ben Benson Reply-To: Ben Benson Cc: Ben Benson Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Added code which optimises the color matrices based off delta E values for the calibration images. Working in LAB color space. Signed-off-by: Ben Benson --- utils/raspberrypi/ctt/colors.py | 30 +++ utils/raspberrypi/ctt/ctt_ccm.py | 265 +++++++++++++++++++++---- utils/raspberrypi/ctt/ctt_visualise.py | 43 ++++ 3 files changed, 300 insertions(+), 38 deletions(-) create mode 100644 utils/raspberrypi/ctt/colors.py create mode 100644 utils/raspberrypi/ctt/ctt_visualise.py diff --git a/utils/raspberrypi/ctt/colors.py b/utils/raspberrypi/ctt/colors.py new file mode 100644 index 00000000..1ab986d6 --- /dev/null +++ b/utils/raspberrypi/ctt/colors.py @@ -0,0 +1,30 @@ +# colors.py - Program to convert from RGB to LAB color space +def RGB_to_LAB(RGB): # where RGB is a 1x3 array. e.g RGB = [100, 255, 230] + num = 0 + XYZ = [0, 0, 0] + # converted all the three R, G, B to X, Y, Z + X = RGB[0] * 0.4124 + RGB[1] * 0.3576 + RGB[2] * 0.1805 + Y = RGB[0] * 0.2126 + RGB[1] * 0.7152 + RGB[2] * 0.0722 + Z = RGB[0] * 0.0193 + RGB[1] * 0.1192 + RGB[2] * 0.9505 + + XYZ[0] = X / 255 * 100 + XYZ[1] = Y / 255 * 100 # XYZ Must be in range 0 -> 100, so scale down from 255 + XYZ[2] = Z / 255 * 100 + XYZ[0] = XYZ[0] / 95.047 # ref_X = 95.047 Observer= 2°, Illuminant= D65 + XYZ[1] = XYZ[1] / 100.0 # ref_Y = 100.000 + XYZ[2] = XYZ[2] / 108.883 # ref_Z = 108.883 + num = 0 + for value in XYZ: + if value > 0.008856: + value = value ** (0.3333333333333333) + else: + value = (7.787 * value) + (16 / 116) + XYZ[num] = value + num = num + 1 + + # L, A, B, values calculated below + L = (116 * XYZ[1]) - 16 + a = 500 * (XYZ[0] - XYZ[1]) + b = 200 * (XYZ[1] - XYZ[2]) + + return [L, a, b] diff --git a/utils/raspberrypi/ctt/ctt_ccm.py b/utils/raspberrypi/ctt/ctt_ccm.py index 376cc712..49159535 100644 --- a/utils/raspberrypi/ctt/ctt_ccm.py +++ b/utils/raspberrypi/ctt/ctt_ccm.py @@ -6,27 +6,66 @@ from ctt_image_load import * from ctt_awb import get_alsc_patches - - +import colors +from scipy.optimize import minimize +from ctt_visualise import visualise_macbeth_chart +import numpy as np """ takes 8-bit macbeth chart values, degammas and returns 16 bit """ + +''' +This program has many options from which to derive the color matrix from. +The first is average. This minimises the average delta E across all patches of +the macbeth chart. Testing across all cameras yeilded this as the most color +accurate and vivid. Other options are avalible however. +Maximum minimises the maximum Delta E of the patches. It iterates through till +a minimum maximum is found (so that there is +not one patch that deviates wildly.) +This yields generally good results but overall the colors are less accurate +Have a fiddle with maximum and see what you think. +The final option allows you to select the patches for which to average across. +This means that you can bias certain patches, for instance if you want the +reds to be more accurate. +''' + +matrix_selection_types = ["average", "maximum", "patches"] +typenum = 0 # select from array above, 0 = average, 1 = maximum, 2 = patches +test_patches = [1, 2, 5, 8, 9, 12, 14] + +''' +Enter patches to test for. Can also be entered twice if you +would like twice as much bias on one patch. +''' + + def degamma(x): - x = x / ((2**8)-1) - x = np.where(x < 0.04045, x/12.92, ((x+0.055)/1.055)**2.4) - x = x * ((2**16)-1) + x = x / ((2 ** 8) - 1) # takes 255 and scales it down to one + x = np.where(x < 0.04045, x / 12.92, ((x + 0.055) / 1.055) ** 2.4) + x = x * ((2 ** 16) - 1) # takes one and scales up to 65535, 16 bit color return x +def gamma(x): + # return (x * * (1 / 2.4) * 1.055 - 0.055) + e = [] + for i in range(len(x)): + e.append(((x[i] / 255) ** (1 / 2.4) * 1.055 - 0.055) * 255) + return e + + """ FInds colour correction matrices for list of images """ + + def ccm(Cam, cal_cr_list, cal_cb_list): + global matrix_selection_types, typenum imgs = Cam.imgs """ standard macbeth chart colour values """ - m_rgb = np.array([ # these are in sRGB + m_rgb = np.array([ # these are in RGB [116, 81, 67], # dark skin [199, 147, 129], # light skin [91, 122, 156], # blue sky @@ -34,7 +73,7 @@ def ccm(Cam, cal_cr_list, cal_cb_list): [130, 128, 176], # blue flower [92, 190, 172], # bluish green [224, 124, 47], # orange - [68, 91, 170], # purplish blue + [68, 91, 170], # purplish blue [198, 82, 97], # moderate red [94, 58, 106], # purple [159, 189, 63], # yellow green @@ -52,16 +91,22 @@ def ccm(Cam, cal_cr_list, cal_cb_list): [82, 84, 86], # neutral 3.5 [49, 49, 51] # black 2 ]) - """ convert reference colours from srgb to rgb """ - m_srgb = degamma(m_rgb) + m_srgb = degamma(m_rgb) # now in 16 bit color. + + m_lab = [] + for col in m_srgb: + m_lab.append(colors.RGB_to_LAB(col / 256)) + # This produces matrix of LAB values for ideal color chart) + """ reorder reference values to match how patches are ordered """ m_srgb = np.array([m_srgb[i::6] for i in range(6)]).reshape((24, 3)) - + m_lab = np.array([m_lab[i::6] for i in range(6)]).reshape((24, 3)) + m_rgb = np.array([m_rgb[i::6] for i in range(6)]).reshape((24, 3)) """ reformat alsc correction tables or set colour_cals to None if alsc is deactivated @@ -76,8 +121,8 @@ def ccm(Cam, cal_cr_list, cal_cb_list): """ normalise tables so min value is 1 """ - cr_tab = cr_tab/np.min(cr_tab) - cb_tab = cb_tab/np.min(cb_tab) + cr_tab = cr_tab / np.min(cr_tab) + cb_tab = cb_tab / np.min(cb_tab) colour_cals[cr['ct']] = [cr_tab, cb_tab] """ @@ -94,6 +139,8 @@ def ccm(Cam, cal_cr_list, cal_cb_list): the function will simply return the macbeth patches """ r, b, g = get_alsc_patches(Img, colour_cals, grey=False) + # 256 values for each patch of sRGB values + """ do awb Note: awb is done by measuring the macbeth chart in the image, rather @@ -101,34 +148,123 @@ def ccm(Cam, cal_cr_list, cal_cb_list): and the ccm matrices will be more accurate. """ r_greys, b_greys, g_greys = r[3::4], b[3::4], g[3::4] - r_g = np.mean(r_greys/g_greys) - b_g = np.mean(b_greys/g_greys) + r_g = np.mean(r_greys / g_greys) + b_g = np.mean(b_greys / g_greys) r = r / r_g b = b / b_g - """ normalise brightness wrt reference macbeth colours and then average each channel for each patch """ - gain = np.mean(m_srgb)/np.mean((r, g, b)) + gain = np.mean(m_srgb) / np.mean((r, g, b)) Cam.log += '\nGain with respect to standard colours: {:.3f}'.format(gain) - r = np.mean(gain*r, axis=1) - b = np.mean(gain*b, axis=1) - g = np.mean(gain*g, axis=1) - + r = np.mean(gain * r, axis=1) + b = np.mean(gain * b, axis=1) + g = np.mean(gain * g, axis=1) """ calculate ccm matrix """ + # ==== All of below should in sRGB ===## + sumde = 0 ccm = do_ccm(r, g, b, m_srgb) + # This is the initial guess that our optimisation code works with. + + r1 = ccm[0] + r2 = ccm[1] + g1 = ccm[3] + g2 = ccm[4] + b1 = ccm[6] + b2 = ccm[7] + ''' + COLOR MATRIX LOOKS AS BELOW + R1 R2 R3 Rval Outr + G1 G2 G3 * Gval = G + B1 B2 B3 Bval B + Will be optimising 6 elements and working out the third element using 1-r1-r2 = r3 + ''' + + x0 = [r1, r2, g1, g2, b1, b2] + ''' + We use our old CCM as the initial guess for the program to find the + optimised matrix + ''' + result = minimize(guess, x0, args=(r, g, b, m_lab), tol=0.0000000001) + ''' + This produces a color matrix which has the lowest delta E possible, + based off the input data. Note it is impossible for this to reach + zero since the input data is imperfect + ''' + + Cam.log += ("\n \n Optimised Matrix Below: \n \n") + [r1, r2, g1, g2, b1, b2] = result.x + # The new, optimised color correction matrix values + optimised_ccm = [r1, r2, (1 - r1 - r2), g1, g2, (1 - g1 - g2), b1, b2, (1 - b1 - b2)] + # This is the optimised Color Matrix (preserving greys by summing rows up to 1) + Cam.log += str(optimised_ccm) + Cam.log += "\n Old Color Correction Matrix Below \n" + Cam.log += str(ccm) + + formatted_ccm = np.array(ccm).reshape((3, 3)) + + ''' + below is a whole load of code that then applies the latest color + matrix, and returns LAB values for color. This can then be used + to calculate the final delta E + ''' + optimised_ccm_rgb = [] # Original Color Corrected Matrix RGB / LAB + optimised_ccm_lab = [] + for w in range(24): + RGB = np.array([r[w], g[w], b[w]]) + ccm_applied_rgb = np.dot(formatted_ccm, (RGB / 256)) + optimised_ccm_rgb.append(gamma(ccm_applied_rgb)) + optimised_ccm_lab.append(colors.RGB_to_LAB(ccm_applied_rgb)) + + formatted_optimised_ccm = np.array(ccm).reshape((3, 3)) + after_gamma_rgb = [] + after_gamma_lab = [] + for w in range(24): + RGB = np.array([r[w], g[w], b[w]]) + optimised_ccm_applied_rgb = np.dot(formatted_optimised_ccm, RGB / 256) + after_gamma_rgb.append(gamma(optimised_ccm_applied_rgb)) + after_gamma_lab.append(colors.RGB_to_LAB(optimised_ccm_applied_rgb)) + ''' + Gamma After RGB / LAB + We now want to spit out some data that shows + how the optimisation has improved the color matrices + ''' + Cam.log += "Here are the Improvements" + + # CALCULATE WORST CASE delta e + old_worst_delta_e = 0 + before_average = transform_and_evaluate(formatted_ccm, r, g, b, m_lab) + new_worst_delta_e = 0 + after_average = transform_and_evaluate(formatted_optimised_ccm, r, g, b, m_lab) + for i in range(24): + old_delta_e = deltae(optimised_ccm_lab[i], m_lab[i]) # Current Old Delta E + new_delta_e = deltae(after_gamma_lab[i], m_lab[i]) # Current New Delta E + if old_delta_e > old_worst_delta_e: + old_worst_delta_e = old_delta_e + if new_delta_e > new_worst_delta_e: + new_worst_delta_e = new_delta_e + + Cam.log += "Before color correction matrix was optimised, we got an average delta E of " + str(before_average) + " and a maximum delta E of " + str(old_worst_delta_e) + Cam.log += "After color correction matrix was optimised, we got an average delta E of " + str(after_average) + " and a maximum delta E of " + str(new_worst_delta_e) + + visualise_macbeth_chart(m_rgb, optimised_ccm_rgb, after_gamma_rgb, str(Img.col) + str(matrix_selection_types[typenum])) + ''' + The program will also save some visualisations of improvements. + Very pretty to look at. Top rectangle is ideal, Left square is + before optimisation, right square is after. + ''' """ if a ccm has already been calculated for that temperature then don't overwrite but save both. They will then be averaged later on - """ + """ # Now going to use optimised color matrix, optimised_ccm if Img.col in ccm_tab.keys(): - ccm_tab[Img.col].append(ccm) + ccm_tab[Img.col].append(optimised_ccm) else: - ccm_tab[Img.col] = [ccm] + ccm_tab[Img.col] = [optimised_ccm] Cam.log += '\n' Cam.log += '\nFinished processing images' @@ -137,8 +273,8 @@ def ccm(Cam, cal_cr_list, cal_cb_list): """ for k, v in ccm_tab.items(): tab = np.mean(v, axis=0) - tab = np.where((10000*tab) % 1 <= 0.05, tab+0.00001, tab) - tab = np.where((10000*tab) % 1 >= 0.95, tab-0.00001, tab) + tab = np.where((10000 * tab) % 1 <= 0.05, tab + 0.00001, tab) + tab = np.where((10000 * tab) % 1 >= 0.95, tab - 0.00001, tab) ccm_tab[k] = list(np.round(tab, 5)) Cam.log += '\nMatrix calculated for colour temperature of {} K'.format(k) @@ -156,20 +292,67 @@ def ccm(Cam, cal_cr_list, cal_cb_list): return ccms +def guess(x0, r, g, b, m_lab): # provides a method of numerical feedback for the optimisation code + [r1, r2, g1, g2, b1, b2] = x0 + ccm = np.array([r1, r2, (1 - r1 - r2), + g1, g2, (1 - g1 - g2), + b1, b2, (1 - b1 - b2)]).reshape((3, 3)) # format the matrix correctly + return transform_and_evaluate(ccm, r, g, b, m_lab) + + +def transform_and_evaluate(ccm, r, g, b, m_lab): # Transforms colors to LAB and applies the correction matrix + # create list of matrix changed colors + realrgb = [] + for i in range(len(r)): + RGB = np.array([r[i], g[i], b[i]]) + rgb_post_ccm = np.dot(ccm, RGB) # This is RGB values after the color correction matrix has been applied + realrgb.append(colors.RGB_to_LAB(rgb_post_ccm)) + # now compare that with m_lab and return numeric result, averaged for each patch + return (sumde(realrgb, m_lab) / 24) # returns an average result of delta E + + +def sumde(listA, listB): + global typenum, test_patches + sumde = 0 + maxde = 0 + patchde = [] + for i in range(len(listA)): + if maxde < (deltae(listA[i], listB[i])): + maxde = deltae(listA[i], listB[i]) + patchde.append(deltae(listA[i], listB[i])) + sumde += deltae(listA[i], listB[i]) + ''' + The different options specified at the start allow for + the maximum to be returned, average or specific patches + ''' + if typenum == 0: + return sumde + if typenum == 1: + return maxde + if typenum == 2: + output = 0 + for y in range(len(test_patches)): + output += patchde[test_patches[y]] # grabs the specific patches (no need for averaging here) + return output + + """ calculates the ccm for an individual image. -ccms are calculate in rgb space, and are fit by hand. Although it is a 3x3 +ccms are calculated in rgb space, and are fit by hand. Although it is a 3x3 matrix, each row must add up to 1 in order to conserve greyness, simplifying calculation. -Should you want to fit them in another space (e.g. LAB) we wish you the best of -luck and send us the code when you are done! :-) +The initial CCM is calculated in RGB, and then optimised in LAB color space +This simplifies the initial calculation but then gets us the accuracy of +using LAB color space. """ + + def do_ccm(r, g, b, m_srgb): rb = r-b gb = g-b - rb_2s = (rb*rb) - rb_gbs = (rb*gb) - gb_2s = (gb*gb) + rb_2s = (rb * rb) + rb_gbs = (rb * gb) + gb_2s = (gb * gb) r_rbs = rb * (m_srgb[..., 0] - b) r_gbs = gb * (m_srgb[..., 0] - b) @@ -191,7 +374,7 @@ def do_ccm(r, g, b, m_srgb): b_rb = np.sum(b_rbs) b_gb = np.sum(b_gbs) - det = rb_2*gb_2 - rb_gb*rb_gb + det = rb_2 * gb_2 - rb_gb * rb_gb """ Raise error if matrix is singular... @@ -201,19 +384,19 @@ def do_ccm(r, g, b, m_srgb): if det < 0.001: raise ArithmeticError - r_a = (gb_2*r_rb - rb_gb*r_gb)/det - r_b = (rb_2*r_gb - rb_gb*r_rb)/det + r_a = (gb_2 * r_rb - rb_gb * r_gb) / det + r_b = (rb_2 * r_gb - rb_gb * r_rb) / det """ Last row can be calculated by knowing the sum must be 1 """ r_c = 1 - r_a - r_b - g_a = (gb_2*g_rb - rb_gb*g_gb)/det - g_b = (rb_2*g_gb - rb_gb*g_rb)/det + g_a = (gb_2 * g_rb - rb_gb * g_gb) / det + g_b = (rb_2 * g_gb - rb_gb * g_rb) / det g_c = 1 - g_a - g_b - b_a = (gb_2*b_rb - rb_gb*b_gb)/det - b_b = (rb_2*b_gb - rb_gb*b_rb)/det + b_a = (gb_2 * b_rb - rb_gb * b_gb) / det + b_b = (rb_2 * b_gb - rb_gb * b_rb) / det b_c = 1 - b_a - b_b """ @@ -222,3 +405,9 @@ def do_ccm(r, g, b, m_srgb): ccm = [r_a, r_b, r_c, g_a, g_b, g_c, b_a, b_b, b_c] return ccm + + +def deltae(colorA, colorB): + return ((colorA[0] - colorB[0]) ** 2 + (colorA[1] - colorB[1]) ** 2 + (colorA[2] - colorB[2]) ** 2) ** 0.5 + # return ((colorA[1]-colorB[1]) * * 2 + (colorA[2]-colorB[2]) * * 2) * * 0.5 + # UNCOMMENT IF YOU WANT TO NEGLECT LUMINANCE FROM CALCULATION OF DELTA E diff --git a/utils/raspberrypi/ctt/ctt_visualise.py b/utils/raspberrypi/ctt/ctt_visualise.py new file mode 100644 index 00000000..ed2339fd --- /dev/null +++ b/utils/raspberrypi/ctt/ctt_visualise.py @@ -0,0 +1,43 @@ +""" +Some code that will save virtual macbeth charts that show the difference between optimised matrices and non optimised matrices + +The function creates an image that is 1550 by 1050 pixels wide, and fills it with patches which are 200x200 pixels in size +Each patch contains the ideal color, the color from the original matrix, and the color from the final matrix +_________________ +| | +| Ideal Color | +|_______________| +| Old | new | +| Color | Color | +|_______|_______| + +Nice way of showing how the optimisation helps change the colors and the color matricies +""" +import numpy as np +from PIL import Image + + +def visualise_macbeth_chart(macbeth_rgb, original_rgb, new_rgb, output_filename): + image = np.zeros((1050, 1550, 3), dtype=np.uint8) + colorindex = -1 + for y in range(6): + for x in range(4): # Creates 6 x 4 grid of macbeth chart + colorindex += 1 + xlocation = 50 + 250 * x # Means there is 50px of black gap between each square, more like the real macbeth chart. + ylocation = 50 + 250 * y + for g in range(200): + for i in range(100): + image[xlocation + i, ylocation + g] = macbeth_rgb[colorindex] + xlocation = 150 + 250 * x + ylocation = 50 + 250 * y + for i in range(100): + for g in range(100): + image[xlocation + i, ylocation + g] = original_rgb[colorindex] # Smaller squares below to compare the old colors with the new ones + xlocation = 150 + 250 * x + ylocation = 150 + 250 * y + for i in range(100): + for g in range(100): + image[xlocation + i, ylocation + g] = new_rgb[colorindex] + + img = Image.fromarray(image, 'RGB') + img.save(str(output_filename) + 'Generated Macbeth Chart.png') From patchwork Fri Jul 7 03:17:01 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ben Benson X-Patchwork-Id: 18817 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id B386EBEFBE for ; Thu, 13 Jul 2023 10:35:23 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 70890628C4; Thu, 13 Jul 2023 12:35:23 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1689244523; bh=skTWx5PCpmZPRau+2uomNEgCwgsBj8cJZQl7nLDi2VU=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To:Cc: From; b=W45cGjfZXJvyBCpDR2j07zwG6cRfZ5fHkLFaqW+9ltXat++fgbvLrvr8LBoqBSM06 twLmgGc2fad2X2G7Vomjdw81nx63Ic+x+bwXK+DXaA+NvuutDXwQ9OR+s8nH5vVURm MA+gsj+FZbQxBs+GA2rK/7D257fmQmd7StVMsiAYW+AKeqgrqwSK9ZDkWVBbwbFtqR 5uajc5ZZUO6J3UErXLchSyJ88nbUgTNBQSjDhP63H986L9Q0cxNYRLpAckk83z8YGH gGP5SKBnAfjy4Lg9As6WopIdeT1Apr2PWvefREniRFYud1Xqa9pWE1Ftiaw4bSMUoj kXeldNjOhvNVw== Received: from mail-wr1-x432.google.com (mail-wr1-x432.google.com [IPv6:2a00:1450:4864:20::432]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 907D061E37 for ; Thu, 13 Jul 2023 12:35:19 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="GZkWZdCD"; dkim-atps=neutral Received: by mail-wr1-x432.google.com with SMTP id ffacd0b85a97d-307d20548adso658500f8f.0 for ; Thu, 13 Jul 2023 03:35:19 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; t=1689244519; x=1691836519; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=DbfFlmL6pEmeV4pBbF+lPZGL5WRhELM3uxBAHcsymr0=; b=GZkWZdCDlB/+tnrZ7p0O8hC4kzdwPa/+0fHvu2VWR3Cc57tBiwrOq6tSqTcMeg3RmH iLwtzudJi76v1aB0zslRbkjrzzJejxjn35cWGTp/FiG/dK4PPR6YUbWaJWLaH9PzDHA6 UNaDU6C5XbgMuKwSDbVBkLspYW0kfOrp2UY9k6rCX0xAJ/HBEaJFCdKVexDSE8yvGNdW 8xrEckrNVETd96BIrZ2VlcvYrXRwx8j9GrhWEbCF9IHtOYZf+7QfSplKLgvL5lz0qeOJ EKo3sMWktXSTg+TLU2c0K/9FAlrlD2thAehLX79Ahvb5yb0bwP4qsq8pbrVczqeG5Vy9 JmAg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1689244519; x=1691836519; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=DbfFlmL6pEmeV4pBbF+lPZGL5WRhELM3uxBAHcsymr0=; b=Zkxbz9y9h+LP/DWQW9Nux04tGw7NKilQU+pbIp1gm5JItpQpmdMyDpLM/ZY2BPdQuh e+JVLeBc602JBV5cLzpM2CuEhFzFjEAz4q7gblKcdAjGJ1ejgTYbDrfmVlrXmyj3Q3QY GJ5CCu8CuM62lyezQvthKIVk7OVoXEV5zA2TTXYhTWNMKcSVJxBQg58xlk2nt7THmLF5 rM9wtkToA06aGYythRVD5c3222mGR+rnZjM4NjQddxdpMh7+0FDDox0DURNPD8yjcB58 7U7g80UYDdFevzmXZp/XmZvvsFVMbSd1upTKMeexBbiSMTPXzaiu6kwp4LybHxGtc15S FvWg== X-Gm-Message-State: ABy/qLb8AeC3K5FcSuuLv9sjeVN2e0DOrdKQx+VyoJoibIQwMOPyZfEE 5KzlCPvWmtLMMy3NiYb2cmhJ+PRu+H2uklOHsxTL450B X-Google-Smtp-Source: APBJJlGHukatixnteqKhwf08fXLNZUamSWfCws2ehoAhPVgjaKraMd3p/Rnon1p4aT/jrOMfeRyMLg== X-Received: by 2002:adf:eb87:0:b0:313:e715:3260 with SMTP id t7-20020adfeb87000000b00313e7153260mr1098716wrn.48.1689244518802; Thu, 13 Jul 2023 03:35:18 -0700 (PDT) Received: from localhost.localdomain ([93.93.133.154]) by smtp.gmail.com with ESMTPSA id r8-20020a056000014800b0030fa3567541sm7551143wrx.48.2023.07.13.03.35.18 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 13 Jul 2023 03:35:18 -0700 (PDT) To: libcamera-devel@lists.libcamera.org Date: Fri, 7 Jul 2023 04:17:01 +0100 Message-Id: <20230707031702.603261-3-ben.benson@raspberrypi.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230707031702.603261-1-ben.benson@raspberrypi.com> References: <20230707031702.603261-1-ben.benson@raspberrypi.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v2 2/3] ipa: rpi: vc4: data: Updated color matrices for RPi Cameras X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Ben Benson via libcamera-devel From: Ben Benson Reply-To: Ben Benson Cc: Ben Benson Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Altered the color matrices for the tuning files for various cameras in order to make them more color accurate. Signed-off-by: Ben Benson --- src/ipa/rpi/vc4/data/imx219.json | 66 ++++------- src/ipa/rpi/vc4/data/imx296.json | 160 +++++--------------------- src/ipa/rpi/vc4/data/imx477.json | 93 ++++----------- src/ipa/rpi/vc4/data/imx708.json | 139 ++++------------------ src/ipa/rpi/vc4/data/imx708_wide.json | 49 ++++---- src/ipa/rpi/vc4/data/ov5647.json | 66 ++++------- 6 files changed, 147 insertions(+), 426 deletions(-) diff --git a/src/ipa/rpi/vc4/data/imx219.json b/src/ipa/rpi/vc4/data/imx219.json index efe7210a..e8fce164 100644 --- a/src/ipa/rpi/vc4/data/imx219.json +++ b/src/ipa/rpi/vc4/data/imx219.json @@ -405,75 +405,57 @@ { "ccms": [ { - "ct": 2498, + "ct": 2860, "ccm": [ - 1.58731, -0.18011, -0.40721, - -0.60639, 2.03422, -0.42782, - -0.19612, -1.69203, 2.88815 + 2.12089, -0.52461, -0.59629, + -0.85342, 2.80445, -0.95103, + -0.26897, -1.14788, 2.41685 ] }, { - "ct": 2811, + "ct": 2960, "ccm": [ - 1.61593, -0.33164, -0.28429, - -0.55048, 1.97779, -0.42731, - -0.12042, -1.42847, 2.54889 + 2.26962, -0.54174, -0.72789, + -0.77008, 2.60271, -0.83262, + -0.26036, -1.51254, 2.77289 ] }, { - "ct": 2911, + "ct": 3603, "ccm": [ - 1.62771, -0.41282, -0.21489, - -0.57991, 2.04176, -0.46186, - -0.07613, -1.13359, 2.20972 + 2.18644, -0.66148, -0.52496, + -0.77828, 2.69474, -0.91645, + -0.25239, -0.83059, 2.08298 ] }, { - "ct": 2919, + "ct": 4650, "ccm": [ - 1.62661, -0.37736, -0.24925, - -0.52519, 1.95233, -0.42714, - -0.10842, -1.34929, 2.45771 + 2.18174, -0.70887, -0.47287, + -0.70196, 2.76426, -1.06231, + -0.25157, -0.71978, 1.97135 ] }, { - "ct": 3627, + "ct": 5858, "ccm": [ - 1.70385, -0.57231, -0.13154, - -0.47763, 1.85998, -0.38235, - -0.07467, -0.82678, 1.90145 + 2.32392, -0.88421, -0.43971, + -0.63821, 2.58348, -0.94527, + -0.28541, -0.54112, 1.82653 ] }, { - "ct": 4600, + "ct": 7580, "ccm": [ - 1.68486, -0.61085, -0.07402, - -0.41927, 2.04016, -0.62089, - -0.08633, -0.67672, 1.76305 - ] - }, - { - "ct": 5716, - "ccm": - [ - 1.80439, -0.73699, -0.06739, - -0.36073, 1.83327, -0.47255, - -0.08378, -0.56403, 1.64781 - ] - }, - { - "ct": 8575, - "ccm": - [ - 1.89357, -0.76427, -0.12931, - -0.27399, 2.15605, -0.88206, - -0.12035, -0.68256, 1.80292 + 2.21175, -0.53242, -0.67933, + -0.57875, 3.07922, -1.50047, + -0.27709, -0.73338, 2.01048 ] } ] diff --git a/src/ipa/rpi/vc4/data/imx296.json b/src/ipa/rpi/vc4/data/imx296.json index 9330d0a9..7621f759 100644 --- a/src/ipa/rpi/vc4/data/imx296.json +++ b/src/ipa/rpi/vc4/data/imx296.json @@ -357,174 +357,66 @@ { "ccms": [ { - "ct": 2000, + "ct": 2500, "ccm": [ - 1.48716, -0.1877, -0.35079, - -0.48577, 1.55088, -0.03387, - 0.24919, -1.4583, 2.12083 - ] - }, - { - "ct": 2200, - "ccm": - [ - 1.53439, -0.28852, -0.29392, - -0.44748, 1.56295, -0.08907, - 0.23529, -1.30488, 1.99784 - ] - }, - { - "ct": 2400, - "ccm": - [ - 1.57619, -0.36904, -0.25181, - -0.41654, 1.57046, -0.13192, - 0.21678, -1.18352, 1.90786 - ] - }, - { - "ct": 2600, - "ccm": - [ - 1.61348, -0.43497, -0.2198, - -0.39075, 1.5753, -0.1665, - 0.19789, -1.08592, 1.83942 + 1.95054, -0.57435, -0.37619, + -0.46945, 1.86661, -0.39716, + 0.07977, -1.14072, 2.06095 ] }, { "ct": 2800, "ccm": [ - 1.64717, -0.49009, -0.1951, - -0.36881, 1.57852, -0.1952, - 0.18016, -1.00609, 1.78575 - ] - }, - { - "ct": 3000, - "ccm": - [ - 1.67798, -0.53693, -0.17591, - -0.34986, 1.58074, -0.21955, - 0.16406, -0.9398, 1.74261 - ] - }, - { - "ct": 3200, - "ccm": - [ - 1.70647, -0.5773, -0.161, - -0.33332, 1.58235, -0.24056, - 0.14961, -0.88398, 1.70721 - ] - }, - { - "ct": 3400, - "ccm": - [ - 1.73305, -0.61248, -0.14951, - -0.31875, 1.58355, -0.25894, - 0.13671, -0.83642, 1.67769 + 1.94104, -0.60261, -0.33844, + -0.43162, 1.85422, -0.42261, + 0.03799, -0.95022, 1.91222 ] }, { - "ct": 3600, + "ct": 2900, "ccm": [ - 1.75802, -0.64343, -0.14077, - -0.30581, 1.5845, -0.27518, - 0.12518, -0.79546, 1.65271 + 1.91828, -0.59569, -0.32258, + -0.51902, 2.09091, -0.57189, + -0.03324, -0.73462, 1.76785 ] }, { - "ct": 4100, + "ct": 3620, "ccm": [ - 1.78116, -0.67459, -0.13048, - -0.26859, 1.58692, -0.31929, - 0.11915, -0.77931, 1.64012 + 1.97199, -0.66403, -0.30797, + -0.46411, 2.02612, -0.56201, + -0.07764, -0.61178, 1.68942 ] }, { - "ct": 4600, + "ct": 4560, "ccm": [ - 1.83867, -0.73605, -0.12044, - -0.24947, 1.58699, -0.34207, - 0.09949, -0.71041, 1.59842 - ] - }, - { - "ct": 5100, - "ccm": - [ - 1.88967, -0.78455, -0.11744, - -0.23398, 1.58806, -0.36172, - 0.08362, -0.6574, 1.56728 + 2.15256, -0.84787, -0.30469, + -0.48422, 2.28962, -0.80541, + -0.15113, -0.53014, 1.68127 ] }, { "ct": 5600, "ccm": [ - 1.93485, -0.82318, -0.1191, - -0.22108, 1.58973, -0.37892, - 0.07074, -0.61609, 1.54362 - ] - }, - { - "ct": 6100, - "ccm": - [ - 1.97481, -0.85423, -0.12371, - -0.21015, 1.59169, -0.39406, - 0.06021, -0.58353, 1.52536 - ] - }, - { - "ct": 6600, - "ccm": - [ - 2.01029, -0.87946, -0.13017, - -0.20074, 1.59378, -0.4075, - 0.05146, -0.55732, 1.51096 - ] - }, - { - "ct": 7100, - "ccm": - [ - 2.04183, -0.9002, -0.13765, - -0.19255, 1.59586, -0.41944, - 0.04414, -0.53603, 1.49947 - ] - }, - { - "ct": 7600, - "ccm": - [ - 2.07001, -0.91744, -0.14566, - -0.18534, 1.59788, -0.43013, - 0.03791, -0.51841, 1.49013 - ] - }, - { - "ct": 8100, - "ccm": - [ - 2.09534, -0.93195, -0.15388, - -0.17893, 1.59981, -0.43974, - 0.03256, -0.50364, 1.48243 + 2.04576, -0.74771, -0.29805, + -0.36332, 1.98993, -0.62662, + -0.09328, -0.46543, 1.55871 ] }, { - "ct": 8600, + "ct": 7400, "ccm": [ - 2.11799, -0.94416, -0.16203, - -0.17324, 1.60161, -0.44836, - 0.02795, -0.4912, 1.47604 + 2.37532, -0.83069, -0.54462, + -0.48279, 2.84309, -1.36031, + -0.21178, -0.66532, 1.87709 ] } ] diff --git a/src/ipa/rpi/vc4/data/imx477.json b/src/ipa/rpi/vc4/data/imx477.json index daffc268..0e39d419 100644 --- a/src/ipa/rpi/vc4/data/imx477.json +++ b/src/ipa/rpi/vc4/data/imx477.json @@ -410,102 +410,57 @@ { "ccms": [ { - "ct": 2360, + "ct": 2850, "ccm": [ - 1.66078, -0.23588, -0.42491, - -0.47456, 1.82763, -0.35307, - -0.00545, -1.44729, 2.45273 + 1.97469, -0.71439, -0.26031, + -0.43521, 2.09769, -0.66248, + -0.04826, -0.84642, 1.89468 ] }, { - "ct": 2870, - "ccm": - [ - 1.78373, -0.55344, -0.23029, - -0.39951, 1.69701, -0.29751, - 0.01986, -1.06525, 2.04539 - ] - }, - { - "ct": 2970, - "ccm": - [ - 1.73511, -0.56973, -0.16537, - -0.36338, 1.69878, -0.33539, - -0.02354, -0.76813, 1.79168 - ] - }, - { - "ct": 3000, - "ccm": - [ - 2.06374, -0.92218, -0.14156, - -0.41721, 1.69289, -0.27568, - -0.00554, -0.92741, 1.93295 - ] - }, - { - "ct": 3700, - "ccm": - [ - 2.13792, -1.08136, -0.05655, - -0.34739, 1.58989, -0.24249, - -0.00349, -0.76789, 1.77138 - ] - }, - { - "ct": 3870, - "ccm": - [ - 1.83834, -0.70528, -0.13307, - -0.30499, 1.60523, -0.30024, - -0.05701, -0.58313, 1.64014 - ] - }, - { - "ct": 4000, + "ct": 2960, "ccm": [ - 2.15741, -1.10295, -0.05447, - -0.34631, 1.61158, -0.26528, - -0.02723, -0.70288, 1.73011 + 2.12952, -0.91185, -0.21768, + -0.38018, 1.90789, -0.52771, + 0.03988, -1.10079, 2.06092 ] }, { - "ct": 4400, + "ct": 3580, "ccm": [ - 2.05729, -0.95007, -0.10723, - -0.41712, 1.78606, -0.36894, - -0.11899, -0.55727, 1.67626 + 2.03422, -0.80048, -0.23374, + -0.39089, 1.97221, -0.58132, + -0.08969, -0.61439, 1.70408 ] }, { - "ct": 4715, + "ct": 4559, "ccm": [ - 1.90255, -0.77478, -0.12777, - -0.31338, 1.88197, -0.56858, - -0.06001, -0.61785, 1.67786 + 2.15423, -0.98143, -0.17279, + -0.38131, 2.14763, -0.76632, + -0.10069, -0.54383, 1.64452 ] }, { - "ct": 5920, + "ct": 5881, "ccm": [ - 1.98691, -0.84671, -0.14019, - -0.26581, 1.70615, -0.44035, - -0.09532, -0.47332, 1.56864 + 2.18464, -0.95493, -0.22971, + -0.36826, 2.00298, -0.63471, + -0.15219, -0.38055, 1.53274 ] }, { - "ct": 9050, + "ct": 7600, "ccm": [ - 2.09255, -0.76541, -0.32714, - -0.28973, 2.27462, -0.98489, - -0.17299, -0.61275, 1.78574 + 2.30687, -0.97295, -0.33392, + -0.30872, 2.32779, -1.01908, + -0.17761, -0.55891, 1.73651 ] } ] diff --git a/src/ipa/rpi/vc4/data/imx708.json b/src/ipa/rpi/vc4/data/imx708.json index 6b26d0a1..c40a5994 100644 --- a/src/ipa/rpi/vc4/data/imx708.json +++ b/src/ipa/rpi/vc4/data/imx708.json @@ -368,147 +368,48 @@ { "ccms": [ { - "ct": 2498, + "ct": 2964, "ccm": [ - 1.14912, 0.28638, -0.43551, - -0.49691, 1.60391, -0.10701, - -0.10513, -1.09534, 2.20047 + 1.72129, -0.45961, -0.26169, + -0.30042, 1.56924, -0.26882, + 0.15133, -1.13293, 1.98161 ] }, { - "ct": 2821, + "ct": 3610, "ccm": [ - 1.18251, 0.15501, -0.33752, - -0.44304, 1.58495, -0.14191, - -0.05077, -0.96422, 2.01498 + 1.54474, -0.35082, -0.19391, + -0.36989, 1.67926, -0.30936, + -0.00524, -0.55197, 1.55722 ] }, { - "ct": 2925, + "ct": 4640, "ccm": [ - 1.18668, 0.00195, -0.18864, - -0.41617, 1.50514, -0.08897, - -0.02675, -0.91143, 1.93818 + 1.52972, -0.35168, -0.17804, + -0.28309, 1.67098, -0.38788, + 0.01695, -0.57209, 1.55515 ] }, { - "ct": 2926, + "ct": 5910, "ccm": [ - 1.50948, -0.44421, -0.06527, - -0.37241, 1.41726, -0.04486, - 0.07098, -0.84694, 1.77596 + 1.56879, -0.42159, -0.14719, + -0.27275, 1.59354, -0.32079, + -0.02862, -0.40662, 1.43525 ] }, { - "ct": 2951, + "ct": 7590, "ccm": [ - 1.52743, -0.47333, -0.05411, - -0.36485, 1.40764, -0.04279, - 0.08672, -0.90479, 1.81807 - ] - }, - { - "ct": 2954, - "ccm": - [ - 1.51683, -0.46841, -0.04841, - -0.36288, 1.39914, -0.03625, - 0.06421, -0.82034, 1.75613 - ] - }, - { - "ct": 3578, - "ccm": - [ - 1.59888, -0.59105, -0.00784, - -0.29366, 1.32037, -0.02671, - 0.06627, -0.76465, 1.69838 - ] - }, - { - "ct": 3717, - "ccm": - [ - 1.59063, -0.58059, -0.01003, - -0.29583, 1.32715, -0.03132, - 0.03613, -0.67431, 1.63817 - ] - }, - { - "ct": 3784, - "ccm": - [ - 1.59379, -0.58861, -0.00517, - -0.29178, 1.33292, -0.04115, - 0.03541, -0.66162, 1.62622 - ] - }, - { - "ct": 4485, - "ccm": - [ - 1.40761, -0.34561, -0.06201, - -0.32388, 1.57221, -0.24832, - -0.01014, -0.63427, 1.64441 - ] - }, - { - "ct": 4615, - "ccm": - [ - 1.41537, -0.35832, -0.05705, - -0.31429, 1.56019, -0.24591, - -0.01761, -0.61859, 1.63621 - ] - }, - { - "ct": 4671, - "ccm": - [ - 1.42941, -0.38178, -0.04764, - -0.31421, 1.55925, -0.24504, - -0.01141, -0.62987, 1.64129 - ] - }, - { - "ct": 5753, - "ccm": - [ - 1.64549, -0.63329, -0.01221, - -0.22431, 1.36423, -0.13992, - -0.00831, -0.55373, 1.56204 - ] - }, - { - "ct": 5773, - "ccm": - [ - 1.63668, -0.63557, -0.00111, - -0.21919, 1.36234, -0.14315, - -0.00399, -0.57428, 1.57827 - ] - }, - { - "ct": 7433, - "ccm": - [ - 1.36007, -0.09277, -0.26729, - -0.36886, 2.09249, -0.72363, - -0.12573, -0.76761, 1.89334 - ] - }, - { - "ct": 55792, - "ccm": - [ - 1.65091, -0.63689, -0.01401, - -0.22277, 1.35752, -0.13475, - -0.00943, -0.55091, 1.56033 + 1.41424, -0.21092, -0.20332, + -0.17646, 1.71734, -0.54087, + 0.01297, -0.63111, 1.61814 ] } ] diff --git a/src/ipa/rpi/vc4/data/imx708_wide.json b/src/ipa/rpi/vc4/data/imx708_wide.json index 14bc918e..65543628 100644 --- a/src/ipa/rpi/vc4/data/imx708_wide.json +++ b/src/ipa/rpi/vc4/data/imx708_wide.json @@ -358,48 +358,57 @@ { "ccms": [ { - "ct": 2750, + "ct": 2868, "ccm": [ - 1.13004, 0.36392, -0.49396, - -0.45885, 1.68171, -0.22286, - -0.06473, -0.86962, 1.93435 + 1.58923, -0.36649, -0.22273, + -0.43591, 1.84858, -0.41268, + 0.02948, -0.77666, 1.74718 ] }, { - "ct": 2940, + "ct": 2965, "ccm": [ - 1.29876, 0.09627, -0.39503, - -0.43085, 1.60258, -0.17172, - -0.02638, -0.92581, 1.95218 + 1.73397, -0.42794, -0.30603, + -0.36504, 1.72431, -0.35926, + 0.12765, -1.10933, 1.98168 ] }, { - "ct": 3650, + "ct": 3603, "ccm": [ - 1.57729, -0.29734, -0.27995, - -0.42965, 1.66231, -0.23265, - -0.02183, -0.62331, 1.64514 + 1.61787, -0.42704, -0.19084, + -0.37819, 1.74588, -0.36769, + 0.00961, -0.59807, 1.58847 ] }, { - "ct": 4625, + "ct": 4620, "ccm": [ - 1.52145, -0.22382, -0.29763, - -0.40445, 1.82186, -0.41742, - -0.05732, -0.56222, 1.61954 + 1.55581, -0.35422, -0.20158, + -0.31805, 1.79309, -0.47505, + -0.01256, -0.54489, 1.55746 ] }, { - "ct": 5715, + "ct": 5901, "ccm": [ - 1.67851, -0.39193, -0.28658, - -0.37169, 1.72949, -0.35781, - -0.09556, -0.41951, 1.51508 + 1.64439, -0.48855, -0.15585, + -0.29149, 1.67122, -0.37972, + -0.03111, -0.44052, 1.47163 + ] + }, + { + "ct": 7610, + "ccm": + [ + 1.48667, -0.26072, -0.22595, + -0.21815, 1.86724, -0.64909, + -0.00985, -0.64485, 1.65471 ] } ] diff --git a/src/ipa/rpi/vc4/data/ov5647.json b/src/ipa/rpi/vc4/data/ov5647.json index d770e44f..a1b42a18 100644 --- a/src/ipa/rpi/vc4/data/ov5647.json +++ b/src/ipa/rpi/vc4/data/ov5647.json @@ -406,75 +406,57 @@ { "ccms": [ { - "ct": 2500, + "ct": 2873, "ccm": [ - 1.70741, -0.05307, -0.65433, - -0.62822, 1.68836, -0.06014, - -0.04452, -1.87628, 2.92079 + 1.88195, -0.26249, -0.61946, + -0.63842, 2.11535, -0.47693, + -0.13531, -0.99739, 2.13271 ] }, { - "ct": 2803, + "ct": 2965, "ccm": [ - 1.74383, -0.18731, -0.55652, - -0.56491, 1.67772, -0.11281, - -0.01522, -1.60635, 2.62157 + 2.15048, -0.51859, -0.63189, + -0.53572, 1.92585, -0.39013, + 0.01831, -1.48576, 2.46744 ] }, { - "ct": 2912, + "ct": 3606, "ccm": [ - 1.75215, -0.22221, -0.52995, - -0.54568, 1.63522, -0.08954, - 0.02633, -1.56997, 2.54364 + 1.97522, -0.43847, -0.53675, + -0.56151, 1.99765, -0.43614, + -0.12438, -0.77056, 1.89493 ] }, { - "ct": 2914, + "ct": 4700, "ccm": [ - 1.72423, -0.28939, -0.43484, - -0.55188, 1.62925, -0.07737, - 0.01959, -1.28661, 2.26702 + 2.00971, -0.51461, -0.49511, + -0.52109, 2.01003, -0.48894, + -0.09527, -0.67318, 1.76845 ] }, { - "ct": 3605, + "ct": 5890, "ccm": [ - 1.80381, -0.43646, -0.36735, - -0.46505, 1.56814, -0.10309, - 0.00929, -1.00424, 1.99495 + 2.13616, -0.65283, -0.48333, + -0.48364, 1.93115, -0.44751, + -0.13465, -0.54831, 1.68295 ] }, { - "ct": 4540, + "ct": 7600, "ccm": [ - 1.85263, -0.46545, -0.38719, - -0.44136, 1.68443, -0.24307, - 0.04108, -0.85599, 1.81491 - ] - }, - { - "ct": 5699, - "ccm": - [ - 1.98595, -0.63542, -0.35054, - -0.34623, 1.54146, -0.19522, - 0.00411, -0.70936, 1.70525 - ] - }, - { - "ct": 8625, - "ccm": - [ - 2.21637, -0.56663, -0.64974, - -0.41133, 1.96625, -0.55492, - -0.02307, -0.83529, 1.85837 + 2.06599, -0.39161, -0.67439, + -0.50883, 2.27467, -0.76583, + -0.13961, -0.66121, 1.80081 ] } ] From patchwork Fri Jul 7 03:17:02 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ben Benson X-Patchwork-Id: 18818 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 0F7CBC3243 for ; Thu, 13 Jul 2023 10:35:25 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id A5098628BE; Thu, 13 Jul 2023 12:35:24 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1689244524; bh=jw+82qi0/UuzPpsk/6k6piAp+aRX3cyESCpbl+L9cMk=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To:Cc: From; b=IBvbIxf8T+2Av3q3r9W62GfHUMHkfhXSdsIn0ovOrK1UV8v/vCEt+WgarWeGI98aQ GjHRC3d2i0i+32tAAFeDAJrrG8Tvhqx56JKdyK1qlyk+0VK1unc2uzt5h/rOIZzOV/ ogp0eIlAulRtHNdlw/Hu8V5lBPPpKUwnsG7/N+pPo9CH4mTwDO2aIoG1/uFbW0ypFW uvuvPN0xtO+ayo1YfMTYf8EYIcbSgnIx52vOsAKoDR9HJ9lNyv5n7mTxSzktxO06Gz lAtcnZIUEzHcnV8GC5dcXa5/jlrWd2tK0kdD37eY6ozOrR8cQpfO7YEU0pFhIYZUIn 0s05o96Xz497w== Received: from mail-wm1-x332.google.com (mail-wm1-x332.google.com [IPv6:2a00:1450:4864:20::332]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 00EBB628BC for ; Thu, 13 Jul 2023 12:35:20 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (2048-bit key; unprotected) header.d=raspberrypi.com header.i=@raspberrypi.com header.b="sNkbTw9c"; dkim-atps=neutral Received: by mail-wm1-x332.google.com with SMTP id 5b1f17b1804b1-3fbc59de009so4449075e9.3 for ; Thu, 13 Jul 2023 03:35:19 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; t=1689244519; x=1691836519; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=uDUF0jTZCrHu9jkvf3VStWYgCNFIJCcJisHmCTREtxY=; b=sNkbTw9cHlceE5z9cUiWHPNsARgYKxSjg8EaexuOE5ZCMOb9iX0a5SDAxQPvVQ7y6t hzMSxs8rkcSLoBpcpZMh05l5GOfPNeChSTvkbAk+SiAPchyeD+3hZyM9K+F5jJKVmfqi XtUJUnBLpF4ENHXPHEShIdKuVHtVcV4PVrnWXMbHzTu+g9ZW1g1WaRHPCtuyD9s6v4Wd HIm+tE9U8qmXWEqZmbxeUV+VwRQp3cN4UItRrevdjqxjIjt9hEdlkuQ9l3sLTNwC9q9P F2ydRu8ZvSF5sH5px7AWu+XYhJm1d8Wgv2J+uC5u/dTaFRDsNnEpgvhH+VW9ZdvEvtmX SEXg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1689244519; x=1691836519; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=uDUF0jTZCrHu9jkvf3VStWYgCNFIJCcJisHmCTREtxY=; b=fB4/EVjZpkm/8mDoLqIYBESEz5ywR4x3N025Za0miPBBNHcO5sOvqNwlAF7ZAmhGyo QDaT12nls6yxOPZ1y3glbyIqlxyIduQMTvEib8yAwwW+8ftKwYBfhZocMxEHm+Ht21wk C2+pspcBWeZwMH2NEuv4rkYYmm6SjQwyb8NsHOC9VPv2qAP0d+7K9l3eztfeSCzTJsSb 8y4HWaXeZ34mn7ZlTGeNKt82cKp4FeE092WqHWjxgObzIhxySPrd7qRjyK0OHBFP17xu w96wlFWbL/I/5tY859g6EB97ftbyBWILPdbcGWO6A0e8U3vpJ249XOXnXk9KCVWQqsBa k5LQ== X-Gm-Message-State: ABy/qLYQkWS7S7OCJj6d71mPqjDpDnKXm0Xdt1A72n2gADWpDRigpmU3 djWSJE+kU371CyZ0SrriJxYquHkIjGIhDgojwZEOVTmo X-Google-Smtp-Source: APBJJlF/7g/yRz47991wE4qKUA33UGYdJcYKj8Dg4ROpcAkbhP3lVAn3Q89KU7RirjvMWiIgVWTxOA== X-Received: by 2002:a5d:4bc7:0:b0:313:f862:6e3e with SMTP id l7-20020a5d4bc7000000b00313f8626e3emr1049626wrt.40.1689244519325; Thu, 13 Jul 2023 03:35:19 -0700 (PDT) Received: from localhost.localdomain ([93.93.133.154]) by smtp.gmail.com with ESMTPSA id r8-20020a056000014800b0030fa3567541sm7551143wrx.48.2023.07.13.03.35.18 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 13 Jul 2023 03:35:19 -0700 (PDT) To: libcamera-devel@lists.libcamera.org Date: Fri, 7 Jul 2023 04:17:02 +0100 Message-Id: <20230707031702.603261-4-ben.benson@raspberrypi.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230707031702.603261-1-ben.benson@raspberrypi.com> References: <20230707031702.603261-1-ben.benson@raspberrypi.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v2 3/3] utils: raspberrypi: ctt: Code tidying X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Ben Benson via libcamera-devel From: Ben Benson Reply-To: Ben Benson Cc: Ben Benson Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Altered the way that some lines are laid out, made functions more attractive to look at, and tidied up messy areas. Signed-off-by: Ben Benson --- utils/raspberrypi/ctt/ctt_ccm.py | 61 ++++++++++++++------------------ 1 file changed, 27 insertions(+), 34 deletions(-) diff --git a/utils/raspberrypi/ctt/ctt_ccm.py b/utils/raspberrypi/ctt/ctt_ccm.py index 49159535..a09bfd09 100644 --- a/utils/raspberrypi/ctt/ctt_ccm.py +++ b/utils/raspberrypi/ctt/ctt_ccm.py @@ -47,11 +47,8 @@ def degamma(x): def gamma(x): - # return (x * * (1 / 2.4) * 1.055 - 0.055) - e = [] - for i in range(len(x)): - e.append(((x[i] / 255) ** (1 / 2.4) * 1.055 - 0.055) * 255) - return e + # Take 3 long array of color values and gamma them + return [((colour / 255) ** (1 / 2.4) * 1.055 - 0.055) * 255 for colour in x] """ @@ -96,10 +93,8 @@ def ccm(Cam, cal_cr_list, cal_cb_list): """ m_srgb = degamma(m_rgb) # now in 16 bit color. - m_lab = [] - for col in m_srgb: - m_lab.append(colors.RGB_to_LAB(col / 256)) - # This produces matrix of LAB values for ideal color chart) + # Produce array of LAB values for ideal color chart + m_lab = [colors.RGB_to_LAB(color / 256) for color in m_srgb] """ reorder reference values to match how patches are ordered @@ -168,7 +163,7 @@ def ccm(Cam, cal_cr_list, cal_cb_list): sumde = 0 ccm = do_ccm(r, g, b, m_srgb) # This is the initial guess that our optimisation code works with. - + original_ccm = ccm r1 = ccm[0] r2 = ccm[1] g1 = ccm[3] @@ -188,7 +183,7 @@ def ccm(Cam, cal_cr_list, cal_cb_list): We use our old CCM as the initial guess for the program to find the optimised matrix ''' - result = minimize(guess, x0, args=(r, g, b, m_lab), tol=0.0000000001) + result = minimize(guess, x0, args=(r, g, b, m_lab), tol=0.01) ''' This produces a color matrix which has the lowest delta E possible, based off the input data. Note it is impossible for this to reach @@ -199,12 +194,13 @@ def ccm(Cam, cal_cr_list, cal_cb_list): [r1, r2, g1, g2, b1, b2] = result.x # The new, optimised color correction matrix values optimised_ccm = [r1, r2, (1 - r1 - r2), g1, g2, (1 - g1 - g2), b1, b2, (1 - b1 - b2)] + # This is the optimised Color Matrix (preserving greys by summing rows up to 1) Cam.log += str(optimised_ccm) Cam.log += "\n Old Color Correction Matrix Below \n" Cam.log += str(ccm) - formatted_ccm = np.array(ccm).reshape((3, 3)) + formatted_ccm = np.array(original_ccm).reshape((3, 3)) ''' below is a whole load of code that then applies the latest color @@ -213,22 +209,21 @@ def ccm(Cam, cal_cr_list, cal_cb_list): ''' optimised_ccm_rgb = [] # Original Color Corrected Matrix RGB / LAB optimised_ccm_lab = [] - for w in range(24): - RGB = np.array([r[w], g[w], b[w]]) - ccm_applied_rgb = np.dot(formatted_ccm, (RGB / 256)) - optimised_ccm_rgb.append(gamma(ccm_applied_rgb)) - optimised_ccm_lab.append(colors.RGB_to_LAB(ccm_applied_rgb)) - formatted_optimised_ccm = np.array(ccm).reshape((3, 3)) + formatted_optimised_ccm = np.array(optimised_ccm).reshape((3, 3)) after_gamma_rgb = [] after_gamma_lab = [] - for w in range(24): - RGB = np.array([r[w], g[w], b[w]]) - optimised_ccm_applied_rgb = np.dot(formatted_optimised_ccm, RGB / 256) + + for RGB in zip(r, g, b): + ccm_applied_rgb = np.dot(formatted_ccm, (np.array(RGB) / 256)) + optimised_ccm_rgb.append(gamma(ccm_applied_rgb)) + optimised_ccm_lab.append(colors.RGB_to_LAB(ccm_applied_rgb)) + + optimised_ccm_applied_rgb = np.dot(formatted_optimised_ccm, np.array(RGB) / 256) after_gamma_rgb.append(gamma(optimised_ccm_applied_rgb)) after_gamma_lab.append(colors.RGB_to_LAB(optimised_ccm_applied_rgb)) ''' - Gamma After RGB / LAB + Gamma After RGB / LAB - not used in calculations, only used for visualisation We now want to spit out some data that shows how the optimisation has improved the color matrices ''' @@ -303,9 +298,8 @@ def guess(x0, r, g, b, m_lab): # provides a method of numerical feedback f def transform_and_evaluate(ccm, r, g, b, m_lab): # Transforms colors to LAB and applies the correction matrix # create list of matrix changed colors realrgb = [] - for i in range(len(r)): - RGB = np.array([r[i], g[i], b[i]]) - rgb_post_ccm = np.dot(ccm, RGB) # This is RGB values after the color correction matrix has been applied + for RGB in zip(r, g, b): + rgb_post_ccm = np.dot(ccm, np.array(RGB) / 256) # This is RGB values after the color correction matrix has been applied realrgb.append(colors.RGB_to_LAB(rgb_post_ccm)) # now compare that with m_lab and return numeric result, averaged for each patch return (sumde(realrgb, m_lab) / 24) # returns an average result of delta E @@ -315,12 +309,12 @@ def sumde(listA, listB): global typenum, test_patches sumde = 0 maxde = 0 - patchde = [] - for i in range(len(listA)): - if maxde < (deltae(listA[i], listB[i])): - maxde = deltae(listA[i], listB[i]) - patchde.append(deltae(listA[i], listB[i])) - sumde += deltae(listA[i], listB[i]) + patchde = [] # Create array of the delta E values for each patch. useful for optimisation of certain patches + for listA_item, listB_item in zip(listA, listB): + if maxde < (deltae(listA_item, listB_item)): + maxde = deltae(listA_item, listB_item) + patchde.append(deltae(listA_item, listB_item)) + sumde += deltae(listA_item, listB_item) ''' The different options specified at the start allow for the maximum to be returned, average or specific patches @@ -330,9 +324,8 @@ def sumde(listA, listB): if typenum == 1: return maxde if typenum == 2: - output = 0 - for y in range(len(test_patches)): - output += patchde[test_patches[y]] # grabs the specific patches (no need for averaging here) + output = sum([patchde[test_patch] for test_patch in test_patches]) + # Selects only certain patches and returns the output for them return output