{"id":17336,"url":"https://patchwork.libcamera.org/api/patches/17336/?format=json","web_url":"https://patchwork.libcamera.org/patch/17336/","project":{"id":1,"url":"https://patchwork.libcamera.org/api/projects/1/?format=json","name":"libcamera","link_name":"libcamera","list_id":"libcamera_core","list_email":"libcamera-devel@lists.libcamera.org","web_url":"","scm_url":"","webscm_url":""},"msgid":"<20220908014200.28728-28-laurent.pinchart@ideasonboard.com>","date":"2022-09-08T01:41:55","name":"[libcamera-devel,v4,27/32] ipa: rkisp1: awb: Add support for RGB means","commit_ref":null,"pull_url":null,"state":"superseded","archived":false,"hash":"29666dc0040aeb4b2c18fcfe181ac84816ec1a3c","submitter":{"id":2,"url":"https://patchwork.libcamera.org/api/people/2/?format=json","name":"Laurent Pinchart","email":"laurent.pinchart@ideasonboard.com"},"delegate":null,"mbox":"https://patchwork.libcamera.org/patch/17336/mbox/","series":[{"id":3476,"url":"https://patchwork.libcamera.org/api/series/3476/?format=json","web_url":"https://patchwork.libcamera.org/project/libcamera/list/?series=3476","date":"2022-09-08T01:41:28","name":"ipa: Frame context queue, IPU3 & RkISP consolidation, and RkISP1 improvements","version":4,"mbox":"https://patchwork.libcamera.org/series/3476/mbox/"}],"comments":"https://patchwork.libcamera.org/api/patches/17336/comments/","check":"pending","checks":"https://patchwork.libcamera.org/api/patches/17336/checks/","tags":{},"headers":{"Return-Path":"","X-Original-To":"parsemail@patchwork.libcamera.org","Delivered-To":"parsemail@patchwork.libcamera.org","Received":["from lancelot.ideasonboard.com (lancelot.ideasonboard.com\n\t[92.243.16.209])\n\tby patchwork.libcamera.org (Postfix) with ESMTPS id BCB66C327D\n\tfor ;\n\tThu, 8 Sep 2022 01:42:57 +0000 (UTC)","from lancelot.ideasonboard.com (localhost [IPv6:::1])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTP id 837F062100;\n\tThu, 8 Sep 2022 03:42:57 +0200 (CEST)","from perceval.ideasonboard.com (perceval.ideasonboard.com\n\t[IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647])\n\tby lancelot.ideasonboard.com (Postfix) with ESMTPS id AE836620F9\n\tfor ;\n\tThu, 8 Sep 2022 03:42:55 +0200 (CEST)","from pendragon.ideasonboard.com (62-78-145-57.bb.dnainternet.fi\n\t[62.78.145.57])\n\tby perceval.ideasonboard.com (Postfix) with ESMTPSA id 36976888;\n\tThu, 8 Sep 2022 03:42:55 +0200 (CEST)"],"DKIM-Signature":["v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org;\n\ts=mail; t=1662601377;\n\tbh=vnC1O9P9yN43lp++s9PPrSzIj+Ios4qwMqahe4dKzi4=;\n\th=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe:\n\tList-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To:Cc:\n\tFrom;\n\tb=miQgbX80ktrqGe3fISBzWQ/WC3eSmZsNID7uuxEYbv03gZ6x2Ltlkmg17HuvtbOYR\n\tPHBW1PoSoaHO3f5PGhkIBPw5hTKHKCz5Sd6tMNI4zG5LuBf+CxLq3ym7CSBPaERkfN\n\tdTjzSuhn6GEHt1JZMslHQ0wEEccaD+uMqdfOVnir3OfsPezj+uRIIuvt8ABIZh/98q\n\t6WPr0QaIWCo5tYSoBLPTQimg0uNDaVnYEkXtCQj8pG2GYvAReWVX+xwALCukTv46/G\n\tnGbw96w6Heh6kdnWtw9NIXsRBuzAgUNOdCHGEUAXrza7EjMJ+n2VLPImgquwkl6TiV\n\tzR3CDmF2jKcwQ==","v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com;\n\ts=mail; t=1662601375;\n\tbh=vnC1O9P9yN43lp++s9PPrSzIj+Ios4qwMqahe4dKzi4=;\n\th=From:To:Cc:Subject:Date:In-Reply-To:References:From;\n\tb=mPuqTj6plo8aRAeg3JjPhCaOuuiTh0V4Wt5hc7wmtTjm2Y+zHYIgbMq9nDe4vQrxd\n\txKuqkEjwD1LTyUfpi+o8ZU36VLtSH5PlvxTGwZV0gEFRcWgEsQOPmHyszr2k4Uz8+x\n\tJlViPI6oOOO0MgKUKJ4REVI3fEU15p9v8nzil62s="],"Authentication-Results":"lancelot.ideasonboard.com; dkim=pass (1024-bit key; \n\tunprotected) header.d=ideasonboard.com\n\theader.i=@ideasonboard.com\n\theader.b=\"mPuqTj6p\"; dkim-atps=neutral","To":"libcamera-devel@lists.libcamera.org","Date":"Thu, 8 Sep 2022 04:41:55 +0300","Message-Id":"<20220908014200.28728-28-laurent.pinchart@ideasonboard.com>","X-Mailer":"git-send-email 2.35.1","In-Reply-To":"<20220908014200.28728-1-laurent.pinchart@ideasonboard.com>","References":"<20220908014200.28728-1-laurent.pinchart@ideasonboard.com>","MIME-Version":"1.0","Content-Transfer-Encoding":"8bit","Subject":"[libcamera-devel] [PATCH v4 27/32] ipa: rkisp1: awb: Add support\n\tfor RGB means","X-BeenThere":"libcamera-devel@lists.libcamera.org","X-Mailman-Version":"2.1.29","Precedence":"list","List-Id":"","List-Unsubscribe":",\n\t","List-Archive":"","List-Post":"","List-Help":"","List-Subscribe":",\n\t","From":"Laurent Pinchart via libcamera-devel\n\t","Reply-To":"Laurent Pinchart ","Cc":"Quentin Schulz ","Errors-To":"libcamera-devel-bounces@lists.libcamera.org","Sender":"\"libcamera-devel\" "},"content":"From: Quentin Schulz \n\nRkISP actually supports two modes for color means, RGB and YCbCr. The\nvariables where the means are stored are identically named regardless of\nthe color means mode that's been selected.\n\nSince the gains are computed in RGB mode, a conversion needs to be done\nwhen the mode is YCbCr, which is unnecessary when RGB mode is selected.\n\nThis adds support for RGB means mode too, by checking at runtime which\nmode is selected at a given time. The default is still set to YCbCr mode\nfor now.\n\nCc: Quentin Schulz \nSigned-off-by: Quentin Schulz \nSigned-off-by: Laurent Pinchart \n---\nChanges since v1:\n\n- Fix compilation error by storing RGB mode flag in Awb class\n- Set ISP AWB configuration based on the selected mode\n---\n src/ipa/rkisp1/algorithms/awb.cpp | 130 +++++++++++++++++++-----------\n src/ipa/rkisp1/algorithms/awb.h | 4 +-\n 2 files changed, 86 insertions(+), 48 deletions(-)","diff":"diff --git a/src/ipa/rkisp1/algorithms/awb.cpp b/src/ipa/rkisp1/algorithms/awb.cpp\nindex b711e93b73ba..2ff67ed98221 100644\n--- a/src/ipa/rkisp1/algorithms/awb.cpp\n+++ b/src/ipa/rkisp1/algorithms/awb.cpp\n@@ -30,6 +30,11 @@ namespace ipa::rkisp1::algorithms {\n \n LOG_DEFINE_CATEGORY(RkISP1Awb)\n \n+Awb::Awb()\n+\t: rgbMode_(false)\n+{\n+}\n+\n /**\n * \\copydoc libcamera::ipa::Algorithm::configure\n */\n@@ -98,38 +103,60 @@ void Awb::prepare(IPAContext &context, const uint32_t frame,\n \t/* Update the gains. */\n \tparams->module_cfg_update |= RKISP1_CIF_ISP_MODULE_AWB_GAIN;\n \n-\t/* If we already have configured the gains and window, return. */\n+\t/* If we already have set the AWB measurement parameters, return. */\n \tif (frame > 0)\n \t\treturn;\n \n-\t/* Configure the gains to apply. */\n+\trkisp1_cif_isp_awb_meas_config &awb_config = params->meas.awb_meas_config;\n+\n+\t/* Configure the measure window for AWB. */\n+\tawb_config.awb_wnd = context.configuration.awb.measureWindow;\n+\n+\t/* Number of frames to use to estimate the means (0 means 1 frame). */\n+\tawb_config.frames = 0;\n+\n+\t/* Select RGB or YCbCr means measurement. */\n+\tif (rgbMode_) {\n+\t\tawb_config.awb_mode = RKISP1_CIF_ISP_AWB_MODE_RGB;\n+\n+\t\t/*\n+\t\t * For RGB-based measurements, pixels are selected with maximum\n+\t\t * red, green and blue thresholds that are set in the\n+\t\t * awb_ref_cr, awb_min_y and awb_ref_cb respectively. The other\n+\t\t * values are not used, set them to 0.\n+\t\t */\n+\t\tawb_config.awb_ref_cr = 250;\n+\t\tawb_config.min_y = 250;\n+\t\tawb_config.awb_ref_cb = 250;\n+\n+\t\tawb_config.max_y = 0;\n+\t\tawb_config.min_c = 0;\n+\t\tawb_config.max_csum = 0;\n+\t} else {\n+\t\tawb_config.awb_mode = RKISP1_CIF_ISP_AWB_MODE_YCBCR;\n+\n+\t\t/* Set the reference Cr and Cb (AWB target) to white. */\n+\t\tawb_config.awb_ref_cb = 128;\n+\t\tawb_config.awb_ref_cr = 128;\n+\n+\t\t/*\n+\t\t * Filter out pixels based on luminance and chrominance values.\n+\t\t * The acceptable luma values are specified as a [16, 250]\n+\t\t * range, while the acceptable chrome values are specified with\n+\t\t * a minimum of 16 and a maximum Cb+Cr sum of 250.\n+\t\t */\n+\t\tawb_config.min_y = 16;\n+\t\tawb_config.max_y = 250;\n+\t\tawb_config.min_c = 16;\n+\t\tawb_config.max_csum = 250;\n+\t}\n+\n+\t/* Enable the AWB gains. */\n \tparams->module_en_update |= RKISP1_CIF_ISP_MODULE_AWB_GAIN;\n-\t/* Update the ISP to apply the gains configured. */\n \tparams->module_ens |= RKISP1_CIF_ISP_MODULE_AWB_GAIN;\n \n-\t/* Configure the measure window for AWB. */\n-\tparams->meas.awb_meas_config.awb_wnd = context.configuration.awb.measureWindow;\n-\t/*\n-\t * Measure Y, Cr and Cb means.\n-\t * \\todo RGB is not working, the kernel seems to not configure it ?\n-\t */\n-\tparams->meas.awb_meas_config.awb_mode = RKISP1_CIF_ISP_AWB_MODE_YCBCR;\n-\t/* Reference Cr and Cb. */\n-\tparams->meas.awb_meas_config.awb_ref_cb = 128;\n-\tparams->meas.awb_meas_config.awb_ref_cr = 128;\n-\t/* Y values to include are between min_y and max_y only. */\n-\tparams->meas.awb_meas_config.min_y = 16;\n-\tparams->meas.awb_meas_config.max_y = 250;\n-\t/* Maximum Cr+Cb value to take into account for awb. */\n-\tparams->meas.awb_meas_config.max_csum = 250;\n-\t/* Minimum Cr and Cb values to take into account. */\n-\tparams->meas.awb_meas_config.min_c = 16;\n-\t/* Number of frames to use to estimate the mean (0 means 1 frame). */\n-\tparams->meas.awb_meas_config.frames = 0;\n-\n-\t/* Update AWB measurement unit configuration. */\n+\t/* Update the AWB measurement parameters and enable the AWB module. */\n \tparams->module_cfg_update |= RKISP1_CIF_ISP_MODULE_AWB;\n-\t/* Make sure the ISP is measuring the means for the next frame. */\n \tparams->module_en_update |= RKISP1_CIF_ISP_MODULE_AWB;\n \tparams->module_ens |= RKISP1_CIF_ISP_MODULE_AWB;\n }\n@@ -182,30 +209,39 @@ void Awb::process(IPAContext &context,\n \tconst rkisp1_cif_isp_stat *params = &stats->params;\n \tconst rkisp1_cif_isp_awb_stat *awb = ¶ms->awb;\n \tIPAActiveState &activeState = context.activeState;\n+\tdouble greenMean;\n+\tdouble redMean;\n+\tdouble blueMean;\n \n-\t/* Get the YCbCr mean values */\n-\tdouble yMean = awb->awb_mean[0].mean_y_or_g;\n-\tdouble crMean = awb->awb_mean[0].mean_cr_or_r;\n-\tdouble cbMean = awb->awb_mean[0].mean_cb_or_b;\n+\tif (rgbMode_) {\n+\t\tgreenMean = awb->awb_mean[0].mean_y_or_g;\n+\t\tredMean = awb->awb_mean[0].mean_cr_or_r;\n+\t\tblueMean = awb->awb_mean[0].mean_cb_or_b;\n+\t} else {\n+\t\t/* Get the YCbCr mean values */\n+\t\tdouble yMean = awb->awb_mean[0].mean_y_or_g;\n+\t\tdouble cbMean = awb->awb_mean[0].mean_cb_or_b;\n+\t\tdouble crMean = awb->awb_mean[0].mean_cr_or_r;\n \n-\t/*\n-\t * Convert from YCbCr to RGB.\n-\t * The hardware uses the following formulas:\n-\t * Y = 16 + 0.2500 R + 0.5000 G + 0.1094 B\n-\t * Cb = 128 - 0.1406 R - 0.2969 G + 0.4375 B\n-\t * Cr = 128 + 0.4375 R - 0.3750 G - 0.0625 B\n-\t *\n-\t * The inverse matrix is thus:\n-\t * [[1,1636, -0,0623, 1,6008]\n-\t * [1,1636, -0,4045, -0,7949]\n-\t * [1,1636, 1,9912, -0,0250]]\n-\t */\n-\tyMean -= 16;\n-\tcbMean -= 128;\n-\tcrMean -= 128;\n-\tdouble redMean = 1.1636 * yMean - 0.0623 * cbMean + 1.6008 * crMean;\n-\tdouble greenMean = 1.1636 * yMean - 0.4045 * cbMean - 0.7949 * crMean;\n-\tdouble blueMean = 1.1636 * yMean + 1.9912 * cbMean - 0.0250 * crMean;\n+\t\t/*\n+\t\t * Convert from YCbCr to RGB.\n+\t\t * The hardware uses the following formulas:\n+\t\t * Y = 16 + 0.2500 R + 0.5000 G + 0.1094 B\n+\t\t * Cb = 128 - 0.1406 R - 0.2969 G + 0.4375 B\n+\t\t * Cr = 128 + 0.4375 R - 0.3750 G - 0.0625 B\n+\t\t *\n+\t\t * The inverse matrix is thus:\n+\t\t * [[1,1636, -0,0623, 1,6008]\n+\t\t * [1,1636, -0,4045, -0,7949]\n+\t\t * [1,1636, 1,9912, -0,0250]]\n+\t\t */\n+\t\tyMean -= 16;\n+\t\tcbMean -= 128;\n+\t\tcrMean -= 128;\n+\t\tredMean = 1.1636 * yMean - 0.0623 * cbMean + 1.6008 * crMean;\n+\t\tgreenMean = 1.1636 * yMean - 0.4045 * cbMean - 0.7949 * crMean;\n+\t\tblueMean = 1.1636 * yMean + 1.9912 * cbMean - 0.0250 * crMean;\n+\t}\n \n \t/*\n \t * The ISP computes the AWB means after applying the colour gains,\ndiff --git a/src/ipa/rkisp1/algorithms/awb.h b/src/ipa/rkisp1/algorithms/awb.h\nindex 9fd156d8aff9..b1b861d930d1 100644\n--- a/src/ipa/rkisp1/algorithms/awb.h\n+++ b/src/ipa/rkisp1/algorithms/awb.h\n@@ -16,7 +16,7 @@ namespace ipa::rkisp1::algorithms {\n class Awb : public Algorithm\n {\n public:\n-\tAwb() = default;\n+\tAwb();\n \t~Awb() = default;\n \n \tint configure(IPAContext &context, const IPACameraSensorInfo &configInfo) override;\n@@ -32,6 +32,8 @@ public:\n \n private:\n \tuint32_t estimateCCT(double red, double green, double blue);\n+\n+\tbool rgbMode_;\n };\n \n } /* namespace ipa::rkisp1::algorithms */\n","prefixes":["libcamera-devel","v4","27/32"]}