From patchwork Tue Sep 27 02:36:37 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Laurent Pinchart X-Patchwork-Id: 17439 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id DAFDDC0DA4 for ; Tue, 27 Sep 2022 02:37:45 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 99053622ED; Tue, 27 Sep 2022 04:37:45 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=libcamera.org; s=mail; t=1664246265; bh=BRr1yWkgRo95TeUJpCd/hiAsq3WVLt7xHy3IY8/QPJs=; h=To:Date:In-Reply-To:References:Subject:List-Id:List-Unsubscribe: List-Archive:List-Post:List-Help:List-Subscribe:From:Reply-To:Cc: From; b=SpwSAx3WWywpjql11SotVDUQaJoWvAtkE2WO8KCdt/FU0khUw2mboy/nS+c+bg2EB WFSbjnEDUt9S/O20CjMVDukQwUO+bEcY9O86YstB1sIXQPfMLIpCz/0+L6aA6wzncw xfuVTUgmGzvPlmpQq6FUrin7do3PR67aa0Iz9PNhbK3c3QHW1gbM0VfyTFpp1VJSS6 5gy8PzllYSxe+wkhi9L2oHbTU1/v6z8c2WY3yFtYs+orKTdJRHZSpW6glVykE0hjdx U72BG0iohnRjGIMkVgEOuf8f+gvRqHtyI8ibk/h2pSQOH+5ZjRN0j7ruWFJl9s/msk an7Gd0tmTZfaA== Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id AC564622EA for ; Tue, 27 Sep 2022 04:37:43 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="MQ2orzer"; dkim-atps=neutral Received: from pendragon.ideasonboard.com (62-78-145-57.bb.dnainternet.fi [62.78.145.57]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 30444823; Tue, 27 Sep 2022 04:37:43 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1664246263; bh=BRr1yWkgRo95TeUJpCd/hiAsq3WVLt7xHy3IY8/QPJs=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=MQ2orzerlD8VyQizcSGmpFyIz3nyRhLOkrPLJ2EwnqbwF+YEM+98PM23e+h+SClEB FGPTcb2JhDWNgXOCmqQhiC1B37WhwRnoq3uLhgKz4DPJcJAiXHgVu4tZX7SiYbfanD PWd69wkWfiQdmx2Tb7fz0SWe5ENUKwCr909dJlKQ= To: libcamera-devel@lists.libcamera.org Date: Tue, 27 Sep 2022 05:36:37 +0300 Message-Id: <20220927023642.12341-29-laurent.pinchart@ideasonboard.com> X-Mailer: git-send-email 2.35.1 In-Reply-To: <20220927023642.12341-1-laurent.pinchart@ideasonboard.com> References: <20220927023642.12341-1-laurent.pinchart@ideasonboard.com> MIME-Version: 1.0 Subject: [libcamera-devel] [PATCH v5 28/33] ipa: rkisp1: awb: Add support for RGB means X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-Patchwork-Original-From: Laurent Pinchart via libcamera-devel From: Laurent Pinchart Reply-To: Laurent Pinchart Cc: Quentin Schulz Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" From: Quentin Schulz RkISP actually supports two modes for color means, RGB and YCbCr. The variables where the means are stored are identically named regardless of the color means mode that's been selected. Since the gains are computed in RGB mode, a conversion needs to be done when the mode is YCbCr, which is unnecessary when RGB mode is selected. This adds support for RGB means mode too, by checking at runtime which mode is selected at a given time. The default is still set to YCbCr mode for now. Cc: Quentin Schulz Signed-off-by: Quentin Schulz Signed-off-by: Laurent Pinchart Reviewed-by: Kieran Bingham Reviewed-by: Jacopo Mondi --- Changes since v4: - Fix grammar mistake and spelling error Changes since v1: - Fix compilation error by storing RGB mode flag in Awb class - Set ISP AWB configuration based on the selected mode --- src/ipa/rkisp1/algorithms/awb.cpp | 130 +++++++++++++++++++----------- src/ipa/rkisp1/algorithms/awb.h | 4 +- 2 files changed, 86 insertions(+), 48 deletions(-) diff --git a/src/ipa/rkisp1/algorithms/awb.cpp b/src/ipa/rkisp1/algorithms/awb.cpp index 694d97cccc07..cde5b981df61 100644 --- a/src/ipa/rkisp1/algorithms/awb.cpp +++ b/src/ipa/rkisp1/algorithms/awb.cpp @@ -30,6 +30,11 @@ namespace ipa::rkisp1::algorithms { LOG_DEFINE_CATEGORY(RkISP1Awb) +Awb::Awb() + : rgbMode_(false) +{ +} + /** * \copydoc libcamera::ipa::Algorithm::configure */ @@ -98,38 +103,60 @@ void Awb::prepare(IPAContext &context, const uint32_t frame, /* Update the gains. */ params->module_cfg_update |= RKISP1_CIF_ISP_MODULE_AWB_GAIN; - /* If we already have configured the gains and window, return. */ + /* If we have already set the AWB measurement parameters, return. */ if (frame > 0) return; - /* Configure the gains to apply. */ + rkisp1_cif_isp_awb_meas_config &awb_config = params->meas.awb_meas_config; + + /* Configure the measure window for AWB. */ + awb_config.awb_wnd = context.configuration.awb.measureWindow; + + /* Number of frames to use to estimate the means (0 means 1 frame). */ + awb_config.frames = 0; + + /* Select RGB or YCbCr means measurement. */ + if (rgbMode_) { + awb_config.awb_mode = RKISP1_CIF_ISP_AWB_MODE_RGB; + + /* + * For RGB-based measurements, pixels are selected with maximum + * red, green and blue thresholds that are set in the + * awb_ref_cr, awb_min_y and awb_ref_cb respectively. The other + * values are not used, set them to 0. + */ + awb_config.awb_ref_cr = 250; + awb_config.min_y = 250; + awb_config.awb_ref_cb = 250; + + awb_config.max_y = 0; + awb_config.min_c = 0; + awb_config.max_csum = 0; + } else { + awb_config.awb_mode = RKISP1_CIF_ISP_AWB_MODE_YCBCR; + + /* Set the reference Cr and Cb (AWB target) to white. */ + awb_config.awb_ref_cb = 128; + awb_config.awb_ref_cr = 128; + + /* + * Filter out pixels based on luminance and chrominance values. + * The acceptable luma values are specified as a [16, 250] + * range, while the acceptable chroma values are specified with + * a minimum of 16 and a maximum Cb+Cr sum of 250. + */ + awb_config.min_y = 16; + awb_config.max_y = 250; + awb_config.min_c = 16; + awb_config.max_csum = 250; + } + + /* Enable the AWB gains. */ params->module_en_update |= RKISP1_CIF_ISP_MODULE_AWB_GAIN; - /* Update the ISP to apply the gains configured. */ params->module_ens |= RKISP1_CIF_ISP_MODULE_AWB_GAIN; - /* Configure the measure window for AWB. */ - params->meas.awb_meas_config.awb_wnd = context.configuration.awb.measureWindow; - /* - * Measure Y, Cr and Cb means. - * \todo RGB is not working, the kernel seems to not configure it ? - */ - params->meas.awb_meas_config.awb_mode = RKISP1_CIF_ISP_AWB_MODE_YCBCR; - /* Reference Cr and Cb. */ - params->meas.awb_meas_config.awb_ref_cb = 128; - params->meas.awb_meas_config.awb_ref_cr = 128; - /* Y values to include are between min_y and max_y only. */ - params->meas.awb_meas_config.min_y = 16; - params->meas.awb_meas_config.max_y = 250; - /* Maximum Cr+Cb value to take into account for awb. */ - params->meas.awb_meas_config.max_csum = 250; - /* Minimum Cr and Cb values to take into account. */ - params->meas.awb_meas_config.min_c = 16; - /* Number of frames to use to estimate the mean (0 means 1 frame). */ - params->meas.awb_meas_config.frames = 0; - - /* Update AWB measurement unit configuration. */ + /* Update the AWB measurement parameters and enable the AWB module. */ params->module_cfg_update |= RKISP1_CIF_ISP_MODULE_AWB; - /* Make sure the ISP is measuring the means for the next frame. */ params->module_en_update |= RKISP1_CIF_ISP_MODULE_AWB; params->module_ens |= RKISP1_CIF_ISP_MODULE_AWB; } @@ -182,30 +209,39 @@ void Awb::process(IPAContext &context, const rkisp1_cif_isp_stat *params = &stats->params; const rkisp1_cif_isp_awb_stat *awb = ¶ms->awb; IPAActiveState &activeState = context.activeState; + double greenMean; + double redMean; + double blueMean; - /* Get the YCbCr mean values */ - double yMean = awb->awb_mean[0].mean_y_or_g; - double crMean = awb->awb_mean[0].mean_cr_or_r; - double cbMean = awb->awb_mean[0].mean_cb_or_b; + if (rgbMode_) { + greenMean = awb->awb_mean[0].mean_y_or_g; + redMean = awb->awb_mean[0].mean_cr_or_r; + blueMean = awb->awb_mean[0].mean_cb_or_b; + } else { + /* Get the YCbCr mean values */ + double yMean = awb->awb_mean[0].mean_y_or_g; + double cbMean = awb->awb_mean[0].mean_cb_or_b; + double crMean = awb->awb_mean[0].mean_cr_or_r; - /* - * Convert from YCbCr to RGB. - * The hardware uses the following formulas: - * Y = 16 + 0.2500 R + 0.5000 G + 0.1094 B - * Cb = 128 - 0.1406 R - 0.2969 G + 0.4375 B - * Cr = 128 + 0.4375 R - 0.3750 G - 0.0625 B - * - * The inverse matrix is thus: - * [[1,1636, -0,0623, 1,6008] - * [1,1636, -0,4045, -0,7949] - * [1,1636, 1,9912, -0,0250]] - */ - yMean -= 16; - cbMean -= 128; - crMean -= 128; - double redMean = 1.1636 * yMean - 0.0623 * cbMean + 1.6008 * crMean; - double greenMean = 1.1636 * yMean - 0.4045 * cbMean - 0.7949 * crMean; - double blueMean = 1.1636 * yMean + 1.9912 * cbMean - 0.0250 * crMean; + /* + * Convert from YCbCr to RGB. + * The hardware uses the following formulas: + * Y = 16 + 0.2500 R + 0.5000 G + 0.1094 B + * Cb = 128 - 0.1406 R - 0.2969 G + 0.4375 B + * Cr = 128 + 0.4375 R - 0.3750 G - 0.0625 B + * + * The inverse matrix is thus: + * [[1,1636, -0,0623, 1,6008] + * [1,1636, -0,4045, -0,7949] + * [1,1636, 1,9912, -0,0250]] + */ + yMean -= 16; + cbMean -= 128; + crMean -= 128; + redMean = 1.1636 * yMean - 0.0623 * cbMean + 1.6008 * crMean; + greenMean = 1.1636 * yMean - 0.4045 * cbMean - 0.7949 * crMean; + blueMean = 1.1636 * yMean + 1.9912 * cbMean - 0.0250 * crMean; + } /* * The ISP computes the AWB means after applying the colour gains, diff --git a/src/ipa/rkisp1/algorithms/awb.h b/src/ipa/rkisp1/algorithms/awb.h index 6d4a39467038..d76b538288ec 100644 --- a/src/ipa/rkisp1/algorithms/awb.h +++ b/src/ipa/rkisp1/algorithms/awb.h @@ -16,7 +16,7 @@ namespace ipa::rkisp1::algorithms { class Awb : public Algorithm { public: - Awb() = default; + Awb(); ~Awb() = default; int configure(IPAContext &context, const IPACameraSensorInfo &configInfo) override; @@ -32,6 +32,8 @@ public: private: uint32_t estimateCCT(double red, double green, double blue); + + bool rgbMode_; }; } /* namespace ipa::rkisp1::algorithms */