From patchwork Thu Jun 18 10:18:52 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jacopo Mondi X-Patchwork-Id: 26928 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 3EA33C330F for ; Thu, 18 Jun 2026 10:19:15 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id B804E62C6B; Thu, 18 Jun 2026 12:19:14 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="kEaykUTi"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id AFDE8629E2 for ; Thu, 18 Jun 2026 12:19:02 +0200 (CEST) Received: from [192.168.125.177] (mob-109-113-4-199.net.vodafone.it [109.113.4.199]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 7D77ADF3; Thu, 18 Jun 2026 12:18:27 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1781777907; bh=gJjQ+ZviYAXTXT0jc81ax4NVY2j+qk1gkvld9egxiI8=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=kEaykUTi/RmABuM4hrEU1rqvEzMiuxI+lHkzuGiLQu2cAT1f2Ka/RkFLPjBZNxRco 2TZQtnv+0kWvVsM3HricuQ/vZABw+gU9GJ0g21Np48ejoEmgnJg+mf2LkP3IXbwD+5 TVR96TcDuuO+KuHBx7qKS8dQ2fWq/GtcPbmF9L7I= From: Jacopo Mondi Date: Thu, 18 Jun 2026 12:18:52 +0200 Subject: [PATCH 13/14] ipa: rppx1: Add AWB algorithm MIME-Version: 1.0 Message-Id: <20260618-rppx1-ipa-v1-13-32337264cfcd@ideasonboard.com> References: <20260618-rppx1-ipa-v1-0-32337264cfcd@ideasonboard.com> In-Reply-To: <20260618-rppx1-ipa-v1-0-32337264cfcd@ideasonboard.com> To: =?utf-8?q?Niklas_S=C3=B6derlund?= , libcamera-devel@lists.libcamera.org Cc: Jacopo Mondi X-Mailer: b4 0.14.3 X-Developer-Signature: v=1; a=openpgp-sha256; l=13952; i=jacopo.mondi@ideasonboard.com; h=from:subject:message-id; bh=gJjQ+ZviYAXTXT0jc81ax4NVY2j+qk1gkvld9egxiI8=; b=owEBbQKS/ZANAwAKAXI0Bo8WoVY8AcsmYgBqM8YKq3P+O2XCdyZusZBrjyJ9+TpkTCPDwJWgP J15uhH7h0GJAjMEAAEKAB0WIQS1xD1IgJogio9YOMByNAaPFqFWPAUCajPGCgAKCRByNAaPFqFW PFy5D/9Gb3ugTRW7s2JiS5j18EMGkExD3+zZL2qWGgTRQxmWbdTgA+fWAeDc+0BXOCwdh82qZb0 BvGnWAgH6qMiREC7nOeyK/sBZscVOB8ftG2rw0hRPwRposAXK0gr28uKvvNc6mom6IKkB9NiHjq EqgjYpVv31jbYKvV4C7+ZtBBUg3mHXzpB1croY2Tem746pEGFZnx77yobdCiGmrHcU+uOjFftLQ xH45hTpP8lLt8Tpe2WRjYwZsLxHuw53QcBNI8JN8qTPOq6kSz2XGI02/WByz6Be9qNbGyDKBn01 tOO02W3/EjKljgyYIj2cY3G35vf9xSfdCcdQd64zEikmEF0iKzPWZ/IBtSponUvEUOV80W++dfM QZmMjPX45NXF2i8Z5STCsXkZt8kGpnDqZ1fBAZ3nddIZdTc6HtWHJ7kkWkAqNd3HCdYrr25ays3 if1y5nbFSf3Iy1rq01pZxQnlsG9dNEOHDxiQ3yAxARH2/jZgimsu9p2/yJ6sDyeoXnPnA72Dt23 gO1SQcmHcWbeK15502JAWU0jJy1XAzJ6a1Im2TjjzDVyU90mrOdIh1/kTKU+SBGsRpnppN+kILf vDlEKUM2jUuLiQzse4NFBn5cLygN3RWGXfahV4NM1csKdguR5NnLGc60CXvpC3JCePT72uKC2+L KrHWZIH6rAAiSeQ== X-Developer-Key: i=jacopo.mondi@ideasonboard.com; a=openpgp; fpr=72392EDC88144A65C701EA9BA5826A2587AD026B X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Add Awb algorithm to the RPP-X1 IPA. The implementation is based on libIPA AwbAlgorithm. Signed-off-by: Jacopo Mondi --- src/ipa/rppx1/algorithms/awb.cpp | 298 +++++++++++++++++++++++++++++++++++ src/ipa/rppx1/algorithms/awb.h | 48 ++++++ src/ipa/rppx1/algorithms/meson.build | 5 + src/ipa/rppx1/ipa_context.cpp | 30 ++++ src/ipa/rppx1/ipa_context.h | 10 ++ src/ipa/rppx1/meson.build | 4 + 6 files changed, 395 insertions(+) diff --git a/src/ipa/rppx1/algorithms/awb.cpp b/src/ipa/rppx1/algorithms/awb.cpp new file mode 100644 index 000000000000..6316606b4b0f --- /dev/null +++ b/src/ipa/rppx1/algorithms/awb.cpp @@ -0,0 +1,298 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2026, Ideas On Board + * + * RPP-X1 AWB control algorithm + */ + + +#include "awb.h" + +#include +#include + +/** + * \file awb.h + * \brief RPP-X1 Auto White Balance + */ + +namespace libcamera { + +namespace ipa::rppx1::algorithms { + +/* \todo RPP-X1 doesn't support the Lux algorithm. */ +static constexpr unsigned int kDefaultLux = 500; + +LOG_DEFINE_CATEGORY(RppX1Awb) + +namespace { + +/* Quantize a gain to the UQ6.12 register format. */ +constexpr uint32_t quantizeGain(double gain) +{ + return UQ<6, 12>(static_cast(gain)).quantized(); +} + +}; /* namespace */ + +/* + * The RPP-X1 ISP has a bus width of 24 bits and all the measurement limits are + * expressed as 24 bit values. In order to make it simpler to express limits + * in code use an 8-bit value and left-shift by 16 to match the 24 bit width of + * the ISP processing pipeline. + */ +constexpr unsigned int kRPPX1BusWidthShift = 16; + +class RppX1AwbStats final : public AwbStats +{ +public: + RppX1AwbStats() {} + RppX1AwbStats(const RGB &rgbMeans) + : AwbStats(rgbMeans) + { + } + + /* Minimum mean value below which AWB can't operate. */ + double minColourValue() const override + { + return 2.0; + } +}; + +/** + * \class Awb + * \brief RPP-X1 white balance correction algorithm + */ + +/** + * \copydoc libcamera::ipa::Algorithm::init + */ +int Awb::init(IPAContext &context, const ValueNode &tuningData) +{ + return awbAlgo_.init(tuningData, context.ctrlMap); +} + +/** + * \copydoc libcamera::ipa::Algorithm::configure + */ +int Awb::configure(IPAContext &context, + const IPACameraSensorInfo &configInfo) +{ + awbAlgo_.configure(context.activeState.awb); + + /* + * Define the measurement window for AWB as a centered rectangle + * covering 3/4 of the image width and height. + */ + context.configuration.awb.measureWindow.h_offs = configInfo.outputSize.width / 8; + context.configuration.awb.measureWindow.v_offs = configInfo.outputSize.height / 8; + context.configuration.awb.measureWindow.h_size = 3 * configInfo.outputSize.width / 4; + context.configuration.awb.measureWindow.v_size = 3 * configInfo.outputSize.height / 4; + + context.configuration.awb.enabled = true; + + return 0; +} + +/** + * \copydoc libcamera::ipa::Algorithm::queueRequest + */ +void Awb::queueRequest(IPAContext &context, const uint32_t frame, + IPAFrameContext &frameContext, + const ControlList &controls) +{ + awbAlgo_.queueRequest(context.activeState.awb, frame, frameContext.awb, + controls); +} + +/** + * \copydoc libcamera::ipa::Algorithm::prepare + */ +void Awb::prepare(IPAContext &context, const uint32_t frame, + IPAFrameContext &frameContext, RppX1Params *params) +{ + awbAlgo_.prepare(context.activeState.awb, frameContext.awb); + + auto gainConfig = params->block(); + gainConfig.setEnabled(true); + + gainConfig->gain_green_b = quantizeGain(frameContext.awb.gains.g()); + gainConfig->gain_blue = quantizeGain(frameContext.awb.gains.b()); + gainConfig->gain_red = quantizeGain(frameContext.awb.gains.r()); + gainConfig->gain_green_r = quantizeGain(frameContext.awb.gains.g()); + + if (frame > 0) + return; + + /* + * If this is the first frame, program the white balance measurement + * engine. + */ + + auto awbConfig = params->block(); + awbConfig.setEnabled(true); + + /* Configure the measure window for AWB. */ + awbConfig->wnd = context.configuration.awb.measureWindow; + + /* Number of frames to use to estimate the means (0 means 1 frame). */ + awbConfig->frames = 0; + + /* Use YCbCr measurement mode. */ + awbConfig->mode = RPPX1_WBMEAS_MODE_YCBCR; + + /* Set the reference Cr and Cb (AWB target) to white. */ + awbConfig->ref_cb_max_b = 128 << kRPPX1BusWidthShift; + awbConfig->ref_cr_max_r = 128 << kRPPX1BusWidthShift; + + /* + * Filter out pixels based on luminance and chrominance values. + * The acceptable luma values are specified as a [16, 250] + * range, while the acceptable chroma values are specified + * with a minimum of 16 and a maximum Cb+Cr sum of 250; + */ + awbConfig->min_y_max_g = 16 << kRPPX1BusWidthShift; + awbConfig->max_y = 250 << kRPPX1BusWidthShift; + awbConfig->min_c = 16 << kRPPX1BusWidthShift; + awbConfig->max_csum = 250 << kRPPX1BusWidthShift; + + /* \todo Disable ymax_cmp even if we program 'max_y' */ + awbConfig->ymax_cmp = 0; + + /* + * Program coefficients and offsets to perform RGB-to-YCbCr + * conversion according to the BT.601 specification for limited + * range YUV. + * + * Y = 16 + 0.2500 R + 0.5000 G + 0.1094 B + * Cb = 128 - 0.1406 R - 0.2969 G + 0.4375 B + * Cr = 128 + 0.4375 R - 0.3750 G - 0.0625 B + * + * which resuls in the following register values: + * + * - coefficients are signed Q4.12 format + * - their numerical value is then 'reg / 2^12' + * - negative numbers need to reverse the 2's complement + * (!(reg & !BIT(16)) + 1) / 2^12 + * + * coeff G0 0x00000800; = 0.500 + * coeff B0 0x000001c0; = 0.1094 + * coeff R0 0x00000400; = 0.2500 + * coeff G1 0x0000fb40; = -0.2969 + * coeff B1 0x00000700; = 0.4375 + * coeff R1 0x0000fdc0; = -0.1406 + * coeff G2 0x0000fa00; = -0.3750 + * coeff B2 0x0000ff00; = -0.0625 + * coeff R2 0x00000700; = 0.4375 + * offset R 0x00100000; offset_r = (16 << 16) + * offset G 0x00800000; offset_g = (128 << 16) + * offset B 0x00800000; offset_b = (128 << 16) + * + * Use the inverse of this matrix in calculateRgbMeans() to + * reverse the colorspace conversion. + */ + awbConfig->ccor_coeff[0][0] = 0x00000800; + awbConfig->ccor_coeff[0][1] = 0x000001c0; + awbConfig->ccor_coeff[0][2] = 0x00000400; + awbConfig->ccor_coeff[1][0] = 0x0000fb40; + awbConfig->ccor_coeff[1][1] = 0x00000700; + awbConfig->ccor_coeff[1][2] = 0x0000fdc0; + awbConfig->ccor_coeff[2][0] = 0x0000fa00; + awbConfig->ccor_coeff[2][1] = 0x0000ff00; + awbConfig->ccor_coeff[2][2] = 0x00000700; + + awbConfig->ccor_offs[0] = 16 << kRPPX1BusWidthShift; + awbConfig->ccor_offs[1] = 128 << kRPPX1BusWidthShift; + awbConfig->ccor_offs[2] = 128 << kRPPX1BusWidthShift; +} + +/** + * \copydoc libcamera::ipa::Algorithm::process + */ +void Awb::process(IPAContext &context, + [[maybe_unused]] const uint32_t frame, + IPAFrameContext &frameContext, + const RppX1Stats *stats, + ControlList &metadata) +{ + RppX1AwbStats awbStats = calculateRgbMeans(frameContext, stats); + + awbAlgo_.process(context.activeState.awb, frameContext.awb, awbStats, + kDefaultLux, metadata); +} + +RppX1AwbStats Awb::calculateRgbMeans(const IPAFrameContext &frameContext, + const RppX1Stats *stats) const +{ + if (!stats) + return {}; + + const auto awb = stats->block(); + if (!awb) + return {}; + + if (awb->cnt == 0) { + LOG(RppX1Awb, Debug) << "AWB statistics are empty"; + return {}; + } + + Vector rgbMeans; + + /* Get the YCbCr mean values: we use YCbCr measurement mode. */ + Vector yuvMeans({ + static_cast(awb->mean_y_or_g), + static_cast(awb->mean_cb_or_b), + static_cast(awb->mean_cr_or_r) + }); + + /* + * Convert from YCbCr to RGB. The statistics engine has been + * programmed with the following matrix: + * + * Y = 16 + 0.2500 R + 0.5000 G + 0.1094 B + * Cb = 128 - 0.1406 R - 0.2969 G + 0.4375 B + * Cr = 128 + 0.4375 R - 0.3750 G - 0.0625 B + * + * Use the inverse matrix here. + */ + static const Matrix yuv2rgbMatrix({ + 1.1636, -0.0623, 1.6008, + 1.1636, -0.4045, -0.7949, + 1.1636, 1.9912, -0.0250 + }); + static const Vector yuv2rgbOffset({ + 16, 128, 128 + }); + + rgbMeans = yuv2rgbMatrix * (yuvMeans - yuv2rgbOffset); + + /* + * Due to hardware rounding errors in the YCbCr means, the + * calculated RGB means may be negative. This would lead to + * negative gains, messing up calculation. Prevent this by + * clamping the means to positive values. + */ + rgbMeans = rgbMeans.max(0.0); + + /* + * \todo + * The ISP computes the AWB means after applying the CCM. Apply + * the inverse as we want to get the raw means before the colour gains. + * rgbMeans = frameContext.ccm.ccm.inverse() * rgbMeans; + */ + + /* + * The ISP computes the AWB means after applying the colour gains, + * divide by the gains that were used to get the raw means from the + * sensor. Apply a minimum value to avoid divisions by near-zero. + */ + rgbMeans /= frameContext.awb.gains.max(0.01); + + return RppX1AwbStats(rgbMeans); +} + +REGISTER_IPA_ALGORITHM(Awb, "Awb") + +} /* namespace ipa::rppx1::algorithms */ + +} /* namespace libcamera */ diff --git a/src/ipa/rppx1/algorithms/awb.h b/src/ipa/rppx1/algorithms/awb.h new file mode 100644 index 000000000000..2e4f67f11bc0 --- /dev/null +++ b/src/ipa/rppx1/algorithms/awb.h @@ -0,0 +1,48 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2026, Ideas On Board + * + * AWB control algorithm + */ + +#pragma once + +#include "libipa/awb.h" +#include "libipa/fixedpoint.h" + +#include "algorithm.h" + +namespace libcamera { + +namespace ipa::rppx1::algorithms { + +class RppX1AwbStats; + +class Awb : public Algorithm +{ +public: + Awb() = default; + ~Awb() = default; + + int init(IPAContext &context, const ValueNode &tuningData) override; + int configure(IPAContext &context, const IPACameraSensorInfo &configInfo) override; + void queueRequest(IPAContext &context, const uint32_t frame, + IPAFrameContext &frameContext, + const ControlList &controls) override; + void prepare(IPAContext &context, const uint32_t frame, + IPAFrameContext &frameContext, + RppX1Params *params) override; + void process(IPAContext &context, const uint32_t frame, + IPAFrameContext &frameContext, + const RppX1Stats *stats, + ControlList &metadata) override; + +private: + RppX1AwbStats calculateRgbMeans(const IPAFrameContext &frameContext, + const RppX1Stats *stats) const; + + AwbAlgorithm> awbAlgo_; +}; + +} /* namespace ipa::rppx1::algorithms */ +} /* namespace libcamera */ diff --git a/src/ipa/rppx1/algorithms/meson.build b/src/ipa/rppx1/algorithms/meson.build new file mode 100644 index 000000000000..70368ad9bac9 --- /dev/null +++ b/src/ipa/rppx1/algorithms/meson.build @@ -0,0 +1,5 @@ +# SPDX-License-Identifier: CC0-1.0 + +rppx1_ipa_algorithms = files([ + 'awb.cpp', +]) diff --git a/src/ipa/rppx1/ipa_context.cpp b/src/ipa/rppx1/ipa_context.cpp index 5ac60dfc7bf6..ea7008f5685d 100644 --- a/src/ipa/rppx1/ipa_context.cpp +++ b/src/ipa/rppx1/ipa_context.cpp @@ -16,21 +16,51 @@ namespace libcamera::ipa::rppx1 { +/** + * \struct RppX1AwbSession + * \brief RPP-X1 Awb session configuration + */ + +/** + * \var RppX1AwbSession::measureWindow + * \brief Awb measurement window + */ + +/** + * \var RppX1AwbSession::enabled + * \brief Awb enabled flag + */ + /** * \struct IPASessionConfiguration * \brief Session configuration for the IPA module */ +/** + * \var IPASessionConfiguration::awb + * \brief Awb session configuration + */ + /** * \struct IPAActiveState * \brief Active state for algorithms */ +/** + * \var IPAActiveState::awb + * \copydoc ipa::awb::ActiveState + */ + /** * \struct IPAFrameContext * \brief Per-frame context for algorithms */ +/** + * \var IPAFrameContext::awb + * \copydoc ipa::awb::FrameContext + */ + /** * \struct IPAContext * \brief Global IPA context data shared between all algorithms diff --git a/src/ipa/rppx1/ipa_context.h b/src/ipa/rppx1/ipa_context.h index f268a35081bc..57197d865d3f 100644 --- a/src/ipa/rppx1/ipa_context.h +++ b/src/ipa/rppx1/ipa_context.h @@ -20,17 +20,27 @@ #include #include +#include "libipa/awb.h" + namespace libcamera { namespace ipa::rppx1 { +struct RppX1AwbSession { + struct rppx1_window measureWindow; + bool enabled; +}; + struct IPASessionConfiguration { + struct RppX1AwbSession awb; }; struct IPAActiveState { + ipa::awb::ActiveState awb; }; struct IPAFrameContext : public FrameContext { + ipa::awb::FrameContext awb; }; struct IPAContext { diff --git a/src/ipa/rppx1/meson.build b/src/ipa/rppx1/meson.build index 8034fe24d241..8b8c79e997c1 100644 --- a/src/ipa/rppx1/meson.build +++ b/src/ipa/rppx1/meson.build @@ -1,4 +1,6 @@ # SPDX-License-Identifier: CC0-1.0 +# +subdir('algorithms') ipa_name = 'ipa_rppx1' @@ -7,6 +9,8 @@ rppx1_ipa_sources = files([ 'rppx1.cpp', ]) +rppx1_ipa_sources += rppx1_ipa_algorithms + mod = shared_module(ipa_name, rppx1_ipa_sources, name_prefix : '', include_directories : [ipa_includes],