From patchwork Fri Jul 3 12:25:07 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Paul Elder X-Patchwork-Id: 27160 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id E2767C328C for ; Fri, 3 Jul 2026 12:26:05 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 8F01C65FC4; Fri, 3 Jul 2026 14:26:05 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="j37oTYxj"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id B634365F9F for ; Fri, 3 Jul 2026 14:26:03 +0200 (CEST) Received: from neptunite.hamster-moth.ts.net (unknown [IPv6:2404:7a81:160:2100:a2cc:2f45:3bd7:2589]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id D19DF1121; Fri, 3 Jul 2026 14:25:14 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1783081517; bh=lcSoXKvD3oFNvACJ3eLN178/kA+dEsoBJf9XrauiyrU=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=j37oTYxjpQdu5UrmjUWVus3eooPKxySeFGm5N/NUFoTwzxhvb5Jo321ex3TP6Lz0R Ow7Q6Eq9G45CBGvyDDgCkYArpsGsDRS8SWixyOZ+x3qciTqNXxxDd71BBzKVkH78+p z1abwCDwqTb9y1pwWkZEKVU58biW5BRDCX+TTuIg= From: Paul Elder To: laurent.pinchart@ideasonboard.com Cc: Paul Elder , michael.riesch@collabora.com, xuhf@rock-chips.com, stefan.klug@ideasonboard.com, kieran.bingham@ideasonboard.com, dan.scally@ideasonboard.com, jacopo.mondi@ideasonboard.com, nicolas.dufresne@collabora.com, libcamera-devel@lists.libcamera.org Subject: [RFC PATCH 01/19] utils: update-kernel-headers: Add rkisp2-config.h Date: Fri, 3 Jul 2026 21:25:07 +0900 Message-ID: <20260703122543.1991189-2-paul.elder@ideasonboard.com> X-Mailer: git-send-email 2.47.2 In-Reply-To: <20260703122543.1991189-1-paul.elder@ideasonboard.com> References: <20260703122543.1991189-1-paul.elder@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Add rkisp2-config.h to the list of kernel headers to update, as we will soon be adding support for the rkisp2 pipeline. Signed-off-by: Paul Elder --- utils/update-kernel-headers.sh | 1 + 1 file changed, 1 insertion(+) diff --git a/utils/update-kernel-headers.sh b/utils/update-kernel-headers.sh index d97840ee0181..ee1c95febe3d 100755 --- a/utils/update-kernel-headers.sh +++ b/utils/update-kernel-headers.sh @@ -57,6 +57,7 @@ headers=" linux/media/arm/mali-c55-config.h linux/media/v4l2-isp.h linux/rkisp1-config.h + linux/rkisp2-config.h linux/stddef.h linux/udmabuf.h linux/v4l2-common.h From patchwork Fri Jul 3 12:25:08 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Paul Elder X-Patchwork-Id: 27161 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id AB719C3302 for ; Fri, 3 Jul 2026 12:26:08 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 2B4F165FCE; Fri, 3 Jul 2026 14:26:08 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="a+uAA+zs"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 4EE2E65F9F for ; Fri, 3 Jul 2026 14:26:07 +0200 (CEST) Received: from neptunite.hamster-moth.ts.net (unknown [IPv6:2404:7a81:160:2100:a2cc:2f45:3bd7:2589]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 688351494; Fri, 3 Jul 2026 14:25:18 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1783081521; bh=VMASAyHP5XydmR7UL3gcxwzBylUin8TniB3R2omdLmI=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=a+uAA+zsF78JgEd7jXZgYaAN0Qqc+Q1YhkaUflOqyNIoloIWkXJ1ZUOFXKOBwmnHn cVnA11cu1bN6T8mG7D8+A/bwiZ66EnD2659Ah7tsCAiYb4mRQXsT/tTGVTvQVSuu9+ OCCz/UdwNDokS48I8rNibKL90pSAOquqACnUIkc8= From: Paul Elder To: laurent.pinchart@ideasonboard.com Cc: Paul Elder , michael.riesch@collabora.com, xuhf@rock-chips.com, stefan.klug@ideasonboard.com, kieran.bingham@ideasonboard.com, dan.scally@ideasonboard.com, jacopo.mondi@ideasonboard.com, nicolas.dufresne@collabora.com, libcamera-devel@lists.libcamera.org Subject: [RFC PATCH 02/19] include: linux: rkisp2-config.h: update to v7.0 Date: Fri, 3 Jul 2026 21:25:08 +0900 Message-ID: <20260703122543.1991189-3-paul.elder@ideasonboard.com> X-Mailer: git-send-email 2.47.2 In-Reply-To: <20260703122543.1991189-1-paul.elder@ideasonboard.com> References: <20260703122543.1991189-1-paul.elder@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Update rkisp2-config.h to the most recent version. Signed-off-by: Paul Elder --- This comes from RFC of "media: rockchip: rkisp2: Add rkisp2 driver": https://lore.kernel.org/all/20260424175853.638202-1-paul.elder@ideasonboard.com/ --- include/linux/README | 2 +- include/linux/rkisp2-config.h | 562 ++++++++++++++++++++++++++++++++++ include/linux/videodev2.h | 4 + 3 files changed, 567 insertions(+), 1 deletion(-) create mode 100644 include/linux/rkisp2-config.h diff --git a/include/linux/README b/include/linux/README index b02952bb28ca..fed0903f5d37 100644 --- a/include/linux/README +++ b/include/linux/README @@ -1,4 +1,4 @@ # SPDX-License-Identifier: CC0-1.0 -Files in this directory are imported from v7.0 of the Linux kernel. Do not +Files in this directory are imported from v7.0-19-g4869cb2fa7d1 of the Linux kernel. Do not modify them manually. diff --git a/include/linux/rkisp2-config.h b/include/linux/rkisp2-config.h new file mode 100644 index 000000000000..5b69490be93d --- /dev/null +++ b/include/linux/rkisp2-config.h @@ -0,0 +1,562 @@ +/* SPDX-License-Identifier: ((GPL-2.0-or-later WITH Linux-syscall-note) OR MIT) */ +/* + * Rockchip ISP2 userspace API + * Copyright (C) 2017 Rockchip Electronics Co., Ltd. + * Copyright (C) 2026 Ideas on Board Oy. + */ + +#ifndef _RKISP2_CONFIG_H +#define _RKISP2_CONFIG_H + +#include + +#include + +#define RKISP2_ISP_GAMMA_OUT_MAX_SEGMENTS 49 + +#define RKISP2_ISP_LSC_SAMPLES_MAX 17 +#define RKISP2_ISP_LSC_SECTORS_TBL_SIZE_MAX 16 + +#define RKISP2_ISP_AE_MEAN_MAX_LITE 25 +#define RKISP2_ISP_AE_MEAN_MAX_BIG 225 + +#define RKISP2_ISP_HIST_WEIGHT_GRIDS_SIZE_LITE 25 +#define RKISP2_ISP_HIST_WEIGHT_GRIDS_SIZE_BIG 225 +#define RKISP2_ISP_HIST_WEIGHT_GRIDS_SIZE_MAX RKISP2_ISP_HIST_WEIGHT_GRIDS_SIZE_BIG + +#define RKISP2_ISP_HIST_BIN_N_MAX 256 + +#define RKISP2_ISP_AWB_COUNTS_SIZE 225 + +/** + * enum rkisp2_isp_version - ISP variants + * + * @RKISP3_V0: Used at least in RK3588 + */ +enum rkisp2_isp_version { + RKISP3_V0 = 30, +}; + +/* See enum rkisp2_isp_goc_segments for logarithmic segment sizes */ +enum rkisp2_isp_goc_mode { + RKISP2_ISP_GOC_MODE_LOGARITHMIC, + RKISP2_ISP_GOC_MODE_EQUIDISTANT +}; + +/* + * The segments are: + * 1 x8, 2 x4, 4 x4, 8 x4, 16 x4, 32 x4, 64 x4, 128 x4, 256 x4, 512 x4 + * In 48-segment mode, the last group of 512 x4 becomes 256 x8 + */ +enum rkisp2_isp_goc_segments { + RKISP2_ISP_GOC_SEGMENTS_44, + RKISP2_ISP_GOC_SEGMENTS_48 +}; + +enum rkisp2_isp_lsc_config { + RKISP2_ISP_LSC_CONFIG_8X8, + RKISP2_ISP_LSC_CONFIG_16X16 +}; + +enum rkisp2_isp_set_active_table_when { + RKISP2_ISP_LSC_SET_ACTIVE_TABLE_AFTER, + RKISP2_ISP_LSC_SET_ACTIVE_TABLE_BEFORE, +}; + +enum rkisp2_isp_histogram_mode { + RKISP2_ISP_HISTOGRAM_MODE_DISABLE, + RKISP2_ISP_HISTOGRAM_MODE_R_HISTOGRAM = 2, + RKISP2_ISP_HISTOGRAM_MODE_G_HISTOGRAM, + RKISP2_ISP_HISTOGRAM_MODE_B_HISTOGRAM, + RKISP2_ISP_HISTOGRAM_MODE_Y_HISTOGRAM +}; + +/* + * This selects which bits are used from the input data to compute the + * histogram + */ +enum rkisp2_isp_histogram_data_sel { + RKISP2_ISP_HISTOGRAM_DATA_SEL_11_4, + RKISP2_ISP_HISTOGRAM_DATA_SEL_10_3, + RKISP2_ISP_HISTOGRAM_DATA_SEL_9_2, + RKISP2_ISP_HISTOGRAM_DATA_SEL_8_1, + RKISP2_ISP_HISTOGRAM_DATA_SEL_7_0, +}; + +/*---------- Statistics ------------*/ + +/** + * struct rkisp2_isp_ae_lite - statistics auto exposure data + * + * @exp_mean_r: Mean luminance value of block xy for r channel + * @exp_mean_g: Mean luminance value of block xy for g channel + * @exp_mean_b: Mean luminance value of block xy for b channel + * @done: This set to nonzero when the stats are ready + * + * Image is divided into 5x5 blocks on lite and 15x15 blocks on big. + */ +struct rkisp2_isp_ae_lite { + __u16 exp_mean_r[RKISP2_ISP_AE_MEAN_MAX_LITE]; + __u16 exp_mean_g[RKISP2_ISP_AE_MEAN_MAX_LITE]; + __u16 exp_mean_b[RKISP2_ISP_AE_MEAN_MAX_LITE]; + __u8 done; +}; + +/** + * struct rkisp2_cif_isp_hist_stat - statistics histogram data + * + * @hist_bins: measured bin counters. Each bin is a 28 bits unsigned fixed point + * type. Bits 0-4 are the fractional part and bits 5-27 are the + * integer part. + * @done: This set to nonzero when the stats are ready + * + * There are 256 bins, at least on 3.x. + */ +struct rkisp2_isp_hist { + __u32 hist_bins[RKISP2_ISP_HIST_BIN_N_MAX]; + __u8 done; +}; + +/** + * struct rkisp2_isp_awb - statistics auto white balance data + * + * @counts_r: Counts of red (18-bits) + * @counts_g: Counts of green (18-bits) + * @counts_b: Counts of blue (18-bits) + * @counts_w: Counts of white point (10-bits) + * @done: This set to nonzero when the stats are ready + * + * TODO Figure out what is being counted + */ +struct rkisp2_isp_awb { + __u32 counts_r[RKISP2_ISP_AWB_COUNTS_SIZE]; + __u32 counts_g[RKISP2_ISP_AWB_COUNTS_SIZE]; + __u32 counts_b[RKISP2_ISP_AWB_COUNTS_SIZE]; + __u16 counts_w[RKISP2_ISP_AWB_COUNTS_SIZE]; + __u8 done; +}; + +/** + * struct rkisp2_stats_buffer - 3A statistics for the RkISP2 + * + * @ae_lite: ae lite stats + * @hist_lite: histogram lite stats + * @hist_big0: histogram big0 stats + * @hist_big1: histogram big0 stats + * @hist_big2: histogram big0 stats + * @awb: awb stats + */ +struct rkisp2_stats_buffer { + struct rkisp2_isp_ae_lite ae_lite; + struct rkisp2_isp_hist hist_lite; + struct rkisp2_isp_hist hist_big0; + struct rkisp2_isp_hist hist_big1; + struct rkisp2_isp_hist hist_big2; + struct rkisp2_isp_awb awb; +}; + +/*---------- Parameters ------------*/ + +/** + * enum rkisp2_params_block_type - RkISP1 extensible params block type + * + * @RKISP2_PARAMS_BLOCK_BLS: Black level subtraction + * @RKISP2_PARAMS_BLOCK_AWB_GAINS: AWB gains + * @RKISP2_PARAMS_BLOCK_CSM: Color conversion coefficients (in the ISP block) + * @RKISP2_PARAMS_BLOCK_CCM: Color correction matrix (in the CCM block) + * @RKISP2_PARAMS_BLOCK_GOC: Gamma out correction + * @RKISP2_PARAMS_BLOCK_LSC: Lens shading correction + * @RKISP2_PARAMS_BLOCK_AE_LITE: AE measurement config (lite) + * @RKISP2_PARAMS_BLOCK_HIST_LITE: Histogram measurement config (lite) + * @RKISP2_PARAMS_BLOCK_HIST_BIG0: Histogram measurement config (zeroth big block) + * @RKISP2_PARAMS_BLOCK_HIST_BIG1: Histogram measurement config (first big block) + * @RKISP2_PARAMS_BLOCK_HIST_BIG2: Histogram measurement config (second big block) + * @RKISP2_PARAMS_BLOCK_AWB_MEAS: AWB measurements config + * */ +enum rkisp2_params_block_type { + RKISP2_PARAMS_BLOCK_BLS, + RKISP2_PARAMS_BLOCK_AWB_GAINS, + RKISP2_PARAMS_BLOCK_CSM, + RKISP2_PARAMS_BLOCK_CCM, + RKISP2_PARAMS_BLOCK_GOC, + RKISP2_PARAMS_BLOCK_LSC, + RKISP2_PARAMS_BLOCK_AE_LITE, + RKISP2_PARAMS_BLOCK_HIST_LITE, + RKISP2_PARAMS_BLOCK_HIST_BIG0, + RKISP2_PARAMS_BLOCK_HIST_BIG1, + RKISP2_PARAMS_BLOCK_HIST_BIG2, + RKISP2_PARAMS_BLOCK_AWB_MEAS, +}; + +/** + * struct rkisp2_isp_window - measurement window. + * + * Measurements are calculated per window inside the frame. + * This struct represents a window for a measurement. + * + * @h_offs: the horizontal offset of the window from the left of the frame in pixels. + * @v_offs: the vertical offset of the window from the top of the frame in pixels. + * @h_size: the horizontal size of the window in pixels + * @v_size: the vertical size of the window in pixels. + */ +struct rkisp2_isp_window { + __u16 h_offs; + __u16 v_offs; + __u16 h_size; + __u16 v_size; +}; + +/** + * struct rkisp2_isp_bls_fixed_val - BLS fixed subtraction values + * + * These are signed 13-bit (-4096 to +4095). + * + * @a: Fixed black level value for Bayer channel 0 + * @b: Fixed black level value for Bayer channel 1 + * @c: Fixed black level value for Bayer channel 2 + * @d: Fixed black level value for Bayer channel 3 + */ +struct rkisp2_isp_bls_fixed_val { + __s16 a; + __s16 b; + __s16 c; + __s16 d; +}; + +/** + * struct rkisp2_isp_awb_gains - Auto white balance gain in the ISP block + * + * All fields in this struct are 16 bit, where: + * 0x100h = 1, unsigned integer value, range 0 to 63 with 8 bit fractional part. + * + * This leaves the upper two msb unaccounted for; it is unknown if these are + * unused or misdocumented. + * + * TODO investigate the upper two bits + * + * @r: gain value for red component. + * @gr: gain value for green component in red line. + * @b: gain value for blue component. + * @gb: gain value for green component in blue line. + */ +struct rkisp2_isp_awb_gains { + __u16 r; + __u16 gr; + __u16 b; + __u16 gb; +}; + +/** + * struct rkisp2_isp_color_cc - Color coefficients + * + * @r: Red coefficient + * @g: Green coefficient + * @b: Blue coefficient + */ +struct rkisp2_isp_color_cc { + __u8 r; + __u8 g; + __u8 b; +}; + +/** + * struct rkisp2_isp_awb_color_quad - Group of RGB and luminance for AWB + * + * TODO redesign this? + * + * @r: Red + * @g: Green + * @b: Blue + * @y: Y (luminance) + */ +struct rkisp2_isp_awb_color_quad { + __u8 r; + __u8 g; + __u8 b; + __u8 y; +}; + +/** + * struct rkisp2_params_bls - RkISP2 params BLS config + * + * RkISP2 parameters Black Level Subtraction configuration block. + * Identified by :c:type:`RKISP2_PARAMS_BLOCK_BLS`. + * + * TODO Check if auto-mode and window selection is for both blocks or just for + * one block (it might be the same as 2.x) + * + * @header: The RkISP2 parameters block header + * @enable_auto: Automatic mode activated means that the measured values + * are subtracted. Otherwise the fixed subtraction + * values will be subtracted. + * @enabled_windows: enabled window (bit 0 for window 1, bit 1 for window 2) + * @bls_window1: Measurement window 1 size + * @bls_window2: Measurement window 2 size + * @bls_samples: Set amount of measured pixels for each Bayer position + * (A, B, C and D) to 2^bls_samples. (TODO needs confirmation) + * @bls_fixed_val: Black Level Subtraction fixed values for the BLS module at + * the front of the pipeline + * @bls1_fixed_val: Black Level Subtraction fixed values for the BLS module after + * bayer noise reduction + */ +struct rkisp2_params_bls { + struct v4l2_isp_params_block_header header; + __u8 enable_auto; + __u8 enabled_windows; + struct rkisp2_isp_window bls_window1; + struct rkisp2_isp_window bls_window2; + __u8 bls_samples; + struct rkisp2_isp_bls_fixed_val bls_fixed_val; + struct rkisp2_isp_bls_fixed_val bls1_fixed_val; +} __attribute__((aligned(8))); + +/** + * struct rkisp2_params_awb_gains - RKISP2 params AWB gains config + * + * RkISP2 parameters auto white balance gains configuration block. + * Identified by :c:type:`RKISP2_PARAMS_BLOCK_AWB_GAINS`. + * + * TODO investigate what the different blocks mean + * + * Block 0 is equivalent to the awb gains block on 2.x, but blocks 1 and + * 2 do not exist on 2.x. + * + * @header: The RkISP2 parameters block header + * @gains: Gains configuration for block i + */ +struct rkisp2_params_awb_gains { + struct v4l2_isp_params_block_header header; + struct rkisp2_isp_awb_gains gains[3]; +} __attribute__((aligned(8))); + +/** + * struct rkisp2_params_csm - Configuration used by Color Space Conversion + * + * RkISP2 parameters histogram configuration block. + * Identified by :c:type:`RKISP2_PARAMS_BLOCK_CSM`. + * + * @header: The RkISP2 parameters block header + * @coeff: color correction matrix. Values are 9-bit signed fixed-point numbers with 2 bit integer + * and 7 bit fractional part, ranging from -2 (0x100) to +1.992 (0x0FF). 0 is + * represented by 0x000 and a coefficient value of 1 as 0x080. + */ +struct rkisp2_params_csm { + struct v4l2_isp_params_block_header header; + __u16 coeff[3][3]; +}; + +/** + * struct rkisp2_params_ccm - Configuration used by Color Correction Matrix + * + * RkISP2 parameters histogram configuration block. + * Identified by :c:type:`RKISP2_PARAMS_BLOCK_CCM`. + * + * @header: The RkISP2 parameters block header + * @high_y_alpha_adj_en: Enable CCM high Y alpha adjustment (TODO figure out what this does) + * @coeff: color correction matrix. Values are 11-bit signed fixed-point numbers with 4 bit integer + * and 7 bit fractional part, ranging from -8 (0x400) to +7.992 (0x3FF). 0 is + * represented by 0x000 and a coefficient value of 1 as 0x080. The + * value is expanded 128 times (TODO figure out what this means). + * @offset: Red, Green, Blue offsets for the color correction matrix. 12-bits + * wide ranging from -4096 to 4095, but only for red; green and blue are 11-bit + * signed fixed-point like coeff, but are still 12-bits wide. + * @y_coeff: Red, Green, Blue coefficients for RGB2Y calculation. red and green + * are 11-bits wide and blue is 12-bits wide. The value is expanded 128 times. + * @alp: CCM curve y-axis point definition for ccm input pixel's luminance. + * 11-bit unsigned ranging from 0 to 1024. The value is expanded 128 times. + * @inflection_point: Inflection point of the ccm alpha interpolation curve. + * The inflection point is 2^inflection_point. Since the maximum y-value is + * 1024, the maximum value of this field is expected to be 10 (0xa), but the + * documentation says 4'b10. + */ +struct rkisp2_params_ccm { + struct v4l2_isp_params_block_header header; + __u8 high_y_alpha_adj_en; + __u16 coeff[3][3]; + __u16 offset[3]; + __u16 y_coeff[3]; + __u16 alp[17]; + __u8 inflection_point; +}; + +/** + * struct rkisp2_params_goc - Configuration used by Gamma Out correction + * + * RkISP2 parameters gamma out correction configuration block. + * Identified by :c:type:`RKISP2_PARAMS_BLOCK_GOC`. + * + * @header: The RkISP2 parameters block header + * @mode: goc mode (from enum rkisp2_isp_goc_mode) + * @segments: segments mode (from enum rkisp2_isp_goc_segments) + * @offset: offset value of the gamma out curve + * @gamma_y: gamma out curve y-axis for all color components + * + * The number of entries of @gamma_y depends on the segments mode. The entries + * are 12-bit unsigned. + */ +struct rkisp2_params_goc { + struct v4l2_isp_params_block_header header; + __u8 mode; + __u8 segments; + __u16 offset; + __u16 gamma_y[RKISP2_ISP_GAMMA_OUT_MAX_SEGMENTS]; +}; + +/** + * struct rkisp2_params_lsc - Configuration used by Lens shading correction + * + * RkISP2 parameters lens shading correction configuration block. + * Identified by :c:type:`RKISP2_PARAMS_BLOCK_LSC`. + * + * The LSC module on the rkisp2 two tables: the 0th table and the 1th table. + * They can be programmed independently and (somewhat) simultaneously, and can be + * swapped by setting a single register. Hence the UAPI here is designed so + * that all these components can be controlled independently. + * + * In the first dimension of {r,gr,gb,b}_data_tbl we can designate which table + * to write the data to. write_table is then used to signal whether to write + * the data, and this can be controlled for both tables. active_table chooses + * which table to activate. set_active_table_when signals whether to set the + * active_table before or after programming the table. This allows + * optimizations such as setting a future table in one parameter buffer while + * swapping before setting it. + * + * This design gives us more control. For example, if we want to only program + * the 0th table without modifying the 1th table, we do not need to also + * populate the 1th table and we can use write_table to designate that we only + * want to program the 0th table. We can also swap tables without needing to + * re-populate the tables by setting active_table and unsetting write_table. + * + * {x,y}_sizes designates the grid of the LSC, and the table entries above + * correspond to the *vertices* of the grid. {x,y}_grads control the bilinear + * interpolation within the grid. + * + * @header: The RkISP2 parameters block header + * @r_data_tbl: Sample table red + * @gr_data_tbl: Sample table green (red) + * @gb_data_tbl: Sample table green (blue) + * @b_data_tbl: Sample table blue + * @write_table: Set to 1 to signal to write the respective table from above + * @active_table: Choose which of the two tables is active (0 or 1) + * @set_active_table_when: From rkisp2_isp_set_active_table_when; switch to the + * active table before or after programming the table + * @x_sizes: Sizes x + * @y_sizes: Sizes y + * @x_grads: Gradients x + * @y_grads: Gradients y + * @window_mode: From enum rkisp2_isp_lsc_config + */ +struct rkisp2_params_lsc { + struct v4l2_isp_params_block_header header; + + __u16 r_data_tbl[2][RKISP2_ISP_LSC_SAMPLES_MAX][RKISP2_ISP_LSC_SAMPLES_MAX]; + __u16 gr_data_tbl[2][RKISP2_ISP_LSC_SAMPLES_MAX][RKISP2_ISP_LSC_SAMPLES_MAX]; + __u16 gb_data_tbl[2][RKISP2_ISP_LSC_SAMPLES_MAX][RKISP2_ISP_LSC_SAMPLES_MAX]; + __u16 b_data_tbl[2][RKISP2_ISP_LSC_SAMPLES_MAX][RKISP2_ISP_LSC_SAMPLES_MAX]; + __u8 write_table[2]; + __u8 active_table; + __u8 set_active_table_when; + + __u16 x_sizes[RKISP2_ISP_LSC_SECTORS_TBL_SIZE_MAX]; + __u16 y_sizes[RKISP2_ISP_LSC_SECTORS_TBL_SIZE_MAX]; + __u16 x_grads[RKISP2_ISP_LSC_SECTORS_TBL_SIZE_MAX]; + __u16 y_grads[RKISP2_ISP_LSC_SECTORS_TBL_SIZE_MAX]; + + __u8 window_mode; +}; + +/** + * struct rkisp2_params_ae_lite - RKISP2 params AE lite config + * + * RkISP2 parameters auto exposure measurement configuration block. + * Identified by :c:type:`RKISP2_PARAMS_BLOCK_AE_LITE`. + * + * TODO change window_num to enum? + * + * @header: The RkISP2 parameters block header + * @window_num: 0 for 1x1, 1 for 5x5 + * @meas_window: Size of measurement window. First window for 5x5. + */ +struct rkisp2_params_ae_lite { + struct v4l2_isp_params_block_header header; + __u8 window_num; + struct rkisp2_isp_window meas_window; +} __attribute__((aligned(8))); + +/** + * struct rkisp2_params_hist_lite RKISP2 params histogram lite config + * + * RkISP2 parameters histogram configuration block. + * Identified by :c:type:`RKISP2_PARAMS_BLOCK_HIST_LITE`. + * + * @header: The RkISP2 parameters block header + * @data_sel: Data selection mode (from enum rkisp2_isp_histogram_data_sel) + * @mode: Histogram mode (from enum rkisp2_isp_histogram_mode) + * @stepsize: Predivider (count every pixel) + * @waterline: Waterline for region statics + * @coeffs: Coefficients for raw2y formula + * @meas_window: Size of first measurement subwindow + * @weights: Weights + */ +struct rkisp2_params_hist_lite { + struct v4l2_isp_params_block_header header; + __u8 data_sel; + __u8 mode; + __u8 stepsize; + __u16 waterline; + struct rkisp2_isp_color_cc coeffs; + struct rkisp2_isp_window meas_window; + __u8 weights[RKISP2_ISP_HIST_WEIGHT_GRIDS_SIZE_LITE]; +} __attribute__((aligned(8))); + +/** + * Same as struct rkisp2_params_hist_lite but for big channel + * + * RkISP2 parameters histogram configuration block. + * Identified by :c:type:`RKISP2_PARAMS_BLOCK_HIST_BIG{0,1,2}`. + * + * @window_num: 0 or 1 for 5x5, 2 or 3 for 15x15 + */ +struct rkisp2_params_hist_big { + struct v4l2_isp_params_block_header header; + __u8 window_num; + __u8 data_sel; + __u8 mode; + __u8 stepsize; + __u16 waterline; + struct rkisp2_isp_color_cc coeffs; + struct rkisp2_isp_window meas_window; + __u8 weights[RKISP2_ISP_HIST_WEIGHT_GRIDS_SIZE_BIG]; +} __attribute__((aligned(8))); + +/** + * struct rkisp2_params_awb_meas - Configuration used by rawawb + * + * RkISP2 parameters AWB measurement configuration block. + * Identified by :c:type:`RKISP2_PARAMS_BLOCK_AWB_MEAS`. + * + * @header: The RkISP2 parameters block header + * @meas_window: Size of first measurement subwindow (13 bits) + * @limits: Limits for white point detection [min, max] (8 bits) + * @weights: Weights (6-bits) + */ +struct rkisp2_params_awb_meas { + struct v4l2_isp_params_block_header header; + struct rkisp2_isp_window meas_window; + struct rkisp2_isp_awb_color_quad limits[2]; + __u8 weights[RKISP2_ISP_AWB_COUNTS_SIZE]; +}; + +#define RKISP2_PARAMS_MAX_SIZE \ + (sizeof(struct rkisp2_params_bls) +\ + sizeof(struct rkisp2_params_awb_gains) +\ + sizeof(struct rkisp2_params_csm) +\ + sizeof(struct rkisp2_params_ccm) +\ + sizeof(struct rkisp2_params_goc) +\ + sizeof(struct rkisp2_params_lsc) +\ + sizeof(struct rkisp2_params_ae_lite) +\ + sizeof(struct rkisp2_params_hist_lite) +\ + sizeof(struct rkisp2_params_hist_big) * 3 +\ + sizeof(struct rkisp2_params_awb_meas)) + +#endif /* _RKISP2_CONFIG_H */ diff --git a/include/linux/videodev2.h b/include/linux/videodev2.h index 8bccad84ff83..d242f3c0ab5f 100644 --- a/include/linux/videodev2.h +++ b/include/linux/videodev2.h @@ -851,6 +851,10 @@ struct v4l2_pix_format { #define V4L2_META_FMT_RK_ISP1_STAT_3A v4l2_fourcc('R', 'K', '1', 'S') /* Rockchip ISP1 3A Statistics */ #define V4L2_META_FMT_RK_ISP1_EXT_PARAMS v4l2_fourcc('R', 'K', '1', 'E') /* Rockchip ISP1 3a Extensible Parameters */ +/* Vendor specific - used for RKISP2 camera sub-system */ +#define V4L2_META_FMT_RKISP2_PARAMS v4l2_fourcc('R', 'K', '2', 'P') /* Rockchip ISP2 Parameters */ +#define V4L2_META_FMT_RKISP2_STATS v4l2_fourcc('R', 'K', '2', 'S') /* Rockchip ISP2 3A Statistics */ + /* Vendor specific - used for C3_ISP */ #define V4L2_META_FMT_C3ISP_PARAMS v4l2_fourcc('C', '3', 'P', 'M') /* Amlogic C3 ISP Parameters */ #define V4L2_META_FMT_C3ISP_STATS v4l2_fourcc('C', '3', 'S', 'T') /* Amlogic C3 ISP Statistics */ From patchwork Fri Jul 3 12:25:09 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Paul Elder X-Patchwork-Id: 27162 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 2815FC328C for ; Fri, 3 Jul 2026 12:26:13 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id C7C5665FCA; Fri, 3 Jul 2026 14:26:12 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="JrddpVvW"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id CD06F65FC3 for ; Fri, 3 Jul 2026 14:26:10 +0200 (CEST) Received: from neptunite.hamster-moth.ts.net (unknown [IPv6:2404:7a81:160:2100:a2cc:2f45:3bd7:2589]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id EFEEA8E0; Fri, 3 Jul 2026 14:25:21 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1783081524; bh=G/6uyCRD1NKUobvC/oTtrFp+jzbE6ZxVDe2pRKvWNwk=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=JrddpVvW4v1FqYjQ+0ZtRokPPMH7ChmLiYTMc7QHDVKKNqcRhm8wY4+ptAto0W0Gw 7uKaLS/NpQG6Jzcx7JNzeoWAiynsvUiKGbGz5CD2Pl9jsFvS/NJRRI/42XJnAiPHuY aqbr6ed4C2LfiR6L0uUgFYw+EGseppfxPcqyOvbw= From: Paul Elder To: laurent.pinchart@ideasonboard.com Cc: Paul Elder , michael.riesch@collabora.com, xuhf@rock-chips.com, stefan.klug@ideasonboard.com, kieran.bingham@ideasonboard.com, dan.scally@ideasonboard.com, jacopo.mondi@ideasonboard.com, nicolas.dufresne@collabora.com, libcamera-devel@lists.libcamera.org Subject: [RFC PATCH 03/19] ipa: rkisp2: params: Add rkisp2 extensible parameters wrapper Date: Fri, 3 Jul 2026 21:25:09 +0900 Message-ID: <20260703122543.1991189-4-paul.elder@ideasonboard.com> X-Mailer: git-send-email 2.47.2 In-Reply-To: <20260703122543.1991189-1-paul.elder@ideasonboard.com> References: <20260703122543.1991189-1-paul.elder@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Add the libipa wrapper for extensible parameters for the rkisp2. This wraps the extensible parameters in the rkisp2 uapi to make them easier to use within libcamera. Signed-off-by: Paul Elder --- This is in a separate patch to reduce the size of the other patches to make them easier to review. --- src/ipa/rkisp2/algorithms/algorithm.h | 42 ++++++++++++++ src/ipa/rkisp2/params.h | 83 +++++++++++++++++++++++++++ 2 files changed, 125 insertions(+) create mode 100644 src/ipa/rkisp2/algorithms/algorithm.h create mode 100644 src/ipa/rkisp2/params.h diff --git a/src/ipa/rkisp2/algorithms/algorithm.h b/src/ipa/rkisp2/algorithms/algorithm.h new file mode 100644 index 000000000000..c30596e3299b --- /dev/null +++ b/src/ipa/rkisp2/algorithms/algorithm.h @@ -0,0 +1,42 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2026, Ideas On Board + * + * RkISP2 control algorithm interface + */ + +#pragma once + +#include + +#include + +#include "module.h" + +namespace libcamera { + +namespace ipa::rkisp2 { + +class Algorithm : public libcamera::ipa::Algorithm +{ +}; + +union rkisp2_params_block { + const struct v4l2_isp_params_block_header *header; + const struct rkisp2_params_bls *bls; + const struct rkisp2_params_awb_gains *awb_gains; + const struct rkisp2_params_csm *csm; + const struct rkisp2_params_ccm *ccm; + const struct rkisp2_params_goc *goc; + const struct rkisp2_params_lsc *lsc; + const struct rkisp2_params_ae_lite *ae_lite; + const struct rkisp2_params_hist_lite *hist_lite; + const struct rkisp2_params_hist_big *hist_big; + const struct rkisp2_params_awb_meas *awb_meas; + const __u8 *data; +}; + +} /* namespace ipa::rkisp2 */ + +} /* namespace libcamera */ + diff --git a/src/ipa/rkisp2/params.h b/src/ipa/rkisp2/params.h new file mode 100644 index 000000000000..c7c15483ee11 --- /dev/null +++ b/src/ipa/rkisp2/params.h @@ -0,0 +1,83 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2026, Ideas On Board + * + * RkISP2 ISP Parameters + */ + +#pragma once + +#include + +#include +#include + +#include + +namespace libcamera { + +namespace ipa::rkisp2 { + +enum class RkISP2Blocks : uint16_t { + Bls, + AwbGains, + Csm, + Ccm, + Goc, + Lsc, + AeLite, + HistLite, + HistBig0, + HistBig1, + HistBig2, + AwbMeas, +}; + +namespace details { + +template +struct block_type { +}; + +#define RKISP2_DEFINE_BLOCK_TYPE(id, cfgType, blkType) \ +template<> \ +struct block_type { \ + using type = struct rkisp2_params_##cfgType; \ + static constexpr rkisp2_params_block_type blockType = \ + rkisp2_params_block_type::RKISP2_PARAMS_##blkType; \ +} + +RKISP2_DEFINE_BLOCK_TYPE(Bls, bls, BLOCK_BLS); +RKISP2_DEFINE_BLOCK_TYPE(AwbGains, awb_gains, BLOCK_AWB_GAINS); +RKISP2_DEFINE_BLOCK_TYPE(Csm, csm, BLOCK_CSM); +RKISP2_DEFINE_BLOCK_TYPE(Ccm, ccm, BLOCK_CCM); +RKISP2_DEFINE_BLOCK_TYPE(Goc, goc, BLOCK_GOC); +RKISP2_DEFINE_BLOCK_TYPE(Lsc, lsc, BLOCK_LSC); +RKISP2_DEFINE_BLOCK_TYPE(AeLite, ae_lite, BLOCK_AE_LITE); +RKISP2_DEFINE_BLOCK_TYPE(HistLite, hist_lite, BLOCK_HIST_LITE); +RKISP2_DEFINE_BLOCK_TYPE(HistBig0, hist_big, BLOCK_HIST_BIG0); +RKISP2_DEFINE_BLOCK_TYPE(HistBig1, hist_big, BLOCK_HIST_BIG1); +RKISP2_DEFINE_BLOCK_TYPE(HistBig2, hist_big, BLOCK_HIST_BIG2); +RKISP2_DEFINE_BLOCK_TYPE(AwbMeas, awb_meas, BLOCK_AWB_MEAS); + +struct param_traits { + using id_type = RkISP2Blocks; + + template + using id_to_details = block_type; +}; + +} /* namespace details */ + +class RkISP2Params : public V4L2Params +{ +public: + RkISP2Params(Span data) + : V4L2Params(data, V4L2_ISP_PARAMS_VERSION_V1) + { + } +}; + +} /* namespace ipa::rkisp2 */ + +} /* namespace libcamera */ From patchwork Fri Jul 3 12:25:10 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Paul Elder X-Patchwork-Id: 27163 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id C782CC328C for ; Fri, 3 Jul 2026 12:26:15 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 5F45865FCA; Fri, 3 Jul 2026 14:26:15 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="IEhX6w7J"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 5DA2165FBF for ; Fri, 3 Jul 2026 14:26:14 +0200 (CEST) Received: from neptunite.hamster-moth.ts.net (unknown [IPv6:2404:7a81:160:2100:a2cc:2f45:3bd7:2589]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 7F3A81121; Fri, 3 Jul 2026 14:25:25 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1783081528; bh=3jYltKqKcr57DNBqTq9A69GfhmEMX5qb+oyxYAgvAvk=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=IEhX6w7JvqQTP5JrUucyxGH7yo6ZjeNn68L7xubVGsxKzK5u8qaawvGJBT7/UmFaO o7wUCXdDDsU/y9eyRKblaIKt6LcmtfLVMAm0ZpOjAYH281Jons+Ja4HIOS/DjvJXjB Gih885aWUJTGCRTkSR/GLTDNGoFD1Kk2w5PxIcMs= From: Paul Elder To: laurent.pinchart@ideasonboard.com Cc: Paul Elder , michael.riesch@collabora.com, xuhf@rock-chips.com, stefan.klug@ideasonboard.com, kieran.bingham@ideasonboard.com, dan.scally@ideasonboard.com, jacopo.mondi@ideasonboard.com, nicolas.dufresne@collabora.com, libcamera-devel@lists.libcamera.org Subject: [RFC PATCH 04/19] include: ipa: rkisp2: Add rkisp2 ipa interface Date: Fri, 3 Jul 2026 21:25:10 +0900 Message-ID: <20260703122543.1991189-5-paul.elder@ideasonboard.com> X-Mailer: git-send-email 2.47.2 In-Reply-To: <20260703122543.1991189-1-paul.elder@ideasonboard.com> References: <20260703122543.1991189-1-paul.elder@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Add the IPA interface for the rkisp2. It is based on and is nearly identical to the rkisp1 IPA interface. Signed-off-by: Paul Elder --- This is also split into a separate patch to make the other patches smaller and easier to review. --- include/libcamera/ipa/meson.build | 1 + include/libcamera/ipa/rkisp2.mojom | 47 ++++++++++++++++++++++++++++++ 2 files changed, 48 insertions(+) create mode 100644 include/libcamera/ipa/rkisp2.mojom diff --git a/include/libcamera/ipa/meson.build b/include/libcamera/ipa/meson.build index 3ee3ada303c0..22c603165fa2 100644 --- a/include/libcamera/ipa/meson.build +++ b/include/libcamera/ipa/meson.build @@ -66,6 +66,7 @@ pipeline_ipa_mojom_mapping = { 'ipu3': 'ipu3.mojom', 'mali-c55': 'mali-c55.mojom', 'rkisp1': 'rkisp1.mojom', + 'rkisp2': 'rkisp2.mojom', 'rpi/pisp': 'raspberrypi.mojom', 'rpi/vc4': 'raspberrypi.mojom', 'simple': 'soft.mojom', diff --git a/include/libcamera/ipa/rkisp2.mojom b/include/libcamera/ipa/rkisp2.mojom new file mode 100644 index 000000000000..52284a29cd24 --- /dev/null +++ b/include/libcamera/ipa/rkisp2.mojom @@ -0,0 +1,47 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ + +/* + * \todo Document the interface and remove the related EXCLUDE_PATTERNS entry. + */ + +module ipa.rkisp2; + +import "include/libcamera/ipa/core.mojom"; + +/* + * colorSpaceEncoding comes from libcamera::ColorSpace::YcbcrEncoding; -1 for invalid + * colorSpaceRange comes from libcamera::ColorSpace::Range; -1 for invalid + */ +struct IPAConfigInfo { + libcamera.IPACameraSensorInfo sensorInfo; + libcamera.ControlInfoMap sensorControls; + int32 colorSpaceEncoding; + int32 colorSpaceRange; +}; + +interface IPARkISP2Interface { + init(libcamera.IPASettings settings, + libcamera.IPACameraSensorInfo sensorInfo, + libcamera.ControlInfoMap sensorControls) + => (int32 ret, libcamera.ControlInfoMap ipaControls); + start() => (int32 ret); + stop(); + + configure(IPAConfigInfo configInfo) + => (int32 ret, libcamera.ControlInfoMap ipaControls); + + mapBuffers(array buffers); + unmapBuffers(array ids); + + [async] queueRequest(uint32 frame, libcamera.ControlList reqControls); + [async] computeParams(uint32 frame, uint32 bufferId); + [async] processStats(uint32 frame, uint32 bufferId, + libcamera.ControlList sensorControls); +}; + +interface IPARkISP2EventInterface { + paramsComputed(uint32 frame, uint32 bufferId, uint32 bytesused); + setSensorControls(uint32 frame, libcamera.ControlList sensorControls); + metadataReady(libcamera.ControlList metadata); +}; + From patchwork Fri Jul 3 12:25:11 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Paul Elder X-Patchwork-Id: 27164 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 58D71C328C for ; Fri, 3 Jul 2026 12:26:19 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 16AD965FCF; Fri, 3 Jul 2026 14:26:19 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="hwnV1OTj"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 2993465FC3 for ; Fri, 3 Jul 2026 14:26:18 +0200 (CEST) Received: from neptunite.hamster-moth.ts.net (unknown [IPv6:2404:7a81:160:2100:a2cc:2f45:3bd7:2589]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 0C4C08E0; Fri, 3 Jul 2026 14:25:28 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1783081532; bh=+fLjhxaR5bCNOB2VrVIIuGV3Gcz1NpGvuAXUHsjk3xU=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=hwnV1OTj3WNoR8T2du0cUdK1yi2C6LF494FHWs38phLu4+p9tadR3FzmUkHlJtXcg F4driUbpp4RJemFF43X5VyGpl7Gc61W0rD1Iq6qhCPl3uP3YwYE8CKt23jEnJBkKmF EZL0OW7haWzxcrjB2lWsMAFB2V9ermFEVLJO/Rrs= From: Paul Elder To: laurent.pinchart@ideasonboard.com Cc: Paul Elder , michael.riesch@collabora.com, xuhf@rock-chips.com, stefan.klug@ideasonboard.com, kieran.bingham@ideasonboard.com, dan.scally@ideasonboard.com, jacopo.mondi@ideasonboard.com, nicolas.dufresne@collabora.com, libcamera-devel@lists.libcamera.org Subject: [RFC PATCH 05/19] ipa: rkisp2: Add rkisp2 ipa Date: Fri, 3 Jul 2026 21:25:11 +0900 Message-ID: <20260703122543.1991189-6-paul.elder@ideasonboard.com> X-Mailer: git-send-email 2.47.2 In-Reply-To: <20260703122543.1991189-1-paul.elder@ideasonboard.com> References: <20260703122543.1991189-1-paul.elder@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Implement the rkisp2 IPA. It is based on the rkisp1 IPA. No control algorithms are implemented yet, but this lays the foundation and plumbing for the IPA to function. The control algorithms will be added in later patches. Signed-off-by: Paul Elder --- This is also split from other patches to make them all easier to review. I put this patch before the pipeline handler patch so that the pipeline handler doesn't error out when it fails to find an IPA. --- meson_options.txt | 4 +- src/ipa/rkisp2/algorithms/meson.build | 5 + src/ipa/rkisp2/algorithms/module.h | 28 ++ src/ipa/rkisp2/ipa_context.h | 201 ++++++++++++ src/ipa/rkisp2/meson.build | 29 ++ src/ipa/rkisp2/rkisp2.cpp | 429 ++++++++++++++++++++++++++ 6 files changed, 694 insertions(+), 2 deletions(-) create mode 100644 src/ipa/rkisp2/algorithms/meson.build create mode 100644 src/ipa/rkisp2/algorithms/module.h create mode 100644 src/ipa/rkisp2/ipa_context.h create mode 100644 src/ipa/rkisp2/meson.build create mode 100644 src/ipa/rkisp2/rkisp2.cpp diff --git a/meson_options.txt b/meson_options.txt index 20baacc4fc65..2638c77cb8b0 100644 --- a/meson_options.txt +++ b/meson_options.txt @@ -48,8 +48,8 @@ option('gstreamer', option('ipas', type : 'array', - choices : ['ipu3', 'mali-c55', 'rkisp1', 'rpi/pisp', 'rpi/vc4', 'simple', - 'vimc'], + choices : ['ipu3', 'mali-c55', 'rkisp1', 'rkisp2', 'rpi/pisp', + 'rpi/vc4', 'simple', 'vimc'], description : 'Select which IPA modules to build') option('lc-compliance', diff --git a/src/ipa/rkisp2/algorithms/meson.build b/src/ipa/rkisp2/algorithms/meson.build new file mode 100644 index 000000000000..4bb81e48eb01 --- /dev/null +++ b/src/ipa/rkisp2/algorithms/meson.build @@ -0,0 +1,5 @@ +# SPDX-License-Identifier: CC0-1.0 + +rkisp2_ipa_algorithms = files([ +]) + diff --git a/src/ipa/rkisp2/algorithms/module.h b/src/ipa/rkisp2/algorithms/module.h new file mode 100644 index 000000000000..23507e1db81f --- /dev/null +++ b/src/ipa/rkisp2/algorithms/module.h @@ -0,0 +1,28 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2026, Ideas on Board Oy. + * + * RkISP2 IPA Module + */ + +#pragma once + +#include + +#include + +#include + +#include "ipa_context.h" +#include "params.h" + +namespace libcamera { + +namespace ipa::rkisp2 { + +using Module = ipa::Module; + +} /* namespace ipa::rkisp2 */ + +} /* namespace libcamera*/ diff --git a/src/ipa/rkisp2/ipa_context.h b/src/ipa/rkisp2/ipa_context.h new file mode 100644 index 000000000000..c806f1fe1bfa --- /dev/null +++ b/src/ipa/rkisp2/ipa_context.h @@ -0,0 +1,201 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2026, Ideas on Board Oy. + * + * RkISP2 IPA Context + * + */ + +#pragma once + +#include + +#include + +#include + +#include +#include +#include + +#include + +#include "libcamera/internal/debug_controls.h" +#include "libcamera/internal/matrix.h" +#include "libcamera/internal/vector.h" + +#include "libipa/agc_mean_luminance.h" +#include "libipa/awb.h" +#include "libipa/camera_sensor_helper.h" +#include "libipa/ccm.h" +#include "libipa/fc_queue.h" +#include "libipa/fixedpoint.h" +#include "libipa/lsc.h" + +namespace libcamera { + +namespace ipa::rkisp2 { + +struct IPAHwSettings { + unsigned int numAeCells; + unsigned int numHistogramBins; + unsigned int numHistogramWeights; + unsigned int numGammaOutSamples; + uint32_t supportedBlocks; + bool compand; +}; + +struct RKISP2AwbSession { + struct rkisp2_isp_window measureWindow; + bool enabled; +}; + +struct IPASessionConfiguration { + struct { + struct rkisp2_isp_window measureWindow; + struct rkisp2_isp_window measureWindow15; + } agc; + + struct RKISP2AwbSession awb; + + struct { + utils::Duration minExposureTime; + utils::Duration maxExposureTime; + double minAnalogueGain; + double maxAnalogueGain; + + int32_t defVBlank; + utils::Duration lineDuration; + Size size; + } sensor; + + struct { + int32_t colorSpaceEncoding; + int32_t colorSpaceRange; + } csm; + + bool raw; +}; + +struct IPAActiveState { + struct { + struct { + uint32_t exposure; + double gain; + } manual; + struct { + uint32_t exposure; + double gain; + double quantizationGain; + double yTarget; + } automatic; + + bool autoExposureEnabled; + bool autoGainEnabled; + double exposureValue; + controls::AeConstraintModeEnum constraintMode; + controls::AeExposureModeEnum exposureMode; + controls::AeMeteringModeEnum meteringMode; + utils::Duration minFrameDuration; + utils::Duration maxFrameDuration; + } agc; + + ipa::awb::ActiveState awb; + + struct { + double gamma; + } goc; + + ipa::ccm::ActiveState ccm; + + struct { + double lux; + } lux; + + struct { + controls::WdrModeEnum mode; + AgcMeanLuminance::AgcConstraint constraint; + double gain; + double strength; + } wdr; + + ipa::lsc::ActiveState lsc; + + struct { + Matrix csm; + bool update; + } csm; +}; + +struct IPAFrameContext : public FrameContext { + struct { + uint32_t exposure; + double gain; + double exposureValue; + double quantizationGain; + uint32_t vblank; + double yTarget; + bool autoExposureEnabled; + bool autoGainEnabled; + controls::AeConstraintModeEnum constraintMode; + controls::AeExposureModeEnum exposureMode; + controls::AeMeteringModeEnum meteringMode; + utils::Duration minFrameDuration; + utils::Duration maxFrameDuration; + utils::Duration frameDuration; + bool updateMetering; + bool autoExposureModeChange; + bool autoGainModeChange; + } agc; + + ipa::awb::FrameContext awb; + + struct { + double gamma; + bool update; + } goc; + + struct { + uint32_t exposure; + double gain; + } sensor; + + ipa::ccm::FrameContext ccm; + + struct { + double lux; + } lux; + + struct { + controls::WdrModeEnum mode; + double strength; + double gain; + } wdr; + + ipa::lsc::FrameContext lsc; +}; + +struct IPAContext { + IPAContext(unsigned int frameContextSize) + : frameContexts(frameContextSize) + { + } + + IPAHwSettings hw; + IPACameraSensorInfo sensorInfo; + IPASessionConfiguration configuration; + IPAActiveState activeState; + + FCQueue frameContexts; + + ControlInfoMap::Map ctrlMap; + + DebugMetadata debugMetadata; + + /* Interface to the Camera Helper */ + std::unique_ptr camHelper; +}; + +} /* namespace ipa::rkisp2 */ + +} /* namespace libcamera*/ diff --git a/src/ipa/rkisp2/meson.build b/src/ipa/rkisp2/meson.build new file mode 100644 index 000000000000..f2f435c2d66f --- /dev/null +++ b/src/ipa/rkisp2/meson.build @@ -0,0 +1,29 @@ +# SPDX-License-Identifier: CC0-1.0 + +subdir('algorithms') + +ipa_name = 'ipa_rkisp2' + +rkisp2_ipa_sources = files([ + 'rkisp2.cpp', +]) + +rkisp2_ipa_sources += rkisp2_ipa_algorithms + +mod = shared_module(ipa_name, rkisp2_ipa_sources, + name_prefix : '', + include_directories : [ipa_includes], + dependencies : [libcamera_private, libipa_dep], + install : true, + install_dir : ipa_install_dir) + +if ipa_sign_module + custom_target(ipa_name + '.so.sign', + input : mod, + output : ipa_name + '.so.sign', + command : [ipa_sign, ipa_priv_key, '@INPUT@', '@OUTPUT@'], + install : false, + build_by_default : true) +endif + +ipa_names += ipa_name diff --git a/src/ipa/rkisp2/rkisp2.cpp b/src/ipa/rkisp2/rkisp2.cpp new file mode 100644 index 000000000000..18053c7d802a --- /dev/null +++ b/src/ipa/rkisp2/rkisp2.cpp @@ -0,0 +1,429 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2026, Ideas on Board Oy. + * + * RkISP2 Image Processing Algorithms + */ + +#include +#include +#include +#include +#include +#include + +#include +#include + +#include +#include + +#include +#include +#include +#include + +#include +#include +#include + +#include "libcamera/internal/formats.h" +#include "libcamera/internal/mapped_framebuffer.h" +#include "libcamera/internal/yaml_parser.h" + +#include "algorithms/algorithm.h" + +#include "ipa_context.h" +#include "params.h" + +namespace libcamera { + +LOG_DEFINE_CATEGORY(IPARkISP2) + +using namespace std::literals::chrono_literals; + +namespace ipa::rkisp2 { + +/* Maximum number of frame contexts to be held */ +static constexpr uint32_t kMaxFrameContexts = 16; + +class IPARkISP2 : public IPARkISP2Interface, public Module +{ +public: + IPARkISP2(); + + int init(const IPASettings &settings, + const IPACameraSensorInfo &sensorInfo, + const ControlInfoMap &sensorControls, + ControlInfoMap *ipaControls) override; + int start() override; + void stop() override; + + int configure(const IPAConfigInfo &ipaConfig, + ControlInfoMap *ipaControls) override; + void mapBuffers(const std::vector &buffers) override; + void unmapBuffers(const std::vector &ids) override; + + void queueRequest(const uint32_t frame, const ControlList &controls) override; + void computeParams(const uint32_t frame, const uint32_t bufferId) override; + void initializeFrameContext(IPAFrameContext &frameContext, + const ControlList &controls); + void processStats(const uint32_t frame, const uint32_t bufferId, + const ControlList &sensorControls) override; + +protected: + std::string logPrefix() const override; + +private: + void updateControls(const IPACameraSensorInfo &sensorInfo, + const ControlInfoMap &sensorControls, + ControlInfoMap *ipaControls); + + void setControls(unsigned int frame, const IPAFrameContext &frameContext); + + std::map buffers_; + std::map mappedBuffers_; + + ControlInfoMap sensorControls_; + + /* Local parameter storage */ + struct IPAContext context_; +}; + +namespace { + +/* List of controls handled by the RkISP2 IPA */ +const ControlInfoMap::Map rkisp2Controls{ + { &controls::DebugMetadataEnable, ControlInfo(false, true, false) }, +}; + +} /* namespace */ + +IPARkISP2::IPARkISP2() + : context_(kMaxFrameContexts) +{ + context_.frameContexts.setInitCallback( + [this](IPAFrameContext &fc, const ControlList &c) { + this->initializeFrameContext(fc, c); + }); +} + +std::string IPARkISP2::logPrefix() const +{ + return "rkisp2"; +} + +int IPARkISP2::init(const IPASettings &settings, + const IPACameraSensorInfo &sensorInfo, + const ControlInfoMap &sensorControls, + ControlInfoMap *ipaControls) +{ + context_.sensorInfo = sensorInfo; + + context_.camHelper = CameraSensorHelperFactoryBase::create(settings.sensorModel); + if (!context_.camHelper) { + LOG(IPARkISP2, Error) + << "Failed to create camera sensor helper for " + << settings.sensorModel; + return -ENODEV; + } + + context_.configuration.sensor.lineDuration = + sensorInfo.minLineLength * 1.0s / sensorInfo.pixelRate; + + /* Load the tuning data file. */ + File file(settings.configurationFile); + if (!file.open(File::OpenModeFlag::ReadOnly)) { + int ret = file.error(); + LOG(IPARkISP2, Error) + << "Failed to open configuration file " + << settings.configurationFile << ": " << strerror(-ret); + return ret; + } + + std::unique_ptr data = YamlParser::parse(file); + if (!data) + return -EINVAL; + + unsigned int version = (*data)["version"].get(0); + if (version != 1) { + LOG(IPARkISP2, Error) + << "Invalid tuning file version " << version; + return -EINVAL; + } + + if (!data->contains("algorithms")) { + LOG(IPARkISP2, Error) + << "Tuning file doesn't contain any algorithm"; + return -EINVAL; + } + + int ret = createAlgorithms(context_, (*data)["algorithms"]); + if (ret) + return ret; + + /* Initialize controls. */ + updateControls(sensorInfo, sensorControls, ipaControls); + + return 0; +} + +int IPARkISP2::start() +{ + /* \todo Properly handle startup controls. */ + return 0; +} + +void IPARkISP2::stop() +{ + context_.frameContexts.clear(); +} + +int IPARkISP2::configure(const IPAConfigInfo &ipaConfig, + ControlInfoMap *ipaControls) +{ + sensorControls_ = ipaConfig.sensorControls; + + const auto itExp = sensorControls_.find(V4L2_CID_EXPOSURE); + int32_t minExposure = itExp->second.min().get(); + int32_t maxExposure = itExp->second.max().get(); + + const auto itGain = sensorControls_.find(V4L2_CID_ANALOGUE_GAIN); + int32_t minGain = itGain->second.min().get(); + int32_t maxGain = itGain->second.max().get(); + + LOG(IPARkISP2, Debug) + << "Exposure: [" << minExposure << ", " << maxExposure + << "], gain: [" << minGain << ", " << maxGain << "]"; + + /* Clear the IPA context before the streaming session. */ + context_.configuration = {}; + context_.activeState = {}; + context_.frameContexts.clear(); + + const IPACameraSensorInfo &info = ipaConfig.sensorInfo; + const ControlInfo vBlank = sensorControls_.find(V4L2_CID_VBLANK)->second; + context_.configuration.sensor.defVBlank = vBlank.def().get(); + context_.configuration.sensor.size = info.outputSize; + context_.configuration.sensor.lineDuration = info.minLineLength * 1.0s / info.pixelRate; + + /* Update the camera controls using the new sensor settings. */ + updateControls(info, sensorControls_, ipaControls); + + /* + * When the AGC computes the new exposure values for a frame, it needs + * to know the limits for exposure time and analogue gain. As it depends + * on the sensor, update it with the controls. + * + * \todo take VBLANK into account for maximum exposure time + */ + context_.configuration.sensor.minExposureTime = + minExposure * context_.configuration.sensor.lineDuration; + context_.configuration.sensor.maxExposureTime = + maxExposure * context_.configuration.sensor.lineDuration; + context_.configuration.sensor.minAnalogueGain = + context_.camHelper->gain(minGain); + context_.configuration.sensor.maxAnalogueGain = + context_.camHelper->gain(maxGain); + + context_.configuration.csm.colorSpaceEncoding = ipaConfig.colorSpaceEncoding; + context_.configuration.csm.colorSpaceRange = ipaConfig.colorSpaceRange; + + for (const auto &a : algorithms()) { + Algorithm *algo = static_cast(a.get()); + + int ret = algo->configure(context_, info); + if (ret) + return ret; + } + + return 0; +} + +void IPARkISP2::mapBuffers(const std::vector &buffers) +{ + for (const IPABuffer &buffer : buffers) { + auto elem = buffers_.emplace(std::piecewise_construct, + std::forward_as_tuple(buffer.id), + std::forward_as_tuple(buffer.planes)); + const FrameBuffer &fb = elem.first->second; + + MappedFrameBuffer mappedBuffer(&fb, MappedFrameBuffer::MapFlag::ReadWrite); + if (!mappedBuffer.isValid()) { + LOG(IPARkISP2, Fatal) << "Failed to mmap buffer: " + << strerror(mappedBuffer.error()); + } + + mappedBuffers_.emplace(buffer.id, std::move(mappedBuffer)); + } +} + +void IPARkISP2::unmapBuffers(const std::vector &ids) +{ + for (unsigned int id : ids) { + const auto fb = buffers_.find(id); + if (fb == buffers_.end()) + continue; + + mappedBuffers_.erase(id); + buffers_.erase(id); + } +} + +void IPARkISP2::queueRequest(const uint32_t frame, const ControlList &controls) +{ + context_.debugMetadata.enableByControl(controls); + + context_.frameContexts.getOrInitContext(frame, controls); +} + +void IPARkISP2::initializeFrameContext(IPAFrameContext &frameContext, + const ControlList &controls) +{ + for (const auto &a : algorithms()) { + Algorithm *algo = static_cast(a.get()); + algo->queueRequest(context_, frameContext.frame(), frameContext, controls); + } +} + +void IPARkISP2::computeParams(const uint32_t frame, const uint32_t bufferId) +{ + IPAFrameContext &frameContext = context_.frameContexts.getOrInitContext(frame); + + RkISP2Params params(mappedBuffers_.at(bufferId).planes()[0]); + + for (const auto &algo : algorithms()) + algo->prepare(context_, frame, frameContext, ¶ms); + + paramsComputed.emit(frame, bufferId, params.bytesused()); +} + +void IPARkISP2::processStats(const uint32_t frame, const uint32_t bufferId, + const ControlList &sensorControls) +{ + IPAFrameContext &frameContext = context_.frameContexts.getOrInitContext(frame); + + const rkisp2_stats_buffer *stats = reinterpret_cast( + mappedBuffers_.at(bufferId).planes()[0].data()); + + frameContext.sensor.exposure = + sensorControls.get(V4L2_CID_EXPOSURE).get(); + frameContext.sensor.gain = + context_.camHelper->gain(sensorControls.get(V4L2_CID_ANALOGUE_GAIN).get()); + + ControlList metadata(controls::controls); + + for (const auto &algo : algorithms()) + algo->process(context_, frame, frameContext, stats, metadata); + + setControls(frame, frameContext); + + metadataReady.emit(metadata); +} + +void IPARkISP2::setControls(unsigned int frame, const IPAFrameContext &frameContext) +{ + /* + * \todo The frame number is most likely wrong here, we need to take + * internal sensor delays and other timing parameters into account. + */ + + uint32_t exposure = frameContext.agc.exposure; + uint32_t gain = context_.camHelper->gainCode(frameContext.agc.gain); + uint32_t vblank = frameContext.agc.vblank; + + LOG(IPARkISP2, Debug) + << "Set controls for frame " << frame << ": exposure " << exposure + << ", gain " << frameContext.agc.gain << ", vblank " << vblank; + + ControlList ctrls(sensorControls_); + if (frameContext.agc.exposure * frameContext.agc.gain > 0) { + ctrls.set(V4L2_CID_EXPOSURE, static_cast(exposure)); + ctrls.set(V4L2_CID_ANALOGUE_GAIN, static_cast(gain)); + } + ctrls.set(V4L2_CID_VBLANK, static_cast(vblank)); + + setSensorControls.emit(frame, ctrls); +} + +void IPARkISP2::updateControls(const IPACameraSensorInfo &sensorInfo, + const ControlInfoMap &sensorControls, + ControlInfoMap *ipaControls) +{ + ControlInfoMap::Map ctrlMap = rkisp2Controls; + + /* + * Compute exposure time limits from the V4L2_CID_EXPOSURE control + * limits and the line duration. + */ + double lineDuration = context_.configuration.sensor.lineDuration.get(); + const ControlInfo &v4l2Exposure = sensorControls.find(V4L2_CID_EXPOSURE)->second; + int32_t minExposure = v4l2Exposure.min().get() * lineDuration; + int32_t maxExposure = v4l2Exposure.max().get() * lineDuration; + int32_t defExposure = v4l2Exposure.def().get() * lineDuration; + ctrlMap.emplace(std::piecewise_construct, + std::forward_as_tuple(&controls::ExposureTime), + std::forward_as_tuple(minExposure, maxExposure, defExposure)); + + /* Compute the analogue gain limits. */ + const ControlInfo &v4l2Gain = sensorControls.find(V4L2_CID_ANALOGUE_GAIN)->second; + float minGain = context_.camHelper->gain(v4l2Gain.min().get()); + float maxGain = context_.camHelper->gain(v4l2Gain.max().get()); + float defGain = context_.camHelper->gain(v4l2Gain.def().get()); + ctrlMap.emplace(std::piecewise_construct, + std::forward_as_tuple(&controls::AnalogueGain), + std::forward_as_tuple(minGain, maxGain, defGain)); + + /* + * Compute the frame duration limits. + * + * The frame length is computed assuming a fixed line length combined + * with the vertical frame sizes. + */ + const ControlInfo &v4l2HBlank = sensorControls.find(V4L2_CID_HBLANK)->second; + uint32_t hblank = v4l2HBlank.def().get(); + uint32_t lineLength = sensorInfo.outputSize.width + hblank; + + const ControlInfo &v4l2VBlank = sensorControls.find(V4L2_CID_VBLANK)->second; + std::array frameHeights{ + v4l2VBlank.min().get() + sensorInfo.outputSize.height, + v4l2VBlank.max().get() + sensorInfo.outputSize.height, + v4l2VBlank.def().get() + sensorInfo.outputSize.height, + }; + + std::array frameDurations; + for (unsigned int i = 0; i < frameHeights.size(); ++i) { + uint64_t frameSize = lineLength * frameHeights[i]; + frameDurations[i] = frameSize / (sensorInfo.pixelRate / 1000000U); + } + + /* \todo Move this (and other agc-related controls) to agc */ + context_.ctrlMap[&controls::FrameDurationLimits] = + ControlInfo(frameDurations[0], frameDurations[1], + ControlValue(Span{ { frameDurations[2], frameDurations[2] } })); + + ctrlMap.insert(context_.ctrlMap.begin(), context_.ctrlMap.end()); + *ipaControls = ControlInfoMap(std::move(ctrlMap), controls::controls); +} + +} /* namespace ipa::rkisp2 */ + +/* + * External IPA module interface + */ + +extern "C" { +const struct IPAModuleInfo ipaModuleInfo = { + IPA_MODULE_API_VERSION, + 1, + "rkisp2", + "rkisp2", +}; + +IPAInterface *ipaCreate() +{ + return new ipa::rkisp2::IPARkISP2(); +} +} + +} /* namespace libcamera */ From patchwork Fri Jul 3 12:25:12 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Paul Elder X-Patchwork-Id: 27165 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 0FA98C328C for ; Fri, 3 Jul 2026 12:26:24 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id B24E065FD3; Fri, 3 Jul 2026 14:26:23 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="LdSLUBx2"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id B05BC65FC3 for ; Fri, 3 Jul 2026 14:26:21 +0200 (CEST) Received: from neptunite.hamster-moth.ts.net (unknown [IPv6:2404:7a81:160:2100:a2cc:2f45:3bd7:2589]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id CC5791121; Fri, 3 Jul 2026 14:25:32 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1783081535; bh=W3siiXg6HQg73xR/LjWsqYVkkFKQb9abJkGtdXeF6hU=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=LdSLUBx2+5Gw6NpQuwiuXJ2MGSx2gxKvbfpYUUMLvxX6/voKSnN7gtx3DuQsZmW3i sAndyftUvANyUKwy6WP4sQD7ySbQxae7QrDPtr8bS+klkoiCNrwxKSRniPeVzCEaQ7 DaKqhI5VCOBnM9Sg/QKKFWxEfTGqXHXLhWKKZO8Q= From: Paul Elder To: laurent.pinchart@ideasonboard.com Cc: Paul Elder , michael.riesch@collabora.com, xuhf@rock-chips.com, stefan.klug@ideasonboard.com, kieran.bingham@ideasonboard.com, dan.scally@ideasonboard.com, jacopo.mondi@ideasonboard.com, nicolas.dufresne@collabora.com, libcamera-devel@lists.libcamera.org Subject: [RFC PATCH 06/19] pipeline: rkisp2: Implement pipeline handler for rkisp2 Date: Fri, 3 Jul 2026 21:25:12 +0900 Message-ID: <20260703122543.1991189-7-paul.elder@ideasonboard.com> X-Mailer: git-send-email 2.47.2 In-Reply-To: <20260703122543.1991189-1-paul.elder@ideasonboard.com> References: <20260703122543.1991189-1-paul.elder@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Implement the pipeline handler for rkisp2. This currently supports the ISP version integrated on the Rockchip RK3588. The current version only supports memory-to-memory mode and only the main path and only raw sensors. Signed-off-by: Paul Elder --- Documentation/runtime_configuration.rst | 9 + meson.build | 1 + meson_options.txt | 1 + src/libcamera/pipeline/rkisp2/meson.build | 5 + src/libcamera/pipeline/rkisp2/rkisp2.cpp | 1305 +++++++++++++++++++++ 5 files changed, 1321 insertions(+) create mode 100644 src/libcamera/pipeline/rkisp2/meson.build create mode 100644 src/libcamera/pipeline/rkisp2/rkisp2.cpp diff --git a/Documentation/runtime_configuration.rst b/Documentation/runtime_configuration.rst index 2cdffb335a66..15dc118bff7b 100644 --- a/Documentation/runtime_configuration.rst +++ b/Documentation/runtime_configuration.rst @@ -81,6 +81,8 @@ Configuration file example supported_devices: - driver: mxc-isi software_isp: true + rkisp2: + isp_enable: true software_isp: copy_input_buffer: false measure: @@ -157,6 +159,13 @@ pipelines.simple.supported_devices.driver, pipelines.simple.supported_devices.so Example `software_isp` value: ``true`` +pipelines.rkisp2.isp_enable + Configure whether or not to use the ISP. Default (when unconfigured) is + true. When set to false the ISP will not be used, so only the VICAP will be + used for capture. + + Example value: ``false`` + software_isp.copy_input_buffer Define whether input buffers should be copied into standard (cached) memory in software ISP. This is done by default to prevent very slow diff --git a/meson.build b/meson.build index 3b8d54a654a3..a86ab176b9ae 100644 --- a/meson.build +++ b/meson.build @@ -219,6 +219,7 @@ pipelines_support = { 'ipu3': arch_x86, 'mali-c55': arch_arm, 'rkisp1': arch_arm, + 'rkisp2': arch_arm, 'rpi/pisp': arch_arm, 'rpi/vc4': arch_arm, 'simple': ['any'], diff --git a/meson_options.txt b/meson_options.txt index 2638c77cb8b0..5443e3698432 100644 --- a/meson_options.txt +++ b/meson_options.txt @@ -82,6 +82,7 @@ option('pipelines', 'ipu3', 'mali-c55', 'rkisp1', + 'rkisp2', 'rpi/pisp', 'rpi/vc4', 'simple', diff --git a/src/libcamera/pipeline/rkisp2/meson.build b/src/libcamera/pipeline/rkisp2/meson.build new file mode 100644 index 000000000000..86f43fb7fcc9 --- /dev/null +++ b/src/libcamera/pipeline/rkisp2/meson.build @@ -0,0 +1,5 @@ +# SPDX-License-Identifier: CC0-1.0 + +libcamera_internal_sources += files([ + 'rkisp2.cpp', +]) diff --git a/src/libcamera/pipeline/rkisp2/rkisp2.cpp b/src/libcamera/pipeline/rkisp2/rkisp2.cpp new file mode 100644 index 000000000000..0d335a980e32 --- /dev/null +++ b/src/libcamera/pipeline/rkisp2/rkisp2.cpp @@ -0,0 +1,1305 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2026, Ideas on Board Oy. + * + * Pipeline handler for Rockchip ISP2 + */ + +#include +#include +#include +#include +#include + +#include +#include + +#include +#include +#include +#include +#include + +#include +#include +#include + +#include "libcamera/internal/buffer_queue.h" +#include "libcamera/internal/camera.h" +#include "libcamera/internal/camera_manager.h" +#include "libcamera/internal/camera_sensor.h" +#include "libcamera/internal/delayed_controls.h" +#include "libcamera/internal/device_enumerator.h" +#include "libcamera/internal/formats.h" +#include "libcamera/internal/framebuffer.h" +#include "libcamera/internal/global_configuration.h" +#include "libcamera/internal/ipa_manager.h" +#include "libcamera/internal/media_device.h" +#include "libcamera/internal/pipeline_handler.h" +#include "libcamera/internal/request.h" +#include "libcamera/internal/sequence_sync_helper.h" +#include "libcamera/internal/v4l2_subdevice.h" +#include "libcamera/internal/v4l2_videodevice.h" + +namespace libcamera { + +static const Size ispMaxSize = Size(4416, 3312); +static const Size vicapMaxSize = Size(8192, 8192); + +/* \todo Re-add NV21 and YUV422 and YVU422 after it's fixed in the driver */ +const std::map formatToMediaBus = { + { formats::UYVY, MEDIA_BUS_FMT_YUYV8_2X8 }, + { formats::NV12, MEDIA_BUS_FMT_YUYV8_2X8 }, + { formats::NV16, MEDIA_BUS_FMT_YUYV8_2X8 }, + { formats::NV61, MEDIA_BUS_FMT_YUYV8_2X8 }, + { formats::YUV420, MEDIA_BUS_FMT_YUYV8_1_5X8 }, + { formats::YVU420, MEDIA_BUS_FMT_YUYV8_1_5X8 }, + { formats::R8, MEDIA_BUS_FMT_YUYV8_2X8 }, +}; + +/* \todo Deduplicate this (we have the same thing in rkisp1) */ +const std::map rawFormats = { + { formats::SBGGR8, MEDIA_BUS_FMT_SBGGR8_1X8 }, + { formats::SGBRG8, MEDIA_BUS_FMT_SGBRG8_1X8 }, + { formats::SGRBG8, MEDIA_BUS_FMT_SGRBG8_1X8 }, + { formats::SRGGB8, MEDIA_BUS_FMT_SRGGB8_1X8 }, + { formats::SBGGR10, MEDIA_BUS_FMT_SBGGR10_1X10 }, + { formats::SGBRG10, MEDIA_BUS_FMT_SGBRG10_1X10 }, + { formats::SGRBG10, MEDIA_BUS_FMT_SGRBG10_1X10 }, + { formats::SRGGB10, MEDIA_BUS_FMT_SRGGB10_1X10 }, + { formats::SBGGR12, MEDIA_BUS_FMT_SBGGR12_1X12 }, + { formats::SGBRG12, MEDIA_BUS_FMT_SGBRG12_1X12 }, + { formats::SGRBG12, MEDIA_BUS_FMT_SGRBG12_1X12 }, + { formats::SRGGB12, MEDIA_BUS_FMT_SRGGB12_1X12 }, +}; + +LOG_DEFINE_CATEGORY(RkISP2) + +class PipelineHandlerRkISP2; + +struct RkISP2FrameInfo { + RkISP2FrameInfo(Request *_request, + FrameBuffer *_buffer, + const ControlList &_metadata = ControlList(controls::controls), + bool _metadataProcessed = false) + : request(_request), mainPathBuffer(_buffer), metadataProcessed(_metadataProcessed) + { + metadata.merge(_metadata); + } + + Request *request = nullptr; + FrameBuffer *mainPathBuffer = nullptr; + ControlList metadata = ControlList(controls::controls); + bool metadataProcessed = false; +}; + +struct RkISP2RequestInfo { + Request *request = nullptr; + size_t sequence = 0; + bool sequenceValid = false; +}; + +class RkISP2CameraData : public Camera::Private +{ +public: + RkISP2CameraData(PipelineHandler *pipe, MediaDevice *media) + : Camera::Private(pipe), media_(media), frame_(0) + { + } + + ~RkISP2CameraData() + { + } + + int init(bool usingIsp, CameraSensor *sensor, V4L2VideoDevice *video, + V4L2VideoDevice *rawrd, V4L2Subdevice *isp, + V4L2VideoDevice *mainPath, V4L2VideoDevice *param, + V4L2VideoDevice *stat); + PixelFormat getSensorFormat(unsigned int mbusCode, const Size &size, + const Size &maxSize); + + PipelineHandlerRkISP2 *pipe(); + const PipelineHandlerRkISP2 *pipe() const; + int loadIPA(); + + void computeParamBuffers(unsigned int frame); + + void tryCompleteRequest(Request *request, FrameBuffer *buffer); + void tryCompleteRequest(const ControlList &metadata); + + void vicapBufferReady(FrameBuffer *buffer); + void rawrdBufferReady(FrameBuffer *buffer); + void ispBufferReady(FrameBuffer *buffer); + void statBufferReady(FrameBuffer *buffer); + + void paramsComputed(unsigned int frame, unsigned int bufferId, unsigned int bytesused); + void setSensorControls(unsigned int frame, const ControlList &sensorControls); + void metadataReady(const ControlList &metadata); + + /* These are the buffers that go between the vicap and the isp */ + std::vector> internalBuffers_; + + std::vector> statBuffers_; + std::unique_ptr paramQueue_; + + std::unique_ptr delayedCtrls_; + + /* + * These are the buffers that are half-complete, as in either metadata + * is ready or the frame has been completed from the ISP. Both + * ready-handlers populate and complete from the front of the queue. + * + * The stat-ready handler flushes the queue however, because in general + * more images complete than stats do, so this prevents the queue from + * perpetually growing and making the metadata out of date. + */ + std::deque pendingCompleteBuffers_; + + /* + * This is for tracking sequence numbers of requests for synchronizing + * between stats and params and images for any dropped frames + */ + std::deque pendingCompleteRequests_; + + std::vector ipaBuffers_; + + bool usingIsp_; + bool isRaw_; + + MediaDevice *media_; + CameraSensor *sensor_; + V4L2VideoDevice *video_; + V4L2VideoDevice *rawrd_; + V4L2Subdevice *isp_; + V4L2VideoDevice *mainPath_; + V4L2VideoDevice *param_; + V4L2VideoDevice *stat_; + Stream stream_; + + std::unique_ptr ipa_; + ControlInfoMap ipaControls_; + + /* + * The sensor frame sequence of the last request queued to the pipeline + * handler + */ + unsigned int frame_; + SequenceSyncHelper syncHelper_; +}; + +class RkISP2CameraConfiguration : public CameraConfiguration +{ +public: + RkISP2CameraConfiguration(const RkISP2CameraData *data); + + Status validate() override; + + const V4L2SubdeviceFormat &sensorFormat() { return sensorFormat_; } + +private: + const RkISP2CameraData *data_; + + V4L2SubdeviceFormat sensorFormat_; +}; + +namespace { + +/* + * This many internal buffers (or rather parameter and statistics buffer + * pairs) ensures that the pipeline runs smoothly, without frame drops. + */ +static constexpr unsigned int kRkISP2MinBufferCount = 4; + +} /* namespace */ + +class PipelineHandlerRkISP2 : public PipelineHandler +{ +public: + PipelineHandlerRkISP2(CameraManager *manager); + + std::unique_ptr + generateConfiguration(Camera *camera, + Span roles) override; + + int configure(Camera *camera, CameraConfiguration *config) override; + + int exportFrameBuffers(Camera *camera, Stream *stream, + std::vector> *buffers) override; + + int start(Camera *camera, const ControlList *controls) override; + void stopDevice(Camera *camera) override; + + int queueRequestDevice(Camera *camera, Request *request) override; + + bool match(DeviceEnumerator *enumerator) override; + +private: + RkISP2CameraData *cameraData(Camera *camera) + { + return static_cast(camera->_d()); + } + + int updateControls(RkISP2CameraData *data); + bool createCamera(bool usingIsp); + int processControls(RkISP2CameraData *data, const ControlList &ctrls); + + int allocateBuffers(Camera *camera); + int freeBuffers(Camera *camera); + + Size clampSensorSize(RkISP2CameraData *data, unsigned int mbus, + const Size &maxSize); + + std::shared_ptr media_; + std::unique_ptr sensor_; + std::unique_ptr csi_; + std::unique_ptr cif_; + std::unique_ptr video_; + + std::shared_ptr ispMedia_; + std::unique_ptr rawrd_; + std::unique_ptr isp_; + std::unique_ptr mainPath_; + std::unique_ptr param_; + std::unique_ptr stat_; +}; + +int RkISP2CameraData::init(bool usingIsp, CameraSensor *sensor, + V4L2VideoDevice *video, V4L2VideoDevice *rawrd, + V4L2Subdevice *isp, + V4L2VideoDevice *mainPath, V4L2VideoDevice *param, + V4L2VideoDevice *stat) +{ + usingIsp_ = usingIsp; + sensor_ = sensor; + video_ = video; + rawrd_ = rawrd; + isp_ = isp; + mainPath_ = mainPath; + param_ = param; + stat_ = stat; + + ControlInfoMap::Map ctrls; + + auto &testPatterns = sensor_->testPatternModes(); + if (testPatterns.size()) { + ctrls.emplace(&controls::draft::TestPatternMode, + ControlInfo(testPatterns.front(), testPatterns.back(), + testPatterns.front())); + } else + LOG(RkISP2, Warning) << "Sensor doesn't support any test pattern modes"; + + controlInfo_ = ControlInfoMap(std::move(ctrls), controls::controls); + + video_->bufferReady.connect(this, &RkISP2CameraData::vicapBufferReady); + if (mainPath_) { + rawrd_->bufferReady.connect(this, &RkISP2CameraData::rawrdBufferReady); + mainPath_->bufferReady.connect(this, &RkISP2CameraData::ispBufferReady); + stat_->bufferReady.connect(this, &RkISP2CameraData::statBufferReady); + } + + return 0; +} + +PixelFormat RkISP2CameraData::getSensorFormat(unsigned int mbusCode, + const Size &size, + const Size &maxSize) +{ + std::vector mbusCodes = { mbusCode }; + V4L2SubdeviceFormat format = sensor_->getFormat(mbusCodes, size, maxSize); + const auto &ret = std::find_if(rawFormats.begin(), rawFormats.end(), + [format](const auto &value) { return value.second == format.code; }); + if (ret != rawFormats.end()) + return (*ret).first; + + LOG(RkISP2, Error) << "No raw format supported by sensor"; + return formats::SRGGB10; +} + +PipelineHandlerRkISP2 *RkISP2CameraData::pipe() +{ + return static_cast(Camera::Private::pipe()); +} + +const PipelineHandlerRkISP2 *RkISP2CameraData::pipe() const +{ + return static_cast(Camera::Private::pipe()); +} + +int RkISP2CameraData::loadIPA() +{ + ipa_ = pipe()->createIPA(1, 1); + if (!ipa_) + return -ENOENT; + + ipa_->paramsComputed.connect(this, &RkISP2CameraData::paramsComputed); + ipa_->setSensorControls.connect(this, &RkISP2CameraData::setSensorControls); + ipa_->metadataReady.connect(this, &RkISP2CameraData::metadataReady); + + /* The IPA tuning file is made from the sensor name. */ + std::string ipaTuningFile = + ipa_->configurationFile(sensor_->model() + ".yaml", "uncalibrated.yaml"); + + IPACameraSensorInfo sensorInfo{}; + int ret = sensor_->sensorInfo(&sensorInfo); + if (ret) { + LOG(RkISP2, Error) << "Camera sensor information not available"; + return ret; + } + + ret = ipa_->init({ ipaTuningFile, sensor_->model() }, + sensorInfo, sensor_->controls(), + &ipaControls_); + if (ret < 0) { + LOG(RkISP2, Error) << "IPA initialization failure"; + return ret; + } + + return 0; +} + +void RkISP2CameraData::computeParamBuffers(unsigned int frame) +{ + while (paramQueue_->nextSequence() <= frame) { + if (paramQueue_->empty(BufferQueue::Idle)) { + LOG(RkISP2, Warning) << "Out of param buffers"; + return; + } + + uint32_t paramsSequence; + FrameBuffer *paramBuffer = paramQueue_->front(BufferQueue::Idle); + paramQueue_->prepareBuffer(¶msSequence); + ipa_->computeParams(paramsSequence, paramBuffer->cookie()); + } +} + +void RkISP2CameraData::vicapBufferReady(FrameBuffer *buffer) +{ + Request *request = buffer->request(); + + if (!mainPath_ || isRaw_) { + pipe()->completeBuffer(request, buffer); + pipe()->completeRequest(request); + return; + } + + if (buffer->metadata().status == FrameMetadata::FrameCancelled) + return; + + rawrd_->queueBuffer(buffer); +} + +void RkISP2CameraData::rawrdBufferReady(FrameBuffer *buffer) +{ + if (buffer->metadata().status == FrameMetadata::FrameCancelled) + return; + + video_->queueBuffer(buffer); +} + +void RkISP2CameraData::ispBufferReady(FrameBuffer *buffer) +{ + Request *request = buffer->request(); + + if (buffer->metadata().status == FrameMetadata::FrameCancelled) { + syncHelper_.cancelFrame(); + pipe()->completeBuffer(request, buffer); + pipe()->completeRequest(request); + return; + } + + RkISP2RequestInfo info = pendingCompleteRequests_.front(); + pendingCompleteRequests_.pop_front(); + ASSERT(info.request == buffer->request()); + + int droppedFrames = syncHelper_.gotFrame(info.sequence, buffer->metadata().sequence); + if (droppedFrames > 0) { + LOG(RkISP2, Debug) + << "Dropped frames " << droppedFrames << " expected " + << info.sequence << " got " << buffer->metadata().sequence; + } + + pipe()->completeBuffer(request, buffer); + tryCompleteRequest(request, buffer); +} + +void RkISP2CameraData::statBufferReady(FrameBuffer *buffer) +{ + size_t sequence = buffer->metadata().sequence; + + if (buffer->metadata().status == FrameMetadata::FrameCancelled) + return; + + ipa_->processStats(sequence, buffer->cookie(), + delayedCtrls_->get(sequence)); + + stat_->queueBuffer(buffer); +} + +void RkISP2CameraData::paramsComputed(unsigned int frame, + unsigned int bufferId, + unsigned int bytesused) +{ + FrameBuffer *buffer = paramQueue_->front(BufferQueue::Preparing); + + ASSERT(buffer->cookie() == bufferId); + + buffer->_d()->metadata().planes()[0].bytesused = bytesused; + + int ret = paramQueue_->preparedBuffer(); + if (ret < 0) { + LOG(RkISP2, Error) + << "Failed to queue parameter buffer for frame " + << frame << ": " << strerror(-ret); + } +} + +void RkISP2CameraData::setSensorControls([[maybe_unused]] unsigned int frame, + const ControlList &sensorControls) +{ + delayedCtrls_->push(sensorControls); +} + +void RkISP2CameraData::metadataReady(const ControlList &metadata) +{ + tryCompleteRequest(metadata); +} + +void RkISP2CameraData::tryCompleteRequest(Request *request, FrameBuffer *buffer) +{ + if (pendingCompleteBuffers_.empty()) { + pendingCompleteBuffers_.emplace_back(request, buffer); + return; + } + + RkISP2FrameInfo &info = pendingCompleteBuffers_.front(); + if (info.request != nullptr || info.mainPathBuffer != nullptr || !info.metadataProcessed) { + pendingCompleteBuffers_.emplace_back(request, buffer); + return; + } + + info = pendingCompleteBuffers_.front(); + request->_d()->metadata().merge(info.metadata); + pendingCompleteBuffers_.pop_front(); + + pipe()->completeRequest(request); +} + +void RkISP2CameraData::tryCompleteRequest(const ControlList &metadata) +{ + if (pendingCompleteBuffers_.empty()) { + pendingCompleteBuffers_.emplace_back(nullptr, nullptr, metadata, true); + return; + } + + RkISP2FrameInfo &info = pendingCompleteBuffers_.front(); + if (!info.metadata.empty() || info.metadataProcessed) { + /* + * Generally more metadata complete than images, so flush the + * queue when adding new metadata to to the queue to prevent + * metadata in the queue from becoming too old + */ + pendingCompleteBuffers_.clear(); + pendingCompleteBuffers_.emplace_back(nullptr, nullptr, metadata, true); + return; + } + + Request *request = info.request; + request->_d()->metadata().merge(metadata); + pendingCompleteBuffers_.pop_front(); + + pipe()->completeRequest(request); +} + +RkISP2CameraConfiguration::RkISP2CameraConfiguration(const RkISP2CameraData *data) + : CameraConfiguration(), data_(data) +{ +} + +CameraConfiguration::Status RkISP2CameraConfiguration::validate() +{ + const CameraSensor *sensor = data_->sensor_; + std::vector mbusCodes; + Status status = Valid; + + if (config_.empty()) + return Invalid; + + /* + * Make sure that if a sensor configuration has been requested it + * is valid. + */ + if (sensorConfig) { + if (!sensorConfig->isValid()) { + LOG(RkISP2, Error) + << "Invalid sensor configuration request"; + + return Invalid; + } + + unsigned int bitDepth = sensorConfig->bitDepth; + if (bitDepth != 8 && bitDepth != 10 && bitDepth != 12) { + LOG(RkISP2, Error) + << "Invalid sensor configuration bit depth"; + + return Invalid; + } + } + + /* \todo Support self path */ + if (config_.size() != 1) { + config_.resize(1); + status = Adjusted; + } + + StreamConfiguration &cfg = config_[0]; + + /* \todo Support YUV sensors */ + const PixelFormatInfo &info = PixelFormatInfo::info(cfg.pixelFormat); + bool usingIsp = data_->usingIsp_; + if (usingIsp && info.colourEncoding == PixelFormatInfo::ColourEncodingRAW) + usingIsp = false; + + const Size &maxSize = usingIsp ? ispMaxSize : vicapMaxSize; + + if (!usingIsp) { + if (!rawFormats.count(cfg.pixelFormat)) { + cfg.pixelFormat = formats::SRGGB10; + status = Adjusted; + } + + unsigned int mbusCode = rawFormats.at(cfg.pixelFormat); + auto sizes = sensor->sizes(mbusCode); + + Size bestSize; + for (const Size &s : sizes) { + /* Ignore smaller sizes. */ + if (s.width < cfg.size.width || + s.height < cfg.size.height) + continue; + + /* Make sure the width stays in the limits. */ + if (s.width > maxSize.width) + continue; + + bestSize = s; + break; + } + + if (bestSize.isNull()) { + LOG(RkISP2, Error) << "Unable to find a suitable sensor format"; + return Invalid; + } + + if (bestSize != cfg.size) + status = Adjusted; + cfg.size = bestSize; + + mbusCodes = { mbusCode }; + sensorFormat_ = sensor->getFormat(mbusCodes, cfg.size, maxSize); + + ASSERT(sensorFormat_.code == mbusCode); + ASSERT(sensorFormat_.size == cfg.size); + + return status; + } + + if (!formatToMediaBus.count(cfg.pixelFormat)) { + cfg.pixelFormat = formats::UYVY; + status = Adjusted; + } + + /* \todo Adjust sizes a bit better */ + if (cfg.size > ispMaxSize) { + cfg.size = ispMaxSize; + status = Adjusted; + } + V4L2DeviceFormat format; + format.fourcc = data_->video_->toV4L2PixelFormat(cfg.pixelFormat); + format.size = cfg.size; + + int ret = data_->mainPath_->tryFormat(&format); + if (ret) + return Invalid; + + cfg.bufferCount = 4; + + std::transform(rawFormats.begin(), rawFormats.end(), + std::back_inserter(mbusCodes), + [](const auto &value) { return value.second; }); + sensorFormat_ = sensor->getFormat(mbusCodes, cfg.size, maxSize); + if (sensorFormat_.size.isNull()) + status = Invalid; + + return status; +} + +PipelineHandlerRkISP2::PipelineHandlerRkISP2(CameraManager *manager) + : PipelineHandler(manager) +{ +} + +Size PipelineHandlerRkISP2::clampSensorSize(RkISP2CameraData *data, unsigned int mbus, + const Size &maxSize) +{ + const std::vector &sizes = data->sensor_->sizes(mbus); + + for (auto it = sizes.rbegin(); it != sizes.rend(); ++it) { + if (it->width <= maxSize.width && + it->height <= maxSize.height) + return *it; + } + + return *sizes.begin(); +} + +std::unique_ptr +PipelineHandlerRkISP2::generateConfiguration(Camera *camera, + Span roles) +{ + RkISP2CameraData *data = cameraData(camera); + auto config = std::make_unique(data); + if (roles.empty()) + return config; + + if (roles.size() > 1) { + LOG(RkISP2, Error) << "Too many roles requested"; + return config; + } + + const StreamRole &role = roles[0]; + + bool isRaw = !data->usingIsp_ || role == StreamRole::Raw; + Size maxSize = isRaw ? vicapMaxSize : ispMaxSize; + + Size defaultSize = { 1920, 1080 }; + unsigned int defaultMbusCode = MEDIA_BUS_FMT_SRGGB10_1X10; + PixelFormat rawFormat = data->getSensorFormat(defaultMbusCode, + defaultSize, maxSize); + + /* Enumerate formats */ + std::vector sizes = { { Size(32, 32), maxSize } }; + auto makeStream = [sizes](auto const &pair) { + return std::make_pair(pair.first, sizes); + }; + std::map> streamFormats; + std::transform(formatToMediaBus.begin(), formatToMediaBus.end(), + std::inserter(streamFormats, streamFormats.end()), + makeStream); + std::transform(rawFormats.begin(), rawFormats.end(), + std::inserter(streamFormats, streamFormats.end()), + makeStream); + + StreamFormats formats(streamFormats); + StreamConfiguration cfg(formats); + /* UYVY is always supported by this ISP */ + cfg.pixelFormat = isRaw ? rawFormat : formats::UYVY; + cfg.size = clampSensorSize(data, defaultMbusCode, maxSize); + cfg.colorSpace = isRaw ? ColorSpace::Raw : ColorSpace::Sycc; + cfg.bufferCount = 4; + + config->addConfiguration(cfg); + + config->validate(); + + return config; +} + +int PipelineHandlerRkISP2::configure(Camera *camera, + CameraConfiguration *c) +{ + RkISP2CameraData *data = cameraData(camera); + RkISP2CameraConfiguration *config = + static_cast(c); + /* \todo Support multiple streams */ + StreamConfiguration &cfg = config->at(0); + int ret; + + const PixelFormatInfo &info = PixelFormatInfo::info(cfg.pixelFormat); + data->isRaw_ = info.colourEncoding == PixelFormatInfo::ColourEncodingRAW; + + V4L2SubdeviceFormat format = config->sensorFormat(); + LOG(RkISP2, Debug) << "Configuring sensor with " << format; + + if (config->sensorConfig) + ret = sensor_->applyConfiguration(*config->sensorConfig, + Transform::Identity, &format); + else + ret = sensor_->setFormat(&format); + if (ret < 0) + return ret; + + LOG(RkISP2, Debug) << "Sensor configured with " << format; + + LOG(RkISP2, Debug) << "Configuring CSI with : " << format; + ret = csi_->setFormat(0, &format); + if (ret) + return ret; + + LOG(RkISP2, Debug) << "Configuring VICAP with : " << format; + ret = cif_->setFormat(0, &format); + if (ret) + return ret; + + ret = cif_->setFormat(1, &format); + if (ret) + return ret; + + Size maxSize = data->isRaw_ ? vicapMaxSize : ispMaxSize; + PixelFormat vicapPixelFormat = + data->getSensorFormat(format.code, format.size, maxSize); + + V4L2DeviceFormat vicapOutputFormat; + vicapOutputFormat.fourcc = video_->toV4L2PixelFormat(vicapPixelFormat); + vicapOutputFormat.size = format.size; + + LOG(RkISP2, Debug) << "Configuring VICAP capture node with : " << vicapOutputFormat; + ret = data->video_->setFormat(&vicapOutputFormat); + if (ret) + return ret; + + if (!data->usingIsp_ || data->isRaw_) { + cfg.setStream(&data->stream_); + cfg.stride = vicapOutputFormat.planes[0].bpl; + return 0; + } + + /* + * \todo Figure out if these should go in the pipeline handler + * or camera data + */ + LOG(RkISP2, Debug) << "Configuring rawrd0 with: " << vicapOutputFormat; + ret = rawrd_->setFormat(&vicapOutputFormat); + if (ret) + return ret; + + LOG(RkISP2, Debug) << "Configuring ISP input with: " << format; + ret = isp_->setFormat(0, &format); + if (ret) + return ret; + + Rectangle ispInCrop(0, 0, format.size); + LOG(RkISP2, Debug) << "Configuring ISP sink crop with: " << ispInCrop; + ret = isp_->setSelection(0, V4L2_SEL_TGT_CROP, &ispInCrop); + if (ret) + return ret; + + format.code = formatToMediaBus.at(cfg.pixelFormat); + format.size = cfg.size; + LOG(RkISP2, Debug) << "Configuring ISP output with: " << format; + ret = isp_->setFormat(1, &format); + if (ret) + return ret; + + Rectangle ispOutCrop(0, 0, cfg.size); + LOG(RkISP2, Debug) << "Configuring ISP source crop with: " << ispOutCrop; + ret = isp_->setSelection(2, V4L2_SEL_TGT_CROP, &ispOutCrop); + if (ret) + return ret; + + V4L2DeviceFormat outputFormat; + outputFormat.fourcc = mainPath_->toV4L2PixelFormat(cfg.pixelFormat); + outputFormat.size = cfg.size; + + LOG(RkISP2, Debug) << "Configuring main path with: " << outputFormat; + ret = mainPath_->setFormat(&outputFormat); + if (ret) + return ret; + + if (outputFormat.size != cfg.size || + outputFormat.fourcc != data->mainPath_->toV4L2PixelFormat(cfg.pixelFormat)) { + LOG(RkISP2, Error) + << "Unable to configure capture in " << cfg.toString(); + return -EINVAL; + } + + cfg.setStream(&data->stream_); + cfg.stride = outputFormat.planes[0].bpl; + + V4L2DeviceFormat paramFormat; + paramFormat.fourcc = V4L2PixelFormat(V4L2_META_FMT_RKISP2_PARAMS); + ret = param_->setFormat(¶mFormat); + if (ret) + return ret; + + V4L2DeviceFormat statFormat; + statFormat.fourcc = V4L2PixelFormat(V4L2_META_FMT_RKISP2_STATS); + ret = stat_->setFormat(&statFormat); + if (ret) + return ret; + + IPACameraSensorInfo sensorInfo; + ret = data->sensor_->sensorInfo(&sensorInfo); + if (ret) + return ret; + + int colorSpaceEncoding = -1; + int colorSpaceRange = -1; + if (cfg.colorSpace) { + colorSpaceEncoding = static_cast(cfg.colorSpace->ycbcrEncoding); + colorSpaceRange = static_cast(cfg.colorSpace->range); + } + + /* Inform IPA of stream configuration and sensor controls. */ + ipa::rkisp2::IPAConfigInfo ipaConfig{ sensorInfo, + data->sensor_->controls(), + colorSpaceEncoding, + colorSpaceRange }; + + ret = data->ipa_->configure(ipaConfig, &data->ipaControls_); + if (ret) { + LOG(RkISP2, Error) << "failed configuring IPA (" << ret << ")"; + return ret; + } + + return updateControls(data); +} + +int PipelineHandlerRkISP2::exportFrameBuffers(Camera *camera, Stream *stream, + std::vector> *buffers) +{ + unsigned int count = stream->configuration().bufferCount; + RkISP2CameraData *data = cameraData(camera); + + if (!data->usingIsp_ || data->isRaw_) + return data->video_->exportBuffers(count, buffers); + + return data->mainPath_->exportBuffers(count, buffers); +} + +int PipelineHandlerRkISP2::start(Camera *camera, + const ControlList *controls) +{ + RkISP2CameraData *data = cameraData(camera); + unsigned int count = data->stream_.configuration().bufferCount; + bool useMP = data->usingIsp_ && !data->isRaw_; + utils::ScopeExitActions actions; + int ret; + + data->frame_ = 0; + + LOG(RkISP2, Debug) << (useMP ? "Using" : "Not using") << " main path"; + + if (useMP) { + /* Allocate buffers for params and stats */ + ret = allocateBuffers(camera); + if (ret) { + LOG(RkISP2, Error) << "Failed to allocate buffers"; + return ret; + } + actions += [&]() { freeBuffers(camera); }; + + /* \todo Support start controls */ + ret = data->ipa_->start(); + if (ret) { + LOG(RkISP2, Error) + << "Failed to start IPA " << camera->id(); + return ret; + } + actions += [&]() { data->ipa_->stop(); }; + + ret = data->param_->streamOn(); + if (ret) { + LOG(RkISP2, Error) + << "Failed to start parameters " << camera->id(); + return ret; + } + actions += [&]() { data->param_->streamOff(); }; + } + + ret = data->video_->importBuffers(kRkISP2MinBufferCount); + if (ret < 0) { + LOG(RkISP2, Error) << "Failed to import buffers to vicap"; + return ret; + } + + actions += [&]() { data->video_->releaseBuffers(); }; + + if (useMP) { + ret = data->rawrd_->importBuffers(count); + if (ret < 0) { + LOG(RkISP2, Error) << "Failed to import buffers to rawrd"; + return ret; + } + + actions += [&]() { data->rawrd_->releaseBuffers(); }; + + ret = data->mainPath_->importBuffers(count); + if (ret < 0) { + LOG(RkISP2, Error) << "Failed to import buffers to main path"; + return ret; + } + + actions += [&]() { data->mainPath_->releaseBuffers(); }; + + auto queueBuffers = [&](const std::vector> &buffers, + V4L2VideoDevice *device, std::string_view name) { + for (const std::unique_ptr &buffer : buffers) { + ret = device->queueBuffer(buffer.get()); + if (ret < 0) { + LOG(RkISP2, Warning) + << "Failed to queue buffer " + << &buffer << " to " << name + << ": " << ret; + } + } + }; + + queueBuffers(data->internalBuffers_, data->video_, "vicap"); + queueBuffers(data->statBuffers_, data->stat_, "stat"); + } + + ret = data->video_->streamOn(); + if (ret < 0) + return ret; + + actions += [&]() { data->video_->streamOff(); }; + + if (useMP) { + ret = data->rawrd_->streamOn(); + if (ret < 0) + return ret; + + actions += [&]() { data->rawrd_->streamOff(); }; + + ret = data->stat_->streamOn(); + if (ret) { + LOG(RkISP2, Error) + << "Failed to start stats " << camera->id(); + return ret; + } + actions += [&]() { data->stat_->streamOff(); }; + + ret = data->mainPath_->streamOn(); + if (ret < 0) + return ret; + + actions += [&]() { data->mainPath_->streamOff(); }; + } + + if (controls) { + ret = processControls(data, *controls); + if (ret < 0) + return ret; + } + + if (useMP) + data->isp_->setFrameStartEnabled(true); + + actions.release(); + return 0; +} + +void PipelineHandlerRkISP2::stopDevice(Camera *camera) +{ + RkISP2CameraData *data = cameraData(camera); + bool useMP = data->usingIsp_ && !data->isRaw_; + + if (useMP) + data->isp_->setFrameStartEnabled(false); + + data->video_->streamOff(); + data->video_->releaseBuffers(); + + if (!useMP) + return; + + data->ipa_->stop(); + + data->rawrd_->streamOff(); + data->rawrd_->releaseBuffers(); + + data->mainPath_->streamOff(); + data->mainPath_->releaseBuffers(); + + data->stat_->streamOff(); + data->stat_->releaseBuffers(); + + data->param_->streamOff(); + + data->internalBuffers_.clear(); + data->statBuffers_.clear(); + freeBuffers(camera); +} + +int PipelineHandlerRkISP2::queueRequestDevice(Camera *camera, Request *request) +{ + RkISP2CameraData *data = cameraData(camera); + FrameBuffer *buffer = request->findBuffer(&data->stream_); + if (!buffer) { + LOG(RkISP2, Error) + << "Attempt to queue request with invalid stream"; + return -ENOENT; + } + + int ret = processControls(data, request->controls()); + if (ret < 0) + return ret; + + if (!data->usingIsp_ || data->isRaw_) + return data->video_->queueBuffer(buffer); + + int correction = data->syncHelper_.correction(); + data->frame_ += correction; + data->syncHelper_.pushCorrection(correction); + + data->pendingCompleteRequests_.push_back({ request, data->frame_, true }); + + data->ipa_->queueRequest(data->frame_, request->controls()); + data->computeParamBuffers(data->frame_); + data->frame_++; + + return data->mainPath_->queueBuffer(buffer); +} + +int PipelineHandlerRkISP2::updateControls(RkISP2CameraData *data) +{ + ControlInfoMap::Map controls; + + /* Add the pipeline handler registered controls to list of camera controls. */ + for (const auto &phControl : data->controlInfo_) + controls[phControl.first] = phControl.second; + + /* Add the IPA registered controls to list of camera controls. */ + for (const auto &ipaControl : data->ipaControls_) + controls[ipaControl.first] = ipaControl.second; + + data->controlInfo_ = ControlInfoMap(std::move(controls), + controls::controls); + + return 0; +} + +bool PipelineHandlerRkISP2::createCamera(bool usingIsp) +{ + std::unique_ptr data = + std::make_unique(this, media_.get()); + if (data->init(usingIsp, sensor_.get(), video_.get(), rawrd_.get(), + isp_.get(), mainPath_.get(), param_.get(), stat_.get())) { + LOG(RkISP2, Error) << "Failed to initialize data"; + return false; + } + + /* Initialize the camera properties. */ + data->properties_ = data->sensor_->properties(); + + const CameraSensorProperties::SensorDelays &delays = data->sensor_->sensorDelays(); + std::unordered_map params = { + { V4L2_CID_ANALOGUE_GAIN, { delays.gainDelay, false } }, + { V4L2_CID_EXPOSURE, { delays.exposureDelay, false } }, + { V4L2_CID_VBLANK, { delays.vblankDelay, true } }, + }; + + if (usingIsp) { + data->delayedCtrls_ = + std::make_unique(data->sensor_->device(), + params); + isp_->frameStart.connect(data->delayedCtrls_.get(), + &DelayedControls::applyControls); + + int ret = data->loadIPA(); + if (ret) { + LOG(RkISP2, Error) << "Failed to load IPA"; + return false; + } + + updateControls(data.get()); + + data->paramQueue_ = + std::make_unique(std::make_unique>(param_.get()), + BufferQueue::PrepareStage, "Params"); + } + + const std::string &id = data->sensor_->id(); + std::set streams{ &data->stream_ }; + std::shared_ptr camera = + Camera::create(std::move(data), id, streams); + registerCamera(std::move(camera)); + + LOG(RkISP2, Debug) + << "RkISP2 device registered " + << (usingIsp ? "with" : "without") << " ISP"; + + return true; +} + +bool PipelineHandlerRkISP2::match(DeviceEnumerator *enumerator) +{ + DeviceMatch dm("rockchip-cif"); + /* \todo Generalize this for the other csi ports */ + /* + * I think we will have one camera per csi port, and then 4 streams + * each that correspond to the 4 channels. Not sure how to handle + * routing to the ISP though. For now we'll just assume one camera. + */ + dm.add("rkcif-mipi2"); + /* \todo Generalize this for the other channels */ + dm.add("rkcif-mipi2-id0"); + dm.add("dw-mipi-csi2rx fdd30000.csi"); + + media_ = acquireMediaDevice(enumerator, dm); + if (!media_) + return false; + + csi_ = V4L2Subdevice::fromEntityName(media_.get(), "dw-mipi-csi2rx fdd30000.csi"); + if (!csi_ || csi_->open() < 0) { + LOG(RkISP2, Error) << "Failed to open csi"; + return false; + } + + cif_ = V4L2Subdevice::fromEntityName(media_.get(), "rkcif-mipi2"); + if (!cif_ || cif_->open() < 0) { + LOG(RkISP2, Error) << "Failed to open cif"; + return false; + } + + /* \todo Support multiple streams */ + video_ = V4L2VideoDevice::fromEntityName(media_.get(), "rkcif-mipi2-id0"); + if (!video_ || video_->open() < 0) { + LOG(RkISP2, Error) << "Failed to open capture device"; + return false; + } + + for (MediaEntity *entity : media_->locateEntities(MEDIA_ENT_F_CAM_SENSOR)) { + LOG(RkISP2, Debug) << "Identified " << entity->name(); + sensor_ = CameraSensorFactoryBase::create(entity); + /* Just get the first sensor for now */ + if (sensor_) + break; + } + + if (!sensor_) { + LOG(RkISP2, Error) << "Failed to find sensor"; + return false; + } + + const GlobalConfiguration &configuration = cameraManager()->_d()->configuration(); + bool usingIsp = configuration.configuration()["pipelines"]["rkisp2"]["isp_enable"].get(true); + if (!usingIsp) { + LOG(RkISP2, Info) << "ISP disabled in configuration file"; + return createCamera(usingIsp); + } + + /* Match ISP */ + + DeviceMatch dmIsp("rkisp2"); + dmIsp.add("rkisp2_isp"); + /* \todo Generalize this for the other channels */ + dmIsp.add("rkisp2_rawrd0"); + /* \todo Support self path */ + dmIsp.add("rkisp2_mainpath"); + + ispMedia_ = acquireMediaDevice(enumerator, dmIsp); + if (!ispMedia_) { + usingIsp = false; + LOG(RkISP2, Debug) << "ISP not found"; + return createCamera(usingIsp); + } + + /* \todo Support the other rawrd nodes */ + rawrd_ = V4L2VideoDevice::fromEntityName(ispMedia_.get(), "rkisp2_rawrd0"); + if (!rawrd_ || rawrd_->open() < 0) { + LOG(RkISP2, Error) << "Failed to open rkisp2 rawrd device"; + return false; + } + + isp_ = V4L2Subdevice::fromEntityName(ispMedia_.get(), "rkisp2_isp"); + if (!isp_ || isp_->open() < 0) { + LOG(RkISP2, Error) << "Failed to open rkisp2 isp"; + return false; + } + + /* \todo Support self path */ + mainPath_ = V4L2VideoDevice::fromEntityName(ispMedia_.get(), "rkisp2_mainpath"); + if (!mainPath_ || mainPath_->open() < 0) { + LOG(RkISP2, Error) << "Failed to open rkisp2 main path"; + return false; + } + + param_ = V4L2VideoDevice::fromEntityName(ispMedia_.get(), "rkisp2_params"); + if (!param_ || param_->open() < 0) { + LOG(RkISP2, Error) << "Failed to open rkisp2 params"; + return false; + } + + stat_ = V4L2VideoDevice::fromEntityName(ispMedia_.get(), "rkisp2_stats"); + if (!stat_ || stat_->open() < 0) { + LOG(RkISP2, Error) << "Failed to open rkisp2 stats"; + return false; + } + + return createCamera(true); +} + +int PipelineHandlerRkISP2::processControls(RkISP2CameraData *data, const ControlList &ctrls) +{ + const auto &testPattern = ctrls.get(controls::draft::TestPatternMode); + if (testPattern) + data->sensor_->setTestPatternMode(static_cast(*testPattern)); + + return 0; +} + +/* This is only called when using the ISP */ +int PipelineHandlerRkISP2::allocateBuffers(Camera *camera) +{ + RkISP2CameraData *data = cameraData(camera); + unsigned int ipaBufferId = 1; + utils::ScopeExitActions actions; + + int ret = data->video_->exportBuffers(kRkISP2MinBufferCount, &data->internalBuffers_); + if (ret < 0) + return ret; + + actions += [&]() { data->video_->releaseBuffers(); }; + + ret = data->stat_->allocateBuffers(kRkISP2MinBufferCount, &data->statBuffers_); + if (ret < 0) + return ret; + + actions += [&]() { data->stat_->releaseBuffers(); }; + + ret = data->paramQueue_->allocateBuffers(kRkISP2MinBufferCount); + if (ret < 0) + return ret; + + actions += [&]() { data->paramQueue_->releaseBuffers(); }; + + auto pushBuffers = [&](const std::vector> &buffers) { + for (const std::unique_ptr &buffer : buffers) { + Span planes = buffer->planes(); + + buffer->setCookie(ipaBufferId++); + data->ipaBuffers_.emplace_back(buffer->cookie(), + std::vector{ planes.begin(), + planes.end() }); + } + }; + + pushBuffers(data->paramQueue_->buffers()); + pushBuffers(data->statBuffers_); + + data->ipa_->mapBuffers(data->ipaBuffers_); + + actions.release(); + return 0; +} + +int PipelineHandlerRkISP2::freeBuffers(Camera *camera) +{ + RkISP2CameraData *data = cameraData(camera); + + std::vector ids; + for (IPABuffer &ipabuf : data->ipaBuffers_) + ids.push_back(ipabuf.id); + + data->ipa_->unmapBuffers(ids); + data->ipaBuffers_.clear(); + + data->paramQueue_->releaseBuffers(); + + return 0; +} + +REGISTER_PIPELINE_HANDLER(PipelineHandlerRkISP2, "rkisp2") + +} /* namespace libcamera */ From patchwork Fri Jul 3 12:25:13 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Paul Elder X-Patchwork-Id: 27166 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id C884BC328C for ; Fri, 3 Jul 2026 12:26:26 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 83E2965FD9; Fri, 3 Jul 2026 14:26:26 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="GirQS2Hz"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 4440C65FCA for ; Fri, 3 Jul 2026 14:26:25 +0200 (CEST) Received: from neptunite.hamster-moth.ts.net (unknown [IPv6:2404:7a81:160:2100:a2cc:2f45:3bd7:2589]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 636208E0; Fri, 3 Jul 2026 14:25:36 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1783081539; bh=0ERL9+WoD24amxrnY2dQ9id8zeeBTQq6BFut/3mpeKA=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=GirQS2HzD7jv2uBdsioD4VXb7Tgb+9A25oEi07ZJM2oAVS0fVC3ytH9/mBJZtjBtR yiWiA4kY4NoDnLSxCbu8ZdPbttTG+6d+VNFhV0hiG/Vo9M7ZLrNKfoHyA9YAdgCYSR ACtpj3IkpwAtHLHwWuHq9uA/m34NZbTAh9JV35OQ= From: Paul Elder To: laurent.pinchart@ideasonboard.com Cc: Paul Elder , michael.riesch@collabora.com, xuhf@rock-chips.com, stefan.klug@ideasonboard.com, kieran.bingham@ideasonboard.com, dan.scally@ideasonboard.com, jacopo.mondi@ideasonboard.com, nicolas.dufresne@collabora.com, libcamera-devel@lists.libcamera.org Subject: [RFC PATCH 07/19] pipeline: rkisp2: Support shared media graph Date: Fri, 3 Jul 2026 21:25:13 +0900 Message-ID: <20260703122543.1991189-8-paul.elder@ideasonboard.com> X-Mailer: git-send-email 2.47.2 In-Reply-To: <20260703122543.1991189-1-paul.elder@ideasonboard.com> References: <20260703122543.1991189-1-paul.elder@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Make the rkisp2 pipeline handler use the shared media graph instead of separate graphs. This makes the rkisp2 pipeline handler incompatible with kernels that do not have shared media graph. Shared media graph puts the VICAP and the ISP in the same media graph, so that it can be switched between inline mode and memory-to-memory mode. The rkisp2 pipeline handler currently does not support inline mode however, so it will only function in memory-to-memory-mode. Signed-off-by: Paul Elder --- I put this in a separate patch so that it's easily revertible. Shared media graph patches for the kernel are here: https://lore.kernel.org/all/20260619052637.1110672-5-paul.elder@ideasonboard.com/ --- src/libcamera/pipeline/rkisp2/rkisp2.cpp | 67 ++++++++++++++++-------- 1 file changed, 46 insertions(+), 21 deletions(-) diff --git a/src/libcamera/pipeline/rkisp2/rkisp2.cpp b/src/libcamera/pipeline/rkisp2/rkisp2.cpp index 0d335a980e32..a89672075f40 100644 --- a/src/libcamera/pipeline/rkisp2/rkisp2.cpp +++ b/src/libcamera/pipeline/rkisp2/rkisp2.cpp @@ -255,7 +255,6 @@ private: std::unique_ptr cif_; std::unique_ptr video_; - std::shared_ptr ispMedia_; std::unique_ptr rawrd_; std::unique_ptr isp_; std::unique_ptr mainPath_; @@ -717,6 +716,36 @@ int PipelineHandlerRkISP2::configure(Camera *camera, const PixelFormatInfo &info = PixelFormatInfo::info(cfg.pixelFormat); data->isRaw_ = info.colourEncoding == PixelFormatInfo::ColourEncodingRAW; + /* + * No need to check usingIsp_ || isRaw_, since if we're capturing from + * just VICAP we still don't want the link to the ISP, and if we're + * using the ISP then we don't support inline mode yet. + * \todo Support inline mode + */ + LOG(RkISP2, Debug) << "Disabling link from VICAP to ISP"; + MediaLink *link = media_->link(cif_->entity(), 2, isp_->entity(), 4); + ret = link->setEnabled(false); + if (ret < 0) { + LOG(RkISP2, Error) << "Failed to disable link between VICAP and ISP"; + return ret; + } + + LOG(RkISP2, Debug) << "Enabling link from VICAP to capture node"; + link = media_->link("rkcif-mipi2", 1, "rkcif-mipi2-id0", 0); + ret = link->setEnabled(true); + if (ret < 0) { + LOG(RkISP2, Error) << "Failed to enable link between VICAP and capture node"; + return ret; + } + + LOG(RkISP2, Debug) << "Enabling link from rawrd to ISP"; + link = media_->link("rkisp2_rawrd0", 0, "rkisp2_isp", 0); + ret = link->setEnabled(true); + if (ret < 0) { + LOG(RkISP2, Error) << "Failed to enable link between rawrd and ISP"; + return ret; + } + V4L2SubdeviceFormat format = config->sensorFormat(); LOG(RkISP2, Debug) << "Configuring sensor with " << format; @@ -1124,6 +1153,10 @@ bool PipelineHandlerRkISP2::createCamera(bool usingIsp) bool PipelineHandlerRkISP2::match(DeviceEnumerator *enumerator) { + /* + * \todo This needs to be reconciled with how shared media graphs are + * named + */ DeviceMatch dm("rockchip-cif"); /* \todo Generalize this for the other csi ports */ /* @@ -1136,6 +1169,12 @@ bool PipelineHandlerRkISP2::match(DeviceEnumerator *enumerator) dm.add("rkcif-mipi2-id0"); dm.add("dw-mipi-csi2rx fdd30000.csi"); + dm.add("rkisp2_isp"); + /* \todo Generalize this for the other channels */ + dm.add("rkisp2_rawrd0"); + /* \todo Support self path */ + dm.add("rkisp2_mainpath"); + media_ = acquireMediaDevice(enumerator, dm); if (!media_) return false; @@ -1179,49 +1218,35 @@ bool PipelineHandlerRkISP2::match(DeviceEnumerator *enumerator) return createCamera(usingIsp); } - /* Match ISP */ - - DeviceMatch dmIsp("rkisp2"); - dmIsp.add("rkisp2_isp"); - /* \todo Generalize this for the other channels */ - dmIsp.add("rkisp2_rawrd0"); - /* \todo Support self path */ - dmIsp.add("rkisp2_mainpath"); - - ispMedia_ = acquireMediaDevice(enumerator, dmIsp); - if (!ispMedia_) { - usingIsp = false; - LOG(RkISP2, Debug) << "ISP not found"; - return createCamera(usingIsp); - } + /* Acquire ISP */ /* \todo Support the other rawrd nodes */ - rawrd_ = V4L2VideoDevice::fromEntityName(ispMedia_.get(), "rkisp2_rawrd0"); + rawrd_ = V4L2VideoDevice::fromEntityName(media_.get(), "rkisp2_rawrd0"); if (!rawrd_ || rawrd_->open() < 0) { LOG(RkISP2, Error) << "Failed to open rkisp2 rawrd device"; return false; } - isp_ = V4L2Subdevice::fromEntityName(ispMedia_.get(), "rkisp2_isp"); + isp_ = V4L2Subdevice::fromEntityName(media_.get(), "rkisp2_isp"); if (!isp_ || isp_->open() < 0) { LOG(RkISP2, Error) << "Failed to open rkisp2 isp"; return false; } /* \todo Support self path */ - mainPath_ = V4L2VideoDevice::fromEntityName(ispMedia_.get(), "rkisp2_mainpath"); + mainPath_ = V4L2VideoDevice::fromEntityName(media_.get(), "rkisp2_mainpath"); if (!mainPath_ || mainPath_->open() < 0) { LOG(RkISP2, Error) << "Failed to open rkisp2 main path"; return false; } - param_ = V4L2VideoDevice::fromEntityName(ispMedia_.get(), "rkisp2_params"); + param_ = V4L2VideoDevice::fromEntityName(media_.get(), "rkisp2_params"); if (!param_ || param_->open() < 0) { LOG(RkISP2, Error) << "Failed to open rkisp2 params"; return false; } - stat_ = V4L2VideoDevice::fromEntityName(ispMedia_.get(), "rkisp2_stats"); + stat_ = V4L2VideoDevice::fromEntityName(media_.get(), "rkisp2_stats"); if (!stat_ || stat_->open() < 0) { LOG(RkISP2, Error) << "Failed to open rkisp2 stats"; return false; From patchwork Fri Jul 3 12:25:14 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Paul Elder X-Patchwork-Id: 27167 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 6CB45C328C for ; Fri, 3 Jul 2026 12:26:31 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 1706F65FD9; Fri, 3 Jul 2026 14:26:31 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="fUVSbv7M"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id C4D9765FCA for ; Fri, 3 Jul 2026 14:26:28 +0200 (CEST) Received: from neptunite.hamster-moth.ts.net (unknown [IPv6:2404:7a81:160:2100:a2cc:2f45:3bd7:2589]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id E45811121; Fri, 3 Jul 2026 14:25:39 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1783081542; bh=b9X1JcNno86vhftBspY6DOelKj8JrllY5rJUTzW1Ne0=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=fUVSbv7Mipyu65I7+rrDc0qSmJQbsUBVpMA5a0BhNfN00UsJruiJ7b0P2sEwSrpEh p4w0S97SQgMKeMrHBmB6BHi2DHiR47Bf3sGHc26SfnfgwfGVamK8J6MmbskBsZe4iK jif/zsrH3eQ9MXOG9TtYGkkXTu88a8DGiAQGcTM8= From: Paul Elder To: laurent.pinchart@ideasonboard.com Cc: Paul Elder , michael.riesch@collabora.com, xuhf@rock-chips.com, stefan.klug@ideasonboard.com, kieran.bingham@ideasonboard.com, dan.scally@ideasonboard.com, jacopo.mondi@ideasonboard.com, nicolas.dufresne@collabora.com, libcamera-devel@lists.libcamera.org Subject: [RFC PATCH 08/19] ipa: rkisp2: algo: bls: Implement black level subtraction Date: Fri, 3 Jul 2026 21:25:14 +0900 Message-ID: <20260703122543.1991189-9-paul.elder@ideasonboard.com> X-Mailer: git-send-email 2.47.2 In-Reply-To: <20260703122543.1991189-1-paul.elder@ideasonboard.com> References: <20260703122543.1991189-1-paul.elder@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Implement a static black level subtraction algorithm for the rkisp2 IPA. Signed-off-by: Paul Elder --- src/ipa/rkisp2/algorithms/bls.cpp | 154 ++++++++++++++++++++++++++ src/ipa/rkisp2/algorithms/bls.h | 41 +++++++ src/ipa/rkisp2/algorithms/meson.build | 1 + 3 files changed, 196 insertions(+) create mode 100644 src/ipa/rkisp2/algorithms/bls.cpp create mode 100644 src/ipa/rkisp2/algorithms/bls.h diff --git a/src/ipa/rkisp2/algorithms/bls.cpp b/src/ipa/rkisp2/algorithms/bls.cpp new file mode 100644 index 000000000000..aba2b7a867be --- /dev/null +++ b/src/ipa/rkisp2/algorithms/bls.cpp @@ -0,0 +1,154 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2026, Ideas On Board + * + * RkISP2 Black Level Subtraction control + */ + +#include "bls.h" + +#include + +#include + +#include + +/** + * \file bls.h + */ + +namespace libcamera { + +namespace ipa::rkisp2::algorithms { + +/** + * \class BlackLevelSubtraction + * \brief RkISP2 Black Level Subtraction control + * + * The pixels output by the camera normally include a black level, because + * sensors do not always report a signal level of '0' for black. Pixels at or + * below this level should be considered black. To achieve that, the RkISP2 BLS + * algorithm subtracts a configurable offset from all pixels. + * + * The black level can be measured at runtime from an optical dark region of the + * camera sensor, or measured during the camera tuning process. The first option + * isn't currently supported. + * + * \todo Add support for black level in the metadata so that we can capture + * proper raw images for tuning + */ + +LOG_DEFINE_CATEGORY(RkISP2Bls) + +/** + * \copydoc libcamera::ipa::Algorithm::init + */ +int BlackLevelSubtraction::init(IPAContext &context, const ValueNode &tuningData) +{ + std::optional levelRed = tuningData["R"].get(); + std::optional levelGreenR = tuningData["Gr"].get(); + std::optional levelGreenB = tuningData["Gb"].get(); + std::optional levelBlue = tuningData["B"].get(); + bool tuningHasLevels = levelRed && levelGreenR && levelGreenB && levelBlue; + + auto blackLevel = context.camHelper->blackLevel(); + if (!blackLevel) { + /* + * Not all camera sensor helpers have been updated with black + * levels. Print a warning and fall back to the levels from the + * tuning data to preserve backward compatibility. This should + * be removed once all helpers provide the data. + */ + LOG(RkISP2Bls, Warning) + << "No black levels provided by camera sensor helper" + << ", please fix"; + + blackLevelRed_ = levelRed.value_or(4096); + blackLevelGreenR_ = levelGreenR.value_or(4096); + blackLevelGreenB_ = levelGreenB.value_or(4096); + blackLevelBlue_ = levelBlue.value_or(4096); + } else if (tuningHasLevels) { + /* + * If black levels are provided in the tuning file, use them to + * avoid breaking existing camera tuning. This is deprecated and + * will be removed. + */ + LOG(RkISP2Bls, Warning) + << "Deprecated: black levels overwritten by tuning file"; + + blackLevelRed_ = *levelRed; + blackLevelGreenR_ = *levelGreenR; + blackLevelGreenB_ = *levelGreenB; + blackLevelBlue_ = *levelBlue; + } else { + blackLevelRed_ = *blackLevel; + blackLevelGreenR_ = *blackLevel; + blackLevelGreenB_ = *blackLevel; + blackLevelBlue_ = *blackLevel; + } + + LOG(RkISP2Bls, Debug) + << "Black levels: red " << blackLevelRed_ + << ", green (red) " << blackLevelGreenR_ + << ", green (blue) " << blackLevelGreenB_ + << ", blue " << blackLevelBlue_; + + return 0; +} + +int BlackLevelSubtraction::configure([[maybe_unused]] IPAContext &context, + [[maybe_unused]] const IPACameraSensorInfo &configInfo) +{ + /* \todo Save the cfa */ + + return 0; +} + +/** + * \copydoc libcamera::ipa::Algorithm::prepare + */ +void BlackLevelSubtraction::prepare(IPAContext &context, + const uint32_t frame, + [[maybe_unused]] IPAFrameContext &frameContext, + RkISP2Params *params) +{ + if (context.configuration.raw) + return; + + if (frame > 1) + return; + + auto config = params->block(); + config.setEnabled(true); + + config->enable_auto = 0; + + /* Scale down to the 12-bit black levels used by the BLS block. */ + /* \todo Handle cfa properly */ + config->bls_fixed_val.a = blackLevelRed_ >> 4; + config->bls_fixed_val.b = blackLevelGreenR_ >> 4; + config->bls_fixed_val.c = blackLevelGreenB_ >> 4; + config->bls_fixed_val.d = blackLevelBlue_ >> 4; +} + +/** + * \copydoc libcamera::ipa::Algorithm::process + */ +void BlackLevelSubtraction::process([[maybe_unused]] IPAContext &context, + [[maybe_unused]] const uint32_t frame, + [[maybe_unused]] IPAFrameContext &frameContext, + [[maybe_unused]] const rkisp2_stats_buffer *stats, + ControlList &metadata) +{ + metadata.set(controls::SensorBlackLevels, + { static_cast(blackLevelRed_), + static_cast(blackLevelGreenR_), + static_cast(blackLevelGreenB_), + static_cast(blackLevelBlue_) }); +} + +REGISTER_IPA_ALGORITHM(BlackLevelSubtraction, "BlackLevelSubtraction") + +} /* namespace ipa::rkisp2::algorithms */ + +} /* namespace libcamera */ diff --git a/src/ipa/rkisp2/algorithms/bls.h b/src/ipa/rkisp2/algorithms/bls.h new file mode 100644 index 000000000000..5377ae76a029 --- /dev/null +++ b/src/ipa/rkisp2/algorithms/bls.h @@ -0,0 +1,41 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2026, Ideas On Board + * + * RkISP2 Black Level Subtraction control + */ + +#pragma once + +#include "algorithm.h" + +namespace libcamera { + +namespace ipa::rkisp2::algorithms { + +class BlackLevelSubtraction : public Algorithm +{ +public: + BlackLevelSubtraction() = default; + ~BlackLevelSubtraction() = default; + + int init(IPAContext &context, const ValueNode &tuningData) override; + int configure(IPAContext &context, + const IPACameraSensorInfo &configInfo) override; + void prepare(IPAContext &context, const uint32_t frame, + IPAFrameContext &frameContext, + RkISP2Params *params) override; + void process(IPAContext &context, const uint32_t frame, + IPAFrameContext &frameContext, + const rkisp2_stats_buffer *stats, + ControlList &metadata) override; + +private: + int16_t blackLevelRed_; + int16_t blackLevelGreenR_; + int16_t blackLevelGreenB_; + int16_t blackLevelBlue_; +}; + +} /* namespace ipa::rkisp2::algorithms */ +} /* namespace libcamera */ diff --git a/src/ipa/rkisp2/algorithms/meson.build b/src/ipa/rkisp2/algorithms/meson.build index 4bb81e48eb01..36996918b39e 100644 --- a/src/ipa/rkisp2/algorithms/meson.build +++ b/src/ipa/rkisp2/algorithms/meson.build @@ -1,5 +1,6 @@ # SPDX-License-Identifier: CC0-1.0 rkisp2_ipa_algorithms = files([ + 'bls.cpp', ]) From patchwork Fri Jul 3 12:25:15 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Paul Elder X-Patchwork-Id: 27168 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 090AAC328C for ; Fri, 3 Jul 2026 12:26:34 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id A5F4165FD8; Fri, 3 Jul 2026 14:26:33 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="rMV/2muO"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 53E2965FD8 for ; Fri, 3 Jul 2026 14:26:32 +0200 (CEST) Received: from neptunite.hamster-moth.ts.net (unknown [IPv6:2404:7a81:160:2100:a2cc:2f45:3bd7:2589]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 739198E0; Fri, 3 Jul 2026 14:25:43 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1783081546; bh=CzSviiRCwi9H5hJ2YKvMHc9OL7RDud3wqlrh3cartjY=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=rMV/2muOy6BeQmPaWFkT487SQkngQpBUoF55BQVEdINGtmJtSRbagljD91LC/M4P2 Gg6GoT8mCslGi+DWAaR4TEE8FFjdfgAC7pfRsJq4Bs4QVMwdTgAndSsz/gMeKrjTch ZA/XUm3fjFi/W8uUX6jF0TttijM+ivGIKoR/59dU= From: Paul Elder To: laurent.pinchart@ideasonboard.com Cc: Paul Elder , michael.riesch@collabora.com, xuhf@rock-chips.com, stefan.klug@ideasonboard.com, kieran.bingham@ideasonboard.com, dan.scally@ideasonboard.com, jacopo.mondi@ideasonboard.com, nicolas.dufresne@collabora.com, libcamera-devel@lists.libcamera.org Subject: [RFC PATCH 09/19] ipa: rkisp2: algo: awb: Implement automatic white balance control Date: Fri, 3 Jul 2026 21:25:15 +0900 Message-ID: <20260703122543.1991189-10-paul.elder@ideasonboard.com> X-Mailer: git-send-email 2.47.2 In-Reply-To: <20260703122543.1991189-1-paul.elder@ideasonboard.com> References: <20260703122543.1991189-1-paul.elder@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Implement auto white balance control algorithm for the rkisp2 IPA. This leverages the libipa awb, so it supports baysian awb, grey world awb, as well as manual white balance. Signed-off-by: Paul Elder --- src/ipa/rkisp2/algorithms/awb.cpp | 174 ++++++++++++++++++++++++++ src/ipa/rkisp2/algorithms/awb.h | 56 +++++++++ src/ipa/rkisp2/algorithms/meson.build | 1 + 3 files changed, 231 insertions(+) create mode 100644 src/ipa/rkisp2/algorithms/awb.cpp create mode 100644 src/ipa/rkisp2/algorithms/awb.h diff --git a/src/ipa/rkisp2/algorithms/awb.cpp b/src/ipa/rkisp2/algorithms/awb.cpp new file mode 100644 index 000000000000..4928ab13d220 --- /dev/null +++ b/src/ipa/rkisp2/algorithms/awb.cpp @@ -0,0 +1,174 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2026, Ideas On Board + * + * AWB control algorithm + */ + +#include "awb.h" + +#include + +#include + +#include + +#include "libcamera/internal/vector.h" + +/** + * \file awb.h + */ + +namespace libcamera { + +namespace ipa::rkisp2::algorithms { + +LOG_DEFINE_CATEGORY(RkISP2Awb) + +class RkISP2AwbStats final : public AwbStats +{ +public: + RkISP2AwbStats() = default; + RkISP2AwbStats(const RGB means) + : AwbStats(means) + { + } + + /* Minimum mean value below which AWB can't operate. */ + double minColourValue() const override + { + return 2.0; + } +}; + +namespace { + +} /* namespace */ + +/** + * \class Awb + * \brief Manage the white balance with automatic and manual controls + */ + +Awb::Awb() +{ +} + +/** + * \copydoc libcamera::ipa::Algorithm::init + */ +int Awb::init(IPAContext &context, const ValueNode &tuningData) +{ + return awbAlgo_.init(tuningData, context.ctrlMap); +} + +int Awb::configure(IPAContext &context, + const IPACameraSensorInfo &configInfo) +{ + awbAlgo_.configure(context.activeState.awb); + + context.configuration.awb.measureWindow.h_offs = 0; + context.configuration.awb.measureWindow.v_offs = 0; + /* + * Unlike ae lite, this seems to still work when height == full window + * height + */ + context.configuration.awb.measureWindow.h_size = configInfo.outputSize.width / 15; + context.configuration.awb.measureWindow.v_size = configInfo.outputSize.height / 15; + + return 0; +} + +/** + * \copydoc libcamera::ipa::Algorithm::queueRequest + */ +void Awb::queueRequest(IPAContext &context, const uint32_t frame, + IPAFrameContext &frameContext, + const ControlList &controls) +{ + awbAlgo_.queueRequest(context.activeState.awb, frame, frameContext.awb, + controls); +} + +/** + * \copydoc libcamera::ipa::Algorithm::prepare + */ +void Awb::prepare([[maybe_unused]] IPAContext &context, + [[maybe_unused]] const uint32_t frame, + IPAFrameContext &frameContext, RkISP2Params *params) +{ + awbAlgo_.prepare(context.activeState.awb, frameContext.awb); + + auto gainConfig = params->block(); + gainConfig.setEnabled(true); + + RGB gains = frameContext.awb.gains; + + /* \todo Use quantized class */ + gainConfig->gains[0].gb = std::clamp(256 * 1.0, 0, 0x3fff); + gainConfig->gains[0].b = std::clamp(256 * gains.b(), 0, 0x3fff); + gainConfig->gains[0].r = std::clamp(256 * gains.r(), 0, 0x3fff); + gainConfig->gains[0].gr = std::clamp(256 * 1.0, 0, 0x3fff); + + auto measConfig = params->block(); + measConfig.setEnabled(true); + + measConfig->meas_window = context.configuration.awb.measureWindow; + struct rkisp2_isp_awb_color_quad minLimits = { 0, 0, 0, 0 }; + struct rkisp2_isp_awb_color_quad maxLimits = { 255, 255, 255, 255 }; + measConfig->limits[0] = minLimits; + measConfig->limits[1] = maxLimits; + + for (unsigned int i = 0; i < RKISP2_ISP_AWB_COUNTS_SIZE; i++) + measConfig->weights[i] = 0x20; +} + +/** + * \copydoc libcamera::ipa::Algorithm::process + */ +void Awb::process([[maybe_unused]] IPAContext &context, + [[maybe_unused]] const uint32_t frame, + IPAFrameContext &frameContext, + const rkisp2_stats_buffer *stats, + ControlList &metadata) +{ + RkISP2AwbStats awbStats = calculateRgbMeans(frameContext, stats); + + awbAlgo_.process(context.activeState.awb, frameContext.awb, awbStats, + frameContext.lux.lux, metadata); +} + +RkISP2AwbStats Awb::calculateRgbMeans([[maybe_unused]] const IPAFrameContext &frameContext, + const rkisp2_stats_buffer *stats) const +{ + if (!stats->awb.done) { + LOG(RkISP2Awb, Error) << "No awb stats"; + return {}; + } + + std::array counts; + RGB means; + + for (size_t i = 0; i < RKISP2_ISP_AWB_COUNTS_SIZE; i++) + counts[i] = static_cast(stats->awb.counts_r[i]); + means.r() = std::accumulate(counts.begin(), counts.end(), 0) / counts.size(); + + for (size_t i = 0; i < RKISP2_ISP_AWB_COUNTS_SIZE; i++) + counts[i] = static_cast(stats->awb.counts_g[i]); + means.g() = std::accumulate(counts.begin(), counts.end(), 0) / counts.size(); + + for (size_t i = 0; i < RKISP2_ISP_AWB_COUNTS_SIZE; i++) + counts[i] = static_cast(stats->awb.counts_b[i]); + means.b() = std::accumulate(counts.begin(), counts.end(), 0) / counts.size(); + + for (size_t i = 0; i < RKISP2_ISP_AWB_COUNTS_SIZE; i++) + counts[i] = static_cast(stats->awb.counts_w[i]); + + return RkISP2AwbStats(means); +} + +REGISTER_IPA_ALGORITHM(Awb, "Awb") + +} /* namespace ipa::rkisp2::algorithms */ + +} /* namespace libcamera */ diff --git a/src/ipa/rkisp2/algorithms/awb.h b/src/ipa/rkisp2/algorithms/awb.h new file mode 100644 index 000000000000..5f25ae1f1f05 --- /dev/null +++ b/src/ipa/rkisp2/algorithms/awb.h @@ -0,0 +1,56 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2026, Ideas On Board + * + * AWB control algorithm + */ + +#pragma once + +#include + +#include + +#include "libcamera/internal/value_node.h" + +#include "libipa/awb.h" +#include "libipa/fixedpoint.h" + +#include "algorithm.h" +#include "ipa_context.h" +#include "params.h" + +namespace libcamera { + +namespace ipa::rkisp2::algorithms { + +class RkISP2AwbStats; + +class Awb : public Algorithm +{ +public: + Awb(); + ~Awb() = default; + + int init(IPAContext &context, const ValueNode &tuningData) override; + int configure(IPAContext &context, const IPACameraSensorInfo &configInfo) override; + void queueRequest(IPAContext &context, const uint32_t frame, + IPAFrameContext &frameContext, + const ControlList &controls) override; + void prepare(IPAContext &context, const uint32_t frame, + IPAFrameContext &frameContext, + RkISP2Params *params) override; + void process(IPAContext &context, const uint32_t frame, + IPAFrameContext &frameContext, + const rkisp2_stats_buffer *stats, + ControlList &metadata) override; + +private: + RkISP2AwbStats calculateRgbMeans(const IPAFrameContext &frameContext, + const rkisp2_stats_buffer *stats) const; + + AwbAlgorithm> awbAlgo_; +}; + +} /* namespace ipa::rkisp2::algorithms */ +} /* namespace libcamera */ diff --git a/src/ipa/rkisp2/algorithms/meson.build b/src/ipa/rkisp2/algorithms/meson.build index 36996918b39e..30133ac4fa68 100644 --- a/src/ipa/rkisp2/algorithms/meson.build +++ b/src/ipa/rkisp2/algorithms/meson.build @@ -1,6 +1,7 @@ # SPDX-License-Identifier: CC0-1.0 rkisp2_ipa_algorithms = files([ + 'awb.cpp', 'bls.cpp', ]) From patchwork Fri Jul 3 12:25:16 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Paul Elder X-Patchwork-Id: 27169 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 923E2C328C for ; Fri, 3 Jul 2026 12:26:37 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 4275B65FE2; Fri, 3 Jul 2026 14:26:37 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="EA5hVS/R"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id D4ED865FDF for ; Fri, 3 Jul 2026 14:26:35 +0200 (CEST) Received: from neptunite.hamster-moth.ts.net (unknown [IPv6:2404:7a81:160:2100:a2cc:2f45:3bd7:2589]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 004E71121; Fri, 3 Jul 2026 14:25:46 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1783081549; bh=aXas7ZXnEXY6006S1EayfAlN8KPowCE2a94tRzIXrxw=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=EA5hVS/Re1nhZw07ojscZ4ZK0uegJhYX4T2tCH/bzIRGkZYdXrI8Ixnrnauq8e8e2 uFmBHUXZyl1HBwX5Sx2DhBOkBuIN2FUgL+VAqpOTTydz5gEZgs9iLDWr3/K/Hbc8Oq DlJUnVki9lS0g7ea3mc0v0lJunOgaIZq9/mKuCb4= From: Paul Elder To: laurent.pinchart@ideasonboard.com Cc: Paul Elder , michael.riesch@collabora.com, xuhf@rock-chips.com, stefan.klug@ideasonboard.com, kieran.bingham@ideasonboard.com, dan.scally@ideasonboard.com, jacopo.mondi@ideasonboard.com, nicolas.dufresne@collabora.com, libcamera-devel@lists.libcamera.org Subject: [RFC PATCH 10/19] ipa: rkisp2: algo: agc: Implement automatic gain control Date: Fri, 3 Jul 2026 21:25:16 +0900 Message-ID: <20260703122543.1991189-11-paul.elder@ideasonboard.com> X-Mailer: git-send-email 2.47.2 In-Reply-To: <20260703122543.1991189-1-paul.elder@ideasonboard.com> References: <20260703122543.1991189-1-paul.elder@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Implement auto gain control algorithm for the rkisp2 IPA. This leverages the libipa agc, and supports manual gain and exposure control as well. It currently does not function in raw mode, meaning that there is no manual gain and exposure control in raw mode, as no IPA is created when running in raw mode for the rkisp2. Signed-off-by: Paul Elder --- todo: support manual gain and exposure in raw mode --- src/ipa/rkisp2/algorithms/agc.cpp | 472 ++++++++++++++++++++++++++ src/ipa/rkisp2/algorithms/agc.h | 58 ++++ src/ipa/rkisp2/algorithms/meson.build | 1 + 3 files changed, 531 insertions(+) create mode 100644 src/ipa/rkisp2/algorithms/agc.cpp create mode 100644 src/ipa/rkisp2/algorithms/agc.h diff --git a/src/ipa/rkisp2/algorithms/agc.cpp b/src/ipa/rkisp2/algorithms/agc.cpp new file mode 100644 index 000000000000..00300ac7c0a4 --- /dev/null +++ b/src/ipa/rkisp2/algorithms/agc.cpp @@ -0,0 +1,472 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2026, Ideas On Board Oy. + * + * AGC/AEC mean-based control algorithm + */ + +#include "agc.h" + +#include +#include +#include +#include +#include +#include + +#include +#include +#include + +#include +#include +#include + +#include "libipa/histogram.h" + +/** + * \file agc.h + */ + +namespace libcamera { + +using namespace std::literals::chrono_literals; + +namespace ipa::rkisp2::algorithms { + +/** + * \class Agc + * \brief A mean-based auto-exposure algorithm + */ + +LOG_DEFINE_CATEGORY(RkISP2Agc) + +int Agc::init(IPAContext &context, const ValueNode &tuningData) +{ + int ret = parseTuningData(tuningData); + if (ret) + return ret; + + context.ctrlMap[&controls::ExposureTimeMode] = + ControlInfo({ { ControlValue(controls::ExposureTimeModeAuto), + ControlValue(controls::ExposureTimeModeManual) } }, + ControlValue(controls::ExposureTimeModeAuto)); + context.ctrlMap[&controls::AnalogueGainMode] = + ControlInfo({ { ControlValue(controls::AnalogueGainModeAuto), + ControlValue(controls::AnalogueGainModeManual) } }, + ControlValue(controls::AnalogueGainModeAuto)); + context.ctrlMap[&controls::ExposureValue] = ControlInfo(-8.0f, 8.0f, 0.0f); + /* \todo Support AnalogueGain and ExposureTime in raw mode */ + context.ctrlMap.merge(controls()); + + return 0; +} + +/** + * \brief Configure the AGC given a configInfo + * \param[in] context The shared IPA context + * \param[in] configInfo The IPA configuration data + * + * \return 0 + */ +int Agc::configure(IPAContext &context, const IPACameraSensorInfo &configInfo) +{ + context.configuration.agc.measureWindow.h_offs = 0; + context.configuration.agc.measureWindow.v_offs = 0; + context.configuration.agc.measureWindow.h_size = (configInfo.outputSize.width / 5); + /* + * ae lite needs the -2 because the total window height must be + * divisible by 2, and it cannot be equal to or greater than the frame + * size, or else the hardware hangs + * + * \todo Move this to the kernel? + * \todo Check if hist lite also needs this + */ + context.configuration.agc.measureWindow.v_size = (configInfo.outputSize.height / 5) - 2; + + context.configuration.agc.measureWindow15.h_size = (configInfo.outputSize.width / 15); + context.configuration.agc.measureWindow15.v_size = (configInfo.outputSize.height / 15) - 2; + + /* Configure the default exposure and gain. */ + context.activeState.agc.automatic.gain = context.configuration.sensor.minAnalogueGain; + context.activeState.agc.automatic.exposure = + 10ms / context.configuration.sensor.lineDuration; + context.activeState.agc.automatic.quantizationGain = 1.0; + context.activeState.agc.manual.gain = context.activeState.agc.automatic.gain; + context.activeState.agc.manual.exposure = context.activeState.agc.automatic.exposure; + context.activeState.agc.autoExposureEnabled = true; + context.activeState.agc.autoGainEnabled = true; + context.activeState.agc.exposureValue = 0.0; + + /* Limit the frame duration to match current initialisation */ + ControlInfo &frameDurationLimits = context.ctrlMap[&controls::FrameDurationLimits]; + context.activeState.agc.minFrameDuration = std::chrono::microseconds(frameDurationLimits.min().get()); + context.activeState.agc.maxFrameDuration = std::chrono::microseconds(frameDurationLimits.max().get()); + + AgcMeanLuminance::configure(context.configuration.sensor.lineDuration, + context.camHelper.get()); + + setLimits(context.configuration.sensor.minExposureTime, + context.configuration.sensor.maxExposureTime, + context.configuration.sensor.minAnalogueGain, + context.configuration.sensor.maxAnalogueGain, {}); + + context.activeState.agc.automatic.yTarget = effectiveYTarget(); + + resetFrameCount(); + + return 0; +} + +/** + * \copydoc libcamera::ipa::Algorithm::queueRequest + */ +void Agc::queueRequest(IPAContext &context, + [[maybe_unused]] const uint32_t frame, + IPAFrameContext &frameContext, + const ControlList &controls) +{ + auto &agc = context.activeState.agc; + + const auto &aeEnable = controls.get(controls::ExposureTimeMode); + if (aeEnable && + (*aeEnable == controls::ExposureTimeModeAuto) != agc.autoExposureEnabled) { + agc.autoExposureEnabled = (*aeEnable == controls::ExposureTimeModeAuto); + + LOG(RkISP2Agc, Debug) + << (agc.autoExposureEnabled ? "Enabling" : "Disabling") + << " AGC (exposure)"; + + /* + * If we go from auto -> manual with no manual control + * set, use the last computed value, which we don't + * know until prepare() so save this information. + * + * \todo Check the previous frame at prepare() time + * instead of saving a flag here + */ + if (!agc.autoExposureEnabled && !controls.get(controls::ExposureTime)) + frameContext.agc.autoExposureModeChange = true; + } + + const auto &agEnable = controls.get(controls::AnalogueGainMode); + if (agEnable && + (*agEnable == controls::AnalogueGainModeAuto) != agc.autoGainEnabled) { + agc.autoGainEnabled = (*agEnable == controls::AnalogueGainModeAuto); + + LOG(RkISP2Agc, Debug) + << (agc.autoGainEnabled ? "Enabling" : "Disabling") + << " AGC (gain)"; + /* + * If we go from auto -> manual with no manual control + * set, use the last computed value, which we don't + * know until prepare() so save this information. + */ + if (!agc.autoGainEnabled && !controls.get(controls::AnalogueGain)) + frameContext.agc.autoGainModeChange = true; + } + + const auto &exposure = controls.get(controls::ExposureTime); + if (exposure && !agc.autoExposureEnabled) { + agc.manual.exposure = *exposure * 1.0us + / context.configuration.sensor.lineDuration; + + LOG(RkISP2Agc, Debug) + << "Set exposure to " << agc.manual.exposure; + } + + const auto &gain = controls.get(controls::AnalogueGain); + if (gain && !agc.autoGainEnabled) { + agc.manual.gain = *gain; + + LOG(RkISP2Agc, Debug) << "Set gain to " << agc.manual.gain; + } + + frameContext.agc.autoExposureEnabled = agc.autoExposureEnabled; + frameContext.agc.autoGainEnabled = agc.autoGainEnabled; + + if (!frameContext.agc.autoExposureEnabled) + frameContext.agc.exposure = agc.manual.exposure; + if (!frameContext.agc.autoGainEnabled) + frameContext.agc.gain = agc.manual.gain; + + if (!frameContext.agc.autoExposureEnabled && + !frameContext.agc.autoGainEnabled) + frameContext.agc.quantizationGain = 1.0; + + const auto &exposureValue = controls.get(controls::ExposureValue); + if (exposureValue) + agc.exposureValue = *exposureValue; + frameContext.agc.exposureValue = agc.exposureValue; + + const auto &frameDurationLimits = controls.get(controls::FrameDurationLimits); + if (frameDurationLimits) { + /* Limit the control value to the limits in ControlInfo */ + ControlInfo &limits = context.ctrlMap[&controls::FrameDurationLimits]; + int64_t minFrameDuration = + std::clamp((*frameDurationLimits).front(), + limits.min().get(), + limits.max().get()); + int64_t maxFrameDuration = + std::clamp((*frameDurationLimits).back(), + limits.min().get(), + limits.max().get()); + + agc.minFrameDuration = std::chrono::microseconds(minFrameDuration); + agc.maxFrameDuration = std::chrono::microseconds(maxFrameDuration); + } + frameContext.agc.minFrameDuration = agc.minFrameDuration; + frameContext.agc.maxFrameDuration = agc.maxFrameDuration; +} + +/** + * \copydoc libcamera::ipa::Algorithm::prepare + */ +void Agc::prepare(IPAContext &context, [[maybe_unused]] const uint32_t frame, + [[maybe_unused]] IPAFrameContext &frameContext, RkISP2Params *params) +{ + uint32_t activeAutoExposure = context.activeState.agc.automatic.exposure; + double activeAutoGain = context.activeState.agc.automatic.gain; + double activeAutoQGain = context.activeState.agc.automatic.quantizationGain; + + /* Populate exposure and gain in auto mode */ + if (frameContext.agc.autoExposureEnabled) { + frameContext.agc.exposure = activeAutoExposure; + frameContext.agc.quantizationGain = activeAutoQGain; + } + if (frameContext.agc.autoGainEnabled) { + frameContext.agc.gain = activeAutoGain; + frameContext.agc.quantizationGain = activeAutoQGain; + } + + /* + * Populate manual exposure and gain from the active auto values when + * transitioning from auto to manual + */ + if (!frameContext.agc.autoExposureEnabled && frameContext.agc.autoExposureModeChange) { + context.activeState.agc.manual.exposure = activeAutoExposure; + frameContext.agc.exposure = activeAutoExposure; + } + if (!frameContext.agc.autoGainEnabled && frameContext.agc.autoGainModeChange) { + context.activeState.agc.manual.gain = activeAutoGain; + frameContext.agc.gain = activeAutoGain; + frameContext.agc.quantizationGain = activeAutoQGain; + } + + frameContext.agc.yTarget = context.activeState.agc.automatic.yTarget; + + if (frame > 1) + return; + + /* + * Configure the AEC measurements. Set the window, measure + * continuously, and estimate Y as (R + G + B) x (85/256). + */ + auto aeLiteConfig = params->block(); + aeLiteConfig.setEnabled(true); + + aeLiteConfig->window_num = 1; + aeLiteConfig->meas_window = context.configuration.agc.measureWindow; + + auto hstConfig = params->block(); + hstConfig.setEnabled(true); + + hstConfig->window_num = 0; + /* \todo choose this based on the bitdepth */ + hstConfig->data_sel = RKISP2_ISP_HISTOGRAM_DATA_SEL_9_2; + hstConfig->mode = RKISP2_ISP_HISTOGRAM_MODE_Y_HISTOGRAM; + /* waterline means to exclude everything above this value */ + hstConfig->waterline = 0x0; + hstConfig->stepsize = 0; + hstConfig->coeffs.r = 0x20; + hstConfig->coeffs.g = 0x20; + hstConfig->coeffs.b = 0x20; + + hstConfig->meas_window = context.configuration.agc.measureWindow15; + + /* \todo Support configuring the weights */ + for (size_t i = 0; i < RKISP2_ISP_HIST_WEIGHT_GRIDS_SIZE_BIG; i++) + hstConfig->weights[i] = 0x20; + + auto hstConfigLite = params->block(); + hstConfigLite.setEnabled(false); +} + +void Agc::fillMetadata(IPAContext &context, IPAFrameContext &frameContext, + ControlList &metadata, [[maybe_unused]] const rkisp2_stats_buffer *stats) +{ + utils::Duration exposureTime = context.configuration.sensor.lineDuration + * frameContext.sensor.exposure; + metadata.set(controls::AnalogueGain, frameContext.sensor.gain); + metadata.set(controls::ExposureTime, exposureTime.get()); + metadata.set(controls::FrameDuration, frameContext.agc.frameDuration.get()); + metadata.set(controls::ExposureTimeMode, + frameContext.agc.autoExposureEnabled + ? controls::ExposureTimeModeAuto + : controls::ExposureTimeModeManual); + metadata.set(controls::AnalogueGainMode, + frameContext.agc.autoGainEnabled + ? controls::AnalogueGainModeAuto + : controls::AnalogueGainModeManual); + + metadata.set(controls::ExposureValue, frameContext.agc.exposureValue); +} + +double Agc::estimateLuminance(double gain) const +{ + /* + * \todo Enforce this check, since lite-like is 5x5 while big is 15x15, + * but I haven't yet figured out how to use all 15x15 weights. At the + * moment we're running everything lite-like + */ + ASSERT(expMeans_.size() == weights_.size()); + double ySum = 0.0; + double wSum = 0.0; + + /* Sum the averages, saturated to 4095. */ + for (unsigned i = 0; i < expMeans_.size(); i++) { + double w = weights_[i] / 0x10; + ySum += std::min(expMeans_[i] * gain, 4095.0) * w; + wSum += w; + } + + /* \todo Weight with the AWB gains */ + + return ySum / wSum / 4095; +} + +void Agc::processFrameDuration(IPAContext &context, + IPAFrameContext &frameContext, + utils::Duration frameDuration) +{ + IPACameraSensorInfo &sensorInfo = context.sensorInfo; + utils::Duration lineDuration = context.configuration.sensor.lineDuration; + + frameContext.agc.vblank = (frameDuration / lineDuration) - sensorInfo.outputSize.height; + + /* Update frame duration accounting for line length quantization. */ + frameContext.agc.frameDuration = (sensorInfo.outputSize.height + frameContext.agc.vblank) * lineDuration; +} + +/** + * \brief Process RkISP2 statistics, and run AGC operations + * \param[in] context The shared IPA context + * \param[in] frame The frame context sequence number + * \param[in] frameContext The current frame context + * \param[in] stats The RKISP2 statistics and ISP results + * \param[out] metadata Metadata for the frame, to be filled by the algorithm + * + * Identify the current image brightness, and use that to estimate the optimal + * new exposure and gain for the scene. + */ +void Agc::process([[maybe_unused]] IPAContext &context, + [[maybe_unused]] const uint32_t frame, + [[maybe_unused]] IPAFrameContext &frameContext, + const rkisp2_stats_buffer *stats, + ControlList &metadata) +{ + const utils::Duration &lineDuration = context.configuration.sensor.lineDuration; + + /* + * \todo Verify that the exposure and gain applied by the sensor for + * this frame match what has been requested. This isn't a hard + * requirement for stability of the AGC (the guarantee we need in + * automatic mode is a perfect match between the frame and the values + * we receive), but is important in manual mode. + */ + + /* The lower 5 bits are fractional and meant to be discarded. */ + Histogram hist({ stats->hist_big0.hist_bins, RKISP2_ISP_HIST_BIN_N_MAX }, + [](uint32_t x) { return x >> 5; }); + + expMeans_.resize(RKISP2_ISP_AE_MEAN_MAX_LITE); + for (size_t i = 0; i < RKISP2_ISP_AE_MEAN_MAX_LITE; i++) { + /* r and b are 0~1023; g is 255~4095 so multiply r and b to match g */ + uint16_t r = stats->ae_lite.exp_mean_r[i] * 4; + uint16_t g = stats->ae_lite.exp_mean_g[i]; + uint16_t b = stats->ae_lite.exp_mean_b[i] * 4; + expMeans_[i] = 0.2126 * r + 0.7152 * g + 0.0722 * b; + } + + /* \todo Support configuring the weights */ + std::vector modeWeights(25, 0x10); + weights_ = { modeWeights.data(), modeWeights.size() }; + + /* + * Set the AGC limits using the fixed exposure time and/or gain in + * manual mode, or the sensor limits in auto mode. + */ + utils::Duration minExposureTime; + utils::Duration maxExposureTime; + double minAnalogueGain; + double maxAnalogueGain; + + if (frameContext.agc.autoExposureEnabled) { + minExposureTime = context.configuration.sensor.minExposureTime; + maxExposureTime = std::clamp(frameContext.agc.maxFrameDuration, + context.configuration.sensor.minExposureTime, + context.configuration.sensor.maxExposureTime); + } else { + minExposureTime = context.configuration.sensor.lineDuration + * frameContext.agc.exposure; + maxExposureTime = minExposureTime; + } + + if (frameContext.agc.autoGainEnabled) { + minAnalogueGain = context.configuration.sensor.minAnalogueGain; + maxAnalogueGain = context.configuration.sensor.maxAnalogueGain; + } else { + minAnalogueGain = frameContext.agc.gain; + maxAnalogueGain = frameContext.agc.gain; + } + + setLimits(minExposureTime, maxExposureTime, minAnalogueGain, maxAnalogueGain, {}); + + /* + * The Agc algorithm needs to know the effective exposure value that was + * applied to the sensor when the statistics were collected. + */ + utils::Duration exposureTime = lineDuration * frameContext.sensor.exposure; + double analogueGain = frameContext.sensor.gain; + utils::Duration effectiveExposureValue = exposureTime * analogueGain; + if (effectiveExposureValue == 0ms) { + LOG(RkISP2Agc, Warning) + << "frame " << frame << ": Effective exposure value is 0: sensor exposure: " + << exposureTime << ", analogue gain: " << analogueGain; + } + + /* \todo Support lux estimation */ + + /* \todo Support setting constraint and exposure modes */ + utils::Duration newExposureTime; + double aGain, qGain, dGain; + std::tie(newExposureTime, aGain, qGain, dGain) = + calculateNewEv(0, 0, hist, effectiveExposureValue); + + LOG(RkISP2Agc, Debug) + << "Divided up exposure time, analogue gain, quantization gain" + << " and digital gain are " << newExposureTime << ", " << aGain + << ", " << qGain << " and " << dGain; + + IPAActiveState &activeState = context.activeState; + /* Update the estimated exposure and gain. */ + activeState.agc.automatic.exposure = newExposureTime / lineDuration; + activeState.agc.automatic.gain = aGain; + activeState.agc.automatic.quantizationGain = qGain; + activeState.agc.automatic.yTarget = effectiveYTarget(); + /* + * Expand the target frame duration so that we do not run faster than + * the minimum frame duration when we have short exposures. + */ + processFrameDuration(context, frameContext, + std::max(frameContext.agc.minFrameDuration, newExposureTime)); + + fillMetadata(context, frameContext, metadata, stats); + expMeans_ = {}; +} + +REGISTER_IPA_ALGORITHM(Agc, "Agc") + +} /* namespace ipa::rkisp2::algorithms */ + +} /* namespace libcamera */ diff --git a/src/ipa/rkisp2/algorithms/agc.h b/src/ipa/rkisp2/algorithms/agc.h new file mode 100644 index 000000000000..18627c32834a --- /dev/null +++ b/src/ipa/rkisp2/algorithms/agc.h @@ -0,0 +1,58 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2026, Ideas On Board Oy. + * + * RkISP2 AGC/AEC mean-based control algorithm + */ + +#pragma once + +#include + +#include + +#include +#include + +#include "libipa/agc_mean_luminance.h" + +#include "algorithm.h" + +namespace libcamera { + +namespace ipa::rkisp2::algorithms { + +class Agc : public Algorithm, public AgcMeanLuminance +{ +public: + Agc() = default; + ~Agc() = default; + + int init(IPAContext &context, [[maybe_unused]] const ValueNode &tuningData) override; + int configure(IPAContext &context, const IPACameraSensorInfo &configInfo) override; + void queueRequest(IPAContext &context, + const uint32_t frame, + IPAFrameContext &frameContext, + const ControlList &controls) override; + void prepare(IPAContext &context, const uint32_t frame, + IPAFrameContext &frameContext, + RkISP2Params *params) override; + void process(IPAContext &context, const uint32_t frame, + IPAFrameContext &frameContext, + const rkisp2_stats_buffer *stats, + ControlList &metadata) override; + +private: + void fillMetadata(IPAContext &context, IPAFrameContext &frameContext, + ControlList &metadata, const rkisp2_stats_buffer *stats); + double estimateLuminance(double gain) const override; + void processFrameDuration(IPAContext &context, + IPAFrameContext &frameContext, + utils::Duration frameDuration); + + std::vector expMeans_; + Span weights_; +}; + +} /* namespace ipa::rkisp2::algorithms */ +} /* namespace libcamera */ diff --git a/src/ipa/rkisp2/algorithms/meson.build b/src/ipa/rkisp2/algorithms/meson.build index 30133ac4fa68..027e40dd0346 100644 --- a/src/ipa/rkisp2/algorithms/meson.build +++ b/src/ipa/rkisp2/algorithms/meson.build @@ -1,6 +1,7 @@ # SPDX-License-Identifier: CC0-1.0 rkisp2_ipa_algorithms = files([ + 'agc.cpp', 'awb.cpp', 'bls.cpp', ]) From patchwork Fri Jul 3 12:25:17 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Paul Elder X-Patchwork-Id: 27170 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 56C23C328C for ; Fri, 3 Jul 2026 12:26:41 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id E360B65FE2; Fri, 3 Jul 2026 14:26:40 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="n6WsooLV"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 623C965FD8 for ; Fri, 3 Jul 2026 14:26:39 +0200 (CEST) Received: from neptunite.hamster-moth.ts.net (unknown [IPv6:2404:7a81:160:2100:a2cc:2f45:3bd7:2589]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 854B68E0; Fri, 3 Jul 2026 14:25:50 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1783081553; bh=VGh1iqBQr5zsnLh9R5lq1HZjJtvMAW93rS/keKbJcc8=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=n6WsooLVLSjqCsjtksiLY7+AVNnazQgV60SdhWVL9a7ZcOWzlNTOCx3raKU96gDuj dvUv/k2VOBPCUcIekPg4Dshubkie+DG5U9+OwvOZacDbyJlzDugXWRY3WbxrHIv9yJ 5lMIXECxE8i6h3DT+mHQvp/5mCb0aFlrk4+6Zhkw= From: Paul Elder To: laurent.pinchart@ideasonboard.com Cc: Paul Elder , michael.riesch@collabora.com, xuhf@rock-chips.com, stefan.klug@ideasonboard.com, kieran.bingham@ideasonboard.com, dan.scally@ideasonboard.com, jacopo.mondi@ideasonboard.com, nicolas.dufresne@collabora.com, libcamera-devel@lists.libcamera.org Subject: [RFC PATCH 11/19] ipa: rkisp2: algo: ccm: Implement color correction matrix Date: Fri, 3 Jul 2026 21:25:17 +0900 Message-ID: <20260703122543.1991189-12-paul.elder@ideasonboard.com> X-Mailer: git-send-email 2.47.2 In-Reply-To: <20260703122543.1991189-1-paul.elder@ideasonboard.com> References: <20260703122543.1991189-1-paul.elder@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Implement a color correction matrix control algorithm for the rkisp2 IPA. This uses the libipa color correction matrix algorithm. Signed-off-by: Paul Elder --- src/ipa/rkisp2/algorithms/ccm.cpp | 126 ++++++++++++++++++++++++++ src/ipa/rkisp2/algorithms/ccm.h | 57 ++++++++++++ src/ipa/rkisp2/algorithms/meson.build | 1 + 3 files changed, 184 insertions(+) create mode 100644 src/ipa/rkisp2/algorithms/ccm.cpp create mode 100644 src/ipa/rkisp2/algorithms/ccm.h diff --git a/src/ipa/rkisp2/algorithms/ccm.cpp b/src/ipa/rkisp2/algorithms/ccm.cpp new file mode 100644 index 000000000000..d9630ccf23bd --- /dev/null +++ b/src/ipa/rkisp2/algorithms/ccm.cpp @@ -0,0 +1,126 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2026, Ideas On Board + * + * RkISP2 Color Correction Matrix control algorithm + */ + +#include "ccm.h" + +#include + +#include +#include + +#include + +#include + +#include "libipa/interpolator.h" + +/** + * \file ccm.h + * \brief RkISP2 CCM algorithm implementation + */ + +namespace libcamera { + +namespace ipa::rkisp2::algorithms { + +/** + * \class Ccm + * \brief RkISP2 color correction matrix algorithm + */ + +LOG_DEFINE_CATEGORY(RkISP2Ccm) + +/** + * \copydoc libcamera::ipa::Algorithm::init + */ +int Ccm::init([[maybe_unused]] IPAContext &context, const ValueNode &tuningData) +{ + return ccmAlgo_.init(tuningData, context.ctrlMap); +} + +/** + * \copydoc libcamera::ipa::Algorithm::configure + */ +int Ccm::configure(IPAContext &context, + [[maybe_unused]] const IPACameraSensorInfo &configInfo) +{ + return ccmAlgo_.configure(context.activeState.ccm, + context.activeState.awb.automatic.temperatureK); +} + +void Ccm::queueRequest(IPAContext &context, + [[maybe_unused]] const uint32_t frame, + IPAFrameContext &frameContext, + const ControlList &controls) +{ + ccmAlgo_.queueRequest(context.activeState.ccm, frameContext.ccm, controls); +} + +void Ccm::setParameters(RkISP2Params *params, IPAFrameContext &context) +{ + const Matrix &matrix = context.ccm.ccm; + const Matrix &offsets = context.ccm.offsets; + + auto config = params->block(); + config.setEnabled(true); + + /* + * 4 bit integer and 7 bit fractional, ranging from -8 (0x400) to + * +7.9921875 (0x3ff) + */ + for (unsigned int i = 0; i < 3; i++) { + for (unsigned int j = 0; j < 3; j++) + config->coeff[i][j] = Q<4, 7>(matrix[i][j]).quantized(); + } + + for (unsigned int i = 0; i < 3; i++) + config->offset[i] = offsets[i][0] & 0xfff; + + config->y_coeff[0] = 0x26; + config->y_coeff[1] = 0x4b; + config->y_coeff[2] = 0x0f; + config->inflection_point = 0xa; + + for (unsigned int i = 0; i < 17; i++) + config->alp[i] = 0x400; + + /* \todo Support alp */ + + LOG(RkISP2Ccm, Debug) << "Setting matrix " << matrix; + LOG(RkISP2Ccm, Debug) << "Setting offsets " << offsets; +} + +/** + * \copydoc libcamera::ipa::Algorithm::prepare + */ +void Ccm::prepare(IPAContext &context, const uint32_t frame, + IPAFrameContext &frameContext, RkISP2Params *params) +{ + if (frameContext.awb.autoEnabled) + ccmAlgo_.prepare(context.activeState.ccm, frameContext.ccm, + frame, frameContext.awb.temperatureK); + + setParameters(params, frameContext); +} + +/** + * \copydoc libcamera::ipa::Algorithm::process + */ +void Ccm::process([[maybe_unused]] IPAContext &context, + [[maybe_unused]] const uint32_t frame, + IPAFrameContext &frameContext, + [[maybe_unused]] const rkisp2_stats_buffer *stats, + ControlList &metadata) +{ + ccmAlgo_.process(frameContext.ccm, metadata); +} + +REGISTER_IPA_ALGORITHM(Ccm, "Ccm") + +} /* namespace ipa::rkisp2::algorithms */ + +} /* namespace libcamera */ diff --git a/src/ipa/rkisp2/algorithms/ccm.h b/src/ipa/rkisp2/algorithms/ccm.h new file mode 100644 index 000000000000..80a87e1c73d0 --- /dev/null +++ b/src/ipa/rkisp2/algorithms/ccm.h @@ -0,0 +1,57 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2026, Ideas On Board + * + * RkISP2 Color Correction Matrix control algorithm + */ + +#pragma once + +#include + +#include + +#include "libcamera/internal/value_node.h" + +#include "libipa/ccm.h" +#include "libipa/fixedpoint.h" + +#include "algorithm.h" +#include "ipa_context.h" +#include "params.h" + +namespace libcamera { + +namespace ipa::rkisp2::algorithms { + +class Ccm : public Algorithm +{ +public: + Ccm() {} + ~Ccm() = default; + + int init(IPAContext &context, const ValueNode &tuningData) override; + int configure(IPAContext &context, + const IPACameraSensorInfo &configInfo) override; + void queueRequest(IPAContext &context, + const uint32_t frame, + IPAFrameContext &frameContext, + const ControlList &controls) override; + void prepare(IPAContext &context, const uint32_t frame, + IPAFrameContext &frameContext, + RkISP2Params *params) override; + void process(IPAContext &context, const uint32_t frame, + IPAFrameContext &frameContext, + const rkisp2_stats_buffer *stats, + ControlList &metadata) override; + +private: + void parseYaml(const ValueNode &tuningData); + void setParameters(RkISP2Params *params, IPAFrameContext &context); + + CcmAlgorithm> ccmAlgo_; +}; + +} /* namespace ipa::rkisp2::algorithms */ + +} /* namespace libcamera */ diff --git a/src/ipa/rkisp2/algorithms/meson.build b/src/ipa/rkisp2/algorithms/meson.build index 027e40dd0346..e7ae3d163bfb 100644 --- a/src/ipa/rkisp2/algorithms/meson.build +++ b/src/ipa/rkisp2/algorithms/meson.build @@ -4,5 +4,6 @@ rkisp2_ipa_algorithms = files([ 'agc.cpp', 'awb.cpp', 'bls.cpp', + 'ccm.cpp', ]) From patchwork Fri Jul 3 12:25:18 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Paul Elder X-Patchwork-Id: 27171 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 003DDC328C for ; Fri, 3 Jul 2026 12:26:44 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 9D7E865FE7; Fri, 3 Jul 2026 14:26:44 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="TR/nWMVm"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id E876965FDF for ; Fri, 3 Jul 2026 14:26:42 +0200 (CEST) Received: from neptunite.hamster-moth.ts.net (unknown [IPv6:2404:7a81:160:2100:a2cc:2f45:3bd7:2589]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 112981121; Fri, 3 Jul 2026 14:25:53 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1783081557; bh=DMsTFHJRsAKgRMApswYooOF/3mCrzmh/8E+8LHvAgdU=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=TR/nWMVmFkiwWDGb/pDy5cbvGII7dBMF0M/8UUCGx64762Nmw35drDGnRJSwnhg6n l2adT9SkCRAXsk9K3S0h8+tVINTvMw+APoszHPUl0qzCJj9kRGYtsSu8Se3MGTI1C/ Z6g9zNCyUAGtK3UU0kvEB9FGw2dBDGuMieBMxmS8= From: Paul Elder To: laurent.pinchart@ideasonboard.com Cc: Paul Elder , michael.riesch@collabora.com, xuhf@rock-chips.com, stefan.klug@ideasonboard.com, kieran.bingham@ideasonboard.com, dan.scally@ideasonboard.com, jacopo.mondi@ideasonboard.com, nicolas.dufresne@collabora.com, libcamera-devel@lists.libcamera.org Subject: [RFC PATCH 12/19] ipa: rkisp2: algo: csm: Implement color space conversion Date: Fri, 3 Jul 2026 21:25:18 +0900 Message-ID: <20260703122543.1991189-13-paul.elder@ideasonboard.com> X-Mailer: git-send-email 2.47.2 In-Reply-To: <20260703122543.1991189-1-paul.elder@ideasonboard.com> References: <20260703122543.1991189-1-paul.elder@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Implement a static color space conversion algorithm for the rkisp2 IPA. Signed-off-by: Paul Elder --- src/ipa/rkisp2/algorithms/csm.cpp | 177 ++++++++++++++++++++++++++ src/ipa/rkisp2/algorithms/csm.h | 34 +++++ src/ipa/rkisp2/algorithms/meson.build | 1 + 3 files changed, 212 insertions(+) create mode 100644 src/ipa/rkisp2/algorithms/csm.cpp create mode 100644 src/ipa/rkisp2/algorithms/csm.h diff --git a/src/ipa/rkisp2/algorithms/csm.cpp b/src/ipa/rkisp2/algorithms/csm.cpp new file mode 100644 index 000000000000..1733c2938d23 --- /dev/null +++ b/src/ipa/rkisp2/algorithms/csm.cpp @@ -0,0 +1,177 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2026, Ideas On Board + * + * RkISP2 Color space conversion + */ + +#include "csm.h" + +#include +#include + +#include +#include + +#include +#include + +#include "linux/rkisp2-config.h" + +/** + * \file csm.h + */ + +namespace libcamera { + +namespace ipa::rkisp2::algorithms { + +/** + * \class ColorSpaceConversion + * \brief RkISP2 Color space conversion + * + * This algorithm implements the color space conversion for the RkISP2. + */ + +LOG_DEFINE_CATEGORY(RkISP2Csm) + +namespace { + struct CsmCoeffs { + uint16_t limited[9]; + uint16_t full[9]; + }; + + static const struct CsmCoeffs rec601Coeffs = { + { + 0x0021, 0x0042, 0x000d, + 0x01ed, 0x01db, 0x0038, + 0x0038, 0x01d1, 0x01f7, + }, + { + 0x0026, 0x004b, 0x000f, + 0x01ea, 0x01d6, 0x0040, + 0x0040, 0x01ca, 0x01f6, + }, + }; + + static const struct CsmCoeffs rec709Coeffs = { + { + 0x0018, 0x0050, 0x0008, + 0x01f3, 0x01d5, 0x0038, + 0x0038, 0x01cd, 0x01fb, + }, + { + 0x001b, 0x005c, 0x0009, + 0x01f1, 0x01cf, 0x0040, + 0x0040, 0x01c6, 0x01fa, + }, + }; + + static const struct CsmCoeffs rec2020Coeffs = { + { + 0x001d, 0x004c, 0x0007, + 0x01f0, 0x01d8, 0x0038, + 0x0038, 0x01cd, 0x01fb, + }, + { + 0x0022, 0x0057, 0x0008, + 0x01ee, 0x01d2, 0x0040, + 0x0040, 0x01c5, 0x01fb, + }, + }; + + static const struct CsmCoeffs smpte240mCoeffs = { + { + 0x0018, 0x004f, 0x000a, + 0x01f3, 0x01d5, 0x0038, + 0x0038, 0x01ce, 0x01fa, + }, + { + 0x001b, 0x005a, 0x000b, + 0x01f1, 0x01cf, 0x0040, + 0x0040, 0x01c7, 0x01f9, + }, + }; + + uint16_t identityCsm[9] = { + 1, 0, 0, + 0, 1, 0, + 0, 0, 1, + }; +} /* namespace */ + +/** + * \copydoc libcamera::ipa::Algorithm::init + */ +int ColorSpaceConversion::init([[maybe_unused]] IPAContext &context, + [[maybe_unused]] const ValueNode &tuningData) +{ + return 0; +} + +/** + * \copydoc libcamera::ipa::Algorithm::configure + */ +int ColorSpaceConversion::configure(IPAContext &context, + [[maybe_unused]] const IPACameraSensorInfo &configInfo) +{ + const CsmCoeffs *coeffs; + switch (context.configuration.csm.colorSpaceEncoding) { + case static_cast(ColorSpace::YcbcrEncoding::Rec601): + coeffs = &rec601Coeffs; + break; + case static_cast(ColorSpace::YcbcrEncoding::Rec709): + coeffs = &rec709Coeffs; + break; + case static_cast(ColorSpace::YcbcrEncoding::Rec2020): + coeffs = &rec2020Coeffs; + break; + default: + coeffs = nullptr; + break; + } + + if (!coeffs) { + context.activeState.csm.csm = Matrix(identityCsm); + return 0; + } + + if (context.configuration.csm.colorSpaceRange == static_cast(ColorSpace::Range::Limited)) + context.activeState.csm.csm = Matrix(coeffs->limited); + else + context.activeState.csm.csm = Matrix(coeffs->full); + context.activeState.csm.update = true; + + return 0; +} + +/** + * \copydoc libcamera::ipa::Algorithm::prepare + */ +void ColorSpaceConversion::prepare(IPAContext &context, + [[maybe_unused]] const uint32_t frame, + [[maybe_unused]] IPAFrameContext &frameContext, + RkISP2Params *params) +{ + if (!context.activeState.csm.update) + return; + + auto config = params->block(); + config.setEnabled(true); + + /* + * We'll use the active state directly as we don't support runtime + * configuration of the csm + */ + for (size_t i = 0; i < 3; i++) + for (size_t j = 0; j < 3; j++) + config->coeff[i][j] = context.activeState.csm.csm[i][j]; + + context.activeState.csm.update = false; +} + +REGISTER_IPA_ALGORITHM(ColorSpaceConversion, "ColorSpaceConversion") + +} /* namespace ipa::rkisp2::algorithms */ + +} /* namespace libcamera */ diff --git a/src/ipa/rkisp2/algorithms/csm.h b/src/ipa/rkisp2/algorithms/csm.h new file mode 100644 index 000000000000..d34e4d3959e8 --- /dev/null +++ b/src/ipa/rkisp2/algorithms/csm.h @@ -0,0 +1,34 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2026, Ideas On Board + * + * RkISP2 Color space conversion + */ + +#pragma once + +#include "libcamera/internal/matrix.h" + +#include "algorithm.h" + +namespace libcamera { + +namespace ipa::rkisp2::algorithms { + +class ColorSpaceConversion : public Algorithm +{ +public: + ColorSpaceConversion() = default; + ~ColorSpaceConversion() = default; + + int init(IPAContext &context, const ValueNode &tuningData) override; + int configure(IPAContext &context, + const IPACameraSensorInfo &configInfo) override; + void prepare(IPAContext &context, const uint32_t frame, + IPAFrameContext &frameContext, + RkISP2Params *params) override; +}; + +} /* namespace ipa::rkisp2::algorithms */ + +} /* namespace libcamera */ diff --git a/src/ipa/rkisp2/algorithms/meson.build b/src/ipa/rkisp2/algorithms/meson.build index e7ae3d163bfb..01829e25d254 100644 --- a/src/ipa/rkisp2/algorithms/meson.build +++ b/src/ipa/rkisp2/algorithms/meson.build @@ -5,5 +5,6 @@ rkisp2_ipa_algorithms = files([ 'awb.cpp', 'bls.cpp', 'ccm.cpp', + 'csm.cpp', ]) From patchwork Fri Jul 3 12:25:19 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Paul Elder X-Patchwork-Id: 27172 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 9BF6CC328C for ; Fri, 3 Jul 2026 12:26:47 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 57CB765FEA; Fri, 3 Jul 2026 14:26:47 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="BzS6oDX7"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 7268265FDF for ; Fri, 3 Jul 2026 14:26:46 +0200 (CEST) Received: from neptunite.hamster-moth.ts.net (unknown [IPv6:2404:7a81:160:2100:a2cc:2f45:3bd7:2589]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 995ED8E0; Fri, 3 Jul 2026 14:25:57 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1783081560; bh=4LOkjL+2uX7mJcASdeNtWhBkBEuTo6ErAi5Zr5hjz7U=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=BzS6oDX7OFtH5J+iUNp5cVdehrIq3ok7Hl01IqLB9bhtn6lKRW3/UBoRLNb5VTSyc kxGcDz45id3IBrj/eyfwk6JFUZgVQWG3pCrWo+hA/w0nCzl1JwP534U0UJHtSgpE6h FuQDQOqcanpXFx7jduEm7S4CGXB8jYeswu++43ik= From: Paul Elder To: laurent.pinchart@ideasonboard.com Cc: Paul Elder , michael.riesch@collabora.com, xuhf@rock-chips.com, stefan.klug@ideasonboard.com, kieran.bingham@ideasonboard.com, dan.scally@ideasonboard.com, jacopo.mondi@ideasonboard.com, nicolas.dufresne@collabora.com, libcamera-devel@lists.libcamera.org Subject: [RFC PATCH 13/19] ipa: rkisp2: algo: goc: Implement gamma out correction Date: Fri, 3 Jul 2026 21:25:19 +0900 Message-ID: <20260703122543.1991189-14-paul.elder@ideasonboard.com> X-Mailer: git-send-email 2.47.2 In-Reply-To: <20260703122543.1991189-1-paul.elder@ideasonboard.com> References: <20260703122543.1991189-1-paul.elder@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Implement a manual gamma out correction algorithm for the rkisp2 IPA. It takes the gamma control and applies the gamma out correction curve to the ISP. Signed-off-by: Paul Elder --- src/ipa/rkisp2/algorithms/goc.cpp | 143 ++++++++++++++++++++++++++ src/ipa/rkisp2/algorithms/goc.h | 42 ++++++++ src/ipa/rkisp2/algorithms/meson.build | 1 + 3 files changed, 186 insertions(+) create mode 100644 src/ipa/rkisp2/algorithms/goc.cpp create mode 100644 src/ipa/rkisp2/algorithms/goc.h diff --git a/src/ipa/rkisp2/algorithms/goc.cpp b/src/ipa/rkisp2/algorithms/goc.cpp new file mode 100644 index 000000000000..d5cabeab8900 --- /dev/null +++ b/src/ipa/rkisp2/algorithms/goc.cpp @@ -0,0 +1,143 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2026, Ideas On Board + * + * RkISP2 Gamma out control + */ + +#include "goc.h" + +#include + +#include +#include + +#include + +#include "linux/rkisp2-config.h" + +/** + * \file goc.h + */ + +namespace libcamera { + +namespace ipa::rkisp2::algorithms { + +/** + * \class GammaOutCorrection + * \brief RkISP2 Gamma out correction + * + * This algorithm implements the gamma out curve for the RkISP2. It defaults to + * a gamma value of 2.2. + * + * As gamma is internally represented as a piecewise linear function with only + * 17 knots, the difference between gamma=2.2 and sRGB gamma is minimal. + * Therefore sRGB gamma was not implemented as special case. + * + * Useful links: + * - https://www.cambridgeincolour.com/tutorials/gamma-correction.htm + * - https://en.wikipedia.org/wiki/SRGB + */ + +LOG_DEFINE_CATEGORY(RkISP2Gamma) + +const float kDefaultGamma = 2.2f; + +/** + * \copydoc libcamera::ipa::Algorithm::init + */ +int GammaOutCorrection::init(IPAContext &context, const ValueNode &tuningData) +{ + defaultGamma_ = tuningData["gamma"].get(kDefaultGamma); + context.ctrlMap[&controls::Gamma] = ControlInfo(0.1f, 10.0f, defaultGamma_); + + return 0; +} + +/** + * \copydoc libcamera::ipa::Algorithm::configure + */ +int GammaOutCorrection::configure(IPAContext &context, + [[maybe_unused]] const IPACameraSensorInfo &configInfo) +{ + context.activeState.goc.gamma = defaultGamma_; + return 0; +} + +/** + * \copydoc libcamera::ipa::Algorithm::queueRequest + */ +void GammaOutCorrection::queueRequest(IPAContext &context, + [[maybe_unused]] const uint32_t frame, + IPAFrameContext &frameContext, + const ControlList &controls) +{ + frameContext.goc.gamma = context.activeState.goc.gamma; + + const auto &gamma = controls.get(controls::Gamma); + if (!gamma) + return; + + context.activeState.goc.gamma = *gamma; + + frameContext.goc.gamma = context.activeState.goc.gamma; +} + +/** + * \copydoc libcamera::ipa::Algorithm::prepare + */ +void GammaOutCorrection::prepare([[maybe_unused]] IPAContext &context, + [[maybe_unused]] const uint32_t frame, + [[maybe_unused]] IPAFrameContext &frameContext, + [[maybe_unused]] RkISP2Params *params) +{ + /* \todo Optimize so we don't have to write it every frame */ + + /* + * The logarithmic segments as specified in the reference, plus an + * additional 0 to make the loop easier + * + * Just use 44 mode for now. 48 mode is when the last four 512 are + * split into eight pieces of 256. + */ + static constexpr std::array segments = { + 0, 1, 1, 1, 1, 1, 1, 1, 1, + 2, 2, 2, 2, 4, 4, 4, 4, + 8, 8, 8, 8, 16, 16, 16, 16, + 32, 32, 32, 32, 64, 64, 64, 64, + 128, 128, 128, 128, 256, 256, 256, 256, + 512, 512, 512, 512 + }; + + auto config = params->block(); + config.setEnabled(true); + + unsigned x = 0; + for (const auto [i, size] : utils::enumerate(segments)) { + config->gamma_y[i] = static_cast(std::pow(x / 4096.0, 1.0 / frameContext.goc.gamma) * 4096.0); + x += size; + } + + config->mode = RKISP2_ISP_GOC_MODE_LOGARITHMIC; + config->segments = RKISP2_ISP_GOC_SEGMENTS_44; + config->offset = 0; +} + +/** + * \copydoc libcamera::ipa::Algorithm::process + */ +void GammaOutCorrection::process([[maybe_unused]] IPAContext &context, + [[maybe_unused]] const uint32_t frame, + IPAFrameContext &frameContext, + [[maybe_unused]] const rkisp2_stats_buffer *stats, + ControlList &metadata) +{ + metadata.set(controls::Gamma, frameContext.goc.gamma); +} + +REGISTER_IPA_ALGORITHM(GammaOutCorrection, "GammaOutCorrection") + +} /* namespace ipa::rkisp2::algorithms */ + +} /* namespace libcamera */ diff --git a/src/ipa/rkisp2/algorithms/goc.h b/src/ipa/rkisp2/algorithms/goc.h new file mode 100644 index 000000000000..e859f6606cd6 --- /dev/null +++ b/src/ipa/rkisp2/algorithms/goc.h @@ -0,0 +1,42 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2026, Ideas On Board + * + * RkISP2 Gamma out control + */ + +#pragma once + +#include "algorithm.h" + +namespace libcamera { + +namespace ipa::rkisp2::algorithms { + +class GammaOutCorrection : public Algorithm +{ +public: + GammaOutCorrection() = default; + ~GammaOutCorrection() = default; + + int init(IPAContext &context, const ValueNode &tuningData) override; + int configure(IPAContext &context, + const IPACameraSensorInfo &configInfo) override; + void queueRequest(IPAContext &context, + const uint32_t frame, + IPAFrameContext &frameContext, + const ControlList &controls) override; + void prepare(IPAContext &context, const uint32_t frame, + IPAFrameContext &frameContext, + RkISP2Params *params) override; + void process(IPAContext &context, const uint32_t frame, + IPAFrameContext &frameContext, + const rkisp2_stats_buffer *stats, + ControlList &metadata) override; + +private: + float defaultGamma_; +}; + +} /* namespace ipa::rkisp2::algorithms */ +} /* namespace libcamera */ diff --git a/src/ipa/rkisp2/algorithms/meson.build b/src/ipa/rkisp2/algorithms/meson.build index 01829e25d254..3d73b0a99320 100644 --- a/src/ipa/rkisp2/algorithms/meson.build +++ b/src/ipa/rkisp2/algorithms/meson.build @@ -6,5 +6,6 @@ rkisp2_ipa_algorithms = files([ 'bls.cpp', 'ccm.cpp', 'csm.cpp', + 'goc.cpp', ]) From patchwork Fri Jul 3 12:25:20 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Paul Elder X-Patchwork-Id: 27173 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 63083C328C for ; Fri, 3 Jul 2026 12:26:52 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id E5C4A65FED; Fri, 3 Jul 2026 14:26:51 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="cJkcWQTG"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 051D665FE2 for ; Fri, 3 Jul 2026 14:26:50 +0200 (CEST) Received: from neptunite.hamster-moth.ts.net (unknown [IPv6:2404:7a81:160:2100:a2cc:2f45:3bd7:2589]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 262261121; Fri, 3 Jul 2026 14:26:00 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1783081564; bh=F6zaFA2WMPuL8yNraGr/Kka8q5Lh7u+ysuifoo/7AxA=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=cJkcWQTGeOw0mbR6ztFR7ES/zg458iuQrNjM+Tp6ZJdpuG8YpO5ltkisFbTAwOy9D zUG0kFrkdzH4IKd6p1YzJNVRcC9cUlEZ2tO3uB3NC9EjNKAp8OCQoKTEdfuzX8pJkT I4kzZ3RsfXlJjE2qyjjoWw7grhvzkLxEdxhoM0qg= From: Paul Elder To: laurent.pinchart@ideasonboard.com Cc: Paul Elder , michael.riesch@collabora.com, xuhf@rock-chips.com, stefan.klug@ideasonboard.com, kieran.bingham@ideasonboard.com, dan.scally@ideasonboard.com, jacopo.mondi@ideasonboard.com, nicolas.dufresne@collabora.com, libcamera-devel@lists.libcamera.org Subject: [RFC PATCH 14/19] ipa: rkisp2: algo: lsc: Implement lens shading correction Date: Fri, 3 Jul 2026 21:25:20 +0900 Message-ID: <20260703122543.1991189-15-paul.elder@ideasonboard.com> X-Mailer: git-send-email 2.47.2 In-Reply-To: <20260703122543.1991189-1-paul.elder@ideasonboard.com> References: <20260703122543.1991189-1-paul.elder@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Implement a lens shading correction algorithm for the rkisp2 IPA. It uses the libipa lens shading correction. Signed-off-by: Paul Elder --- src/ipa/rkisp2/algorithms/lsc.cpp | 263 ++++++++++++++++++++++++++ src/ipa/rkisp2/algorithms/lsc.h | 71 +++++++ src/ipa/rkisp2/algorithms/meson.build | 1 + 3 files changed, 335 insertions(+) create mode 100644 src/ipa/rkisp2/algorithms/lsc.cpp create mode 100644 src/ipa/rkisp2/algorithms/lsc.h diff --git a/src/ipa/rkisp2/algorithms/lsc.cpp b/src/ipa/rkisp2/algorithms/lsc.cpp new file mode 100644 index 000000000000..dd93583dffdc --- /dev/null +++ b/src/ipa/rkisp2/algorithms/lsc.cpp @@ -0,0 +1,263 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2026, Ideas On Board + * + * RkISP2 Lens Shading Correction control + */ + +#include "lsc.h" + +#include +#include +#include + +#include +#include + +/** + * \file lsc.h + */ + +namespace libcamera { + +namespace ipa::rkisp2::algorithms { + +/** + * \class LensShadingCorrection + * \brief RkISP2 Lens Shading Correction control + */ + +LOG_DEFINE_CATEGORY(RkISP2Lsc) + +namespace { + +constexpr int kColourTemperatureQuantization = 10; + +unsigned int quantize(unsigned int value, unsigned int step) +{ + return std::lround(value / static_cast(step)) * step; +} + +} /* namespace */ + +LensShadingCorrection::LensShadingCorrection() + : lastAppliedCt_(0), lastAppliedQuantizedCt_(0) +{ +} + +std::vector LensShadingCorrection::parseSizes(const ValueNode &tuningData, + const char *prop) +{ + std::vector sizes = + tuningData[prop].get>().value_or(utils::defopt); + /* Nobody cares about 8x8 mirrored mode; we'll just use 16x16 mode */ + if (sizes.size() != RKISP2_ISP_LSC_SECTORS_TBL_SIZE_MAX) { + LOG(RkISP2Lsc, Error) + << "Invalid '" << prop << "' values: expected " + << RKISP2_ISP_LSC_SECTORS_TBL_SIZE_MAX + << " elements, got " << sizes.size(); + return {}; + } + + /* + * The sum of all elements must be 1 to satisfy hardware constraints. + * Validate it here, allowing a 1% tolerance as rounding errors may + * prevent an exact match (further adjustments will be performed in + * LensShadingCorrection::prepare()). + * + * If we were in 8x8 mode then we'd have to mirror the quadrants like + * in rkisp1, but in 16x16 mode we get to configure the entire table. + * Since 8x8 table support is a todo, we only need to handle the 16x16 + * case here thus the sum should be 1. + * + * \todo Support 8x8 mode? + */ + double sum = std::accumulate(sizes.begin(), sizes.end(), 0.0); + if (sum < 0.95 || sum > 1.05) { + LOG(RkISP2Lsc, Error) + << "Invalid '" << prop << "' values: sum of the elements" + << " should be 1.0, got " << sum; + return {}; + } + + return sizes; +} + +std::vector LensShadingCorrection::sizesToPositions(Span sizes) +{ + std::vector positions(sizes.size() + 1); + + positions[0] = 0.0; + for (size_t i = 1; i < positions.size(); i++) + positions[i] = positions[i - 1] + sizes[i - 1]; + + return positions; +} + +/** + * \copydoc libcamera::ipa::Algorithm::init + */ +int LensShadingCorrection::init(IPAContext &context, + const ValueNode &tuningData) +{ + xSize_ = parseSizes(tuningData, "x-size"); + ySize_ = parseSizes(tuningData, "y-size"); + + if (xSize_.empty() || ySize_.empty()) + return -EINVAL; + + xPos_ = sizesToPositions(xSize_); + yPos_ = sizesToPositions(ySize_); + + return lscAlgo_.init(tuningData, context.ctrlMap, { + .keys = { "r", "gr", "gb", "b" }, + .numHCells = RKISP2_ISP_LSC_SAMPLES_MAX, + .numVCells = RKISP2_ISP_LSC_SAMPLES_MAX, + .sensorSize = context.sensorInfo.activeAreaSize + }); +} + +/** + * \copydoc libcamera::ipa::Algorithm::configure + */ +int LensShadingCorrection::configure(IPAContext &context, + const IPACameraSensorInfo &configInfo) +{ + const Size &size = context.configuration.sensor.size; + Size totalSize{}; + + /* Calculate gradients. */ + for (unsigned int i = 0; i < RKISP2_ISP_LSC_SECTORS_TBL_SIZE_MAX; ++i) { + xSizes_[i] = xSize_[i] * size.width; + ySizes_[i] = ySize_[i] * size.height; + + /* + * To prevent unexpected behavior of the ISP, the sum of + * x_sizes and y_sizes items shall be equal to + * respectively size.width and size.height. Enforce it by + * computing the last tables value to avoid + * rounding-induced errors. + */ + if (i == RKISP2_ISP_LSC_SECTORS_TBL_SIZE_MAX - 1) { + xSizes_[i] = size.width - totalSize.width; + ySizes_[i] = size.height - totalSize.height; + } + + totalSize.width += xSizes_[i]; + totalSize.height += ySizes_[i]; + + xGrad_[i] = std::round(32768 / xSizes_[i]); + yGrad_[i] = std::round(32768 / ySizes_[i]); + } + + return lscAlgo_.configure(context.activeState.lsc, configInfo.analogCrop, + xPos_, yPos_); +} + +void LensShadingCorrection::setParameters(rkisp2_params_lsc &config) +{ + memcpy(config.x_grads, xGrad_, sizeof(config.x_grads)); + memcpy(config.y_grads, yGrad_, sizeof(config.y_grads)); + memcpy(config.x_sizes, xSizes_, sizeof(config.x_sizes)); + memcpy(config.y_sizes, ySizes_, sizeof(config.y_sizes)); +} + +void LensShadingCorrection::copyTable(rkisp2_params_lsc &config, + const ipa::lsc::Components &set) +{ + const auto &r = set.at("r"); + std::copy(r.begin(), r.end(), &config.r_data_tbl[0][0][0]); + const auto &gr = set.at("gr"); + std::copy(gr.begin(), gr.end(), &config.gr_data_tbl[0][0][0]); + const auto &gb = set.at("gb"); + std::copy(gb.begin(), gb.end(), &config.gb_data_tbl[0][0][0]); + const auto &b = set.at("b"); + std::copy(b.begin(), b.end(), &config.b_data_tbl[0][0][0]); +} + +/** + * \copydoc libcamera::ipa::Algorithm::queueRequest + */ +void LensShadingCorrection::queueRequest(IPAContext &context, + [[maybe_unused]] const uint32_t frame, + IPAFrameContext &frameContext, + const ControlList &controls) +{ + lscAlgo_.queueRequest(context.activeState.lsc, frameContext.lsc, + controls); +} + +/** + * \copydoc libcamera::ipa::Algorithm::prepare + */ +void LensShadingCorrection::prepare([[maybe_unused]] IPAContext &context, + [[maybe_unused]] const uint32_t frame, + IPAFrameContext &frameContext, + RkISP2Params *params) +{ + uint32_t ct = frameContext.awb.temperatureK; + unsigned int quantizedCt = quantize(ct, kColourTemperatureQuantization); + + /* Check if we can skip the update. */ + if (!frameContext.lsc.update) { + if (!frameContext.lsc.enabled) + return; + + /* + * Add a threshold so that oscillations around a quantization + * step don't lead to constant changes. + */ + if (utils::abs_diff(ct, lastAppliedCt_) < kColourTemperatureQuantization / 2) + return; + + if (quantizedCt == lastAppliedQuantizedCt_) + return; + } + + auto config = params->block(); + config.setEnabled(frameContext.lsc.enabled); + + if (!frameContext.lsc.enabled) + return; + + /* + * \todo Should add support for the lsc table swapping functionality? + * Or maybe we don't need it because the lsc doesn't change very + * frequently. Just use the 0th table for now. + */ + config->window_mode = RKISP2_ISP_LSC_CONFIG_16X16; + config->write_table[0] = 1; + config->write_table[1] = 0; + config->active_table = 0; + config->set_active_table_when = RKISP2_ISP_LSC_SET_ACTIVE_TABLE_AFTER; + + setParameters(*config); + + const auto &set = lscAlgo_.interpolateComponents(quantizedCt); + copyTable(*config, set); + + lastAppliedCt_ = ct; + lastAppliedQuantizedCt_ = quantizedCt; + + LOG(RkISP2Lsc, Debug) + << "ct is " << ct << ", quantized to " + << quantizedCt; +} + +/** + * \copydoc libcamera::ipa::Algorithm::process + */ +void LensShadingCorrection::process([[maybe_unused]] IPAContext &context, + [[maybe_unused]] const uint32_t frame, + IPAFrameContext &frameContext, + [[maybe_unused]] const rkisp2_stats_buffer *stats, + ControlList &metadata) +{ + lscAlgo_.process(frameContext.lsc, metadata); +} + +REGISTER_IPA_ALGORITHM(LensShadingCorrection, "LensShadingCorrection") + +} /* namespace ipa::rkisp2::algorithms */ + +} /* namespace libcamera */ diff --git a/src/ipa/rkisp2/algorithms/lsc.h b/src/ipa/rkisp2/algorithms/lsc.h new file mode 100644 index 000000000000..78cbb094ec64 --- /dev/null +++ b/src/ipa/rkisp2/algorithms/lsc.h @@ -0,0 +1,71 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2026, Ideas On Board + * + * RkISP2 Lens Shading Correction algorithm + */ + +#pragma once + +#include + +#include + +#include "libcamera/internal/value_node.h" + +#include "libipa/fixedpoint.h" +#include "libipa/lsc.h" + +#include "algorithm.h" +#include "ipa_context.h" +#include "params.h" + +namespace libcamera { + +namespace ipa::rkisp2::algorithms { + +class LensShadingCorrection : public Algorithm +{ +public: + LensShadingCorrection(); + ~LensShadingCorrection() = default; + + int init(IPAContext &context, const ValueNode &tuningData) override; + int configure(IPAContext &context, const IPACameraSensorInfo &configInfo) override; + void queueRequest(IPAContext &context, const uint32_t frame, + IPAFrameContext &frameContext, + const ControlList &controls) override; + void prepare(IPAContext &context, const uint32_t frame, + IPAFrameContext &frameContext, + RkISP2Params *params) override; + void process(IPAContext &context, const uint32_t frame, + IPAFrameContext &frameContext, + const rkisp2_stats_buffer *stats, + ControlList &metadata) override; + +private: + std::vector parseSizes(const ValueNode &tuningData, + const char *prop); + std::vector sizesToPositions(Span sizes); + + void setParameters(rkisp2_params_lsc &config); + void copyTable(rkisp2_params_lsc &config, + const ipa::lsc::Components &set0); + + std::vector xSize_; + std::vector ySize_; + uint16_t xGrad_[RKISP2_ISP_LSC_SECTORS_TBL_SIZE_MAX]; + uint16_t yGrad_[RKISP2_ISP_LSC_SECTORS_TBL_SIZE_MAX]; + uint16_t xSizes_[RKISP2_ISP_LSC_SECTORS_TBL_SIZE_MAX]; + uint16_t ySizes_[RKISP2_ISP_LSC_SECTORS_TBL_SIZE_MAX]; + std::vector xPos_; + std::vector yPos_; + + unsigned int lastAppliedCt_; + unsigned int lastAppliedQuantizedCt_; + + LscAlgorithm> lscAlgo_; +}; + +} /* namespace ipa::rkisp2::algorithms */ +} /* namespace libcamera */ diff --git a/src/ipa/rkisp2/algorithms/meson.build b/src/ipa/rkisp2/algorithms/meson.build index 3d73b0a99320..bcc947fabdb4 100644 --- a/src/ipa/rkisp2/algorithms/meson.build +++ b/src/ipa/rkisp2/algorithms/meson.build @@ -7,5 +7,6 @@ rkisp2_ipa_algorithms = files([ 'ccm.cpp', 'csm.cpp', 'goc.cpp', + 'lsc.cpp', ]) From patchwork Fri Jul 3 12:25:21 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Paul Elder X-Patchwork-Id: 27174 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id B4B31C328C for ; Fri, 3 Jul 2026 12:26:54 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 6D95465FED; Fri, 3 Jul 2026 14:26:54 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="ttfNjBkn"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 8856E65FE2 for ; Fri, 3 Jul 2026 14:26:53 +0200 (CEST) Received: from neptunite.hamster-moth.ts.net (unknown [IPv6:2404:7a81:160:2100:a2cc:2f45:3bd7:2589]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id AAED68E0; Fri, 3 Jul 2026 14:26:04 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1783081567; bh=FCiGnoB4Og2o4qyO7umIfuJjmc37YufL40KZPeR89kE=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=ttfNjBknqTHg0h/C2WNO675WYHhpjQT3qFIhzvr0TVMO/rie7VWUjW//X2gXhkWtf OsgTjG5iP+siuZqhcNSuavKJoopbHQKA+Jsp/Tyc4nsqZPF1IhPV8lu8o8Aam7E4Au TFVs++NxFnucrBcFdtMk9+OzaPxHv4zxxyBlADsU= From: Paul Elder To: laurent.pinchart@ideasonboard.com Cc: Paul Elder , michael.riesch@collabora.com, xuhf@rock-chips.com, stefan.klug@ideasonboard.com, kieran.bingham@ideasonboard.com, dan.scally@ideasonboard.com, jacopo.mondi@ideasonboard.com, nicolas.dufresne@collabora.com, libcamera-devel@lists.libcamera.org Subject: [RFC PATCH 15/19] utils: tuning: libtuning: image: Enable images missing some metadata Date: Fri, 3 Jul 2026 21:25:21 +0900 Message-ID: <20260703122543.1991189-16-paul.elder@ideasonboard.com> X-Mailer: git-send-email 2.47.2 In-Reply-To: <20260703122543.1991189-1-paul.elder@ideasonboard.com> References: <20260703122543.1991189-1-paul.elder@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Some pipeline handlers, like the rkisp2 pipeline handler, are capable of taking raw images but cannot set: - exposure time - gain - black level yet as they have not been implemented for raw mode yet. Arguably black mode will never be supported in raw mode as it bypasses the ISP completely and black level subtraction is in the ISP. This prevents the images from being used for tuning in the current libtuning-based tuning scripts as it requires exposure time and iso and black level, so modify libtuning to allow images with no exposure time and iso and black level. Signed-off-by: Paul Elder --- utils/tuning/libtuning/image.py | 25 ++++++++++++++++++++----- 1 file changed, 20 insertions(+), 5 deletions(-) diff --git a/utils/tuning/libtuning/image.py b/utils/tuning/libtuning/image.py index ecd334bdc67f..b1ada0d0867c 100644 --- a/utils/tuning/libtuning/image.py +++ b/utils/tuning/libtuning/image.py @@ -70,12 +70,27 @@ class Image: white = metadata[f'Exif.{subimage}.WhiteLevel'].value self.sigbits = int(white).bit_length() self.fmt = (self.sigbits - 4) // 2 - self.exposure = int(metadata[f'Exif.{photo}.ExposureTime'].value * 1000000) - self.againQ8 = metadata[f'Exif.{photo}.ISOSpeedRatings'].value * 256 / 100 - self.againQ8_norm = self.againQ8 / 256 + + self.exposure = None + exposure_key = f'Exif.{photo}.ExposureTime' + if exposure_key in metadata: + self.exposure = int(metadata[exposure_key].value * 1000000) + + self.againQ8 = None + self.againQ8_norm = None + iso_key = f'Exif.{photo}.ISOSpeedRatings' + if iso_key in metadata: + self.againQ8 = metadata[iso_key].value * 256 / 100 + self.againQ8_norm = self.againQ8 / 256 + + self.blacklevel = 0 + self.blacklevel_16 = 0 + bl_key = f'Exif.{subimage}.BlackLevel' + if bl_key in metadata: + self.blacklevel = int(metadata[bl_key].value[0]) + self.blacklevel_16 = self.blacklevel << (16 - self.sigbits) + self.camName = metadata['Exif.Image.Model'].value - self.blacklevel = int(metadata[f'Exif.{subimage}.BlackLevel'].value[0]) - self.blacklevel_16 = self.blacklevel << (16 - self.sigbits) # Channel order depending on bayer pattern # The key is the order given by exif, where 0 is R, 1 is G, and 2 is B From patchwork Fri Jul 3 12:25:22 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Paul Elder X-Patchwork-Id: 27175 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 4FD6BC328C for ; Fri, 3 Jul 2026 12:26:59 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 02EA365FF0; Fri, 3 Jul 2026 14:26:59 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="jE8GAeBf"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 195FB65FC2 for ; Fri, 3 Jul 2026 14:26:57 +0200 (CEST) Received: from neptunite.hamster-moth.ts.net (unknown [IPv6:2404:7a81:160:2100:a2cc:2f45:3bd7:2589]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 371051121; Fri, 3 Jul 2026 14:26:07 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1783081571; bh=Zlm0egukM4G+0+A8YxhawoYkXbpnOYicQACr04edykw=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=jE8GAeBfH3V8TkuJpyOZ3WCe/+PqxpnmZIilyUUNdqxM0U3uG9OPFU245EDSZKARS OrFVtmm0REkbdmW3KSsI7Jq23Uu5f0xiPKW0nMqU1+FZLSMdeTefMfpDb/kUW5Yu4g XiWrVe2FAxgBDr6ISn2ONLgYpsogccIBAxwuvDyg= From: Paul Elder To: laurent.pinchart@ideasonboard.com Cc: Paul Elder , michael.riesch@collabora.com, xuhf@rock-chips.com, stefan.klug@ideasonboard.com, kieran.bingham@ideasonboard.com, dan.scally@ideasonboard.com, jacopo.mondi@ideasonboard.com, nicolas.dufresne@collabora.com, libcamera-devel@lists.libcamera.org Subject: [RFC PATCH 16/19] utils: tuning: rkisp2.py: Add tuning script for rkisp2 Date: Fri, 3 Jul 2026 21:25:22 +0900 Message-ID: <20260703122543.1991189-17-paul.elder@ideasonboard.com> X-Mailer: git-send-email 2.47.2 In-Reply-To: <20260703122543.1991189-1-paul.elder@ideasonboard.com> References: <20260703122543.1991189-1-paul.elder@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Add a tuning script for the rkisp2. It is based on the rkisp1 tuning script. Signed-off-by: Paul Elder --- lsc is not output currently as it needs a bit more investigation. --- utils/tuning/rkisp2.py | 42 ++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 42 insertions(+) create mode 100755 utils/tuning/rkisp2.py diff --git a/utils/tuning/rkisp2.py b/utils/tuning/rkisp2.py new file mode 100755 index 000000000000..13feff03e6cc --- /dev/null +++ b/utils/tuning/rkisp2.py @@ -0,0 +1,42 @@ +#!/usr/bin/env python3 +# SPDX-License-Identifier: GPL-2.0-or-later +# +# Copyright (C) 2026, Paul Elder +# Copyright (C) 2026, Ideas On Board +# +# Tuning script for rkisp2, based on rkisp1 + +import logging +import sys + +import coloredlogs +import libtuning as lt +from libtuning.generators import YamlOutput +from libtuning.modules.agc import AGCRkISP1 +from libtuning.modules.awb import AWBRkISP1 +from libtuning.modules.ccm import CCMRkISP1 +from libtuning.modules.lsc import LSCRkISP1 +from libtuning.modules.lux import LuxRkISP1 +from libtuning.modules.static import StaticModule +from libtuning.parsers import YamlParser + +coloredlogs.install(level=logging.INFO, fmt='%(name)s %(levelname)s %(message)s') + +csm = StaticModule('ColorSpaceConversion') +agc = AGCRkISP1(debug=[lt.Debug.Plot]) +awb = AWBRkISP1(debug=[lt.Debug.Plot]) +bls = StaticModule('BlackLevelSubtraction') +ccm = CCMRkISP1(debug=[lt.Debug.Plot]) +goc = StaticModule('GammaOutCorrection', {'gamma': 2.2}) + +tuner = lt.Tuner('RkISP2') +tuner.add([csm, agc, awb, bls, ccm, goc]) +tuner.set_input_parser(YamlParser()) +tuner.set_output_formatter(YamlOutput()) + +# Bayesian AWB uses the lux value, so insert the lux algorithm before AWB. +# Compress is parameterized by others, so add it at the end. +tuner.set_output_order([csm, agc, awb, bls, ccm, goc]) + +if __name__ == '__main__': + sys.exit(tuner.run(sys.argv)) From patchwork Fri Jul 3 12:25:23 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Paul Elder X-Patchwork-Id: 27176 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id E3D8EC328C for ; Fri, 3 Jul 2026 12:27:01 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 8AEB865FF9; Fri, 3 Jul 2026 14:27:01 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="cniLnRR1"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 9B68265FC2 for ; Fri, 3 Jul 2026 14:27:00 +0200 (CEST) Received: from neptunite.hamster-moth.ts.net (unknown [IPv6:2404:7a81:160:2100:a2cc:2f45:3bd7:2589]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id BB9CD8E0; Fri, 3 Jul 2026 14:26:11 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1783081574; bh=uaCh4rYhCX9VLJFsHR4070jxBfSjedftOaB8zXo7w+U=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=cniLnRR1XANvk5hgIdfcpOTUnIpmagT5uqFxxm47X28828pQ8wTLIVU6GI+3GfqGB o4HZybr/EbM3IPuXKNHUZOTfh+sMwctL1lc8LosI2iBOtLo5Cworrf/8/69WhrweHQ sT4vNG91X0iqgsQ92CF2YSWaHaOecX1YcS/GeTDg= From: Paul Elder To: laurent.pinchart@ideasonboard.com Cc: Paul Elder , michael.riesch@collabora.com, xuhf@rock-chips.com, stefan.klug@ideasonboard.com, kieran.bingham@ideasonboard.com, dan.scally@ideasonboard.com, jacopo.mondi@ideasonboard.com, nicolas.dufresne@collabora.com, libcamera-devel@lists.libcamera.org Subject: [RFC PATCH 17/19] ipa: rkisp2: data: Add uncalibrated tuning file Date: Fri, 3 Jul 2026 21:25:23 +0900 Message-ID: <20260703122543.1991189-18-paul.elder@ideasonboard.com> X-Mailer: git-send-email 2.47.2 In-Reply-To: <20260703122543.1991189-1-paul.elder@ideasonboard.com> References: <20260703122543.1991189-1-paul.elder@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Add uncalibrated tuning file for rkisp2 that just enables black level subtraction. Signed-off-by: Paul Elder --- src/ipa/rkisp2/data/meson.build | 10 ++++++++++ src/ipa/rkisp2/data/uncalibrated.yaml | 7 +++++++ src/ipa/rkisp2/meson.build | 1 + 3 files changed, 18 insertions(+) create mode 100644 src/ipa/rkisp2/data/meson.build create mode 100644 src/ipa/rkisp2/data/uncalibrated.yaml diff --git a/src/ipa/rkisp2/data/meson.build b/src/ipa/rkisp2/data/meson.build new file mode 100644 index 000000000000..1ecd8b20ec90 --- /dev/null +++ b/src/ipa/rkisp2/data/meson.build @@ -0,0 +1,10 @@ +# SPDX-License-Identifier: CC0-1.0 + +conf_files = files([ + 'uncalibrated.yaml', +]) + +install_data(conf_files, + install_dir : ipa_data_dir / 'rkisp2', + install_tag : 'runtime') + diff --git a/src/ipa/rkisp2/data/uncalibrated.yaml b/src/ipa/rkisp2/data/uncalibrated.yaml new file mode 100644 index 000000000000..6d60bf1e6e98 --- /dev/null +++ b/src/ipa/rkisp2/data/uncalibrated.yaml @@ -0,0 +1,7 @@ +# SPDX-License-Identifier: CC0-1.0 +%YAML 1.1 +--- +version: 1 +algorithms: + - BlackLevelSubtraction: +... diff --git a/src/ipa/rkisp2/meson.build b/src/ipa/rkisp2/meson.build index f2f435c2d66f..db6b06aeec25 100644 --- a/src/ipa/rkisp2/meson.build +++ b/src/ipa/rkisp2/meson.build @@ -1,6 +1,7 @@ # SPDX-License-Identifier: CC0-1.0 subdir('algorithms') +subdir('data') ipa_name = 'ipa_rkisp2' From patchwork Fri Jul 3 12:25:24 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Paul Elder X-Patchwork-Id: 27177 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id E68E7C328C for ; Fri, 3 Jul 2026 12:27:05 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 98DA865FF9; Fri, 3 Jul 2026 14:27:05 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="noXwvgwV"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 33AEF65FED for ; Fri, 3 Jul 2026 14:27:04 +0200 (CEST) Received: from neptunite.hamster-moth.ts.net (unknown [IPv6:2404:7a81:160:2100:a2cc:2f45:3bd7:2589]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 4B4A91552; Fri, 3 Jul 2026 14:26:15 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1783081578; bh=JM1lHcZkSCQpwTs0aLSSUnhlfSc1lwA8/pkT1um6OG4=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=noXwvgwVum8mudSrpMAyDQ9+W030eoYw9KO4Wok0Pi2p2e1w++NekRdYtSm13EcgO o0sfCGLsUBTvFU+phv+gar2qUF10SHLzyIilh/SZWDGwP98eRYceHE/El2NPrqZuV8 sBbL4YyGyE0P1uHnDB3HghBeHr0wTw5N/lKDRN10= From: Paul Elder To: laurent.pinchart@ideasonboard.com Cc: Paul Elder , michael.riesch@collabora.com, xuhf@rock-chips.com, stefan.klug@ideasonboard.com, kieran.bingham@ideasonboard.com, dan.scally@ideasonboard.com, jacopo.mondi@ideasonboard.com, nicolas.dufresne@collabora.com, libcamera-devel@lists.libcamera.org Subject: [RFC PATCH 18/19] ipa: rkisp2: data: Add rudimentary tuning file for imx219 Date: Fri, 3 Jul 2026 21:25:24 +0900 Message-ID: <20260703122543.1991189-19-paul.elder@ideasonboard.com> X-Mailer: git-send-email 2.47.2 In-Reply-To: <20260703122543.1991189-1-paul.elder@ideasonboard.com> References: <20260703122543.1991189-1-paul.elder@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Add a tuning file for rkisp2 for imx219. The ccms come from rpi, the lsc tables are just all 1.0 for now, and the agc parameters are copied from the output of the rkisp1 tuning script. Signed-off-by: Paul Elder --- src/ipa/rkisp2/data/imx219.yaml | 139 ++++++++++++++++++++++++++++++++ src/ipa/rkisp2/data/meson.build | 1 + 2 files changed, 140 insertions(+) create mode 100644 src/ipa/rkisp2/data/imx219.yaml diff --git a/src/ipa/rkisp2/data/imx219.yaml b/src/ipa/rkisp2/data/imx219.yaml new file mode 100644 index 000000000000..eef413a490cb --- /dev/null +++ b/src/ipa/rkisp2/data/imx219.yaml @@ -0,0 +1,139 @@ +# SPDX-License-Identifier: CC0-1.0 +%YAML 1.1 +--- +version: 1 +algorithms: + - LensShadingCorrection: + x-size: [ 0.0625, 0.0625, 0.0625, 0.0625, 0.0625, 0.0625, 0.0625, 0.0625, + 0.0625, 0.0625, 0.0625, 0.0625, 0.0625, 0.0625, 0.0625, 0.0625 ] + y-size: [ 0.0625, 0.0625, 0.0625, 0.0625, 0.0625, 0.0625, 0.0625, 0.0625, + 0.0625, 0.0625, 0.0625, 0.0625, 0.0625, 0.0625, 0.0625, 0.0625 ] + sets: + - ct: 5800 + r: [ + 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, + 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, + 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, + 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, + 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, + 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, + 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, + 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, + 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, + 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, + 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, + 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, + 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, + 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, + 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, + 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, + 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, + ] + gr: [ + 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, + 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, + 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, + 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, + 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, + 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, + 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, + 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, + 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, + 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, + 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, + 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, + 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, + 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, + 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, + 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, + 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, + ] + gb: [ + 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, + 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, + 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, + 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, + 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, + 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, + 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, + 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, + 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, + 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, + 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, + 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, + 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, + 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, + 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, + 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, + 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, + ] + b: [ + 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, + 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, + 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, + 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, + 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, + 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, + 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, + 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, + 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, + 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, + 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, + 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, + 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, + 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, + 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, + 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, + 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, + ] + - Agc: + AeMeteringMode: + MeteringCentreWeighted: [0, 0, 0, 0, 0, 0, 6, 8, 6, 0, 0, 8, 16, 8, 0, 0, 6, 8, 6, 0, 0, 0, 0, 0, 0] + MeteringSpot: [0, 0, 0, 0, 0, 0, 2, 4, 2, 0, 0, 4, 16, 4, 0, 0, 2, 4, 2, 0, 0, 0, 0, 0, 0] + MeteringMatrix: [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1] + AeExposureMode: + ExposureNormal: + exposureTime: [ 100, 10000, 30000, 60000, 120000 ] + gain: [ 1.0, 2.0, 4.0, 6.0, 6.0 ] + ExposureShort: + exposureTime: [ 100, 5000, 10000, 20000, 120000 ] + gain: [ 1.0, 2.0, 4.0, 6.0, 6.0 ] + AeConstraintMode: + ConstraintNormal: + lower: + qLo: 0.98 + qHi: 1.0 + yTarget: [0, 0.65, 1000, 0.65] + ConstraintHighlight: + lower: + qLo: 0.98 + qHi: 1.0 + yTarget: [0, 0.5, 1000, 0.5] + upper: + qLo: 0.98 + qHi: 1.0 + yTarget: [0, 0.8, 1000, 0.5] + relativeLuminanceTarget: 0.5 + - Awb: + - BlackLevelSubtraction: + R: 4096 + Gr: 4096 + Gb: 4096 + B: 4096 + - Ccm: + ccms: + - ct: 2860 + ccm: [ 2.12089, -0.52461, -0.59629, -0.85342, 2.80445, -0.95103, -0.26897, -1.14788, 2.41685 ] + - ct: 2960 + ccm: [ 2.26962, -0.54174, -0.72789, -0.77008, 2.60271, -0.83262, -0.26036, -1.51254, 2.77289 ] + - ct: 3603 + ccm: [ 2.18644, -0.66148, -0.52496, -0.77828, 2.69474, -0.91645, -0.25239, -0.83059, 2.08298 ] + - ct: 4650 + ccm: [ 2.18174, -0.70887, -0.47287, -0.70196, 2.76426, -1.06231, -0.25157, -0.71978, 1.97135 ] + - ct: 5858 + ccm: [ 2.32392, -0.88421, -0.43971, -0.63821, 2.58348, -0.94527, -0.28541, -0.54112, 1.82653 ] + - ct: 7580 + ccm: [ 2.21175, -0.53242, -0.67933, -0.57875, 3.07922, -1.50047, -0.27709, -0.73338, 2.01048 ] + - GammaOutCorrection: + - ColorSpaceConversion: +... diff --git a/src/ipa/rkisp2/data/meson.build b/src/ipa/rkisp2/data/meson.build index 1ecd8b20ec90..6b21718e408f 100644 --- a/src/ipa/rkisp2/data/meson.build +++ b/src/ipa/rkisp2/data/meson.build @@ -1,6 +1,7 @@ # SPDX-License-Identifier: CC0-1.0 conf_files = files([ + 'imx219.yaml', 'uncalibrated.yaml', ]) From patchwork Fri Jul 3 12:25:25 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Paul Elder X-Patchwork-Id: 27178 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 7E53BC328C for ; Fri, 3 Jul 2026 12:27:09 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 2A97565FFF; Fri, 3 Jul 2026 14:27:09 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="pK07OReq"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id B1FE465FED for ; Fri, 3 Jul 2026 14:27:07 +0200 (CEST) Received: from neptunite.hamster-moth.ts.net (unknown [IPv6:2404:7a81:160:2100:a2cc:2f45:3bd7:2589]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id D54921121; Fri, 3 Jul 2026 14:26:18 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1783081581; bh=jE97IpXxHWdvScNkVYbhDJHNEYa8nGsdBx/m4Ph7sg8=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=pK07OReqHfXwbmJbG7xeVah8IUNINBqKe9JQnVCb5yzCprgO77LwOGqh1yU6g8SAs PSdBweTd2TagYsmId1U82iWonCgVdsAQ3G5cz8puuM5B+Ylf6zI9HzcLlK3IVzaG6G JfkE3q34oMyyXhu6HeC+8paB50ukQIBH/lwWXIG8= From: Paul Elder To: laurent.pinchart@ideasonboard.com Cc: Paul Elder , michael.riesch@collabora.com, xuhf@rock-chips.com, stefan.klug@ideasonboard.com, kieran.bingham@ideasonboard.com, dan.scally@ideasonboard.com, jacopo.mondi@ideasonboard.com, nicolas.dufresne@collabora.com, libcamera-devel@lists.libcamera.org Subject: [RFC PATCH 19/19] ipa: rkisp2: data: Add tuning file for imx708 Date: Fri, 3 Jul 2026 21:25:25 +0900 Message-ID: <20260703122543.1991189-20-paul.elder@ideasonboard.com> X-Mailer: git-send-email 2.47.2 In-Reply-To: <20260703122543.1991189-1-paul.elder@ideasonboard.com> References: <20260703122543.1991189-1-paul.elder@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Add a tuning file for rkisp2 for imx708. This was tuned using the rkisp2 tuning script using images from [0] (at branch raspberrypi/imx708 in directory 2026-05-22), with some hand-modifications, such as removing lsc because it needs further investigation to function properly. [0] https://gitlab.freedesktop.org/camera/tuning Signed-off-by: Paul Elder --- src/ipa/rkisp2/data/imx708.yaml | 154 ++++++++++++++++++++++++++++++++ src/ipa/rkisp2/data/meson.build | 1 + 2 files changed, 155 insertions(+) create mode 100644 src/ipa/rkisp2/data/imx708.yaml diff --git a/src/ipa/rkisp2/data/imx708.yaml b/src/ipa/rkisp2/data/imx708.yaml new file mode 100644 index 000000000000..d15367ba97cc --- /dev/null +++ b/src/ipa/rkisp2/data/imx708.yaml @@ -0,0 +1,154 @@ +# SPDX-License-Identifier: CC0-1.0 +%YAML 1.1 +--- +version: 1 +algorithms: + - ColorSpaceConversion: + - Agc: + AeMeteringMode: + MeteringCentreWeighted: [ 0, 0, 0, 0, 0, 0, 6, 8, 6, 0, 0, 8, 16, 8, 0, 0, 6, 8, 6, 0, 0, 0, 0, 0, 0 ] + AeExposureMode: + ExposureNormal: + exposureTime: [ 100, 10000, 30000, 60000, 120000 ] + gain: [ 1.2, 2.0, 4.0, 6.0, 6.0 ] + AeConstraintMode: + ConstraintNormal: + lower: + qLo: 0.98 + qHi: 1.0 + yTarget: 0.1 + relativeLuminanceTarget: 0.1 + - Awb: + algorithm: "grey" + AwbMode: + AwbAuto: + lo: 2500 + hi: 9000 + AwbIncandescent: + lo: 2500 + hi: 3000 + AwbTungsten: + lo: 3000 + hi: 3500 + AwbFluorescent: + lo: 4000 + hi: 4700 + AwbIndoor: + lo: 3000 + hi: 5000 + AwbDaylight: + lo: 5500 + hi: 6500 + AwbCloudy: + lo: 6500 + hi: 8000 + priors: + - lux: 0 + ct: [ 2000, 13000 ] + probability: [ 1.0, 1.0 ] + colourGains: + - ct: 2500 + gains: [ 1.1199462425803561, 1.9142419601837672 ] + - ct: 3000 + gains: [ 1.1242270938729624, 1.9015021867275146 ] + - ct: 3500 + gains: [ 1.1583458820803894, 1.8109380659181455 ] + - ct: 4000 + gains: [ 1.2714558169103625, 1.6165535079211122 ] + - ct: 4500 + gains: [ 1.476886722788362, 1.4486455164421266 ] + - ct: 5000 + gains: [ 1.64446637066272, 1.3819789939192924 ] + - ct: 5500 + gains: [ 1.7822135091783995, 1.3487995683841383 ] + - ct: 6000 + gains: [ 1.8677624206200971, 1.33422281521014 ] + - ct: 6500 + gains: [ 1.9219680953296177, 1.3266118333775536 ] + - ct: 7000 + gains: [ 1.9516003122560501, 1.3229263130043656 ] + - ct: 7500 + gains: [ 1.9673421207948063, 1.321178491214163 ] + - ct: 8000 + gains: [ 1.9845207382417145, 1.3190871916633689 ] + - ct: 8500 + gains: [ 1.993620414673046, 1.3182177695755337 ] + - ct: 9000 + gains: [ 2.006823198876179, 1.3170025023047545 ] + transversePos: 0.01269 + transverseNeg: 0.01561 + - BlackLevelSubtraction: + - Ccm: + ccms: + - ct: 2500 + ccm: [ + 1.72201, 0.27067, -0.99269, -0.64829, 2.48777, -0.83947, -0.08905, -0.54176, + 1.63081, + ] + - ct: 3000 + ccm: [ + 1.80587, 0.27132, -1.07719, -0.67445, 2.55729, -0.88284, -0.11544, -0.50189, + 1.61734, + ] + - ct: 3500 + ccm: [ + 1.83389, 0.37559, -1.20949, -0.69373, 2.67018, -0.97646, -0.15308, -0.47581, + 1.62889, + ] + - ct: 4000 + ccm: [ + 1.86271, 0.39818, -1.26089, -0.66884, 2.62966, -0.96082, -0.19298, -0.45096, + 1.64394, + ] + - ct: 4500 + ccm: [ + 1.78213, 0.40302, -1.18514, -0.61892, 2.47338, -0.85445, -0.24296, -0.38901, + 1.63197, + ] + - ct: 5000 + ccm: [ + 1.75328, 0.39113, -1.14441, -0.60457, 2.40168, -0.79711, -0.28013, -0.33477, + 1.61489, + ] + - ct: 5500 + ccm: [ + 1.67989, 0.42725, -1.10714, -0.60576, 2.35811, -0.75235, -0.30859, -0.29885, + 1.60744, + ] + - ct: 6000 + ccm: [ + 1.62571, 0.45052, -1.07622, -0.60271, 2.32574, -0.72302, -0.31931, -0.27971, + 1.59902, + ] + - ct: 6500 + ccm: [ + 1.60018, 0.43641, -1.03658, -0.59726, 2.29742, -0.70016, -0.31271, -0.28088, + 1.59358, + ] + - ct: 7000 + ccm: [ + 1.59364, 0.41222, -1.00585, -0.58752, 2.27837, -0.69085, -0.30352, -0.28323, + 1.58675, + ] + - ct: 7500 + ccm: [ + 1.59057, 0.40006, -0.99063, -0.58525, 2.26603, -0.68078, -0.29994, -0.28105, + 1.58099, + ] + - ct: 8000 + ccm: [ + 1.59551, 0.36878, -0.96428, -0.57195, 2.24572, -0.67377, -0.28793, -0.28858, + 1.57651, + ] + - ct: 8500 + ccm: [ + 1.60576, 0.33929, -0.94504, -0.56235, 2.23411, -0.67176, -0.27817, -0.29265, + 1.57082, + ] + - ct: 9000 + ccm: [ + 1.60867, 0.30611, -0.91478, -0.54939, 2.21074, -0.66135, -0.26915, -0.30127, + 1.57042, + ] + - GammaOutCorrection: + gamma: 2.2 diff --git a/src/ipa/rkisp2/data/meson.build b/src/ipa/rkisp2/data/meson.build index 6b21718e408f..f4dc569db277 100644 --- a/src/ipa/rkisp2/data/meson.build +++ b/src/ipa/rkisp2/data/meson.build @@ -2,6 +2,7 @@ conf_files = files([ 'imx219.yaml', + 'imx708.yaml', 'uncalibrated.yaml', ])