From patchwork Thu Aug 27 10:49:54 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Paul Elder X-Patchwork-Id: 28121 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 4EE0DBDDFC for ; Thu, 27 Aug 2026 10:50:34 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id C04ED68460; Thu, 27 Aug 2026 12:50:33 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="MUl5NI+f"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 1BD7368454 for ; Thu, 27 Aug 2026 12:50:32 +0200 (CEST) Received: from neptunite.hamster-moth.ts.net (unknown [IPv6:2400:2411:160:2f00:306c:5b5b:3c7f:eabd]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 78CC2145C; Thu, 27 Aug 2026 12:49:02 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1787827745; bh=JmomUbSF76fMGyZp8e0/3EvIoj5oB0WSJOAJvQyxav0=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=MUl5NI+foGYIlx7xgnW4+bb2SMPeSV0M0BJ6uKynYRaK7Dk0UApcC/j61ce0oQzpx z23Ls3SQmzbumfOJGLyVwO+Aj0ooQJTxNEqqOmvDlQHUk1OarJHASEd6ZyvLSXPLrV M19wGYe6H83PYglaLTgMJIz+MOig91NnSbjPHEBg= From: Paul Elder To: laurent.pinchart@ideasonboard.com Cc: Paul Elder , michael.riesch@collabora.com, xuhf@rock-chips.com, stefan.klug@ideasonboard.com, kieran.bingham@ideasonboard.com, dan.scally@ideasonboard.com, jacopo.mondi@ideasonboard.com, nicolas.dufresne@collabora.com, mehdi.djait@linux.intel.com, libcamera-devel@lists.libcamera.org Subject: [PATCH v3 01/20] utils: update-kernel-headers: Add rkisp2-config.h Date: Thu, 27 Aug 2026 19:49:54 +0900 Message-ID: <20260827105018.2781166-2-paul.elder@ideasonboard.com> X-Mailer: git-send-email 2.47.2 In-Reply-To: <20260827105018.2781166-1-paul.elder@ideasonboard.com> References: <20260827105018.2781166-1-paul.elder@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Add rkisp2-config.h to the list of kernel headers to update, as we will soon be adding support for the rkisp2 pipeline. Signed-off-by: Paul Elder --- Changes in v3: - pull from the new directory No change in v2 --- utils/update-kernel-headers.sh | 1 + 1 file changed, 1 insertion(+) diff --git a/utils/update-kernel-headers.sh b/utils/update-kernel-headers.sh index 3fdea5f1eb5d..7b6f2373613a 100755 --- a/utils/update-kernel-headers.sh +++ b/utils/update-kernel-headers.sh @@ -56,6 +56,7 @@ headers=" linux/media-bus-format.h linux/media.h linux/media/arm/mali-c55-config.h + linux/media/rockchip/rkisp2-config.h linux/media/v4l2-isp.h linux/rkisp1-config.h linux/stddef.h From patchwork Thu Aug 27 10:49:55 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Paul Elder X-Patchwork-Id: 28122 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id E4BC4C333C for ; Thu, 27 Aug 2026 10:50:36 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 82C2868465; Thu, 27 Aug 2026 12:50:36 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="fqFrO0gX"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id BA4F368454 for ; Thu, 27 Aug 2026 12:50:35 +0200 (CEST) Received: from neptunite.hamster-moth.ts.net (unknown [IPv6:2400:2411:160:2f00:306c:5b5b:3c7f:eabd]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 2B77BC1; Thu, 27 Aug 2026 12:49:05 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1787827749; bh=+CULSgEAilr9G6TOYQ7L0AAvsjwTniG0nus2T6mzU1M=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=fqFrO0gXP9SrBSUfHAEsrTsqMVdhZYFfBhuTJBlDcfWPdianBOcHdskiq/9cF4Rfv Lv1VisGYKlt/Am9kcw8A7ONagheMJ9oTIz25Gg+M9BLoCiGrVsOWxgj5vqXYkQLEux icH/oDL8ylBFPLgGx2XftwHfl/cgz9rWdo4paUFk= From: Paul Elder To: laurent.pinchart@ideasonboard.com Cc: Paul Elder , michael.riesch@collabora.com, xuhf@rock-chips.com, stefan.klug@ideasonboard.com, kieran.bingham@ideasonboard.com, dan.scally@ideasonboard.com, jacopo.mondi@ideasonboard.com, nicolas.dufresne@collabora.com, mehdi.djait@linux.intel.com, libcamera-devel@lists.libcamera.org Subject: [PATCH v3 02/20] include: linux: rkisp2-config.h: update to v7.2-rc1 Date: Thu, 27 Aug 2026 19:49:55 +0900 Message-ID: <20260827105018.2781166-3-paul.elder@ideasonboard.com> X-Mailer: git-send-email 2.47.2 In-Reply-To: <20260827105018.2781166-1-paul.elder@ideasonboard.com> References: <20260827105018.2781166-1-paul.elder@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Update rkisp2-config.h to the most recent version. Signed-off-by: Paul Elder --- This comes from v3 of "media: rockchip: rkisp2: Add rkisp2 driver": https://lore.kernel.org/linux-media/20260827-rkisp2-v3-0-e91ff0a604d6@ideasonboard.com/T/#t Changes in v3: - move the header file location to the new one Changes in v2: - update to the headers from v2 of the kernel patches (obviously...) - use extensible stats instead of non-extensible stats - add crop parameter --- include/linux/README | 2 +- include/linux/media/rockchip/rkisp2-config.h | 640 +++++++++++++++++++ include/linux/videodev2.h | 4 + 3 files changed, 645 insertions(+), 1 deletion(-) create mode 100644 include/linux/media/rockchip/rkisp2-config.h diff --git a/include/linux/README b/include/linux/README index 9303e0f14ae6..0f61052c0450 100644 --- a/include/linux/README +++ b/include/linux/README @@ -1,4 +1,4 @@ # SPDX-License-Identifier: CC0-1.0 -Files in this directory are imported from v7.1 of the Linux kernel. Do not +Files in this directory are imported from v7.2-rc1-324-gd4c801b3cf6d of the Linux kernel. Do not modify them manually. diff --git a/include/linux/media/rockchip/rkisp2-config.h b/include/linux/media/rockchip/rkisp2-config.h new file mode 100644 index 000000000000..b17267c001c9 --- /dev/null +++ b/include/linux/media/rockchip/rkisp2-config.h @@ -0,0 +1,640 @@ +/* SPDX-License-Identifier: ((GPL-2.0-or-later WITH Linux-syscall-note) OR MIT) */ +/* + * Rockchip ISP2 userspace API + * Copyright (C) 2017 Rockchip Electronics Co., Ltd. + * Copyright (C) 2026 Ideas on Board Oy. + */ + +#ifndef _RKISP2_CONFIG_H +#define _RKISP2_CONFIG_H + +#include + +#include + +#define RKISP2_ISP_GAMMA_OUT_MAX_SEGMENTS 49 + +#define RKISP2_ISP_LSC_SAMPLES_MAX 17 +#define RKISP2_ISP_LSC_SECTORS_TBL_SIZE_MAX 16 + +#define RKISP2_ISP_AE_MEAN_MAX_LITE 25 +#define RKISP2_ISP_AE_MEAN_MAX_BIG 225 + +#define RKISP2_ISP_HIST_WEIGHT_GRIDS_SIZE_LITE 25 +#define RKISP2_ISP_HIST_WEIGHT_GRIDS_SIZE_BIG 225 +#define RKISP2_ISP_HIST_WEIGHT_GRIDS_SIZE_MAX RKISP2_ISP_HIST_WEIGHT_GRIDS_SIZE_BIG + +#define RKISP2_ISP_HIST_BIN_N_MAX 256 + +#define RKISP2_ISP_AWB_COUNTS_SIZE 225 + +/** + * enum rkisp2_isp_version - ISP variants + * + * @RKISP3_V0: Used at least in RK3588 + */ +enum rkisp2_isp_version { + RKISP3_V0 = 30, +}; + +/* See enum rkisp2_isp_goc_segments for logarithmic segment sizes */ +enum rkisp2_isp_goc_mode { + RKISP2_ISP_GOC_MODE_LOGARITHMIC, + RKISP2_ISP_GOC_MODE_EQUIDISTANT +}; + +/* + * The segments are: + * 1 x8, 2 x4, 4 x4, 8 x4, 16 x4, 32 x4, 64 x4, 128 x4, 256 x4, 512 x4 + * In 48-segment mode, the last group of 512 x4 becomes 256 x8 + */ +enum rkisp2_isp_goc_segments { + RKISP2_ISP_GOC_SEGMENTS_44, + RKISP2_ISP_GOC_SEGMENTS_48 +}; + +enum rkisp2_isp_lsc_config { + RKISP2_ISP_LSC_CONFIG_8X8, + RKISP2_ISP_LSC_CONFIG_16X16 +}; + +enum rkisp2_isp_set_active_table_when { + RKISP2_ISP_LSC_SET_ACTIVE_TABLE_AFTER, + RKISP2_ISP_LSC_SET_ACTIVE_TABLE_BEFORE, +}; + +enum rkisp2_isp_crop_en { + RKISP2_ISP_CROP_ENABLE_MAIN = 0x1, + RKISP2_ISP_CROP_ENABLE_SELF = 0x2, +}; + +enum rkisp2_isp_histogram_mode { + RKISP2_ISP_HISTOGRAM_MODE_DISABLE, + RKISP2_ISP_HISTOGRAM_MODE_R_HISTOGRAM = 2, + RKISP2_ISP_HISTOGRAM_MODE_G_HISTOGRAM, + RKISP2_ISP_HISTOGRAM_MODE_B_HISTOGRAM, + RKISP2_ISP_HISTOGRAM_MODE_Y_HISTOGRAM +}; + +/* + * This selects which bits are used from the input data to compute the + * histogram + */ +enum rkisp2_isp_histogram_data_sel { + RKISP2_ISP_HISTOGRAM_DATA_SEL_11_4, + RKISP2_ISP_HISTOGRAM_DATA_SEL_10_3, + RKISP2_ISP_HISTOGRAM_DATA_SEL_9_2, + RKISP2_ISP_HISTOGRAM_DATA_SEL_8_1, + RKISP2_ISP_HISTOGRAM_DATA_SEL_7_0, +}; + +/*---------- Statistics ------------*/ + +/** + * enum rkisp2_stats_block_type - RkISP2 extensible stats block types + * + * @RKISP2_STATS_BLOCK_AE_LITE: Auto exposite (lite) measurements + * @RKISP2_STATS_BLOCK_HIST_LITE: Histogram (lite) + * @RKISP2_STATS_BLOCK_HIST_BIG0: Histogram (big 0) + * @RKISP2_STATS_BLOCK_HIST_BIG1: Histogram (big 1) + * @RKISP2_STATS_BLOCK_HIST_BIG2: Histogram (big 2) + * @RKISP2_STATS_BLOCK_AWB: Auto white balance counts + */ +enum rkisp2_stats_block_type { + RKISP2_STATS_BLOCK_AE_LITE, + RKISP2_STATS_BLOCK_HIST_LITE, + RKISP2_STATS_BLOCK_HIST_BIG0, + RKISP2_STATS_BLOCK_HIST_BIG1, + RKISP2_STATS_BLOCK_HIST_BIG2, + RKISP2_STATS_BLOCK_AWB, +}; + +/** + * struct rkisp2_stats_ae_lite - statistics auto exposure data + * + * @header: block header (type = RKISP2_STATS_BLOCK_AE_LITE) + * @exp_mean_r: Mean luminance value of block xy for r channel + * @exp_mean_g: Mean luminance value of block xy for g channel + * @exp_mean_b: Mean luminance value of block xy for b channel + * @done: This set to nonzero when the stats are ready + * @reserved: padding + * + * Image is divided into 5x5 blocks on lite and 15x15 blocks on big. + */ +struct rkisp2_stats_ae_lite { + struct v4l2_isp_block_header header; + __u16 exp_mean_r[RKISP2_ISP_AE_MEAN_MAX_LITE]; + __u16 exp_mean_g[RKISP2_ISP_AE_MEAN_MAX_LITE]; + __u16 exp_mean_b[RKISP2_ISP_AE_MEAN_MAX_LITE]; + __u8 done; + __u8 reserved; +}; + +/** + * struct rkisp2_stats_hist - statistics histogram data + * + * @header: block header (type = RKISP2_STATS_BLOCK_HIST_{LITE,BIG0,BIG1,BIG2}) + * @hist_bins: measured bin counters. Each bin is a 28 bits unsigned fixed point + * type. Bits 0-4 are the fractional part and bits 5-27 are the + * integer part. + * @done: This set to nonzero when the stats are ready + * @reserved: padding + * + * There are 256 bins, at least on 3.x. + */ +struct rkisp2_stats_hist { + struct v4l2_isp_block_header header; + __u32 hist_bins[RKISP2_ISP_HIST_BIN_N_MAX]; + __u8 done; + __u8 reserved[7]; +}; + +/** + * struct rkisp2_stats_awb - statistics auto white balance data + * + * @header: block header (type = RKISP2_STATS_BLOCK_AWB) + * @counts_r: Counts of red (18-bits) + * @counts_g: Counts of green (18-bits) + * @counts_b: Counts of blue (18-bits) + * @counts_w: Counts of white point (10-bits) + * @done: This set to nonzero when the stats are ready + * @reserved: padding + * + * TODO Figure out what is being counted + */ +struct rkisp2_stats_awb { + struct v4l2_isp_block_header header; + __u32 counts_r[RKISP2_ISP_AWB_COUNTS_SIZE]; + __u32 counts_g[RKISP2_ISP_AWB_COUNTS_SIZE]; + __u32 counts_b[RKISP2_ISP_AWB_COUNTS_SIZE]; + __u16 counts_w[RKISP2_ISP_AWB_COUNTS_SIZE]; + __u8 done; + __u8 reserved; +}; + +#define RKISP2_STATS_MAX_SIZE \ + (sizeof(struct rkisp2_stats_ae_lite) +\ + sizeof(struct rkisp2_stats_hist) * 4 +\ + sizeof(struct rkisp2_stats_awb)) + +/*---------- Parameters ------------*/ + +/** + * enum rkisp2_params_block_type - RkISP2 extensible params block types + * + * @RKISP2_PARAMS_BLOCK_BLS: Black level subtraction + * @RKISP2_PARAMS_BLOCK_AWB_GAINS: AWB gains + * @RKISP2_PARAMS_BLOCK_CSM: Color conversion coefficients (in the ISP block) + * @RKISP2_PARAMS_BLOCK_CCM: Color correction matrix (in the CCM block) + * @RKISP2_PARAMS_BLOCK_GOC: Gamma out correction + * @RKISP2_PARAMS_BLOCK_LSC: Lens shading correction + * @RKISP2_PARAMS_BLOCK_CROP: Crop config + * @RKISP2_PARAMS_BLOCK_AE_LITE: AE measurement config (lite) + * @RKISP2_PARAMS_BLOCK_HIST_LITE: Histogram measurement config (lite) + * @RKISP2_PARAMS_BLOCK_HIST_BIG0: Histogram measurement config (zeroth big block) + * @RKISP2_PARAMS_BLOCK_HIST_BIG1: Histogram measurement config (first big block) + * @RKISP2_PARAMS_BLOCK_HIST_BIG2: Histogram measurement config (second big block) + * @RKISP2_PARAMS_BLOCK_AWB_MEAS: AWB measurements config + */ +enum rkisp2_params_block_type { + RKISP2_PARAMS_BLOCK_BLS, + RKISP2_PARAMS_BLOCK_AWB_GAINS, + RKISP2_PARAMS_BLOCK_CSM, + RKISP2_PARAMS_BLOCK_CCM, + RKISP2_PARAMS_BLOCK_GOC, + RKISP2_PARAMS_BLOCK_LSC, + RKISP2_PARAMS_BLOCK_CROP, + RKISP2_PARAMS_BLOCK_AE_LITE, + RKISP2_PARAMS_BLOCK_HIST_LITE, + RKISP2_PARAMS_BLOCK_HIST_BIG0, + RKISP2_PARAMS_BLOCK_HIST_BIG1, + RKISP2_PARAMS_BLOCK_HIST_BIG2, + RKISP2_PARAMS_BLOCK_AWB_MEAS, +}; + +/** + * struct rkisp2_isp_window - measurement window. + * + * Measurements are calculated per window inside the frame. + * This struct represents a window for a measurement. + * + * @h_offs: the horizontal offset of the window from the left of the frame in pixels. + * @v_offs: the vertical offset of the window from the top of the frame in pixels. + * @h_size: the horizontal size of the window in pixels + * @v_size: the vertical size of the window in pixels. + */ +struct rkisp2_isp_window { + __u16 h_offs; + __u16 v_offs; + __u16 h_size; + __u16 v_size; +}; + +/** + * struct rkisp2_isp_bls_fixed_val - BLS fixed subtraction values + * + * These are signed 13-bit (-4096 to +4095). + * + * @a: Fixed black level value for Bayer channel 0 + * @b: Fixed black level value for Bayer channel 1 + * @c: Fixed black level value for Bayer channel 2 + * @d: Fixed black level value for Bayer channel 3 + */ +struct rkisp2_isp_bls_fixed_val { + __s16 a; + __s16 b; + __s16 c; + __s16 d; +}; + +/** + * struct rkisp2_isp_awb_gains - Auto white balance gain in the ISP block + * + * All fields in this struct are 16 bit, where: + * 0x100h = 1, unsigned integer value, range 0 to 63 with 8 bit fractional part. + * + * This leaves the upper two msb unaccounted for; it is unknown if these are + * unused or misdocumented. + * + * TODO investigate the upper two bits + * + * @r: gain value for red component. + * @gr: gain value for green component in red line. + * @b: gain value for blue component. + * @gb: gain value for green component in blue line. + */ +struct rkisp2_isp_awb_gains { + __u16 r; + __u16 gr; + __u16 b; + __u16 gb; +}; + +/** + * struct rkisp2_isp_color_cc - Color coefficients + * + * @r: Red coefficient + * @g: Green coefficient + * @b: Blue coefficient + */ +struct rkisp2_isp_color_cc { + __u8 r; + __u8 g; + __u8 b; +}; + +/** + * struct rkisp2_isp_awb_color_quad - Group of RGB and luminance for AWB + * + * TODO redesign this? + * + * @r: Red + * @g: Green + * @b: Blue + * @y: Y (luminance) + */ +struct rkisp2_isp_awb_color_quad { + __u8 r; + __u8 g; + __u8 b; + __u8 y; +}; + +/** + * struct rkisp2_params_bls - RkISP2 params BLS config + * + * RkISP2 parameters Black Level Subtraction configuration block. + * Identified by :c:type:`RKISP2_PARAMS_BLOCK_BLS`. + * + * TODO Check if auto-mode and window selection is for both blocks or just for + * one block (it might be the same as 2.x) + * + * @header: The RkISP2 parameters block header + * @enable_auto: Automatic mode activated means that the measured values + * are subtracted. Otherwise the fixed subtraction + * values will be subtracted. + * @enabled_windows: enabled window (bit 0 for window 1, bit 1 for window 2) + * @bls_window1: Measurement window 1 size + * @bls_window2: Measurement window 2 size + * @bls_samples: Set amount of measured pixels for each Bayer position + * (A, B, C and D) to 2^bls_samples. (TODO needs confirmation) + * @reserved: padding + * @bls_fixed_val: Black Level Subtraction fixed values for the BLS module at + * the front of the pipeline + * @bls1_fixed_val: Black Level Subtraction fixed values for the BLS module after + * bayer noise reduction + * @reserved1: padding + */ +struct rkisp2_params_bls { + struct v4l2_isp_params_block_header header; + __u8 enable_auto; + __u8 enabled_windows; + struct rkisp2_isp_window bls_window1; + struct rkisp2_isp_window bls_window2; + __u8 bls_samples; + __u8 reserved; + struct rkisp2_isp_bls_fixed_val bls_fixed_val; + struct rkisp2_isp_bls_fixed_val bls1_fixed_val; + __u8 reserved1[4]; +} __attribute__((aligned(8))); + +/** + * struct rkisp2_params_awb_gains - RKISP2 params AWB gains config + * + * RkISP2 parameters auto white balance gains configuration block. + * Identified by :c:type:`RKISP2_PARAMS_BLOCK_AWB_GAINS`. + * + * TODO investigate what the different blocks mean + * + * Block 0 is equivalent to the awb gains block on 2.x, but blocks 1 and + * 2 do not exist on 2.x. + * + * @header: The RkISP2 parameters block header + * @gains: Gains configuration for block i + */ +struct rkisp2_params_awb_gains { + struct v4l2_isp_params_block_header header; + struct rkisp2_isp_awb_gains gains[3]; +} __attribute__((aligned(8))); + +/** + * struct rkisp2_params_csm - Configuration used by Color Space Conversion + * + * RkISP2 parameters histogram configuration block. + * Identified by :c:type:`RKISP2_PARAMS_BLOCK_CSM`. + * + * @header: The RkISP2 parameters block header + * @coeff: color correction matrix. Values are 9-bit signed fixed-point numbers with 2 bit integer + * and 7 bit fractional part, ranging from -2 (0x100) to +1.992 (0x0FF). 0 is + * represented by 0x000 and a coefficient value of 1 as 0x080. + * @reserved: padding + */ +struct rkisp2_params_csm { + struct v4l2_isp_params_block_header header; + __u16 coeff[3][3]; + __u8 reserved[6]; +} __attribute__((aligned(8))); + +/** + * struct rkisp2_params_ccm - Configuration used by Color Correction Matrix + * + * RkISP2 parameters histogram configuration block. + * Identified by :c:type:`RKISP2_PARAMS_BLOCK_CCM`. + * + * @header: The RkISP2 parameters block header + * @high_y_alpha_adj_en: Enable CCM high Y alpha adjustment (TODO figure out what this does) + * @reserved: padding + * @coeff: color correction matrix. Values are 11-bit signed fixed-point numbers with 4 bit integer + * and 7 bit fractional part, ranging from -8 (0x400) to +7.992 (0x3FF). 0 is + * represented by 0x000 and a coefficient value of 1 as 0x080. The + * value is expanded 128 times (TODO figure out what this means). + * @offset: Red, Green, Blue offsets for the color correction matrix. 12-bits + * wide ranging from -4096 to 4095, but only for red; green and blue are 11-bit + * signed fixed-point like coeff, but are still 12-bits wide. + * @y_coeff: Red, Green, Blue coefficients for RGB2Y calculation. red and green + * are 11-bits wide and blue is 12-bits wide. The value is expanded 128 times. + * @alp: CCM curve y-axis point definition for ccm input pixel's luminance. + * 11-bit unsigned ranging from 0 to 1024. The value is expanded 128 times. + * @inflection_point: Inflection point of the ccm alpha interpolation curve. + * The inflection point is 2^inflection_point. Since the maximum y-value is + * 1024, the maximum value of this field is expected to be 10 (0xa), but the + * documentation says 4'b10. + * @reserved1: padding + */ +struct rkisp2_params_ccm { + struct v4l2_isp_params_block_header header; + __u8 high_y_alpha_adj_en; + __u8 reserved; + __u16 coeff[3][3]; + __u16 offset[3]; + __u16 y_coeff[3]; + __u16 alp[17]; + __u8 inflection_point; + __u8 reserved1[5]; +} __attribute__((aligned(8))); + +/** + * struct rkisp2_params_goc - Configuration used by Gamma Out correction + * + * RkISP2 parameters gamma out correction configuration block. + * Identified by :c:type:`RKISP2_PARAMS_BLOCK_GOC`. + * + * @header: The RkISP2 parameters block header + * @mode: goc mode (from enum rkisp2_isp_goc_mode) + * @segments: segments mode (from enum rkisp2_isp_goc_segments) + * @offset: offset value of the gamma out curve + * @gamma_y: gamma out curve y-axis for all color components + * @reserved: padding + * + * The number of entries of @gamma_y depends on the segments mode. The entries + * are 12-bit unsigned. + */ +struct rkisp2_params_goc { + struct v4l2_isp_params_block_header header; + __u8 mode; + __u8 segments; + __u16 offset; + __u16 gamma_y[RKISP2_ISP_GAMMA_OUT_MAX_SEGMENTS]; + __u8 reserved[2]; +} __attribute__((aligned(8))); + +/** + * struct rkisp2_params_lsc - Configuration used by Lens shading correction + * + * RkISP2 parameters lens shading correction configuration block. + * Identified by :c:type:`RKISP2_PARAMS_BLOCK_LSC`. + * + * The LSC module on the rkisp2 two tables: the 0th table and the 1th table. + * They can be programmed independently and (somewhat) simultaneously, and can be + * swapped by setting a single register. Hence the UAPI here is designed so + * that all these components can be controlled independently. + * + * In the first dimension of {r,gr,gb,b}_data_tbl we can designate which table + * to write the data to. write_table is then used to signal whether to write + * the data, and this can be controlled for both tables. active_table chooses + * which table to activate. set_active_table_when signals whether to set the + * active_table before or after programming the table. This allows + * optimizations such as setting a future table in one parameter buffer while + * swapping before setting it. + * + * This design gives us more control. For example, if we want to only program + * the 0th table without modifying the 1th table, we do not need to also + * populate the 1th table and we can use write_table to designate that we only + * want to program the 0th table. We can also swap tables without needing to + * re-populate the tables by setting active_table and unsetting write_table. + * + * {x,y}_sizes designates the grid of the LSC, and the table entries above + * correspond to the *vertices* of the grid. {x,y}_grads control the bilinear + * interpolation within the grid. + * + * @header: The RkISP2 parameters block header + * @r_data_tbl: Sample table red + * @gr_data_tbl: Sample table green (red) + * @gb_data_tbl: Sample table green (blue) + * @b_data_tbl: Sample table blue + * @write_table: Set to 1 to signal to write the respective table from above + * @active_table: Choose which of the two tables is active (0 or 1) + * @set_active_table_when: From rkisp2_isp_set_active_table_when; switch to the + * active table before or after programming the table + * @x_sizes: Sizes x + * @y_sizes: Sizes y + * @x_grads: Gradients x + * @y_grads: Gradients y + * @window_mode: From enum rkisp2_isp_lsc_config + * @reserved: padding + */ +struct rkisp2_params_lsc { + struct v4l2_isp_params_block_header header; + + __u16 r_data_tbl[2][RKISP2_ISP_LSC_SAMPLES_MAX][RKISP2_ISP_LSC_SAMPLES_MAX]; + __u16 gr_data_tbl[2][RKISP2_ISP_LSC_SAMPLES_MAX][RKISP2_ISP_LSC_SAMPLES_MAX]; + __u16 gb_data_tbl[2][RKISP2_ISP_LSC_SAMPLES_MAX][RKISP2_ISP_LSC_SAMPLES_MAX]; + __u16 b_data_tbl[2][RKISP2_ISP_LSC_SAMPLES_MAX][RKISP2_ISP_LSC_SAMPLES_MAX]; + __u8 write_table[2]; + __u8 active_table; + __u8 set_active_table_when; + + __u16 x_sizes[RKISP2_ISP_LSC_SECTORS_TBL_SIZE_MAX]; + __u16 y_sizes[RKISP2_ISP_LSC_SECTORS_TBL_SIZE_MAX]; + __u16 x_grads[RKISP2_ISP_LSC_SECTORS_TBL_SIZE_MAX]; + __u16 y_grads[RKISP2_ISP_LSC_SECTORS_TBL_SIZE_MAX]; + + __u8 window_mode; + __u8 reserved[3]; +} __attribute__((aligned(8))); + +/** + * struct rkisp2_params_crop - Crop configuration + * + * RkISP2 parameters crop configuration block. + * Identified by :c:type:`RKISP2_PARAMS_BLOCK_CROP`. + * This block is responsible for configuring main and self path crops. The + * driver will automatically compute the scaling factors for the resizer and + * any format conversions (eg. for converting subsampling factors). + * + * @header: The RkISP2 parameters block header + * @crop_en: Enable bits for main path and self path crop below. Comes from enum rkisp2_isp_crop_en + * @reserved: padding + * @mp_crop: Crop window for the main path, relative to the ISP source format size + * @sp_crop: Crop window for the self path, relative to the ISP source format size + * @reserved1: padding + */ +struct rkisp2_params_crop { + struct v4l2_isp_params_block_header header; + __u8 crop_en; + __u8 reserved; + struct rkisp2_isp_window mp_crop; + struct rkisp2_isp_window sp_crop; + __u8 reserved1[6]; +} __attribute__((aligned(8))); + +/** + * struct rkisp2_params_ae_lite - RKISP2 params AE lite config + * + * RkISP2 parameters auto exposure measurement configuration block. + * Identified by :c:type:`RKISP2_PARAMS_BLOCK_AE_LITE`. + * + * TODO change window_num to enum? + * + * @header: The RkISP2 parameters block header + * @window_num: 0 for 1x1, 1 for 5x5 + * @reserved: padding + * @meas_window: Size of measurement window. First window for 5x5. + * @reserved1: padding + */ +struct rkisp2_params_ae_lite { + struct v4l2_isp_params_block_header header; + __u8 window_num; + __u8 reserved; + struct rkisp2_isp_window meas_window; + __u8 reserved1[6]; +} __attribute__((aligned(8))); + +/** + * struct rkisp2_params_hist_lite RKISP2 params histogram lite config + * + * RkISP2 parameters histogram configuration block. + * Identified by :c:type:`RKISP2_PARAMS_BLOCK_HIST_LITE`. + * + * @header: The RkISP2 parameters block header + * @data_sel: Data selection mode (from enum rkisp2_isp_histogram_data_sel) + * @mode: Histogram mode (from enum rkisp2_isp_histogram_mode) + * @stepsize: Predivider (count every pixel) + * @reserved: padding + * @waterline: Waterline for region statics + * @coeffs: Coefficients for raw2y formula + * @reserved1: padding + * @meas_window: Size of first measurement subwindow + * @weights: Weights + * @reserved2: padding + */ +struct rkisp2_params_hist_lite { + struct v4l2_isp_params_block_header header; + __u8 data_sel; + __u8 mode; + __u8 stepsize; + __u8 reserved; + __u16 waterline; + struct rkisp2_isp_color_cc coeffs; + __u8 reserved1; + struct rkisp2_isp_window meas_window; + __u8 weights[RKISP2_ISP_HIST_WEIGHT_GRIDS_SIZE_LITE]; + __u8 reserved2[5]; +} __attribute__((aligned(8))); + +/** + * Same as struct rkisp2_params_hist_lite but for big channel + * + * RkISP2 parameters histogram configuration block. + * Identified by :c:type:`RKISP2_PARAMS_BLOCK_HIST_BIG{0,1,2}`. + * + * @window_num: 0 or 1 for 5x5, 2 or 3 for 15x15 + */ +struct rkisp2_params_hist_big { + struct v4l2_isp_params_block_header header; + __u8 data_sel; + __u8 mode; + __u8 stepsize; + __u8 window_num; + __u16 waterline; + struct rkisp2_isp_color_cc coeffs; + __u8 reserved1; + struct rkisp2_isp_window meas_window; + __u8 weights[RKISP2_ISP_HIST_WEIGHT_GRIDS_SIZE_BIG]; + __u8 reserved2[5]; +} __attribute__((aligned(8))); + +/** + * struct rkisp2_params_awb_meas - Configuration used by rawawb + * + * RkISP2 parameters AWB measurement configuration block. + * Identified by :c:type:`RKISP2_PARAMS_BLOCK_AWB_MEAS`. + * + * @header: The RkISP2 parameters block header + * @meas_window: Size of first measurement subwindow (13 bits) + * @limits: Limits for white point detection [min, max] (8 bits) + * @weights: Weights (6-bits) + * @reserved: padding + */ +struct rkisp2_params_awb_meas { + struct v4l2_isp_params_block_header header; + struct rkisp2_isp_window meas_window; + struct rkisp2_isp_awb_color_quad limits[2]; + __u8 weights[RKISP2_ISP_AWB_COUNTS_SIZE]; + __u8 reserved[7]; +} __attribute__((aligned(8))); + +#define RKISP2_PARAMS_MAX_SIZE \ + (sizeof(struct rkisp2_params_bls) +\ + sizeof(struct rkisp2_params_awb_gains) +\ + sizeof(struct rkisp2_params_csm) +\ + sizeof(struct rkisp2_params_ccm) +\ + sizeof(struct rkisp2_params_goc) +\ + sizeof(struct rkisp2_params_lsc) +\ + sizeof(struct rkisp2_params_crop) +\ + sizeof(struct rkisp2_params_ae_lite) +\ + sizeof(struct rkisp2_params_hist_lite) +\ + sizeof(struct rkisp2_params_hist_big) * 3 +\ + sizeof(struct rkisp2_params_awb_meas)) + +#endif /* _RKISP2_CONFIG_H */ diff --git a/include/linux/videodev2.h b/include/linux/videodev2.h index 8bccad84ff83..d242f3c0ab5f 100644 --- a/include/linux/videodev2.h +++ b/include/linux/videodev2.h @@ -851,6 +851,10 @@ struct v4l2_pix_format { #define V4L2_META_FMT_RK_ISP1_STAT_3A v4l2_fourcc('R', 'K', '1', 'S') /* Rockchip ISP1 3A Statistics */ #define V4L2_META_FMT_RK_ISP1_EXT_PARAMS v4l2_fourcc('R', 'K', '1', 'E') /* Rockchip ISP1 3a Extensible Parameters */ +/* Vendor specific - used for RKISP2 camera sub-system */ +#define V4L2_META_FMT_RKISP2_PARAMS v4l2_fourcc('R', 'K', '2', 'P') /* Rockchip ISP2 Parameters */ +#define V4L2_META_FMT_RKISP2_STATS v4l2_fourcc('R', 'K', '2', 'S') /* Rockchip ISP2 3A Statistics */ + /* Vendor specific - used for C3_ISP */ #define V4L2_META_FMT_C3ISP_PARAMS v4l2_fourcc('C', '3', 'P', 'M') /* Amlogic C3 ISP Parameters */ #define V4L2_META_FMT_C3ISP_STATS v4l2_fourcc('C', '3', 'S', 'T') /* Amlogic C3 ISP Statistics */ From patchwork Thu Aug 27 10:49:56 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Paul Elder X-Patchwork-Id: 28123 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 9F0BEBDDFC for ; Thu, 27 Aug 2026 10:50:41 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 1569868482; Thu, 27 Aug 2026 12:50:41 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="lnVNJxYL"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 5A7E068454 for ; Thu, 27 Aug 2026 12:50:39 +0200 (CEST) Received: from neptunite.hamster-moth.ts.net (unknown [IPv6:2400:2411:160:2f00:306c:5b5b:3c7f:eabd]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id C98B45B3; Thu, 27 Aug 2026 12:49:09 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1787827752; bh=ytEqaRVIa8OcNlp4cntNovjAt7EhrWUjlGQfmMmtBlg=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=lnVNJxYLjxo33Z7nGEcn3kwqhiRlHYXFTmTuWAROS+LnHDWzOV3CR6y/el2LMrl2+ LMPO8oEQ6K81vV284bQY3GNOBC44qMHIzbUT99I+rq2ScuWDc1HMuACLPw8kD9Ummx G2DhsSk7mzhj4co2llLHQX9De82lqsAN3WIbYwus= From: Paul Elder To: laurent.pinchart@ideasonboard.com Cc: Paul Elder , michael.riesch@collabora.com, xuhf@rock-chips.com, stefan.klug@ideasonboard.com, kieran.bingham@ideasonboard.com, dan.scally@ideasonboard.com, jacopo.mondi@ideasonboard.com, nicolas.dufresne@collabora.com, mehdi.djait@linux.intel.com, libcamera-devel@lists.libcamera.org Subject: [PATCH v3 03/20] ipa: rkisp2: params: Add rkisp2 extensible parameters wrapper Date: Thu, 27 Aug 2026 19:49:56 +0900 Message-ID: <20260827105018.2781166-4-paul.elder@ideasonboard.com> X-Mailer: git-send-email 2.47.2 In-Reply-To: <20260827105018.2781166-1-paul.elder@ideasonboard.com> References: <20260827105018.2781166-1-paul.elder@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Add the libipa wrapper for extensible parameters for the rkisp2. This wraps the extensible parameters in the rkisp2 uapi to make them easier to use within libcamera. Signed-off-by: Paul Elder --- This is in a separate patch to reduce the size of the other patches to make them easier to review. Changes in v2: - rename things so that it doesn't conflict with extensible stats that have been added - add crop --- src/ipa/rkisp2/algorithms/algorithm.h | 27 +++++++++ src/ipa/rkisp2/params.h | 85 +++++++++++++++++++++++++++ 2 files changed, 112 insertions(+) create mode 100644 src/ipa/rkisp2/algorithms/algorithm.h create mode 100644 src/ipa/rkisp2/params.h diff --git a/src/ipa/rkisp2/algorithms/algorithm.h b/src/ipa/rkisp2/algorithms/algorithm.h new file mode 100644 index 000000000000..5dbe00def07f --- /dev/null +++ b/src/ipa/rkisp2/algorithms/algorithm.h @@ -0,0 +1,27 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2026, Ideas On Board + * + * RkISP2 control algorithm interface + */ + +#pragma once + +#include + +#include + +#include "module.h" + +namespace libcamera { + +namespace ipa::rkisp2 { + +class Algorithm : public libcamera::ipa::Algorithm +{ +}; + +} /* namespace ipa::rkisp2 */ + +} /* namespace libcamera */ + diff --git a/src/ipa/rkisp2/params.h b/src/ipa/rkisp2/params.h new file mode 100644 index 000000000000..ee4fb3b2c30a --- /dev/null +++ b/src/ipa/rkisp2/params.h @@ -0,0 +1,85 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2026, Ideas On Board + * + * RkISP2 ISP Parameters + */ + +#pragma once + +#include + +#include +#include + +#include + +namespace libcamera { + +namespace ipa::rkisp2 { + +enum class RkISP2ParamsBlocks : uint16_t { + Bls, + AwbGains, + Csm, + Ccm, + Goc, + Lsc, + Crop, + AeLite, + HistLite, + HistBig0, + HistBig1, + HistBig2, + AwbMeas, +}; + +namespace details { + +template +struct block_type { +}; + +#define RkISP2_DEFINE_PARAMS_BLOCK_TYPE(id, cfgType, blkType) \ +template<> \ +struct block_type { \ + using type = struct rkisp2_params_##cfgType; \ + static constexpr rkisp2_params_block_type blockType = \ + rkisp2_params_block_type::RKISP2_PARAMS_BLOCK_##blkType;\ +} + +RkISP2_DEFINE_PARAMS_BLOCK_TYPE(Bls, bls, BLS); +RkISP2_DEFINE_PARAMS_BLOCK_TYPE(AwbGains, awb_gains, AWB_GAINS); +RkISP2_DEFINE_PARAMS_BLOCK_TYPE(Csm, csm, CSM); +RkISP2_DEFINE_PARAMS_BLOCK_TYPE(Ccm, ccm, CCM); +RkISP2_DEFINE_PARAMS_BLOCK_TYPE(Goc, goc, GOC); +RkISP2_DEFINE_PARAMS_BLOCK_TYPE(Lsc, lsc, LSC); +RkISP2_DEFINE_PARAMS_BLOCK_TYPE(Crop, crop, CROP); +RkISP2_DEFINE_PARAMS_BLOCK_TYPE(AeLite, ae_lite, AE_LITE); +RkISP2_DEFINE_PARAMS_BLOCK_TYPE(HistLite, hist_lite, HIST_LITE); +RkISP2_DEFINE_PARAMS_BLOCK_TYPE(HistBig0, hist_big, HIST_BIG0); +RkISP2_DEFINE_PARAMS_BLOCK_TYPE(HistBig1, hist_big, HIST_BIG1); +RkISP2_DEFINE_PARAMS_BLOCK_TYPE(HistBig2, hist_big, HIST_BIG2); +RkISP2_DEFINE_PARAMS_BLOCK_TYPE(AwbMeas, awb_meas, AWB_MEAS); + +struct param_traits { + using id_type = RkISP2ParamsBlocks; + + template + using id_to_details = block_type; +}; + +} /* namespace details */ + +class RkISP2Params : public V4L2Params +{ +public: + RkISP2Params(Span data) + : V4L2Params(data, V4L2_ISP_VERSION_V1) + { + } +}; + +} /* namespace ipa::rkisp2 */ + +} /* namespace libcamera */ From patchwork Thu Aug 27 10:49:57 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Paul Elder X-Patchwork-Id: 28124 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 3EE66BDDFC for ; Thu, 27 Aug 2026 10:50:44 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id E4D8F6847D; Thu, 27 Aug 2026 12:50:43 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="Egedwv4Y"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id F3C7D68454 for ; Thu, 27 Aug 2026 12:50:42 +0200 (CEST) Received: from neptunite.hamster-moth.ts.net (unknown [IPv6:2400:2411:160:2f00:306c:5b5b:3c7f:eabd]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 6EEC0C1; Thu, 27 Aug 2026 12:49:13 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1787827756; bh=uosIflOYxlRN0/1CmcDBD7iyHzs1+//trWqQ5Wluhbo=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=Egedwv4YkgmqWDwlb57IV1Rvq+ggbMzYJKKXOk521/WOFHi+tjDevkZMKmSVS6P5M H6FzphHy3ZleLCVeNaOK5q1R16v2C6ELeRL8StFRRTHUSa6M1h3hS7Z7y/Iu2fxl8u 8KnQ1RSj6k0I6cMycWPhhDeuCFsq9YBPluogtcv4= From: Paul Elder To: laurent.pinchart@ideasonboard.com Cc: Paul Elder , michael.riesch@collabora.com, xuhf@rock-chips.com, stefan.klug@ideasonboard.com, kieran.bingham@ideasonboard.com, dan.scally@ideasonboard.com, jacopo.mondi@ideasonboard.com, nicolas.dufresne@collabora.com, mehdi.djait@linux.intel.com, libcamera-devel@lists.libcamera.org Subject: [PATCH v3 04/20] ipa: rkisp2: stats: Add rkisp2 extensible statistics wrapper Date: Thu, 27 Aug 2026 19:49:57 +0900 Message-ID: <20260827105018.2781166-5-paul.elder@ideasonboard.com> X-Mailer: git-send-email 2.47.2 In-Reply-To: <20260827105018.2781166-1-paul.elder@ideasonboard.com> References: <20260827105018.2781166-1-paul.elder@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Add the libipa wrapper for extensible statistics for the rkisp2. This wraps the extensible statistics in the rkisp2 uapi to make them easier to use within libcamera. Signed-off-by: Paul Elder --- This is in a separate patch to reduce the size of the other patches to make them easier to review. New in v2 --- src/ipa/rkisp2/stats.h | 65 ++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 65 insertions(+) create mode 100644 src/ipa/rkisp2/stats.h diff --git a/src/ipa/rkisp2/stats.h b/src/ipa/rkisp2/stats.h new file mode 100644 index 000000000000..7d50183ef01e --- /dev/null +++ b/src/ipa/rkisp2/stats.h @@ -0,0 +1,65 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2026, Ideas On Board + * + * RkISP2 ISP Statistics + */ + +#pragma once + +#include + +#include +#include + +#include + +namespace libcamera { + +namespace ipa::rkisp2 { + +enum class RkISP2StatsBlocks : uint16_t { + AeLite, + HistLite, + HistBig0, + HistBig1, + HistBig2, + Awb, +}; + +template +struct block_type { +}; + +#define RKISP2_DEFINE_STATS_BLOCK_TYPE(id, cfgType, blkType) \ +template<> \ +struct block_type { \ + using type = struct rkisp2_stats_##cfgType; \ + static constexpr rkisp2_stats_block_type blockType = \ + rkisp2_stats_block_type::RKISP2_STATS_BLOCK_##blkType; \ +} + +RKISP2_DEFINE_STATS_BLOCK_TYPE(AeLite, ae_lite, AE_LITE); +RKISP2_DEFINE_STATS_BLOCK_TYPE(HistLite, hist, HIST_LITE); +RKISP2_DEFINE_STATS_BLOCK_TYPE(HistBig0, hist, HIST_BIG0); +RKISP2_DEFINE_STATS_BLOCK_TYPE(HistBig1, hist, HIST_BIG1); +RKISP2_DEFINE_STATS_BLOCK_TYPE(HistBig2, hist, HIST_BIG2); +RKISP2_DEFINE_STATS_BLOCK_TYPE(Awb, awb, AWB); + +struct stats_traits { + using id_type = RkISP2StatsBlocks; + template using id_to_details = block_type; +}; + +class RkISP2Stats : public V4L2Stats +{ +public: + RkISP2Stats(Span data) + : V4L2Stats(data, V4L2_ISP_VERSION_V1) + { + } +}; + +} /* namespace ipa::rkisp2 */ + +} /* namespace libcamera */ From patchwork Thu Aug 27 10:49:58 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Paul Elder X-Patchwork-Id: 28125 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 1B765BDDFC for ; Thu, 27 Aug 2026 10:50:48 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id C2B956847D; Thu, 27 Aug 2026 12:50:47 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="SSoEMmgV"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id A3FDE68454 for ; Thu, 27 Aug 2026 12:50:46 +0200 (CEST) Received: from neptunite.hamster-moth.ts.net (unknown [IPv6:2400:2411:160:2f00:306c:5b5b:3c7f:eabd]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 145CB5B3; Thu, 27 Aug 2026 12:49:16 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1787827760; bh=RCr7Egp/mMIrcJAlCiT/38vZ5WdMFnsHjXF/U8nDX8I=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=SSoEMmgVMxu9f6ki2/TMXqlz6GE0mlnAY/4nggqoMBRveiy9JwpBzrFehXKe8Hd3M /hs14h1/rel5tYICVp17anZT+fno7YbV1Dbyfz8hZTfeXT2/FG0lOkyHMVqVpBndyq yP57epAETnEsXik8mhy9ZmceXaG76UullnF6xH9k= From: Paul Elder To: laurent.pinchart@ideasonboard.com Cc: Paul Elder , michael.riesch@collabora.com, xuhf@rock-chips.com, stefan.klug@ideasonboard.com, kieran.bingham@ideasonboard.com, dan.scally@ideasonboard.com, jacopo.mondi@ideasonboard.com, nicolas.dufresne@collabora.com, mehdi.djait@linux.intel.com, libcamera-devel@lists.libcamera.org Subject: [PATCH v3 05/20] include: ipa: rkisp2: Add rkisp2 ipa interface Date: Thu, 27 Aug 2026 19:49:58 +0900 Message-ID: <20260827105018.2781166-6-paul.elder@ideasonboard.com> X-Mailer: git-send-email 2.47.2 In-Reply-To: <20260827105018.2781166-1-paul.elder@ideasonboard.com> References: <20260827105018.2781166-1-paul.elder@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Add the IPA interface for the rkisp2. It is based on and is nearly identical to the rkisp1 IPA interface. Signed-off-by: Paul Elder --- This is also split into a separate patch to make the other patches smaller and easier to review. No change in v2 --- include/libcamera/ipa/meson.build | 1 + include/libcamera/ipa/rkisp2.mojom | 47 ++++++++++++++++++++++++++++++ 2 files changed, 48 insertions(+) create mode 100644 include/libcamera/ipa/rkisp2.mojom diff --git a/include/libcamera/ipa/meson.build b/include/libcamera/ipa/meson.build index fb518526a2a0..32854928fdc5 100644 --- a/include/libcamera/ipa/meson.build +++ b/include/libcamera/ipa/meson.build @@ -66,6 +66,7 @@ pipeline_ipa_mojom_mapping = { 'ipu3': 'ipu3.mojom', 'mali-c55': 'mali-c55.mojom', 'rkisp1': 'rkisp1.mojom', + 'rkisp2': 'rkisp2.mojom', 'rpi/pisp': 'raspberrypi.mojom', 'rpi/vc4': 'raspberrypi.mojom', 'simple': 'softisp.mojom', diff --git a/include/libcamera/ipa/rkisp2.mojom b/include/libcamera/ipa/rkisp2.mojom new file mode 100644 index 000000000000..52284a29cd24 --- /dev/null +++ b/include/libcamera/ipa/rkisp2.mojom @@ -0,0 +1,47 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ + +/* + * \todo Document the interface and remove the related EXCLUDE_PATTERNS entry. + */ + +module ipa.rkisp2; + +import "include/libcamera/ipa/core.mojom"; + +/* + * colorSpaceEncoding comes from libcamera::ColorSpace::YcbcrEncoding; -1 for invalid + * colorSpaceRange comes from libcamera::ColorSpace::Range; -1 for invalid + */ +struct IPAConfigInfo { + libcamera.IPACameraSensorInfo sensorInfo; + libcamera.ControlInfoMap sensorControls; + int32 colorSpaceEncoding; + int32 colorSpaceRange; +}; + +interface IPARkISP2Interface { + init(libcamera.IPASettings settings, + libcamera.IPACameraSensorInfo sensorInfo, + libcamera.ControlInfoMap sensorControls) + => (int32 ret, libcamera.ControlInfoMap ipaControls); + start() => (int32 ret); + stop(); + + configure(IPAConfigInfo configInfo) + => (int32 ret, libcamera.ControlInfoMap ipaControls); + + mapBuffers(array buffers); + unmapBuffers(array ids); + + [async] queueRequest(uint32 frame, libcamera.ControlList reqControls); + [async] computeParams(uint32 frame, uint32 bufferId); + [async] processStats(uint32 frame, uint32 bufferId, + libcamera.ControlList sensorControls); +}; + +interface IPARkISP2EventInterface { + paramsComputed(uint32 frame, uint32 bufferId, uint32 bytesused); + setSensorControls(uint32 frame, libcamera.ControlList sensorControls); + metadataReady(libcamera.ControlList metadata); +}; + From patchwork Thu Aug 27 10:49:59 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Paul Elder X-Patchwork-Id: 28126 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 9D6B1BDDFC for ; Thu, 27 Aug 2026 10:50:52 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 6149C6847D; Thu, 27 Aug 2026 12:50:52 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="pdGouto+"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 48D9168454 for ; Thu, 27 Aug 2026 12:50:50 +0200 (CEST) Received: from neptunite.hamster-moth.ts.net (unknown [IPv6:2400:2411:160:2f00:306c:5b5b:3c7f:eabd]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id B27C5C1; Thu, 27 Aug 2026 12:49:20 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1787827763; bh=cp3XaEUIgyI/K/wmYCAIoGlQZvBejJOuWfq9D+lxoSc=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=pdGouto+i1ccl7As9f3fCoqNWaphscBuMynxMSnPqAMHIXC+Dwt5YunT7zTspfFn5 SbNGF4G/II8t1PW5fIzdgYVlHlvqw0a12hgzZcVVRv0c+qClK+vxnA7maffwkSui4Z mchDnDOCIkWtGDU480beYIoPp0laKbwIvISsrO+g= From: Paul Elder To: laurent.pinchart@ideasonboard.com Cc: Paul Elder , michael.riesch@collabora.com, xuhf@rock-chips.com, stefan.klug@ideasonboard.com, kieran.bingham@ideasonboard.com, dan.scally@ideasonboard.com, jacopo.mondi@ideasonboard.com, nicolas.dufresne@collabora.com, mehdi.djait@linux.intel.com, libcamera-devel@lists.libcamera.org Subject: [PATCH v3 06/20] ipa: rkisp2: Add rkisp2 ipa Date: Thu, 27 Aug 2026 19:49:59 +0900 Message-ID: <20260827105018.2781166-7-paul.elder@ideasonboard.com> X-Mailer: git-send-email 2.47.2 In-Reply-To: <20260827105018.2781166-1-paul.elder@ideasonboard.com> References: <20260827105018.2781166-1-paul.elder@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Implement the rkisp2 IPA. It is based on the rkisp1 IPA. No control algorithms are implemented yet, but this lays the foundation and plumbing for the IPA to function. The control algorithms will be added in later patches. Signed-off-by: Paul Elder --- This is also split from other patches to make them all easier to review. I put this patch before the pipeline handler patch so that the pipeline handler doesn't error out when it fails to find an IPA. Changes in v2: - initialize/declare ScalerCrop control - use extensible stats --- meson_options.txt | 2 +- src/ipa/meson.build | 1 + src/ipa/rkisp2/algorithms/meson.build | 5 + src/ipa/rkisp2/algorithms/module.h | 29 ++ src/ipa/rkisp2/ipa_context.h | 211 ++++++++++++ src/ipa/rkisp2/meson.build | 29 ++ src/ipa/rkisp2/rkisp2.cpp | 440 ++++++++++++++++++++++++++ 7 files changed, 716 insertions(+), 1 deletion(-) create mode 100644 src/ipa/rkisp2/algorithms/meson.build create mode 100644 src/ipa/rkisp2/algorithms/module.h create mode 100644 src/ipa/rkisp2/ipa_context.h create mode 100644 src/ipa/rkisp2/meson.build create mode 100644 src/ipa/rkisp2/rkisp2.cpp diff --git a/meson_options.txt b/meson_options.txt index e829ebe60490..56a5e2085990 100644 --- a/meson_options.txt +++ b/meson_options.txt @@ -48,7 +48,7 @@ option('gstreamer', option('ipas', type : 'array', - choices : ['ipu3', 'mali-c55', 'rkisp1', 'rpi/pisp', 'rpi/vc4', + choices : ['ipu3', 'mali-c55', 'rkisp1', 'rkisp2', 'rpi/pisp', 'rpi/vc4', 'softisp', 'vimc'], description : 'Select which IPA modules to build') diff --git a/src/ipa/meson.build b/src/ipa/meson.build index ab4a1897f58a..3e8f5bffcd3a 100644 --- a/src/ipa/meson.build +++ b/src/ipa/meson.build @@ -28,6 +28,7 @@ supported_ipas = { 'ipu3': 'ipu3', 'mali-c55': 'mali-c55', 'rkisp1': 'rkisp1', + 'rkisp2': 'rkisp2', 'rpi/pisp': 'rpi/pisp', 'rpi/vc4': 'rpi/vc4', 'simple': 'softisp', diff --git a/src/ipa/rkisp2/algorithms/meson.build b/src/ipa/rkisp2/algorithms/meson.build new file mode 100644 index 000000000000..4bb81e48eb01 --- /dev/null +++ b/src/ipa/rkisp2/algorithms/meson.build @@ -0,0 +1,5 @@ +# SPDX-License-Identifier: CC0-1.0 + +rkisp2_ipa_algorithms = files([ +]) + diff --git a/src/ipa/rkisp2/algorithms/module.h b/src/ipa/rkisp2/algorithms/module.h new file mode 100644 index 000000000000..fbab898c5e8b --- /dev/null +++ b/src/ipa/rkisp2/algorithms/module.h @@ -0,0 +1,29 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2026, Ideas on Board Oy. + * + * RkISP2 IPA Module + */ + +#pragma once + +#include + +#include + +#include + +#include "ipa_context.h" +#include "params.h" +#include "stats.h" + +namespace libcamera { + +namespace ipa::rkisp2 { + +using Module = ipa::Module; + +} /* namespace ipa::rkisp2 */ + +} /* namespace libcamera*/ diff --git a/src/ipa/rkisp2/ipa_context.h b/src/ipa/rkisp2/ipa_context.h new file mode 100644 index 000000000000..29bdb5f19857 --- /dev/null +++ b/src/ipa/rkisp2/ipa_context.h @@ -0,0 +1,211 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2026, Ideas on Board Oy. + * + * RkISP2 IPA Context + * + */ + +#pragma once + +#include + +#include + +#include + +#include +#include +#include + +#include + +#include "libcamera/internal/debug_controls.h" +#include "libcamera/internal/matrix.h" +#include "libcamera/internal/vector.h" + +#include "libipa/agc_mean_luminance.h" +#include "libipa/awb.h" +#include "libipa/camera_sensor_helper.h" +#include "libipa/ccm.h" +#include "libipa/fc_queue.h" +#include "libipa/fixedpoint.h" +#include "libipa/lsc.h" + +namespace libcamera { + +namespace ipa::rkisp2 { + +struct IPAHwSettings { + unsigned int numAeCells; + unsigned int numHistogramBins; + unsigned int numHistogramWeights; + unsigned int numGammaOutSamples; + uint32_t supportedBlocks; + bool compand; +}; + +struct RKISP2AwbSession { + struct rkisp2_isp_window measureWindow; + bool enabled; +}; + +struct IPASessionConfiguration { + struct { + struct rkisp2_isp_window measureWindow; + struct rkisp2_isp_window measureWindow15; + } agc; + + struct RKISP2AwbSession awb; + + struct { + utils::Duration minExposureTime; + utils::Duration maxExposureTime; + double minAnalogueGain; + double maxAnalogueGain; + + int32_t defVBlank; + utils::Duration lineDuration; + Size size; + } sensor; + + struct { + int32_t colorSpaceEncoding; + int32_t colorSpaceRange; + } csm; + + bool raw; +}; + +struct IPAActiveState { + struct { + struct { + uint32_t exposure; + double gain; + } manual; + struct { + uint32_t exposure; + double gain; + double quantizationGain; + double yTarget; + } automatic; + + bool autoExposureEnabled; + bool autoGainEnabled; + double exposureValue; + controls::AeConstraintModeEnum constraintMode; + controls::AeExposureModeEnum exposureMode; + controls::AeMeteringModeEnum meteringMode; + utils::Duration minFrameDuration; + utils::Duration maxFrameDuration; + } agc; + + ipa::awb::ActiveState awb; + + struct { + double gamma; + } goc; + + ipa::ccm::ActiveState ccm; + + struct { + double lux; + } lux; + + struct { + controls::WdrModeEnum mode; + AgcMeanLuminance::AgcConstraint constraint; + double gain; + double strength; + } wdr; + + ipa::lsc::ActiveState lsc; + + struct { + Matrix csm; + bool update; + } csm; + + struct { + Rectangle crop; + bool set; + } crop; +}; + +struct IPAFrameContext : public FrameContext { + struct { + uint32_t exposure; + double gain; + double exposureValue; + double quantizationGain; + uint32_t vblank; + double yTarget; + bool autoExposureEnabled; + bool autoGainEnabled; + controls::AeConstraintModeEnum constraintMode; + controls::AeExposureModeEnum exposureMode; + controls::AeMeteringModeEnum meteringMode; + utils::Duration minFrameDuration; + utils::Duration maxFrameDuration; + utils::Duration frameDuration; + bool updateMetering; + bool autoExposureModeChange; + bool autoGainModeChange; + } agc; + + ipa::awb::FrameContext awb; + + struct { + double gamma; + bool update; + } goc; + + struct { + uint32_t exposure; + double gain; + } sensor; + + ipa::ccm::FrameContext ccm; + + struct { + double lux; + } lux; + + struct { + controls::WdrModeEnum mode; + double strength; + double gain; + } wdr; + + ipa::lsc::FrameContext lsc; + + struct { + Rectangle crop; + bool set; + } crop; +}; + +struct IPAContext { + IPAContext(unsigned int frameContextSize) + : frameContexts(frameContextSize) + { + } + + IPAHwSettings hw; + IPACameraSensorInfo sensorInfo; + IPASessionConfiguration configuration; + IPAActiveState activeState; + + FCQueue frameContexts; + + ControlInfoMap::Map ctrlMap; + + DebugMetadata debugMetadata; + + /* Interface to the Camera Helper */ + std::unique_ptr camHelper; +}; + +} /* namespace ipa::rkisp2 */ + +} /* namespace libcamera*/ diff --git a/src/ipa/rkisp2/meson.build b/src/ipa/rkisp2/meson.build new file mode 100644 index 000000000000..f2f435c2d66f --- /dev/null +++ b/src/ipa/rkisp2/meson.build @@ -0,0 +1,29 @@ +# SPDX-License-Identifier: CC0-1.0 + +subdir('algorithms') + +ipa_name = 'ipa_rkisp2' + +rkisp2_ipa_sources = files([ + 'rkisp2.cpp', +]) + +rkisp2_ipa_sources += rkisp2_ipa_algorithms + +mod = shared_module(ipa_name, rkisp2_ipa_sources, + name_prefix : '', + include_directories : [ipa_includes], + dependencies : [libcamera_private, libipa_dep], + install : true, + install_dir : ipa_install_dir) + +if ipa_sign_module + custom_target(ipa_name + '.so.sign', + input : mod, + output : ipa_name + '.so.sign', + command : [ipa_sign, ipa_priv_key, '@INPUT@', '@OUTPUT@'], + install : false, + build_by_default : true) +endif + +ipa_names += ipa_name diff --git a/src/ipa/rkisp2/rkisp2.cpp b/src/ipa/rkisp2/rkisp2.cpp new file mode 100644 index 000000000000..1ed2771bf1fc --- /dev/null +++ b/src/ipa/rkisp2/rkisp2.cpp @@ -0,0 +1,440 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2026, Ideas on Board Oy. + * + * RkISP2 Image Processing Algorithms + */ + +#include +#include +#include +#include +#include +#include + +#include +#include + +#include +#include + +#include +#include +#include +#include + +#include +#include +#include + +#include "libcamera/internal/formats.h" +#include "libcamera/internal/mapped_framebuffer.h" +#include "libcamera/internal/yaml_parser.h" + +#include "algorithms/algorithm.h" + +#include "ipa_context.h" +#include "params.h" + +namespace libcamera { + +LOG_DEFINE_CATEGORY(IPARkISP2) + +using namespace std::literals::chrono_literals; + +namespace ipa::rkisp2 { + +/* Maximum number of frame contexts to be held */ +static constexpr uint32_t kMaxFrameContexts = 16; + +class IPARkISP2 : public IPARkISP2Interface, public Module +{ +public: + IPARkISP2(); + + int init(const IPASettings &settings, + const IPACameraSensorInfo &sensorInfo, + const ControlInfoMap &sensorControls, + ControlInfoMap *ipaControls) override; + int start() override; + void stop() override; + + int configure(const IPAConfigInfo &ipaConfig, + ControlInfoMap *ipaControls) override; + void mapBuffers(const std::vector &buffers) override; + void unmapBuffers(const std::vector &ids) override; + + void queueRequest(const uint32_t frame, const ControlList &controls) override; + void computeParams(const uint32_t frame, const uint32_t bufferId) override; + void initializeFrameContext(IPAFrameContext &frameContext, + const ControlList &controls); + void processStats(const uint32_t frame, const uint32_t bufferId, + const ControlList &sensorControls) override; + +protected: + std::string logPrefix() const override; + +private: + void updateControls(const IPACameraSensorInfo &sensorInfo, + const ControlInfoMap &sensorControls, + ControlInfoMap *ipaControls); + + void setControls(unsigned int frame, const IPAFrameContext &frameContext); + + std::map buffers_; + std::map mappedBuffers_; + + ControlInfoMap sensorControls_; + + /* Local parameter storage */ + struct IPAContext context_; +}; + +namespace { + +/* List of controls handled by the RkISP2 IPA */ +const ControlInfoMap::Map rkisp2Controls{ + { &controls::DebugMetadataEnable, ControlInfo(false, true, false) }, +}; + +} /* namespace */ + +IPARkISP2::IPARkISP2() + : context_(kMaxFrameContexts) +{ + context_.frameContexts.setInitCallback( + [this](IPAFrameContext &fc, const ControlList &c) { + this->initializeFrameContext(fc, c); + }); +} + +std::string IPARkISP2::logPrefix() const +{ + return "rkisp2"; +} + +int IPARkISP2::init(const IPASettings &settings, + const IPACameraSensorInfo &sensorInfo, + const ControlInfoMap &sensorControls, + ControlInfoMap *ipaControls) +{ + context_.sensorInfo = sensorInfo; + + context_.camHelper = CameraSensorHelperFactoryBase::create(settings.sensorModel); + if (!context_.camHelper) { + LOG(IPARkISP2, Error) + << "Failed to create camera sensor helper for " + << settings.sensorModel; + return -ENODEV; + } + + context_.configuration.sensor.lineDuration = + sensorInfo.minLineLength * 1.0s / sensorInfo.pixelRate; + + /* Load the tuning data file. */ + File file(settings.configurationFile); + if (!file.open(File::OpenModeFlag::ReadOnly)) { + int ret = file.error(); + LOG(IPARkISP2, Error) + << "Failed to open configuration file " + << settings.configurationFile << ": " << strerror(-ret); + return ret; + } + + std::unique_ptr data = YamlParser::parse(file); + if (!data) + return -EINVAL; + + unsigned int version = (*data)["version"].get(0); + if (version != 1) { + LOG(IPARkISP2, Error) + << "Invalid tuning file version " << version; + return -EINVAL; + } + + if (!data->contains("algorithms")) { + LOG(IPARkISP2, Error) + << "Tuning file doesn't contain any algorithm"; + return -EINVAL; + } + + int ret = createAlgorithms(context_, (*data)["algorithms"]); + if (ret) + return ret; + + /* Initialize controls. */ + updateControls(sensorInfo, sensorControls, ipaControls); + + return 0; +} + +int IPARkISP2::start() +{ + /* \todo Properly handle startup controls. */ + return 0; +} + +void IPARkISP2::stop() +{ + context_.frameContexts.clear(); +} + +int IPARkISP2::configure(const IPAConfigInfo &ipaConfig, + ControlInfoMap *ipaControls) +{ + sensorControls_ = ipaConfig.sensorControls; + + const auto itExp = sensorControls_.find(V4L2_CID_EXPOSURE); + int32_t minExposure = itExp->second.min().get(); + int32_t maxExposure = itExp->second.max().get(); + + const auto itGain = sensorControls_.find(V4L2_CID_ANALOGUE_GAIN); + int32_t minGain = itGain->second.min().get(); + int32_t maxGain = itGain->second.max().get(); + + LOG(IPARkISP2, Debug) + << "Exposure: [" << minExposure << ", " << maxExposure + << "], gain: [" << minGain << ", " << maxGain << "]"; + + /* Clear the IPA context before the streaming session. */ + context_.configuration = {}; + context_.activeState = {}; + context_.frameContexts.clear(); + + const IPACameraSensorInfo &info = ipaConfig.sensorInfo; + const ControlInfo vBlank = sensorControls_.find(V4L2_CID_VBLANK)->second; + context_.configuration.sensor.defVBlank = vBlank.def().get(); + context_.configuration.sensor.size = info.outputSize; + context_.configuration.sensor.lineDuration = info.minLineLength * 1.0s / info.pixelRate; + + /* Update the camera controls using the new sensor settings. */ + updateControls(info, sensorControls_, ipaControls); + + /* + * When the AGC computes the new exposure values for a frame, it needs + * to know the limits for exposure time and analogue gain. As it depends + * on the sensor, update it with the controls. + * + * \todo take VBLANK into account for maximum exposure time + */ + context_.configuration.sensor.minExposureTime = + minExposure * context_.configuration.sensor.lineDuration; + context_.configuration.sensor.maxExposureTime = + maxExposure * context_.configuration.sensor.lineDuration; + context_.configuration.sensor.minAnalogueGain = + context_.camHelper->gain(minGain); + context_.configuration.sensor.maxAnalogueGain = + context_.camHelper->gain(maxGain); + + context_.configuration.csm.colorSpaceEncoding = ipaConfig.colorSpaceEncoding; + context_.configuration.csm.colorSpaceRange = ipaConfig.colorSpaceRange; + + for (const auto &a : algorithms()) { + Algorithm *algo = static_cast(a.get()); + + int ret = algo->configure(context_, info); + if (ret) + return ret; + } + + return 0; +} + +void IPARkISP2::mapBuffers(const std::vector &buffers) +{ + for (const IPABuffer &buffer : buffers) { + auto elem = buffers_.emplace(std::piecewise_construct, + std::forward_as_tuple(buffer.id), + std::forward_as_tuple(buffer.planes)); + const FrameBuffer &fb = elem.first->second; + + MappedFrameBuffer mappedBuffer(&fb, MappedFrameBuffer::MapFlag::ReadWrite); + if (!mappedBuffer.isValid()) { + LOG(IPARkISP2, Fatal) << "Failed to mmap buffer: " + << strerror(mappedBuffer.error()); + } + + mappedBuffers_.emplace(buffer.id, std::move(mappedBuffer)); + } +} + +void IPARkISP2::unmapBuffers(const std::vector &ids) +{ + for (unsigned int id : ids) { + const auto fb = buffers_.find(id); + if (fb == buffers_.end()) + continue; + + mappedBuffers_.erase(id); + buffers_.erase(id); + } +} + +void IPARkISP2::queueRequest(const uint32_t frame, const ControlList &controls) +{ + context_.debugMetadata.enableByControl(controls); + + context_.frameContexts.getOrInitContext(frame, controls); +} + +void IPARkISP2::initializeFrameContext(IPAFrameContext &frameContext, + const ControlList &controls) +{ + for (const auto &a : algorithms()) { + Algorithm *algo = static_cast(a.get()); + algo->queueRequest(context_, frameContext.frame(), frameContext, controls); + } +} + +void IPARkISP2::computeParams(const uint32_t frame, const uint32_t bufferId) +{ + IPAFrameContext &frameContext = context_.frameContexts.getOrInitContext(frame); + + RkISP2Params params(mappedBuffers_.at(bufferId).planes()[0]); + + for (const auto &algo : algorithms()) + algo->prepare(context_, frame, frameContext, ¶ms); + + paramsComputed.emit(frame, bufferId, params.bytesused()); +} + +void IPARkISP2::processStats(const uint32_t frame, const uint32_t bufferId, + const ControlList &sensorControls) +{ + IPAFrameContext &frameContext = context_.frameContexts.getOrInitContext(frame); + + RkISP2Stats stats(mappedBuffers_.at(bufferId).planes()[0]); + + frameContext.sensor.exposure = + sensorControls.get(V4L2_CID_EXPOSURE).get(); + frameContext.sensor.gain = + context_.camHelper->gain(sensorControls.get(V4L2_CID_ANALOGUE_GAIN).get()); + + ControlList metadata(controls::controls); + + for (const auto &algo : algorithms()) + algo->process(context_, frame, frameContext, &stats, metadata); + + setControls(frame, frameContext); + + metadataReady.emit(metadata); +} + +void IPARkISP2::setControls(unsigned int frame, const IPAFrameContext &frameContext) +{ + /* + * \todo The frame number is most likely wrong here, we need to take + * internal sensor delays and other timing parameters into account. + */ + + uint32_t exposure = frameContext.agc.exposure; + uint32_t gain = context_.camHelper->gainCode(frameContext.agc.gain); + uint32_t vblank = frameContext.agc.vblank; + + LOG(IPARkISP2, Debug) + << "Set controls for frame " << frame << ": exposure " << exposure + << ", gain " << frameContext.agc.gain << ", vblank " << vblank; + + ControlList ctrls(sensorControls_); + if (frameContext.agc.exposure * frameContext.agc.gain > 0) { + ctrls.set(V4L2_CID_EXPOSURE, static_cast(exposure)); + ctrls.set(V4L2_CID_ANALOGUE_GAIN, static_cast(gain)); + } + ctrls.set(V4L2_CID_VBLANK, static_cast(vblank)); + + setSensorControls.emit(frame, ctrls); +} + +void IPARkISP2::updateControls(const IPACameraSensorInfo &sensorInfo, + const ControlInfoMap &sensorControls, + ControlInfoMap *ipaControls) +{ + ControlInfoMap::Map ctrlMap = rkisp2Controls; + + /* + * Compute exposure time limits from the V4L2_CID_EXPOSURE control + * limits and the line duration. + */ + double lineDuration = context_.configuration.sensor.lineDuration.get(); + const ControlInfo &v4l2Exposure = sensorControls.find(V4L2_CID_EXPOSURE)->second; + int32_t minExposure = v4l2Exposure.min().get() * lineDuration; + int32_t maxExposure = v4l2Exposure.max().get() * lineDuration; + int32_t defExposure = v4l2Exposure.def().get() * lineDuration; + ctrlMap.emplace(std::piecewise_construct, + std::forward_as_tuple(&controls::ExposureTime), + std::forward_as_tuple(minExposure, maxExposure, defExposure)); + + /* Compute the analogue gain limits. */ + const ControlInfo &v4l2Gain = sensorControls.find(V4L2_CID_ANALOGUE_GAIN)->second; + float minGain = context_.camHelper->gain(v4l2Gain.min().get()); + float maxGain = context_.camHelper->gain(v4l2Gain.max().get()); + float defGain = context_.camHelper->gain(v4l2Gain.def().get()); + ctrlMap.emplace(std::piecewise_construct, + std::forward_as_tuple(&controls::AnalogueGain), + std::forward_as_tuple(minGain, maxGain, defGain)); + + /* + * Compute the frame duration limits. + * + * The frame length is computed assuming a fixed line length combined + * with the vertical frame sizes. + */ + const ControlInfo &v4l2HBlank = sensorControls.find(V4L2_CID_HBLANK)->second; + uint32_t hblank = v4l2HBlank.def().get(); + uint32_t lineLength = sensorInfo.outputSize.width + hblank; + + const ControlInfo &v4l2VBlank = sensorControls.find(V4L2_CID_VBLANK)->second; + std::array frameHeights{ + v4l2VBlank.min().get() + sensorInfo.outputSize.height, + v4l2VBlank.max().get() + sensorInfo.outputSize.height, + v4l2VBlank.def().get() + sensorInfo.outputSize.height, + }; + + std::array frameDurations; + for (unsigned int i = 0; i < frameHeights.size(); ++i) { + uint64_t frameSize = lineLength * frameHeights[i]; + frameDurations[i] = frameSize / (sensorInfo.pixelRate / 1000000U); + } + + /* \todo Move this (and other agc-related controls) to agc */ + context_.ctrlMap[&controls::FrameDurationLimits] = + ControlInfo(frameDurations[0], frameDurations[1], + ControlValue(Span{ { frameDurations[2], frameDurations[2] } })); + + Rectangle ispMinCrop{ 0, 0, 32, 32 }; + /* + * No need to clamp this as the hardware will hang anyway if sensor + * size > isp max size + */ + Rectangle ispMaxCrop{ 0, 0, sensorInfo.outputSize }; + /* + * \todo Either always enable Crop algo or make this conditional on + * when the Crop algo is present + */ + context_.ctrlMap[&controls::ScalerCrop] = + ControlInfo(ispMinCrop, ispMaxCrop, ispMaxCrop); + + ctrlMap.insert(context_.ctrlMap.begin(), context_.ctrlMap.end()); + *ipaControls = ControlInfoMap(std::move(ctrlMap), controls::controls); +} + +} /* namespace ipa::rkisp2 */ + +/* + * External IPA module interface + */ + +extern "C" { +const struct IPAModuleInfo ipaModuleInfo = { + IPA_MODULE_API_VERSION, + 1, + "rkisp2", +}; + +IPAInterface *ipaCreate() +{ + return new ipa::rkisp2::IPARkISP2(); +} +} + +} /* namespace libcamera */ From patchwork Thu Aug 27 10:50:00 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Paul Elder X-Patchwork-Id: 28127 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 3CF72BDDFC for ; Thu, 27 Aug 2026 10:50:56 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id E263B68487; Thu, 27 Aug 2026 12:50:55 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="CTutuHSq"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 2D6F168476 for ; Thu, 27 Aug 2026 12:50:54 +0200 (CEST) Received: from neptunite.hamster-moth.ts.net (unknown [IPv6:2400:2411:160:2f00:306c:5b5b:3c7f:eabd]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 5D06B145C; Thu, 27 Aug 2026 12:49:24 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1787827767; bh=QxUz/zCRZt5t7Df8ulREtPTwJD6sQQtHXsy00sONejo=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=CTutuHSqXTYXm9n8r1dbZK4Vb+oU8PVYFMschvnEOxizpxQR/3RfciiIRn8V+YC1L o0rGEN6v2vFz+1TWwOi+cul675ICng4/zWhwXlZwXdf8FTQ/zhtB/eMGblYMZpFWle KrFuO15VO7RucdWBuzJffG3ynYPKw5k2JdaNXukg= From: Paul Elder To: laurent.pinchart@ideasonboard.com Cc: Paul Elder , michael.riesch@collabora.com, xuhf@rock-chips.com, stefan.klug@ideasonboard.com, kieran.bingham@ideasonboard.com, dan.scally@ideasonboard.com, jacopo.mondi@ideasonboard.com, nicolas.dufresne@collabora.com, mehdi.djait@linux.intel.com, libcamera-devel@lists.libcamera.org Subject: [PATCH v3 07/20] pipeline: rkisp2: Implement pipeline handler for rkisp2 Date: Thu, 27 Aug 2026 19:50:00 +0900 Message-ID: <20260827105018.2781166-8-paul.elder@ideasonboard.com> X-Mailer: git-send-email 2.47.2 In-Reply-To: <20260827105018.2781166-1-paul.elder@ideasonboard.com> References: <20260827105018.2781166-1-paul.elder@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Implement the pipeline handler for rkisp2. This currently supports the ISP version integrated on the Rockchip RK3588. The current version only supports memory-to-memory mode and only the main path and only raw sensors. Signed-off-by: Paul Elder --- Changes in v2: - split dma rx pads - split main/self path capture pads - no more configuring crop rectangles, as they have been moved to parameters buffers --- Documentation/runtime_configuration.rst | 9 + meson.build | 1 + meson_options.txt | 1 + src/libcamera/pipeline/rkisp2/meson.build | 5 + src/libcamera/pipeline/rkisp2/rkisp2.cpp | 1293 +++++++++++++++++++++ 5 files changed, 1309 insertions(+) create mode 100644 src/libcamera/pipeline/rkisp2/meson.build create mode 100644 src/libcamera/pipeline/rkisp2/rkisp2.cpp diff --git a/Documentation/runtime_configuration.rst b/Documentation/runtime_configuration.rst index a71b715a1724..53a5a29744c1 100644 --- a/Documentation/runtime_configuration.rst +++ b/Documentation/runtime_configuration.rst @@ -81,6 +81,8 @@ Configuration file example supported_devices: - driver: mxc-isi software_isp: true + rkisp2: + isp_enable: true software_isp: copy_input_buffer: false measure: @@ -157,6 +159,13 @@ pipelines.simple.supported_devices.driver, pipelines.simple.supported_devices.so Example `software_isp` value: ``true`` +pipelines.rkisp2.isp_enable + Configure whether or not to use the ISP. Default (when unconfigured) is + true. When set to false the ISP will not be used, so only the VICAP will be + used for capture. + + Example value: ``false`` + software_isp.copy_input_buffer Define whether input buffers should be copied into standard (cached) memory in software ISP. This is done by default to prevent very slow diff --git a/meson.build b/meson.build index d8e78ed6c01a..516014ffc93a 100644 --- a/meson.build +++ b/meson.build @@ -219,6 +219,7 @@ pipelines_support = { 'ipu3': arch_x86, 'mali-c55': arch_arm, 'rkisp1': arch_arm, + 'rkisp2': arch_arm, 'rpi/pisp': arch_arm, 'rpi/vc4': arch_arm, 'simple': ['any'], diff --git a/meson_options.txt b/meson_options.txt index 56a5e2085990..55ab557d2953 100644 --- a/meson_options.txt +++ b/meson_options.txt @@ -82,6 +82,7 @@ option('pipelines', 'ipu3', 'mali-c55', 'rkisp1', + 'rkisp2', 'rpi/pisp', 'rpi/vc4', 'simple', diff --git a/src/libcamera/pipeline/rkisp2/meson.build b/src/libcamera/pipeline/rkisp2/meson.build new file mode 100644 index 000000000000..86f43fb7fcc9 --- /dev/null +++ b/src/libcamera/pipeline/rkisp2/meson.build @@ -0,0 +1,5 @@ +# SPDX-License-Identifier: CC0-1.0 + +libcamera_internal_sources += files([ + 'rkisp2.cpp', +]) diff --git a/src/libcamera/pipeline/rkisp2/rkisp2.cpp b/src/libcamera/pipeline/rkisp2/rkisp2.cpp new file mode 100644 index 000000000000..fa17d08381f2 --- /dev/null +++ b/src/libcamera/pipeline/rkisp2/rkisp2.cpp @@ -0,0 +1,1293 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2026, Ideas on Board Oy. + * + * Pipeline handler for Rockchip ISP2 + */ + +#include +#include +#include +#include +#include + +#include +#include + +#include +#include +#include +#include +#include + +#include +#include +#include + +#include "libcamera/internal/buffer_queue.h" +#include "libcamera/internal/camera.h" +#include "libcamera/internal/camera_manager.h" +#include "libcamera/internal/camera_sensor.h" +#include "libcamera/internal/delayed_controls.h" +#include "libcamera/internal/device_enumerator.h" +#include "libcamera/internal/formats.h" +#include "libcamera/internal/framebuffer.h" +#include "libcamera/internal/global_configuration.h" +#include "libcamera/internal/ipa_manager.h" +#include "libcamera/internal/media_device.h" +#include "libcamera/internal/pipeline_handler.h" +#include "libcamera/internal/request.h" +#include "libcamera/internal/sequence_sync_helper.h" +#include "libcamera/internal/v4l2_subdevice.h" +#include "libcamera/internal/v4l2_videodevice.h" + +namespace libcamera { + +static const Size ispMaxSize = Size(4416, 3312); +static const Size vicapMaxSize = Size(8192, 8192); + +/* \todo Re-add NV21 and YUV422 and YVU422 after it's fixed in the driver */ +const std::map formatToMediaBus = { + { formats::UYVY, MEDIA_BUS_FMT_YUYV8_2X8 }, + { formats::NV12, MEDIA_BUS_FMT_YUYV8_2X8 }, + { formats::NV16, MEDIA_BUS_FMT_YUYV8_2X8 }, + { formats::NV61, MEDIA_BUS_FMT_YUYV8_2X8 }, + { formats::YUV420, MEDIA_BUS_FMT_YUYV8_1_5X8 }, + { formats::YVU420, MEDIA_BUS_FMT_YUYV8_1_5X8 }, + { formats::R8, MEDIA_BUS_FMT_YUYV8_2X8 }, +}; + +/* \todo Deduplicate this (we have the same thing in rkisp1) */ +const std::map rawFormats = { + { formats::SBGGR8, MEDIA_BUS_FMT_SBGGR8_1X8 }, + { formats::SGBRG8, MEDIA_BUS_FMT_SGBRG8_1X8 }, + { formats::SGRBG8, MEDIA_BUS_FMT_SGRBG8_1X8 }, + { formats::SRGGB8, MEDIA_BUS_FMT_SRGGB8_1X8 }, + { formats::SBGGR10, MEDIA_BUS_FMT_SBGGR10_1X10 }, + { formats::SGBRG10, MEDIA_BUS_FMT_SGBRG10_1X10 }, + { formats::SGRBG10, MEDIA_BUS_FMT_SGRBG10_1X10 }, + { formats::SRGGB10, MEDIA_BUS_FMT_SRGGB10_1X10 }, + { formats::SBGGR12, MEDIA_BUS_FMT_SBGGR12_1X12 }, + { formats::SGBRG12, MEDIA_BUS_FMT_SGBRG12_1X12 }, + { formats::SGRBG12, MEDIA_BUS_FMT_SGRBG12_1X12 }, + { formats::SRGGB12, MEDIA_BUS_FMT_SRGGB12_1X12 }, +}; + +LOG_DEFINE_CATEGORY(RkISP2) + +class PipelineHandlerRkISP2; + +struct RkISP2FrameInfo { + RkISP2FrameInfo(Request *_request, + FrameBuffer *_buffer, + const ControlList &_metadata = ControlList(controls::controls), + bool _metadataProcessed = false) + : request(_request), mainPathBuffer(_buffer), metadataProcessed(_metadataProcessed) + { + metadata.merge(_metadata); + } + + Request *request = nullptr; + FrameBuffer *mainPathBuffer = nullptr; + ControlList metadata = ControlList(controls::controls); + bool metadataProcessed = false; +}; + +struct RkISP2RequestInfo { + Request *request = nullptr; + size_t sequence = 0; + bool sequenceValid = false; +}; + +class RkISP2CameraData : public Camera::Private +{ +public: + RkISP2CameraData(PipelineHandler *pipe, MediaDevice *media) + : Camera::Private(pipe), media_(media), frame_(0) + { + } + + ~RkISP2CameraData() + { + } + + int init(bool usingIsp, CameraSensor *sensor, V4L2VideoDevice *video, + V4L2VideoDevice *rawrd, V4L2Subdevice *isp, + V4L2VideoDevice *mainPath, V4L2VideoDevice *param, + V4L2VideoDevice *stat); + PixelFormat getSensorFormat(unsigned int mbusCode, const Size &size, + const Size &maxSize); + + PipelineHandlerRkISP2 *pipe(); + const PipelineHandlerRkISP2 *pipe() const; + int loadIPA(); + + void computeParamBuffers(unsigned int frame); + + void tryCompleteRequest(Request *request, FrameBuffer *buffer); + void tryCompleteRequest(const ControlList &metadata); + + void vicapBufferReady(FrameBuffer *buffer); + void rawrdBufferReady(FrameBuffer *buffer); + void ispBufferReady(FrameBuffer *buffer); + void statBufferReady(FrameBuffer *buffer); + + void paramsComputed(unsigned int frame, unsigned int bufferId, unsigned int bytesused); + void setSensorControls(unsigned int frame, const ControlList &sensorControls); + void metadataReady(const ControlList &metadata); + + /* These are the buffers that go between the vicap and the isp */ + std::vector> internalBuffers_; + + std::vector> statBuffers_; + std::unique_ptr paramQueue_; + + std::unique_ptr delayedCtrls_; + + /* + * These are the buffers that are half-complete, as in either metadata + * is ready or the frame has been completed from the ISP. Both + * ready-handlers populate and complete from the front of the queue. + * + * The stat-ready handler flushes the queue however, because in general + * more images complete than stats do, so this prevents the queue from + * perpetually growing and making the metadata out of date. + */ + std::deque pendingCompleteBuffers_; + + /* + * This is for tracking sequence numbers of requests for synchronizing + * between stats and params and images for any dropped frames + */ + std::deque pendingCompleteRequests_; + + std::vector ipaBuffers_; + + bool usingIsp_; + bool isRaw_; + + MediaDevice *media_; + CameraSensor *sensor_; + V4L2VideoDevice *video_; + V4L2VideoDevice *rawrd_; + V4L2Subdevice *isp_; + V4L2VideoDevice *mainPath_; + V4L2VideoDevice *param_; + V4L2VideoDevice *stat_; + Stream stream_; + + std::unique_ptr ipa_; + ControlInfoMap ipaControls_; + + /* + * The sensor frame sequence of the last request queued to the pipeline + * handler + */ + unsigned int frame_; + SequenceSyncHelper syncHelper_; +}; + +class RkISP2CameraConfiguration : public CameraConfiguration +{ +public: + RkISP2CameraConfiguration(const RkISP2CameraData *data); + + Status validate() override; + + const V4L2SubdeviceFormat &sensorFormat() { return sensorFormat_; } + +private: + const RkISP2CameraData *data_; + + V4L2SubdeviceFormat sensorFormat_; +}; + +namespace { + +/* + * This many internal buffers (or rather parameter and statistics buffer + * pairs) ensures that the pipeline runs smoothly, without frame drops. + */ +static constexpr unsigned int kRkISP2MinBufferCount = 4; + +} /* namespace */ + +class PipelineHandlerRkISP2 : public PipelineHandler +{ +public: + PipelineHandlerRkISP2(CameraManager *manager); + + std::unique_ptr + generateConfiguration(Camera *camera, + Span roles) override; + + int configure(Camera *camera, CameraConfiguration *config) override; + + int exportFrameBuffers(Camera *camera, Stream *stream, + std::vector> *buffers) override; + + int start(Camera *camera, const ControlList *controls) override; + void stopDevice(Camera *camera) override; + + int queueRequestDevice(Camera *camera, Request *request) override; + + bool match(DeviceEnumerator *enumerator) override; + +private: + RkISP2CameraData *cameraData(Camera *camera) + { + return static_cast(camera->_d()); + } + + int updateControls(RkISP2CameraData *data); + bool createCamera(bool usingIsp); + int processControls(RkISP2CameraData *data, const ControlList &ctrls); + + int allocateBuffers(Camera *camera); + int freeBuffers(Camera *camera); + + Size clampSensorSize(RkISP2CameraData *data, unsigned int mbus, + const Size &maxSize); + + std::shared_ptr media_; + std::unique_ptr sensor_; + std::unique_ptr csi_; + std::unique_ptr cif_; + std::unique_ptr video_; + + std::shared_ptr ispMedia_; + std::unique_ptr rawrd_; + std::unique_ptr isp_; + std::unique_ptr mainPath_; + std::unique_ptr param_; + std::unique_ptr stat_; +}; + +int RkISP2CameraData::init(bool usingIsp, CameraSensor *sensor, + V4L2VideoDevice *video, V4L2VideoDevice *rawrd, + V4L2Subdevice *isp, + V4L2VideoDevice *mainPath, V4L2VideoDevice *param, + V4L2VideoDevice *stat) +{ + usingIsp_ = usingIsp; + sensor_ = sensor; + video_ = video; + rawrd_ = rawrd; + isp_ = isp; + mainPath_ = mainPath; + param_ = param; + stat_ = stat; + + ControlInfoMap::Map ctrls; + + auto &testPatterns = sensor_->testPatternModes(); + if (testPatterns.size()) { + ctrls.emplace(&controls::draft::TestPatternMode, + ControlInfo(testPatterns.front(), testPatterns.back(), + testPatterns.front())); + } else + LOG(RkISP2, Warning) << "Sensor doesn't support any test pattern modes"; + + controlInfo_ = ControlInfoMap(std::move(ctrls), controls::controls); + + video_->bufferReady.connect(this, &RkISP2CameraData::vicapBufferReady); + if (mainPath_) { + rawrd_->bufferReady.connect(this, &RkISP2CameraData::rawrdBufferReady); + mainPath_->bufferReady.connect(this, &RkISP2CameraData::ispBufferReady); + stat_->bufferReady.connect(this, &RkISP2CameraData::statBufferReady); + } + + return 0; +} + +PixelFormat RkISP2CameraData::getSensorFormat(unsigned int mbusCode, + const Size &size, + const Size &maxSize) +{ + std::vector mbusCodes = { mbusCode }; + V4L2SubdeviceFormat format = sensor_->getFormat(mbusCodes, size, maxSize); + const auto &ret = std::find_if(rawFormats.begin(), rawFormats.end(), + [format](const auto &value) { return value.second == format.code; }); + if (ret != rawFormats.end()) + return (*ret).first; + + LOG(RkISP2, Error) << "No raw format supported by sensor"; + return formats::SRGGB10; +} + +PipelineHandlerRkISP2 *RkISP2CameraData::pipe() +{ + return static_cast(Camera::Private::pipe()); +} + +const PipelineHandlerRkISP2 *RkISP2CameraData::pipe() const +{ + return static_cast(Camera::Private::pipe()); +} + +int RkISP2CameraData::loadIPA() +{ + ipa_ = pipe()->createIPA(1, 1); + if (!ipa_) + return -ENOENT; + + ipa_->paramsComputed.connect(this, &RkISP2CameraData::paramsComputed); + ipa_->setSensorControls.connect(this, &RkISP2CameraData::setSensorControls); + ipa_->metadataReady.connect(this, &RkISP2CameraData::metadataReady); + + /* The IPA tuning file is made from the sensor name. */ + std::string ipaTuningFile = + ipa_->configurationFile(sensor_->model() + ".yaml", "uncalibrated.yaml"); + + IPACameraSensorInfo sensorInfo{}; + int ret = sensor_->sensorInfo(&sensorInfo); + if (ret) { + LOG(RkISP2, Error) << "Camera sensor information not available"; + return ret; + } + + ret = ipa_->init({ ipaTuningFile, sensor_->model() }, + sensorInfo, sensor_->controls(), + &ipaControls_); + if (ret < 0) { + LOG(RkISP2, Error) << "IPA initialization failure"; + return ret; + } + + return 0; +} + +void RkISP2CameraData::computeParamBuffers(unsigned int frame) +{ + while (paramQueue_->nextSequence() <= frame) { + if (paramQueue_->empty(BufferQueue::Idle)) { + LOG(RkISP2, Warning) << "Out of param buffers"; + return; + } + + uint32_t paramsSequence; + FrameBuffer *paramBuffer = paramQueue_->front(BufferQueue::Idle); + paramQueue_->prepareBuffer(¶msSequence); + ipa_->computeParams(paramsSequence, paramBuffer->cookie()); + } +} + +void RkISP2CameraData::vicapBufferReady(FrameBuffer *buffer) +{ + Request *request = buffer->request(); + + if (!mainPath_ || isRaw_) { + pipe()->completeBuffer(request, buffer); + pipe()->completeRequest(request); + return; + } + + if (buffer->metadata().status == FrameMetadata::FrameCancelled) + return; + + rawrd_->queueBuffer(buffer); +} + +void RkISP2CameraData::rawrdBufferReady(FrameBuffer *buffer) +{ + if (buffer->metadata().status == FrameMetadata::FrameCancelled) + return; + + video_->queueBuffer(buffer); +} + +void RkISP2CameraData::ispBufferReady(FrameBuffer *buffer) +{ + Request *request = buffer->request(); + + if (buffer->metadata().status == FrameMetadata::FrameCancelled) { + syncHelper_.cancelFrame(); + pipe()->completeBuffer(request, buffer); + pipe()->completeRequest(request); + return; + } + + RkISP2RequestInfo info = pendingCompleteRequests_.front(); + pendingCompleteRequests_.pop_front(); + ASSERT(info.request == buffer->request()); + + int droppedFrames = syncHelper_.gotFrame(info.sequence, buffer->metadata().sequence); + if (droppedFrames > 0) { + LOG(RkISP2, Debug) + << "Dropped frames " << droppedFrames << " expected " + << info.sequence << " got " << buffer->metadata().sequence; + } + + pipe()->completeBuffer(request, buffer); + tryCompleteRequest(request, buffer); +} + +void RkISP2CameraData::statBufferReady(FrameBuffer *buffer) +{ + size_t sequence = buffer->metadata().sequence; + + if (buffer->metadata().status == FrameMetadata::FrameCancelled) + return; + + ipa_->processStats(sequence, buffer->cookie(), + delayedCtrls_->get(sequence)); + + stat_->queueBuffer(buffer); +} + +void RkISP2CameraData::paramsComputed(unsigned int frame, + unsigned int bufferId, + unsigned int bytesused) +{ + FrameBuffer *buffer = paramQueue_->front(BufferQueue::Preparing); + + ASSERT(buffer->cookie() == bufferId); + + buffer->_d()->metadata().planes()[0].bytesused = bytesused; + + int ret = paramQueue_->preparedBuffer(); + if (ret < 0) { + LOG(RkISP2, Error) + << "Failed to queue parameter buffer for frame " + << frame << ": " << strerror(-ret); + } +} + +void RkISP2CameraData::setSensorControls([[maybe_unused]] unsigned int frame, + const ControlList &sensorControls) +{ + delayedCtrls_->push(sensorControls); +} + +void RkISP2CameraData::metadataReady(const ControlList &metadata) +{ + tryCompleteRequest(metadata); +} + +void RkISP2CameraData::tryCompleteRequest(Request *request, FrameBuffer *buffer) +{ + if (pendingCompleteBuffers_.empty()) { + pendingCompleteBuffers_.emplace_back(request, buffer); + return; + } + + RkISP2FrameInfo &info = pendingCompleteBuffers_.front(); + if (info.request != nullptr || info.mainPathBuffer != nullptr || !info.metadataProcessed) { + pendingCompleteBuffers_.emplace_back(request, buffer); + return; + } + + info = pendingCompleteBuffers_.front(); + request->_d()->metadata().merge(info.metadata); + pendingCompleteBuffers_.pop_front(); + + pipe()->completeRequest(request); +} + +void RkISP2CameraData::tryCompleteRequest(const ControlList &metadata) +{ + if (pendingCompleteBuffers_.empty()) { + pendingCompleteBuffers_.emplace_back(nullptr, nullptr, metadata, true); + return; + } + + RkISP2FrameInfo &info = pendingCompleteBuffers_.front(); + if (!info.metadata.empty() || info.metadataProcessed) { + /* + * Generally more metadata complete than images, so flush the + * queue when adding new metadata to to the queue to prevent + * metadata in the queue from becoming too old + */ + pendingCompleteBuffers_.clear(); + pendingCompleteBuffers_.emplace_back(nullptr, nullptr, metadata, true); + return; + } + + Request *request = info.request; + request->_d()->metadata().merge(metadata); + pendingCompleteBuffers_.pop_front(); + + pipe()->completeRequest(request); +} + +RkISP2CameraConfiguration::RkISP2CameraConfiguration(const RkISP2CameraData *data) + : CameraConfiguration(), data_(data) +{ +} + +CameraConfiguration::Status RkISP2CameraConfiguration::validate() +{ + const CameraSensor *sensor = data_->sensor_; + std::vector mbusCodes; + Status status = Valid; + + if (config_.empty()) + return Invalid; + + /* + * Make sure that if a sensor configuration has been requested it + * is valid. + */ + if (sensorConfig) { + if (!sensorConfig->isValid()) { + LOG(RkISP2, Error) + << "Invalid sensor configuration request"; + + return Invalid; + } + + unsigned int bitDepth = sensorConfig->bitDepth; + if (bitDepth != 8 && bitDepth != 10 && bitDepth != 12) { + LOG(RkISP2, Error) + << "Invalid sensor configuration bit depth"; + + return Invalid; + } + } + + /* \todo Support self path */ + if (config_.size() != 1) { + config_.resize(1); + status = Adjusted; + } + + StreamConfiguration &cfg = config_[0]; + + /* \todo Support YUV sensors */ + const PixelFormatInfo &info = PixelFormatInfo::info(cfg.pixelFormat); + bool usingIsp = data_->usingIsp_; + if (usingIsp && info.colourEncoding == PixelFormatInfo::ColourEncodingRAW) + usingIsp = false; + + const Size &maxSize = usingIsp ? ispMaxSize : vicapMaxSize; + + if (!usingIsp) { + if (!rawFormats.count(cfg.pixelFormat)) { + cfg.pixelFormat = formats::SRGGB10; + status = Adjusted; + } + + unsigned int mbusCode = rawFormats.at(cfg.pixelFormat); + auto sizes = sensor->sizes(mbusCode); + + Size bestSize; + for (const Size &s : sizes) { + /* Ignore smaller sizes. */ + if (s.width < cfg.size.width || + s.height < cfg.size.height) + continue; + + /* Make sure the width stays in the limits. */ + if (s.width > maxSize.width) + continue; + + bestSize = s; + break; + } + + if (bestSize.isNull()) { + LOG(RkISP2, Error) << "Unable to find a suitable sensor format"; + return Invalid; + } + + if (bestSize != cfg.size) + status = Adjusted; + cfg.size = bestSize; + + mbusCodes = { mbusCode }; + sensorFormat_ = sensor->getFormat(mbusCodes, cfg.size, maxSize); + + ASSERT(sensorFormat_.code == mbusCode); + ASSERT(sensorFormat_.size == cfg.size); + + return status; + } + + if (!formatToMediaBus.count(cfg.pixelFormat)) { + cfg.pixelFormat = formats::UYVY; + status = Adjusted; + } + + /* \todo Adjust sizes a bit better */ + if (cfg.size > ispMaxSize) { + cfg.size = ispMaxSize; + status = Adjusted; + } + V4L2DeviceFormat format; + format.fourcc = data_->video_->toV4L2PixelFormat(cfg.pixelFormat); + format.size = cfg.size; + + int ret = data_->mainPath_->tryFormat(&format); + if (ret) + return Invalid; + + cfg.bufferCount = 4; + + std::transform(rawFormats.begin(), rawFormats.end(), + std::back_inserter(mbusCodes), + [](const auto &value) { return value.second; }); + sensorFormat_ = sensor->getFormat(mbusCodes, cfg.size, maxSize); + if (sensorFormat_.size.isNull()) + status = Invalid; + + return status; +} + +PipelineHandlerRkISP2::PipelineHandlerRkISP2(CameraManager *manager) + : PipelineHandler(manager) +{ +} + +Size PipelineHandlerRkISP2::clampSensorSize(RkISP2CameraData *data, unsigned int mbus, + const Size &maxSize) +{ + const std::vector &sizes = data->sensor_->sizes(mbus); + + for (auto it = sizes.rbegin(); it != sizes.rend(); ++it) { + if (it->width <= maxSize.width && + it->height <= maxSize.height) + return *it; + } + + return *sizes.begin(); +} + +std::unique_ptr +PipelineHandlerRkISP2::generateConfiguration(Camera *camera, + Span roles) +{ + RkISP2CameraData *data = cameraData(camera); + auto config = std::make_unique(data); + if (roles.empty()) + return config; + + if (roles.size() > 1) { + LOG(RkISP2, Error) << "Too many roles requested"; + return config; + } + + const StreamRole &role = roles[0]; + + bool isRaw = !data->usingIsp_ || role == StreamRole::Raw; + Size maxSize = isRaw ? vicapMaxSize : ispMaxSize; + + Size defaultSize = { 1920, 1080 }; + unsigned int defaultMbusCode = MEDIA_BUS_FMT_SRGGB10_1X10; + PixelFormat rawFormat = data->getSensorFormat(defaultMbusCode, + defaultSize, maxSize); + + /* Enumerate formats */ + std::vector sizes = { { Size(32, 32), maxSize } }; + auto makeStream = [sizes](auto const &pair) { + return std::make_pair(pair.first, sizes); + }; + std::map> streamFormats; + std::transform(formatToMediaBus.begin(), formatToMediaBus.end(), + std::inserter(streamFormats, streamFormats.end()), + makeStream); + std::transform(rawFormats.begin(), rawFormats.end(), + std::inserter(streamFormats, streamFormats.end()), + makeStream); + + StreamFormats formats(streamFormats); + StreamConfiguration cfg(formats); + /* UYVY is always supported by this ISP */ + cfg.pixelFormat = isRaw ? rawFormat : formats::UYVY; + cfg.size = clampSensorSize(data, defaultMbusCode, maxSize); + cfg.colorSpace = isRaw ? ColorSpace::Raw : ColorSpace::Sycc; + cfg.bufferCount = 4; + + config->addConfiguration(cfg); + + config->validate(); + + return config; +} + +int PipelineHandlerRkISP2::configure(Camera *camera, + CameraConfiguration *c) +{ + RkISP2CameraData *data = cameraData(camera); + RkISP2CameraConfiguration *config = + static_cast(c); + /* \todo Support multiple streams */ + StreamConfiguration &cfg = config->at(0); + int ret; + + const PixelFormatInfo &info = PixelFormatInfo::info(cfg.pixelFormat); + data->isRaw_ = info.colourEncoding == PixelFormatInfo::ColourEncodingRAW; + + V4L2SubdeviceFormat format = config->sensorFormat(); + LOG(RkISP2, Debug) << "Configuring sensor with " << format; + + if (config->sensorConfig) + ret = sensor_->applyConfiguration(*config->sensorConfig, + Transform::Identity, &format); + else + ret = sensor_->setFormat(&format); + if (ret < 0) + return ret; + + LOG(RkISP2, Debug) << "Sensor configured with " << format; + + LOG(RkISP2, Debug) << "Configuring CSI with : " << format; + ret = csi_->setFormat(0, &format); + if (ret) + return ret; + + LOG(RkISP2, Debug) << "Configuring VICAP with : " << format; + ret = cif_->setFormat(0, &format); + if (ret) + return ret; + + ret = cif_->setFormat(1, &format); + if (ret) + return ret; + + Size maxSize = data->isRaw_ ? vicapMaxSize : ispMaxSize; + PixelFormat vicapPixelFormat = + data->getSensorFormat(format.code, format.size, maxSize); + + V4L2DeviceFormat vicapOutputFormat; + vicapOutputFormat.fourcc = video_->toV4L2PixelFormat(vicapPixelFormat); + vicapOutputFormat.size = format.size; + + LOG(RkISP2, Debug) << "Configuring VICAP capture node with : " << vicapOutputFormat; + ret = data->video_->setFormat(&vicapOutputFormat); + if (ret) + return ret; + + if (!data->usingIsp_ || data->isRaw_) { + cfg.setStream(&data->stream_); + cfg.stride = vicapOutputFormat.planes[0].bpl; + return 0; + } + + /* + * \todo Figure out if these should go in the pipeline handler + * or camera data + */ + LOG(RkISP2, Debug) << "Configuring rawrd0 with: " << vicapOutputFormat; + ret = rawrd_->setFormat(&vicapOutputFormat); + if (ret) + return ret; + + LOG(RkISP2, Debug) << "Configuring ISP input with: " << format; + ret = isp_->setFormat(0, &format); + if (ret) + return ret; + + format.code = formatToMediaBus.at(cfg.pixelFormat); + format.size = cfg.size; + LOG(RkISP2, Debug) << "Configuring ISP output with: " << format; + ret = isp_->setFormat(5, &format); + if (ret) + return ret; + + V4L2DeviceFormat outputFormat; + outputFormat.fourcc = mainPath_->toV4L2PixelFormat(cfg.pixelFormat); + outputFormat.size = cfg.size; + + LOG(RkISP2, Debug) << "Configuring main path with: " << outputFormat; + ret = mainPath_->setFormat(&outputFormat); + if (ret) + return ret; + + if (outputFormat.size != cfg.size || + outputFormat.fourcc != data->mainPath_->toV4L2PixelFormat(cfg.pixelFormat)) { + LOG(RkISP2, Error) + << "Unable to configure capture in " << cfg.toString(); + return -EINVAL; + } + + cfg.setStream(&data->stream_); + cfg.stride = outputFormat.planes[0].bpl; + + V4L2DeviceFormat paramFormat; + paramFormat.fourcc = V4L2PixelFormat(V4L2_META_FMT_RKISP2_PARAMS); + ret = param_->setFormat(¶mFormat); + if (ret) + return ret; + + V4L2DeviceFormat statFormat; + statFormat.fourcc = V4L2PixelFormat(V4L2_META_FMT_RKISP2_STATS); + ret = stat_->setFormat(&statFormat); + if (ret) + return ret; + + IPACameraSensorInfo sensorInfo; + ret = data->sensor_->sensorInfo(&sensorInfo); + if (ret) + return ret; + + int colorSpaceEncoding = -1; + int colorSpaceRange = -1; + if (cfg.colorSpace) { + colorSpaceEncoding = static_cast(cfg.colorSpace->ycbcrEncoding); + colorSpaceRange = static_cast(cfg.colorSpace->range); + } + + /* Inform IPA of stream configuration and sensor controls. */ + ipa::rkisp2::IPAConfigInfo ipaConfig{ sensorInfo, + data->sensor_->controls(), + colorSpaceEncoding, + colorSpaceRange }; + + ret = data->ipa_->configure(ipaConfig, &data->ipaControls_); + if (ret) { + LOG(RkISP2, Error) << "failed configuring IPA (" << ret << ")"; + return ret; + } + + return updateControls(data); +} + +int PipelineHandlerRkISP2::exportFrameBuffers(Camera *camera, Stream *stream, + std::vector> *buffers) +{ + unsigned int count = stream->configuration().bufferCount; + RkISP2CameraData *data = cameraData(camera); + + if (!data->usingIsp_ || data->isRaw_) + return data->video_->exportBuffers(count, buffers); + + return data->mainPath_->exportBuffers(count, buffers); +} + +int PipelineHandlerRkISP2::start(Camera *camera, + const ControlList *controls) +{ + RkISP2CameraData *data = cameraData(camera); + unsigned int count = data->stream_.configuration().bufferCount; + bool useMP = data->usingIsp_ && !data->isRaw_; + utils::ScopeExitActions actions; + int ret; + + data->frame_ = 0; + + LOG(RkISP2, Debug) << (useMP ? "Using" : "Not using") << " main path"; + + if (useMP) { + /* Allocate buffers for params and stats */ + ret = allocateBuffers(camera); + if (ret) { + LOG(RkISP2, Error) << "Failed to allocate buffers"; + return ret; + } + actions += [&]() { freeBuffers(camera); }; + + /* \todo Support start controls */ + ret = data->ipa_->start(); + if (ret) { + LOG(RkISP2, Error) + << "Failed to start IPA " << camera->id(); + return ret; + } + actions += [&]() { data->ipa_->stop(); }; + + ret = data->param_->streamOn(); + if (ret) { + LOG(RkISP2, Error) + << "Failed to start parameters " << camera->id(); + return ret; + } + actions += [&]() { data->param_->streamOff(); }; + } + + ret = data->video_->importBuffers(kRkISP2MinBufferCount); + if (ret < 0) { + LOG(RkISP2, Error) << "Failed to import buffers to vicap"; + return ret; + } + + actions += [&]() { data->video_->releaseBuffers(); }; + + if (useMP) { + ret = data->rawrd_->importBuffers(count); + if (ret < 0) { + LOG(RkISP2, Error) << "Failed to import buffers to rawrd"; + return ret; + } + + actions += [&]() { data->rawrd_->releaseBuffers(); }; + + ret = data->mainPath_->importBuffers(count); + if (ret < 0) { + LOG(RkISP2, Error) << "Failed to import buffers to main path"; + return ret; + } + + actions += [&]() { data->mainPath_->releaseBuffers(); }; + + auto queueBuffers = [&](const std::vector> &buffers, + V4L2VideoDevice *device, std::string_view name) { + for (const std::unique_ptr &buffer : buffers) { + ret = device->queueBuffer(buffer.get()); + if (ret < 0) { + LOG(RkISP2, Warning) + << "Failed to queue buffer " + << &buffer << " to " << name + << ": " << ret; + } + } + }; + + queueBuffers(data->internalBuffers_, data->video_, "vicap"); + queueBuffers(data->statBuffers_, data->stat_, "stat"); + } + + ret = data->video_->streamOn(); + if (ret < 0) + return ret; + + actions += [&]() { data->video_->streamOff(); }; + + if (useMP) { + ret = data->rawrd_->streamOn(); + if (ret < 0) + return ret; + + actions += [&]() { data->rawrd_->streamOff(); }; + + ret = data->stat_->streamOn(); + if (ret) { + LOG(RkISP2, Error) + << "Failed to start stats " << camera->id(); + return ret; + } + actions += [&]() { data->stat_->streamOff(); }; + + ret = data->mainPath_->streamOn(); + if (ret < 0) + return ret; + + actions += [&]() { data->mainPath_->streamOff(); }; + } + + if (controls) { + ret = processControls(data, *controls); + if (ret < 0) + return ret; + } + + if (useMP) + data->isp_->setFrameStartEnabled(true); + + actions.release(); + return 0; +} + +void PipelineHandlerRkISP2::stopDevice(Camera *camera) +{ + RkISP2CameraData *data = cameraData(camera); + bool useMP = data->usingIsp_ && !data->isRaw_; + + if (useMP) + data->isp_->setFrameStartEnabled(false); + + data->video_->streamOff(); + data->video_->releaseBuffers(); + + if (!useMP) + return; + + data->ipa_->stop(); + + data->rawrd_->streamOff(); + data->rawrd_->releaseBuffers(); + + data->mainPath_->streamOff(); + data->mainPath_->releaseBuffers(); + + data->stat_->streamOff(); + data->stat_->releaseBuffers(); + + data->param_->streamOff(); + + data->internalBuffers_.clear(); + data->statBuffers_.clear(); + freeBuffers(camera); +} + +int PipelineHandlerRkISP2::queueRequestDevice(Camera *camera, Request *request) +{ + RkISP2CameraData *data = cameraData(camera); + FrameBuffer *buffer = request->findBuffer(&data->stream_); + if (!buffer) { + LOG(RkISP2, Error) + << "Attempt to queue request with invalid stream"; + return -ENOENT; + } + + int ret = processControls(data, request->controls()); + if (ret < 0) + return ret; + + if (!data->usingIsp_ || data->isRaw_) + return data->video_->queueBuffer(buffer); + + int correction = data->syncHelper_.correction(); + data->frame_ += correction; + data->syncHelper_.pushCorrection(correction); + + data->pendingCompleteRequests_.push_back({ request, data->frame_, true }); + + data->ipa_->queueRequest(data->frame_, request->controls()); + data->computeParamBuffers(data->frame_); + data->frame_++; + + return data->mainPath_->queueBuffer(buffer); +} + +int PipelineHandlerRkISP2::updateControls(RkISP2CameraData *data) +{ + ControlInfoMap::Map controls; + + /* Add the pipeline handler registered controls to list of camera controls. */ + for (const auto &phControl : data->controlInfo_) + controls[phControl.first] = phControl.second; + + /* Add the IPA registered controls to list of camera controls. */ + for (const auto &ipaControl : data->ipaControls_) + controls[ipaControl.first] = ipaControl.second; + + data->controlInfo_ = ControlInfoMap(std::move(controls), + controls::controls); + + return 0; +} + +bool PipelineHandlerRkISP2::createCamera(bool usingIsp) +{ + std::unique_ptr data = + std::make_unique(this, media_.get()); + if (data->init(usingIsp, sensor_.get(), video_.get(), rawrd_.get(), + isp_.get(), mainPath_.get(), param_.get(), stat_.get())) { + LOG(RkISP2, Error) << "Failed to initialize data"; + return false; + } + + /* Initialize the camera properties. */ + data->properties_ = data->sensor_->properties(); + + const CameraSensorProperties::SensorDelays &delays = data->sensor_->sensorDelays(); + std::unordered_map params = { + { V4L2_CID_ANALOGUE_GAIN, { delays.gainDelay, false } }, + { V4L2_CID_EXPOSURE, { delays.exposureDelay, false } }, + { V4L2_CID_VBLANK, { delays.vblankDelay, true } }, + }; + + if (usingIsp) { + data->delayedCtrls_ = + std::make_unique(data->sensor_->device(), + params); + isp_->frameStart.connect(data->delayedCtrls_.get(), + &DelayedControls::applyControls); + + int ret = data->loadIPA(); + if (ret) { + LOG(RkISP2, Error) << "Failed to load IPA"; + return false; + } + + updateControls(data.get()); + + data->paramQueue_ = + std::make_unique(std::make_unique>(param_.get()), + BufferQueue::PrepareStage, "Params"); + } + + const std::string &id = data->sensor_->id(); + std::set streams{ &data->stream_ }; + std::shared_ptr camera = + Camera::create(std::move(data), id, streams); + registerCamera(std::move(camera)); + + LOG(RkISP2, Debug) + << "RkISP2 device registered " + << (usingIsp ? "with" : "without") << " ISP"; + + return true; +} + +bool PipelineHandlerRkISP2::match(DeviceEnumerator *enumerator) +{ + DeviceMatch dm("rockchip-cif"); + /* \todo Generalize this for the other csi ports */ + /* + * I think we will have one camera per csi port, and then 4 streams + * each that correspond to the 4 channels. Not sure how to handle + * routing to the ISP though. For now we'll just assume one camera. + */ + dm.add("rkcif-mipi2"); + /* \todo Generalize this for the other channels */ + dm.add("rkcif-mipi2-id0"); + dm.add("dw-mipi-csi2rx fdd30000.csi"); + + media_ = acquireMediaDevice(enumerator, dm); + if (!media_) + return false; + + csi_ = V4L2Subdevice::fromEntityName(media_.get(), "dw-mipi-csi2rx fdd30000.csi"); + if (!csi_ || csi_->open() < 0) { + LOG(RkISP2, Error) << "Failed to open csi"; + return false; + } + + cif_ = V4L2Subdevice::fromEntityName(media_.get(), "rkcif-mipi2"); + if (!cif_ || cif_->open() < 0) { + LOG(RkISP2, Error) << "Failed to open cif"; + return false; + } + + /* \todo Support multiple streams */ + video_ = V4L2VideoDevice::fromEntityName(media_.get(), "rkcif-mipi2-id0"); + if (!video_ || video_->open() < 0) { + LOG(RkISP2, Error) << "Failed to open capture device"; + return false; + } + + for (MediaEntity *entity : media_->locateEntities(MEDIA_ENT_F_CAM_SENSOR)) { + LOG(RkISP2, Debug) << "Identified " << entity->name(); + sensor_ = CameraSensorFactoryBase::create(entity); + /* Just get the first sensor for now */ + if (sensor_) + break; + } + + if (!sensor_) { + LOG(RkISP2, Error) << "Failed to find sensor"; + return false; + } + + const GlobalConfiguration &configuration = cameraManager()->_d()->configuration(); + bool usingIsp = configuration.configuration()["pipelines"]["rkisp2"]["isp_enable"].get(true); + if (!usingIsp) { + LOG(RkISP2, Info) << "ISP disabled in configuration file"; + return createCamera(usingIsp); + } + + /* Match ISP */ + + DeviceMatch dmIsp("rkisp2"); + dmIsp.add("rkisp2_isp"); + /* \todo Generalize this for the other channels */ + dmIsp.add("rkisp2_rawrd0"); + /* \todo Support self path */ + dmIsp.add("rkisp2_mainpath"); + + ispMedia_ = acquireMediaDevice(enumerator, dmIsp); + if (!ispMedia_) { + usingIsp = false; + LOG(RkISP2, Debug) << "ISP not found"; + return createCamera(usingIsp); + } + + /* \todo Support the other rawrd nodes */ + rawrd_ = V4L2VideoDevice::fromEntityName(ispMedia_.get(), "rkisp2_rawrd0"); + if (!rawrd_ || rawrd_->open() < 0) { + LOG(RkISP2, Error) << "Failed to open rkisp2 rawrd device"; + return false; + } + + isp_ = V4L2Subdevice::fromEntityName(ispMedia_.get(), "rkisp2_isp"); + if (!isp_ || isp_->open() < 0) { + LOG(RkISP2, Error) << "Failed to open rkisp2 isp"; + return false; + } + + /* \todo Support self path */ + mainPath_ = V4L2VideoDevice::fromEntityName(ispMedia_.get(), "rkisp2_mainpath"); + if (!mainPath_ || mainPath_->open() < 0) { + LOG(RkISP2, Error) << "Failed to open rkisp2 main path"; + return false; + } + + param_ = V4L2VideoDevice::fromEntityName(ispMedia_.get(), "rkisp2_params"); + if (!param_ || param_->open() < 0) { + LOG(RkISP2, Error) << "Failed to open rkisp2 params"; + return false; + } + + stat_ = V4L2VideoDevice::fromEntityName(ispMedia_.get(), "rkisp2_stats"); + if (!stat_ || stat_->open() < 0) { + LOG(RkISP2, Error) << "Failed to open rkisp2 stats"; + return false; + } + + return createCamera(true); +} + +int PipelineHandlerRkISP2::processControls(RkISP2CameraData *data, const ControlList &ctrls) +{ + const auto &testPattern = ctrls.get(controls::draft::TestPatternMode); + if (testPattern) + data->sensor_->setTestPatternMode(static_cast(*testPattern)); + + return 0; +} + +/* This is only called when using the ISP */ +int PipelineHandlerRkISP2::allocateBuffers(Camera *camera) +{ + RkISP2CameraData *data = cameraData(camera); + unsigned int ipaBufferId = 1; + utils::ScopeExitActions actions; + + int ret = data->video_->exportBuffers(kRkISP2MinBufferCount, &data->internalBuffers_); + if (ret < 0) + return ret; + + actions += [&]() { data->video_->releaseBuffers(); }; + + ret = data->stat_->allocateBuffers(kRkISP2MinBufferCount, &data->statBuffers_); + if (ret < 0) + return ret; + + actions += [&]() { data->stat_->releaseBuffers(); }; + + ret = data->paramQueue_->allocateBuffers(kRkISP2MinBufferCount); + if (ret < 0) + return ret; + + actions += [&]() { data->paramQueue_->releaseBuffers(); }; + + auto pushBuffers = [&](const std::vector> &buffers) { + for (const std::unique_ptr &buffer : buffers) { + Span planes = buffer->planes(); + + buffer->setCookie(ipaBufferId++); + data->ipaBuffers_.emplace_back(buffer->cookie(), + std::vector{ planes.begin(), + planes.end() }); + } + }; + + pushBuffers(data->paramQueue_->buffers()); + pushBuffers(data->statBuffers_); + + data->ipa_->mapBuffers(data->ipaBuffers_); + + actions.release(); + return 0; +} + +int PipelineHandlerRkISP2::freeBuffers(Camera *camera) +{ + RkISP2CameraData *data = cameraData(camera); + + std::vector ids; + for (IPABuffer &ipabuf : data->ipaBuffers_) + ids.push_back(ipabuf.id); + + data->ipa_->unmapBuffers(ids); + data->ipaBuffers_.clear(); + + data->paramQueue_->releaseBuffers(); + + return 0; +} + +REGISTER_PIPELINE_HANDLER(PipelineHandlerRkISP2, "rkisp2") + +} /* namespace libcamera */ From patchwork Thu Aug 27 10:50:01 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Paul Elder X-Patchwork-Id: 28128 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id B5D3FBDDFC for ; Thu, 27 Aug 2026 10:50:59 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 7857568494; Thu, 27 Aug 2026 12:50:59 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="geLBZbAB"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id CAB1768476 for ; Thu, 27 Aug 2026 12:50:57 +0200 (CEST) Received: from neptunite.hamster-moth.ts.net (unknown [IPv6:2400:2411:160:2f00:306c:5b5b:3c7f:eabd]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 40015C1; Thu, 27 Aug 2026 12:49:28 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1787827771; bh=LwU47QUkDJUKvTxCsltyLYQegARHTOtCF6u48eSM+bE=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=geLBZbABY0bndewJKoNr1ELYkVPTNU643A6wNFTR/u63SDSKWLBofUA/PTUa8Ew8c 74q8W9aHgT/s1vgTLVTT6Z69zl8YjIYm16x23xwoS3aEVcs5RnjaOfxX5qTKpdxeqD JA/Tf4VQ6RuXrJjLDoNuhqyqMJ4+kWaA1oAWs3WQ= From: Paul Elder To: laurent.pinchart@ideasonboard.com Cc: Paul Elder , michael.riesch@collabora.com, xuhf@rock-chips.com, stefan.klug@ideasonboard.com, kieran.bingham@ideasonboard.com, dan.scally@ideasonboard.com, jacopo.mondi@ideasonboard.com, nicolas.dufresne@collabora.com, mehdi.djait@linux.intel.com, libcamera-devel@lists.libcamera.org Subject: [PATCH v3 08/20] ipa: rkisp2: algo: bls: Implement black level subtraction Date: Thu, 27 Aug 2026 19:50:01 +0900 Message-ID: <20260827105018.2781166-9-paul.elder@ideasonboard.com> X-Mailer: git-send-email 2.47.2 In-Reply-To: <20260827105018.2781166-1-paul.elder@ideasonboard.com> References: <20260827105018.2781166-1-paul.elder@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Implement a static black level subtraction algorithm for the rkisp2 IPA. Signed-off-by: Paul Elder --- No change in v2 --- src/ipa/rkisp2/algorithms/bls.cpp | 154 ++++++++++++++++++++++++++ src/ipa/rkisp2/algorithms/bls.h | 41 +++++++ src/ipa/rkisp2/algorithms/meson.build | 1 + 3 files changed, 196 insertions(+) create mode 100644 src/ipa/rkisp2/algorithms/bls.cpp create mode 100644 src/ipa/rkisp2/algorithms/bls.h diff --git a/src/ipa/rkisp2/algorithms/bls.cpp b/src/ipa/rkisp2/algorithms/bls.cpp new file mode 100644 index 000000000000..316c057d1046 --- /dev/null +++ b/src/ipa/rkisp2/algorithms/bls.cpp @@ -0,0 +1,154 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2026, Ideas On Board + * + * RkISP2 Black Level Subtraction control + */ + +#include "bls.h" + +#include + +#include + +#include + +/** + * \file bls.h + */ + +namespace libcamera { + +namespace ipa::rkisp2::algorithms { + +/** + * \class BlackLevelSubtraction + * \brief RkISP2 Black Level Subtraction control + * + * The pixels output by the camera normally include a black level, because + * sensors do not always report a signal level of '0' for black. Pixels at or + * below this level should be considered black. To achieve that, the RkISP2 BLS + * algorithm subtracts a configurable offset from all pixels. + * + * The black level can be measured at runtime from an optical dark region of the + * camera sensor, or measured during the camera tuning process. The first option + * isn't currently supported. + * + * \todo Add support for black level in the metadata so that we can capture + * proper raw images for tuning + */ + +LOG_DEFINE_CATEGORY(RkISP2Bls) + +/** + * \copydoc libcamera::ipa::Algorithm::init + */ +int BlackLevelSubtraction::init(IPAContext &context, const ValueNode &tuningData) +{ + std::optional levelRed = tuningData["R"].get(); + std::optional levelGreenR = tuningData["Gr"].get(); + std::optional levelGreenB = tuningData["Gb"].get(); + std::optional levelBlue = tuningData["B"].get(); + bool tuningHasLevels = levelRed && levelGreenR && levelGreenB && levelBlue; + + auto blackLevel = context.camHelper->blackLevel(); + if (!blackLevel) { + /* + * Not all camera sensor helpers have been updated with black + * levels. Print a warning and fall back to the levels from the + * tuning data to preserve backward compatibility. This should + * be removed once all helpers provide the data. + */ + LOG(RkISP2Bls, Warning) + << "No black levels provided by camera sensor helper" + << ", please fix"; + + blackLevelRed_ = levelRed.value_or(4096); + blackLevelGreenR_ = levelGreenR.value_or(4096); + blackLevelGreenB_ = levelGreenB.value_or(4096); + blackLevelBlue_ = levelBlue.value_or(4096); + } else if (tuningHasLevels) { + /* + * If black levels are provided in the tuning file, use them to + * avoid breaking existing camera tuning. This is deprecated and + * will be removed. + */ + LOG(RkISP2Bls, Warning) + << "Deprecated: black levels overwritten by tuning file"; + + blackLevelRed_ = *levelRed; + blackLevelGreenR_ = *levelGreenR; + blackLevelGreenB_ = *levelGreenB; + blackLevelBlue_ = *levelBlue; + } else { + blackLevelRed_ = *blackLevel; + blackLevelGreenR_ = *blackLevel; + blackLevelGreenB_ = *blackLevel; + blackLevelBlue_ = *blackLevel; + } + + LOG(RkISP2Bls, Debug) + << "Black levels: red " << blackLevelRed_ + << ", green (red) " << blackLevelGreenR_ + << ", green (blue) " << blackLevelGreenB_ + << ", blue " << blackLevelBlue_; + + return 0; +} + +int BlackLevelSubtraction::configure([[maybe_unused]] IPAContext &context, + [[maybe_unused]] const IPACameraSensorInfo &configInfo) +{ + /* \todo Save the cfa */ + + return 0; +} + +/** + * \copydoc libcamera::ipa::Algorithm::prepare + */ +void BlackLevelSubtraction::prepare(IPAContext &context, + const uint32_t frame, + [[maybe_unused]] IPAFrameContext &frameContext, + RkISP2Params *params) +{ + if (context.configuration.raw) + return; + + if (frame > 1) + return; + + auto config = params->block(); + config.setEnabled(true); + + config->enable_auto = 0; + + /* Scale down to the 12-bit black levels used by the BLS block. */ + /* \todo Handle cfa properly */ + config->bls_fixed_val.a = blackLevelRed_ >> 4; + config->bls_fixed_val.b = blackLevelGreenR_ >> 4; + config->bls_fixed_val.c = blackLevelGreenB_ >> 4; + config->bls_fixed_val.d = blackLevelBlue_ >> 4; +} + +/** + * \copydoc libcamera::ipa::Algorithm::process + */ +void BlackLevelSubtraction::process([[maybe_unused]] IPAContext &context, + [[maybe_unused]] const uint32_t frame, + [[maybe_unused]] IPAFrameContext &frameContext, + [[maybe_unused]] const RkISP2Stats *stats, + ControlList &metadata) +{ + metadata.set(controls::SensorBlackLevels, + { static_cast(blackLevelRed_), + static_cast(blackLevelGreenR_), + static_cast(blackLevelGreenB_), + static_cast(blackLevelBlue_) }); +} + +REGISTER_IPA_ALGORITHM(BlackLevelSubtraction, "BlackLevelSubtraction") + +} /* namespace ipa::rkisp2::algorithms */ + +} /* namespace libcamera */ diff --git a/src/ipa/rkisp2/algorithms/bls.h b/src/ipa/rkisp2/algorithms/bls.h new file mode 100644 index 000000000000..31f3fe21c8c7 --- /dev/null +++ b/src/ipa/rkisp2/algorithms/bls.h @@ -0,0 +1,41 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2026, Ideas On Board + * + * RkISP2 Black Level Subtraction control + */ + +#pragma once + +#include "algorithm.h" + +namespace libcamera { + +namespace ipa::rkisp2::algorithms { + +class BlackLevelSubtraction : public Algorithm +{ +public: + BlackLevelSubtraction() = default; + ~BlackLevelSubtraction() = default; + + int init(IPAContext &context, const ValueNode &tuningData) override; + int configure(IPAContext &context, + const IPACameraSensorInfo &configInfo) override; + void prepare(IPAContext &context, const uint32_t frame, + IPAFrameContext &frameContext, + RkISP2Params *params) override; + void process(IPAContext &context, const uint32_t frame, + IPAFrameContext &frameContext, + const RkISP2Stats *stats, + ControlList &metadata) override; + +private: + int16_t blackLevelRed_; + int16_t blackLevelGreenR_; + int16_t blackLevelGreenB_; + int16_t blackLevelBlue_; +}; + +} /* namespace ipa::rkisp2::algorithms */ +} /* namespace libcamera */ diff --git a/src/ipa/rkisp2/algorithms/meson.build b/src/ipa/rkisp2/algorithms/meson.build index 4bb81e48eb01..36996918b39e 100644 --- a/src/ipa/rkisp2/algorithms/meson.build +++ b/src/ipa/rkisp2/algorithms/meson.build @@ -1,5 +1,6 @@ # SPDX-License-Identifier: CC0-1.0 rkisp2_ipa_algorithms = files([ + 'bls.cpp', ]) From patchwork Thu Aug 27 10:50:02 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Paul Elder X-Patchwork-Id: 28129 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id AFDDAC333C for ; Thu, 27 Aug 2026 10:51:03 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 21F2068497; Thu, 27 Aug 2026 12:51:03 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="eJAoFhWV"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 76C8F68487 for ; Thu, 27 Aug 2026 12:51:01 +0200 (CEST) Received: from neptunite.hamster-moth.ts.net (unknown [IPv6:2400:2411:160:2f00:306c:5b5b:3c7f:eabd]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id DDD905B3; Thu, 27 Aug 2026 12:49:31 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1787827775; bh=YNbTCefrBPiHqnhsnE0xUdu8292o2jhctnckMbe/09Y=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=eJAoFhWV3f59oGLH3l/dgOD1gtr0hji3IxByv1t4MYP/ILxx3JzpsL3NK+g2wYwqS h1JH5TsIE35Qkotx1oQi9wMygUy4Yv03zPbpIzjVYak36mpNuRHeOOtAiiX9lPzG6d D1dwPmmUpptV1MOHyfwYRM0zlJA5tdxXE3oacBDk= From: Paul Elder To: laurent.pinchart@ideasonboard.com Cc: Paul Elder , michael.riesch@collabora.com, xuhf@rock-chips.com, stefan.klug@ideasonboard.com, kieran.bingham@ideasonboard.com, dan.scally@ideasonboard.com, jacopo.mondi@ideasonboard.com, nicolas.dufresne@collabora.com, mehdi.djait@linux.intel.com, libcamera-devel@lists.libcamera.org Subject: [PATCH v3 09/20] ipa: rkisp2: algo: awb: Implement automatic white balance control Date: Thu, 27 Aug 2026 19:50:02 +0900 Message-ID: <20260827105018.2781166-10-paul.elder@ideasonboard.com> X-Mailer: git-send-email 2.47.2 In-Reply-To: <20260827105018.2781166-1-paul.elder@ideasonboard.com> References: <20260827105018.2781166-1-paul.elder@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Implement auto white balance control algorithm for the rkisp2 IPA. This leverages the libipa awb, so it supports baysian awb, grey world awb, as well as manual white balance. Signed-off-by: Paul Elder --- Changes in v2: - rebased on libipa rework - use extensible stats --- src/ipa/rkisp2/algorithms/awb.cpp | 211 ++++++++++++++++++++++++++ src/ipa/rkisp2/algorithms/awb.h | 56 +++++++ src/ipa/rkisp2/algorithms/meson.build | 1 + 3 files changed, 268 insertions(+) create mode 100644 src/ipa/rkisp2/algorithms/awb.cpp create mode 100644 src/ipa/rkisp2/algorithms/awb.h diff --git a/src/ipa/rkisp2/algorithms/awb.cpp b/src/ipa/rkisp2/algorithms/awb.cpp new file mode 100644 index 000000000000..e3b284a2d79f --- /dev/null +++ b/src/ipa/rkisp2/algorithms/awb.cpp @@ -0,0 +1,211 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2026, Ideas On Board + * + * AWB control algorithm + */ + +#include "awb.h" + +#include + +#include + +#include + +#include "libcamera/internal/vector.h" + +/** + * \file awb.h + */ + +namespace libcamera { + +namespace ipa::rkisp2::algorithms { + +LOG_DEFINE_CATEGORY(RkISP2Awb) + +class RkISP2AwbStats final : public AwbStats +{ +public: + RkISP2AwbStats() = default; + RkISP2AwbStats(const RGB means) + { + rgbMeans_[0] = means.r(); + rgbMeans_[1] = means.g(); + rgbMeans_[2] = means.b(); + + rg_ = rgbMeans_.r(); + bg_ = rgbMeans_.b(); + } + + double computeColourError(const RGB &gains) const override + { + /* + * Compute the sum of the squared colour error (non-greyness) as + * it appears in the log likelihood equation. + */ + double deltaR = gains.r() * rg_ - 1.0; + double deltaB = gains.b() * bg_ - 1.0; + double delta2 = deltaR * deltaR + deltaB * deltaB; + + return delta2; + } + + RGB rgbMeans() const override + { + return rgbMeans_; + } + + bool valid() const override + { + /* Minimum mean value below which AWB can't operate. */ + constexpr double minValue = 0.2; + + return rgbMeans_.r() > minValue || rgbMeans_.g() > minValue || + rgbMeans_.b() > minValue; + } + +private: + RGB rgbMeans_; + double rg_; + double bg_; +}; + +namespace { + +} /* namespace */ + +/** + * \class Awb + * \brief Manage the white balance with automatic and manual controls + */ + +Awb::Awb() +{ +} + +/** + * \copydoc libcamera::ipa::Algorithm::init + */ +int Awb::init(IPAContext &context, const ValueNode &tuningData) +{ + return awbAlgo_.init(tuningData, context.ctrlMap); +} + +int Awb::configure(IPAContext &context, + const IPACameraSensorInfo &configInfo) +{ + awbAlgo_.configure(context.activeState.awb); + + context.configuration.awb.measureWindow.h_offs = 0; + context.configuration.awb.measureWindow.v_offs = 0; + /* + * Unlike ae lite, this seems to still work when height == full window + * height + */ + context.configuration.awb.measureWindow.h_size = configInfo.outputSize.width / 15; + context.configuration.awb.measureWindow.v_size = configInfo.outputSize.height / 15; + + return 0; +} + +/** + * \copydoc libcamera::ipa::Algorithm::queueRequest + */ +void Awb::queueRequest(IPAContext &context, const uint32_t frame, + IPAFrameContext &frameContext, + const ControlList &controls) +{ + awbAlgo_.queueRequest(context.activeState.awb, frame, frameContext.awb, + controls); +} + +/** + * \copydoc libcamera::ipa::Algorithm::prepare + */ +void Awb::prepare([[maybe_unused]] IPAContext &context, + [[maybe_unused]] const uint32_t frame, + IPAFrameContext &frameContext, RkISP2Params *params) +{ + awbAlgo_.prepare(context.activeState.awb, frameContext.awb); + + auto gainConfig = params->block(); + gainConfig.setEnabled(true); + + RGB gains = frameContext.awb.gains; + + /* \todo Use quantized class */ + gainConfig->gains[0].gb = std::clamp(256 * 1.0, 0, 0x3fff); + gainConfig->gains[0].b = std::clamp(256 * gains.b(), 0, 0x3fff); + gainConfig->gains[0].r = std::clamp(256 * gains.r(), 0, 0x3fff); + gainConfig->gains[0].gr = std::clamp(256 * 1.0, 0, 0x3fff); + + auto measConfig = params->block(); + measConfig.setEnabled(true); + + measConfig->meas_window = context.configuration.awb.measureWindow; + struct rkisp2_isp_awb_color_quad minLimits = { 0, 0, 0, 0 }; + struct rkisp2_isp_awb_color_quad maxLimits = { 255, 255, 255, 255 }; + measConfig->limits[0] = minLimits; + measConfig->limits[1] = maxLimits; + + for (unsigned int i = 0; i < RKISP2_ISP_AWB_COUNTS_SIZE; i++) + measConfig->weights[i] = 0x20; +} + +/** + * \copydoc libcamera::ipa::Algorithm::process + */ +void Awb::process([[maybe_unused]] IPAContext &context, + const uint32_t frame, + IPAFrameContext &frameContext, + const RkISP2Stats *stats, + ControlList &metadata) +{ + /* The first frame has no stats so skip processing */ + if (frame < 1) + return; + + RkISP2AwbStats awbStats = calculateRgbMeans(frameContext, stats); + + awbAlgo_.process(context.activeState.awb, frameContext.awb, awbStats, + frameContext.lux.lux, metadata); +} + +RkISP2AwbStats Awb::calculateRgbMeans([[maybe_unused]] const IPAFrameContext &frameContext, + const RkISP2Stats *stats) const +{ + auto awb = stats->block(); + + if (!awb->done) { + LOG(RkISP2Awb, Error) << "No awb stats"; + return {}; + } + + std::array counts; + RGB means; + + for (size_t i = 0; i < RKISP2_ISP_AWB_COUNTS_SIZE; i++) + counts[i] = static_cast(awb->counts_r[i]); + means.r() = std::accumulate(counts.begin(), counts.end(), 0) / counts.size(); + + for (size_t i = 0; i < RKISP2_ISP_AWB_COUNTS_SIZE; i++) + counts[i] = static_cast(awb->counts_g[i]); + means.g() = std::accumulate(counts.begin(), counts.end(), 0) / counts.size(); + + for (size_t i = 0; i < RKISP2_ISP_AWB_COUNTS_SIZE; i++) + counts[i] = static_cast(awb->counts_b[i]); + means.b() = std::accumulate(counts.begin(), counts.end(), 0) / counts.size(); + + for (size_t i = 0; i < RKISP2_ISP_AWB_COUNTS_SIZE; i++) + counts[i] = static_cast(awb->counts_w[i]); + + return RkISP2AwbStats(means); +} + +REGISTER_IPA_ALGORITHM(Awb, "Awb") + +} /* namespace ipa::rkisp2::algorithms */ + +} /* namespace libcamera */ diff --git a/src/ipa/rkisp2/algorithms/awb.h b/src/ipa/rkisp2/algorithms/awb.h new file mode 100644 index 000000000000..7f8393339654 --- /dev/null +++ b/src/ipa/rkisp2/algorithms/awb.h @@ -0,0 +1,56 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2026, Ideas On Board + * + * AWB control algorithm + */ + +#pragma once + +#include + +#include + +#include "libcamera/internal/value_node.h" + +#include "libipa/awb.h" +#include "libipa/fixedpoint.h" + +#include "algorithm.h" +#include "ipa_context.h" +#include "params.h" + +namespace libcamera { + +namespace ipa::rkisp2::algorithms { + +class RkISP2AwbStats; + +class Awb : public Algorithm +{ +public: + Awb(); + ~Awb() = default; + + int init(IPAContext &context, const ValueNode &tuningData) override; + int configure(IPAContext &context, const IPACameraSensorInfo &configInfo) override; + void queueRequest(IPAContext &context, const uint32_t frame, + IPAFrameContext &frameContext, + const ControlList &controls) override; + void prepare(IPAContext &context, const uint32_t frame, + IPAFrameContext &frameContext, + RkISP2Params *params) override; + void process(IPAContext &context, const uint32_t frame, + IPAFrameContext &frameContext, + const RkISP2Stats *stats, + ControlList &metadata) override; + +private: + RkISP2AwbStats calculateRgbMeans(const IPAFrameContext &frameContext, + const RkISP2Stats *stats) const; + + AwbAlgorithm> awbAlgo_; +}; + +} /* namespace ipa::rkisp2::algorithms */ +} /* namespace libcamera */ diff --git a/src/ipa/rkisp2/algorithms/meson.build b/src/ipa/rkisp2/algorithms/meson.build index 36996918b39e..30133ac4fa68 100644 --- a/src/ipa/rkisp2/algorithms/meson.build +++ b/src/ipa/rkisp2/algorithms/meson.build @@ -1,6 +1,7 @@ # SPDX-License-Identifier: CC0-1.0 rkisp2_ipa_algorithms = files([ + 'awb.cpp', 'bls.cpp', ]) From patchwork Thu Aug 27 10:50:03 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Paul Elder X-Patchwork-Id: 28130 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 75215BDDFC for ; Thu, 27 Aug 2026 10:51:06 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 0CBF36849A; Thu, 27 Aug 2026 12:51:06 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="WOEiPZfN"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 1BF1468487 for ; Thu, 27 Aug 2026 12:51:05 +0200 (CEST) Received: from neptunite.hamster-moth.ts.net (unknown [IPv6:2400:2411:160:2f00:306c:5b5b:3c7f:eabd]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 86EDDC1; Thu, 27 Aug 2026 12:49:35 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1787827778; bh=wqX5JnGfj3rDxQorHr7PaSBPNBpbYJodKcqENNfIKCM=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=WOEiPZfNBonP4eJWtl2uF10ztgrOwXwptv3UALGVjfqpaH4gZ9GAVDnDWBo4Grwjf r4F7vXowXi2rpWIW3LkgBpYiwjIzaRdYUUDRdWKDGXoVvuY4yANX9Ag6jKzBKaqxN1 HGJYQmHxExjYuP5ohTcjw0S8uJWbyiqpEFiNgyI8= From: Paul Elder To: laurent.pinchart@ideasonboard.com Cc: Paul Elder , michael.riesch@collabora.com, xuhf@rock-chips.com, stefan.klug@ideasonboard.com, kieran.bingham@ideasonboard.com, dan.scally@ideasonboard.com, jacopo.mondi@ideasonboard.com, nicolas.dufresne@collabora.com, mehdi.djait@linux.intel.com, libcamera-devel@lists.libcamera.org Subject: [PATCH v3 10/20] ipa: rkisp2: algo: agc: Implement automatic gain control Date: Thu, 27 Aug 2026 19:50:03 +0900 Message-ID: <20260827105018.2781166-11-paul.elder@ideasonboard.com> X-Mailer: git-send-email 2.47.2 In-Reply-To: <20260827105018.2781166-1-paul.elder@ideasonboard.com> References: <20260827105018.2781166-1-paul.elder@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Implement auto gain control algorithm for the rkisp2 IPA. This leverages the libipa agc, and supports manual gain and exposure control as well. It currently does not function in raw mode, meaning that there is no manual gain and exposure control in raw mode, as no IPA is created when running in raw mode for the rkisp2. Signed-off-by: Paul Elder --- todo: support manual gain and exposure in raw mode Changes in v2: - rebased on libipa rework - use hist lite instead of big0 - use extensible stats --- src/ipa/rkisp2/algorithms/agc.cpp | 479 ++++++++++++++++++++++++++ src/ipa/rkisp2/algorithms/agc.h | 58 ++++ src/ipa/rkisp2/algorithms/meson.build | 1 + 3 files changed, 538 insertions(+) create mode 100644 src/ipa/rkisp2/algorithms/agc.cpp create mode 100644 src/ipa/rkisp2/algorithms/agc.h diff --git a/src/ipa/rkisp2/algorithms/agc.cpp b/src/ipa/rkisp2/algorithms/agc.cpp new file mode 100644 index 000000000000..4a7f111ccf0d --- /dev/null +++ b/src/ipa/rkisp2/algorithms/agc.cpp @@ -0,0 +1,479 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2026, Ideas On Board Oy. + * + * AGC/AEC mean-based control algorithm + */ + +#include "agc.h" + +#include +#include +#include +#include +#include +#include + +#include +#include +#include + +#include +#include +#include + +#include "libipa/histogram.h" + +/** + * \file agc.h + */ + +namespace libcamera { + +using namespace std::literals::chrono_literals; + +namespace ipa::rkisp2::algorithms { + +/** + * \class Agc + * \brief A mean-based auto-exposure algorithm + */ + +LOG_DEFINE_CATEGORY(RkISP2Agc) + +int Agc::init(IPAContext &context, const ValueNode &tuningData) +{ + int ret = parseTuningData(tuningData); + if (ret) + return ret; + + context.ctrlMap[&controls::ExposureTimeMode] = + ControlInfo({ { ControlValue(controls::ExposureTimeModeAuto), + ControlValue(controls::ExposureTimeModeManual) } }, + ControlValue(controls::ExposureTimeModeAuto)); + context.ctrlMap[&controls::AnalogueGainMode] = + ControlInfo({ { ControlValue(controls::AnalogueGainModeAuto), + ControlValue(controls::AnalogueGainModeManual) } }, + ControlValue(controls::AnalogueGainModeAuto)); + context.ctrlMap[&controls::ExposureValue] = ControlInfo(-8.0f, 8.0f, 0.0f); + /* \todo Support AnalogueGain and ExposureTime in raw mode */ + context.ctrlMap.merge(controls()); + + return 0; +} + +/** + * \brief Configure the AGC given a configInfo + * \param[in] context The shared IPA context + * \param[in] configInfo The IPA configuration data + * + * \return 0 + */ +int Agc::configure(IPAContext &context, const IPACameraSensorInfo &configInfo) +{ + context.configuration.agc.measureWindow.h_offs = 0; + context.configuration.agc.measureWindow.v_offs = 0; + context.configuration.agc.measureWindow.h_size = (configInfo.outputSize.width / 5); + /* + * ae lite needs the -2 because the total window height must be + * divisible by 2, and it cannot be equal to or greater than the frame + * size, or else the hardware hangs + * + * \todo Move this to the kernel? + * \todo Check if hist lite also needs this + */ + context.configuration.agc.measureWindow.v_size = (configInfo.outputSize.height / 5) - 2; + + context.configuration.agc.measureWindow15.h_size = (configInfo.outputSize.width / 15); + context.configuration.agc.measureWindow15.v_size = (configInfo.outputSize.height / 15) - 2; + + /* Configure the default exposure and gain. */ + context.activeState.agc.automatic.gain = context.configuration.sensor.minAnalogueGain; + context.activeState.agc.automatic.exposure = + 10ms / context.configuration.sensor.lineDuration; + context.activeState.agc.automatic.quantizationGain = 1.0; + context.activeState.agc.manual.gain = context.activeState.agc.automatic.gain; + context.activeState.agc.manual.exposure = context.activeState.agc.automatic.exposure; + context.activeState.agc.autoExposureEnabled = true; + context.activeState.agc.autoGainEnabled = true; + context.activeState.agc.exposureValue = 0.0; + + /* Limit the frame duration to match current initialisation */ + ControlInfo &frameDurationLimits = context.ctrlMap[&controls::FrameDurationLimits]; + context.activeState.agc.minFrameDuration = std::chrono::microseconds(frameDurationLimits.min().get()); + context.activeState.agc.maxFrameDuration = std::chrono::microseconds(frameDurationLimits.max().get()); + + AgcMeanLuminance::configure(context.configuration.sensor.lineDuration, + context.camHelper.get()); + + setLimits(context.configuration.sensor.minExposureTime, + context.configuration.sensor.maxExposureTime, + context.configuration.sensor.minAnalogueGain, + context.configuration.sensor.maxAnalogueGain, {}); + + context.activeState.agc.automatic.yTarget = effectiveYTarget(); + + resetFrameCount(); + + return 0; +} + +/** + * \copydoc libcamera::ipa::Algorithm::queueRequest + */ +void Agc::queueRequest(IPAContext &context, + [[maybe_unused]] const uint32_t frame, + IPAFrameContext &frameContext, + const ControlList &controls) +{ + auto &agc = context.activeState.agc; + + const auto &aeEnable = controls.get(controls::ExposureTimeMode); + if (aeEnable && + (*aeEnable == controls::ExposureTimeModeAuto) != agc.autoExposureEnabled) { + agc.autoExposureEnabled = (*aeEnable == controls::ExposureTimeModeAuto); + + LOG(RkISP2Agc, Debug) + << (agc.autoExposureEnabled ? "Enabling" : "Disabling") + << " AGC (exposure)"; + + /* + * If we go from auto -> manual with no manual control + * set, use the last computed value, which we don't + * know until prepare() so save this information. + * + * \todo Check the previous frame at prepare() time + * instead of saving a flag here + */ + if (!agc.autoExposureEnabled && !controls.get(controls::ExposureTime)) + frameContext.agc.autoExposureModeChange = true; + } + + const auto &agEnable = controls.get(controls::AnalogueGainMode); + if (agEnable && + (*agEnable == controls::AnalogueGainModeAuto) != agc.autoGainEnabled) { + agc.autoGainEnabled = (*agEnable == controls::AnalogueGainModeAuto); + + LOG(RkISP2Agc, Debug) + << (agc.autoGainEnabled ? "Enabling" : "Disabling") + << " AGC (gain)"; + /* + * If we go from auto -> manual with no manual control + * set, use the last computed value, which we don't + * know until prepare() so save this information. + */ + if (!agc.autoGainEnabled && !controls.get(controls::AnalogueGain)) + frameContext.agc.autoGainModeChange = true; + } + + const auto &exposure = controls.get(controls::ExposureTime); + if (exposure && !agc.autoExposureEnabled) { + agc.manual.exposure = *exposure * 1.0us + / context.configuration.sensor.lineDuration; + + LOG(RkISP2Agc, Debug) + << "Set exposure to " << agc.manual.exposure; + } + + const auto &gain = controls.get(controls::AnalogueGain); + if (gain && !agc.autoGainEnabled) { + agc.manual.gain = *gain; + + LOG(RkISP2Agc, Debug) << "Set gain to " << agc.manual.gain; + } + + frameContext.agc.autoExposureEnabled = agc.autoExposureEnabled; + frameContext.agc.autoGainEnabled = agc.autoGainEnabled; + + if (!frameContext.agc.autoExposureEnabled) + frameContext.agc.exposure = agc.manual.exposure; + if (!frameContext.agc.autoGainEnabled) + frameContext.agc.gain = agc.manual.gain; + + if (!frameContext.agc.autoExposureEnabled && + !frameContext.agc.autoGainEnabled) + frameContext.agc.quantizationGain = 1.0; + + const auto &exposureValue = controls.get(controls::ExposureValue); + if (exposureValue) + agc.exposureValue = *exposureValue; + frameContext.agc.exposureValue = agc.exposureValue; + + const auto &frameDurationLimits = controls.get(controls::FrameDurationLimits); + if (frameDurationLimits) { + /* Limit the control value to the limits in ControlInfo */ + ControlInfo &limits = context.ctrlMap[&controls::FrameDurationLimits]; + int64_t minFrameDuration = + std::clamp((*frameDurationLimits).front(), + limits.min().get(), + limits.max().get()); + int64_t maxFrameDuration = + std::clamp((*frameDurationLimits).back(), + limits.min().get(), + limits.max().get()); + + agc.minFrameDuration = std::chrono::microseconds(minFrameDuration); + agc.maxFrameDuration = std::chrono::microseconds(maxFrameDuration); + } + frameContext.agc.minFrameDuration = agc.minFrameDuration; + frameContext.agc.maxFrameDuration = agc.maxFrameDuration; +} + +/** + * \copydoc libcamera::ipa::Algorithm::prepare + */ +void Agc::prepare(IPAContext &context, [[maybe_unused]] const uint32_t frame, + [[maybe_unused]] IPAFrameContext &frameContext, RkISP2Params *params) +{ + uint32_t activeAutoExposure = context.activeState.agc.automatic.exposure; + double activeAutoGain = context.activeState.agc.automatic.gain; + double activeAutoQGain = context.activeState.agc.automatic.quantizationGain; + + /* Populate exposure and gain in auto mode */ + if (frameContext.agc.autoExposureEnabled) { + frameContext.agc.exposure = activeAutoExposure; + frameContext.agc.quantizationGain = activeAutoQGain; + } + if (frameContext.agc.autoGainEnabled) { + frameContext.agc.gain = activeAutoGain; + frameContext.agc.quantizationGain = activeAutoQGain; + } + + /* + * Populate manual exposure and gain from the active auto values when + * transitioning from auto to manual + */ + if (!frameContext.agc.autoExposureEnabled && frameContext.agc.autoExposureModeChange) { + context.activeState.agc.manual.exposure = activeAutoExposure; + frameContext.agc.exposure = activeAutoExposure; + } + if (!frameContext.agc.autoGainEnabled && frameContext.agc.autoGainModeChange) { + context.activeState.agc.manual.gain = activeAutoGain; + frameContext.agc.gain = activeAutoGain; + frameContext.agc.quantizationGain = activeAutoQGain; + } + + frameContext.agc.yTarget = context.activeState.agc.automatic.yTarget; + + if (frame > 1) + return; + + /* + * Configure the AEC measurements. Set the window, measure + * continuously, and estimate Y as (R + G + B) x (85/256). + */ + auto aeLiteConfig = params->block(); + aeLiteConfig.setEnabled(true); + + aeLiteConfig->window_num = 1; + aeLiteConfig->meas_window = context.configuration.agc.measureWindow; + + auto hstConfig = params->block(); + hstConfig.setEnabled(true); + + hstConfig->window_num = 0; + /* \todo choose this based on the bitdepth */ + hstConfig->data_sel = RKISP2_ISP_HISTOGRAM_DATA_SEL_9_2; + hstConfig->mode = RKISP2_ISP_HISTOGRAM_MODE_Y_HISTOGRAM; + /* waterline means to exclude everything above this value */ + hstConfig->waterline = 0x0; + hstConfig->stepsize = 0; + hstConfig->coeffs.r = 0x21; + hstConfig->coeffs.g = 0x20; + hstConfig->coeffs.b = 0x0d; + + hstConfig->meas_window = context.configuration.agc.measureWindow15; + + /* \todo Support configuring the weights */ + for (size_t i = 0; i < RKISP2_ISP_HIST_WEIGHT_GRIDS_SIZE_BIG; i++) + hstConfig->weights[i] = 0x20; + + auto hstConfigLite = params->block(); + hstConfigLite.setEnabled(false); +} + +void Agc::fillMetadata(IPAContext &context, IPAFrameContext &frameContext, + ControlList &metadata, [[maybe_unused]] const RkISP2Stats *stats) +{ + utils::Duration exposureTime = context.configuration.sensor.lineDuration + * frameContext.sensor.exposure; + metadata.set(controls::AnalogueGain, frameContext.sensor.gain); + metadata.set(controls::ExposureTime, exposureTime.get()); + metadata.set(controls::FrameDuration, frameContext.agc.frameDuration.get()); + metadata.set(controls::ExposureTimeMode, + frameContext.agc.autoExposureEnabled + ? controls::ExposureTimeModeAuto + : controls::ExposureTimeModeManual); + metadata.set(controls::AnalogueGainMode, + frameContext.agc.autoGainEnabled + ? controls::AnalogueGainModeAuto + : controls::AnalogueGainModeManual); + + metadata.set(controls::ExposureValue, frameContext.agc.exposureValue); +} + +double Agc::estimateLuminance(double gain) const +{ + /* + * \todo Enforce this check, since lite-like is 5x5 while big is 15x15, + * but I haven't yet figured out how to use all 15x15 weights. At the + * moment we're running everything lite-like + */ + ASSERT(expMeans_.size() == weights_.size()); + double ySum = 0.0; + double wSum = 0.0; + + /* Sum the averages, saturated to 4095. */ + for (unsigned i = 0; i < expMeans_.size(); i++) { + double w = weights_[i] / 0x10; + ySum += std::min(expMeans_[i] * gain, 4095.0) * w; + wSum += w; + } + + /* \todo Weight with the AWB gains */ + + return ySum / wSum / 4095; +} + +void Agc::processFrameDuration(IPAContext &context, + IPAFrameContext &frameContext, + utils::Duration frameDuration) +{ + IPACameraSensorInfo &sensorInfo = context.sensorInfo; + utils::Duration lineDuration = context.configuration.sensor.lineDuration; + + frameContext.agc.vblank = (frameDuration / lineDuration) - sensorInfo.outputSize.height; + + /* Update frame duration accounting for line length quantization. */ + frameContext.agc.frameDuration = (sensorInfo.outputSize.height + frameContext.agc.vblank) * lineDuration; +} + +/** + * \brief Process RkISP2 statistics, and run AGC operations + * \param[in] context The shared IPA context + * \param[in] frame The frame context sequence number + * \param[in] frameContext The current frame context + * \param[in] stats The RKISP2 statistics and ISP results + * \param[out] metadata Metadata for the frame, to be filled by the algorithm + * + * Identify the current image brightness, and use that to estimate the optimal + * new exposure and gain for the scene. + */ +void Agc::process([[maybe_unused]] IPAContext &context, + const uint32_t frame, + [[maybe_unused]] IPAFrameContext &frameContext, + const RkISP2Stats *stats, + ControlList &metadata) +{ + /* The first frame has no stats so skip processing */ + if (frame < 1) + return; + + const utils::Duration &lineDuration = context.configuration.sensor.lineDuration; + + auto aeLite = stats->block(); + auto histBig = stats->block(); + + /* + * \todo Verify that the exposure and gain applied by the sensor for + * this frame match what has been requested. This isn't a hard + * requirement for stability of the AGC (the guarantee we need in + * automatic mode is a perfect match between the frame and the values + * we receive), but is important in manual mode. + */ + + /* The lower 5 bits are fractional and meant to be discarded. */ + Histogram hist({ histBig->hist_bins, RKISP2_ISP_HIST_BIN_N_MAX }, + [](uint32_t x) { return x >> 5; }); + + expMeans_.resize(RKISP2_ISP_AE_MEAN_MAX_LITE); + for (size_t i = 0; i < RKISP2_ISP_AE_MEAN_MAX_LITE; i++) { + /* r and b are 0~1023; g is 255~4095 so multiply r and b to match g */ + uint16_t r = aeLite->exp_mean_r[i] * 4; + uint16_t g = aeLite->exp_mean_g[i]; + uint16_t b = aeLite->exp_mean_b[i] * 4; + expMeans_[i] = 0.2126 * r + 0.7152 * g + 0.0722 * b; + } + + /* \todo Support configuring the weights */ + std::vector modeWeights(25, 0x10); + weights_ = { modeWeights.data(), modeWeights.size() }; + + /* + * Set the AGC limits using the fixed exposure time and/or gain in + * manual mode, or the sensor limits in auto mode. + */ + utils::Duration minExposureTime; + utils::Duration maxExposureTime; + double minAnalogueGain; + double maxAnalogueGain; + + if (frameContext.agc.autoExposureEnabled) { + minExposureTime = context.configuration.sensor.minExposureTime; + maxExposureTime = std::clamp(frameContext.agc.maxFrameDuration, + context.configuration.sensor.minExposureTime, + context.configuration.sensor.maxExposureTime); + } else { + minExposureTime = context.configuration.sensor.lineDuration + * frameContext.agc.exposure; + maxExposureTime = minExposureTime; + } + + if (frameContext.agc.autoGainEnabled) { + minAnalogueGain = context.configuration.sensor.minAnalogueGain; + maxAnalogueGain = context.configuration.sensor.maxAnalogueGain; + } else { + minAnalogueGain = frameContext.agc.gain; + maxAnalogueGain = frameContext.agc.gain; + } + + setLimits(minExposureTime, maxExposureTime, minAnalogueGain, maxAnalogueGain, {}); + + /* + * The Agc algorithm needs to know the effective exposure value that was + * applied to the sensor when the statistics were collected. + */ + utils::Duration exposureTime = lineDuration * frameContext.sensor.exposure; + double analogueGain = frameContext.sensor.gain; + utils::Duration effectiveExposureValue = exposureTime * analogueGain; + if (effectiveExposureValue == 0ms) { + LOG(RkISP2Agc, Warning) + << "frame " << frame << ": Effective exposure value is 0: sensor exposure: " + << exposureTime << ", analogue gain: " << analogueGain; + } + + /* \todo Support lux estimation */ + + /* \todo Support setting constraint and exposure modes */ + utils::Duration newExposureTime; + double aGain, qGain, dGain; + std::tie(newExposureTime, aGain, qGain, dGain) = + calculateNewEv(0, 0, hist, effectiveExposureValue); + + LOG(RkISP2Agc, Debug) + << "Divided up exposure time, analogue gain, quantization gain" + << " and digital gain are " << newExposureTime << ", " << aGain + << ", " << qGain << " and " << dGain; + + IPAActiveState &activeState = context.activeState; + /* Update the estimated exposure and gain. */ + activeState.agc.automatic.exposure = newExposureTime / lineDuration; + activeState.agc.automatic.gain = aGain; + activeState.agc.automatic.quantizationGain = qGain; + activeState.agc.automatic.yTarget = effectiveYTarget(); + /* + * Expand the target frame duration so that we do not run faster than + * the minimum frame duration when we have short exposures. + */ + processFrameDuration(context, frameContext, + std::max(frameContext.agc.minFrameDuration, newExposureTime)); + + fillMetadata(context, frameContext, metadata, stats); + expMeans_ = {}; +} + +REGISTER_IPA_ALGORITHM(Agc, "Agc") + +} /* namespace ipa::rkisp2::algorithms */ + +} /* namespace libcamera */ diff --git a/src/ipa/rkisp2/algorithms/agc.h b/src/ipa/rkisp2/algorithms/agc.h new file mode 100644 index 000000000000..6737c5f3d795 --- /dev/null +++ b/src/ipa/rkisp2/algorithms/agc.h @@ -0,0 +1,58 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2026, Ideas On Board Oy. + * + * RkISP2 AGC/AEC mean-based control algorithm + */ + +#pragma once + +#include + +#include + +#include +#include + +#include "libipa/agc_mean_luminance.h" + +#include "algorithm.h" + +namespace libcamera { + +namespace ipa::rkisp2::algorithms { + +class Agc : public Algorithm, public AgcMeanLuminance +{ +public: + Agc() = default; + ~Agc() = default; + + int init(IPAContext &context, [[maybe_unused]] const ValueNode &tuningData) override; + int configure(IPAContext &context, const IPACameraSensorInfo &configInfo) override; + void queueRequest(IPAContext &context, + const uint32_t frame, + IPAFrameContext &frameContext, + const ControlList &controls) override; + void prepare(IPAContext &context, const uint32_t frame, + IPAFrameContext &frameContext, + RkISP2Params *params) override; + void process(IPAContext &context, const uint32_t frame, + IPAFrameContext &frameContext, + const RkISP2Stats *stats, + ControlList &metadata) override; + +private: + void fillMetadata(IPAContext &context, IPAFrameContext &frameContext, + ControlList &metadata, const RkISP2Stats *stats); + double estimateLuminance(double gain) const override; + void processFrameDuration(IPAContext &context, + IPAFrameContext &frameContext, + utils::Duration frameDuration); + + std::vector expMeans_; + Span weights_; +}; + +} /* namespace ipa::rkisp2::algorithms */ +} /* namespace libcamera */ diff --git a/src/ipa/rkisp2/algorithms/meson.build b/src/ipa/rkisp2/algorithms/meson.build index 30133ac4fa68..027e40dd0346 100644 --- a/src/ipa/rkisp2/algorithms/meson.build +++ b/src/ipa/rkisp2/algorithms/meson.build @@ -1,6 +1,7 @@ # SPDX-License-Identifier: CC0-1.0 rkisp2_ipa_algorithms = files([ + 'agc.cpp', 'awb.cpp', 'bls.cpp', ]) From patchwork Thu Aug 27 10:50:04 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Paul Elder X-Patchwork-Id: 28131 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 32188C333C for ; Thu, 27 Aug 2026 10:51:11 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id C328568490; Thu, 27 Aug 2026 12:51:10 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="V2tzHL0F"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id B774F68487 for ; Thu, 27 Aug 2026 12:51:08 +0200 (CEST) Received: from neptunite.hamster-moth.ts.net (unknown [IPv6:2400:2411:160:2f00:306c:5b5b:3c7f:eabd]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 2F8465B3; Thu, 27 Aug 2026 12:49:38 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1787827782; bh=XRI7KOZr+9eaqsByST0jfURf8qO58w4pggLzxdhHTaQ=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=V2tzHL0FYreIc5XR5bb3igLG/eEIHdc25X/Y5SCDnRfjKS+qan2HpoH7l8RTu+oi0 0jyyO/nop6YXF7BHqdQcwiPyPF0BR/mOSJAsN4MKnUcZqoE4pk2wZjmnaFDlUfzOMf XCADIc3+7g6IJ+jZb7c+qr9s4pXcgNMPrIUdnRUw= From: Paul Elder To: laurent.pinchart@ideasonboard.com Cc: Paul Elder , michael.riesch@collabora.com, xuhf@rock-chips.com, stefan.klug@ideasonboard.com, kieran.bingham@ideasonboard.com, dan.scally@ideasonboard.com, jacopo.mondi@ideasonboard.com, nicolas.dufresne@collabora.com, mehdi.djait@linux.intel.com, libcamera-devel@lists.libcamera.org Subject: [PATCH v3 11/20] ipa: rkisp2: algo: ccm: Implement color correction matrix Date: Thu, 27 Aug 2026 19:50:04 +0900 Message-ID: <20260827105018.2781166-12-paul.elder@ideasonboard.com> X-Mailer: git-send-email 2.47.2 In-Reply-To: <20260827105018.2781166-1-paul.elder@ideasonboard.com> References: <20260827105018.2781166-1-paul.elder@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Implement a color correction matrix control algorithm for the rkisp2 IPA. This uses the libipa color correction matrix algorithm. Signed-off-by: Paul Elder --- Changes in v2: - rebased on libipa rework --- src/ipa/rkisp2/algorithms/ccm.cpp | 126 ++++++++++++++++++++++++++ src/ipa/rkisp2/algorithms/ccm.h | 57 ++++++++++++ src/ipa/rkisp2/algorithms/meson.build | 1 + 3 files changed, 184 insertions(+) create mode 100644 src/ipa/rkisp2/algorithms/ccm.cpp create mode 100644 src/ipa/rkisp2/algorithms/ccm.h diff --git a/src/ipa/rkisp2/algorithms/ccm.cpp b/src/ipa/rkisp2/algorithms/ccm.cpp new file mode 100644 index 000000000000..803a22264c04 --- /dev/null +++ b/src/ipa/rkisp2/algorithms/ccm.cpp @@ -0,0 +1,126 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2026, Ideas On Board + * + * RkISP2 Color Correction Matrix control algorithm + */ + +#include "ccm.h" + +#include + +#include +#include + +#include + +#include + +#include "libipa/interpolator.h" + +/** + * \file ccm.h + * \brief RkISP2 CCM algorithm implementation + */ + +namespace libcamera { + +namespace ipa::rkisp2::algorithms { + +/** + * \class Ccm + * \brief RkISP2 color correction matrix algorithm + */ + +LOG_DEFINE_CATEGORY(RkISP2Ccm) + +/** + * \copydoc libcamera::ipa::Algorithm::init + */ +int Ccm::init([[maybe_unused]] IPAContext &context, const ValueNode &tuningData) +{ + return ccmAlgo_.init(tuningData, context.ctrlMap); +} + +/** + * \copydoc libcamera::ipa::Algorithm::configure + */ +int Ccm::configure(IPAContext &context, + [[maybe_unused]] const IPACameraSensorInfo &configInfo) +{ + return ccmAlgo_.configure(context.activeState.ccm, + context.activeState.awb.automatic.colourTemperature); +} + +void Ccm::queueRequest(IPAContext &context, + [[maybe_unused]] const uint32_t frame, + IPAFrameContext &frameContext, + const ControlList &controls) +{ + ccmAlgo_.queueRequest(context.activeState.ccm, frameContext.ccm, controls); +} + +void Ccm::setParameters(RkISP2Params *params, IPAFrameContext &context) +{ + const Matrix &matrix = context.ccm.ccm; + const Matrix &offsets = context.ccm.offsets; + + auto config = params->block(); + config.setEnabled(true); + + /* + * 4 bit integer and 7 bit fractional, ranging from -8 (0x400) to + * +7.9921875 (0x3ff) + */ + for (unsigned int i = 0; i < 3; i++) { + for (unsigned int j = 0; j < 3; j++) + config->coeff[i][j] = Q<4, 7>(matrix[i][j]).quantized(); + } + + for (unsigned int i = 0; i < 3; i++) + config->offset[i] = offsets[i][0] & 0xfff; + + config->y_coeff[0] = 0x26; + config->y_coeff[1] = 0x4b; + config->y_coeff[2] = 0x0f; + config->inflection_point = 0xa; + + for (unsigned int i = 0; i < 17; i++) + config->alp[i] = 0x400; + + /* \todo Support alp */ + + LOG(RkISP2Ccm, Debug) << "Setting matrix " << matrix; + LOG(RkISP2Ccm, Debug) << "Setting offsets " << offsets; +} + +/** + * \copydoc libcamera::ipa::Algorithm::prepare + */ +void Ccm::prepare(IPAContext &context, const uint32_t frame, + IPAFrameContext &frameContext, RkISP2Params *params) +{ + if (frameContext.awb.autoEnabled) + ccmAlgo_.prepare(context.activeState.ccm, frameContext.ccm, + frame, frameContext.awb.colourTemperature); + + setParameters(params, frameContext); +} + +/** + * \copydoc libcamera::ipa::Algorithm::process + */ +void Ccm::process([[maybe_unused]] IPAContext &context, + [[maybe_unused]] const uint32_t frame, + IPAFrameContext &frameContext, + [[maybe_unused]] const RkISP2Stats *stats, + ControlList &metadata) +{ + ccmAlgo_.process(frameContext.ccm, metadata); +} + +REGISTER_IPA_ALGORITHM(Ccm, "Ccm") + +} /* namespace ipa::rkisp2::algorithms */ + +} /* namespace libcamera */ diff --git a/src/ipa/rkisp2/algorithms/ccm.h b/src/ipa/rkisp2/algorithms/ccm.h new file mode 100644 index 000000000000..6c2bff77c1b5 --- /dev/null +++ b/src/ipa/rkisp2/algorithms/ccm.h @@ -0,0 +1,57 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2026, Ideas On Board + * + * RkISP2 Color Correction Matrix control algorithm + */ + +#pragma once + +#include + +#include + +#include "libcamera/internal/value_node.h" + +#include "libipa/ccm.h" +#include "libipa/fixedpoint.h" + +#include "algorithm.h" +#include "ipa_context.h" +#include "params.h" + +namespace libcamera { + +namespace ipa::rkisp2::algorithms { + +class Ccm : public Algorithm +{ +public: + Ccm() {} + ~Ccm() = default; + + int init(IPAContext &context, const ValueNode &tuningData) override; + int configure(IPAContext &context, + const IPACameraSensorInfo &configInfo) override; + void queueRequest(IPAContext &context, + const uint32_t frame, + IPAFrameContext &frameContext, + const ControlList &controls) override; + void prepare(IPAContext &context, const uint32_t frame, + IPAFrameContext &frameContext, + RkISP2Params *params) override; + void process(IPAContext &context, const uint32_t frame, + IPAFrameContext &frameContext, + const RkISP2Stats *stats, + ControlList &metadata) override; + +private: + void parseYaml(const ValueNode &tuningData); + void setParameters(RkISP2Params *params, IPAFrameContext &context); + + CcmAlgorithm> ccmAlgo_; +}; + +} /* namespace ipa::rkisp2::algorithms */ + +} /* namespace libcamera */ diff --git a/src/ipa/rkisp2/algorithms/meson.build b/src/ipa/rkisp2/algorithms/meson.build index 027e40dd0346..e7ae3d163bfb 100644 --- a/src/ipa/rkisp2/algorithms/meson.build +++ b/src/ipa/rkisp2/algorithms/meson.build @@ -4,5 +4,6 @@ rkisp2_ipa_algorithms = files([ 'agc.cpp', 'awb.cpp', 'bls.cpp', + 'ccm.cpp', ]) From patchwork Thu Aug 27 10:50:05 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Paul Elder X-Patchwork-Id: 28132 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id C1E3DBDDFC for ; Thu, 27 Aug 2026 10:51:13 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 5F92F68490; Thu, 27 Aug 2026 12:51:13 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="jskOYCwf"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 6766B6848E for ; Thu, 27 Aug 2026 12:51:12 +0200 (CEST) Received: from neptunite.hamster-moth.ts.net (unknown [IPv6:2400:2411:160:2f00:306c:5b5b:3c7f:eabd]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id C923CC1; Thu, 27 Aug 2026 12:49:42 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1787827785; bh=KXmv3KbPQlgypy6jjRQdhSc9t/jan3RZ2S2S6jcsVts=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=jskOYCwfgqFJUSOMl2+2QY71kW5drfzXhYCsPffg8Xpu8qvYsCZHNWm67CGFdIHJh zvGEAcLv7mKKmXkY3MPb7b7qxkIBGTeh6sEByn5SFQjhVAxY6dovtnU01UtFxvzk6t 64s+FLtMUiYif7kFICBjCnvilbQdAFVk2DpGkTuE= From: Paul Elder To: laurent.pinchart@ideasonboard.com Cc: Paul Elder , michael.riesch@collabora.com, xuhf@rock-chips.com, stefan.klug@ideasonboard.com, kieran.bingham@ideasonboard.com, dan.scally@ideasonboard.com, jacopo.mondi@ideasonboard.com, nicolas.dufresne@collabora.com, mehdi.djait@linux.intel.com, libcamera-devel@lists.libcamera.org Subject: [PATCH v3 12/20] ipa: rkisp2: algo: csm: Implement color space conversion Date: Thu, 27 Aug 2026 19:50:05 +0900 Message-ID: <20260827105018.2781166-13-paul.elder@ideasonboard.com> X-Mailer: git-send-email 2.47.2 In-Reply-To: <20260827105018.2781166-1-paul.elder@ideasonboard.com> References: <20260827105018.2781166-1-paul.elder@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Implement a static color space conversion algorithm for the rkisp2 IPA. Signed-off-by: Paul Elder --- No change in v2 --- src/ipa/rkisp2/algorithms/csm.cpp | 177 ++++++++++++++++++++++++++ src/ipa/rkisp2/algorithms/csm.h | 34 +++++ src/ipa/rkisp2/algorithms/meson.build | 1 + 3 files changed, 212 insertions(+) create mode 100644 src/ipa/rkisp2/algorithms/csm.cpp create mode 100644 src/ipa/rkisp2/algorithms/csm.h diff --git a/src/ipa/rkisp2/algorithms/csm.cpp b/src/ipa/rkisp2/algorithms/csm.cpp new file mode 100644 index 000000000000..35adc94c6574 --- /dev/null +++ b/src/ipa/rkisp2/algorithms/csm.cpp @@ -0,0 +1,177 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2026, Ideas On Board + * + * RkISP2 Color space conversion + */ + +#include "csm.h" + +#include +#include + +#include + +#include +#include + +#include +#include + +/** + * \file csm.h + */ + +namespace libcamera { + +namespace ipa::rkisp2::algorithms { + +/** + * \class ColorSpaceConversion + * \brief RkISP2 Color space conversion + * + * This algorithm implements the color space conversion for the RkISP2. + */ + +LOG_DEFINE_CATEGORY(RkISP2Csm) + +namespace { + struct CsmCoeffs { + uint16_t limited[9]; + uint16_t full[9]; + }; + + static const struct CsmCoeffs rec601Coeffs = { + { + 0x0021, 0x0042, 0x000d, + 0x01ed, 0x01db, 0x0038, + 0x0038, 0x01d1, 0x01f7, + }, + { + 0x0026, 0x004b, 0x000f, + 0x01ea, 0x01d6, 0x0040, + 0x0040, 0x01ca, 0x01f6, + }, + }; + + static const struct CsmCoeffs rec709Coeffs = { + { + 0x0018, 0x0050, 0x0008, + 0x01f3, 0x01d5, 0x0038, + 0x0038, 0x01cd, 0x01fb, + }, + { + 0x001b, 0x005c, 0x0009, + 0x01f1, 0x01cf, 0x0040, + 0x0040, 0x01c6, 0x01fa, + }, + }; + + static const struct CsmCoeffs rec2020Coeffs = { + { + 0x001d, 0x004c, 0x0007, + 0x01f0, 0x01d8, 0x0038, + 0x0038, 0x01cd, 0x01fb, + }, + { + 0x0022, 0x0057, 0x0008, + 0x01ee, 0x01d2, 0x0040, + 0x0040, 0x01c5, 0x01fb, + }, + }; + + static const struct CsmCoeffs smpte240mCoeffs = { + { + 0x0018, 0x004f, 0x000a, + 0x01f3, 0x01d5, 0x0038, + 0x0038, 0x01ce, 0x01fa, + }, + { + 0x001b, 0x005a, 0x000b, + 0x01f1, 0x01cf, 0x0040, + 0x0040, 0x01c7, 0x01f9, + }, + }; + + uint16_t identityCsm[9] = { + 1, 0, 0, + 0, 1, 0, + 0, 0, 1, + }; +} /* namespace */ + +/** + * \copydoc libcamera::ipa::Algorithm::init + */ +int ColorSpaceConversion::init([[maybe_unused]] IPAContext &context, + [[maybe_unused]] const ValueNode &tuningData) +{ + return 0; +} + +/** + * \copydoc libcamera::ipa::Algorithm::configure + */ +int ColorSpaceConversion::configure(IPAContext &context, + [[maybe_unused]] const IPACameraSensorInfo &configInfo) +{ + const CsmCoeffs *coeffs; + switch (context.configuration.csm.colorSpaceEncoding) { + case static_cast(ColorSpace::YcbcrEncoding::Rec601): + coeffs = &rec601Coeffs; + break; + case static_cast(ColorSpace::YcbcrEncoding::Rec709): + coeffs = &rec709Coeffs; + break; + case static_cast(ColorSpace::YcbcrEncoding::Rec2020): + coeffs = &rec2020Coeffs; + break; + default: + coeffs = nullptr; + break; + } + + if (!coeffs) { + context.activeState.csm.csm = Matrix(identityCsm); + return 0; + } + + if (context.configuration.csm.colorSpaceRange == static_cast(ColorSpace::Range::Limited)) + context.activeState.csm.csm = Matrix(coeffs->limited); + else + context.activeState.csm.csm = Matrix(coeffs->full); + context.activeState.csm.update = true; + + return 0; +} + +/** + * \copydoc libcamera::ipa::Algorithm::prepare + */ +void ColorSpaceConversion::prepare(IPAContext &context, + [[maybe_unused]] const uint32_t frame, + [[maybe_unused]] IPAFrameContext &frameContext, + RkISP2Params *params) +{ + if (!context.activeState.csm.update) + return; + + auto config = params->block(); + config.setEnabled(true); + + /* + * We'll use the active state directly as we don't support runtime + * configuration of the csm + */ + for (size_t i = 0; i < 3; i++) + for (size_t j = 0; j < 3; j++) + config->coeff[i][j] = context.activeState.csm.csm[i][j]; + + context.activeState.csm.update = false; +} + +REGISTER_IPA_ALGORITHM(ColorSpaceConversion, "ColorSpaceConversion") + +} /* namespace ipa::rkisp2::algorithms */ + +} /* namespace libcamera */ diff --git a/src/ipa/rkisp2/algorithms/csm.h b/src/ipa/rkisp2/algorithms/csm.h new file mode 100644 index 000000000000..d34e4d3959e8 --- /dev/null +++ b/src/ipa/rkisp2/algorithms/csm.h @@ -0,0 +1,34 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2026, Ideas On Board + * + * RkISP2 Color space conversion + */ + +#pragma once + +#include "libcamera/internal/matrix.h" + +#include "algorithm.h" + +namespace libcamera { + +namespace ipa::rkisp2::algorithms { + +class ColorSpaceConversion : public Algorithm +{ +public: + ColorSpaceConversion() = default; + ~ColorSpaceConversion() = default; + + int init(IPAContext &context, const ValueNode &tuningData) override; + int configure(IPAContext &context, + const IPACameraSensorInfo &configInfo) override; + void prepare(IPAContext &context, const uint32_t frame, + IPAFrameContext &frameContext, + RkISP2Params *params) override; +}; + +} /* namespace ipa::rkisp2::algorithms */ + +} /* namespace libcamera */ diff --git a/src/ipa/rkisp2/algorithms/meson.build b/src/ipa/rkisp2/algorithms/meson.build index e7ae3d163bfb..01829e25d254 100644 --- a/src/ipa/rkisp2/algorithms/meson.build +++ b/src/ipa/rkisp2/algorithms/meson.build @@ -5,5 +5,6 @@ rkisp2_ipa_algorithms = files([ 'awb.cpp', 'bls.cpp', 'ccm.cpp', + 'csm.cpp', ]) From patchwork Thu Aug 27 10:50:06 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Paul Elder X-Patchwork-Id: 28133 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 950C8BDDFC for ; Thu, 27 Aug 2026 10:51:17 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 4673368487; Thu, 27 Aug 2026 12:51:17 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="aiLv/Cee"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 2DF1368461 for ; Thu, 27 Aug 2026 12:51:16 +0200 (CEST) Received: from neptunite.hamster-moth.ts.net (unknown [IPv6:2400:2411:160:2f00:306c:5b5b:3c7f:eabd]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 771235B3; Thu, 27 Aug 2026 12:49:46 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1787827789; bh=RJ+6V4rgHiJpBJiybA7uxGa+mM0Mr05x22m9QP/nmto=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=aiLv/Ceev0j2i0P/GPRbMHE4u88VvYRkN0HMyd2o52NrjHqIf9M6y2JQajxs3aYwn i1TyX1OZLkn4kN0mSLZSGQ00kYZPi5CUDiE4TypbNHBXZrgANs9vKX+UuxmIAnt8HL M+1XVuvuJhh20z8m6sv7Yq8RbljMSzOk4B162cmo= From: Paul Elder To: laurent.pinchart@ideasonboard.com Cc: Paul Elder , michael.riesch@collabora.com, xuhf@rock-chips.com, stefan.klug@ideasonboard.com, kieran.bingham@ideasonboard.com, dan.scally@ideasonboard.com, jacopo.mondi@ideasonboard.com, nicolas.dufresne@collabora.com, mehdi.djait@linux.intel.com, libcamera-devel@lists.libcamera.org Subject: [PATCH v3 13/20] ipa: rkisp2: algo: goc: Implement gamma out correction Date: Thu, 27 Aug 2026 19:50:06 +0900 Message-ID: <20260827105018.2781166-14-paul.elder@ideasonboard.com> X-Mailer: git-send-email 2.47.2 In-Reply-To: <20260827105018.2781166-1-paul.elder@ideasonboard.com> References: <20260827105018.2781166-1-paul.elder@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Implement a manual gamma out correction algorithm for the rkisp2 IPA. It takes the gamma control and applies the gamma out correction curve to the ISP. Signed-off-by: Paul Elder --- No change in v2 --- src/ipa/rkisp2/algorithms/goc.cpp | 143 ++++++++++++++++++++++++++ src/ipa/rkisp2/algorithms/goc.h | 42 ++++++++ src/ipa/rkisp2/algorithms/meson.build | 1 + 3 files changed, 186 insertions(+) create mode 100644 src/ipa/rkisp2/algorithms/goc.cpp create mode 100644 src/ipa/rkisp2/algorithms/goc.h diff --git a/src/ipa/rkisp2/algorithms/goc.cpp b/src/ipa/rkisp2/algorithms/goc.cpp new file mode 100644 index 000000000000..d80212d4f2e0 --- /dev/null +++ b/src/ipa/rkisp2/algorithms/goc.cpp @@ -0,0 +1,143 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2026, Ideas On Board + * + * RkISP2 Gamma out control + */ + +#include "goc.h" + +#include + +#include + +#include +#include + +#include + +/** + * \file goc.h + */ + +namespace libcamera { + +namespace ipa::rkisp2::algorithms { + +/** + * \class GammaOutCorrection + * \brief RkISP2 Gamma out correction + * + * This algorithm implements the gamma out curve for the RkISP2. It defaults to + * a gamma value of 2.2. + * + * As gamma is internally represented as a piecewise linear function with only + * 17 knots, the difference between gamma=2.2 and sRGB gamma is minimal. + * Therefore sRGB gamma was not implemented as special case. + * + * Useful links: + * - https://www.cambridgeincolour.com/tutorials/gamma-correction.htm + * - https://en.wikipedia.org/wiki/SRGB + */ + +LOG_DEFINE_CATEGORY(RkISP2Gamma) + +const float kDefaultGamma = 2.2f; + +/** + * \copydoc libcamera::ipa::Algorithm::init + */ +int GammaOutCorrection::init(IPAContext &context, const ValueNode &tuningData) +{ + defaultGamma_ = tuningData["gamma"].get(kDefaultGamma); + context.ctrlMap[&controls::Gamma] = ControlInfo(0.1f, 10.0f, defaultGamma_); + + return 0; +} + +/** + * \copydoc libcamera::ipa::Algorithm::configure + */ +int GammaOutCorrection::configure(IPAContext &context, + [[maybe_unused]] const IPACameraSensorInfo &configInfo) +{ + context.activeState.goc.gamma = defaultGamma_; + return 0; +} + +/** + * \copydoc libcamera::ipa::Algorithm::queueRequest + */ +void GammaOutCorrection::queueRequest(IPAContext &context, + [[maybe_unused]] const uint32_t frame, + IPAFrameContext &frameContext, + const ControlList &controls) +{ + frameContext.goc.gamma = context.activeState.goc.gamma; + + const auto &gamma = controls.get(controls::Gamma); + if (!gamma) + return; + + context.activeState.goc.gamma = *gamma; + + frameContext.goc.gamma = context.activeState.goc.gamma; +} + +/** + * \copydoc libcamera::ipa::Algorithm::prepare + */ +void GammaOutCorrection::prepare([[maybe_unused]] IPAContext &context, + [[maybe_unused]] const uint32_t frame, + [[maybe_unused]] IPAFrameContext &frameContext, + [[maybe_unused]] RkISP2Params *params) +{ + /* \todo Optimize so we don't have to write it every frame */ + + /* + * The logarithmic segments as specified in the reference, plus an + * additional 0 to make the loop easier + * + * Just use 44 mode for now. 48 mode is when the last four 512 are + * split into eight pieces of 256. + */ + static constexpr std::array segments = { + 0, 1, 1, 1, 1, 1, 1, 1, 1, + 2, 2, 2, 2, 4, 4, 4, 4, + 8, 8, 8, 8, 16, 16, 16, 16, + 32, 32, 32, 32, 64, 64, 64, 64, + 128, 128, 128, 128, 256, 256, 256, 256, + 512, 512, 512, 512 + }; + + auto config = params->block(); + config.setEnabled(true); + + unsigned x = 0; + for (const auto [i, size] : utils::enumerate(segments)) { + config->gamma_y[i] = static_cast(std::pow(x / 4096.0, 1.0 / frameContext.goc.gamma) * 4096.0); + x += size; + } + + config->mode = RKISP2_ISP_GOC_MODE_LOGARITHMIC; + config->segments = RKISP2_ISP_GOC_SEGMENTS_44; + config->offset = 0; +} + +/** + * \copydoc libcamera::ipa::Algorithm::process + */ +void GammaOutCorrection::process([[maybe_unused]] IPAContext &context, + [[maybe_unused]] const uint32_t frame, + IPAFrameContext &frameContext, + [[maybe_unused]] const RkISP2Stats *stats, + ControlList &metadata) +{ + metadata.set(controls::Gamma, frameContext.goc.gamma); +} + +REGISTER_IPA_ALGORITHM(GammaOutCorrection, "GammaOutCorrection") + +} /* namespace ipa::rkisp2::algorithms */ + +} /* namespace libcamera */ diff --git a/src/ipa/rkisp2/algorithms/goc.h b/src/ipa/rkisp2/algorithms/goc.h new file mode 100644 index 000000000000..4353dda5ff14 --- /dev/null +++ b/src/ipa/rkisp2/algorithms/goc.h @@ -0,0 +1,42 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2026, Ideas On Board + * + * RkISP2 Gamma out control + */ + +#pragma once + +#include "algorithm.h" + +namespace libcamera { + +namespace ipa::rkisp2::algorithms { + +class GammaOutCorrection : public Algorithm +{ +public: + GammaOutCorrection() = default; + ~GammaOutCorrection() = default; + + int init(IPAContext &context, const ValueNode &tuningData) override; + int configure(IPAContext &context, + const IPACameraSensorInfo &configInfo) override; + void queueRequest(IPAContext &context, + const uint32_t frame, + IPAFrameContext &frameContext, + const ControlList &controls) override; + void prepare(IPAContext &context, const uint32_t frame, + IPAFrameContext &frameContext, + RkISP2Params *params) override; + void process(IPAContext &context, const uint32_t frame, + IPAFrameContext &frameContext, + const RkISP2Stats *stats, + ControlList &metadata) override; + +private: + float defaultGamma_; +}; + +} /* namespace ipa::rkisp2::algorithms */ +} /* namespace libcamera */ diff --git a/src/ipa/rkisp2/algorithms/meson.build b/src/ipa/rkisp2/algorithms/meson.build index 01829e25d254..3d73b0a99320 100644 --- a/src/ipa/rkisp2/algorithms/meson.build +++ b/src/ipa/rkisp2/algorithms/meson.build @@ -6,5 +6,6 @@ rkisp2_ipa_algorithms = files([ 'bls.cpp', 'ccm.cpp', 'csm.cpp', + 'goc.cpp', ]) From patchwork Thu Aug 27 10:50:07 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Paul Elder X-Patchwork-Id: 28134 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 2FF38BDDFC for ; Thu, 27 Aug 2026 10:51:22 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id CC0A9684A6; Thu, 27 Aug 2026 12:51:21 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="PaCo8xSu"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id CE31D68461 for ; Thu, 27 Aug 2026 12:51:19 +0200 (CEST) Received: from neptunite.hamster-moth.ts.net (unknown [IPv6:2400:2411:160:2f00:306c:5b5b:3c7f:eabd]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 455E8C1; Thu, 27 Aug 2026 12:49:50 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1787827793; bh=LgBSBE2WMbadPrGnfc1v0wSaRfo/uDgcntA4LnFUXVI=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=PaCo8xSuBL2S/uArZrJhFUwntjKwz73fRNaf2H8HFwgyHW5NrFneqYnTFf/xakw68 4j6dSdhBu5mWUt3OcTTDZRFSg+qUXWHGGpHOREKOx+3vxTL7cSvhIPLJ85aijrgmU6 XhRPM27IIEDrc9SDJndF3yLlXhgXk9boQVgf1qNE= From: Paul Elder To: laurent.pinchart@ideasonboard.com Cc: Paul Elder , michael.riesch@collabora.com, xuhf@rock-chips.com, stefan.klug@ideasonboard.com, kieran.bingham@ideasonboard.com, dan.scally@ideasonboard.com, jacopo.mondi@ideasonboard.com, nicolas.dufresne@collabora.com, mehdi.djait@linux.intel.com, libcamera-devel@lists.libcamera.org Subject: [PATCH v3 14/20] ipa: rkisp2: algo: lsc: Implement lens shading correction Date: Thu, 27 Aug 2026 19:50:07 +0900 Message-ID: <20260827105018.2781166-15-paul.elder@ideasonboard.com> X-Mailer: git-send-email 2.47.2 In-Reply-To: <20260827105018.2781166-1-paul.elder@ideasonboard.com> References: <20260827105018.2781166-1-paul.elder@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Implement a lens shading correction algorithm for the rkisp2 IPA. It uses the libipa lens shading correction. Signed-off-by: Paul Elder --- Changes in v2: - rebased on libipa rework --- src/ipa/rkisp2/algorithms/lsc.cpp | 263 ++++++++++++++++++++++++++ src/ipa/rkisp2/algorithms/lsc.h | 71 +++++++ src/ipa/rkisp2/algorithms/meson.build | 1 + 3 files changed, 335 insertions(+) create mode 100644 src/ipa/rkisp2/algorithms/lsc.cpp create mode 100644 src/ipa/rkisp2/algorithms/lsc.h diff --git a/src/ipa/rkisp2/algorithms/lsc.cpp b/src/ipa/rkisp2/algorithms/lsc.cpp new file mode 100644 index 000000000000..b7c67dc983f7 --- /dev/null +++ b/src/ipa/rkisp2/algorithms/lsc.cpp @@ -0,0 +1,263 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2026, Ideas On Board + * + * RkISP2 Lens Shading Correction control + */ + +#include "lsc.h" + +#include +#include +#include + +#include +#include + +/** + * \file lsc.h + */ + +namespace libcamera { + +namespace ipa::rkisp2::algorithms { + +/** + * \class LensShadingCorrection + * \brief RkISP2 Lens Shading Correction control + */ + +LOG_DEFINE_CATEGORY(RkISP2Lsc) + +namespace { + +constexpr int kColourTemperatureQuantization = 10; + +unsigned int quantize(unsigned int value, unsigned int step) +{ + return std::lround(value / static_cast(step)) * step; +} + +} /* namespace */ + +LensShadingCorrection::LensShadingCorrection() + : lastAppliedCt_(0), lastAppliedQuantizedCt_(0) +{ +} + +std::vector LensShadingCorrection::parseSizes(const ValueNode &tuningData, + const char *prop) +{ + std::vector sizes = + tuningData[prop].get>().value_or(utils::defopt); + /* Nobody cares about 8x8 mirrored mode; we'll just use 16x16 mode */ + if (sizes.size() != RKISP2_ISP_LSC_SECTORS_TBL_SIZE_MAX) { + LOG(RkISP2Lsc, Error) + << "Invalid '" << prop << "' values: expected " + << RKISP2_ISP_LSC_SECTORS_TBL_SIZE_MAX + << " elements, got " << sizes.size(); + return {}; + } + + /* + * The sum of all elements must be 1 to satisfy hardware constraints. + * Validate it here, allowing a 1% tolerance as rounding errors may + * prevent an exact match (further adjustments will be performed in + * LensShadingCorrection::prepare()). + * + * If we were in 8x8 mode then we'd have to mirror the quadrants like + * in rkisp1, but in 16x16 mode we get to configure the entire table. + * Since 8x8 table support is a todo, we only need to handle the 16x16 + * case here thus the sum should be 1. + * + * \todo Support 8x8 mode? + */ + double sum = std::accumulate(sizes.begin(), sizes.end(), 0.0); + if (sum < 0.95 || sum > 1.05) { + LOG(RkISP2Lsc, Error) + << "Invalid '" << prop << "' values: sum of the elements" + << " should be 1.0, got " << sum; + return {}; + } + + return sizes; +} + +std::vector LensShadingCorrection::sizesToPositions(Span sizes) +{ + std::vector positions(sizes.size() + 1); + + positions[0] = 0.0; + for (size_t i = 1; i < positions.size(); i++) + positions[i] = positions[i - 1] + sizes[i - 1]; + + return positions; +} + +/** + * \copydoc libcamera::ipa::Algorithm::init + */ +int LensShadingCorrection::init(IPAContext &context, + const ValueNode &tuningData) +{ + xSize_ = parseSizes(tuningData, "x-size"); + ySize_ = parseSizes(tuningData, "y-size"); + + if (xSize_.empty() || ySize_.empty()) + return -EINVAL; + + xPos_ = sizesToPositions(xSize_); + yPos_ = sizesToPositions(ySize_); + + return lscAlgo_.init(tuningData, context.ctrlMap, LscDescriptor{ + .keys = { "r", "gr", "gb", "b" }, + .numHSamples = RKISP2_ISP_LSC_SAMPLES_MAX, + .numVSamples = RKISP2_ISP_LSC_SAMPLES_MAX, + .sensorSize = context.sensorInfo.activeAreaSize + }); +} + +/** + * \copydoc libcamera::ipa::Algorithm::configure + */ +int LensShadingCorrection::configure(IPAContext &context, + const IPACameraSensorInfo &configInfo) +{ + const Size &size = context.configuration.sensor.size; + Size totalSize{}; + + /* Calculate gradients. */ + for (unsigned int i = 0; i < RKISP2_ISP_LSC_SECTORS_TBL_SIZE_MAX; ++i) { + xSizes_[i] = xSize_[i] * size.width; + ySizes_[i] = ySize_[i] * size.height; + + /* + * To prevent unexpected behavior of the ISP, the sum of + * x_sizes and y_sizes items shall be equal to + * respectively size.width and size.height. Enforce it by + * computing the last tables value to avoid + * rounding-induced errors. + */ + if (i == RKISP2_ISP_LSC_SECTORS_TBL_SIZE_MAX - 1) { + xSizes_[i] = size.width - totalSize.width; + ySizes_[i] = size.height - totalSize.height; + } + + totalSize.width += xSizes_[i]; + totalSize.height += ySizes_[i]; + + xGrad_[i] = std::round(32768 / xSizes_[i]); + yGrad_[i] = std::round(32768 / ySizes_[i]); + } + + return lscAlgo_.configure(context.activeState.lsc, configInfo.analogCrop, + xPos_, yPos_); +} + +void LensShadingCorrection::setParameters(rkisp2_params_lsc &config) +{ + memcpy(config.x_grads, xGrad_, sizeof(config.x_grads)); + memcpy(config.y_grads, yGrad_, sizeof(config.y_grads)); + memcpy(config.x_sizes, xSizes_, sizeof(config.x_sizes)); + memcpy(config.y_sizes, ySizes_, sizeof(config.y_sizes)); +} + +void LensShadingCorrection::copyTable(rkisp2_params_lsc &config, + const ipa::lsc::Components &set) +{ + const auto &r = set.at("r"); + std::copy(r.begin(), r.end(), &config.r_data_tbl[0][0][0]); + const auto &gr = set.at("gr"); + std::copy(gr.begin(), gr.end(), &config.gr_data_tbl[0][0][0]); + const auto &gb = set.at("gb"); + std::copy(gb.begin(), gb.end(), &config.gb_data_tbl[0][0][0]); + const auto &b = set.at("b"); + std::copy(b.begin(), b.end(), &config.b_data_tbl[0][0][0]); +} + +/** + * \copydoc libcamera::ipa::Algorithm::queueRequest + */ +void LensShadingCorrection::queueRequest(IPAContext &context, + [[maybe_unused]] const uint32_t frame, + IPAFrameContext &frameContext, + const ControlList &controls) +{ + lscAlgo_.queueRequest(context.activeState.lsc, frameContext.lsc, + controls); +} + +/** + * \copydoc libcamera::ipa::Algorithm::prepare + */ +void LensShadingCorrection::prepare([[maybe_unused]] IPAContext &context, + [[maybe_unused]] const uint32_t frame, + IPAFrameContext &frameContext, + RkISP2Params *params) +{ + uint32_t ct = frameContext.awb.colourTemperature; + unsigned int quantizedCt = quantize(ct, kColourTemperatureQuantization); + + /* Check if we can skip the update. */ + if (!frameContext.lsc.update) { + if (!frameContext.lsc.enabled) + return; + + /* + * Add a threshold so that oscillations around a quantization + * step don't lead to constant changes. + */ + if (utils::abs_diff(ct, lastAppliedCt_) < kColourTemperatureQuantization / 2) + return; + + if (quantizedCt == lastAppliedQuantizedCt_) + return; + } + + auto config = params->block(); + config.setEnabled(frameContext.lsc.enabled); + + if (!frameContext.lsc.enabled) + return; + + /* + * \todo Should add support for the lsc table swapping functionality? + * Or maybe we don't need it because the lsc doesn't change very + * frequently. Just use the 0th table for now. + */ + config->window_mode = RKISP2_ISP_LSC_CONFIG_16X16; + config->write_table[0] = 1; + config->write_table[1] = 0; + config->active_table = 0; + config->set_active_table_when = RKISP2_ISP_LSC_SET_ACTIVE_TABLE_AFTER; + + setParameters(*config); + + const auto &set = lscAlgo_.interpolateComponents(quantizedCt); + copyTable(*config, set); + + lastAppliedCt_ = ct; + lastAppliedQuantizedCt_ = quantizedCt; + + LOG(RkISP2Lsc, Debug) + << "ct is " << ct << ", quantized to " + << quantizedCt; +} + +/** + * \copydoc libcamera::ipa::Algorithm::process + */ +void LensShadingCorrection::process([[maybe_unused]] IPAContext &context, + [[maybe_unused]] const uint32_t frame, + IPAFrameContext &frameContext, + [[maybe_unused]] const RkISP2Stats *stats, + ControlList &metadata) +{ + lscAlgo_.process(frameContext.lsc, metadata); +} + +REGISTER_IPA_ALGORITHM(LensShadingCorrection, "LensShadingCorrection") + +} /* namespace ipa::rkisp2::algorithms */ + +} /* namespace libcamera */ diff --git a/src/ipa/rkisp2/algorithms/lsc.h b/src/ipa/rkisp2/algorithms/lsc.h new file mode 100644 index 000000000000..f7b879e66456 --- /dev/null +++ b/src/ipa/rkisp2/algorithms/lsc.h @@ -0,0 +1,71 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2026, Ideas On Board + * + * RkISP2 Lens Shading Correction algorithm + */ + +#pragma once + +#include + +#include + +#include "libcamera/internal/value_node.h" + +#include "libipa/fixedpoint.h" +#include "libipa/lsc.h" + +#include "algorithm.h" +#include "ipa_context.h" +#include "params.h" + +namespace libcamera { + +namespace ipa::rkisp2::algorithms { + +class LensShadingCorrection : public Algorithm +{ +public: + LensShadingCorrection(); + ~LensShadingCorrection() = default; + + int init(IPAContext &context, const ValueNode &tuningData) override; + int configure(IPAContext &context, const IPACameraSensorInfo &configInfo) override; + void queueRequest(IPAContext &context, const uint32_t frame, + IPAFrameContext &frameContext, + const ControlList &controls) override; + void prepare(IPAContext &context, const uint32_t frame, + IPAFrameContext &frameContext, + RkISP2Params *params) override; + void process(IPAContext &context, const uint32_t frame, + IPAFrameContext &frameContext, + const RkISP2Stats *stats, + ControlList &metadata) override; + +private: + std::vector parseSizes(const ValueNode &tuningData, + const char *prop); + std::vector sizesToPositions(Span sizes); + + void setParameters(rkisp2_params_lsc &config); + void copyTable(rkisp2_params_lsc &config, + const ipa::lsc::Components &set0); + + std::vector xSize_; + std::vector ySize_; + uint16_t xGrad_[RKISP2_ISP_LSC_SECTORS_TBL_SIZE_MAX]; + uint16_t yGrad_[RKISP2_ISP_LSC_SECTORS_TBL_SIZE_MAX]; + uint16_t xSizes_[RKISP2_ISP_LSC_SECTORS_TBL_SIZE_MAX]; + uint16_t ySizes_[RKISP2_ISP_LSC_SECTORS_TBL_SIZE_MAX]; + std::vector xPos_; + std::vector yPos_; + + unsigned int lastAppliedCt_; + unsigned int lastAppliedQuantizedCt_; + + LscAlgorithm> lscAlgo_; +}; + +} /* namespace ipa::rkisp2::algorithms */ +} /* namespace libcamera */ diff --git a/src/ipa/rkisp2/algorithms/meson.build b/src/ipa/rkisp2/algorithms/meson.build index 3d73b0a99320..bcc947fabdb4 100644 --- a/src/ipa/rkisp2/algorithms/meson.build +++ b/src/ipa/rkisp2/algorithms/meson.build @@ -7,5 +7,6 @@ rkisp2_ipa_algorithms = files([ 'ccm.cpp', 'csm.cpp', 'goc.cpp', + 'lsc.cpp', ]) From patchwork Thu Aug 27 10:50:08 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Paul Elder X-Patchwork-Id: 28135 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id DA691BDDFC for ; Thu, 27 Aug 2026 10:51:24 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 85CF5684AB; Thu, 27 Aug 2026 12:51:24 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="Wz5bpBS8"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 7E93D68487 for ; Thu, 27 Aug 2026 12:51:23 +0200 (CEST) Received: from neptunite.hamster-moth.ts.net (unknown [IPv6:2400:2411:160:2f00:306c:5b5b:3c7f:eabd]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id E157B5B3; Thu, 27 Aug 2026 12:49:53 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1787827797; bh=ypgaeSKGTecFJxmyEszOvRBHjkql1IoNR37SL1UYewM=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=Wz5bpBS8TF/4fWLi+pHq3kaYN6mVv5+QqPWxvEy6O9u0o7tyESwsn0oiOBCJun6S/ 4mVY5Bajc8rgPpKRi+vNt7dmD48k1RzYc+M4s4F+FnIZLk/XeZO3sz5Ja+p3PvmVT0 Yq8oa4Z53WFAQJkIpMn/QJfK5VlOR1emfPARINCg= From: Paul Elder To: laurent.pinchart@ideasonboard.com Cc: Paul Elder , michael.riesch@collabora.com, xuhf@rock-chips.com, stefan.klug@ideasonboard.com, kieran.bingham@ideasonboard.com, dan.scally@ideasonboard.com, jacopo.mondi@ideasonboard.com, nicolas.dufresne@collabora.com, mehdi.djait@linux.intel.com, libcamera-devel@lists.libcamera.org Subject: [PATCH v3 15/20] ipa: rkisp2: algo: crop: Implement scaler crop Date: Thu, 27 Aug 2026 19:50:08 +0900 Message-ID: <20260827105018.2781166-16-paul.elder@ideasonboard.com> X-Mailer: git-send-email 2.47.2 In-Reply-To: <20260827105018.2781166-1-paul.elder@ideasonboard.com> References: <20260827105018.2781166-1-paul.elder@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Implement a thin algorithm for handling the ScalerCrop control. As rkisp2 uses parameter buffers to configure scaling and cropping (as opposed to v4l2 subdev pads), it is cleaner to implement scaler crop as an algorithm in the IPA. Signed-off-by: Paul Elder --- New in v2 --- src/ipa/rkisp2/algorithms/crop.cpp | 99 +++++++++++++++++++++++++++ src/ipa/rkisp2/algorithms/crop.h | 39 +++++++++++ src/ipa/rkisp2/algorithms/meson.build | 1 + 3 files changed, 139 insertions(+) create mode 100644 src/ipa/rkisp2/algorithms/crop.cpp create mode 100644 src/ipa/rkisp2/algorithms/crop.h diff --git a/src/ipa/rkisp2/algorithms/crop.cpp b/src/ipa/rkisp2/algorithms/crop.cpp new file mode 100644 index 000000000000..df275173c497 --- /dev/null +++ b/src/ipa/rkisp2/algorithms/crop.cpp @@ -0,0 +1,99 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2026, Ideas On Board + * + * RkISP2 Crop + */ + +#include "crop.h" + +#include + +#include + +#include +#include + +#include + +/** + * \file crop.h + */ + +namespace libcamera { + +namespace ipa::rkisp2::algorithms { + +/** + * \class Crop + * \brief RkISP2 Crop + * + * This is a thin algorithm that implements ScalerCrop for the RkISP2. + * + * As the cropping of the RkISP2 is controlled by parameter buffers instead of + * by V4L2 crop rectangles, it is more practical to implement the ScalerCrop + * control here instead of in the pipeline handler. This also has the positive + * side effect of supporting per-frame ScalerCrop. + */ + +LOG_DEFINE_CATEGORY(RkISP2Crop) + +/** + * \copydoc libcamera::ipa::Algorithm::queueRequest + */ +void Crop::queueRequest(IPAContext &context, + [[maybe_unused]] const uint32_t frame, + IPAFrameContext &frameContext, + const ControlList &controls) +{ + frameContext.crop.crop = context.activeState.crop.crop; + + const auto &scalerCrop = controls.get(controls::ScalerCrop); + if (!scalerCrop) + return; + + context.activeState.crop.crop = *scalerCrop; + + frameContext.crop.crop = context.activeState.crop.crop; + frameContext.crop.set = true; +} + +/** + * \copydoc libcamera::ipa::Algorithm::prepare + */ +void Crop::prepare([[maybe_unused]] IPAContext &context, + [[maybe_unused]] const uint32_t frame, + IPAFrameContext &frameContext, + [[maybe_unused]] RkISP2Params *params) +{ + if (!frameContext.crop.set) + return; + + auto config = params->block(); + config.setEnabled(true); + + config->crop_en = RKISP2_ISP_CROP_ENABLE_MAIN; + config->mp_crop.h_offs = frameContext.crop.crop.x; + config->mp_crop.v_offs = frameContext.crop.crop.y; + config->mp_crop.h_size = frameContext.crop.crop.width; + config->mp_crop.v_size = frameContext.crop.crop.height; +} + +/** + * \copydoc libcamera::ipa::Algorithm::process + */ +void Crop::process([[maybe_unused]] IPAContext &context, + [[maybe_unused]] const uint32_t frame, + IPAFrameContext &frameContext, + [[maybe_unused]] const RkISP2Stats *stats, + ControlList &metadata) +{ + /* \todo Adjust this to match the spec */ + metadata.set(controls::ScalerCrop, frameContext.crop.crop); +} + +REGISTER_IPA_ALGORITHM(Crop, "Crop") + +} /* namespace ipa::rkisp2::algorithms */ + +} /* namespace libcamera */ diff --git a/src/ipa/rkisp2/algorithms/crop.h b/src/ipa/rkisp2/algorithms/crop.h new file mode 100644 index 000000000000..7c21e20c86ec --- /dev/null +++ b/src/ipa/rkisp2/algorithms/crop.h @@ -0,0 +1,39 @@ +/* SPDX-License-Identifier: LGPL-2.1-or-later */ +/* + * Copyright (C) 2026, Ideas On Board + * + * RkISP2 Crop + */ + +#pragma once + +#include "algorithm.h" + +namespace libcamera { + +namespace ipa::rkisp2::algorithms { + +class Crop : public Algorithm +{ +public: + Crop() = default; + ~Crop() = default; + + void queueRequest(IPAContext &context, + const uint32_t frame, + IPAFrameContext &frameContext, + const ControlList &controls) override; + void prepare(IPAContext &context, const uint32_t frame, + IPAFrameContext &frameContext, + RkISP2Params *params) override; + void process(IPAContext &context, const uint32_t frame, + IPAFrameContext &frameContext, + const RkISP2Stats *stats, + ControlList &metadata) override; + +private: + Rectangle defaultCrop_; +}; + +} /* namespace ipa::rkisp2::algorithms */ +} /* namespace libcamera */ diff --git a/src/ipa/rkisp2/algorithms/meson.build b/src/ipa/rkisp2/algorithms/meson.build index bcc947fabdb4..9ccc9653233a 100644 --- a/src/ipa/rkisp2/algorithms/meson.build +++ b/src/ipa/rkisp2/algorithms/meson.build @@ -5,6 +5,7 @@ rkisp2_ipa_algorithms = files([ 'awb.cpp', 'bls.cpp', 'ccm.cpp', + 'crop.cpp', 'csm.cpp', 'goc.cpp', 'lsc.cpp', From patchwork Thu Aug 27 10:50:09 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Paul Elder X-Patchwork-Id: 28136 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id B0272BDDFC for ; Thu, 27 Aug 2026 10:51:28 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 6125D684AE; Thu, 27 Aug 2026 12:51:28 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="vR5xoRj6"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 24C0E68487 for ; Thu, 27 Aug 2026 12:51:27 +0200 (CEST) Received: from neptunite.hamster-moth.ts.net (unknown [IPv6:2400:2411:160:2f00:306c:5b5b:3c7f:eabd]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 903ADC1; Thu, 27 Aug 2026 12:49:57 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1787827800; bh=ANkI6m1mn4QEmtkEsIpMdABg4JfnT3xfJK0XI6ZekSQ=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=vR5xoRj6nfxuJB2qfF5qaEp7hu/m/YIDfIpcbQGaUwNtDV4UrShM3NzbX6xEtjCGH Uo79IkYfNb6m+Z7bBonFbM8Z3MvdOX/IXmpTiuVOKDox4iJE5lYbeNRrEhumDZ2q8u y38r0jTHSV31dG0BxtLgVAHRuRMx+r/9IdgG3TWY= From: Paul Elder To: laurent.pinchart@ideasonboard.com Cc: Paul Elder , michael.riesch@collabora.com, xuhf@rock-chips.com, stefan.klug@ideasonboard.com, kieran.bingham@ideasonboard.com, dan.scally@ideasonboard.com, jacopo.mondi@ideasonboard.com, nicolas.dufresne@collabora.com, mehdi.djait@linux.intel.com, libcamera-devel@lists.libcamera.org Subject: [PATCH v3 16/20] utils: tuning: libtuning: image: Enable images missing some metadata Date: Thu, 27 Aug 2026 19:50:09 +0900 Message-ID: <20260827105018.2781166-17-paul.elder@ideasonboard.com> X-Mailer: git-send-email 2.47.2 In-Reply-To: <20260827105018.2781166-1-paul.elder@ideasonboard.com> References: <20260827105018.2781166-1-paul.elder@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Some pipeline handlers, like the rkisp2 pipeline handler, are capable of taking raw images but cannot set: - exposure time - gain - black level yet as they have not been implemented for raw mode yet. Arguably black mode will never be supported in raw mode as it bypasses the ISP completely and black level subtraction is in the ISP. This prevents the images from being used for tuning in the current libtuning-based tuning scripts as it requires exposure time and iso and black level, so modify libtuning to allow images with no exposure time and iso and black level. Signed-off-by: Paul Elder --- No change in v2 --- utils/tuning/libtuning/image.py | 25 ++++++++++++++++++++----- 1 file changed, 20 insertions(+), 5 deletions(-) diff --git a/utils/tuning/libtuning/image.py b/utils/tuning/libtuning/image.py index ecd334bdc67f..b1ada0d0867c 100644 --- a/utils/tuning/libtuning/image.py +++ b/utils/tuning/libtuning/image.py @@ -70,12 +70,27 @@ class Image: white = metadata[f'Exif.{subimage}.WhiteLevel'].value self.sigbits = int(white).bit_length() self.fmt = (self.sigbits - 4) // 2 - self.exposure = int(metadata[f'Exif.{photo}.ExposureTime'].value * 1000000) - self.againQ8 = metadata[f'Exif.{photo}.ISOSpeedRatings'].value * 256 / 100 - self.againQ8_norm = self.againQ8 / 256 + + self.exposure = None + exposure_key = f'Exif.{photo}.ExposureTime' + if exposure_key in metadata: + self.exposure = int(metadata[exposure_key].value * 1000000) + + self.againQ8 = None + self.againQ8_norm = None + iso_key = f'Exif.{photo}.ISOSpeedRatings' + if iso_key in metadata: + self.againQ8 = metadata[iso_key].value * 256 / 100 + self.againQ8_norm = self.againQ8 / 256 + + self.blacklevel = 0 + self.blacklevel_16 = 0 + bl_key = f'Exif.{subimage}.BlackLevel' + if bl_key in metadata: + self.blacklevel = int(metadata[bl_key].value[0]) + self.blacklevel_16 = self.blacklevel << (16 - self.sigbits) + self.camName = metadata['Exif.Image.Model'].value - self.blacklevel = int(metadata[f'Exif.{subimage}.BlackLevel'].value[0]) - self.blacklevel_16 = self.blacklevel << (16 - self.sigbits) # Channel order depending on bayer pattern # The key is the order given by exif, where 0 is R, 1 is G, and 2 is B From patchwork Thu Aug 27 10:50:10 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Paul Elder X-Patchwork-Id: 28137 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 89B45BDDFC for ; Thu, 27 Aug 2026 10:51:32 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 3EFEF684B2; Thu, 27 Aug 2026 12:51:32 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="frNOOOMx"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id C1559684A0 for ; Thu, 27 Aug 2026 12:51:30 +0200 (CEST) Received: from neptunite.hamster-moth.ts.net (unknown [IPv6:2400:2411:160:2f00:306c:5b5b:3c7f:eabd]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 384EB5B3; Thu, 27 Aug 2026 12:50:00 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1787827804; bh=F/Y/d2aaUo/KjEVQEBSfZBGUTXdyBMfYDvcP8qtFxsY=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=frNOOOMxZ3h/c4nlIWxqAyKZ8eWZBcS+4R9pjK+wC5lQ8xtY7M8cekyi668KV9QH0 HhG5dY+ig6L33bn4/ybABs6yje922siHjVzKvyF2ewHBekxrybS7MKEw4VBqC5dKgM Os1BeIjrbT4qWHTj5CEZ3FiyxcU8NetBTihhb8gE= From: Paul Elder To: laurent.pinchart@ideasonboard.com Cc: Paul Elder , michael.riesch@collabora.com, xuhf@rock-chips.com, stefan.klug@ideasonboard.com, kieran.bingham@ideasonboard.com, dan.scally@ideasonboard.com, jacopo.mondi@ideasonboard.com, nicolas.dufresne@collabora.com, mehdi.djait@linux.intel.com, libcamera-devel@lists.libcamera.org Subject: [PATCH v3 17/20] utils: tuning: rkisp2.py: Add tuning script for rkisp2 Date: Thu, 27 Aug 2026 19:50:10 +0900 Message-ID: <20260827105018.2781166-18-paul.elder@ideasonboard.com> X-Mailer: git-send-email 2.47.2 In-Reply-To: <20260827105018.2781166-1-paul.elder@ideasonboard.com> References: <20260827105018.2781166-1-paul.elder@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Add a tuning script for the rkisp2. It is based on the rkisp1 tuning script. Signed-off-by: Paul Elder --- lsc is not output currently as it needs a bit more investigation. No change in v2 --- utils/tuning/rkisp2.py | 42 ++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 42 insertions(+) create mode 100755 utils/tuning/rkisp2.py diff --git a/utils/tuning/rkisp2.py b/utils/tuning/rkisp2.py new file mode 100755 index 000000000000..13feff03e6cc --- /dev/null +++ b/utils/tuning/rkisp2.py @@ -0,0 +1,42 @@ +#!/usr/bin/env python3 +# SPDX-License-Identifier: GPL-2.0-or-later +# +# Copyright (C) 2026, Paul Elder +# Copyright (C) 2026, Ideas On Board +# +# Tuning script for rkisp2, based on rkisp1 + +import logging +import sys + +import coloredlogs +import libtuning as lt +from libtuning.generators import YamlOutput +from libtuning.modules.agc import AGCRkISP1 +from libtuning.modules.awb import AWBRkISP1 +from libtuning.modules.ccm import CCMRkISP1 +from libtuning.modules.lsc import LSCRkISP1 +from libtuning.modules.lux import LuxRkISP1 +from libtuning.modules.static import StaticModule +from libtuning.parsers import YamlParser + +coloredlogs.install(level=logging.INFO, fmt='%(name)s %(levelname)s %(message)s') + +csm = StaticModule('ColorSpaceConversion') +agc = AGCRkISP1(debug=[lt.Debug.Plot]) +awb = AWBRkISP1(debug=[lt.Debug.Plot]) +bls = StaticModule('BlackLevelSubtraction') +ccm = CCMRkISP1(debug=[lt.Debug.Plot]) +goc = StaticModule('GammaOutCorrection', {'gamma': 2.2}) + +tuner = lt.Tuner('RkISP2') +tuner.add([csm, agc, awb, bls, ccm, goc]) +tuner.set_input_parser(YamlParser()) +tuner.set_output_formatter(YamlOutput()) + +# Bayesian AWB uses the lux value, so insert the lux algorithm before AWB. +# Compress is parameterized by others, so add it at the end. +tuner.set_output_order([csm, agc, awb, bls, ccm, goc]) + +if __name__ == '__main__': + sys.exit(tuner.run(sys.argv)) From patchwork Thu Aug 27 10:50:11 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Paul Elder X-Patchwork-Id: 28138 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id 62AFFBDDFC for ; Thu, 27 Aug 2026 10:51:36 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 123E3684A6; Thu, 27 Aug 2026 12:51:36 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="hSxBKr/1"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [IPv6:2001:4b98:dc2:55:216:3eff:fef7:d647]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 690E9684A0 for ; Thu, 27 Aug 2026 12:51:34 +0200 (CEST) Received: from neptunite.hamster-moth.ts.net (unknown [IPv6:2400:2411:160:2f00:306c:5b5b:3c7f:eabd]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id D6E83C1; Thu, 27 Aug 2026 12:50:04 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1787827807; bh=shC7mLhx5RhE1nFOnvWMHiGL38nfZ9U8mCVUf4nrfvI=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=hSxBKr/1MIcsL8yqEfeqNMvnRjyJ27IeQ6Qd2oyG+JFgOKEnQ0ItVJrvVgcew72Cr SY0WDCG+3yQx7stDe6AK7FOxYY5vS1NrgucxW/dNIDjYrRq49V8krh8Gw6+kAX4XXp tcc47ihYMXi4/X8q6jmpMW3K83/XQIgiJ0u/Hfzg= From: Paul Elder To: laurent.pinchart@ideasonboard.com Cc: Paul Elder , michael.riesch@collabora.com, xuhf@rock-chips.com, stefan.klug@ideasonboard.com, kieran.bingham@ideasonboard.com, dan.scally@ideasonboard.com, jacopo.mondi@ideasonboard.com, nicolas.dufresne@collabora.com, mehdi.djait@linux.intel.com, libcamera-devel@lists.libcamera.org Subject: [PATCH v3 18/20] ipa: rkisp2: data: Add uncalibrated tuning file Date: Thu, 27 Aug 2026 19:50:11 +0900 Message-ID: <20260827105018.2781166-19-paul.elder@ideasonboard.com> X-Mailer: git-send-email 2.47.2 In-Reply-To: <20260827105018.2781166-1-paul.elder@ideasonboard.com> References: <20260827105018.2781166-1-paul.elder@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Add uncalibrated tuning file for rkisp2 that just enables black level subtraction. Signed-off-by: Paul Elder --- No change in v2 --- src/ipa/rkisp2/data/meson.build | 10 ++++++++++ src/ipa/rkisp2/data/uncalibrated.yaml | 7 +++++++ src/ipa/rkisp2/meson.build | 1 + 3 files changed, 18 insertions(+) create mode 100644 src/ipa/rkisp2/data/meson.build create mode 100644 src/ipa/rkisp2/data/uncalibrated.yaml diff --git a/src/ipa/rkisp2/data/meson.build b/src/ipa/rkisp2/data/meson.build new file mode 100644 index 000000000000..1ecd8b20ec90 --- /dev/null +++ b/src/ipa/rkisp2/data/meson.build @@ -0,0 +1,10 @@ +# SPDX-License-Identifier: CC0-1.0 + +conf_files = files([ + 'uncalibrated.yaml', +]) + +install_data(conf_files, + install_dir : ipa_data_dir / 'rkisp2', + install_tag : 'runtime') + diff --git a/src/ipa/rkisp2/data/uncalibrated.yaml b/src/ipa/rkisp2/data/uncalibrated.yaml new file mode 100644 index 000000000000..6d60bf1e6e98 --- /dev/null +++ b/src/ipa/rkisp2/data/uncalibrated.yaml @@ -0,0 +1,7 @@ +# SPDX-License-Identifier: CC0-1.0 +%YAML 1.1 +--- +version: 1 +algorithms: + - BlackLevelSubtraction: +... diff --git a/src/ipa/rkisp2/meson.build b/src/ipa/rkisp2/meson.build index f2f435c2d66f..db6b06aeec25 100644 --- a/src/ipa/rkisp2/meson.build +++ b/src/ipa/rkisp2/meson.build @@ -1,6 +1,7 @@ # SPDX-License-Identifier: CC0-1.0 subdir('algorithms') +subdir('data') ipa_name = 'ipa_rkisp2' From patchwork Thu Aug 27 10:50:12 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Paul Elder X-Patchwork-Id: 28139 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id E8812BDDFC for ; Thu, 27 Aug 2026 10:51:39 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 98F3E684AE; Thu, 27 Aug 2026 12:51:39 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="wlewrnV2"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id 154EA684A0 for ; Thu, 27 Aug 2026 12:51:38 +0200 (CEST) Received: from neptunite.hamster-moth.ts.net (unknown [IPv6:2400:2411:160:2f00:306c:5b5b:3c7f:eabd]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 7C2215B3; Thu, 27 Aug 2026 12:50:08 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1787827811; bh=HOXfdKeFzRjzgCEZHHnv5k6VATnM/WzKxgLKB6/AU1E=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=wlewrnV2UYdcC08cpnp6NWtRuXoOxGFQhCNpiIlzYw6Tef07x0E/mD5R5E1k/I5zn DfWPRtISFm7iOKAw667WYgmCj37u75z/MJdfUmLGJRrJrKyjOeaOoU8e5D1YhCGRt6 BA5M4n8xWrTVrScCjVHJFOBlhPwK+jWFetg/eCJ8= From: Paul Elder To: laurent.pinchart@ideasonboard.com Cc: Paul Elder , michael.riesch@collabora.com, xuhf@rock-chips.com, stefan.klug@ideasonboard.com, kieran.bingham@ideasonboard.com, dan.scally@ideasonboard.com, jacopo.mondi@ideasonboard.com, nicolas.dufresne@collabora.com, mehdi.djait@linux.intel.com, libcamera-devel@lists.libcamera.org Subject: [PATCH v3 19/20] ipa: rkisp2: data: Add rudimentary tuning file for imx219 Date: Thu, 27 Aug 2026 19:50:12 +0900 Message-ID: <20260827105018.2781166-20-paul.elder@ideasonboard.com> X-Mailer: git-send-email 2.47.2 In-Reply-To: <20260827105018.2781166-1-paul.elder@ideasonboard.com> References: <20260827105018.2781166-1-paul.elder@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Add a tuning file for rkisp2 for imx219. The ccms come from rpi, the lsc tables are just all 1.0 for now, and the agc parameters are copied from the output of the rkisp1 tuning script. Signed-off-by: Paul Elder --- No change in v2 --- src/ipa/rkisp2/data/imx219.yaml | 139 ++++++++++++++++++++++++++++++++ src/ipa/rkisp2/data/meson.build | 1 + 2 files changed, 140 insertions(+) create mode 100644 src/ipa/rkisp2/data/imx219.yaml diff --git a/src/ipa/rkisp2/data/imx219.yaml b/src/ipa/rkisp2/data/imx219.yaml new file mode 100644 index 000000000000..eef413a490cb --- /dev/null +++ b/src/ipa/rkisp2/data/imx219.yaml @@ -0,0 +1,139 @@ +# SPDX-License-Identifier: CC0-1.0 +%YAML 1.1 +--- +version: 1 +algorithms: + - LensShadingCorrection: + x-size: [ 0.0625, 0.0625, 0.0625, 0.0625, 0.0625, 0.0625, 0.0625, 0.0625, + 0.0625, 0.0625, 0.0625, 0.0625, 0.0625, 0.0625, 0.0625, 0.0625 ] + y-size: [ 0.0625, 0.0625, 0.0625, 0.0625, 0.0625, 0.0625, 0.0625, 0.0625, + 0.0625, 0.0625, 0.0625, 0.0625, 0.0625, 0.0625, 0.0625, 0.0625 ] + sets: + - ct: 5800 + r: [ + 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, + 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, + 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, + 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, + 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, + 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, + 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, + 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, + 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, + 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, + 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, + 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, + 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, + 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, + 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, + 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, + 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, + ] + gr: [ + 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, + 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, + 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, + 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, + 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, + 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, + 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, + 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, + 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, + 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, + 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, + 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, + 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, + 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, + 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, + 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, + 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, + ] + gb: [ + 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, + 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, + 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, + 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, + 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, + 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, + 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, + 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, + 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, + 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, + 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, + 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, + 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, + 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, + 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, + 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, + 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, + ] + b: [ + 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, + 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, + 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, + 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, + 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, + 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, + 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, + 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, + 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, + 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, + 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, + 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, + 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, + 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, + 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, + 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, + 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, 1024, + ] + - Agc: + AeMeteringMode: + MeteringCentreWeighted: [0, 0, 0, 0, 0, 0, 6, 8, 6, 0, 0, 8, 16, 8, 0, 0, 6, 8, 6, 0, 0, 0, 0, 0, 0] + MeteringSpot: [0, 0, 0, 0, 0, 0, 2, 4, 2, 0, 0, 4, 16, 4, 0, 0, 2, 4, 2, 0, 0, 0, 0, 0, 0] + MeteringMatrix: [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1] + AeExposureMode: + ExposureNormal: + exposureTime: [ 100, 10000, 30000, 60000, 120000 ] + gain: [ 1.0, 2.0, 4.0, 6.0, 6.0 ] + ExposureShort: + exposureTime: [ 100, 5000, 10000, 20000, 120000 ] + gain: [ 1.0, 2.0, 4.0, 6.0, 6.0 ] + AeConstraintMode: + ConstraintNormal: + lower: + qLo: 0.98 + qHi: 1.0 + yTarget: [0, 0.65, 1000, 0.65] + ConstraintHighlight: + lower: + qLo: 0.98 + qHi: 1.0 + yTarget: [0, 0.5, 1000, 0.5] + upper: + qLo: 0.98 + qHi: 1.0 + yTarget: [0, 0.8, 1000, 0.5] + relativeLuminanceTarget: 0.5 + - Awb: + - BlackLevelSubtraction: + R: 4096 + Gr: 4096 + Gb: 4096 + B: 4096 + - Ccm: + ccms: + - ct: 2860 + ccm: [ 2.12089, -0.52461, -0.59629, -0.85342, 2.80445, -0.95103, -0.26897, -1.14788, 2.41685 ] + - ct: 2960 + ccm: [ 2.26962, -0.54174, -0.72789, -0.77008, 2.60271, -0.83262, -0.26036, -1.51254, 2.77289 ] + - ct: 3603 + ccm: [ 2.18644, -0.66148, -0.52496, -0.77828, 2.69474, -0.91645, -0.25239, -0.83059, 2.08298 ] + - ct: 4650 + ccm: [ 2.18174, -0.70887, -0.47287, -0.70196, 2.76426, -1.06231, -0.25157, -0.71978, 1.97135 ] + - ct: 5858 + ccm: [ 2.32392, -0.88421, -0.43971, -0.63821, 2.58348, -0.94527, -0.28541, -0.54112, 1.82653 ] + - ct: 7580 + ccm: [ 2.21175, -0.53242, -0.67933, -0.57875, 3.07922, -1.50047, -0.27709, -0.73338, 2.01048 ] + - GammaOutCorrection: + - ColorSpaceConversion: +... diff --git a/src/ipa/rkisp2/data/meson.build b/src/ipa/rkisp2/data/meson.build index 1ecd8b20ec90..6b21718e408f 100644 --- a/src/ipa/rkisp2/data/meson.build +++ b/src/ipa/rkisp2/data/meson.build @@ -1,6 +1,7 @@ # SPDX-License-Identifier: CC0-1.0 conf_files = files([ + 'imx219.yaml', 'uncalibrated.yaml', ]) From patchwork Thu Aug 27 10:50:13 2026 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Paul Elder X-Patchwork-Id: 28140 Return-Path: X-Original-To: parsemail@patchwork.libcamera.org Delivered-To: parsemail@patchwork.libcamera.org Received: from lancelot.ideasonboard.com (lancelot.ideasonboard.com [92.243.16.209]) by patchwork.libcamera.org (Postfix) with ESMTPS id D4283C333C for ; Thu, 27 Aug 2026 10:51:42 +0000 (UTC) Received: from lancelot.ideasonboard.com (localhost [IPv6:::1]) by lancelot.ideasonboard.com (Postfix) with ESMTP id 5A1D2684AE; Thu, 27 Aug 2026 12:51:42 +0200 (CEST) Authentication-Results: lancelot.ideasonboard.com; dkim=pass (1024-bit key; unprotected) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="RJqnYC2P"; dkim-atps=neutral Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lancelot.ideasonboard.com (Postfix) with ESMTPS id B2B2D684A0 for ; Thu, 27 Aug 2026 12:51:41 +0200 (CEST) Received: from neptunite.hamster-moth.ts.net (unknown [IPv6:2400:2411:160:2f00:306c:5b5b:3c7f:eabd]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 28E01C1; Thu, 27 Aug 2026 12:50:11 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1787827815; bh=H0RiDQw1XheJP92AQlO2vD0Fh2CkLcw/Ytfjtw9z6F8=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=RJqnYC2P4bS0m511bN9xhLxFnfepvOqIts37FaL4ItXStjldW/kBm0XC2BH9B4fSb qXlfay92iqEgvILtrLTrtsxzgRtVUaqNUCrw1Yr85MvMjj8kLwhQ+eTil8GY2JaPsb 5boES4RxQ6YHVBcxOflB+HhpbnjnvApDmUzbyZnc= From: Paul Elder To: laurent.pinchart@ideasonboard.com Cc: Paul Elder , michael.riesch@collabora.com, xuhf@rock-chips.com, stefan.klug@ideasonboard.com, kieran.bingham@ideasonboard.com, dan.scally@ideasonboard.com, jacopo.mondi@ideasonboard.com, nicolas.dufresne@collabora.com, mehdi.djait@linux.intel.com, libcamera-devel@lists.libcamera.org Subject: [PATCH v3 20/20] ipa: rkisp2: data: Add tuning file for imx708 Date: Thu, 27 Aug 2026 19:50:13 +0900 Message-ID: <20260827105018.2781166-21-paul.elder@ideasonboard.com> X-Mailer: git-send-email 2.47.2 In-Reply-To: <20260827105018.2781166-1-paul.elder@ideasonboard.com> References: <20260827105018.2781166-1-paul.elder@ideasonboard.com> MIME-Version: 1.0 X-BeenThere: libcamera-devel@lists.libcamera.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libcamera-devel-bounces@lists.libcamera.org Sender: "libcamera-devel" Add a tuning file for rkisp2 for imx708. This was tuned using the rkisp2 tuning script using images from [0] (at branch raspberrypi/imx708 in directory 2026-05-22), with some hand-modifications, such as removing lsc because it needs further investigation to function properly. [0] https://gitlab.freedesktop.org/camera/tuning Signed-off-by: Paul Elder --- No change in v2 --- src/ipa/rkisp2/data/imx708.yaml | 155 ++++++++++++++++++++++++++++++++ src/ipa/rkisp2/data/meson.build | 1 + 2 files changed, 156 insertions(+) create mode 100644 src/ipa/rkisp2/data/imx708.yaml diff --git a/src/ipa/rkisp2/data/imx708.yaml b/src/ipa/rkisp2/data/imx708.yaml new file mode 100644 index 000000000000..3296ca58b71e --- /dev/null +++ b/src/ipa/rkisp2/data/imx708.yaml @@ -0,0 +1,155 @@ +# SPDX-License-Identifier: CC0-1.0 +%YAML 1.1 +--- +version: 1 +algorithms: + - Crop: + - ColorSpaceConversion: + - Agc: + AeMeteringMode: + MeteringCentreWeighted: [ 0, 0, 0, 0, 0, 0, 6, 8, 6, 0, 0, 8, 16, 8, 0, 0, 6, 8, 6, 0, 0, 0, 0, 0, 0 ] + AeExposureMode: + ExposureNormal: + exposureTime: [ 100, 10000, 30000, 60000, 120000 ] + gain: [ 1.2, 2.0, 4.0, 6.0, 6.0 ] + AeConstraintMode: + ConstraintNormal: + lower: + qLo: 0.98 + qHi: 1.0 + yTarget: 0.1 + relativeLuminanceTarget: 0.1 + - Awb: + algorithm: "grey" + AwbMode: + AwbAuto: + lo: 2500 + hi: 9000 + AwbIncandescent: + lo: 2500 + hi: 3000 + AwbTungsten: + lo: 3000 + hi: 3500 + AwbFluorescent: + lo: 4000 + hi: 4700 + AwbIndoor: + lo: 3000 + hi: 5000 + AwbDaylight: + lo: 5500 + hi: 6500 + AwbCloudy: + lo: 6500 + hi: 8000 + priors: + - lux: 0 + ct: [ 2000, 13000 ] + probability: [ 1.0, 1.0 ] + colourGains: + - ct: 2500 + gains: [ 1.1199462425803561, 1.9142419601837672 ] + - ct: 3000 + gains: [ 1.1242270938729624, 1.9015021867275146 ] + - ct: 3500 + gains: [ 1.1583458820803894, 1.8109380659181455 ] + - ct: 4000 + gains: [ 1.2714558169103625, 1.6165535079211122 ] + - ct: 4500 + gains: [ 1.476886722788362, 1.4486455164421266 ] + - ct: 5000 + gains: [ 1.64446637066272, 1.3819789939192924 ] + - ct: 5500 + gains: [ 1.7822135091783995, 1.3487995683841383 ] + - ct: 6000 + gains: [ 1.8677624206200971, 1.33422281521014 ] + - ct: 6500 + gains: [ 1.9219680953296177, 1.3266118333775536 ] + - ct: 7000 + gains: [ 1.9516003122560501, 1.3229263130043656 ] + - ct: 7500 + gains: [ 1.9673421207948063, 1.321178491214163 ] + - ct: 8000 + gains: [ 1.9845207382417145, 1.3190871916633689 ] + - ct: 8500 + gains: [ 1.993620414673046, 1.3182177695755337 ] + - ct: 9000 + gains: [ 2.006823198876179, 1.3170025023047545 ] + transversePos: 0.01269 + transverseNeg: 0.01561 + - BlackLevelSubtraction: + - Ccm: + ccms: + - ct: 2500 + ccm: [ + 1.72201, 0.27067, -0.99269, -0.64829, 2.48777, -0.83947, -0.08905, -0.54176, + 1.63081, + ] + - ct: 3000 + ccm: [ + 1.80587, 0.27132, -1.07719, -0.67445, 2.55729, -0.88284, -0.11544, -0.50189, + 1.61734, + ] + - ct: 3500 + ccm: [ + 1.83389, 0.37559, -1.20949, -0.69373, 2.67018, -0.97646, -0.15308, -0.47581, + 1.62889, + ] + - ct: 4000 + ccm: [ + 1.86271, 0.39818, -1.26089, -0.66884, 2.62966, -0.96082, -0.19298, -0.45096, + 1.64394, + ] + - ct: 4500 + ccm: [ + 1.78213, 0.40302, -1.18514, -0.61892, 2.47338, -0.85445, -0.24296, -0.38901, + 1.63197, + ] + - ct: 5000 + ccm: [ + 1.75328, 0.39113, -1.14441, -0.60457, 2.40168, -0.79711, -0.28013, -0.33477, + 1.61489, + ] + - ct: 5500 + ccm: [ + 1.67989, 0.42725, -1.10714, -0.60576, 2.35811, -0.75235, -0.30859, -0.29885, + 1.60744, + ] + - ct: 6000 + ccm: [ + 1.62571, 0.45052, -1.07622, -0.60271, 2.32574, -0.72302, -0.31931, -0.27971, + 1.59902, + ] + - ct: 6500 + ccm: [ + 1.60018, 0.43641, -1.03658, -0.59726, 2.29742, -0.70016, -0.31271, -0.28088, + 1.59358, + ] + - ct: 7000 + ccm: [ + 1.59364, 0.41222, -1.00585, -0.58752, 2.27837, -0.69085, -0.30352, -0.28323, + 1.58675, + ] + - ct: 7500 + ccm: [ + 1.59057, 0.40006, -0.99063, -0.58525, 2.26603, -0.68078, -0.29994, -0.28105, + 1.58099, + ] + - ct: 8000 + ccm: [ + 1.59551, 0.36878, -0.96428, -0.57195, 2.24572, -0.67377, -0.28793, -0.28858, + 1.57651, + ] + - ct: 8500 + ccm: [ + 1.60576, 0.33929, -0.94504, -0.56235, 2.23411, -0.67176, -0.27817, -0.29265, + 1.57082, + ] + - ct: 9000 + ccm: [ + 1.60867, 0.30611, -0.91478, -0.54939, 2.21074, -0.66135, -0.26915, -0.30127, + 1.57042, + ] + - GammaOutCorrection: + gamma: 2.2 diff --git a/src/ipa/rkisp2/data/meson.build b/src/ipa/rkisp2/data/meson.build index 6b21718e408f..f4dc569db277 100644 --- a/src/ipa/rkisp2/data/meson.build +++ b/src/ipa/rkisp2/data/meson.build @@ -2,6 +2,7 @@ conf_files = files([ 'imx219.yaml', + 'imx708.yaml', 'uncalibrated.yaml', ])